Tabnine Logo
Stream.partitionPersist
Code IndexAdd Tabnine to your IDE (free)

How to use
partitionPersist
method
in
storm.trident.Stream

Best Java code snippets using storm.trident.Stream.partitionPersist (Showing top 20 results out of 315)

origin: alibaba/jstorm

public TridentState partitionPersist(StateSpec stateSpec, StateUpdater updater, Fields functionFields) {
  return partitionPersist(stateSpec, null, updater, functionFields);
}
origin: alibaba/jstorm

public TridentState partitionPersist(StateSpec stateSpec, StateUpdater updater) {
  return partitionPersist(stateSpec, updater, new Fields());
}
origin: alibaba/jstorm

public TridentState partitionPersist(StateSpec stateSpec, Fields inputFields, StateUpdater updater) {
  return partitionPersist(stateSpec, inputFields, updater, new Fields());
}
origin: alibaba/jstorm

public TridentState partitionPersist(StateFactory stateFactory, StateUpdater updater) {
  return partitionPersist(stateFactory, updater, new Fields());
}
origin: alibaba/jstorm

public TridentState partitionPersist(StateFactory stateFactory, Fields inputFields, StateUpdater updater, Fields functionFields) {
 return partitionPersist(new StateSpec(stateFactory), inputFields, updater, functionFields);
}
origin: alibaba/jstorm

public TridentState partitionPersist(StateFactory stateFactory, Fields inputFields, StateUpdater updater) {
  return partitionPersist(stateFactory, inputFields, updater, new Fields());
}
origin: alibaba/jstorm

public TridentState partitionPersist(StateFactory stateFactory, StateUpdater updater, Fields functionFields) {
  return partitionPersist(new StateSpec(stateFactory), updater, functionFields);
}
origin: alibaba/jstorm

public TridentState persistentAggregate(StateSpec spec, Fields inputFields, ReducerAggregator agg, Fields functionFields) {
  projectionValidation(inputFields);
  return global().partitionPersist(spec, inputFields, new ReducerAggStateUpdater(agg), functionFields);
}
origin: alibaba/jstorm

public TridentState persistentAggregate(StateSpec spec, Fields inputFields, ReducerAggregator agg, Fields functionFields) {
  return _stream.partitionBy(_groupFields).partitionPersist(spec, TridentUtils.fieldsUnion(_groupFields, inputFields),
      new MapReducerAggStateUpdater(agg, _groupFields, inputFields), TridentUtils.fieldsConcat(_groupFields, functionFields));
}
origin: alibaba/jstorm

public TridentState persistentAggregate(StateSpec spec, Fields inputFields, CombinerAggregator agg, Fields functionFields) {
  return aggregate(inputFields, agg, functionFields).partitionPersist(spec, TridentUtils.fieldsUnion(_groupFields, functionFields),
      new MapCombinerAggStateUpdater(agg, _groupFields, functionFields), TridentUtils.fieldsConcat(_groupFields, functionFields));
}
origin: alibaba/jstorm

public TridentState persistentAggregate(StateSpec spec, Fields inputFields, CombinerAggregator agg, Fields functionFields) {
  projectionValidation(inputFields);
  // replaces normal aggregation here with a global grouping because it needs to be consistent across batches 
  return new ChainedAggregatorDeclarer(this, new GlobalAggScheme())
      .aggregate(inputFields, agg, functionFields)
      .chainEnd()
      .partitionPersist(spec, functionFields, new CombinerAggStateUpdater(agg), functionFields);
}
origin: alibaba/jstorm

private Stream window(WindowConfig windowConfig, WindowsStoreFactory windowStoreFactory, Fields inputFields, Aggregator aggregator,
           Fields functionFields, boolean storeTuplesInStore) {
  projectionValidation(inputFields);
  windowConfig.validate();
  Fields fields = addTriggerField(functionFields);
  // when storeTuplesInStore is false then the given windowStoreFactory is only used to store triggers and
  // that store is passed to WindowStateUpdater to remove them after committing the batch.
  Stream stream = _topology.addSourcedNode(this,
      new ProcessorNode(_topology.getUniqueStreamId(),
          _name,
          fields,
          fields,
          new WindowTridentProcessor(windowConfig, _topology.getUniqueWindowId(), windowStoreFactory,
              inputFields, aggregator, storeTuplesInStore)));
  Stream effectiveStream = stream.project(functionFields);
  // create StateUpdater with the given windowStoreFactory to remove triggered aggregation results form store
  // when they are successfully processed.
  StateFactory stateFactory = new WindowsStateFactory();
  StateUpdater stateUpdater = new WindowsStateUpdater(windowStoreFactory);
  stream.partitionPersist(stateFactory, new Fields(WindowTridentProcessor.TRIGGER_FIELD_NAME), stateUpdater, new Fields());
  return effectiveStream;
}
origin: com.n3twork.storm/storm-core

public TridentState persistentAggregate(StateSpec spec, Fields inputFields, ReducerAggregator agg, Fields functionFields) {
  projectionValidation(inputFields);
  return global().partitionPersist(spec, inputFields, new ReducerAggStateUpdater(agg), functionFields);
}

origin: com.alibaba.jstorm/jstorm-core

public TridentState persistentAggregate(StateSpec spec, Fields inputFields, ReducerAggregator agg, Fields functionFields) {
  projectionValidation(inputFields);
  return global().partitionPersist(spec, inputFields, new ReducerAggStateUpdater(agg), functionFields);
}
origin: com.n3twork.storm/storm-core

public TridentState persistentAggregate(StateSpec spec, Fields inputFields, CombinerAggregator agg, Fields functionFields) {
  return aggregate(inputFields, agg, functionFields)
      .partitionPersist(spec,
          TridentUtils.fieldsUnion(_groupFields, functionFields),
          new MapCombinerAggStateUpdater(agg, _groupFields, functionFields),
          TridentUtils.fieldsConcat(_groupFields, functionFields)); 
}
origin: com.n3twork.storm/storm-core

public TridentState persistentAggregate(StateSpec spec, Fields inputFields, ReducerAggregator agg, Fields functionFields) {
  return _stream.partitionBy(_groupFields)
      .partitionPersist(spec,
        TridentUtils.fieldsUnion(_groupFields, inputFields),
        new MapReducerAggStateUpdater(agg, _groupFields, inputFields),
        TridentUtils.fieldsConcat(_groupFields, functionFields));
}
origin: com.alibaba.jstorm/jstorm-core

public TridentState persistentAggregate(StateSpec spec, Fields inputFields, CombinerAggregator agg, Fields functionFields) {
  return aggregate(inputFields, agg, functionFields).partitionPersist(spec, TridentUtils.fieldsUnion(_groupFields, functionFields),
      new MapCombinerAggStateUpdater(agg, _groupFields, functionFields), TridentUtils.fieldsConcat(_groupFields, functionFields));
}
origin: com.alibaba.jstorm/jstorm-core

public TridentState persistentAggregate(StateSpec spec, Fields inputFields, ReducerAggregator agg, Fields functionFields) {
  return _stream.partitionBy(_groupFields).partitionPersist(spec, TridentUtils.fieldsUnion(_groupFields, inputFields),
      new MapReducerAggStateUpdater(agg, _groupFields, inputFields), TridentUtils.fieldsConcat(_groupFields, functionFields));
}
origin: com.n3twork.storm/storm-core

public TridentState persistentAggregate(StateSpec spec, Fields inputFields, CombinerAggregator agg, Fields functionFields) {
  projectionValidation(inputFields);
  // replaces normal aggregation here with a global grouping because it needs to be consistent across batches 
  return new ChainedAggregatorDeclarer(this, new GlobalAggScheme())
      .aggregate(inputFields, agg, functionFields)
      .chainEnd()
      .partitionPersist(spec, functionFields, new CombinerAggStateUpdater(agg), functionFields);
}
origin: com.alibaba.jstorm/jstorm-core

public TridentState persistentAggregate(StateSpec spec, Fields inputFields, CombinerAggregator agg, Fields functionFields) {
  projectionValidation(inputFields);
  // replaces normal aggregation here with a global grouping because it needs to be consistent across batches 
  return new ChainedAggregatorDeclarer(this, new GlobalAggScheme())
      .aggregate(inputFields, agg, functionFields)
      .chainEnd()
      .partitionPersist(spec, functionFields, new CombinerAggStateUpdater(agg), functionFields);
}
storm.tridentStreampartitionPersist

Popular methods of Stream

  • each
  • groupBy
    ## Grouping Operation
  • project
    Filters out fields from a stream, resulting in a Stream containing only the fields specified by `kee
  • aggregate
  • partitionBy
    ## Repartitioning Operation
  • shuffle
    ## Repartitioning Operation Use random round robin algorithm to evenly redistribute tuples across al
  • stateQuery
  • parallelismHint
    Applies a parallelism hint to a stream.
  • partitionAggregate
  • global
    ## Repartitioning Operation All tuples are sent to the same partition. The same partition is chosen
  • batchGlobal
    ## Repartitioning Operation All tuples in the batch are sent to the same partition. Different batche
  • chainedAgg
  • batchGlobal,
  • chainedAgg,
  • <init>,
  • broadcast,
  • getOutputFields,
  • name,
  • partition,
  • persistentAggregate,
  • projectionValidation

Popular in Java

  • Parsing JSON documents to java classes using gson
  • orElseThrow (Optional)
    Return the contained value, if present, otherwise throw an exception to be created by the provided s
  • setScale (BigDecimal)
  • getSystemService (Context)
  • MalformedURLException (java.net)
    This exception is thrown when a program attempts to create an URL from an incorrect specification.
  • Enumeration (java.util)
    A legacy iteration interface.New code should use Iterator instead. Iterator replaces the enumeration
  • Queue (java.util)
    A collection designed for holding elements prior to processing. Besides basic java.util.Collection o
  • ReentrantLock (java.util.concurrent.locks)
    A reentrant mutual exclusion Lock with the same basic behavior and semantics as the implicit monitor
  • BoxLayout (javax.swing)
  • Runner (org.openjdk.jmh.runner)
  • Best IntelliJ plugins
Tabnine Logo
  • Products

    Search for Java codeSearch for JavaScript code
  • IDE Plugins

    IntelliJ IDEAWebStormVisual StudioAndroid StudioEclipseVisual Studio CodePyCharmSublime TextPhpStormVimGoLandRubyMineEmacsJupyter NotebookJupyter LabRiderDataGripAppCode
  • Company

    About UsContact UsCareers
  • Resources

    FAQBlogTabnine AcademyTerms of usePrivacy policyJava Code IndexJavascript Code Index
Get Tabnine for your IDE now