congrats Icon
New! Tabnine Pro 14-day free trial
Start a free trial
Tabnine Logo
Stream.getOutputFields
Code IndexAdd Tabnine to your IDE (free)

How to use
getOutputFields
method
in
storm.trident.Stream

Best Java code snippets using storm.trident.Stream.getOutputFields (Showing top 20 results out of 315)

origin: alibaba/jstorm

@Override
public Fields getOutputFields() {
  return _stream.getOutputFields();
}
origin: alibaba/jstorm

  private void projectionValidation(Fields projFields) {
    if (projFields == null) {
      return;
    }

    Fields allFields = this.getOutputFields();
    for (String field : projFields) {
      if (!allFields.contains(field)) {
        throw new IllegalArgumentException("Trying to select non-existent field: '" + field + "' from stream containing fields fields: <" + allFields + ">");
      }
    }
  }
}
origin: alibaba/jstorm

/**
 * Returns a stream consisting of the elements of this stream that match the given filter.
 *
 * @param filter the filter to apply to each trident tuple to determine if it should be included.
 * @return the new stream
 */
public Stream filter(Filter filter) {
  return each(getOutputFields(), filter);
}
origin: alibaba/jstorm

private static List<Fields> strippedInputFields(List<Stream> streams, List<Fields> joinFields) {
  List<Fields> ret = new ArrayList<>();
  for(int i=0; i<streams.size(); i++) {
    ret.add(TridentUtils.fieldsSubtract(streams.get(i).getOutputFields(), joinFields.get(i)));
  }
  return ret;
}

origin: alibaba/jstorm

public Stream merge(List<Stream> streams) {
  return merge(streams.get(0).getOutputFields(), streams);
} 

origin: alibaba/jstorm

private <T> Stream comparableAggregateStream(String inputFieldName, Aggregator<T> aggregator) {
  if(inputFieldName != null) {
    projectionValidation(new Fields(inputFieldName));
  }
  return partitionAggregate(getOutputFields(), aggregator, getOutputFields());
}
origin: alibaba/jstorm

/**
 * Returns a stream consisting of the result of applying the given mapping function to the values of this stream.
 *
 * @param function a mapping function to be applied to each value in this stream.
 * @return the new stream
 */
public Stream map(MapFunction function) {
  projectionValidation(getOutputFields());
  return _topology.addSourcedNode(this,
                  new ProcessorNode(
                      _topology.getUniqueStreamId(),
                      _name,
                      getOutputFields(),
                      getOutputFields(),
                      new MapProcessor(getOutputFields(), new MapFunctionExecutor(function))));
}
origin: alibaba/jstorm

/**
 * Returns a stream consisting of the results of replacing each value of this stream with the contents
 * produced by applying the provided mapping function to each value. This has the effect of applying
 * a one-to-many transformation to the values of the stream, and then flattening the resulting elements into a new stream.
 *
 * @param function a mapping function to be applied to each value in this stream which produces new values.
 * @return the new stream
 */
public Stream flatMap(FlatMapFunction function) {
  projectionValidation(getOutputFields());
  return _topology.addSourcedNode(this,
                  new ProcessorNode(
                      _topology.getUniqueStreamId(),
                      _name,
                      getOutputFields(),
                      getOutputFields(),
                      new MapProcessor(getOutputFields(), new FlatMapFunctionExecutor(function))));
}
origin: alibaba/jstorm

/**
 * Returns a stream consisting of the trident tuples of this stream, additionally performing the provided action on
 * each trident tuple as they are consumed from the resulting stream. This is mostly useful for debugging
 * to see the tuples as they flow past a certain point in a pipeline.
 *
 * @param action the action to perform on the trident tuple as they are consumed from the stream
 * @return the new stream
 */
public Stream peek(Consumer action) {
  projectionValidation(getOutputFields());
  return _topology.addSourcedNode(this,
                  new ProcessorNode(
                      _topology.getUniqueStreamId(),
                      _name,
                      getOutputFields(),
                      getOutputFields(),
                      new MapProcessor(getOutputFields(), new ConsumerExecutor(action))));
}
origin: alibaba/jstorm

@Override
public Stream apply(Stream input) {
  Fields outputFields = input.getOutputFields();
  return input.partitionAggregate(outputFields, _agg, outputFields)
        .global()
        .partitionAggregate(outputFields, _agg, outputFields);             
}
origin: alibaba/jstorm

public Stream stateQuery(TridentState state, Fields inputFields, QueryFunction function, Fields functionFields) {
  projectionValidation(inputFields);
  String stateId = state._node.stateInfo.id;
  Node n = new ProcessorNode(_topology.getUniqueStreamId(),
          _name,
          TridentUtils.fieldsConcat(getOutputFields(), functionFields),
          functionFields,
          new StateQueryProcessor(stateId, inputFields, function));
  _topology._colocate.get(stateId).add(n);
  return _topology.addSourcedNode(this, n);
}
origin: alibaba/jstorm

@Override
public Stream each(Fields inputFields, Function function, Fields functionFields) {
  projectionValidation(inputFields);
  return _topology.addSourcedNode(this,
      new ProcessorNode(_topology.getUniqueStreamId(),
          _name,
          TridentUtils.fieldsConcat(getOutputFields(), functionFields),
          functionFields,
          new EachProcessor(inputFields, function)));
}
//creates brand new tuples with brand new fields
origin: alibaba/jstorm

/**
 * ## Repartitioning Operation
 *
 * This method takes in a custom partitioning function that implements
 * {@link org.apache.storm.grouping.CustomStreamGrouping}
 *
 * @param grouping
 * @return
 */
public Stream partition(Grouping grouping) {
  if (_node instanceof PartitionNode) {
    return each(new Fields(), new TrueFilter()).partition(grouping);
  } else {
    return _topology.addSourcedNode(this, new PartitionNode(_node.streamId, _name, getOutputFields(), grouping));
  }
}
origin: com.alibaba.jstorm/jstorm-core

  private void projectionValidation(Fields projFields) {
    if (projFields == null) {
      return;
    }

    Fields allFields = this.getOutputFields();
    for (String field : projFields) {
      if (!allFields.contains(field)) {
        throw new IllegalArgumentException("Trying to select non-existent field: '" + field + "' from stream containing fields fields: <" + allFields + ">");
      }
    }
  }
}
origin: com.alibaba.jstorm/jstorm-core

private static List<Fields> strippedInputFields(List<Stream> streams, List<Fields> joinFields) {
  List<Fields> ret = new ArrayList<>();
  for(int i=0; i<streams.size(); i++) {
    ret.add(TridentUtils.fieldsSubtract(streams.get(i).getOutputFields(), joinFields.get(i)));
  }
  return ret;
}

origin: com.n3twork.storm/storm-core

  private void projectionValidation(Fields projFields) {
    if (projFields == null) {
      return;
    }

    Fields allFields = this.getOutputFields();
    for (String field : projFields) {
      if (!allFields.contains(field)) {
        throw new IllegalArgumentException("Trying to select non-existent field: '" + field + "' from stream containing fields fields: <" + allFields + ">");
      }
    }
  }
}
origin: com.alibaba.jstorm/jstorm-core

private <T> Stream comparableAggregateStream(String inputFieldName, Aggregator<T> aggregator) {
  if(inputFieldName != null) {
    projectionValidation(new Fields(inputFieldName));
  }
  return partitionAggregate(getOutputFields(), aggregator, getOutputFields());
}
origin: com.n3twork.storm/storm-core

@Override
public Stream apply(Stream input) {
  Fields outputFields = input.getOutputFields();
  return input.partitionAggregate(outputFields, _agg, outputFields)
        .global()
        .partitionAggregate(outputFields, _agg, outputFields);             
}

origin: com.alibaba.jstorm/jstorm-core

public Stream stateQuery(TridentState state, Fields inputFields, QueryFunction function, Fields functionFields) {
  projectionValidation(inputFields);
  String stateId = state._node.stateInfo.id;
  Node n = new ProcessorNode(_topology.getUniqueStreamId(),
          _name,
          TridentUtils.fieldsConcat(getOutputFields(), functionFields),
          functionFields,
          new StateQueryProcessor(stateId, inputFields, function));
  _topology._colocate.get(stateId).add(n);
  return _topology.addSourcedNode(this, n);
}
origin: com.alibaba.jstorm/jstorm-core

@Override
public Stream each(Fields inputFields, Function function, Fields functionFields) {
  projectionValidation(inputFields);
  return _topology.addSourcedNode(this,
      new ProcessorNode(_topology.getUniqueStreamId(),
          _name,
          TridentUtils.fieldsConcat(getOutputFields(), functionFields),
          functionFields,
          new EachProcessor(inputFields, function)));
}
//creates brand new tuples with brand new fields
storm.tridentStreamgetOutputFields

Popular methods of Stream

  • each
  • groupBy
    ## Grouping Operation
  • project
    Filters out fields from a stream, resulting in a Stream containing only the fields specified by `kee
  • aggregate
  • partitionBy
    ## Repartitioning Operation
  • shuffle
    ## Repartitioning Operation Use random round robin algorithm to evenly redistribute tuples across al
  • stateQuery
  • parallelismHint
    Applies a parallelism hint to a stream.
  • partitionAggregate
  • partitionPersist
  • global
    ## Repartitioning Operation All tuples are sent to the same partition. The same partition is chosen
  • batchGlobal
    ## Repartitioning Operation All tuples in the batch are sent to the same partition. Different batche
  • global,
  • batchGlobal,
  • chainedAgg,
  • <init>,
  • broadcast,
  • name,
  • partition,
  • persistentAggregate,
  • projectionValidation

Popular in Java

  • Making http post requests using okhttp
  • onCreateOptionsMenu (Activity)
  • getContentResolver (Context)
  • startActivity (Activity)
  • BufferedImage (java.awt.image)
    The BufferedImage subclass describes an java.awt.Image with an accessible buffer of image data. All
  • Runnable (java.lang)
    Represents a command that can be executed. Often used to run code in a different Thread.
  • Proxy (java.net)
    This class represents proxy server settings. A created instance of Proxy stores a type and an addres
  • Permission (java.security)
    Legacy security code; do not use.
  • Time (java.sql)
    Java representation of an SQL TIME value. Provides utilities to format and parse the time's represen
  • JOptionPane (javax.swing)
  • Top 17 Free Sublime Text Plugins
Tabnine Logo
  • Products

    Search for Java codeSearch for JavaScript code
  • IDE Plugins

    IntelliJ IDEAWebStormVisual StudioAndroid StudioEclipseVisual Studio CodePyCharmSublime TextPhpStormVimAtomGoLandRubyMineEmacsJupyter NotebookJupyter LabRiderDataGripAppCode
  • Company

    About UsContact UsCareers
  • Resources

    FAQBlogTabnine AcademyStudentsTerms of usePrivacy policyJava Code IndexJavascript Code Index
Get Tabnine for your IDE now