Tabnine Logo
StreamExecutionEnvironment.fromParallelCollection
Code IndexAdd Tabnine to your IDE (free)

How to use
fromParallelCollection
method
in
org.apache.flink.streaming.api.environment.StreamExecutionEnvironment

Best Java code snippets using org.apache.flink.streaming.api.environment.StreamExecutionEnvironment.fromParallelCollection (Showing top 11 results out of 315)

origin: apache/flink

/**
 * Creates a new data stream that contains elements in the iterator. The iterator is splittable,
 * allowing the framework to create a parallel data stream source that returns the elements in
 * the iterator.
 *
 * <p>Because the iterator will remain unmodified until the actual execution happens, the type
 * of data returned by the iterator must be given explicitly in the form of the type
 * information. This method is useful for cases where the type is generic. In that case, the
 * type class (as given in
 * {@link #fromParallelCollection(org.apache.flink.util.SplittableIterator, Class)} does not
 * supply all type information.
 *
 * @param iterator
 *         The iterator that produces the elements of the data stream
 * @param typeInfo
 *         The TypeInformation for the produced data stream.
 * @param <OUT>
 *         The type of the returned data stream
 * @return A data stream representing the elements in the iterator
 */
public <OUT> DataStreamSource<OUT> fromParallelCollection(SplittableIterator<OUT> iterator, TypeInformation<OUT>
    typeInfo) {
  return fromParallelCollection(iterator, typeInfo, "Parallel Collection Source");
}
origin: apache/flink

/**
 * Creates a new data stream that contains elements in the iterator. The iterator is splittable,
 * allowing the framework to create a parallel data stream source that returns the elements in
 * the iterator.
 *
 * <p>Because the iterator will remain unmodified until the actual execution happens, the type
 * of data returned by the iterator must be given explicitly in the form of the type class
 * (this is due to the fact that the Java compiler erases the generic type information).
 *
 * @param iterator
 *         The iterator that produces the elements of the data stream
 * @param type
 *         The class of the data produced by the iterator. Must not be a generic class.
 * @param <OUT>
 *         The type of the returned data stream
 * @return A data stream representing the elements in the iterator
 */
public <OUT> DataStreamSource<OUT> fromParallelCollection(SplittableIterator<OUT> iterator, Class<OUT> type) {
  return fromParallelCollection(iterator, TypeExtractor.getForClass(type));
}
origin: apache/flink

@Test
@SuppressWarnings("unchecked")
public void testFromCollectionParallelism() {
  try {
    TypeInformation<Integer> typeInfo = BasicTypeInfo.INT_TYPE_INFO;
    StreamExecutionEnvironment env = StreamExecutionEnvironment.getExecutionEnvironment();
    DataStreamSource<Integer> dataStream1 = env.fromCollection(new DummySplittableIterator<Integer>(), typeInfo);
    try {
      dataStream1.setParallelism(4);
      fail("should throw an exception");
    }
    catch (IllegalArgumentException e) {
      // expected
    }
    dataStream1.addSink(new DiscardingSink<Integer>());
    DataStreamSource<Integer> dataStream2 = env.fromParallelCollection(new DummySplittableIterator<Integer>(),
        typeInfo).setParallelism(4);
    dataStream2.addSink(new DiscardingSink<Integer>());
    env.getExecutionPlan();
    assertEquals("Parallelism of collection source must be 1.", 1, env.getStreamGraph().getStreamNode(dataStream1.getId()).getParallelism());
    assertEquals("Parallelism of parallel collection source must be 4.",
        4,
        env.getStreamGraph().getStreamNode(dataStream2.getId()).getParallelism());
  }
  catch (Exception e) {
    e.printStackTrace();
    fail(e.getMessage());
  }
}
origin: DTStack/flinkx

/**
 * Creates a new data stream that contains elements in the iterator. The iterator is splittable,
 * allowing the framework to create a parallel data stream source that returns the elements in
 * the iterator.
 *
 * <p>Because the iterator will remain unmodified until the actual execution happens, the type
 * of data returned by the iterator must be given explicitly in the form of the type
 * information. This method is useful for cases where the type is generic. In that case, the
 * type class (as given in
 * {@link #fromParallelCollection(org.apache.flink.util.SplittableIterator, Class)} does not
 * supply all type information.
 *
 * @param iterator
 *         The iterator that produces the elements of the data stream
 * @param typeInfo
 *         The TypeInformation for the produced data stream.
 * @param <OUT>
 *         The type of the returned data stream
 * @return A data stream representing the elements in the iterator
 */
public <OUT> DataStreamSource<OUT> fromParallelCollection(SplittableIterator<OUT> iterator, TypeInformation<OUT>
    typeInfo) {
  return fromParallelCollection(iterator, typeInfo, "Parallel Collection Source");
}
origin: org.apache.flink/flink-streaming-java_2.11

/**
 * Creates a new data stream that contains elements in the iterator. The iterator is splittable,
 * allowing the framework to create a parallel data stream source that returns the elements in
 * the iterator.
 *
 * <p>Because the iterator will remain unmodified until the actual execution happens, the type
 * of data returned by the iterator must be given explicitly in the form of the type
 * information. This method is useful for cases where the type is generic. In that case, the
 * type class (as given in
 * {@link #fromParallelCollection(org.apache.flink.util.SplittableIterator, Class)} does not
 * supply all type information.
 *
 * @param iterator
 *         The iterator that produces the elements of the data stream
 * @param typeInfo
 *         The TypeInformation for the produced data stream.
 * @param <OUT>
 *         The type of the returned data stream
 * @return A data stream representing the elements in the iterator
 */
public <OUT> DataStreamSource<OUT> fromParallelCollection(SplittableIterator<OUT> iterator, TypeInformation<OUT>
    typeInfo) {
  return fromParallelCollection(iterator, typeInfo, "Parallel Collection Source");
}
origin: org.apache.flink/flink-streaming-java_2.10

/**
 * Creates a new data stream that contains elements in the iterator. The iterator is splittable,
 * allowing the framework to create a parallel data stream source that returns the elements in
 * the iterator.
 *
 * <p>Because the iterator will remain unmodified until the actual execution happens, the type
 * of data returned by the iterator must be given explicitly in the form of the type
 * information. This method is useful for cases where the type is generic. In that case, the
 * type class (as given in
 * {@link #fromParallelCollection(org.apache.flink.util.SplittableIterator, Class)} does not
 * supply all type information.
 *
 * @param iterator
 *         The iterator that produces the elements of the data stream
 * @param typeInfo
 *         The TypeInformation for the produced data stream.
 * @param <OUT>
 *         The type of the returned data stream
 * @return A data stream representing the elements in the iterator
 */
public <OUT> DataStreamSource<OUT> fromParallelCollection(SplittableIterator<OUT> iterator, TypeInformation<OUT>
    typeInfo) {
  return fromParallelCollection(iterator, typeInfo, "Parallel Collection Source");
}
origin: org.apache.flink/flink-streaming-java

/**
 * Creates a new data stream that contains elements in the iterator. The iterator is splittable,
 * allowing the framework to create a parallel data stream source that returns the elements in
 * the iterator.
 *
 * <p>Because the iterator will remain unmodified until the actual execution happens, the type
 * of data returned by the iterator must be given explicitly in the form of the type
 * information. This method is useful for cases where the type is generic. In that case, the
 * type class (as given in
 * {@link #fromParallelCollection(org.apache.flink.util.SplittableIterator, Class)} does not
 * supply all type information.
 *
 * @param iterator
 *         The iterator that produces the elements of the data stream
 * @param typeInfo
 *         The TypeInformation for the produced data stream.
 * @param <OUT>
 *         The type of the returned data stream
 * @return A data stream representing the elements in the iterator
 */
public <OUT> DataStreamSource<OUT> fromParallelCollection(SplittableIterator<OUT> iterator, TypeInformation<OUT>
    typeInfo) {
  return fromParallelCollection(iterator, typeInfo, "Parallel Collection Source");
}
origin: org.apache.flink/flink-streaming-java_2.10

/**
 * Creates a new data stream that contains elements in the iterator. The iterator is splittable,
 * allowing the framework to create a parallel data stream source that returns the elements in
 * the iterator.
 *
 * <p>Because the iterator will remain unmodified until the actual execution happens, the type
 * of data returned by the iterator must be given explicitly in the form of the type class
 * (this is due to the fact that the Java compiler erases the generic type information).
 *
 * @param iterator
 *         The iterator that produces the elements of the data stream
 * @param type
 *         The class of the data produced by the iterator. Must not be a generic class.
 * @param <OUT>
 *         The type of the returned data stream
 * @return A data stream representing the elements in the iterator
 */
public <OUT> DataStreamSource<OUT> fromParallelCollection(SplittableIterator<OUT> iterator, Class<OUT> type) {
  return fromParallelCollection(iterator, TypeExtractor.getForClass(type));
}
origin: org.apache.flink/flink-streaming-java_2.11

/**
 * Creates a new data stream that contains elements in the iterator. The iterator is splittable,
 * allowing the framework to create a parallel data stream source that returns the elements in
 * the iterator.
 *
 * <p>Because the iterator will remain unmodified until the actual execution happens, the type
 * of data returned by the iterator must be given explicitly in the form of the type class
 * (this is due to the fact that the Java compiler erases the generic type information).
 *
 * @param iterator
 *         The iterator that produces the elements of the data stream
 * @param type
 *         The class of the data produced by the iterator. Must not be a generic class.
 * @param <OUT>
 *         The type of the returned data stream
 * @return A data stream representing the elements in the iterator
 */
public <OUT> DataStreamSource<OUT> fromParallelCollection(SplittableIterator<OUT> iterator, Class<OUT> type) {
  return fromParallelCollection(iterator, TypeExtractor.getForClass(type));
}
origin: org.apache.flink/flink-streaming-java

/**
 * Creates a new data stream that contains elements in the iterator. The iterator is splittable,
 * allowing the framework to create a parallel data stream source that returns the elements in
 * the iterator.
 *
 * <p>Because the iterator will remain unmodified until the actual execution happens, the type
 * of data returned by the iterator must be given explicitly in the form of the type class
 * (this is due to the fact that the Java compiler erases the generic type information).
 *
 * @param iterator
 *         The iterator that produces the elements of the data stream
 * @param type
 *         The class of the data produced by the iterator. Must not be a generic class.
 * @param <OUT>
 *         The type of the returned data stream
 * @return A data stream representing the elements in the iterator
 */
public <OUT> DataStreamSource<OUT> fromParallelCollection(SplittableIterator<OUT> iterator, Class<OUT> type) {
  return fromParallelCollection(iterator, TypeExtractor.getForClass(type));
}
origin: DTStack/flinkx

/**
 * Creates a new data stream that contains elements in the iterator. The iterator is splittable,
 * allowing the framework to create a parallel data stream source that returns the elements in
 * the iterator.
 *
 * <p>Because the iterator will remain unmodified until the actual execution happens, the type
 * of data returned by the iterator must be given explicitly in the form of the type class
 * (this is due to the fact that the Java compiler erases the generic type information).
 *
 * @param iterator
 *         The iterator that produces the elements of the data stream
 * @param type
 *         The class of the data produced by the iterator. Must not be a generic class.
 * @param <OUT>
 *         The type of the returned data stream
 * @return A data stream representing the elements in the iterator
 */
public <OUT> DataStreamSource<OUT> fromParallelCollection(SplittableIterator<OUT> iterator, Class<OUT> type) {
  return fromParallelCollection(iterator, TypeExtractor.getForClass(type));
}
org.apache.flink.streaming.api.environmentStreamExecutionEnvironmentfromParallelCollection

Javadoc

Creates a new data stream that contains elements in the iterator. The iterator is splittable, allowing the framework to create a parallel data stream source that returns the elements in the iterator.

Because the iterator will remain unmodified until the actual execution happens, the type of data returned by the iterator must be given explicitly in the form of the type class (this is due to the fact that the Java compiler erases the generic type information).

Popular methods of StreamExecutionEnvironment

  • execute
  • getExecutionEnvironment
    Creates an execution environment that represents the context in which the program is currently execu
  • addSource
    Ads a data source with a custom type information thus opening a DataStream. Only in very special cas
  • getConfig
    Gets the config object.
  • enableCheckpointing
    Enables checkpointing for the streaming job. The distributed state of the streaming dataflow will be
  • setStreamTimeCharacteristic
    Sets the time characteristic for all streams create from this environment, e.g., processing time, ev
  • setParallelism
    Sets the parallelism for operations executed through this environment. Setting a parallelism of x he
  • fromElements
    Creates a new data stream that contains the given elements. The elements must all be of the same typ
  • setStateBackend
    Sets the state backend that describes how to store and checkpoint operator state. It defines both wh
  • createLocalEnvironment
    Creates a LocalStreamEnvironment. The local execution environment will run the program in a multi-th
  • fromCollection
    Creates a data stream from the given iterator.Because the iterator will remain unmodified until the
  • getCheckpointConfig
    Gets the checkpoint config, which defines values like checkpoint interval, delay between checkpoints
  • fromCollection,
  • getCheckpointConfig,
  • getParallelism,
  • getStreamGraph,
  • setRestartStrategy,
  • socketTextStream,
  • readTextFile,
  • generateSequence,
  • clean,
  • getStreamTimeCharacteristic

Popular in Java

  • Finding current android device location
  • getOriginalFilename (MultipartFile)
    Return the original filename in the client's filesystem.This may contain path information depending
  • getSharedPreferences (Context)
  • onRequestPermissionsResult (Fragment)
  • MalformedURLException (java.net)
    This exception is thrown when a program attempts to create an URL from an incorrect specification.
  • Map (java.util)
    A Map is a data structure consisting of a set of keys and values in which each key is mapped to a si
  • Executor (java.util.concurrent)
    An object that executes submitted Runnable tasks. This interface provides a way of decoupling task s
  • ThreadPoolExecutor (java.util.concurrent)
    An ExecutorService that executes each submitted task using one of possibly several pooled threads, n
  • Stream (java.util.stream)
    A sequence of elements supporting sequential and parallel aggregate operations. The following exampl
  • Modifier (javassist)
    The Modifier class provides static methods and constants to decode class and member access modifiers
  • Top plugins for WebStorm
Tabnine Logo
  • Products

    Search for Java codeSearch for JavaScript code
  • IDE Plugins

    IntelliJ IDEAWebStormVisual StudioAndroid StudioEclipseVisual Studio CodePyCharmSublime TextPhpStormVimGoLandRubyMineEmacsJupyter NotebookJupyter LabRiderDataGripAppCode
  • Company

    About UsContact UsCareers
  • Resources

    FAQBlogTabnine AcademyTerms of usePrivacy policyJava Code IndexJavascript Code Index
Get Tabnine for your IDE now