congrats Icon
New! Tabnine Pro 14-day free trial
Start a free trial
Tabnine Logo
StreamExecutionEnvironment.disableOperatorChaining
Code IndexAdd Tabnine to your IDE (free)

How to use
disableOperatorChaining
method
in
org.apache.flink.streaming.api.environment.StreamExecutionEnvironment

Best Java code snippets using org.apache.flink.streaming.api.environment.StreamExecutionEnvironment.disableOperatorChaining (Showing top 8 results out of 315)

origin: apache/flink

env1.getConfig().setRestartStrategy(RestartStrategies.noRestart());
env1.getConfig().disableSysoutLogging();
env1.disableOperatorChaining(); // let the source read everything into the network buffers
origin: apache/flink

/**
 * Tests that there are no collisions with two identical sources.
 *
 * <pre>
 * [ (src0) ] --\
 *               +--> [ (sink) ]
 * [ (src1) ] --/
 * </pre>
 */
@Test
public void testNodeHashIdenticalSources() throws Exception {
  StreamExecutionEnvironment env = StreamExecutionEnvironment.createLocalEnvironment();
  env.setParallelism(4);
  env.disableOperatorChaining();
  DataStream<String> src0 = env.addSource(new NoOpSourceFunction());
  DataStream<String> src1 = env.addSource(new NoOpSourceFunction());
  src0.union(src1).addSink(new NoOpSinkFunction());
  JobGraph jobGraph = env.getStreamGraph().getJobGraph();
  List<JobVertex> vertices = jobGraph.getVerticesSortedTopologicallyFromSources();
  assertTrue(vertices.get(0).isInputVertex());
  assertTrue(vertices.get(1).isInputVertex());
  assertNotNull(vertices.get(0).getID());
  assertNotNull(vertices.get(1).getID());
  assertNotEquals(vertices.get(0).getID(), vertices.get(1).getID());
}
origin: apache/flink

/**
 * Tests that there are no collisions with two identical intermediate nodes connected to the
 * same predecessor.
 *
 * <pre>
 *             /-> [ (map) ] -> [ (sink) ]
 * [ (src) ] -+
 *             \-> [ (map) ] -> [ (sink) ]
 * </pre>
 */
@Test
public void testNodeHashIdenticalNodes() throws Exception {
  StreamExecutionEnvironment env = StreamExecutionEnvironment.createLocalEnvironment();
  env.setParallelism(4);
  env.disableOperatorChaining();
  DataStream<String> src = env.addSource(new NoOpSourceFunction());
  src.map(new NoOpMapFunction()).addSink(new NoOpSinkFunction());
  src.map(new NoOpMapFunction()).addSink(new NoOpSinkFunction());
  JobGraph jobGraph = env.getStreamGraph().getJobGraph();
  Set<JobVertexID> vertexIds = new HashSet<>();
  for (JobVertex vertex : jobGraph.getVertices()) {
    assertTrue(vertexIds.add(vertex.getID()));
  }
}
origin: apache/flink

/**
 * Tests that a collision on the manual hash throws an Exception.
 */
@Test(expected = IllegalArgumentException.class)
public void testManualHashAssignmentCollisionThrowsException() throws Exception {
  StreamExecutionEnvironment env = StreamExecutionEnvironment.createLocalEnvironment();
  env.setParallelism(4);
  env.disableOperatorChaining();
  env.addSource(new NoOpSourceFunction()).uid("source")
      .map(new NoOpMapFunction()).uid("source") // Collision
      .addSink(new NoOpSinkFunction());
  // This call is necessary to generate the job graph
  env.getStreamGraph().getJobGraph();
}
origin: apache/flink

/**
 * Creates a streaming JobGraph from the StreamEnvironment.
 */
private JobGraph createJobGraph(
  int parallelism,
  int numberOfRetries,
  long restartDelay) {
  StreamExecutionEnvironment env = StreamExecutionEnvironment.getExecutionEnvironment();
  env.setParallelism(parallelism);
  env.disableOperatorChaining();
  env.getConfig().setRestartStrategy(RestartStrategies.fixedDelayRestart(numberOfRetries, restartDelay));
  env.getConfig().disableSysoutLogging();
  DataStream<Integer> stream = env
    .addSource(new InfiniteTestSource())
    .shuffle()
    .map(new StatefulCounter());
  stream.addSink(new DiscardingSink<>());
  return env.getStreamGraph().getJobGraph();
}
origin: apache/flink

StreamExecutionEnvironment env = StreamExecutionEnvironment.createLocalEnvironment();
env.setParallelism(4);
env.disableOperatorChaining();
env.disableOperatorChaining();
origin: dataArtisans/oscon

env.enableCheckpointing(1000);
env.setParallelism(1);
env.disableOperatorChaining();
origin: dataArtisans/oscon

env.setMaxParallelism(8);
env.setParallelism(1);
env.disableOperatorChaining();
env.getConfig().setLatencyTrackingInterval(1000);
org.apache.flink.streaming.api.environmentStreamExecutionEnvironmentdisableOperatorChaining

Javadoc

Disables operator chaining for streaming operators. Operator chaining allows non-shuffle operations to be co-located in the same thread fully avoiding serialization and de-serialization.

Popular methods of StreamExecutionEnvironment

  • execute
  • getExecutionEnvironment
    Creates an execution environment that represents the context in which the program is currently execu
  • addSource
    Ads a data source with a custom type information thus opening a DataStream. Only in very special cas
  • getConfig
    Gets the config object.
  • enableCheckpointing
    Enables checkpointing for the streaming job. The distributed state of the streaming dataflow will be
  • setStreamTimeCharacteristic
    Sets the time characteristic for all streams create from this environment, e.g., processing time, ev
  • setParallelism
    Sets the parallelism for operations executed through this environment. Setting a parallelism of x he
  • fromElements
    Creates a new data stream that contains the given elements. The elements must all be of the same typ
  • setStateBackend
    Sets the state backend that describes how to store and checkpoint operator state. It defines both wh
  • createLocalEnvironment
    Creates a LocalStreamEnvironment. The local execution environment will run the program in a multi-th
  • fromCollection
    Creates a data stream from the given iterator.Because the iterator will remain unmodified until the
  • getCheckpointConfig
    Gets the checkpoint config, which defines values like checkpoint interval, delay between checkpoints
  • fromCollection,
  • getCheckpointConfig,
  • getParallelism,
  • getStreamGraph,
  • setRestartStrategy,
  • socketTextStream,
  • readTextFile,
  • generateSequence,
  • clean,
  • getStreamTimeCharacteristic

Popular in Java

  • Parsing JSON documents to java classes using gson
  • onCreateOptionsMenu (Activity)
  • scheduleAtFixedRate (ScheduledExecutorService)
  • startActivity (Activity)
  • Menu (java.awt)
  • BigDecimal (java.math)
    An immutable arbitrary-precision signed decimal.A value is represented by an arbitrary-precision "un
  • SocketTimeoutException (java.net)
    This exception is thrown when a timeout expired on a socket read or accept operation.
  • URLConnection (java.net)
    A connection to a URL for reading or writing. For HTTP connections, see HttpURLConnection for docume
  • NumberFormat (java.text)
    The abstract base class for all number formats. This class provides the interface for formatting and
  • Random (java.util)
    This class provides methods that return pseudo-random values.It is dangerous to seed Random with the
  • Top plugins for WebStorm
Tabnine Logo
  • Products

    Search for Java codeSearch for JavaScript code
  • IDE Plugins

    IntelliJ IDEAWebStormVisual StudioAndroid StudioEclipseVisual Studio CodePyCharmSublime TextPhpStormVimAtomGoLandRubyMineEmacsJupyter NotebookJupyter LabRiderDataGripAppCode
  • Company

    About UsContact UsCareers
  • Resources

    FAQBlogTabnine AcademyStudentsTerms of usePrivacy policyJava Code IndexJavascript Code Index
Get Tabnine for your IDE now