congrats Icon
New! Announcing Tabnine Chat Beta
Learn More
Tabnine Logo
StreamExecutionEnvironment.getCachedFiles
Code IndexAdd Tabnine to your IDE (free)

How to use
getCachedFiles
method
in
org.apache.flink.streaming.api.environment.StreamExecutionEnvironment

Best Java code snippets using org.apache.flink.streaming.api.environment.StreamExecutionEnvironment.getCachedFiles (Showing top 4 results out of 315)

origin: apache/flink

private JobGraph createJobGraph() {
  // make sure that all vertices start immediately
  jobGraph.setScheduleMode(ScheduleMode.EAGER);
  // Generate deterministic hashes for the nodes in order to identify them across
  // submission iff they didn't change.
  Map<Integer, byte[]> hashes = defaultStreamGraphHasher.traverseStreamGraphAndGenerateHashes(streamGraph);
  // Generate legacy version hashes for backwards compatibility
  List<Map<Integer, byte[]>> legacyHashes = new ArrayList<>(legacyStreamGraphHashers.size());
  for (StreamGraphHasher hasher : legacyStreamGraphHashers) {
    legacyHashes.add(hasher.traverseStreamGraphAndGenerateHashes(streamGraph));
  }
  Map<Integer, List<Tuple2<byte[], byte[]>>> chainedOperatorHashes = new HashMap<>();
  setChaining(hashes, legacyHashes, chainedOperatorHashes);
  setPhysicalEdges();
  setSlotSharingAndCoLocation();
  configureCheckpointing();
  JobGraphGenerator.addUserArtifactEntries(streamGraph.getEnvironment().getCachedFiles(), jobGraph);
  // set the ExecutionConfig last when it has been finalized
  try {
    jobGraph.setExecutionConfig(streamGraph.getExecutionConfig());
  }
  catch (IOException e) {
    throw new IllegalConfigurationException("Could not serialize the ExecutionConfig." +
        "This indicates that non-serializable types (like custom serializers) were registered");
  }
  return jobGraph;
}
origin: org.apache.flink/flink-streaming-java_2.11

private JobGraph createJobGraph() {
  // make sure that all vertices start immediately
  jobGraph.setScheduleMode(ScheduleMode.EAGER);
  // Generate deterministic hashes for the nodes in order to identify them across
  // submission iff they didn't change.
  Map<Integer, byte[]> hashes = defaultStreamGraphHasher.traverseStreamGraphAndGenerateHashes(streamGraph);
  // Generate legacy version hashes for backwards compatibility
  List<Map<Integer, byte[]>> legacyHashes = new ArrayList<>(legacyStreamGraphHashers.size());
  for (StreamGraphHasher hasher : legacyStreamGraphHashers) {
    legacyHashes.add(hasher.traverseStreamGraphAndGenerateHashes(streamGraph));
  }
  Map<Integer, List<Tuple2<byte[], byte[]>>> chainedOperatorHashes = new HashMap<>();
  setChaining(hashes, legacyHashes, chainedOperatorHashes);
  setPhysicalEdges();
  setSlotSharingAndCoLocation();
  configureCheckpointing();
  JobGraphGenerator.addUserArtifactEntries(streamGraph.getEnvironment().getCachedFiles(), jobGraph);
  // set the ExecutionConfig last when it has been finalized
  try {
    jobGraph.setExecutionConfig(streamGraph.getExecutionConfig());
  }
  catch (IOException e) {
    throw new IllegalConfigurationException("Could not serialize the ExecutionConfig." +
        "This indicates that non-serializable types (like custom serializers) were registered");
  }
  return jobGraph;
}
origin: org.apache.flink/flink-streaming-java

private JobGraph createJobGraph() {
  // make sure that all vertices start immediately
  jobGraph.setScheduleMode(ScheduleMode.EAGER);
  // Generate deterministic hashes for the nodes in order to identify them across
  // submission iff they didn't change.
  Map<Integer, byte[]> hashes = defaultStreamGraphHasher.traverseStreamGraphAndGenerateHashes(streamGraph);
  // Generate legacy version hashes for backwards compatibility
  List<Map<Integer, byte[]>> legacyHashes = new ArrayList<>(legacyStreamGraphHashers.size());
  for (StreamGraphHasher hasher : legacyStreamGraphHashers) {
    legacyHashes.add(hasher.traverseStreamGraphAndGenerateHashes(streamGraph));
  }
  Map<Integer, List<Tuple2<byte[], byte[]>>> chainedOperatorHashes = new HashMap<>();
  setChaining(hashes, legacyHashes, chainedOperatorHashes);
  setPhysicalEdges();
  setSlotSharingAndCoLocation();
  configureCheckpointing();
  JobGraphGenerator.addUserArtifactEntries(streamGraph.getEnvironment().getCachedFiles(), jobGraph);
  // set the ExecutionConfig last when it has been finalized
  try {
    jobGraph.setExecutionConfig(streamGraph.getExecutionConfig());
  }
  catch (IOException e) {
    throw new IllegalConfigurationException("Could not serialize the ExecutionConfig." +
        "This indicates that non-serializable types (like custom serializers) were registered");
  }
  return jobGraph;
}
origin: org.apache.flink/flink-streaming-java_2.10

private JobGraph createJobGraph() {
  // make sure that all vertices start immediately
  jobGraph.setScheduleMode(ScheduleMode.EAGER);
  // Generate deterministic hashes for the nodes in order to identify them across
  // submission iff they didn't change.
  Map<Integer, byte[]> hashes = defaultStreamGraphHasher.traverseStreamGraphAndGenerateHashes(streamGraph);
  // Generate legacy version hashes for backwards compatibility
  List<Map<Integer, byte[]>> legacyHashes = new ArrayList<>(legacyStreamGraphHashers.size());
  for (StreamGraphHasher hasher : legacyStreamGraphHashers) {
    legacyHashes.add(hasher.traverseStreamGraphAndGenerateHashes(streamGraph));
  }
  Map<Integer, List<Tuple2<byte[], byte[]>>> chainedOperatorHashes = new HashMap<>();
  setChaining(hashes, legacyHashes, chainedOperatorHashes);
  setPhysicalEdges();
  setSlotSharing();
  configureCheckpointing();
  // add registered cache file into job configuration
  for (Tuple2<String, DistributedCache.DistributedCacheEntry> e : streamGraph.getEnvironment().getCachedFiles()) {
    DistributedCache.writeFileInfoToConfig(e.f0, e.f1, jobGraph.getJobConfiguration());
  }
  // set the ExecutionConfig last when it has been finalized
  try {
    jobGraph.setExecutionConfig(streamGraph.getExecutionConfig());
  }
  catch (IOException e) {
    throw new IllegalConfigurationException("Could not serialize the ExecutionConfig." +
        "This indicates that non-serializable types (like custom serializers) were registered");
  }
  return jobGraph;
}
org.apache.flink.streaming.api.environmentStreamExecutionEnvironmentgetCachedFiles

Javadoc

Get the list of cached files that were registered for distribution among the task managers.

Popular methods of StreamExecutionEnvironment

  • execute
  • getExecutionEnvironment
    Creates an execution environment that represents the context in which the program is currently execu
  • addSource
    Ads a data source with a custom type information thus opening a DataStream. Only in very special cas
  • getConfig
    Gets the config object.
  • enableCheckpointing
    Enables checkpointing for the streaming job. The distributed state of the streaming dataflow will be
  • setStreamTimeCharacteristic
    Sets the time characteristic for all streams create from this environment, e.g., processing time, ev
  • setParallelism
    Sets the parallelism for operations executed through this environment. Setting a parallelism of x he
  • fromElements
    Creates a new data stream that contains the given elements. The elements must all be of the same typ
  • setStateBackend
    Sets the state backend that describes how to store and checkpoint operator state. It defines both wh
  • createLocalEnvironment
    Creates a LocalStreamEnvironment. The local execution environment will run the program in a multi-th
  • fromCollection
    Creates a data stream from the given iterator.Because the iterator will remain unmodified until the
  • getCheckpointConfig
    Gets the checkpoint config, which defines values like checkpoint interval, delay between checkpoints
  • fromCollection,
  • getCheckpointConfig,
  • getParallelism,
  • getStreamGraph,
  • setRestartStrategy,
  • socketTextStream,
  • readTextFile,
  • generateSequence,
  • clean,
  • getStreamTimeCharacteristic

Popular in Java

  • Making http requests using okhttp
  • getOriginalFilename (MultipartFile)
    Return the original filename in the client's filesystem.This may contain path information depending
  • addToBackStack (FragmentTransaction)
  • requestLocationUpdates (LocationManager)
  • GridLayout (java.awt)
    The GridLayout class is a layout manager that lays out a container's components in a rectangular gri
  • BufferedReader (java.io)
    Wraps an existing Reader and buffers the input. Expensive interaction with the underlying reader is
  • CountDownLatch (java.util.concurrent)
    A synchronization aid that allows one or more threads to wait until a set of operations being perfor
  • JOptionPane (javax.swing)
  • Options (org.apache.commons.cli)
    Main entry-point into the library. Options represents a collection of Option objects, which describ
  • IsNull (org.hamcrest.core)
    Is the value null?
  • Best plugins for Eclipse
Tabnine Logo
  • Products

    Search for Java codeSearch for JavaScript code
  • IDE Plugins

    IntelliJ IDEAWebStormVisual StudioAndroid StudioEclipseVisual Studio CodePyCharmSublime TextPhpStormVimGoLandRubyMineEmacsJupyter NotebookJupyter LabRiderDataGripAppCode
  • Company

    About UsContact UsCareers
  • Resources

    FAQBlogTabnine AcademyTerms of usePrivacy policyJava Code IndexJavascript Code Index
Get Tabnine for your IDE now