Tabnine Logo
StreamExecutionEnvironment.clean
Code IndexAdd Tabnine to your IDE (free)

How to use
clean
method
in
org.apache.flink.streaming.api.environment.StreamExecutionEnvironment

Best Java code snippets using org.apache.flink.streaming.api.environment.StreamExecutionEnvironment.clean (Showing top 20 results out of 315)

origin: apache/flink

public <F> F clean(F f) {
  return environment.clean(f);
}
origin: apache/flink

/**
 * Invokes the {@link org.apache.flink.api.java.ClosureCleaner}
 * on the given function if closure cleaning is enabled in the {@link ExecutionConfig}.
 *
 * @return The cleaned Function
 */
protected <F> F clean(F f) {
  return getExecutionEnvironment().clean(f);
}
origin: apache/flink

/**
 * Invokes the {@link org.apache.flink.api.java.ClosureCleaner}
 * on the given function if closure cleaning is enabled in the {@link ExecutionConfig}.
 *
 * @return The cleaned Function
 */
<F> F clean(F f) {
  return inputStream.getExecutionEnvironment().clean(f);
}
origin: apache/flink

  protected <F> F clean(F f) {
    return getExecutionEnvironment().clean(f);
  }
}
origin: apache/flink

/**
 * Send late arriving data to the side output identified by the given {@link OutputTag}. Data
 * is considered late after the watermark has passed the end of the window plus the allowed
 * lateness set using {@link #allowedLateness(Time)}.
 *
 * <p>You can get the stream of late data using
 * {@link SingleOutputStreamOperator#getSideOutput(OutputTag)} on the
 * {@link SingleOutputStreamOperator} resulting from the windowed operation
 * with the same {@link OutputTag}.
 */
@PublicEvolving
public AllWindowedStream<T, W> sideOutputLateData(OutputTag<T> outputTag) {
  Preconditions.checkNotNull(outputTag, "Side output tag must not be null.");
  this.lateDataOutputTag = input.getExecutionEnvironment().clean(outputTag);
  return this;
}
origin: apache/flink

/**
 * Applies the given window function to each window. The window function is called for each
 * evaluation of the window for each key individually. The output of the window function is
 * interpreted as a regular non-windowed stream.
 *
 * <p>Note that this function requires that all data in the windows is buffered until the window
 * is evaluated, as the function provides no means of incremental aggregation.
 *
 * @param function The window function.
 * @param resultType Type information for the result type of the window function
 * @return The data stream that is the result of applying the window function to the window.
 */
public <R> SingleOutputStreamOperator<R> apply(WindowFunction<T, R, K, W> function, TypeInformation<R> resultType) {
  function = input.getExecutionEnvironment().clean(function);
  return apply(new InternalIterableWindowFunction<>(function), resultType, function);
}
origin: apache/flink

/**
 * Applies a reduce function to the window. The window function is called for each evaluation
 * of the window for each key individually. The output of the reduce function is interpreted
 * as a regular non-windowed stream.
 *
 * <p>This window will try and incrementally aggregate data as much as the window policies
 * permit. For example, tumbling time windows can aggregate the data, meaning that only one
 * element per key is stored. Sliding time windows will aggregate on the granularity of the
 * slide interval, so a few elements are stored per key (one per slide interval).
 * Custom windows may not be able to incrementally aggregate, or may need to store extra values
 * in an aggregation tree.
 *
 * @param function The reduce function.
 * @return The data stream that is the result of applying the reduce function to the window.
 */
@SuppressWarnings("unchecked")
public SingleOutputStreamOperator<T> reduce(ReduceFunction<T> function) {
  if (function instanceof RichFunction) {
    throw new UnsupportedOperationException("ReduceFunction of reduce can not be a RichFunction. " +
      "Please use reduce(ReduceFunction, WindowFunction) instead.");
  }
  //clean the closure
  function = input.getExecutionEnvironment().clean(function);
  return reduce(function, new PassThroughWindowFunction<K, W, T>());
}
origin: apache/flink

/**
 * Send late arriving data to the side output identified by the given {@link OutputTag}. Data
 * is considered late after the watermark has passed the end of the window plus the allowed
 * lateness set using {@link #allowedLateness(Time)}.
 *
 * <p>You can get the stream of late data using
 * {@link SingleOutputStreamOperator#getSideOutput(OutputTag)} on the
 * {@link SingleOutputStreamOperator} resulting from the windowed operation
 * with the same {@link OutputTag}.
 */
@PublicEvolving
public WindowedStream<T, K, W> sideOutputLateData(OutputTag<T> outputTag) {
  Preconditions.checkNotNull(outputTag, "Side output tag must not be null.");
  this.lateDataOutputTag = input.getExecutionEnvironment().clean(outputTag);
  return this;
}
origin: apache/flink

/**
 * Applies a reduce function to the window. The window function is called for each evaluation
 * of the window for each key individually. The output of the reduce function is interpreted
 * as a regular non-windowed stream.
 *
 * <p>This window will try and incrementally aggregate data as much as the window policies permit.
 * For example, tumbling time windows can aggregate the data, meaning that only one element per
 * key is stored. Sliding time windows will aggregate on the granularity of the slide interval,
 * so a few elements are stored per key (one per slide interval).
 * Custom windows may not be able to incrementally aggregate, or may need to store extra values
 * in an aggregation tree.
 *
 * @param function The reduce function.
 * @return The data stream that is the result of applying the reduce function to the window.
 */
@SuppressWarnings("unchecked")
public SingleOutputStreamOperator<T> reduce(ReduceFunction<T> function) {
  if (function instanceof RichFunction) {
    throw new UnsupportedOperationException("ReduceFunction of reduce can not be a RichFunction. " +
        "Please use reduce(ReduceFunction, WindowFunction) instead.");
  }
  //clean the closure
  function = input.getExecutionEnvironment().clean(function);
  String callLocation = Utils.getCallLocationName();
  String udfName = "AllWindowedStream." + callLocation;
  return reduce(function, new PassThroughAllWindowFunction<W, T>());
}
origin: apache/flink

/**
 * Applies the given window function to each window. The window function is called for each
 * evaluation of the window. The output of the window function is
 * interpreted as a regular non-windowed stream.
 *
 * <p>Not that this function requires that all data in the windows is buffered until the window
 * is evaluated, as the function provides no means of incremental aggregation.
 *
 * @param function The window function.
 * @return The data stream that is the result of applying the window function to the window.
 */
public <R> SingleOutputStreamOperator<R> apply(AllWindowFunction<T, R, W> function, TypeInformation<R> resultType) {
  String callLocation = Utils.getCallLocationName();
  function = input.getExecutionEnvironment().clean(function);
  return apply(new InternalIterableAllWindowFunction<>(function), resultType, callLocation);
}
origin: apache/flink

/**
 * Applies the given window function to each window. The window function is called for each
 * evaluation of the window for each key individually. The output of the window function is
 * interpreted as a regular non-windowed stream.
 *
 * <p>Note that this function requires that all data in the windows is buffered until the window
 * is evaluated, as the function provides no means of incremental aggregation.
 *
 * @param function The window function.
 * @param resultType Type information for the result type of the window function
 * @return The data stream that is the result of applying the window function to the window.
 */
@Internal
public <R> SingleOutputStreamOperator<R> process(ProcessWindowFunction<T, R, K, W> function, TypeInformation<R> resultType) {
  function = input.getExecutionEnvironment().clean(function);
  return apply(new InternalIterableProcessWindowFunction<>(function), resultType, function);
}
origin: apache/flink

/**
 * Applies the given window function to each window. The window function is called for each
 * evaluation of the window. The output of the window function is
 * interpreted as a regular non-windowed stream.
 *
 * <p>Not that this function requires that all data in the windows is buffered until the window
 * is evaluated, as the function provides no means of incremental aggregation.
 *
 * @param function The process window function.
 * @return The data stream that is the result of applying the window function to the window.
 */
@PublicEvolving
public <R> SingleOutputStreamOperator<R> process(ProcessAllWindowFunction<T, R, W> function, TypeInformation<R> resultType) {
  String callLocation = Utils.getCallLocationName();
  function = input.getExecutionEnvironment().clean(function);
  return apply(new InternalIterableProcessAllWindowFunction<>(function), resultType, callLocation);
}
origin: apache/flink

/**
 * Applies the given window function to each window. The window function is called for each
 * evaluation of the window. The output of the window function is
 * interpreted as a regular non-windowed stream.
 *
 * <p>Not that this function requires that all data in the windows is buffered until the window
 * is evaluated, as the function provides no means of incremental aggregation.
 *
 * @param function The window function.
 * @return The data stream that is the result of applying the window function to the window.
 */
public <R> SingleOutputStreamOperator<R> apply(AllWindowFunction<T, R, W> function) {
  String callLocation = Utils.getCallLocationName();
  function = input.getExecutionEnvironment().clean(function);
  TypeInformation<R> resultType = getAllWindowFunctionReturnType(function, getInputType());
  return apply(new InternalIterableAllWindowFunction<>(function), resultType, callLocation);
}
origin: apache/flink

clean(function);
StreamSource<OUT, ?> sourceOperator;
if (function instanceof StoppableFunction) {
origin: apache/flink

/**
 * Applies the given window function to each window. The window function is called for each
 * evaluation of the window. The output of the window function is
 * interpreted as a regular non-windowed stream.
 *
 * <p>Not that this function requires that all data in the windows is buffered until the window
 * is evaluated, as the function provides no means of incremental aggregation.
 *
 * @param function The process window function.
 * @return The data stream that is the result of applying the window function to the window.
 */
@PublicEvolving
public <R> SingleOutputStreamOperator<R> process(ProcessAllWindowFunction<T, R, W> function) {
  String callLocation = Utils.getCallLocationName();
  function = input.getExecutionEnvironment().clean(function);
  TypeInformation<R> resultType = getProcessAllWindowFunctionReturnType(function, getInputType());
  return apply(new InternalIterableProcessAllWindowFunction<>(function), resultType, callLocation);
}
origin: apache/flink

in.getExecutionEnvironment().clean(func),
timeout,
bufSize,
origin: apache/flink

/**
 * Completes the join operation with the user function that is executed
 * for each combination of elements with the same key in a window.
 *
 * <p>Note: This method's return type does not support setting an operator-specific parallelism.
 * Due to binary backwards compatibility, this cannot be altered. Use the
 * {@link #with(JoinFunction, TypeInformation)}, method to set an operator-specific parallelism.
 */
public <T> DataStream<T> apply(JoinFunction<T1, T2, T> function, TypeInformation<T> resultType) {
  //clean the closure
  function = input1.getExecutionEnvironment().clean(function);
  coGroupedWindowedStream = input1.coGroup(input2)
    .where(keySelector1)
    .equalTo(keySelector2)
    .window(windowAssigner)
    .trigger(trigger)
    .evictor(evictor)
    .allowedLateness(allowedLateness);
  return coGroupedWindowedStream
      .apply(new JoinCoGroupFunction<>(function), resultType);
}
origin: apache/flink

/**
 * Completes the join operation with the user function that is executed
 * for each combination of elements with the same key in a window.
 *
 * <p>Note: This method's return type does not support setting an operator-specific parallelism.
 * Due to binary backwards compatibility, this cannot be altered. Use the
 * {@link #with(JoinFunction, TypeInformation)}, method to set an operator-specific parallelism.
 */
public <T> DataStream<T> apply(FlatJoinFunction<T1, T2, T> function, TypeInformation<T> resultType) {
  //clean the closure
  function = input1.getExecutionEnvironment().clean(function);
  coGroupedWindowedStream = input1.coGroup(input2)
    .where(keySelector1)
    .equalTo(keySelector2)
    .window(windowAssigner)
    .trigger(trigger)
    .evictor(evictor)
    .allowedLateness(allowedLateness);
  return coGroupedWindowedStream
      .apply(new FlatJoinCoGroupFunction<>(function), resultType);
}
origin: apache/flink

Preconditions.checkNotNull(outputType);
final ProcessJoinFunction<IN1, IN2, OUT> cleanedUdf = left.getExecutionEnvironment().clean(processJoinFunction);
origin: apache/flink

function = input1.getExecutionEnvironment().clean(function);
org.apache.flink.streaming.api.environmentStreamExecutionEnvironmentclean

Javadoc

Returns a "closure-cleaned" version of the given function. Cleans only if closure cleaning is not disabled in the org.apache.flink.api.common.ExecutionConfig

Popular methods of StreamExecutionEnvironment

  • execute
  • getExecutionEnvironment
    Creates an execution environment that represents the context in which the program is currently execu
  • addSource
    Ads a data source with a custom type information thus opening a DataStream. Only in very special cas
  • getConfig
    Gets the config object.
  • enableCheckpointing
    Enables checkpointing for the streaming job. The distributed state of the streaming dataflow will be
  • setStreamTimeCharacteristic
    Sets the time characteristic for all streams create from this environment, e.g., processing time, ev
  • setParallelism
    Sets the parallelism for operations executed through this environment. Setting a parallelism of x he
  • fromElements
    Creates a new data stream that contains the given elements. The elements must all be of the same typ
  • setStateBackend
    Sets the state backend that describes how to store and checkpoint operator state. It defines both wh
  • createLocalEnvironment
    Creates a LocalStreamEnvironment. The local execution environment will run the program in a multi-th
  • fromCollection
    Creates a data stream from the given iterator.Because the iterator will remain unmodified until the
  • getCheckpointConfig
    Gets the checkpoint config, which defines values like checkpoint interval, delay between checkpoints
  • fromCollection,
  • getCheckpointConfig,
  • getParallelism,
  • getStreamGraph,
  • setRestartStrategy,
  • socketTextStream,
  • readTextFile,
  • generateSequence,
  • getStreamTimeCharacteristic

Popular in Java

  • Finding current android device location
  • scheduleAtFixedRate (Timer)
  • getSharedPreferences (Context)
  • onRequestPermissionsResult (Fragment)
  • Window (java.awt)
    A Window object is a top-level window with no borders and no menubar. The default layout for a windo
  • ServerSocket (java.net)
    This class represents a server-side socket that waits for incoming client connections. A ServerSocke
  • SecureRandom (java.security)
    This class generates cryptographically secure pseudo-random numbers. It is best to invoke SecureRand
  • Random (java.util)
    This class provides methods that return pseudo-random values.It is dangerous to seed Random with the
  • Executors (java.util.concurrent)
    Factory and utility methods for Executor, ExecutorService, ScheduledExecutorService, ThreadFactory,
  • JarFile (java.util.jar)
    JarFile is used to read jar entries and their associated data from jar files.
  • Github Copilot alternatives
Tabnine Logo
  • Products

    Search for Java codeSearch for JavaScript code
  • IDE Plugins

    IntelliJ IDEAWebStormVisual StudioAndroid StudioEclipseVisual Studio CodePyCharmSublime TextPhpStormVimGoLandRubyMineEmacsJupyter NotebookJupyter LabRiderDataGripAppCode
  • Company

    About UsContact UsCareers
  • Resources

    FAQBlogTabnine AcademyTerms of usePrivacy policyJava Code IndexJavascript Code Index
Get Tabnine for your IDE now