Tabnine Logo
OutputCollector
Code IndexAdd Tabnine to your IDE (free)

How to use
OutputCollector
in
org.apache.nemo.common.ir

Best Java code snippets using org.apache.nemo.common.ir.OutputCollector (Showing top 20 results out of 315)

origin: org.apache.nemo/nemo-common

@Override
public void close() {
 outputCollector.emit(aggregatedDynOptData);
}
origin: apache/incubator-nemo

 /**
  * This just forwards watermarks to the next operator because it has one data stream.
  * @param edgeIndex edge index
  * @param watermark watermark
  */
 @Override
 public void trackAndEmitWatermarks(final int edgeIndex,
                   final Watermark watermark) {
  watermarkCollector.emitWatermark(watermark);
 }
}
origin: apache/incubator-nemo

@Override
public void onWatermark(final Watermark inputWatermark) {
 // If no data, just forwards the watermark
 if (windowListMap.size() == 0 && currentOutputWatermark < inputWatermark.getTimestamp()) {
  currentOutputWatermark = inputWatermark.getTimestamp();
  outputCollector.emitWatermark(inputWatermark);
  return;
 }
 final Iterator<Map.Entry<BoundedWindow, List<I>>> iterator = windowListMap.entrySet().iterator();
 long minOutputTimestampOfEmittedWindows = Long.MAX_VALUE;
 while (iterator.hasNext()) {
  final Map.Entry<BoundedWindow, List<I>> entry = iterator.next();
  if (entry.getKey().maxTimestamp().getMillis() <= inputWatermark.getTimestamp()) {
   // emit the windowed data if the watermark timestamp > the window max boundary
   final O output = viewFn.apply(new MultiView<>(entry.getValue()));
   outputCollector.emit(WindowedValue.of(
    output, entry.getKey().maxTimestamp(), entry.getKey(), PaneInfo.ON_TIME_AND_ONLY_FIRING));
   iterator.remove();
   minOutputTimestampOfEmittedWindows =
    Math.min(minOutputTimestampOfEmittedWindows, entry.getKey().maxTimestamp().getMillis());
  }
 }
 if (minOutputTimestampOfEmittedWindows != Long.MAX_VALUE
  && currentOutputWatermark < minOutputTimestampOfEmittedWindows) {
  // update current output watermark and emit to next operators
  currentOutputWatermark = minOutputTimestampOfEmittedWindows;
  outputCollector.emitWatermark(new Watermark(currentOutputWatermark));
 }
}
origin: org.apache.nemo/nemo-runtime-executor

/**
 * Process a data element down the DAG dependency.
 */
private void processElement(final OutputCollector outputCollector, final Object dataElement) {
 outputCollector.emit(dataElement);
}
origin: org.apache.nemo/nemo-runtime-executor

 /**
  * This just forwards watermarks to the next operator because it has one data stream.
  * @param edgeIndex edge index
  * @param watermark watermark
  */
 @Override
 public void trackAndEmitWatermarks(final int edgeIndex,
                   final Watermark watermark) {
  watermarkCollector.emitWatermark(watermark);
 }
}
origin: apache/incubator-nemo

@Override
public void onData(final T element) {
 outputCollector.emit(element);
}
origin: apache/incubator-nemo

@Override
public void onWatermark(final Watermark watermark) {
 outputCollector.emitWatermark(watermark);
}
origin: org.apache.nemo/nemo-common

@Override
public void onData(final T element) {
 outputCollector.emit(element);
}
origin: apache/incubator-nemo

@Override
public void onWatermark(final Watermark watermark) {
 outputCollector.emitWatermark(watermark);
}
origin: apache/incubator-nemo

@Override
public void close() {
 outputCollector.emit(aggregatedDynOptData);
}
origin: org.apache.nemo/nemo-runtime-executor

private void processWatermark(final OutputCollector outputCollector,
               final Watermark watermark) {
 outputCollector.emitWatermark(watermark);
}
origin: apache/incubator-nemo

/**
 * Process a data element down the DAG dependency.
 */
private void processElement(final OutputCollector outputCollector, final Object dataElement) {
 outputCollector.emit(dataElement);
}
origin: apache/incubator-nemo

@Override
public void onWatermark(final Watermark watermark) {
 outputCollector.emitWatermark(watermark);
}
origin: apache/incubator-nemo

@Override
public void onData(final T element) {
 outputCollector.emit(element);
}
origin: apache/incubator-nemo

@Override
public void onWatermark(final Watermark watermark) {
 outputCollector.emitWatermark(watermark);
}
origin: apache/incubator-nemo

 @Override
 public <T> void output(final TupleTag<T> tag, final WindowedValue<T> output) {
  if (tag.equals(mainOutputTag)) {
   outputCollector.emit((WindowedValue<OutputT>) output);
  } else {
   outputCollector.emit(tag.getId(), output);
  }
 }
}
origin: apache/incubator-nemo

private void processWatermark(final OutputCollector outputCollector,
               final Watermark watermark) {
 outputCollector.emitWatermark(watermark);
}
origin: apache/incubator-nemo

@Override
public void onData(final I element) {
  try {
   outputCollector.emit(func.call(element));
  } catch (Exception e) {
   throw new RuntimeException(e);
  }
}
origin: apache/incubator-nemo

@Override
public void onWatermark(final Watermark watermark) {
 outputCollector.emitWatermark(watermark);
}
origin: apache/incubator-nemo

@Override
public void onData(final T element) {
 try {
  Tuple2<K, V> data = func.call(element);
  outputCollector.emit(data);
 } catch (Exception e) {
  throw new RuntimeException(e);
 }
}
org.apache.nemo.common.irOutputCollector

Javadoc

Interface through which Transform emits outputs. This is to be implemented in the runtime with runtime-specific distributed data movement and storage mechanisms.

Most used methods

  • emit
    Multi-destination emit. Currently unused, but might come in handy for operations like multi-output m
  • emitWatermark
    Emit watermark to downstream vertices.

Popular in Java

  • Updating database using SQL prepared statement
  • orElseThrow (Optional)
    Return the contained value, if present, otherwise throw an exception to be created by the provided s
  • addToBackStack (FragmentTransaction)
  • getApplicationContext (Context)
  • ServerSocket (java.net)
    This class represents a server-side socket that waits for incoming client connections. A ServerSocke
  • Stack (java.util)
    Stack is a Last-In/First-Out(LIFO) data structure which represents a stack of objects. It enables u
  • Stream (java.util.stream)
    A sequence of elements supporting sequential and parallel aggregate operations. The following exampl
  • JCheckBox (javax.swing)
  • JPanel (javax.swing)
  • Join (org.hibernate.mapping)
  • Top plugins for Android Studio
Tabnine Logo
  • Products

    Search for Java codeSearch for JavaScript code
  • IDE Plugins

    IntelliJ IDEAWebStormVisual StudioAndroid StudioEclipseVisual Studio CodePyCharmSublime TextPhpStormVimGoLandRubyMineEmacsJupyter NotebookJupyter LabRiderDataGripAppCode
  • Company

    About UsContact UsCareers
  • Resources

    FAQBlogTabnine AcademyTerms of usePrivacy policyJava Code IndexJavascript Code Index
Get Tabnine for your IDE now