Tabnine Logo
ChannelLineageNode.addPredecessor
Code IndexAdd Tabnine to your IDE (free)

How to use
addPredecessor
method
in
org.qcri.rheem.core.platform.lineage.ChannelLineageNode

Best Java code snippets using org.qcri.rheem.core.platform.lineage.ChannelLineageNode.addPredecessor (Showing top 9 results out of 315)

origin: org.qcri.rheem/rheem-core

/**
 * Set all of the {@code inputs} as predecessors of the {@code operatorContext} each of the {@code outputs}.
 *
 * @param inputs               input {@link ChannelInstance}s
 * @param executionLineageNode in-between {@link ExecutionLineageNode}
 * @param outputs              output {@link ChannelInstance}s
 * @see #addPredecessor(LazyExecutionLineageNode)
 */
public static void connectAll(ChannelInstance[] inputs,
               ExecutionLineageNode executionLineageNode,
               ChannelInstance[] outputs) {
  for (ChannelInstance input : inputs) {
    if (input != null) executionLineageNode.addPredecessor(input.getLineage());
  }
  for (ChannelInstance output : outputs) {
    if (output != null) output.getLineage().addPredecessor(executionLineageNode);
  }
}
origin: org.qcri.rheem/rheem-java

/**
 * Utility method to forward a {@link JavaChannelInstance} to another.
 *
 * @param input  that should be forwarded
 * @param output to that should be forwarded
 */
static void forward(ChannelInstance input, ChannelInstance output) {
  // Do the forward.
  if (output instanceof CollectionChannel.Instance) {
    ((CollectionChannel.Instance) output).accept(((CollectionChannel.Instance) input).provideCollection());
  } else if (output instanceof StreamChannel.Instance) {
    ((StreamChannel.Instance) output).accept(((JavaChannelInstance) input).provideStream());
  } else {
    throw new RheemException(String.format("Cannot forward %s to %s.", input, output));
  }
  // Manipulate the lineage.
  output.getLineage().addPredecessor(input.getLineage());
}
origin: org.qcri.rheem/rheem-java

@Override
public Tuple<Collection<ExecutionLineageNode>, Collection<ChannelInstance>> evaluate(
    ChannelInstance[] inputs,
    ChannelInstance[] outputs,
    JavaExecutor javaExecutor,
    OptimizationContext.OperatorContext operatorContext) {
  assert inputs.length == this.getNumInputs();
  assert outputs.length == this.getNumOutputs();
  String url = this.getInputUrl().trim();
  FileSystem fs = FileSystems.getFileSystem(url).orElseThrow(
      () -> new RheemException(String.format("Cannot access file system of %s.", url))
  );
  try {
    final InputStream inputStream = fs.open(url);
    Stream<String> lines = new BufferedReader(new InputStreamReader(inputStream)).lines();
    ((StreamChannel.Instance) outputs[0]).accept(lines);
  } catch (IOException e) {
    throw new RheemException(String.format("Reading %s failed.", url), e);
  }
  ExecutionLineageNode prepareLineageNode = new ExecutionLineageNode(operatorContext);
  prepareLineageNode.add(LoadProfileEstimators.createFromSpecification(
      "rheem.java.textfilesource.load.prepare", javaExecutor.getConfiguration()
  ));
  ExecutionLineageNode mainLineageNode = new ExecutionLineageNode(operatorContext);
  mainLineageNode.add(LoadProfileEstimators.createFromSpecification(
      "rheem.java.textfilesource.load.main", javaExecutor.getConfiguration()
  ));
  outputs[0].getLineage().addPredecessor(mainLineageNode);
  return prepareLineageNode.collectAndMark();
}
origin: org.qcri.rheem/rheem-java

outputs[0].getLineage().addPredecessor(probingExecutionLineageNode);
origin: org.qcri.rheem/rheem-spark

/**
 * Utility method to forward a {@link RddChannel.Instance} to another.
 *
 * @param input  that should be forwarded
 * @param output to that should be forwarded
 */
public void forward(ChannelInstance input, ChannelInstance output) {
  final RddChannel.Instance rddInput = (RddChannel.Instance) input;
  final RddChannel.Instance rddOutput = (RddChannel.Instance) output;
  // Do the forward.
  assert rddInput.getChannel().getDescriptor() == RddChannel.CACHED_DESCRIPTOR ||
      rddOutput.getChannel().getDescriptor() != RddChannel.CACHED_DESCRIPTOR;
  rddOutput.accept(rddInput.provideRdd(), this);
  // Manipulate the lineage.
  output.getLineage().addPredecessor(input.getLineage());
}
origin: org.qcri.rheem/rheem-java

output.getLineage().addPredecessor(probingExecutionLineageNode);
return new Tuple<>(executionLineageNodes, producedChannelInstances);
origin: org.qcri.rheem/rheem-graphchi

    "rheem.graphchi.pagerank.load.output", configuration
));
outputChannelInstance.getLineage().addPredecessor(outputExecutionLineage);
origin: org.qcri.rheem/rheem-spark

@Override
public Tuple<Collection<ExecutionLineageNode>, Collection<ChannelInstance>> evaluate(
    ChannelInstance[] inputs,
    ChannelInstance[] outputs,
    SparkExecutor sparkExecutor,
    OptimizationContext.OperatorContext operatorContext) {
  assert inputs.length == this.getNumInputs();
  assert outputs.length == this.getNumOutputs();
  RddChannel.Instance output = (RddChannel.Instance) outputs[0];
  final JavaRDD<String> rdd = sparkExecutor.sc.textFile(this.getInputUrl());
  this.name(rdd);
  output.accept(rdd, sparkExecutor);
  ExecutionLineageNode prepareLineageNode = new ExecutionLineageNode(operatorContext);
  prepareLineageNode.add(LoadProfileEstimators.createFromSpecification(
      "rheem.spark.textfilesource.load.prepare", sparkExecutor.getConfiguration()
  ));
  ExecutionLineageNode mainLineageNode = new ExecutionLineageNode(operatorContext);
  mainLineageNode.add(LoadProfileEstimators.createFromSpecification(
      "rheem.spark.textfilesource.load.main", sparkExecutor.getConfiguration()
  ));
  output.getLineage().addPredecessor(mainLineageNode);
  return prepareLineageNode.collectAndMark();
}
origin: org.qcri.rheem/rheem-java

outputs[0].getLineage().addPredecessor(probingExecutionLineageNode);
org.qcri.rheem.core.platform.lineageChannelLineageNodeaddPredecessor

Popular methods of ChannelLineageNode

  • <init>
  • collectAndMark
  • getChannelInstance
    Retrieve the encapsulated ChannelInstance.

Popular in Java

  • Reading from database using SQL prepared statement
  • getOriginalFilename (MultipartFile)
    Return the original filename in the client's filesystem.This may contain path information depending
  • getSharedPreferences (Context)
  • getResourceAsStream (ClassLoader)
  • Pointer (com.sun.jna)
    An abstraction for a native pointer data type. A Pointer instance represents, on the Java side, a na
  • Window (java.awt)
    A Window object is a top-level window with no borders and no menubar. The default layout for a windo
  • BufferedWriter (java.io)
    Wraps an existing Writer and buffers the output. Expensive interaction with the underlying reader is
  • DecimalFormat (java.text)
    A concrete subclass of NumberFormat that formats decimal numbers. It has a variety of features desig
  • BasicDataSource (org.apache.commons.dbcp)
    Basic implementation of javax.sql.DataSource that is configured via JavaBeans properties. This is no
  • Get (org.apache.hadoop.hbase.client)
    Used to perform Get operations on a single row. To get everything for a row, instantiate a Get objec
  • Top 12 Jupyter Notebook extensions
Tabnine Logo
  • Products

    Search for Java codeSearch for JavaScript code
  • IDE Plugins

    IntelliJ IDEAWebStormVisual StudioAndroid StudioEclipseVisual Studio CodePyCharmSublime TextPhpStormVimGoLandRubyMineEmacsJupyter NotebookJupyter LabRiderDataGripAppCode
  • Company

    About UsContact UsCareers
  • Resources

    FAQBlogTabnine AcademyTerms of usePrivacy policyJava Code IndexJavascript Code Index
Get Tabnine for your IDE now