Tabnine Logo
PartitionConsumerResult.getPartitionConsumerState
Code IndexAdd Tabnine to your IDE (free)

How to use
getPartitionConsumerState
method
in
co.cask.cdap.api.dataset.lib.PartitionConsumerResult

Best Java code snippets using co.cask.cdap.api.dataset.lib.PartitionConsumerResult.getPartitionConsumerState (Showing top 3 results out of 315)

origin: cdapio/cdap

 /**
  * @param limit limit to be applied while consuming partitions
  * @param predicate predicate to be applied while consuming partitions
  * @return a list of {@link Partition}s of the underlying {@link PartitionedFileSet} created since the last call
  *         to this method. This excludes partitions created in in-progress transactions including the one in which the
  *         call to this method is made.
  */
 public List<PartitionDetail> consumePartitions(int limit, Predicate<PartitionDetail> predicate) {
  PartitionConsumerResult partitionConsumerResult =
   partitionedFileSet.consumePartitions(partitionConsumerState, limit, predicate);
  partitionConsumerState = partitionConsumerResult.getPartitionConsumerState();
  return partitionConsumerResult.getPartitions();
 }
}
origin: co.cask.cdap/cdap-data-fabric

 /**
  * @param limit limit to be applied while consuming partitions
  * @param predicate predicate to be applied while consuming partitions
  * @return a list of {@link Partition}s of the underlying {@link PartitionedFileSet} created since the last call
  *         to this method. This excludes partitions created in in-progress transactions including the one in which the
  *         call to this method is made.
  */
 public List<PartitionDetail> consumePartitions(int limit, Predicate<PartitionDetail> predicate) {
  PartitionConsumerResult partitionConsumerResult =
   partitionedFileSet.consumePartitions(partitionConsumerState, limit, predicate);
  partitionConsumerState = partitionConsumerResult.getPartitionConsumerState();
  return partitionConsumerResult.getPartitions();
 }
}
origin: cdapio/cdap

/**
 * Populates the ConsumerWorkingSet by fetching partitions from the given PartitionedFileSet.
 *
 * @param partitionedFileSet the PartitionedFileSet to fetch partitions from
 * @param configuration the ConsumerConfiguration which defines parameters for consuming
 */
public void populate(PartitionedFileSet partitionedFileSet, ConsumerConfiguration configuration) {
 int numToPopulate = configuration.getMaxWorkingSetSize() - partitions.size();
 Predicate<PartitionDetail> predicate = configuration.getPartitionPredicate();
 co.cask.cdap.api.dataset.lib.PartitionConsumerResult result =
  partitionedFileSet.consumePartitions(partitionConsumerState, numToPopulate, predicate);
 List<PartitionDetail> partitions = result.getPartitions();
 for (PartitionDetail partition : partitions) {
  addPartition(partition.getPartitionKey());
 }
 partitionConsumerState = result.getPartitionConsumerState();
}
co.cask.cdap.api.dataset.libPartitionConsumerResultgetPartitionConsumerState

Popular methods of PartitionConsumerResult

  • getPartitions
  • <init>

Popular in Java

  • Running tasks concurrently on multiple threads
  • scheduleAtFixedRate (ScheduledExecutorService)
  • startActivity (Activity)
  • getSharedPreferences (Context)
  • EOFException (java.io)
    Thrown when a program encounters the end of a file or stream during an input operation.
  • FileInputStream (java.io)
    An input stream that reads bytes from a file. File file = ...finally if (in != null) in.clos
  • BigDecimal (java.math)
    An immutable arbitrary-precision signed decimal.A value is represented by an arbitrary-precision "un
  • Time (java.sql)
    Java representation of an SQL TIME value. Provides utilities to format and parse the time's represen
  • Modifier (javassist)
    The Modifier class provides static methods and constants to decode class and member access modifiers
  • Project (org.apache.tools.ant)
    Central representation of an Ant project. This class defines an Ant project with all of its targets,
  • From CI to AI: The AI layer in your organization
Tabnine Logo
  • Products

    Search for Java codeSearch for JavaScript code
  • IDE Plugins

    IntelliJ IDEAWebStormVisual StudioAndroid StudioEclipseVisual Studio CodePyCharmSublime TextPhpStormVimGoLandRubyMineEmacsJupyter NotebookJupyter LabRiderDataGripAppCode
  • Company

    About UsContact UsCareers
  • Resources

    FAQBlogTabnine AcademyTerms of usePrivacy policyJava Code IndexJavascript Code Index
Get Tabnine for your IDE now