congrats Icon
New! Tabnine Pro 14-day free trial
Start a free trial
Tabnine Logo
PartitionedFileSet.getPartitioning
Code IndexAdd Tabnine to your IDE (free)

How to use
getPartitioning
method
in
co.cask.cdap.api.dataset.lib.PartitionedFileSet

Best Java code snippets using co.cask.cdap.api.dataset.lib.PartitionedFileSet.getPartitioning (Showing top 6 results out of 315)

origin: cdapio/cdap

DynamicPartitionerWriterWrapper(TaskAttemptContext job) {
 this.job = job;
 Configuration configuration = job.getConfiguration();
 Class<? extends DynamicPartitioner> partitionerClass = configuration
  .getClass(PartitionedFileSetArguments.DYNAMIC_PARTITIONER_CLASS_NAME, null, DynamicPartitioner.class);
 this.dynamicPartitioner = new InstantiatorFactory(false).get(TypeToken.of(partitionerClass)).create();
 this.partitionWriteOption =
  DynamicPartitioner.PartitionWriteOption.valueOf(
   configuration.get(PartitionedFileSetArguments.DYNAMIC_PARTITIONER_WRITE_OPTION));
 MapReduceClassLoader classLoader = MapReduceClassLoader.getFromConfiguration(configuration);
 this.taskContext = classLoader.getTaskContextProvider().get(job);
 // name the output file 'part-<RunId>-m-00000' instead of 'part-m-00000'
 String outputName = DynamicPartitioningOutputFormat.getOutputName(job);
 if (partitionWriteOption == DynamicPartitioner.PartitionWriteOption.CREATE_OR_APPEND) {
  outputName = outputName + "-" + taskContext.getProgramRunId().getRun();
 }
 this.outputName = outputName;
 String outputDatasetName = configuration.get(Constants.Dataset.Partitioned.HCONF_ATTR_OUTPUT_DATASET);
 this.outputDataset = taskContext.getDataset(outputDatasetName);
 this.partitioning = outputDataset.getPartitioning();
 this.dynamicPartitioner.initialize(taskContext);
 this.fileOutputFormatName = job.getConfiguration()
  .getClass(Constants.Dataset.Partitioned.HCONF_ATTR_OUTPUT_FORMAT_CLASS_NAME, null, FileOutputFormat.class)
  .getName();
}
origin: co.cask.cdap/cdap-app-fabric

DynamicPartitionerWriterWrapper(TaskAttemptContext job) {
 this.job = job;
 Configuration configuration = job.getConfiguration();
 Class<? extends DynamicPartitioner> partitionerClass = configuration
  .getClass(PartitionedFileSetArguments.DYNAMIC_PARTITIONER_CLASS_NAME, null, DynamicPartitioner.class);
 this.dynamicPartitioner = new InstantiatorFactory(false).get(TypeToken.of(partitionerClass)).create();
 this.partitionWriteOption =
  DynamicPartitioner.PartitionWriteOption.valueOf(
   configuration.get(PartitionedFileSetArguments.DYNAMIC_PARTITIONER_WRITE_OPTION));
 MapReduceClassLoader classLoader = MapReduceClassLoader.getFromConfiguration(configuration);
 this.taskContext = classLoader.getTaskContextProvider().get(job);
 // name the output file 'part-<RunId>-m-00000' instead of 'part-m-00000'
 String outputName = DynamicPartitioningOutputFormat.getOutputName(job);
 if (partitionWriteOption == DynamicPartitioner.PartitionWriteOption.CREATE_OR_APPEND) {
  outputName = outputName + "-" + taskContext.getProgramRunId().getRun();
 }
 this.outputName = outputName;
 String outputDatasetName = configuration.get(Constants.Dataset.Partitioned.HCONF_ATTR_OUTPUT_DATASET);
 this.outputDataset = taskContext.getDataset(outputDatasetName);
 this.partitioning = outputDataset.getPartitioning();
 this.dynamicPartitioner.initialize(taskContext);
 this.fileOutputFormatName = job.getConfiguration()
  .getClass(Constants.Dataset.Partitioned.HCONF_ATTR_OUTPUT_FORMAT_CLASS_NAME, null, FileOutputFormat.class)
  .getName();
}
origin: cdapio/cdap

 return;
Partitioning partitioning = ((PartitionedFileSet) dataset).getPartitioning();
origin: co.cask.cdap/cdap-explore

 return;
Partitioning partitioning = ((PartitionedFileSet) dataset).getPartitioning();
origin: cdapio/cdap

Partitioning partitioning = null;
if (dataset instanceof PartitionedFileSet) {
 partitioning = ((PartitionedFileSet) dataset).getPartitioning();
 baseLocation = ((PartitionedFileSet) dataset).getEmbeddedFileSet().getBaseLocation();
} else {
origin: co.cask.cdap/cdap-explore

Partitioning partitioning = null;
if (dataset instanceof PartitionedFileSet) {
 partitioning = ((PartitionedFileSet) dataset).getPartitioning();
 baseLocation = ((PartitionedFileSet) dataset).getEmbeddedFileSet().getBaseLocation();
} else {
co.cask.cdap.api.dataset.libPartitionedFileSetgetPartitioning

Javadoc

Get the partitioning declared for the file set.

Popular methods of PartitionedFileSet

  • getPartition
  • getEmbeddedFileSet
  • dropPartition
  • getPartitionOutput
    Return a partition output for a specific partition key, in preparation for creating a new partition.
  • getPartitions
  • addPartition
    Add a partition for a given partition key, stored at a given path (relative to the file set's base p
  • consumePartitions
  • addMetadata
    Adds a set of new metadata entries for a particular partition. Note that existing entries cannot be
  • concatenatePartition
    Asynchronous operation to concatenate the partition in Hive. Note that Hive only supports certain fo
  • removeMetadata
    Removes a set of metadata entries for a particular partition. If any metadata key does not exist, no
  • setMetadata
    Sets metadata entries for a particular partition. If the metadata entry key does not already exist,
  • setMetadata

Popular in Java

  • Start an intent from android
  • setContentView (Activity)
  • orElseThrow (Optional)
    Return the contained value, if present, otherwise throw an exception to be created by the provided s
  • requestLocationUpdates (LocationManager)
  • ResourceBundle (java.util)
    ResourceBundle is an abstract class which is the superclass of classes which provide Locale-specifi
  • SortedMap (java.util)
    A map that has its keys ordered. The sorting is according to either the natural ordering of its keys
  • Modifier (javassist)
    The Modifier class provides static methods and constants to decode class and member access modifiers
  • HttpServletRequest (javax.servlet.http)
    Extends the javax.servlet.ServletRequest interface to provide request information for HTTP servlets.
  • LogFactory (org.apache.commons.logging)
    Factory for creating Log instances, with discovery and configuration features similar to that employ
  • Project (org.apache.tools.ant)
    Central representation of an Ant project. This class defines an Ant project with all of its targets,
  • Top 17 Plugins for Android Studio
Tabnine Logo
  • Products

    Search for Java codeSearch for JavaScript code
  • IDE Plugins

    IntelliJ IDEAWebStormVisual StudioAndroid StudioEclipseVisual Studio CodePyCharmSublime TextPhpStormVimAtomGoLandRubyMineEmacsJupyter NotebookJupyter LabRiderDataGripAppCode
  • Company

    About UsContact UsCareers
  • Resources

    FAQBlogTabnine AcademyStudentsTerms of usePrivacy policyJava Code IndexJavascript Code Index
Get Tabnine for your IDE now