How to use
getPartitioning
method
in
co.cask.cdap.api.dataset.lib.PartitionedFileSet

Best Java code snippets using co.cask.cdap.api.dataset.lib.PartitionedFileSet.getPartitioning (Showing top 6 results out of 315)

Add the Codota plugin to your IDE and get smart completions

private void myMethod () {
 ArrayList a =new ArrayList<String>()
new ArrayList()
new ArrayList<Object>()
Smart code suggestions by Tabnine
}

DynamicPartitionerWriterWrapper(TaskAttemptContext job) {
 this.job = job;
 Configuration configuration = job.getConfiguration();
 Class<? extends DynamicPartitioner> partitionerClass = configuration
  .getClass(PartitionedFileSetArguments.DYNAMIC_PARTITIONER_CLASS_NAME, null, DynamicPartitioner.class);
 this.dynamicPartitioner = new InstantiatorFactory(false).get(TypeToken.of(partitionerClass)).create();
 this.partitionWriteOption =
  DynamicPartitioner.PartitionWriteOption.valueOf(
   configuration.get(PartitionedFileSetArguments.DYNAMIC_PARTITIONER_WRITE_OPTION));
 MapReduceClassLoader classLoader = MapReduceClassLoader.getFromConfiguration(configuration);
 this.taskContext = classLoader.getTaskContextProvider().get(job);
 // name the output file 'part-<RunId>-m-00000' instead of 'part-m-00000'
 String outputName = DynamicPartitioningOutputFormat.getOutputName(job);
 if (partitionWriteOption == DynamicPartitioner.PartitionWriteOption.CREATE_OR_APPEND) {
  outputName = outputName + "-" + taskContext.getProgramRunId().getRun();
 }
 this.outputName = outputName;
 String outputDatasetName = configuration.get(Constants.Dataset.Partitioned.HCONF_ATTR_OUTPUT_DATASET);
 this.outputDataset = taskContext.getDataset(outputDatasetName);
 this.partitioning = outputDataset.getPartitioning();
 this.dynamicPartitioner.initialize(taskContext);
 this.fileOutputFormatName = job.getConfiguration()
  .getClass(Constants.Dataset.Partitioned.HCONF_ATTR_OUTPUT_FORMAT_CLASS_NAME, null, FileOutputFormat.class)
  .getName();
}

DynamicPartitionerWriterWrapper(TaskAttemptContext job) {
 this.job = job;
 Configuration configuration = job.getConfiguration();
 Class<? extends DynamicPartitioner> partitionerClass = configuration
  .getClass(PartitionedFileSetArguments.DYNAMIC_PARTITIONER_CLASS_NAME, null, DynamicPartitioner.class);
 this.dynamicPartitioner = new InstantiatorFactory(false).get(TypeToken.of(partitionerClass)).create();
 this.partitionWriteOption =
  DynamicPartitioner.PartitionWriteOption.valueOf(
   configuration.get(PartitionedFileSetArguments.DYNAMIC_PARTITIONER_WRITE_OPTION));
 MapReduceClassLoader classLoader = MapReduceClassLoader.getFromConfiguration(configuration);
 this.taskContext = classLoader.getTaskContextProvider().get(job);
 // name the output file 'part-<RunId>-m-00000' instead of 'part-m-00000'
 String outputName = DynamicPartitioningOutputFormat.getOutputName(job);
 if (partitionWriteOption == DynamicPartitioner.PartitionWriteOption.CREATE_OR_APPEND) {
  outputName = outputName + "-" + taskContext.getProgramRunId().getRun();
 }
 this.outputName = outputName;
 String outputDatasetName = configuration.get(Constants.Dataset.Partitioned.HCONF_ATTR_OUTPUT_DATASET);
 this.outputDataset = taskContext.getDataset(outputDatasetName);
 this.partitioning = outputDataset.getPartitioning();
 this.dynamicPartitioner.initialize(taskContext);
 this.fileOutputFormatName = job.getConfiguration()
  .getClass(Constants.Dataset.Partitioned.HCONF_ATTR_OUTPUT_FORMAT_CLASS_NAME, null, FileOutputFormat.class)
  .getName();
}

 return;
Partitioning partitioning = ((PartitionedFileSet) dataset).getPartitioning();

 return;
Partitioning partitioning = ((PartitionedFileSet) dataset).getPartitioning();

Partitioning partitioning = null;
if (dataset instanceof PartitionedFileSet) {
 partitioning = ((PartitionedFileSet) dataset).getPartitioning();
 baseLocation = ((PartitionedFileSet) dataset).getEmbeddedFileSet().getBaseLocation();
} else {

Partitioning partitioning = null;
if (dataset instanceof PartitionedFileSet) {
 partitioning = ((PartitionedFileSet) dataset).getPartitioning();
 baseLocation = ((PartitionedFileSet) dataset).getEmbeddedFileSet().getBaseLocation();
} else {

Javadoc

Get the partitioning declared for the file set.

Popular methods of PartitionedFileSet

getPartition
getEmbeddedFileSet
dropPartition
getPartitionOutput
Return a partition output for a specific partition key, in preparation for creating a new partition.
getPartitions
addPartition
Add a partition for a given partition key, stored at a given path (relative to the file set's base p
consumePartitions
addMetadata
Adds a set of new metadata entries for a particular partition. Note that existing entries cannot be
concatenatePartition
Asynchronous operation to concatenate the partition in Hive. Note that Hive only supports certain fo
removeMetadata
Removes a set of metadata entries for a particular partition. If any metadata key does not exist, no
setMetadata
Sets metadata entries for a particular partition. If the metadata entry key does not already exist,

setMetadata

Popular in Java

Running tasks concurrently on multiple threads
getExternalFilesDir (Context)
getSupportFragmentManager (FragmentActivity)
getSharedPreferences (Context)
FileNotFoundException (java.io)
Thrown when a file specified by a program cannot be found.
ConnectException (java.net)
A ConnectException is thrown if a connection cannot be established to a remote host on a specific po
List (java.util)
An ordered collection (also known as a sequence). The user of this interface has precise control ove
Vector (java.util)
Vector is an implementation of List, backed by an array and synchronized. All optional operations in
JCheckBox (javax.swing)
Join (org.hibernate.mapping)

How to use getPartitioningmethodin co.cask.cdap.api.dataset.lib.PartitionedFileSet

Best Java code snippets using co.cask.cdap.api.dataset.lib.PartitionedFileSet.getPartitioning (Showing top 6 results out of 315)

How to use
getPartitioning
method
in
co.cask.cdap.api.dataset.lib.PartitionedFileSet