Tabnine Logo
Mapper$Context.getOutputCommitter
Code IndexAdd Tabnine to your IDE (free)

How to use
getOutputCommitter
method
in
org.apache.hadoop.mapreduce.Mapper$Context

Best Java code snippets using org.apache.hadoop.mapreduce.Mapper$Context.getOutputCommitter (Showing top 7 results out of 315)

origin: apache/incubator-gobblin

GobblinOutputCommitter gobblinOutputCommitter = (GobblinOutputCommitter) context.getOutputCommitter();
gobblinOutputCommitter.getAttemptIdToMultiTaskAttempt()
  .put(context.getTaskAttemptID().toString(), gobblinMultiTaskAttempt);
origin: ShifuML/shifu

@SuppressWarnings("unchecked")
private Context createSubContextForHadoop2(Context context, Class<?> mapContextImplClazz)
    throws NoSuchMethodException, InstantiationException, IllegalAccessException,
    InvocationTargetException, ClassNotFoundException {
  Constructor<?> constructor = mapContextImplClazz.getDeclaredConstructor(Configuration.class,
      TaskAttemptID.class, RecordReader.class, RecordWriter.class, OutputCommitter.class,
      StatusReporter.class, InputSplit.class);
  constructor.setAccessible(true);
  Object mapContext = constructor.newInstance(outer.getConfiguration(), outer.getTaskAttemptID(), reader,
      new SubMapRecordWriter(), context.getOutputCommitter(), new SubMapStatusReporter(),
      outer.getInputSplit());
  Class<?> wrappedMapperClazz = Class.forName("org.apache.hadoop.mapreduce.lib.map.WrappedMapper");
  Object wrappedMapper = wrappedMapperClazz.newInstance();
  Method method = wrappedMapperClazz.getDeclaredMethod("getMapContext",
      Class.forName("org.apache.hadoop.mapreduce.MapContext"));
  return (Context) (method.invoke(wrappedMapper, mapContext));
}
origin: ShifuML/shifu

@SuppressWarnings("unchecked")
private Context createSubContextForHadoop2(Context context, Class<?> mapContextImplClazz)
    throws NoSuchMethodException, InstantiationException, IllegalAccessException,
    InvocationTargetException, ClassNotFoundException {
  Constructor<?> constructor = mapContextImplClazz.getDeclaredConstructor(Configuration.class,
      TaskAttemptID.class, RecordReader.class, RecordWriter.class, OutputCommitter.class,
      StatusReporter.class, InputSplit.class);
  constructor.setAccessible(true);
  Object mapContext = constructor.newInstance(outer.getConfiguration(), outer.getTaskAttemptID(), reader,
      new SubMapRecordWriter(), context.getOutputCommitter(), new SubMapStatusReporter(),
      outer.getInputSplit());
  Class<?> wrappedMapperClazz = Class.forName("org.apache.hadoop.mapreduce.lib.map.WrappedMapper");
  Object wrappedMapper = wrappedMapperClazz.newInstance();
  Method method = wrappedMapperClazz.getDeclaredMethod("getMapContext",
      Class.forName("org.apache.hadoop.mapreduce.MapContext"));
  return (Context) (method.invoke(wrappedMapper, mapContext));
}
origin: com.linkedin.gobblin/gobblin-runtime

GobblinOutputCommitter gobblinOutputCommitter = (GobblinOutputCommitter) context.getOutputCommitter();
gobblinOutputCommitter.getAttemptIdToMultiTaskAttempt()
  .put(context.getTaskAttemptID().toString(), gobblinMultiTaskAttempt);
origin: org.apache.gobblin/gobblin-runtime

GobblinOutputCommitter gobblinOutputCommitter = (GobblinOutputCommitter) context.getOutputCommitter();
gobblinOutputCommitter.getAttemptIdToMultiTaskAttempt()
  .put(context.getTaskAttemptID().toString(), gobblinMultiTaskAttempt);
origin: ShifuML/shifu

@SuppressWarnings("unchecked")
private Context createSubContextForHadoop1(Context context) throws NoSuchMethodException,
    InstantiationException, IllegalAccessException, InvocationTargetException {
  Constructor<?> constructor = Context.class.getDeclaredConstructor(Mapper.class, Configuration.class,
      TaskAttemptID.class, RecordReader.class, RecordWriter.class, OutputCommitter.class,
      StatusReporter.class, InputSplit.class);
  constructor.setAccessible(true);
  return (Context) constructor.newInstance(mapper, outer.getConfiguration(), outer.getTaskAttemptID(),
      reader, new SubMapRecordWriter(), context.getOutputCommitter(), new SubMapStatusReporter(),
      outer.getInputSplit());
}
origin: ShifuML/shifu

@SuppressWarnings("unchecked")
private Context createSubContextForHadoop1(Context context) throws NoSuchMethodException,
    InstantiationException, IllegalAccessException, InvocationTargetException {
  Constructor<?> constructor = Context.class.getDeclaredConstructor(Mapper.class, Configuration.class,
      TaskAttemptID.class, RecordReader.class, RecordWriter.class, OutputCommitter.class,
      StatusReporter.class, InputSplit.class);
  constructor.setAccessible(true);
  return (Context) constructor.newInstance(mapper, outer.getConfiguration(), outer.getTaskAttemptID(),
      reader, new SubMapRecordWriter(), context.getOutputCommitter(), new SubMapStatusReporter(),
      outer.getInputSplit());
}
org.apache.hadoop.mapreduceMapper$ContextgetOutputCommitter

Popular methods of Mapper$Context

  • write
  • getConfiguration
  • getCounter
  • getInputSplit
  • progress
  • setStatus
  • getTaskAttemptID
  • nextKeyValue
  • getCurrentValue
  • getCurrentKey
  • getNumReduceTasks
  • getJobID
  • getNumReduceTasks,
  • getJobID,
  • getInputFormatClass,
  • getLocalCacheFiles,
  • getCredentials,
  • getLocalCacheArchives,
  • getStatus,
  • getCacheArchives,
  • getCacheFiles

Popular in Java

  • Creating JSON documents from java classes using gson
  • getContentResolver (Context)
  • getSupportFragmentManager (FragmentActivity)
  • scheduleAtFixedRate (ScheduledExecutorService)
  • FileInputStream (java.io)
    An input stream that reads bytes from a file. File file = ...finally if (in != null) in.clos
  • MalformedURLException (java.net)
    This exception is thrown when a program attempts to create an URL from an incorrect specification.
  • HashSet (java.util)
    HashSet is an implementation of a Set. All optional operations (adding and removing) are supported.
  • TreeMap (java.util)
    Walk the nodes of the tree left-to-right or right-to-left. Note that in descending iterations, next
  • Handler (java.util.logging)
    A Handler object accepts a logging request and exports the desired messages to a target, for example
  • BasicDataSource (org.apache.commons.dbcp)
    Basic implementation of javax.sql.DataSource that is configured via JavaBeans properties. This is no
  • Best IntelliJ plugins
Tabnine Logo
  • Products

    Search for Java codeSearch for JavaScript code
  • IDE Plugins

    IntelliJ IDEAWebStormVisual StudioAndroid StudioEclipseVisual Studio CodePyCharmSublime TextPhpStormVimGoLandRubyMineEmacsJupyter NotebookJupyter LabRiderDataGripAppCode
  • Company

    About UsContact UsCareers
  • Resources

    FAQBlogTabnine AcademyTerms of usePrivacy policyJava Code IndexJavascript Code Index
Get Tabnine for your IDE now