Tabnine Logo
Mapper$Context.getOutputCommitter
Code IndexAdd Tabnine to your IDE (free)

How to use
getOutputCommitter
method
in
org.apache.hadoop.mapreduce.Mapper$Context

Best Java code snippets using org.apache.hadoop.mapreduce.Mapper$Context.getOutputCommitter (Showing top 7 results out of 315)

origin: apache/incubator-gobblin

GobblinOutputCommitter gobblinOutputCommitter = (GobblinOutputCommitter) context.getOutputCommitter();
gobblinOutputCommitter.getAttemptIdToMultiTaskAttempt()
  .put(context.getTaskAttemptID().toString(), gobblinMultiTaskAttempt);
origin: ShifuML/shifu

@SuppressWarnings("unchecked")
private Context createSubContextForHadoop2(Context context, Class<?> mapContextImplClazz)
    throws NoSuchMethodException, InstantiationException, IllegalAccessException,
    InvocationTargetException, ClassNotFoundException {
  Constructor<?> constructor = mapContextImplClazz.getDeclaredConstructor(Configuration.class,
      TaskAttemptID.class, RecordReader.class, RecordWriter.class, OutputCommitter.class,
      StatusReporter.class, InputSplit.class);
  constructor.setAccessible(true);
  Object mapContext = constructor.newInstance(outer.getConfiguration(), outer.getTaskAttemptID(), reader,
      new SubMapRecordWriter(), context.getOutputCommitter(), new SubMapStatusReporter(),
      outer.getInputSplit());
  Class<?> wrappedMapperClazz = Class.forName("org.apache.hadoop.mapreduce.lib.map.WrappedMapper");
  Object wrappedMapper = wrappedMapperClazz.newInstance();
  Method method = wrappedMapperClazz.getDeclaredMethod("getMapContext",
      Class.forName("org.apache.hadoop.mapreduce.MapContext"));
  return (Context) (method.invoke(wrappedMapper, mapContext));
}
origin: ShifuML/shifu

@SuppressWarnings("unchecked")
private Context createSubContextForHadoop2(Context context, Class<?> mapContextImplClazz)
    throws NoSuchMethodException, InstantiationException, IllegalAccessException,
    InvocationTargetException, ClassNotFoundException {
  Constructor<?> constructor = mapContextImplClazz.getDeclaredConstructor(Configuration.class,
      TaskAttemptID.class, RecordReader.class, RecordWriter.class, OutputCommitter.class,
      StatusReporter.class, InputSplit.class);
  constructor.setAccessible(true);
  Object mapContext = constructor.newInstance(outer.getConfiguration(), outer.getTaskAttemptID(), reader,
      new SubMapRecordWriter(), context.getOutputCommitter(), new SubMapStatusReporter(),
      outer.getInputSplit());
  Class<?> wrappedMapperClazz = Class.forName("org.apache.hadoop.mapreduce.lib.map.WrappedMapper");
  Object wrappedMapper = wrappedMapperClazz.newInstance();
  Method method = wrappedMapperClazz.getDeclaredMethod("getMapContext",
      Class.forName("org.apache.hadoop.mapreduce.MapContext"));
  return (Context) (method.invoke(wrappedMapper, mapContext));
}
origin: com.linkedin.gobblin/gobblin-runtime

GobblinOutputCommitter gobblinOutputCommitter = (GobblinOutputCommitter) context.getOutputCommitter();
gobblinOutputCommitter.getAttemptIdToMultiTaskAttempt()
  .put(context.getTaskAttemptID().toString(), gobblinMultiTaskAttempt);
origin: org.apache.gobblin/gobblin-runtime

GobblinOutputCommitter gobblinOutputCommitter = (GobblinOutputCommitter) context.getOutputCommitter();
gobblinOutputCommitter.getAttemptIdToMultiTaskAttempt()
  .put(context.getTaskAttemptID().toString(), gobblinMultiTaskAttempt);
origin: ShifuML/shifu

@SuppressWarnings("unchecked")
private Context createSubContextForHadoop1(Context context) throws NoSuchMethodException,
    InstantiationException, IllegalAccessException, InvocationTargetException {
  Constructor<?> constructor = Context.class.getDeclaredConstructor(Mapper.class, Configuration.class,
      TaskAttemptID.class, RecordReader.class, RecordWriter.class, OutputCommitter.class,
      StatusReporter.class, InputSplit.class);
  constructor.setAccessible(true);
  return (Context) constructor.newInstance(mapper, outer.getConfiguration(), outer.getTaskAttemptID(),
      reader, new SubMapRecordWriter(), context.getOutputCommitter(), new SubMapStatusReporter(),
      outer.getInputSplit());
}
origin: ShifuML/shifu

@SuppressWarnings("unchecked")
private Context createSubContextForHadoop1(Context context) throws NoSuchMethodException,
    InstantiationException, IllegalAccessException, InvocationTargetException {
  Constructor<?> constructor = Context.class.getDeclaredConstructor(Mapper.class, Configuration.class,
      TaskAttemptID.class, RecordReader.class, RecordWriter.class, OutputCommitter.class,
      StatusReporter.class, InputSplit.class);
  constructor.setAccessible(true);
  return (Context) constructor.newInstance(mapper, outer.getConfiguration(), outer.getTaskAttemptID(),
      reader, new SubMapRecordWriter(), context.getOutputCommitter(), new SubMapStatusReporter(),
      outer.getInputSplit());
}
org.apache.hadoop.mapreduceMapper$ContextgetOutputCommitter

Popular methods of Mapper$Context

  • write
  • getConfiguration
  • getCounter
  • getInputSplit
  • progress
  • setStatus
  • getTaskAttemptID
  • nextKeyValue
  • getCurrentValue
  • getCurrentKey
  • getNumReduceTasks
  • getJobID
  • getNumReduceTasks,
  • getJobID,
  • getInputFormatClass,
  • getLocalCacheFiles,
  • getCredentials,
  • getLocalCacheArchives,
  • getStatus,
  • getCacheArchives,
  • getCacheFiles

Popular in Java

  • Reactive rest calls using spring rest template
  • getResourceAsStream (ClassLoader)
  • getOriginalFilename (MultipartFile)
    Return the original filename in the client's filesystem.This may contain path information depending
  • onRequestPermissionsResult (Fragment)
  • VirtualMachine (com.sun.tools.attach)
    A Java virtual machine. A VirtualMachine represents a Java virtual machine to which this Java vir
  • FileWriter (java.io)
    A specialized Writer that writes to a file in the file system. All write requests made by calling me
  • UnknownHostException (java.net)
    Thrown when a hostname can not be resolved.
  • SimpleDateFormat (java.text)
    Formats and parses dates in a locale-sensitive manner. Formatting turns a Date into a String, and pa
  • JTextField (javax.swing)
  • Runner (org.openjdk.jmh.runner)
  • Top 17 Plugins for Android Studio
Tabnine Logo
  • Products

    Search for Java codeSearch for JavaScript code
  • IDE Plugins

    IntelliJ IDEAWebStormVisual StudioAndroid StudioEclipseVisual Studio CodePyCharmSublime TextPhpStormVimAtomGoLandRubyMineEmacsJupyter NotebookJupyter LabRiderDataGripAppCode
  • Company

    About UsContact UsCareers
  • Resources

    FAQBlogTabnine AcademyStudentsTerms of usePrivacy policyJava Code IndexJavascript Code Index
Get Tabnine for your IDE now