congrats Icon
New! Announcing Tabnine Chat Beta
Learn More
Tabnine Logo
HCatUtil.getInputJobInfosFromConf
Code IndexAdd Tabnine to your IDE (free)

How to use
getInputJobInfosFromConf
method
in
org.apache.hive.hcatalog.common.HCatUtil

Best Java code snippets using org.apache.hive.hcatalog.common.HCatUtil.getInputJobInfosFromConf (Showing top 3 results out of 315)

origin: apache/hive

public static InputJobInfo getLastInputJobInfosFromConf(Configuration conf)
    throws IOException {
 LinkedList<InputJobInfo> inputJobInfos = getInputJobInfosFromConf(conf);
 if (inputJobInfos == null || inputJobInfos.isEmpty()) {
  return null;
 } else {
  return getInputJobInfosFromConf(conf).getLast();
 }
}
origin: apache/hive

/**
 * Get statistics about the data to be loaded. Only input data size is implemented at this time.
 */
@Override
public ResourceStatistics getStatistics(String location, Job job) throws IOException {
 try {
  if (dbName == null || tableName == null) {
   throw new IOException("DB or table name unset. setLocation() must be invoked on this " +
       "loader to set them");
  }
  ResourceStatistics stats = new ResourceStatistics();
  long inputSize = -1;
  LinkedList<InputJobInfo> inputJobInfos = HCatUtil.getInputJobInfosFromConf(
      job.getConfiguration());
  for (InputJobInfo inputJobInfo : inputJobInfos) {
   if (dbName.equals(inputJobInfo.getDatabaseName()) && tableName.equals(inputJobInfo.getTableName())){
    inputSize = getSizeInBytes(inputJobInfo);
    break;
   }
  }
  if (inputSize == -1) {
   throw new IOException("Could not calculate input size for database: " + dbName + ", " +
       "table: " + tableName + ". Requested location:" + location);
  }
  stats.setSizeInBytes(inputSize);
  return stats;
 } catch (Exception e) {
  throw new IOException(e);
 }
}
origin: apache/hive

 @Test
 public void testInputJobInfoInConf() throws Exception {
  Configuration conf = new Configuration(false);

  InputJobInfo inputJobInfo = HCatUtil.getLastInputJobInfosFromConf(conf);
  Assert.assertNull(inputJobInfo);
  List<InputJobInfo> inputJobInfos = HCatUtil.getInputJobInfosFromConf(conf);
  Assert.assertNull(inputJobInfos);

  InputJobInfo inputJobInfo0 = InputJobInfo.create("db", "table", "", new Properties());
  InputJobInfo inputJobInfo1 = InputJobInfo.create("db", "table2", "", new Properties());

  HCatUtil.putInputJobInfoToConf(inputJobInfo0, conf);
  HCatUtil.putInputJobInfoToConf(inputJobInfo1, conf);

  inputJobInfo = HCatUtil.getLastInputJobInfosFromConf(conf);
  inputJobInfos = HCatUtil.getInputJobInfosFromConf(conf);

  Assert.assertEquals(inputJobInfo1.getDatabaseName(), inputJobInfo.getDatabaseName());
  Assert.assertEquals(inputJobInfo1.getTableName(), inputJobInfo.getTableName());
  Assert.assertEquals(inputJobInfo0.getDatabaseName(), inputJobInfos.get(0).getDatabaseName());
  Assert.assertEquals(inputJobInfo0.getTableName(), inputJobInfos.get(0).getTableName());

 }
}
org.apache.hive.hcatalog.commonHCatUtilgetInputJobInfosFromConf

Popular methods of HCatUtil

  • getHiveMetastoreClient
    Get or create a hive client depending on whether it exits in cache or not
  • getTable
  • closeHiveClientQuietly
  • deserialize
  • getHiveConf
  • extractSchema
  • getStorageHandler
    Create an instance of a storage handler defined in storerInfo. If one cannot be found then FosterSto
  • serialize
  • validateExecuteBitPresentIfReadOrWrite
    Ensure that read or write permissions are not granted without also granting execute permissions. Ess
  • validateMorePermissive
    Test if the first FsAction is more permissive than the second. This is useful in cases where we want
  • configureOutputStorageHandler
  • copyConf
    Replace the contents of dest with the contents of src
  • configureOutputStorageHandler,
  • copyConf,
  • copyJobPropertiesToJobConf,
  • decodeBytes,
  • encodeBytes,
  • extractThriftToken,
  • getFieldSchemaList,
  • getHCatFieldSchemaList,
  • getInputJobProperties,
  • getJobConfFromContext

Popular in Java

  • Finding current android device location
  • getApplicationContext (Context)
  • addToBackStack (FragmentTransaction)
  • onCreateOptionsMenu (Activity)
  • BorderLayout (java.awt)
    A border layout lays out a container, arranging and resizing its components to fit in five regions:
  • RandomAccessFile (java.io)
    Allows reading from and writing to a file in a random-access manner. This is different from the uni-
  • Format (java.text)
    The base class for all formats. This is an abstract base class which specifies the protocol for clas
  • Calendar (java.util)
    Calendar is an abstract base class for converting between a Date object and a set of integer fields
  • SortedMap (java.util)
    A map that has its keys ordered. The sorting is according to either the natural ordering of its keys
  • HttpServletRequest (javax.servlet.http)
    Extends the javax.servlet.ServletRequest interface to provide request information for HTTP servlets.
  • Top Sublime Text plugins
Tabnine Logo
  • Products

    Search for Java codeSearch for JavaScript code
  • IDE Plugins

    IntelliJ IDEAWebStormVisual StudioAndroid StudioEclipseVisual Studio CodePyCharmSublime TextPhpStormVimGoLandRubyMineEmacsJupyter NotebookJupyter LabRiderDataGripAppCode
  • Company

    About UsContact UsCareers
  • Resources

    FAQBlogTabnine AcademyTerms of usePrivacy policyJava Code IndexJavascript Code Index
Get Tabnine for your IDE now