Tabnine Logo
HdfsSpout.setArchiveDir
Code IndexAdd Tabnine to your IDE (free)

How to use
setArchiveDir
method
in
org.apache.storm.hdfs.spout.HdfsSpout

Best Java code snippets using org.apache.storm.hdfs.spout.HdfsSpout.setArchiveDir (Showing top 3 results out of 315)

origin: apache/storm

.setHdfsUri(hdfsUri)
.setSourceDir(sourceDir)
.setArchiveDir(archiveDir)
.setBadFilesDir(badDir);
origin: apache/storm

private AutoCloseableHdfsSpout makeSpout(String readerType, String[] outputFields) {
  HdfsSpout spout = new HdfsSpout().withOutputFields(outputFields)
                   .setReaderType(readerType)
                   .setHdfsUri(DFS_CLUSTER_RULE.getDfscluster().getURI().toString())
                   .setSourceDir(source.toString())
                   .setArchiveDir(archive.toString())
                   .setBadFilesDir(badfiles.toString());
  return new AutoCloseableHdfsSpout(spout);
}
origin: apache/storm

static StormTopology getTopology(Map<String, Object> config) {
  final int spoutNum = Helper.getInt(config, SPOUT_NUM, DEFAULT_SPOUT_NUM);
  final int boltNum = Helper.getInt(config, BOLT_NUM, DEFAULT_BOLT_NUM);
  final String fileFormat = Helper.getStr(config, "text");
  final String hdfsUri = Helper.getStr(config, HDFS_URI);
  final String sourceDir = Helper.getStr(config, SOURCE_DIR);
  final String archiveDir = Helper.getStr(config, ARCHIVE_DIR);
  final String badDir = Helper.getStr(config, BAD_DIR);
  // 1 -  Setup Hdfs Spout   --------
  HdfsSpout spout = new HdfsSpout()
    .setReaderType(fileFormat)
    .setHdfsUri(hdfsUri)
    .setSourceDir(sourceDir)
    .setArchiveDir(archiveDir)
    .setBadFilesDir(badDir)
    .withOutputFields(TextFileReader.defaultFields);
  // 2 -   DevNull Bolt   --------
  DevNullBolt bolt = new DevNullBolt();
  // 3 - Setup Topology  --------
  TopologyBuilder builder = new TopologyBuilder();
  builder.setSpout(SPOUT_ID, spout, spoutNum);
  builder.setBolt(BOLT_ID, bolt, boltNum)
      .localOrShuffleGrouping(SPOUT_ID);
  return builder.createTopology();
}
org.apache.storm.hdfs.spoutHdfsSpoutsetArchiveDir

Popular methods of HdfsSpout

  • <init>
  • setBadFilesDir
  • setHdfsUri
  • setReaderType
  • setSourceDir
  • withOutputFields
    Output field names. Number of fields depends upon the reader type
  • ack
  • canCommitNow
  • checkValidReader
  • close
  • closeReaderAndResetTrackers
  • commitProgress
  • closeReaderAndResetTrackers,
  • commitProgress,
  • createFileReader,
  • emitData,
  • fail,
  • getCollector,
  • getDefaultLockDir,
  • getFileForLockFile,
  • getFileProgress

Popular in Java

  • Updating database using SQL prepared statement
  • getSharedPreferences (Context)
  • getSupportFragmentManager (FragmentActivity)
  • getExternalFilesDir (Context)
  • Container (java.awt)
    A generic Abstract Window Toolkit(AWT) container object is a component that can contain other AWT co
  • Graphics2D (java.awt)
    This Graphics2D class extends the Graphics class to provide more sophisticated control overgraphics
  • BufferedWriter (java.io)
    Wraps an existing Writer and buffers the output. Expensive interaction with the underlying reader is
  • PrintStream (java.io)
    Fake signature of an existing Java class.
  • URL (java.net)
    A Uniform Resource Locator that identifies the location of an Internet resource as specified by RFC
  • Handler (java.util.logging)
    A Handler object accepts a logging request and exports the desired messages to a target, for example
  • Top PhpStorm plugins
Tabnine Logo
  • Products

    Search for Java codeSearch for JavaScript code
  • IDE Plugins

    IntelliJ IDEAWebStormVisual StudioAndroid StudioEclipseVisual Studio CodePyCharmSublime TextPhpStormVimGoLandRubyMineEmacsJupyter NotebookJupyter LabRiderDataGripAppCode
  • Company

    About UsContact UsCareers
  • Resources

    FAQBlogTabnine AcademyTerms of usePrivacy policyJava Code IndexJavascript Code Index
Get Tabnine for your IDE now