Tabnine Logo
AvroIO.parseAllGenericRecords
Code IndexAdd Tabnine to your IDE (free)

How to use
parseAllGenericRecords
method
in
org.apache.beam.sdk.io.AvroIO

Best Java code snippets using org.apache.beam.sdk.io.AvroIO.parseAllGenericRecords (Showing top 4 results out of 315)

origin: org.apache.beam/beam-sdks-java-core

@Override
public PCollection<T> expand(PBegin input) {
 checkNotNull(getFilepattern(), "filepattern");
 Coder<T> coder = inferCoder(getCoder(), getParseFn(), input.getPipeline().getCoderRegistry());
 if (getMatchConfiguration().getWatchInterval() == null && !getHintMatchesManyFiles()) {
  return input.apply(
    org.apache.beam.sdk.io.Read.from(
      AvroSource.from(getFilepattern()).withParseFn(getParseFn(), coder)));
 }
 // All other cases go through ParseAllGenericRecords.
 return input
   .apply("Create filepattern", Create.ofProvider(getFilepattern(), StringUtf8Coder.of()))
   .apply(
     "Via ParseAll",
     parseAllGenericRecords(getParseFn())
       .withCoder(coder)
       .withMatchConfiguration(getMatchConfiguration()));
}
origin: org.apache.beam/beam-sdks-java-core

paths.apply(
  "Parse all",
  AvroIO.parseAllGenericRecords(new ParseGenericClass())
    .withCoder(AvroCoder.of(GenericClass.class))
    .withDesiredBundleSizeBytes(10)))
origin: org.apache.beam/beam-sdks-java-core

path.apply(
  "ParseAll",
  AvroIO.parseAllGenericRecords(new ParseGenericClass())
    .withCoder(AvroCoder.of(GenericClass.class))
    .withDesiredBundleSizeBytes(10)))
origin: org.apache.beam/beam-sdks-java-core

paths.apply(
  "Parse all",
  AvroIO.parseAllGenericRecords(new ParseGenericClass())
    .withCoder(AvroCoder.of(GenericClass.class))
    .watchForNewFiles(
org.apache.beam.sdk.ioAvroIOparseAllGenericRecords

Javadoc

Like #parseGenericRecords(SerializableFunction), but reads each filepattern in the input PCollection.

Popular methods of AvroIO

  • readGenericRecords
    Reads Avro file(s) containing records of the specified schema.
  • writeGenericRecords
    Writes Avro records of the specified schema.
  • constantDestinations
    Returns a DynamicAvroDestinations that always returns the same FilenamePolicy, schema, metadata, and
  • read
    Reads records of the given type from an Avro file (or multiple Avro files matching a pattern).The sc
  • readAll
    Like #read, but reads each filepattern in the input PCollection.
  • write
    Writes a PCollection to an Avro file (or multiple Avro files matching a sharding pattern).
  • writeCustomTypeToGenericRecords
    Similar to #writeCustomType(), but specialized for the case where the output type is GenericRecord.
  • defaultWriteBuilder
  • parseGenericRecords
    Reads Avro file(s) containing records of an unspecified schema and converting each record to a custo
  • readAllGenericRecords
    Like #readGenericRecords(Schema), but reads each filepattern in the input PCollection.
  • sink
    A Sink for use with FileIO#write and FileIO#writeDynamic, writing elements of the given generated cl
  • sinkViaGenericRecords
    A Sink for use with FileIO#write and FileIO#writeDynamic, writing elements by converting each one to
  • sink,
  • sinkViaGenericRecords,
  • writeCustomType

Popular in Java

  • Making http requests using okhttp
  • setRequestProperty (URLConnection)
  • getApplicationContext (Context)
  • getExternalFilesDir (Context)
  • GridBagLayout (java.awt)
    The GridBagLayout class is a flexible layout manager that aligns components vertically and horizonta
  • FileWriter (java.io)
    A specialized Writer that writes to a file in the file system. All write requests made by calling me
  • Proxy (java.net)
    This class represents proxy server settings. A created instance of Proxy stores a type and an addres
  • SimpleDateFormat (java.text)
    Formats and parses dates in a locale-sensitive manner. Formatting turns a Date into a String, and pa
  • Executor (java.util.concurrent)
    An object that executes submitted Runnable tasks. This interface provides a way of decoupling task s
  • LoggerFactory (org.slf4j)
    The LoggerFactory is a utility class producing Loggers for various logging APIs, most notably for lo
  • Github Copilot alternatives
Tabnine Logo
  • Products

    Search for Java codeSearch for JavaScript code
  • IDE Plugins

    IntelliJ IDEAWebStormVisual StudioAndroid StudioEclipseVisual Studio CodePyCharmSublime TextPhpStormVimGoLandRubyMineEmacsJupyter NotebookJupyter LabRiderDataGripAppCode
  • Company

    About UsContact UsCareers
  • Resources

    FAQBlogTabnine AcademyTerms of usePrivacy policyJava Code IndexJavascript Code Index
Get Tabnine for your IDE now