Tabnine Logo
FixedWidthDatastore
Code IndexAdd Tabnine to your IDE (free)

How to use
FixedWidthDatastore
in
org.datacleaner.connection

Best Java code snippets using org.datacleaner.connection.FixedWidthDatastore (Showing top 10 results out of 315)

origin: datacleaner/DataCleaner

public Element toElement(final FixedWidthDatastore datastore, final String filename) {
  final Element ds = getDocument().createElement("fixed-width-datastore");
  ds.setAttribute("name", datastore.getName());
  if (!Strings.isNullOrEmpty(datastore.getDescription())) {
    ds.setAttribute("description", datastore.getDescription());
  appendElement(ds, "encoding", datastore.getEncoding());
  final int fixedValueWidth = datastore.getFixedValueWidth();
  if (fixedValueWidth > -1) {
    final String valueOf = String.valueOf(fixedValueWidth);
    appendElement(widthElement, "fixed-value-width", valueOf);
  } else {
    final int[] valueWidths = datastore.getValueWidths();
    for (int i = 0; i < valueWidths.length; i++) {
      appendElement(widthElement, "value-width", String.valueOf(valueWidths[i]));
  final List<String> customColumnNames = datastore.getCustomColumnNames();
  if (customColumnNames != null && !customColumnNames.isEmpty()) {
    final Element columnNamesElement = getDocument().createElement("custom-column-names");
  appendElement(ds, "header-line-number", datastore.getHeaderLineNumber());
  appendElement(ds, "fail-on-inconsistencies", String.valueOf(datastore.isFailOnInconsistencies()));
  appendElement(ds, "skip-ebcdic-header", String.valueOf(datastore.isSkipEbcdicHeader()));
  appendElement(ds, "eol-present", String.valueOf(datastore.isEolPresent()));
origin: datacleaner/DataCleaner

_encodingComboBox.setSelectedItem(originalDatastore.getEncoding());
_failOnInconsistenciesCheckBox.setSelected(originalDatastore.isFailOnInconsistencies());
_skipEbcdicHeaderCheckBox.setSelected(originalDatastore.isSkipEbcdicHeader());
_eolPresentCheckBox.setSelected(originalDatastore.isEolPresent());
final List<String> customColumnNames = originalDatastore.getCustomColumnNames();
final int[] valueWidths = originalDatastore.getValueWidths();
_headerLineComboBox.setSelectedIndex(originalDatastore.getHeaderLineNumber());
origin: datacleaner/DataCleaner

private boolean isEbcdic() {
  // This is just a way how to differentiate between EBCDIC and normal FixedWidth configuration.
  // Perhaps there is a better way how to do this.
  return getEncoding().startsWith(EBCDIC_PREFIX);
}
origin: datacleaner/DataCleaner

final Resource resource = fixedWidthDatastore.getResource();
final String datastorePath = resource.getQualifiedPath();
final FixedWidthConfiguration fixedWidthConfiguration = fixedWidthDatastore.getConfiguration();
final JavaRDD<String> rawInput;
if (_minPartitions != null) {
origin: datacleaner/DataCleaner

private FixedWidthDatastore createDatastore(final String name, final Resource resource,
    final boolean failOnInconsistencies, final boolean skipEbcdicHeader, final boolean eolPresent) {
  final int[] valueWidths = getValueWidths(failOnInconsistencies);
  return new FixedWidthDatastore(name, resource, resource.getQualifiedPath(), _encodingComboBox.getSelectedItem(),
      valueWidths, failOnInconsistencies, skipEbcdicHeader, eolPresent, getHeaderLine(), getColumnNames());
}
origin: datacleaner/DataCleaner

public static boolean isHdfsResourcedDatastore(final Datastore datastore) {
  if (datastore instanceof CsvDatastore) {
    final CsvDatastore csvDatastore = (CsvDatastore) datastore;
    final Resource resource = csvDatastore.getResource();
    if (!isHdfsResource(resource)) {
      return false;
    }
  } else if (datastore instanceof JsonDatastore) {
    final JsonDatastore jsonDatastore = (JsonDatastore) datastore;
    final Resource resource = jsonDatastore.getResource();
    if (!isHdfsResource(resource)) {
      return false;
    }
  } else if (datastore instanceof FixedWidthDatastore) {
    final FixedWidthDatastore fixedWidthDatastore = (FixedWidthDatastore) datastore;
    final Resource resource = fixedWidthDatastore.getResource();
    if (!isHdfsResource(resource)) {
      return false;
    }
  } else {
    // other type of datastore
    return false;
  }
  return true;
}
origin: datacleaner/DataCleaner

  @Override
  public String toString() {
    return "FixedWidthDatastore[name=" + getName() + ", filename=" + _filename + ", encoding=" + _encoding
        + ", headerLineNumber=" + _headerLineNumber + ", valueWidths=" + Arrays.toString(_valueWidths)
        + ", fixedValueWidth=" + _fixedValueWidth + "]";
  }
}
origin: datacleaner/DataCleaner

@Override
protected UsageAwareDatastoreConnection<DataContext> createDatastoreConnection() {
  final FixedWidthConfiguration configuration = getConfiguration();
  final Resource resource = _resourceRef.get();
  final DataContext dataContext;
  if (resource == null) {
    logger.warn("Resource was not available, a local file reference will be created with path: {}", _filename);
    dataContext = DataContextFactory.createFixedWidthDataContext(new File(_filename), configuration);
  } else {
    dataContext = DataContextFactory.createFixedWidthDataContext(resource, configuration);
  }
  return new DatastoreConnectionImpl<>(dataContext, this);
}
origin: datacleaner/DataCleaner

  ds = new FixedWidthDatastore(name, resource, filename, encoding, valueWidths, failOnInconsistencies,
      skipEbcdicHeader, eolPresent, headerLineNumber, customColumnNames);
} else {
  ds = new FixedWidthDatastore(name, resource, filename, encoding, fixedValueWidth, failOnInconsistencies,
      skipEbcdicHeader, eolPresent, headerLineNumber);
origin: datacleaner/DataCleaner

  elem = toElement((JsonDatastore) datastore, filename);
} else if (datastore instanceof FixedWidthDatastore) {
  final Resource resource = ((FixedWidthDatastore) datastore).getResource();
  final String filename = toFilename(resource);
  elem = toElement((FixedWidthDatastore) datastore, filename);
org.datacleaner.connectionFixedWidthDatastore

Javadoc

Datastore based on fixed width files

Most used methods

  • getEncoding
  • <init>
  • getConfiguration
  • getCustomColumnNames
  • getHeaderLineNumber
  • getName
  • getResource
  • getValueWidths
  • isEolPresent
  • isFailOnInconsistencies
  • isSkipEbcdicHeader
  • getDescription
  • isSkipEbcdicHeader,
  • getDescription,
  • getFixedValueWidth,
  • isEbcdic

Popular in Java

  • Parsing JSON documents to java classes using gson
  • setContentView (Activity)
  • getResourceAsStream (ClassLoader)
  • onRequestPermissionsResult (Fragment)
  • GridLayout (java.awt)
    The GridLayout class is a layout manager that lays out a container's components in a rectangular gri
  • MessageDigest (java.security)
    Uses a one-way hash function to turn an arbitrary number of bytes into a fixed-length byte sequence.
  • NoSuchElementException (java.util)
    Thrown when trying to retrieve an element past the end of an Enumeration or Iterator.
  • TimerTask (java.util)
    The TimerTask class represents a task to run at a specified time. The task may be run once or repeat
  • Vector (java.util)
    Vector is an implementation of List, backed by an array and synchronized. All optional operations in
  • HttpServletRequest (javax.servlet.http)
    Extends the javax.servlet.ServletRequest interface to provide request information for HTTP servlets.
  • Top 12 Jupyter Notebook extensions
Tabnine Logo
  • Products

    Search for Java codeSearch for JavaScript code
  • IDE Plugins

    IntelliJ IDEAWebStormVisual StudioAndroid StudioEclipseVisual Studio CodePyCharmSublime TextPhpStormVimGoLandRubyMineEmacsJupyter NotebookJupyter LabRiderDataGripAppCode
  • Company

    About UsContact UsCareers
  • Resources

    FAQBlogTabnine AcademyTerms of usePrivacy policyJava Code IndexJavascript Code Index
Get Tabnine for your IDE now