congrats Icon
New! Tabnine Pro 14-day free trial
Start a free trial
Tabnine Logo
SourceProcessors.readMapP
Code IndexAdd Tabnine to your IDE (free)

How to use
readMapP
method
in
com.hazelcast.jet.core.processor.SourceProcessors

Best Java code snippets using com.hazelcast.jet.core.processor.SourceProcessors.readMapP (Showing top 9 results out of 315)

origin: hazelcast/hazelcast-jet

/**
 * Convenience for {@link #map(String, Predicate, Projection)}
 * which uses a {@link DistributedFunction} as the projection function.
 */
@Nonnull
public static <T, K, V> BatchSource<T> map(
    @Nonnull String mapName,
    @Nonnull Predicate<? super K, ? super V> predicate,
    @Nonnull DistributedFunction<? super Map.Entry<K, V>, ? extends T> projectionFn
) {
  return batchFromProcessor("mapSource(" + mapName + ')', readMapP(mapName, predicate, projectionFn));
}
origin: hazelcast/hazelcast-jet

/**
 * Returns a source that fetches entries from a local Hazelcast {@code IMap}
 * with the specified name and emits them as {@code Map.Entry}. It leverages
 * data locality by making each of the underlying processors fetch only those
 * entries that are stored on the member where it is running.
 * <p>
 * The source does not save any state to snapshot. If the job is restarted,
 * it will re-emit all entries.
 * <p>
 * If the {@code IMap} is modified while being read, or if there is a
 * cluster topology change (triggering data migration), the source may
 * miss and/or duplicate some entries.
 * <p>
 * The default local parallelism for this processor is 2 (or 1 if just 1
 * CPU is available).
 */
@Nonnull
public static <K, V> BatchSource<Entry<K, V>> map(@Nonnull String mapName) {
  return batchFromProcessor("mapSource(" + mapName + ')', readMapP(mapName));
}
origin: hazelcast/hazelcast-jet

    @Nonnull Projection<? super Entry<K, V>, ? extends T> projection
) {
  return batchFromProcessor("mapSource(" + mapName + ')', readMapP(mapName, predicate, projection));
origin: hazelcast/hazelcast-jet-code-samples

  public static void main(String[] args) throws Exception {
    System.setProperty("hazelcast.logging.type", "log4j");
    Jet.newJetInstance();
    JetInstance jet = Jet.newJetInstance();
    try {

      IMapJet<Object, Object> map = jet.getMap("map");
      range(0, COUNT).parallel().forEach(i -> map.put("key-" + i, i));

      DAG dag = new DAG();

      Vertex source = dag.newVertex("map-source", SourceProcessors.readMapP(map.getName()));
      Vertex sink = dag.newVertex("file-sink", new WriteFilePSupplier(OUTPUT_FOLDER));
      dag.edge(between(source, sink));

      jet.newJob(dag).join();
      System.out.println("\nHazelcast IMap dumped to folder " + new File(OUTPUT_FOLDER).getAbsolutePath());
    } finally {
      Jet.shutdownAll();
    }
  }
}
origin: hazelcast/hazelcast-jet

private void rewriteDagWithSnapshotRestore(DAG dag, long snapshotId, String mapName) {
  IMap<Object, Object> snapshotMap = nodeEngine.getHazelcastInstance().getMap(mapName);
  snapshotId = SnapshotValidator.validateSnapshot(snapshotId, jobIdString(), snapshotMap);
  logger.info("State of " + jobIdString() + " will be restored from snapshot " + snapshotId + ", map=" + mapName);
  List<Vertex> originalVertices = new ArrayList<>();
  dag.iterator().forEachRemaining(originalVertices::add);
  Map<String, Integer> vertexToOrdinal = new HashMap<>();
  Vertex readSnapshotVertex = dag.newVertex(SNAPSHOT_VERTEX_PREFIX + "read",
      readMapP(mapName));
  long finalSnapshotId = snapshotId;
  Vertex explodeVertex = dag.newVertex(SNAPSHOT_VERTEX_PREFIX + "explode",
      () -> new ExplodeSnapshotP(vertexToOrdinal, finalSnapshotId));
  dag.edge(between(readSnapshotVertex, explodeVertex).isolated());
  int index = 0;
  // add the edges
  for (Vertex userVertex : originalVertices) {
    vertexToOrdinal.put(userVertex.getName(), index);
    int destOrdinal = dag.getInboundEdges(userVertex.getName()).size();
    dag.edge(new SnapshotRestoreEdge(explodeVertex, index, userVertex, destOrdinal));
    index++;
  }
}
origin: hazelcast/hazelcast-jet

  public static CompletableFuture<Void> copyMapUsingJob(JetInstance instance, int queueSize,
                             String sourceMap, String targetMap) {
    DAG dag = new DAG();
    Vertex source = dag.newVertex("readMap(" + sourceMap + ')', readMapP(sourceMap));
    Vertex sink = dag.newVertex("writeMap(" + targetMap + ')', writeMapP(targetMap));
    dag.edge(between(source, sink).setConfig(new EdgeConfig().setQueueSize(queueSize)));
    JobConfig jobConfig = new JobConfig()
        .setName("copy-" + sourceMap + "-to-" + targetMap);
    return instance.newJob(dag, jobConfig).getFuture();
  }
}
origin: hazelcast/hazelcast-jet-code-samples

Vertex source = dag.newVertex("source", readMapP(DOCID_NAME));
origin: hazelcast/hazelcast-jet-code-samples

Vertex docSource = dag.newVertex("doc-source", readMapP(DOCID_NAME));
origin: hazelcast/hazelcast-jet-code-samples

Vertex readTickerInfoMap = dag.newVertex("readTickerInfoMap", readMapP(TICKER_INFO_MAP_NAME));
Vertex collectToMap = dag.newVertex("collectToMap",
    Processors.aggregateP(AggregateOperations.toMap(entryKey(), entryValue())));
com.hazelcast.jet.core.processorSourceProcessorsreadMapP

Javadoc

Returns a supplier of processors for Sources#map(String).

Popular methods of SourceProcessors

  • streamMapP
    Returns a supplier of processors for Sources#mapJournal(String,JournalInitialPosition) )}.
  • readCacheP
    Returns a supplier of processors for Sources#cache(String).
  • readFilesP
    Returns a supplier of processors for Sources#filesBuilder. See FileSourceBuilder#build for more deta
  • readJdbcP
    Returns a supplier of processors for Sources#jdbc(String,String,DistributedFunction).
  • readListP
    Returns a supplier of processors for Sources#list(String).
  • readRemoteCacheP
    Returns a supplier of processors for Sources#remoteCache(String,ClientConfig).
  • readRemoteListP
    Returns a supplier of processors for Sources#remoteList(String,ClientConfig).
  • readRemoteMapP
    Returns a supplier of processors for Sources#remoteMap(String,ClientConfig,Predicate,Projection).
  • streamCacheP
    Returns a supplier of processors for Sources#cacheJournal(String,JournalInitialPosition).
  • streamFilesP
    Returns a supplier of processors for Sources#filesBuilder. See FileSourceBuilder#buildWatcher for mo
  • streamJmsQueueP
    Returns a supplier of processors for Sources#jmsQueueBuilder.
  • streamJmsTopicP
    Returns a supplier of processors for Sources#jmsTopicBuilder.
  • streamJmsQueueP,
  • streamJmsTopicP,
  • streamRemoteCacheP,
  • streamRemoteMapP,
  • streamSocketP,
  • toProjection

Popular in Java

  • Reading from database using SQL prepared statement
  • setContentView (Activity)
  • addToBackStack (FragmentTransaction)
  • setRequestProperty (URLConnection)
  • Font (java.awt)
    The Font class represents fonts, which are used to render text in a visible way. A font provides the
  • Runnable (java.lang)
    Represents a command that can be executed. Often used to run code in a different Thread.
  • BitSet (java.util)
    The BitSet class implements abit array [http://en.wikipedia.org/wiki/Bit_array]. Each element is eit
  • GregorianCalendar (java.util)
    GregorianCalendar is a concrete subclass of Calendarand provides the standard calendar used by most
  • JList (javax.swing)
  • Options (org.apache.commons.cli)
    Main entry-point into the library. Options represents a collection of Option objects, which describ
  • Sublime Text for Python
Tabnine Logo
  • Products

    Search for Java codeSearch for JavaScript code
  • IDE Plugins

    IntelliJ IDEAWebStormVisual StudioAndroid StudioEclipseVisual Studio CodePyCharmSublime TextPhpStormVimAtomGoLandRubyMineEmacsJupyter NotebookJupyter LabRiderDataGripAppCode
  • Company

    About UsContact UsCareers
  • Resources

    FAQBlogTabnine AcademyStudentsTerms of usePrivacy policyJava Code IndexJavascript Code Index
Get Tabnine for your IDE now