private void addAllRowsToResult(RowResultIterator it, int recordcount, List<String> querySchema, Vector<HashMap<String, ByteIterator>> result) throws Exception { RowResult row; HashMap<String, ByteIterator> rowResult = new HashMap<>(querySchema.size()); if (it == null) { return; } while (it.hasNext()) { if (result.size() == recordcount) { return; } row = it.next(); int colIdx = 0; for (String col : querySchema) { rowResult.put(col, new StringByteIterator(row.getString(colIdx))); colIdx++; } result.add(rowResult); } }
public boolean advanceNextPosition() boolean needNextRows = !started || !nextRows.hasNext(); while (!nextRows.hasNext()); log.debug("Fetched " + nextRows.getNumRows() + " rows"); currentRow = nextRows.next(); totalBytes += getRowLength(); return true;
@Override public RowResultIterator call(final Response resp) { numRowsReturned += resp.data.getNumRows(); if (!resp.more) { // We're done scanning this tablet. scanFinished(); return resp.data; } sequenceId++; hasMore = resp.more; //LOG.info("Scan.next is returning rows: " + resp.data.getNumRows()); return resp.data; }
private static RowResult getOneRowResult(KuduScanner scanner) throws KuduException { RowResultIterator rowResults; rowResults = scanner.nextRows(); if (rowResults.getNumRows() == 0) { return null; } if (rowResults.getNumRows() > 1) { throw new RuntimeException("Received too many rows from scanner " + scanner); } return rowResults.next(); }
private void buildScannerAndCheckColumnsCount(AsyncKuduScanner.AsyncKuduScannerBuilder builder, int count) throws Exception { AsyncKuduScanner scanner = builder.build(); scanner.nextRows().join(DEFAULT_SLEEP); RowResultIterator rri = scanner.nextRows().join(DEFAULT_SLEEP); assertEquals(count, rri.next().getSchema().getColumns().size()); }
static RowResultIterator makeRowResultIterator(long ellapsedMillis, String tsUUID, Schema schema, WireProtocol.RowwiseRowBlockPB data, final CallResponse callResponse) throws KuduException { if (data == null || data.getNumRows() == 0) { return new RowResultIterator(ellapsedMillis, tsUUID, schema, 0, null, null); } Slice bs = callResponse.getSidecar(data.getRowsSidecar()); Slice indirectBs = callResponse.getSidecar(data.getIndirectDataSidecar()); int numRows = data.getNumRows(); // Integrity check int rowSize = schema.getRowSize(); int expectedSize = numRows * rowSize; if (expectedSize != bs.length()) { Status statusIllegalState = Status.IllegalState("RowResult block has " + bs.length() + " bytes of data but expected " + expectedSize + " for " + numRows + " rows"); throw new NonRecoverableException(statusIllegalState); } return new RowResultIterator(ellapsedMillis, tsUUID, schema, numRows, bs, indirectBs); }
@Override public Object call(RowResultIterator arg) throws Exception { if (arg == null) return null; counter.addAndGet(arg.getNumRows()); return null; } };
final RowResult next = rows.next();
@Override public int next() { int rowCount = 0; try { while (iterator == null || !iterator.hasNext()) { if (!scanner.hasMoreRows()) { iterator = null; return 0; } context.getStats().startWait(); try { iterator = scanner.nextRows(); } finally { context.getStats().stopWait(); } } for (; rowCount < TARGET_RECORD_COUNT && iterator.hasNext(); rowCount++) { addRowResult(iterator.next(), rowCount); } } catch (Exception ex) { throw new RuntimeException(ex); } for (ProjectedColumnInfo pci : projectedCols) { pci.vv.getMutator().setValueCount(rowCount); } return rowCount; }
/** * get next Row/Page */ @Override public boolean advanceNextPosition() { boolean needNextRows = !started || !nextRows.hasNext(); if (!started) { started = true; nanoStart = System.nanoTime(); } if (needNextRows) { currentRow = null; try { do { if (!scanner.hasMoreRows()) { return false; } nextRows = scanner.nextRows(); } while (!nextRows.hasNext()); log.debug("Fetched " + nextRows.getNumRows() + " rows"); } catch (KuduException e) { currentRow = null; throw new RuntimeException(e); } } currentRow = nextRows.next(); totalBytes += getRowLength(); return true; }
public static int countRowsInScan(KuduScanner scanner) throws KuduException { int counter = 0; while (scanner.hasMoreRows()) { counter += scanner.nextRows().getNumRows(); } return counter; }
RowResult rr = scanner.nextRows().next(); assertEquals(magicNumber, rr.getInt(1)); assertEquals(magicNumber, rr.getInt(2));
public boolean advanceNextPosition() boolean needNextRows = !started || !nextRows.hasNext(); while (!nextRows.hasNext()); log.debug("Fetched " + nextRows.getNumRows() + " rows"); currentRow = nextRows.next(); totalBytes += getRowLength(); return true;
numRowsReturned += resp.data.getNumRows();
private int countRows(KuduTable table, KuduPredicate... predicates) throws Exception { KuduScanner.KuduScannerBuilder scanBuilder = new KuduScanner.KuduScannerBuilder(asyncClient, table); for (KuduPredicate predicate : predicates) { scanBuilder.addPredicate(predicate); } KuduScanner scanner = scanBuilder.build(); int count = 0; while (scanner.hasMoreRows()) { count += scanner.nextRows().getNumRows(); } return count; }
int accum = scanner.nextRows().getNumRows(); int rows = scanner.nextRows().getNumRows(); accum += rows; accum += scanner.nextRows().getNumRows(); accum += scanner.nextRows().getNumRows();
@Override public KuduRow nextRecord(KuduRow reuse) throws IOException { // check that current iterator has next rows if (this.resultIterator.hasNext()) { RowResult row = this.resultIterator.next(); return KuduMapper.toKuduRow(row); } // if not, check that current scanner has more iterators else if (scanner.hasMoreRows()) { this.resultIterator = scanner.nextRows(); return nextRecord(reuse); } else { endReached = true; } return null; }