protected void doPushLimitIntoGroupScan(RelOptRuleCall call, LimitPrel limit, final ProjectPrel project, ScanPrel scan, GroupScan groupScan) { try { final GroupScan newGroupScan = getGroupScanWithLimit(groupScan, limit); if (newGroupScan == null) { return; } final ScanPrel newScan = new ScanPrel(scan.getCluster(), scan.getTraitSet(), newGroupScan, scan.getRowType(), scan.getTable()); final RelNode newChild; if (project != null) { final ProjectPrel newProject = new ProjectPrel(project.getCluster(), project.getTraitSet(), newScan, project.getProjects(), project.getRowType()); newChild = newProject; } else { newChild = newScan; } call.transformTo(newChild); logger.debug("pushLimitIntoGroupScan: Converted to a new ScanPrel " + newScan.getGroupScan()); } catch (Exception e) { logger.warn("pushLimitIntoGroupScan: Exception while trying limit pushdown!", e); } }
protected void doPushLimitIntoRowKeyJoin(RelOptRuleCall call, LimitPrel limit, final ProjectPrel project, RowKeyJoinPrel join) { final RelNode newChild; try { RelNode left = join.getLeft(); RelNode right = join.getRight(); final RelNode limitOnLeft = new LimitPrel(left.getCluster(), left.getTraitSet(), left, limit.getOffset(), limit.getFetch()); RowKeyJoinPrel newJoin = new RowKeyJoinPrel(join.getCluster(), join.getTraitSet(), limitOnLeft, right, join.getCondition(), join.getJoinType()); if (project != null) { final ProjectPrel newProject = new ProjectPrel(project.getCluster(), project.getTraitSet(), newJoin, project.getProjects(), project.getRowType()); newChild = newProject; } else { newChild = newJoin; } call.transformTo(newChild); logger.debug("pushLimitIntoRowKeyJoin: Pushed limit on left side of Join " + join.toString()); } catch (Exception e) { logger.warn("pushLimitIntoRowKeyJoin: Exception while trying limit pushdown!", e); } } }
@Override public void onMatch(RelOptRuleCall call) { final ProjectPrel project = call.rel(1); final LimitPrel limit = call.rel(0); RelNode child = project.getInput(); final RelNode limitUnderProject = new LimitPrel(child.getCluster(), child.getTraitSet(), child, limit.getOffset(), limit.getFetch()); final RelNode newProject = new ProjectPrel(project.getCluster(), project.getTraitSet(), limitUnderProject, project.getProjects(), project.getRowType()); if (DrillRelOptUtil.isProjectFlatten(project)) { //Preserve limit above the project since Flatten can produce more rows. Also mark it so we do not fire the rule again. child = newProject; final RelNode limitAboveProject = new LimitPrel(child.getCluster(), child.getTraitSet(), child, limit.getOffset(), limit.getFetch(), true); call.transformTo(limitAboveProject); } else { call.transformTo(newProject); } }
new ProjectPrel(project.getCluster(), project.getTraitSet().plus(Prel.DRILL_PHYSICAL), newScan,
@Override public Project copy(RelTraitSet traitSet, RelNode input, List<RexNode> exps, RelDataType rowType) { return new ProjectPrel(getCluster(), traitSet, input, exps, rowType, this.outputProj); }
ProjectPrel cap = new ProjectPrel(finalRel.getCluster(), finalRel.getTraitSet(), finalRel, IndexPlanUtils.getProjects(upperProject), upperProject.getRowType()); finalRel = cap;
private Prel addTrivialOrderedProjectPrel(Prel prel) { RelDataType t = prel.getRowType(); RexBuilder b = prel.getCluster().getRexBuilder(); List<RexNode> projections = Lists.newArrayList(); int projectCount = t.getFieldList().size(); // no point in reordering if we only have one column if (projectCount < 2) { return prel; } for (int i = 0; i < projectCount; i++) { projections.add(b.makeInputRef(prel, i)); } return new ProjectPrel(prel.getCluster(), prel.getTraitSet(), prel, projections, prel.getRowType()); }
@Override public RelNode createProject(RelNode child, List<? extends RexNode> childExprs, List<String> fieldNames) { final RelOptCluster cluster = child.getCluster(); final RelDataType rowType = RexUtil.createStructType(cluster.getTypeFactory(), childExprs, fieldNames); final RelNode project = new ProjectPrel(cluster, child.getTraitSet().plus(Prel.DRILL_PHYSICAL), child, Lists.newArrayList(childExprs), rowType); return project; } }
private RelNode rename(RelNode input, List<RelDataTypeField> inputFields, List<String> outputFieldNames) { if (outputFieldNames.size() == 0) { return input; } List<RexNode> exprs = Lists.newArrayList(); for (RelDataTypeField field : inputFields) { RexNode expr = input.getCluster().getRexBuilder().makeInputRef(field.getType(), field.getIndex()); exprs.add(expr); } RelDataType rowType = RexUtil.createStructType(input.getCluster().getTypeFactory(), exprs, outputFieldNames, null); ProjectPrel proj = new ProjectPrel(input.getCluster(), input.getTraitSet(), input, exprs, rowType); return proj; }
public RelNode buildRowKeyProject(RelNode inputRel, int fieldIndex) { List<RelDataTypeField> inputFields = inputRel.getRowType().getFieldList(); final RelDataTypeField rowKeyField = inputFields.get(fieldIndex); RexNode expr = builder.makeInputRef(rowKeyField.getType(), rowKeyField.getIndex()); List<RexNode> exprs = Lists.newArrayList(); exprs.add(expr); final RelDataTypeFactory.FieldInfoBuilder rightFieldTypeBuilder = inputRel.getCluster().getTypeFactory().builder(); rightFieldTypeBuilder.add(rowKeyField); final RelDataType projectRowType = rightFieldTypeBuilder.build(); ProjectPrel proj = new ProjectPrel(inputRel.getCluster(), inputRel.getTraitSet(), inputRel, exprs, projectRowType); return proj; }
private RelNode rename(RelNode input, List<RelDataTypeField> inputFields, List<String> outputFieldNames) { List<RexNode> exprs = Lists.newArrayList(); for (RelDataTypeField field : inputFields) { RexNode expr = input.getCluster().getRexBuilder().makeInputRef(field.getType(), field.getIndex()); exprs.add(expr); } RelDataType rowType = RexUtil.createStructType(input.getCluster().getTypeFactory(), exprs, outputFieldNames, null); ProjectPrel proj = new ProjectPrel(input.getCluster(), input.getTraitSet(), input, exprs, rowType); return proj; }
@Override public RelNode convertChild(DrillProjectRel project, RelNode rel) throws RuntimeException { DrillDistributionTrait childDist = rel.getTraitSet().getTrait(DrillDistributionTraitDef.INSTANCE); RelCollation childCollation = rel.getTraitSet().getTrait(RelCollationTraitDef.INSTANCE); DrillDistributionTrait newDist = convertDist(childDist, distributionMap); RelCollation newCollation = convertRelCollation(childCollation, collationMap); RelTraitSet newProjectTraits = newTraitSet(Prel.DRILL_PHYSICAL, newDist, newCollation); return new ProjectPrel(project.getCluster(), newProjectTraits, rel, project.getProjects(), project.getRowType()); }
public RelNode buildOriginalProject (RelNode newRel) { RelDataType origRowType = origProject == null ? origScan.getRowType() : origProject.getRowType(); final RelDataTypeFactory.FieldInfoBuilder finalFieldTypeBuilder = origScan.getCluster().getTypeFactory().builder(); List<RelDataTypeField> hjRowFields = newRel.getRowType().getFieldList(); int toRemoveRowKeyCount = 1; if (getRowKeyIndex(origRowType, origScan) < 0 ) { toRemoveRowKeyCount = 2; } finalFieldTypeBuilder.addAll(hjRowFields.subList(0, hjRowFields.size()-toRemoveRowKeyCount)); final RelDataType finalProjectRowType = finalFieldTypeBuilder.build(); List<RexNode> resetExprs = Lists.newArrayList(); for (int idx=0; idx<hjRowFields.size()-toRemoveRowKeyCount; ++idx) { resetExprs.add(RexInputRef.of(idx, newRel.getRowType())); } final ProjectPrel resetProjectPrel = new ProjectPrel(newRel.getCluster(), newRel.getTraitSet(), newRel, resetExprs, finalProjectRowType); newRel = resetProjectPrel; RelNode finalRel = Prule.convert(newRel, newRel.getTraitSet()); return finalRel; }
protected void doPushLimitIntoGroupScan(RelOptRuleCall call, LimitPrel limit, final ProjectPrel project, ScanPrel scan, GroupScan groupScan) { try { final GroupScan newGroupScan = getGroupScanWithLimit(groupScan, limit); if (newGroupScan == null) { return; } final ScanPrel newScan = new ScanPrel(scan.getCluster(), scan.getTraitSet(), newGroupScan, scan.getRowType(), scan.getTable()); final RelNode newChild; if (project != null) { final ProjectPrel newProject = new ProjectPrel(project.getCluster(), project.getTraitSet(), newScan, project.getProjects(), project.getRowType()); newChild = newProject; } else { newChild = newScan; } call.transformTo(newChild); logger.debug("pushLimitIntoGroupScan: Converted to a new ScanPrel " + newScan.getGroupScan()); } catch (Exception e) { logger.warn("pushLimitIntoGroupScan: Exception while trying limit pushdown!", e); } }
private Prel prefixTabNameToStar(Prel prel, Void value) throws RuntimeException { if (StarColumnHelper.containsStarColumn(prel.getRowType()) && prefixedForStar) { List<RexNode> exprs = Lists.newArrayList(); for (RelDataTypeField field : prel.getRowType().getFieldList()) { RexNode expr = prel.getCluster().getRexBuilder().makeInputRef(field.getType(), field.getIndex()); exprs.add(expr); } List<String> fieldNames = Lists.newArrayList(); long tableId = tableNumber.getAndIncrement(); for (String name : prel.getRowType().getFieldNames()) { if (StarColumnHelper.isNonPrefixedStarColumn(name)) { fieldNames.add("T" + tableId + StarColumnHelper.PREFIX_DELIMITER + name); // Add prefix to * column. } else { fieldNames.add(name); // Keep regular column as it is. } } RelDataType rowType = RexUtil.createStructType(prel.getCluster().getTypeFactory(), exprs, fieldNames, null); // insert a PAS. ProjectPrel proj = new ProjectPrel(prel.getCluster(), prel.getTraitSet(), prel, exprs, rowType); return proj; } else { return visitPrel(prel, value); } }
scan.getTraitSet().plus(Prel.DRILL_PHYSICAL).plus(DrillDistributionTrait.SINGLETON), directScan, scanRowType); final ProjectPrel newProject = new ProjectPrel(agg.getCluster(), agg.getTraitSet().plus(Prel.DRILL_PHYSICAL) .plus(DrillDistributionTrait.SINGLETON), newScan, prepareFieldExpressions(scanRowType), agg.getRowType());
protected void doPushLimitIntoRowKeyJoin(RelOptRuleCall call, LimitPrel limit, final ProjectPrel project, RowKeyJoinPrel join) { final RelNode newChild; try { RelNode left = join.getLeft(); RelNode right = join.getRight(); final RelNode limitOnLeft = new LimitPrel(left.getCluster(), left.getTraitSet(), left, limit.getOffset(), limit.getFetch()); RowKeyJoinPrel newJoin = new RowKeyJoinPrel(join.getCluster(), join.getTraitSet(), limitOnLeft, right, join.getCondition(), join.getJoinType()); if (project != null) { final ProjectPrel newProject = new ProjectPrel(project.getCluster(), project.getTraitSet(), newJoin, project.getProjects(), project.getRowType()); newChild = newProject; } else { newChild = newJoin; } call.transformTo(newChild); logger.debug("pushLimitIntoRowKeyJoin: Pushed limit on left side of Join " + join.toString()); } catch (Exception e) { logger.warn("pushLimitIntoRowKeyJoin: Exception while trying limit pushdown!", e); } } }
private Prel insertProjUnderScreenOrWriter(Prel prel, RelDataType origRowType, Prel child) { ProjectPrel proj; List<RelNode> children = Lists.newArrayList(); List<RexNode> exprs = Lists.newArrayList(); for (int i = 0; i < origRowType.getFieldCount(); i++) { RexNode expr = child.getCluster().getRexBuilder().makeInputRef(origRowType.getFieldList().get(i).getType(), i); exprs.add(expr); } RelDataType newRowType = RexUtil.createStructType(child.getCluster().getTypeFactory(), exprs, origRowType.getFieldNames(), null); int fieldCount = prel.getRowType().isStruct() ? prel.getRowType().getFieldCount() : 1; // Insert PUS/PUW : remove the prefix and keep the original field name. if (fieldCount > 1) { // no point in allowing duplicates if we only have one column proj = new ProjectAllowDupPrel(child.getCluster(), child.getTraitSet(), child, exprs, newRowType, true); //outputProj = true : will allow to build the schema for PUS Project, see ProjectRecordBatch#handleNullInput() } else { proj = new ProjectPrel(child.getCluster(), child.getTraitSet(), child, exprs, newRowType, true); //outputProj = true : will allow to build the schema for PUS Project, see ProjectRecordBatch#handleNullInput() } children.add(proj); return (Prel) prel.copy(prel.getTraitSet(), children); }
@Override public void onMatch(RelOptRuleCall call) { final DrillProjectRel project = call.rel(0); final RelNode input = project.getInput(); RelTraitSet traits = input.getTraitSet().plus(Prel.DRILL_PHYSICAL); RelNode convertedInput = convert(input, traits); // Maintain two different map for distribution trait and collation trait. // For now, the only difference comes from the way how cast function impacts propagating trait. final Map<Integer, Integer> distributionMap = getDistributionMap(project); final Map<Integer, Integer> collationMap = getCollationMap(project); boolean traitPull = new ProjectTraitPull(call, distributionMap, collationMap).go(project, convertedInput); if(!traitPull){ call.transformTo(new ProjectPrel(project.getCluster(), convertedInput.getTraitSet(), convertedInput, project.getProjects(), project.getRowType())); } }
@Override public void onMatch(RelOptRuleCall call) { final ProjectPrel project = call.rel(1); final LimitPrel limit = call.rel(0); RelNode child = project.getInput(); final RelNode limitUnderProject = new LimitPrel(child.getCluster(), child.getTraitSet(), child, limit.getOffset(), limit.getFetch()); final RelNode newProject = new ProjectPrel(project.getCluster(), project.getTraitSet(), limitUnderProject, project.getProjects(), project.getRowType()); if (DrillRelOptUtil.isProjectFlatten(project)) { //Preserve limit above the project since Flatten can produce more rows. Also mark it so we do not fire the rule again. child = newProject; final RelNode limitAboveProject = new LimitPrel(child.getCluster(), child.getTraitSet(), child, limit.getOffset(), limit.getFetch(), true); call.transformTo(limitAboveProject); } else { call.transformTo(newProject); } }