ReadRel carries filter, best_effort_filter and projection (a MaskExpression), and AbstractReadRel exposes all three. SubstraitRelNodeConverter reads none of them: visit(NamedScan) (205), visit(LocalFiles) (211) and visit(VirtualTableScan) (808) each build their Calcite relation from the schema alone, and getFilter() / getBestEffortFilter() / getProjection() appear nowhere in the file. SubstraitRelVisitor never sets them on the way back, so the loss is silent in both directions.
A read carrying a mandatory filter therefore converts to a Calcite plan that reads every row, with no error — wrong results rather than a refusal. The projection mask is worse than dropped: AbstractReadRel.deriveRecordType applies it (MaskExpressionTypeProjector.project(projection, base)), so the POJO's record type is the masked schema while the emitted Calcite row type is the unmasked one, and every field index a parent relation computed against the read is off.
NamedScan(names=[t], schema=[a i64, b string], filter=<pred>, projection=<mask>)
converts as if it were
NamedScan(names=[t], schema=[a i64, b string])
Refusing an unsupported field would be safer than dropping it, the way visitOther and visit(Join) refuse what they cannot express.
Reproduced on main at 7310fc8. Distinct from #1160, which is about the emit mapping (Rel.getRemap()), not these three fields. Found while reviewing #1151.
ReadRelcarriesfilter,best_effort_filterandprojection(aMaskExpression), andAbstractReadRelexposes all three.SubstraitRelNodeConverterreads none of them:visit(NamedScan)(205),visit(LocalFiles)(211) andvisit(VirtualTableScan)(808) each build their Calcite relation from the schema alone, andgetFilter()/getBestEffortFilter()/getProjection()appear nowhere in the file.SubstraitRelVisitornever sets them on the way back, so the loss is silent in both directions.A read carrying a mandatory filter therefore converts to a Calcite plan that reads every row, with no error — wrong results rather than a refusal. The projection mask is worse than dropped:
AbstractReadRel.deriveRecordTypeapplies it (MaskExpressionTypeProjector.project(projection, base)), so the POJO's record type is the masked schema while the emitted Calcite row type is the unmasked one, and every field index a parent relation computed against the read is off.converts as if it were
Refusing an unsupported field would be safer than dropping it, the way
visitOtherandvisit(Join)refuse what they cannot express.Reproduced on
mainat 7310fc8. Distinct from #1160, which is about the emit mapping (Rel.getRemap()), not these three fields. Found while reviewing #1151.