Skip to content

feat(rest): add scan plan endpoint support to REST catalog client - #783

Open
gsandeep1241 wants to merge 36 commits into
apache:mainfrom
gsandeep1241:sandeepg-scan-plan-endpoint-for-rest-catalog-client-2-impl
Open

gsandeep1241 wants to merge 36 commits into
apache:mainfrom
gsandeep1241:sandeepg-scan-plan-endpoint-for-rest-catalog-client-2-impl

Conversation

@gsandeep1241

@gsandeep1241 gsandeep1241 commented Jun 26, 2026 •

Copy link
Copy Markdown
Contributor

When a table is loaded from a REST catalog with scan-planning-mode=server (table config takes precedence over catalog config), NewScan() now returns a RestTableScanBuilder whose Build() produces a RestTableScan. PlanFiles() and PlanFilesStream() on that scan delegate manifest resolution to the server via POST /plan, GET /plan/{id} (with exponential backoff), POST /tasks, and DELETE /plan/{id} (best-effort cancel), instead of reading manifests locally.

  • Add RestTable, RestTableScanBuilder, RestTableScan and RestScanContext
  • Promote DataTableScan::PlanFilesStream, TableScan::io and TableScanBuilder::Build to virtual
  • Convert RestCatalog::client_ and paths_ to shared_ptr so RestScanContext can share ownership with live scans
  • Add REST incremental append planning for scans with an exclusive start snapshot; scans without one (including an inclusive root snapshot) return NotSupported
  • Use storage credentials returned by planning responses in a scan-scoped FileIO

Comment thread src/iceberg/catalog/rest/rest_catalog.cc Outdated
Comment thread src/iceberg/catalog/rest/rest_table_scan.cc Outdated
Comment thread src/iceberg/catalog/rest/rest_table_scan.cc
Comment thread src/iceberg/catalog/rest/rest_table_scan.cc Outdated
@github-actions

Copy link
Copy Markdown

This pull request has been marked as stale due to 30 days of inactivity. It will be closed in 1 week if no further activity occurs. If you think that’s incorrect or this pull request requires a review, please simply write any comment. If closed, you can revive the PR at any time and @mention a reviewer or discuss it on the dev@iceberg.apache.org list. Thank you for your contributions.

@github-actions github-actions Bot added the stale label Aug 29, 2026
@github-actions

github-actions Bot commented Sep 5, 2026

Copy link
Copy Markdown

This pull request has been closed due to lack of activity. This is not a judgement on the merit of the PR in any way. It is just a way of keeping the PR queue manageable. If you think that is incorrect, or the pull request requires review, you can revive the PR at any time.

@github-actions github-actions Bot closed this Sep 5, 2026
@wgtmac

wgtmac commented Sep 7, 2026

Copy link
Copy Markdown
Member

Hey @gsandeep1241, do you want to revive this?

@wgtmac wgtmac reopened this Sep 7, 2026
@github-actions github-actions Bot removed the stale label Sep 8, 2026
@gsandeep1241

Copy link
Copy Markdown
Contributor Author

@wgtmac Thanks for re-opening it. Apologies for the long delay, I'll get back on it this week - next set of changes should be out for review in the next couple of days!

@gsandeep1241
gsandeep1241 force-pushed the sandeepg-scan-plan-endpoint-for-rest-catalog-client-2-impl branch from fb8da85 to d8a7d6c Compare September 13, 2026 19:53
Comment thread src/iceberg/catalog/rest/rest_catalog.cc Outdated
Comment thread src/iceberg/catalog/rest/rest_table_scan.cc Outdated
Comment thread src/iceberg/catalog/rest/rest_table_scan.cc Outdated
Comment thread src/iceberg/catalog/rest/rest_table_scan.cc
Comment thread src/iceberg/catalog/rest/rest_table_scan.cc Outdated
@gsandeep1241
gsandeep1241 requested a review from wgtmac September 13, 2026 21:33
@gsandeep1241
gsandeep1241 marked this pull request as ready for review September 13, 2026 21:38
@gsandeep1241
gsandeep1241 force-pushed the sandeepg-scan-plan-endpoint-for-rest-catalog-client-2-impl branch 2 times, most recently from 2bab861 to bcc9da4 Compare September 13, 2026 22:48
@gsandeep1241

Copy link
Copy Markdown
Contributor Author

Hey @gsandeep1241, do you want to revive this?

Thanks @wgtmac for reviving this! This is now ready for review. Please take a look when you can :)

Comment thread src/iceberg/catalog/rest/rest_table_scan.h Outdated
Comment thread src/iceberg/catalog/rest/rest_table_scan.h Outdated
Comment thread src/iceberg/catalog/rest/rest_table_scan.cc Outdated
Comment thread src/iceberg/catalog/rest/rest_table_scan.cc Outdated
Comment thread src/iceberg/catalog/rest/rest_table.h
Comment thread src/iceberg/test/rest_table_scan_test.cc Outdated
Comment thread src/iceberg/test/rest_table_scan_test.cc Outdated
Comment thread src/iceberg/test/rest_table_scan_test.cc Outdated
Comment thread src/iceberg/test/rest_table_scan_test.cc Outdated
Comment thread src/iceberg/test/rest_table_scan_test.cc Outdated
@gsandeep1241
gsandeep1241 force-pushed the sandeepg-scan-plan-endpoint-for-rest-catalog-client-2-impl branch from a9cbb7d to 5de0344 Compare September 29, 2026 03:22
@gsandeep1241
gsandeep1241 requested a review from wgtmac September 29, 2026 06:33
@gsandeep1241

Copy link
Copy Markdown
Contributor Author

@wgtmac Thank you for the detailed review. I've addressed your comments. Can you please take a look once more when you have the chance?

if (metadata_->current_snapshot_id == kInvalidSnapshotId) return {};
ICEBERG_ASSIGN_OR_RAISE(auto snapshot, metadata_->Snapshot());
if (!snapshot) return {};
request.end_snapshot_id = snapshot->snapshot_id;

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

When no explicit to_snapshot_id is set, this always uses the table's current snapshot and ignores context_.branch. A branch whose head differs from main will send the wrong end-snapshot-id; please resolve the branch head here and validate explicit endpoints against it.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Addressed by overriding the other PlanFiles method here and letting the base handle it.

RestScanContext rest_context);

RestScanContext rest_context_;
mutable std::shared_ptr<FileIO> scan_io_;

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This stores the vended scan IO, but the incremental scan does not override TableScan::io(). Normal callers therefore keep using the table IO after PlanFiles(), so credentials returned for incremental planning are never used to read the tasks. Please expose the effective IO here as well.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done.

Result<std::unique_ptr<DataTableScan>> RestTableScanBuilder::Build() {
ICEBERG_RETURN_UNEXPECTED(CheckErrors());
ICEBERG_RETURN_UNEXPECTED(context_.Validate());
ICEBERG_ASSIGN_OR_RAISE(auto schema, ResolveSnapshotSchema());

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ResolveSnapshotSchema() always returns the snapshot schema when snapshot_id is set, even when UseRef() marked a branch scan with use_snapshot_schema=false. Java uses the current table schema for branches and the snapshot schema for tags/time travel; the schema passed to this REST scan should follow that flag.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done. Fixed in table_scan.cc

.catalog_config = config_.configs(),
.table_config = table_config,
};
return RestTable::Make(identifier, std::move(result.metadata),

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This creates a RestTable only when the table is loaded. MakeTableFromCommitResponse() still returns a plain Table, so committing through a server-planning table silently switches later scans back to local manifest planning. Please preserve the REST table type and scan context after commit.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done

std::shared_ptr<FileIO>& scan_io) {
if (credentials.empty()) return {};
ICEBERG_ASSIGN_OR_RAISE(
auto io, MakeTableFileIO(ctx.catalog_config, ctx.table_config, credentials));

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This builds the scan IO from static credentials only. It does not install a StorageCredentialProvider or pass the scan plan-id; Java adds rest-scan-plan-id and refreshes vended credentials through the credentials endpoint. Temporary credentials can otherwise expire during a scan.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done

ICEBERG_UNWRAP_OR_FAIL(auto tasks1, scan->PlanFiles());
EXPECT_TRUE(tasks1.empty());

// Second call: no credentials. Verifies scan_io_ was cleared so stale

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

These assertions only check that planning returns tasks. Please also call scan->io() after vended credentials are returned; the public incremental-scan path currently exposes the original table IO.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done

// UseSnapshot(): the POST body sent to the server must contain both
// "snapshot-id" and "use-snapshot-schema": true.
// --------------------------------------------------------------------------
TEST_F(RestTableScanTest, UseSnapshotPropagatesUseSnapshotSchemaInContext) {

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please add request-body tests for Project() and IncludeColumnStats({"..."}). The current suite checks snapshot flags, but it does not catch the missing select projection or stats-fields described above.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done

// --------------------------------------------------------------------------
// PlanFilesStream: SUBMITTED → poll until COMPLETED, stream yields all tasks.
// --------------------------------------------------------------------------
TEST_F(RestTableScanTest, PlanFilesStreamSubmittedThenCompleted) {

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This submitted case has no plan tasks, so it cannot verify lazy fetching. Please return multiple plan-task tokens from the poll response and assert that /tasks is not called until Next() consumes the stream.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done

EXPECT_FALSE(scan->context().use_snapshot_schema);
}

INSTANTIATE_TEST_SUITE_P(TableScanVersions, TableScanTest, testing::Values(1, 2, 3));

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

These use_snapshot_schema checks do not depend on format version, but the parameterization runs each one for versions 1, 2, and 3. Please make these non-parameterized or avoid repeating the same assertions three times.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done

// PlanFilesStream: stream destroyed with no Next() calls at all →
// DELETE /plan called by the destructor.
// --------------------------------------------------------------------------
TEST_F(RestTableScanTest, PlanFilesStreamCancelOnPartialConsumption) {

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The test name says partial consumption, but the stream is destroyed without any Next() call. Please rename it to make the scenario clear, such as CancelOnNoConsumption.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done

@gsandeep1241
gsandeep1241 force-pushed the sandeepg-scan-plan-endpoint-for-rest-catalog-client-2-impl branch 2 times, most recently from 6dcfb2e to 5ff7165 Compare October 11, 2026 23:17

@gsandeep1241 gsandeep1241 left a comment

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Addressed all comments.

if (metadata_->current_snapshot_id == kInvalidSnapshotId) return {};
ICEBERG_ASSIGN_OR_RAISE(auto snapshot, metadata_->Snapshot());
if (!snapshot) return {};
request.end_snapshot_id = snapshot->snapshot_id;

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Addressed by overriding the other PlanFiles method here and letting the base handle it.

RestScanContext rest_context);

RestScanContext rest_context_;
mutable std::shared_ptr<FileIO> scan_io_;

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done.

Result<std::unique_ptr<DataTableScan>> RestTableScanBuilder::Build() {
ICEBERG_RETURN_UNEXPECTED(CheckErrors());
ICEBERG_RETURN_UNEXPECTED(context_.Validate());
ICEBERG_ASSIGN_OR_RAISE(auto schema, ResolveSnapshotSchema());

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done. Fixed in table_scan.cc

.catalog_config = config_.configs(),
.table_config = table_config,
};
return RestTable::Make(identifier, std::move(result.metadata),

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done


Result<std::unique_ptr<DataTableScan>> RestTableScanBuilder::Build() {
ICEBERG_RETURN_UNEXPECTED(CheckErrors());
ICEBERG_RETURN_UNEXPECTED(context_.Validate());

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done, thanks!


constexpr int64_t kMinSleepMs = 1'000;
constexpr int64_t kMaxSleepMs = 60'000;
constexpr int kMaxRetries = 10;

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done. Kept the sleeps here to match java.

Comment thread src/iceberg/catalog/rest/rest_table_scan.cc
ICEBERG_ASSIGN_OR_RAISE(auto json, FromJsonString(response.body()));
ICEBERG_ASSIGN_OR_RAISE(auto result,
PlanTableScanResponseFromJson(json, specs, schema));
ICEBERG_RETURN_UNEXPECTED(result.Validate());

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done

std::shared_ptr<FileIO>& scan_io) {
if (credentials.empty()) return {};
ICEBERG_ASSIGN_OR_RAISE(
auto io, MakeTableFileIO(ctx.catalog_config, ctx.table_config, credentials));

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done

Comment thread src/iceberg/table_scan.cc
TableScanBuilder<ScanType>::ResolveSnapshotSchema() {
if (snapshot_schema_ == nullptr) {
if (context_.snapshot_id.has_value()) {
if (context_.snapshot_id.has_value() && context_.use_snapshot_schema) {

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This changes existing behavior for locally planned scans too.

@gsandeep1241
gsandeep1241 requested a review from wgtmac October 11, 2026 23:19
Sandeep Gottimukkala and others added 2 commits October 13, 2026 16:10
When a table is loaded from a REST catalog that advertises the PlanTableScan
endpoint, NewScan() now returns a RestTableScanBuilder whose Build() produces
a RestTableScan. PlanFiles() on that scan delegates manifest resolution to
the server via POST /plan, GET /plan/{id} (with exponential backoff),
POST /tasks/{id}, and DELETE /plan/{id} (best-effort cancel), instead of
reading manifests locally.

- Add RestTable, RestTableScanBuilder, RestTableScan and RestScanContext
- Promote DataTableScan::PlanFiles and TableScanBuilder::Build to virtual
- Convert RestCatalog::client_ and paths_ to shared_ptr so RestScanContext
  can share ownership with live scans
- Cancel server-side plan when ResolveScanTasks fails partway through
- Propagate use_snapshot_schema from scan context to PlanTableScanRequest:
  true for UseSnapshot/AsOfTime/tag refs and incremental scans, false for
  branch refs and default scans
- Gate RestTable creation on effective scan-planning-mode config (table
  config overrides client config, default is client); error if server mode
  is requested but endpoint is not advertised
- Add ScanPlanningMode enum and ScanPlanningModeFrom() parser to
  RestCatalogProperties
- Make HttpClient methods virtual and add HttpResponse::MakeForTesting()
  to support unit test mocking
- Add tests: use_snapshot_schema in table_scan_test, ScanPlanningModeFrom
  parsing in catalog_properties_test, and RestTableScan HTTP flow tests
  in rest_table_scan_test
Sandeep Gottimukkala and others added 28 commits October 13, 2026 16:10
PlanFiles() was made virtual in this PR to support a RestTableScan override.
That override has since been replaced by RestTableScan::PlanFilesStream(), so
PlanFiles() no longer needs to be virtual — callers dispatch through the
virtual PlanFilesStream() hook instead.
… and virtual io()

- PlanFilesStream: COMPLETED, lazy token fetching, cancel-on-partial-consume,
  SUBMITTED→poll→COMPLETED, FAILED
- RestIncrementalAppendScan: COMPLETED, plan-tasks, SUBMITTED→poll→COMPLETED,
  FAILED, endpoint-not-supported, no-current-snapshot returns empty,
  explicit to_snapshot_id, exclusive from_snapshot_id, inclusive
  from_snapshot_id with and without a parent snapshot
- RestTable::NewIncrementalAppendScan returns RestIncrementalAppendScanBuilder
FetchPlanningResult returned early on GET failure, JSON parse error, response
deserialization error, Validate() failure, and ApplyStorageCredentials failure
without cancelling, leaving the server holding plan resources.

ExecuteScanPlanStream and ExecuteScanPlan had the same gap for the kFailed
status and ApplyStorageCredentials failure in the kCompleted branch.

Replace ICEBERG_ASSIGN_OR_RAISE / ICEBERG_RETURN_UNEXPECTED with explicit
checks in FetchPlanningResult's poll loop, and add CancelPlanning before every
error return in ExecuteScanPlanStream and ExecuteScanPlan where plan_id is known.
If PlanFilesStream()/PlanFiles() is called more than once and only the first
response vends storage credentials, the second plan would inherit the stale
credential-scoped FileIO from the first. Reset the slot to null before each
planning call so that io() falls back to the table IO when the server does
not vend credentials in the new response.
…ial fixes

- Add JsonBodyHas/JsonBodyLacks GMock matchers that parse the POST body
  so request field propagation is verified against actual server JSON,
  not just internal context state
- Update UseSnapshot and DefaultScan tests to call PlanFiles() and match
  "snapshot-id"/"use-snapshot-schema" in the POST body
- Update incremental scan snapshot routing tests to match
  "start-snapshot-id"/"end-snapshot-id" values in the POST body
- Add PlanFilesStreamFailedWithPlanIdCancels: verifies DELETE is called
  when ExecuteScanPlanStream receives FAILED with a plan-id
- Add CancelCalledOnFetchPlanningResultGetError: verifies DELETE is called
  when FetchPlanningResult GET fails after SUBMITTED
- Add SecondPlanFilesStreamCallClearsStaleCredentials: verifies scan_io_slot_
  is reset so second plan response with no credentials reverts io() to table IO
- Add PlanFilesFailedWithPlanIdCancels for RestIncrementalAppendScan
- Add SecondPlanFilesCallClearsStaleCredentials for RestIncrementalAppendScan
…partial-consume tests

- Rename PlanFilesStreamLazilyFetchesPlanTasks to
  PlanFilesStreamFetchesEachTokenSeparately; verify task content and count
  rather than empty responses, since the old comment implied laziness was
  being tested when only call-count was verified
- Add PlanFilesStreamFetchesTokenOnlyWhenBufferExhausted: uses a counter
  incremented inside WillOnce lambdas to assert the second FetchScanTasks
  POST is not made until the first token buffer is fully drained
- Add PlanFilesStreamNextReturnsErrorOnFetchFailure: calls Next() directly
  and asserts it returns IOError when FetchScanTasks fails
- Add PlanFilesStreamCancelAfterPartialConsumption: consumes one task then
  destroys the stream, verifying DELETE called with a token still pending
- Retain PlanFilesStreamCancelOnPartialConsumption with updated comment
  clarifying it tests destroy-before-any-Next
…ests

- Add MakeForTesting() factory to RestCatalog that bypasses FetchServerConfig,
  enabling unit tests to inject pre-built client/paths/endpoints.
- Add rest_catalog_unit_test.cc with four tests covering:
  table config server scan -> RestTable, client config server scan -> RestTable,
  server scan with missing PlanTableScan endpoint -> NotSupported,
  and default (no config) -> plain Table.
- Fix RestIncrementalAppendScan::PlanFiles() to check kInvalidSnapshotId
  before calling Snapshot(), so a table with no current snapshot returns
  empty results rather than a NotFound error.
- Fix make_unique<RestFileScanTaskStream> deduction failure by replacing
  brace-init {} with std::vector<std::string>{} for plan_task_tokens arg.
- Relax PlanTableScanResponse::Validate() to allow plan-id in failed
  responses, enabling the cancel-on-fail code path to be reached.
- Update rest_json_serde_test.cc: remove FailedWithPlanId invalid case,
  add FailedWithPlanId roundtrip test.
- Add iceberg/manifest/manifest_entry.h and iceberg/constants.h includes
  to rest_table_scan_test.cc to fix DataFile and kInvalidSnapshotId access.
Override the two-arg PlanFiles so the snapshot range is resolved by
ResolvePlanFiles, which uses the branch head when UseBranch() is set and
validates explicit start/end snapshots against it. Return NotSupported
when there is no start snapshot, since the REST API requires both
start-snapshot-id and end-snapshot-id.
Override io() so callers read incremental scan tasks with the FileIO
built from storage credentials returned during planning, falling back
to the table's FileIO when none are returned.
… commit

ResolveSnapshotSchema() now honors use_snapshot_schema, so branch scans
use the current table schema while tags and time travel use the snapshot
schema, matching Java's SnapshotUtil.schemaFor(table, ref).

MakeTableFromCommitResponse() now goes through the same scan planning
mode resolution as LoadTable, so committing through a server-planning
table returns a RestTable instead of a plain Table.
REST scan builders now resolve IncludeColumnStats() columns, so
stats-fields is sent and unknown columns are rejected.

The plan request's select is derived from the scan's projected schema,
matching Java's RESTTableScan, so Project() is honored, filter columns
are included, and Select("*") expands to all columns.
FetchPlanningResult now returns the completed response instead of
resolving its plan tasks, so a SUBMITTED plan builds the same lazy
stream as an immediate COMPLETED one. The stream also queues nested
plan tasks returned by fetchScanTasks instead of resolving them
eagerly, matching Java's ScanTaskIterable.
REST scans fetch plan tasks serially and do not use an executor set via
PlanWith(). Document this, add a TODO for concurrent fetching, and test
that the executor is not used.
PlanTableScanResponseFromJson validates the response, so a failure
dropped the plan id before it could be cancelled. Read the plan id from
the raw JSON first and cancel the plan if parsing or validation fails.
Add rest-scan-planning.poll-timeout-ms (default 300000, must be
positive) and rest-scan-planning.poll-num-retries (default 10, must be
non-negative), matching Java. They are validated before the plan is
submitted, and polling no longer sleeps after the final attempt.
Main now aliases Result to ::iceberg::expected so the library builds in
C++20 compat mode; std::unexpected no longer converts to it.
Vended credentials from scan planning responses now get a
StorageCredentialProvider that refreshes through the LoadCredentials
endpoint with the planId query parameter, matching the table FileIO
provider and Java's plan-scoped credential refresh.
Restore the PlanTableScanResponse validation that allows plan-id only for
submitted or completed plans. The plan is still cancelled because the
plan-id is read from the raw response before parsing.
A completed plan response may omit plan-id. Refresh credentials table-scoped in that case instead of sending an empty planId query parameter.
The spec only defines storage-credentials on completed planning results, and Java ignores them on fetchScanTasks. Drop the field from FetchScanTasksResponse so later responses can no longer replace the plan's scan FileIO, and keep the scan FileIO as a plain member now that the stream no longer writes it.
Preserve residual filters, keep scan sessions alive, and centralize
plan cleanup. Share planning context and consolidate REST tests.
@wgtmac
wgtmac force-pushed the sandeepg-scan-plan-endpoint-for-rest-catalog-client-2-impl branch from e848b44 to b40f915 Compare October 14, 2026 06:14

@wgtmac wgtmac left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for working on this! I rebased onto main and fixed residual handling, plan cleanup, and auth-session lifetime. I also simplified the APIs and tests to reduce duplication.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants