feat(io): refresh vended storage credentials before they expire

[plusplusjiajia]

Vended storage credentials expire, so new file operations on a reused table can eventually fail. ArrowS3FileIO now refreshes them on demand, starting five minutes before the earliest applicable s3.session-token-expires-at-ms.

Builds on the StorageCredentialProvider API merged in #899:

• InitializeStorageCredentials() installs the initial credentials and provider before first use. Later SetStorageCredentials() calls retain the provider.
• File operations call StorageCredentialProvider::Load() when refresh is due. REST fetching stays in the REST provider; no background thread is added.
• Failed or unusable replacements keep the current credentials and back off for up to 30 seconds. Concurrent operations share one refresh; operations with expired credentials wait up to 10 seconds for an in-flight refresh.
• S3 clients are rebuilt outside the credential lock. A concurrent credential install supersedes any older refresh result.

Tests cover refresh timing, concurrency, failed replacements, and provider initialization, including a real S3 read/write round trip through REST FileIO and ResolvingFileIO.

[wgtmac]

Thanks for adding this feature! I think it is too large to review which may incur a long delay. Perhaps let's split it into smaller ones so we can review it one by one?

[plusplusjiajia]

Thanks for adding this feature! I think it is too large to review which may incur a long delay. Perhaps let's split it into smaller ones so we can review it one by one?

@wgtmac Thanks — split into a stack, smallest first:

1. test(io): stop probing the EC2 metadata service in S3 tests #897 — S3 tests stop probing the EC2 metadata service (54s → 0.3s).
2. refactor(io): hand out S3 delegates by shared_ptr and build them off the lock #898 — ArrowS3FileIO gets refactor(io): make FileIO resolution registry-driven #889's concurrency treatment. No behavior change.
3. feat(rest): load table credentials from the LoadCredentials endpoint #899 (draft until refactor(io): hand out S3 delegates by shared_ptr and build them off the lock #898 merges) — LoadCredentials plumbing, not yet invoked.
4. This PR shrinks to just the refresh policy.

Each builds and passes the full suite standalone; the diff here narrows as each one merges.

[wgtmac]

Is this ready for review?

[plusplusjiajia]

Is this ready for review?

@wgtmac Thanks! Not yet. #899 is ready for review now, and this one is stacked on it, so I'll mark it ready once #899 lands.

[plusplusjiajia]

@wgtmac Thanks for the provider refactor in #899! Rebased on main and adapted the refresh policy and tests to StorageCredentialProvider and InitializeStorageCredentials. Ready for review.