Repository navigation
Conversation
Adds user_metadata and content_type to CopyOptions with matching capability flags. S3 sends x-amz-metadata-directive: REPLACE, so a same-key copy updates metadata in one zero-body CopyObject instead of a read and rewrite.
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Which issue does this PR close?
Closes #8430.
Rationale for this change
A metadata-only change, such as setting a file's mtime after upload, currently costs a full
read and rewrite. OpenDAL's copy cannot carry new metadata, and it refuses
from == to.S3 can make this change with one zero-body CopyObject that sends
x-amz-metadata-directive: REPLACE. The issue has the details.What changes are included in this PR?
CopyOptionsgetsuser_metadataandcontent_type, named as inWriteOptionsandComposeOptions.copy_withandcopier_withget matching builder methods, andOpCopygets matching getters.copy_with_user_metadataandcopy_with_content_type.CorrectnessCheckLayerreturnsUnsupportedwhen a service does not advertise them.Operator::copy*acceptsfrom == towhen either option is set.x-amz-metadata-directive: REPLACE,Content-Typeandx-amz-meta-*.OpCopyoptions round tripUnsupportedwithout the capability, and same-path copyallowed only with metadata
test_copy_with_user_metadata_in_place, gated oncopy_with_user_metadataTested against:
including the new one. The two failures,
test_copy_with_source_version_*, fail the sameway on
mainagainst the same bucket ("must have version") because the bucket has noversioning. A same-key
copy_withwith user metadata, content type and a source lengthhint sent one request and kept the ETag; the metadata changed and the content did not.
fixtures/s3image): full s3 behavior suite, 128 of 128.main(conditional writes, responseoverrides, special characters), none of them copy.
Are there any user-facing changes?
Yes. This PR adds two options, two builder methods and two capability flags; nothing
existing changes. The behavior is documented on
CopyOptions::user_metadata: when eitheroption is set, the destination does not inherit source metadata, and unset fields such as
Cache-Control take the service default.
Breaking changes
AI Usage Statement
claude-opus-5-5).reviewed the change and submits it.
compatible_services.md. All of them are advertised with the new flags.🤖 Generated with Claude Code