Repository navigation
Conversation
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Which issue does this PR close?
No linked issue. This draft proposes native 3FS data I/O for review.
Rationale for this change
Using
Fson a 3FS mount sends file data through the regular FUSE read/write path. Add an opt-in service that uses 3FS's USRBIO shared-memory interface for data I/O while keeping filesystem metadata operations on the mount.What changes are included in this PR?
opendal-service-hf3fs, theservices-hf3fsfeature, and service registration.Fs.ubuntu-22.04, with a cached, pinned 3FS build.Validation: the existing behavior workflow passed on a GitHub-hosted Ubuntu runner using the implementation and fixture from this draft: 5 focused USRBIO tests and 116 behavior tests passed against real 3FS. The validation ran from a separate probe branch. Focused Clippy, formatting, and 12 behavior-planner tests also passed.
Are there any user-facing changes?
Yes. Linux users can enable
services-hf3fsand constructHf3fswith an existing 3FS directory and a compatiblelibhf3fs_api_shared.so. Building OpenDAL does not require the 3FS SDK. Running the service requires a real 3FS deployment and its FUSE client.Reads avoid a copy while shared-memory slots are available; retained buffers trigger a copy fallback to prevent pool exhaustion. Writes are not atomic. USRBIO has no cancellation fence, so fatal driver failures can retain in-flight resources until process exit. These constraints are documented.
The CI fixture selects Ubuntu 22.04 because the tested Ubuntu 24.04 Azure kernel lacks RXE support. The initial 3FS build is expensive; subsequent jobs restore it from cache.
Breaking changes
None.
AI Usage Statement