Skip to content

Chunked uploads#121

Open
petarjakopec wants to merge 3 commits into
3.0from
chunked-uploads
Open

Chunked uploads#121
petarjakopec wants to merge 3 commits into
3.0from
chunked-uploads

Conversation

@petarjakopec
Copy link
Copy Markdown

Enables chunk uploads for larger files.

CloudinaryApiGateway::upload() now injects the chunk_size option into the Cloudinary SDK's upload() call when the source is a local readable file above a configurable threshold.

The SDK internally routes the transfer through postLargeFileAsync() (multipart PUT with Content-Range headers) instead of postSingleChunkAsync().

In cloudinary/cloudinary_php 2.x, chunking is purely option-driven via the standard upload() method. Implementation and docs have been synced to this SDK contract.

Configuration

Two new optional config keys added:
YAML

netgen_remote_media:
cloudinary:
large_upload_threshold: 100000000 # bytes, default: 100 MB
upload_chunk_size: 20000000 # bytes, default: 20 MB

📊 Observability

log_requests: true now distinguishes between chunked and single-request uploads in var/log/cloudinary.log.
Plaintext

[API][FREE] upload("/tmp/foo") [size=314572800B, chunked=yes (threshold=100000000B, chunk_size=20000000B)] -> Cloudinary\Uploader::upload("/tmp/foo")
[API][FREE] upload("/tmp/bar") [size=485762B, chunked=no (threshold=100000000B, chunk_size=20000000B)] -> Cloudinary\Uploader::upload("/tmp/bar")
[API][FREE] upload("https://...") [size=external/unknown, chunked=no] -> Cloudinary\Uploader::upload("https://...")

📁 Files Touched

Gateways:

    CloudinaryApiGateway.php: Added shouldChunk() helper and option injection.

    MonologLoggedGateway.php: Enhanced logging visibility.

DI & Config:

    Configuration.php: Defined new integer nodes.

    NetgenRemoteMediaExtension.php: Parameter mapping.

    core.yaml: Service wiring.

Tests:

    12 new tests covering branching logic, logging, and DI wiring.

Docs:

    Updated INSTALL.md

⚠️ Backwards Compatibility

Zero breaking changes:

GatewayInterface and ProviderInterface signatures remain unchanged.

Gateway classes are @internal and public: false, so constructor changes won't affect consumers.

Existing behavior is preserved for files ≤ 100 MB.

✅ Test Plan

Static Analysis: composer fix, phpstan (lib + bundle + tests) — All Clean.

Unit Tests: composer test — 595 tests, 3643 assertions passed.

Smoke Test: Lowered threshold to 5MB in a consuming project; confirmed chunked=yes in logs.

Real-world: Successfully uploaded >100 MB file (verified with adjusted Nginx/PHP limits).

💡 Caveats (documented in INSTALL.md)

Chunk Size: Cloudinary requires non-final chunks to be ≥ 5 MB. Our default (20 MB) safely covers this.

Server Limits: Host Nginx/PHP limits (client_max_body_size, post_max_size, etc.) must be raised to handle large payloads.

Account Limits: Ensure the Cloudinary plan's raw upload cap is sufficient for your target file sizes.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant