[CoronRing]

Summary

Closes #1690.

S3Loader, GCSLoader and AzureBlobLoader decoded every object with the text encoding, including objects they had already tagged DocumentType.PDF. A real PDF raised UnicodeDecodeError and stopped the whole load, and any other binary file (an image, a .docx) did the same.

• PDFs are parsed, not decoded. The page extraction in PyPDFLoader._stream_file moves into a module-level stream_pdf() in pdf_loader.py, which reads either a path or bytes. PyPDFLoader and the cloud loaders both call it, so a PDF in a bucket produces the same documents and metadata as the same file on disk, plus the provider metadata (bucket, key, ...). Matching on .pdf ignores case.
• New pdf_breakdown_strategy option on the three cloud loaders, with the same values and default ("page") as PyPDFLoader.breakdown_strategy.
• pypdf stays optional. Text-only loads don't need the pdf extra. If any listed key is a PDF and pypdf is missing, the load raises an ImportError with the install hint before any object is downloaded, so a store is never left half-ingested.
• Undecodable objects: a .txt, .md, .markdown, .csv, .json or .jsonl object that is not valid text in encoding raises a ValueError naming the object and encoding=, since that means the encoding is wrong. Anything else that is not valid text (images, Office files, extension-less binaries) is skipped with a warning.
• One shared loop. The three loaders now subclass ObjectStoreLoader in cloud/_common.py, which owns option validation, astream, and the parse-or-decode step. Each provider only lists keys, downloads bytes, and names its source and metadata.
• Off the event loop: text decoding and the document strategy's page extraction now run in worker threads, like the other strategies.
• Shared validation: validate_breakdown_strategy in _pdf_options.py (no pypdf import) now takes the allowed values, and PyPDFOCRLoader and GLMOCRLoader use it too.
• Docs: the three storage pages describe PDF parsing, the PDF metadata keys and the decode behaviour, and module_packages.md notes that the cloud extras need [pdf] to parse PDFs.

Type of change

• Bug fix
• Feature
• Breaking change
• Docs
• Refactor / chore / build / tests

Checklist

• Lint & format pass (ruff check . && ruff format .)
• Tests added/updated and pass locally (pytest tests)
• Docs updated if user-facing behavior changed
• Breaking changes include migration notes

Notes

Depends on #1687 for page ids. Every page or paragraph of one PDF still shares a source, so it shares a Document.id, and the runtime keeps only the last one. That bug is #1648, already on main for PyPDFLoader, and #1687 fixes it by putting the page into source (path#page=N). The plan is to merge #1687 first, then rebase this PR onto it so stream_pdf carries that scheme and cloud PDFs get it too.

Tests: test_binary_objects.py runs every case against all three loaders: page, paragraph and document splitting of a real PDF, empty pages skipped, metadata merged, an uppercase .PDF key, a corrupt PDF (raises), an invalid strategy rejected, PDF and text objects kept in order, a PNG skipped with a warning, a .txt in the wrong encoding raising, a custom encoding=, and pypdf unavailable (text still loads; a PDF raises before anything is yielded). The PDFs come from a small make_pdf() builder in the cloud conftest.py, so the tests run the real pypdf path. All 631 retrieval unit tests pass, and mkdocs build --strict passes.

Behaviour changes:

• A PDF in a bucket now yields one Document per non-empty page by default, where it used to raise. Pass pdf_breakdown_strategy="document" for one Document per object.
• A non-text object that used to raise UnicodeDecodeError is now skipped with a warning, unless it has a known text extension, which raises a ValueError instead.
• Invalid breakdown_strategy errors from the PDF and OCR loaders now read ... must be one of 'page', 'paragraph', 'document', got ....

Left out:

• An encrypted or corrupt PDF still raises from pypdf and stops the load, as it does in PyPDFLoader. Containing one document's failure is [09] [Bug] [Retrieval] One bad document aborts the whole ingest run #1654.
• PDFs are recognised by extension only. An extension-less PDF is not valid text, so it is skipped with the warning.
• Other binary formats (images, Office files) are skipped, not loaded.

[Pooria90]

Thanks Guan. I just have two blocking comments on the pdf_loader. They are carried over from previous code; I appreciate you fixing them.

[CoronRing]

@Pooria90 For the record, following our chat: the page-id fix is out of this PR. It's #1648's bug, and #1687 already fixes it by putting the page into source (path#page=N), the same pattern the CSV/JSON/HF loaders use. That scheme also avoids a trap with an explicit id: the re-ingest skip check matches on source + content hash, so two identical pages in one PDF would wrongly skip the second. Plan is to merge #1687 first and rebase this onto it, so cloud PDFs pick up the same ids through stream_pdf.

Your other two points are in: the document strategy extracts pages via asyncio.to_thread, and the tests now hide pypdf itself and cover paragraph, an uppercase .PDF key and a corrupt PDF. The description is updated with the full scope.

Thanks for the sharp eyes! (๑•̀ㅂ•́)و✧