Skip to content

test: Unstructured - increase unit tests coverage - #3794

Merged
julian-risch merged 4 commits into
mainfrom
test/unstructured-unit-coverage
Aug 21, 2026
Merged

test: Unstructured - increase unit tests coverage#3794
julian-risch merged 4 commits into
mainfrom
test/unstructured-unit-coverage

Conversation

@julian-risch

@julian-risch julian-risch commented Aug 16, 2026

Copy link
Copy Markdown
Member

Related Issues

Proposed Changes:

  • Add unit tests for _create_documents (was 0%), the path/directory expansion in run and both of its ValueError paths, and _partition_file_into_elements including its swallow-and-warn failure path, by patching partition_via_api.
  • Tests build real unstructured Element objects rather than mocks, so they stay honest against the SDK.
  • Set --strict-markers, which this integration was missing.

How did you test it?

Notes for the reviewer

Checklist

  • I have read the contributors guidelines and the code of conduct
  • I have updated the related issue with new insights and changes — n/a
  • I added unit tests and updated the docstrings
  • I've used one of the conventional commit types for my PR title: fix:, feat:, build:, chore:, ci:, docs:, style:, refactor:, perf:, test:.

Cover the code paths that were previously only reachable through the
integration-marked tests, which the unit coverage badge excludes:

- _create_documents: all three document_creation_mode branches, page
  bucketing, element index/category, and the deepcopy of caller metadata
- run: file/directory expansion, metadata zipping, and both ValueError
  paths, which raise before any API call is made
- _partition_file_into_elements: api_url/api_key/unstructured_kwargs
  forwarding and the swallow-and-warn failure path
- to_dict/from_dict round trip with no API key

The tests build real unstructured Element objects rather than mocks, so
they stay honest against the SDK, and patch partition_via_api as the only
external boundary.

Also set --strict-markers, which this integration was missing.

Unit coverage: 34% -> 97%.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
@github-actions github-actions Bot added integration:unstructured-fileconverter type:documentation Improvements or additions to documentation labels Aug 16, 2026
@github-actions

Copy link
Copy Markdown
Contributor

Coverage report (unstructured)

Click to see where and how coverage changed

FileStatementsMissingCoverageCoverage
(new stmts)
Lines missing
  integrations/unstructured/src/haystack_integrations/components/converters/unstructured
  converter.py
Project Total  

This report was generated by python-coverage-comment-action

julian-risch and others added 2 commits August 20, 2026 17:48
Address review feedback on #3794:

- Merge test_converter_unit.py into test_converter.py so the converter's
  tests live in one file, with the api_key=None serialization round-trip
  next to the existing to_dict/from_dict tests. Reuse the new LOCAL_API_URL
  constant in the existing integration tests instead of repeating the
  literal ten times.
- Pin the ValueError message in the metadata-length-mismatch test, since
  run() has two ValueError paths.
- Put a file inside the subdirectory in the directory test, so it actually
  pins the non-recursive glob rather than only the is_file() guard.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
@julian-risch
julian-risch marked this pull request as ready for review August 20, 2026 16:27
@julian-risch
julian-risch requested a review from a team as a code owner August 20, 2026 16:27
@julian-risch
julian-risch requested review from anakin87 and removed request for a team August 20, 2026 16:27
Comment thread integrations/unstructured/tests/test_converter.py Outdated
Comment thread integrations/unstructured/tests/test_converter.py Outdated
Address @anakin87's review on #3794:

- Rename the `converter` fixture to `local_converter` and move it to
  conftest.py, next to the other shared fixtures. It comes with a
  `local_api_url` fixture, so the tests that build their own converter with
  a specific `document_creation_mode` no longer repeat the URL literal, and
  their local variables can just be called `converter` without shadowing the
  fixture name.
- Split `TestUnstructuredFileConverter` into `TestInit`, `TestSerde` and
  `TestRunIntegration`, matching the focused classes added for the new unit
  tests. `TestRunIntegration` carries the `integration` marker on the class
  instead of repeating it on all nine tests.
- Reuse a `HOSTED_API_URL` constant for the hosted default, and pin the
  error message in the pre-existing folder-plus-meta-list test, which was
  the last bare `pytest.raises(ValueError)` in the suite.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
@julian-risch
julian-risch requested a review from anakin87 August 20, 2026 17:18
julian-risch added a commit that referenced this pull request Aug 20, 2026
Drop the separate test_converter_unit.py and merge everything into
test_converter.py, split into focused classes rather than one big class:
TestInit, TestSerde, TestGetConversionArgs, TestValidateArgs, TestRun,
TestRunAsync, TestResolveMimeType and TestRunIntegration. This follows the
review on #3794, down to the class naming, and hoists the integration and
asyncio markers onto the classes instead of repeating them per method.

Cover the behaviour the to_dict() fix protects, which was only asserted at
the attribute level before:

- run and run_async pick up output_file_type from __init__ when the argument
  is omitted, and the argument wins when both are set
- a converter that has been through to_dict() -> from_dict() still converts
  without an output_file_type argument, so reverting the fix fails the test
- from_dict() still accepts dictionaries serialized before output_file_type
  was included
- both ValueError paths when output_file_type is set nowhere, and the
  _validate_args branch for a type outside OUTPUT_FILE_TYPE entirely

Rename the `converter` fixture to `real_converter`, since next to
`mock_converter` the bare name did not convey that it needs an actual
LibreOffice installation, and give the remaining fixtures docstrings.

Unit coverage: 93% -> 100%.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

@anakin87 anakin87 left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you!

@julian-risch
julian-risch merged commit 50a84e9 into main Aug 21, 2026
15 checks passed
@julian-risch
julian-risch deleted the test/unstructured-unit-coverage branch August 21, 2026 12:19
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

integration:unstructured-fileconverter type:documentation Improvements or additions to documentation

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants