Skip to content

fix: don't modify the input dictionary in from_dict - #12420

Closed
Ayushhgit wants to merge 1 commit into
deepset-ai:mainfrom
Ayushhgit:fix-from-dict-not-mutate-input
Closed

fix: don't modify the input dictionary in from_dict#12420
Ayushhgit wants to merge 1 commit into
deepset-ai:mainfrom
Ayushhgit:fix-from-dict-not-mutate-input

Conversation

@Ayushhgit

Copy link
Copy Markdown
Contributor

Related Issues

Proposed Changes:

from_dict() deserializes in place, so it replaced values inside the dictionary the caller passed in with live objects: a callable path became the callable, a nested component dict became the component, "overwrite" became DuplicatePolicy.OVERWRITE. Afterwards the caller's dict could no longer be serialized, and a second from_dict() call on it failed with errors like 'function' object has no attribute 'split' or The value of 'chat_generator' is not a dictionary.

Two places in the codebase already treat this as the expected contract:

  • PipelineBase._from_dict_impl copies its input first — data_copy = _deepcopy_with_exceptions(data) # to prevent modification of original data
  • test_component_from_dict_does_not_mutate_input asserts it for the secrets default_from_dict deserializes

This PR extends the same contract to the hand-written from_dict implementations:

  • adds _copy_serialized_data to haystack/utils/deserialization.py, a thin wrapper over the existing _deepcopy_with_exceptions (so components, tools and toolsets that are already live objects are passed through, not copied)
  • calls it as the first statement of the 49 from_dict implementations that deserialize in place — components, tools, toolsets, hooks and SuperComponent

No public behavior other than the mutation changes: the objects returned by from_dict are the same as before, and the deserialization helpers themselves are untouched.

Document.from_dict and ChatMessage.from_dict needed no change, they already build a new dict.

How did you test it?

  • new test/test_from_dict_no_mutation.py: for one component per deserialization style (callable, tools, nested chat generator, chat messages, type, enum, filter policy, pipeline, tool) it asserts that from_dict leaves the input dict equal to what it was, that calling from_dict twice on the same data returns equal components, and that the data is still JSON-serializable afterwards. All 25 of these tests fail on main and pass with this change.
  • the same file sweeps component.registry and fails if any component that can be built without arguments mutates its input, so new components are covered automatically.
  • hatch run test:unit: 6212 passed. The 2 failures on my machine (test_run_with_bad_filter, test_from_file_path_guess_mime_type[spam.csv]) also fail on a clean checkout of main — they are a jq error-message and a Windows mimetypes difference, unrelated to this change.
  • ruff check / ruff format are clean. hatch run test:types reports 25 errors on my machine, all of them unused type: ignore comments and openai stub mismatches in the OpenAI generators and their tests, none in the files this PR touches — they come from the unpinned openai version locally.

Notes for the reviewer

  • The one-line guard is repeated rather than hidden in a decorator or in component_from_dict, because Cls.from_dict(data) is called directly by users and by the deserialization helpers, so a central hook would not cover it. If you would rather solve this centrally, I'm happy to rework it.
  • Pipeline.from_dict now copies twice: once for the whole pipeline, then once per component. The data is plain configuration and _deepcopy_with_exceptions skips objects that are expensive or impossible to copy, so this should not be noticeable, but say the word if you want the pipeline-level copy dropped instead.
  • The AI disclaimer below is per CONTRIBUTING.md. I reviewed every hunk and ran the tests and checks above locally.

This PR was generated with an AI assistant. I have reviewed the changes and run the relevant tests.

Checklist

  • I have read the contributors guidelines and the code of conduct.
  • I have updated the related issue with new insights and changes.
  • I have added unit tests and updated the docstrings.
  • I've used one of the conventional commit types for my PR title: fix:, feat:, build:, chore:, ci:, docs:, style:, refactor:, perf:, test: and added ! in case the PR includes breaking changes.
  • I have documented my code.
  • I have added a release note file, following the contributors guidelines.
  • I have run pre-commit hooks and fixed any issue.

`from_dict` deserialized in place, replacing values inside the caller's
dictionary with live objects: a callable path became the callable, a
nested component dict became the component, "overwrite" became
DuplicatePolicy.OVERWRITE. The dictionary was then no longer
serializable and a second `from_dict` call on it failed.

`Pipeline.from_dict` already copies its input to prevent this. Do the
same in the 49 `from_dict` implementations that deserialize in place,
through a `_copy_serialized_data` helper wrapping the existing
`_deepcopy_with_exceptions`, so components and tools that are already
live objects are passed through instead of copied.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
@Ayushhgit
Ayushhgit requested a review from a team as a code owner August 20, 2026 12:29
@Ayushhgit
Ayushhgit requested review from davidsbatista and removed request for a team August 20, 2026 12:29
@vercel

vercel Bot commented Aug 20, 2026

Copy link
Copy Markdown

@Ayushhgit is attempting to deploy a commit to the deepset Team on Vercel.

A member of the Team first needs to authorize it.

@sjrl

sjrl commented Aug 20, 2026

Copy link
Copy Markdown
Contributor

Closing, see #12418 (comment)

@sjrl sjrl closed this Aug 20, 2026
@github-actions

Copy link
Copy Markdown
Contributor

Coverage report

Click to see where and how coverage changed

FileStatementsMissingCoverageCoverage
(new stmts)
Lines missing
  haystack/components/builders
  chat_prompt_builder.py
  haystack/components/converters
  docx.py
  pdfminer.py
  pypdf.py
  haystack/components/embedders
  azure_document_embedder.py
  azure_text_embedder.py
  mock_document_embedder.py
  mock_text_embedder.py
  openai_document_embedder.py
  haystack/components/evaluators
  context_relevance.py
  faithfulness.py
  haystack/components/extractors
  llm_metadata_extractor.py
  haystack/components/extractors/image
  llm_document_content_extractor.py
  haystack/components/generators/chat
  fallback.py
  mock.py
  haystack/components/joiners
  branch.py
  list_joiner.py
  haystack/components/preprocessors
  document_preprocessor.py 199
  embedding_based_document_splitter.py 588-590
  haystack/components/rankers
  llm_ranker.py
  haystack/components/retrievers
  multi_retriever.py
  haystack/core/pipeline
  pipeline.py
  haystack/hooks/compaction
  hooks.py
  summarization.py
  haystack/hooks/human_in_the_loop
  strategies.py
Project Total  

The report is truncated to 25 files out of 52. To see the full report, please visit the workflow summary page.

This report was generated by python-coverage-comment-action

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

from_dict() modifies the dictionary it receives, breaking re-serialization and repeated deserialization

2 participants