Skip to content

Fix tool schemas degrading to {"type": "object"} - #1487

Closed
CoronRing wants to merge 5 commits into
mainfrom
1458-tool-schema-annotation-resolution
Closed

CoronRing wants to merge 5 commits into
mainfrom
1458-tool-schema-annotation-resolution

Conversation

@CoronRing

Copy link
Copy Markdown
Collaborator

Closes #1458.

ParameterType.from_python_type is an identity lookup over 8 types with OBJECT as
the default. Anything it doesn't recognise silently becomes {"type": "object"}
which under from __future__ import annotations is everything. This PR also fix a major bug where, if any unrecognized class is resolved to an object, it cannot be corrected with a manual manifest.

Fixes

annotation before after
pattern: str under PEP 563 {"type": "object"} {"type": "string"}
Literal["content", "files"] {"type": "object"} {"type": "string", "enum": [...]}
List[str] | None {"anyOf": [{"type": "object"}]} {"type": "array", "items": {"type": "string"}}
correct ToolManifest under PEP 563 NodeCreationError accepted

Changes

  • llm/tools/annotations.py (new): resolved_signature() evaluates string/ForwardRef
    annotations. Fast-paths to plain inspect.signature when nothing is deferred, so no
    behaviour change for normal code. Unresolvable names warn and pass through.
  • llm/tools/parameter_handlers.py: UnionParameterHandler and
    SequenceParameterHandler now recurse through the handler chain so nested generics keep their structure.
  • llm/tools/tool.py: two call-site swaps
  • validation/node_creation/validation.py: resolve annotations before validating.
    Manifest type mismatches now warn instead of raising

For review

  • One output change for code that already worked: Optional[str] was
    {"anyOf": [{"type": "string"}]}, now {"type": "string"}. Single-branch anyOf is
    noise to a provider and optionality already rides on required. Nothing in-repo
    depended on the old shape.
  • _flatten_union_options is required, not cosmetic: UnionParameter.__init__ raises
    on a nested union, newly reachable now that union arms recurse (Union[Tuple[str, int], bool]).

@CoronRing
CoronRing marked this pull request as ready for review August 25, 2026 18:14
@Amir-R25 Amir-R25 added the area:core Core SDK such as nodes, sessions, flows, requests, etc label Sep 9, 2026

@Amir-R25 Amir-R25 left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks, good overall direction. A few edge cases and code styles need a little work.

A few were happening before but this PR is a good place to address.

I left a few comments:

  • Nested forward references such as list["Payload"] are still not fully resolved.
  • Tuple handling loses the tuple container and turns its element types into top-level union options.
  • I also had a question about changing all manifest type mismatches from errors to warnings, rather than only relaxing validation when inference is unreliable.

One general readability note: there are quite a few inline comments that narrate what the next line is doing. I’d trim those and keep comments for non-obvious constraints or reasoning. A few comments also overstate the behaviour, for example, “get_type_hints()’s exact behaviour” and `“nested generics keep their structure” so simplifying them would make the implementation easier to review and maintain.

Comment thread packages/railtracks/src/railtracks/llm/tools/annotations.py Outdated
Comment thread packages/railtracks/src/railtracks/llm/tools/annotations.py Outdated
Comment thread packages/railtracks/src/railtracks/llm/tools/parameter_handlers.py Outdated
Comment thread packages/railtracks/src/railtracks/llm/tools/parameter_handlers.py
Comment thread packages/railtracks/src/railtracks/validation/node_creation/validation.py Outdated
Comment thread packages/railtracks/src/railtracks/validation/node_creation/validation.py Outdated
Comment thread packages/railtracks/src/railtracks/llm/tools/parameter_handlers.py Outdated

@Aryan-Railtown Aryan-Railtown left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Amazing Work Guan!!
Just 1-2 minor things that i could find

Comment thread packages/railtracks/src/railtracks/llm/tools/parameter_handlers.py Outdated
Comment thread packages/railtracks/src/railtracks/validation/node_creation/validation.py Outdated
Comment thread packages/railtracks/src/railtracks/llm/tools/parameter_handlers.py Outdated
Resolve forward references at every nesting level, keep tuples as array
containers under their own parameter name, align Literal[..., None] with
Optional[Literal[...]], and reject manifests that contradict a reliable
annotation while staying silent where inference is unreliable.
Walking a model through model_fields follows resolved field types, so a model
that refers back to itself gives the walk an infinite annotation tree. Track
the models already on the current path and stop at a repeat.

Also covers the model_fields switch with tests: a dict field reached through a
PEP 563 model, a dict field reached through inheritance, and a dict nested
inside a self-referential model.
@CoronRing

Copy link
Copy Markdown
Collaborator Author

Closing in favour of #1580.

Thanks @Amir-R25 and @Aryan-Railtown for the thorough reviews. Every round found a real edge case, and that is what convinced me this is the wrong shape for the fix.

The core of this PR is a hand-rolled annotation resolver (annotations.py) that evaluates strings and rebuilds generic aliases by hand. typing.get_type_hints(func, include_extras=True) already handles every case in the table above, including the nested list["Payload"] Amir found, and it follows Python's own annotation changes (3.14's annotationlib) without us maintaining a parallel implementation. The one thing the custom resolver bought was per-parameter tolerance when a single name can't be resolved, and a clear warning covers that well enough. On top of that the PR grew well past #1458: the tuple rework, recursive union/sequence handling and the Optional[str] output change are each their own change.

#1580 takes the stdlib route at a fraction of the size. I've left a review there with what it still needs, mainly the manifest escape hatch and a warning when resolution fails.

Where the rest goes:

@CoronRing CoronRing closed this Sep 24, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area:core Core SDK such as nodes, sessions, flows, requests, etc

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Tool schemas silently degrade to {"type": "object"} under from __future__ import annotations

3 participants