Skip to content

fix(graphs): repair CodeGeneratorGraph HTML state and reference comparison - #1157

Closed
DRAKMANXP wants to merge 2 commits into
ScrapeGraphAI:pre/betafrom
DRAKMANXP:fix/code-generator-state-and-reference
Closed

fix(graphs): repair CodeGeneratorGraph HTML state and reference comparison#1157
DRAKMANXP wants to merge 2 commits into
ScrapeGraphAI:pre/betafrom
DRAKMANXP:fix/code-generator-state-and-reference

Conversation

@DRAKMANXP

@DRAKMANXP DRAKMANXP commented Sep 22, 2026 •
edited
Loading

Copy link
Copy Markdown

CodeGeneratorGraph.run() cannot reach code generation because both HTML consumers expect original_html, but no node writes that state key. Once that is repaired, a list returned as the reference answer raises TypeError in semantic_comparison() because it is unpacked with **.

The official OpenAI example constructs this graph and calls run() directly, so it uses the affected path too.

Reproduction

On the base commit, this reaches ValueError: Error parsing input keys for HtmlAnalyzer without a provider call:

from langchain_core.language_models.fake_chat_models import FakeListChatModel
from pydantic import BaseModel
from scrapegraphai.graphs import CodeGeneratorGraph

class Projects(BaseModel):
    projects: list[dict[str, str]]

graph = CodeGeneratorGraph(
    prompt="List the project titles.",
    source="<html><body><h1>Example project</h1></body></html>",
    schema=Projects,
    config={"llm": {
        "model_instance": FakeListChatModel(responses=[
            '{"projects": [{"title": "Example project"}]}',
            "Extract project titles.",
        ]),
        "model_tokens": 8192,
    }},
)
graph.run()

The second failure can be reproduced independently on that graph:

graph.graph.nodes[-1].semantic_comparison(
    {"projects": [{"title": "Example project"}]},
    [{"title": "Example project"}],
)
# TypeError: argument after ** must be a mapping, not list

GenerateAnswerNode uses JsonOutputParser(pydantic_object=schema), which supplies format instructions but does not validate the returned JSON against the schema. A list or a nonconforming object can therefore reach this method.

Changes

  • Read the existing doc state in both HTML consumers. Enable FetchNode's existing script_creator option for this graph so OpenAI/forced conversion does not replace the HTML with Markdown. The shared fetch node is unchanged.
  • Validate reference answers with Pydantic v2. When a reference does not conform, preserve its complete JSON for the existing semantic comparison instead of unpacking it or dropping elements. Restrict the dictionary equality shortcut to dictionaries. Generated-code output still goes through its existing schema validation.
  • Add offline tests covering run() with local HTML and a mocked URL loader, OpenAI and forced-conversion paths, valid reference objects, lists, and schema-invalid reference objects. The run tests execute the fixed test-generated extractor and its validation loops.

Validation

  • uv run --frozen --with pytest-timeout pytest tests/test_code_generator_graph.py: 11 passed. The initial seven-case regression set on the base commit produced 6 failed / 1 passed, reproducing the missing state and reference-type failures before the fixes.
  • Ruff passes for all three changed files; Black and isort pass for the new test file; git diff --check passes.
  • UV_FROZEN=true make lint still fails with the same six pre-existing F401 errors in batch_generate_answer_node.py and utils/batch_api.py, reproduced before the patch.
  • The existing tests/test_generate_answer_node.py has three pre-existing failures / five passes on the base commit (test_execute_multiple_chunks, test_execute_merge_json_decode_error, test_init_chat_ollama_format). No changes to that node or its tests.
  • Python 3.13.12. --with pytest-timeout supplies the plugin required by the existing strict pytest configuration without changing project dependencies or the lockfile. No live LLM/browser integration tests, full suite, mypy, or all-files pre-commit run.

Based on pre/beta, in two commits, one per defect.

DKM

@DRAKMANXP
fix(graphs): preserve HTML for CodeGeneratorGraph consumers
20629e5 
Use the existing doc state and disable HTML-to-Markdown conversion for this graph. Cover run() with local HTML and a mocked URL loader, including force mode.

DKM Gaming
@DRAKMANXP
fix(nodes): compare unvalidated reference JSON without unpacking it
39c7d3d 
Validate matching references with Pydantic v2. Preserve other JSON for semantic comparison, and only use the dictionary equality shortcut for dictionary inputs. Add regression coverage for lists and schema-invalid objects.

DKM Gaming
@DRAKMANXP DRAKMANXP closed this Sep 22, 2026
@DRAKMANXP ChatGPT Codex Connector

DRAKMANXP commented Sep 22, 2026

Copy link
Copy Markdown
Author

DKM Gaming — Closing this PR pending end-to-end verification, including the full test suite. We will reopen it once that verification is complete.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

1 participant

@DRAKMANXP