Skip to content

feat: add evaluation tables and data to test skills against updates #104

Description

@ryzizub

Description

Add evaluation tables and reference data so this plugin's skills can be tested against Claude Code updates. Today there is no structured way to verify a skill still behaves correctly after a Claude Code change. Define eval cases (inputs and expected outcomes) per skill so regressions are caught automatically rather than discovered in use.

Requirements

  • Define an eval data format (input scenario, expected behavior/output, pass criteria) reusable across skills.
  • Store eval tables/data co-located with each skill or under a shared evals/ directory.
  • Seed eval cases for the core skills (bloc, testing, navigation, layered-architecture, create-project).
  • Provide a runner that executes the evals and reports pass/fail.
  • Document how to add a new eval case when adding or changing a skill.

Additional Context

Consumed by the weekly changelog-check bot: when a skill is flagged as potentially impacted, its eval table is the gate for confirming it still works. See #103.

Metadata

Metadata

Assignees

No one assigned

    Labels

    featureA new feature or request

    Type

    No type

    Fields

    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions