Skip to content

evalgate list hardcodes the provider names instead of asking the registry #3

Description

@royalpinto007

What is wanted

evalgate list should ask the provider registry what is registered instead of printing a hardcoded string.

The problem

src/cli/index.ts, cmdList (lines 236-242):

const scorers = defaultScorerRegistry().list().sort();
console.log("Registered scorers:");
for (const s of scorers) console.log(`  - ${s}`);
console.log("\nRegistered providers: mock, openai, anthropic, groq, openrouter");

The scorer half is derived from the registry. The provider half is a string literal. It happens to be correct right now (it matches the five register calls in defaultRegistry, src/providers/registry.ts lines 54-60), but nothing keeps it correct, and it can never reflect a provider registered at runtime, which is exactly what registryFor does for --degrade (line 60).

The reason it was written this way is that ScorerRegistry has a list() method (src/scorers/registry.ts lines 98-101) and ProviderRegistry does not. ProviderRegistry has register, has, and get, and already reads [...this.factories.keys()] internally at line 37 to build its unknown-provider error message.

Steps

  1. Add a list(): string[] method to ProviderRegistry in src/providers/registry.ts, returning [...this.factories.keys()]. Mirror the docstring style of ScorerRegistry.list().
  2. Use it in cmdList in src/cli/index.ts so providers print in the same bulleted, sorted form as scorers. cmdList currently takes no arguments; passing the registry built by registryFor(args) would also make evalgate list --degrade honest, though a plain defaultRegistry(process.env) is acceptable if you would rather keep the signature.
  3. Optionally simplify line 37 of src/providers/registry.ts to reuse the new list().
  4. Add a small test, for example in tests/providers.test.ts, asserting defaultRegistry({}).list() contains all five built-ins and that a register("custom", ...) call shows up.

Self-contained, under an hour. Comment below to claim it; I usually reply within a day.

Metadata

Metadata

Assignees

No one assigned

    Labels

    claimedSomeone asked first and is working on thisenhancementNew feature or requestgood first issueGood for newcomers

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions