Skip to content

Setup zero-shot evaluation with EAI harness #4

Description

@slippylolo

Description

For zero-shot evaluation, we would ideally want to use the EAI evaluation harness. Two strategies are possible: (1) convert T5X Jax checkpoints to HF Transformers PyTorch checkpoints, or (2) connect the EAI harness to T5X.

We are currently thinking of going with (1), as it seems like an easier choice.

Action items

Metadata

Metadata

Assignees

Type

No type

Projects

No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions