feature/implement the medexqa

[chakravarthik27]

prompt:

prompt = """
You are a medical expert specializing in clinical reasoning and medical knowledge.

You will be given:
1. A medical multiple-choice question.
2. Four answer options labeled A, B, C, and D.

Your task is to:
- Select the single best answer based on established medical knowledge.
- Return only the corresponding option letter.

Rules:
- Output exactly one uppercase letter: A, B, C, or D.
- Do not provide any explanation, reasoning, punctuation, or additional text.
- If the question contains placeholders such as {question} or {options}, treat them as input variables that will be replaced before execution.

Example:

Question:
What is the most common cause of hypothyroidism in the United States?

Options:
A) Iodine deficiency
B) Hashimoto's thyroiditis
C) Graves' disease
D) Thyroidectomy

Output:
B

Now answer the following question.

Question:
{question}

Options:
{options}

Output:

"""

Harness Setup:

from langtest import Harness

harness = Harness(
    task="question-answering",
    model={
        "model": "gpt-4.1-mini",
        "hub": "openai"
    },
    data={
        'data_source': 'MedExQA',
        'subset': 'biomedical_engineer'
    },
    config={
        "model_parameters": {
            "user_prompt": prompt,
        },
        "tests": {
            "defaults": {
                "min_pass_rate": 0.8
            },
            "robustness": {
                "add_ocr_typo": {"min_pass_rate": 0.8},
                "uppercase": {"min_pass_rate": 0.8},
                "add_speech_to_text_typo": {"min_pass_rate": 0.8}
            }
        }
    })

harness.generate().run().report()

[blidiselalin]

Looks good