Accuracy with Given Answer

Example showing how to evaluate the accuracy of an Kern Agent's response with a given answer.

Create a Python file

1from typing import Optional
2
3from kern.eval.accuracy import AccuracyEval, AccuracyResult
4from kern.models.openai import OpenAIResponses
5
6evaluation = AccuracyEval(
7    name="Given Answer Evaluation",
8    model=OpenAIResponses(id="gpt-5.2"),
9    input="What is 10*5 then to the power of 2? do it step by step",
10    expected_output="2500",
11)
12result_with_given_answer: Optional[AccuracyResult] = evaluation.run_with_output(
13    output="2500", print_results=True
14)
15assert result_with_given_answer is not None and result_with_given_answer.avg_score >= 8

Set up your virtual environment

1uv venv --python 3.12
2source .venv/bin/activate

1uv venv --python 3.12
2.venv\Scripts\activate

Install dependencies

1uv pip install -U openai kern-ai

Export your OpenAI API key

1export OPENAI_API_KEY="your_openai_api_key_here"

1$Env:OPENAI_API_KEY="your_openai_api_key_here"

Run Agent

1python accuracy_with_given_answer.py