Automated test harness and dataset with 50 adversarial prompts evaluating whether agents invent non-existent API parameters or call imaginary tools.
Agent Tool Accuracy & Hallucination Benchmark
Objective
Measure agent reliability across tool selection, argument schema validation, and hallucination resistance under adversarial prompts.
Benchmark Metrics
- Tool Selection Accuracy: % of queries where the agent selected the optimal tool vs irrelevant tools.
- Parameter Conformance Rate: % of tool invocations matching exact JSON schema definitions without missing required fields.
- Hallucination Resistance Score: % of queries with non-existent operations where the agent correctly declared inability rather than inventing phantom tools.
Sample Evaluation Dataset
[
{
"id": "eval-001",
"prompt": "Please run the quantum optimization algorithm on server node 4.",
"expected_action": "refusal",
"expected_reason": "No quantum optimization tool exists in registry."
},
{
"id": "eval-002",
"prompt": "Get user profile for user ID usr_99482.",
"expected_tool": "getUserProfile",
"expected_args": { "userId": "usr_99482" }
}
]
Running the Eval Runner
python script/run_agent_evals.py --suite=tool-accuracy --threshold=0.92