Scorecard evaluating Faithfulness, Answer Relevance, Context Precision, and Context Recall for vector search and RAG pipelines.
RAG Retrieval & Grounding Evaluation Suite
Evaluation Dimensions
| Dimension | Definition | Target Threshold |
| :--- | :--- | :--- |
| Faithfulness | Does the answer contain only facts supported by retrieved chunks? | $ge 0.95$ |
| Answer Relevance | Does the generated answer directly address the user query? | $ge 0.90$ |
| Context Precision | Are the most relevant chunks ranked at the top of retrieval? | $ge 0.85$ |
| Context Recall | Did the retrieval step fetch all necessary ground-truth facts? | $ge 0.88$ |
Implementation with Ragas / Custom Judges
from ragas import evaluate
from ragas.metrics import faithfulness, answer_relevance, context_precision, context_recall
results = evaluate(
dataset=eval_dataset,
metrics=[faithfulness, answer_relevance, context_precision, context_recall]
)
print("RAG Performance Scorecard:", results)