Comprehensive evaluation harness to score agent refactoring quality across AST preservation, test suite coverage, zero API breaking changes, and performance benchmarking.
Automated Code Refactoring Safety & Equivalence Scorecard
Overview
Evaluating AI agent code refactoring requires measuring whether the refactored code maintains 100% behavioral equivalence while improving maintainability and performance.
Scorecard Metrics
| Metric | Weight | Description | Pass Target |
|---|---|---|---|
| Test Suite Pass Rate | 40% | Percentage of existing unit/integration tests passing without modifications | 100% |
| API Contract Preservation | 30% | Zero breaking changes to function signatures or exported interfaces | 100% |
| Code Complexity (Cyclomatic) | 15% | Reduction in cyclomatic complexity score | ≥ 20% reduction |
| Performance Latency | 15% | Benchmark runtime execution comparison | ≤ baseline latency |
Evaluation Runner Script
import { execSync } from "child_process";
export function evaluateRefactorSafety(): { score: number; status: "PASS" | "FAIL" } {
try {
execSync("npx tsc --noEmit");
execSync("npm run test");
return { score: 100, status: "PASS" };
} catch (error) {
return { score: 0, status: "FAIL" };
}
}