Agent Eval Suite

Agent Eval Suite

Agent 评估套件，提供基准测试、A/B测试、性能回归检测和模拟环境测试。

功能特性

1. 基准测试框架

2. A/B 测试

3. 性能回归检测

4. 模拟环境测试

安装

npm install

使用方法

Benchmark

const { Benchmark } = require('./src');
const benchmark = new Benchmark({ iterations: 100 });
benchmark.addTest('task-completion', {
  execute: async () => await agent.completeTask(task)
});
const results = await benchmark.run();
console.log(results);

ABTester

const { ABTester } = require('./src');
const ab = new ABTester({ confidenceLevel: 0.95 });
ab.createExperiment('new-prompt', {
  control: async () => await oldPrompt(),
  treatment: async () => await newPrompt()
});
const result = await ab.run('new-prompt', { sampleSize: 200 });
console.log(result); // { winner: 'treatment', confidence: 0.97 }

RegressionDetector

const { RegressionDetector } = require('./src');
const detector = new RegressionDetector({ threshold: 0.1 });
detector.record('response-time', { version: 'v1.1.0', value: 1200 });
const regressions = detector.detect();
console.log(regressions);

测试

npm test

License

MIT