benchmark-optimization-loop
Convert 'make it faster' requests into a bounded measured optimization loop — baseline first, generate one-hypothesis variants, benchmark each against a correctness gate, and promote the fastest safe variant with reproducible commands.
- Kind
- Skill
- Works with
- Claude Code, Codex
- Source
- affaan-m/ECC @ d3b8a3e90890
- Path
- skills/benchmark-optimization-loop/SKILL.md
- License
- MIT
- Jobs
- Automation, Implementation, Testing
InstallCopy it from the source repository, or sign in and let the assistant adapt it for your project.
# Benchmark Optimization Loop Use this skill to convert "make it 20x faster" or "try 50 recursive optimizations" into a bounded measured loop that can actually improve a system. ## Required Baseline Do not optimize until these exist: - the operation being optimized; - the correctness gate that must stay green; - the metric: wall time, p95 latency, rows/sec, cost/run, memory, error rate; - the current baseline; - the search budget: max variants, max time, max spend, max data impact. If the user asks for an unrealistic target, keep the ambition but make the loop bounded and measurable. ## Loop 1. Measure the baseline. 2. Identify bottlenecks from evidence. 3. Generate variants that test one hypothesis each. 4. Run variants with the same input shape. 5. Reject variants that fail correctness, safety, or reproducibility. 6. Promote the fastest safe variant. 7. Codify the winning path in a script, command, test, config, or doc. 8. Rerun the baseline and winner to confirm the delta. ## Variant Table Track variants like this: ```text Variant | Hypothesis | Command | Time | Correct? | Notes baseline | current path | npm run job | 120s | yes | stable …