Grafana Labs Launches o11y-bench 1.1 For AI Evaluation

Sixty-four tasks across six areas now test if AI agents can actually handle system outages. Grafana Labs just pushed out version one-point-one of o11y-bench to grade how well models work with real telemetry. The benchmark runs inside a live Grafana stack and checks sixty-four distinct tasks. Most importantly, every single task runs three separate times to test for reliability. A control test using zero tools failed every single task on the board. That proves sounding smart is not enough to fix a broken dashboard. Fireworks also just added the benchmark to its Specialized Intelligence Index. Now teams can compare open and closed models on real incident workflows.