SRE for AI agents: why uptime beats prompt engineering

LLM observability exploded in 2024–2025. LangSmith, Langfuse, Braintrust, Helicone — each optimizes a different slice of the AI development loop. Prompt versioning, eval datasets, A/B tests. Essential for iteration.

But production teams keep learning about failures the same way: a customer complains. The model call failed, the retry loop burned $40, and nothing paged anyone.

The reliability gap

SRE culture asks four questions: Is it up? Is it erroring? What does it cost? Who gets told? Generic APM tools weren't built for token-priced, streaming, provider-dependent workloads. Prompt tools weren't built for pager duty.

Puse sits in that gap. We trace every call — model, tokens, latency, cost, errors — and alert on regressions. Not "your prompt score dropped 2%," but "error rate hit 12% in the last 15 minutes."

What to monitor first

• Error rate by project (rolling 15m vs 24h baseline)
• Cost per hour (catch retry loops early)
• p95 latency (provider degradation)
• Top error messages (actionable context)

Start with one project, one Slack channel, two alert thresholds. You can add prompt evals later — but you can't un-churn a customer who hit a silent failure.