Arize Phoenix
Open-source AI observability platform from Arize AI ($70M Series C, 2024).
Datadog
Unified SaaS observability platform correlating metrics, distributed traces, logs, and synthetic tests across multi-cloud stacks — strongest when cross-service correlation and zero operational overhead outweigh the per-host cost.
Helicone
Open-source LLM observability platform and AI gateway. One-line integration. Combines monitoring, tracing, prompt management, semantic caching, and multi-provider routing in a single proxy layer.
Langfuse
Open-source LLM observability platform. The standard self-hosted choice for teams that want full data ownership.
LLM Observability
LLM observability platform comparison — Langfuse (best self-hosted, MIT, ClickHouse), LangSmith (LangChain shops), Arize Phoenix (ML+LLM unified) — plus cost gates and online eval patterns.
LLM Tracing with OpenTelemetry
OTel GenAI semantic conventions, manual and auto-instrumentation for Anthropic/LangChain, Langfuse native SDK patterns, cost tracking per trace, and Prometheus alerting thresholds.
LLMOps
LLMOps treats prompts as versioned production artifacts — a registry replaces hardcoded strings, eval gates block regressions, and A/B testing on real traffic replaces intuition-driven prompt changes.