LLM-Ops + Observability

Langfuse Observability

LLM-Tracing, Eval, Prompt-Management

Sehe jeden Call, jeden Token, jeden Cent.
Zum Tool UI → Alle Tools
Beschreibung

Langfuse traced jeden LLM-Call der Plattform: Prompt-Input, Response-Output, Latency, Token-Counts, Cost in EUR/USD, User, Session, Feedback-Score. Prompt-Versioning + A/B-Testing. Evaluations gegen Golden-Sets. Alerts bei Anomalien (Cost-Spike, Latency-Drift). Integration über LiteLLM-Hooks + nativen Langfuse-SDKs in Cowork-AI, NemoClaw, OpenClaw.

Features & Capabilities · 8 Punkte
F01
Complete LLM-Trace pro Call: Prompt, Response, Tokens, Cost, Latency
F02
Prompt-Management mit Git-like Versioning + Rollback
F03
Evaluations: User-Feedback, Model-Graded, Custom-Scoring
F04
Datasets: Golden-Sets für Regression-Tests
F05
Sessions + Users: Multi-Turn-Conversations zusammenfassen
F06
Cost-Dashboards pro User, Project, Model, Tag
F07
OpenTelemetry-Export zu Grafana/Tempo
F08
PostHog-Integration für Product-Analytics
API & Endpoints
UI
https://trace.ben-e-fit.ai
Tracing-API
https://trace.ben-e-fit.ai/api/public/ingestion
Metrics
https://trace.ben-e-fit.ai/api/public/metrics
Use-Cases · 4 Beispiele

Wie das Tool real genutzt wird.

01
CFO-View: Welche Custom-Models verursachen am meisten Cost? Welche User?
02
Quality-Review: Welche Sonnet-Antworten bekamen 1-Stern-Feedback?
03
Prompt-Engineering: A/B-Test SystemPrompt-v3 vs v4 über 200 Eval-Cases
04
Anomalie-Detection: Pruefe Latency-Drift bei Opus -> Alert in Slack
Tech-Stack
Next.js Postgres ClickHouse Redis
Compliance & Governance
DSGVO Art. 5/30
EU-AI-Act Art. 12 (Logging)
Audit-Trail