Opik

Opik is an open-source AI observability platform designed to help developers debug, evaluate, and monitor Large Language Model (LLM) applications, RAG systems, and agentic workflows throughout their entire development lifecycle. LLM Tracing & Observability – Log and analyze traces and spans to understand each step of your LLM application's execution, from development through production Evaluation Metrics – Define custom evaluation metrics or use pre-configured ones to benchmark LLM performance, including built-in LLM judges for hallucination detection and factuality checks Automated Prompt & Agent Optimization – Leverage four powerful optimizers (Few-shot Bayesian, MIPRO, evolutionary, and MetaPrompt) to automatically optimize prompts and agents based on your evaluation metrics Safety & Guardrails – Screen inputs and outputs to detect and redact PII, competitor mentions, and off-topic content using built-in models or third-party guardrail libraries CI/CD Integration – Build comprehensive test suites with PyTest-based LLM unit tests to evaluate pipelines on every deployment Production Monitoring – Track and analyze production traces to identify issues and generate new datasets for continuous improvement Opik is fully open-source and available for free on GitHub, with an enterprise-ready version available for scalable, compliance-focused deployments. The complete LLM evaluation feature set is included in the free, open-source version.