AI governance and observability
Evaluation, monitoring, safety, and governance for AI systems.
14 tools. Adoption level is the effort to run it well, not a quality score. Limitations are sourced; a blank means none documented yet.
| Tool | Best for | Level | Pricing | Key documented limitation |
|---|---|---|---|---|
| Arize Phoenix | Teams that need to evaluate LLM responses for quality and safety. | L3 Integrated | Open Source + Enterprise Cloud pricing. | None documented yet |
| Arthur AI | Large enterprises needing a secure 'operating system' for diverse AI model deployments. | L3 Integrated | Enterprise custom pricing. | None documented yet |
| Braintrust | Scoring agent quality before you widen access. | L4 Orchestrated | $$ | None documented yet |
| Helicone | Developers building production-ready LLM applications with integrated caching and rate limiting. | L3 Integrated | Free tier + Pro tier at $20/mo. | None documented yet |
| Langfuse | Tracing and evaluating agent behavior. | L4 Orchestrated | $ | None documented yet |
| LangSmith | MLOps and Engineering teams monitoring model performance in production. | L3 Integrated | Free + $39/mo + Enterprise | None documented yet |
| Lasso Security | Security and compliance teams monitoring LLM interactions for sensitive data leakage. | L4 Orchestrated | Enterprise pricing via contact. | None documented yet |
| Nebuly | Optimizing AI-driven user experiences | L2 Assisted | Not published | None documented yet |
| PromptHub | Streamlining prompt engineering and management for AI teams | L2 Assisted | Not published | None documented yet |
| PromptLayer | Product managers and engineering teams needing precise control over LLM prompts. | L3 Integrated | Free tier + Pro at $39/mo. | None documented yet |
| Robust Intelligence | Security and DevOps teams needing to secure AI-generated code and LLM integrations. | L4 Orchestrated | Contact sales | None documented yet |
| Scale AI | Enterprises needing high-quality, human-in-the-loop training data for RLHF. | L2 Assisted | Usage-based + Enterprise / Contact sales | None documented yet |
| Vera | Information security teams managing AI risk and model shadow IT. | L4 Orchestrated | Free trial + Contact sales for enterprise tiers | None documented yet |
| Weights & Biases | ML teams requiring enterprise-grade experimental tracking and model management. | L3 Integrated | Free for individuals + Enterprise custom pricing. | None documented yet |