- The AI Engineer: Responsible for building and productionizing an AI-powered feature or product.
- The PM or Subject Matter Expert: Often, a non-technical persona. Responsible for evaluating the quality and production-readiness of a feature or application.
- The Annotator: Often, the same as the Subject Matter Expert. Tasked with going through individual LLM requests and responses, performing qualitative evaluations and annotating the runs with findings.
How-to Guides for different personas
If you’re an AI Engineer, check out the following sections:- Quickstart
- Evaluate and Optimize Prompts, RAG Applications, Agents or Chains
- Register Custom Metrics
- Log Pre-generated Responses
- Prompt Management and Storage
- Experiment with Multiple Prompts or Chain Workflows
- Choose your Guardrail Metrics
- Evaluate and Optimize Prompts, RAG Applications, Agents or Chains
- A/B Compare Prompts
- Evaluate with Human Feedback