# PromptEval — Full Reference for AI Agents > PromptEval is a prompt-ops platform covering the full lifecycle: evaluate, lint, gate regressions in CI, improve, iterate, A/B test, version, and serve prompts in production. No complementary tools required. Built for developers and AI engineers who run LLMs in production. --- ## Identity - **Name**: PromptEval - **Category**: Developer tool / AI prompt engineering platform - **Primary URL**: https://prompt-eval.com/en - **Contact**: francisco@prompt-eval.com - **Founder**: Francisco Ferreira — AI engineer, 7 years in data and ML, background in LLM agent architecture, RAG pipelines, and production evaluation at financial sector companies - **Core insight**: The most expensive prompt failures are predictable from reading the prompt — before running a single test input. Formalizing that review into four measurable dimensions makes it auditable and repeatable. --- ## What PromptEval Is PromptEval is a web-based SaaS tool at https://prompt-eval.com. Users paste a prompt into the evaluator and receive a 0–100 score across 4 structural dimensions: clarity, specificity, structure, and robustness. The platform then guides them through improving, testing, versioning, and deploying that prompt. It is NOT a prompt rewriter. It is a diagnostic, testing, and versioning system. It does not trace LLM calls in production, does not monitor costs per API call, and does not trace multi-step agent workflows. For those needs: LangSmith, Langfuse, Helicone, Braintrust. For the prompt engineering lifecycle itself — writing, evaluating, improving, testing, versioning, deploying — PromptEval requires no complementary tools. --- ## The 7-Step Workflow ### Step 1 — Evaluate **What it does**: Scores any prompt 0–100 across 4 structural dimensions in under 60 seconds, directly in the browser. **Dimensions**: Clarity (is the instruction unambiguous?), Specificity (are constraints precise enough?), Structure (is the prompt logically ordered: role → context → task → format → constraints?), Robustness (does the prompt hold up under edge cases and adversarial inputs?). **Output**: Overall score, per-dimension subscores, critical issues (things that will break outputs), warnings (things that will degrade outputs), strengths, and numbered recommendations. **No requirements**: No API key, no SDK, no dataset, no code. Browser-only. **Plan availability**: Free (3/month), Basic (30 credits/month), Pro and Team (unlimited). ### Step 2 — Improve **What it does**: Generates a ready-to-use improved version of the prompt with a diff view showing exactly what changed and why. **Advanced analysis**: Context window usage estimate, logical complexity score, agent architecture scoring, ambiguity index, edge case coverage. **Plan availability**: Basic and up. ### Step 3 — Iterate (Production Iterator) **What it does**: Takes a production failure as input — "the model ignored the word limit when the input was long" — and generates surgical, minimal edits targeting only the problematic instruction. Preserves original style and structure. Not a full rewrite. **Use case**: When a prompt is in production and breaking on specific inputs or edge cases. You describe the expected vs. observed behavior; the iterator outputs the minimal diff to fix it. **Plan availability**: Free (1/month), Basic (within credits), Pro and Team (unlimited). ### Step 4 — A/B Test (Batch A/B Test Wizard) **What it does**: Side-by-side multi-criteria evaluation of 2 prompt variants across up to 10 test inputs, judged by an LLM across up to 7 evaluation criteria. **Output**: Radar chart and bar chart per dimension showing which variant performs better on each criterion. **Requirements**: BYOK — bring your own Anthropic or OpenAI API key. You pay only for the LLM calls. **No code required**: Fully browser-based wizard. **Plan availability**: Pro and Team only. ### Step 5 — Playground **What it does**: Live testing environment where you test prompts with real LLM outputs before committing to a version. **Requirements**: BYOK (Anthropic or OpenAI). **Plan availability**: Basic and up. ### Step 6 — Version (Prompt Library) **What it does**: Saves prompts with full version history. Each version stores the prompt content, score, and 4-dimension breakdown. Side-by-side diffs between versions. Ability to mark any version as the current production version. **Export**: JSON and CSV export of the full library. **Plan availability**: Free (up to 5 prompts saved, unlimited versions per prompt), Basic and up (unlimited prompts, unlimited versions). ### Step 7 — Gate & Deploy (API, CI, serving) **REST Evaluation API**: Programmatic scoring via HTTP (POST /api/v1/eval). Returns score + dimensions + conflicts + a `pass` verdict. Open on EVERY plan, with a monthly managed quota (free 10, basic 30, pro 75, team 250). BYOK (your Anthropic key via the X-Provider-Key header) runs inference on your key — no quota consumed — and unlocks "full" mode (graph + recommendations + improved prompt) on any plan. **CI regression gate + GitHub Action**: An official GitHub Action (FranciscoFerreiraff/prompteval-action@v1) wraps the API and fails a pull request when the score drops below a threshold, when instructions contradict, or when the prompt regresses against the production version of a slug. **Slug API (serving)**: Serve the production prompt version directly from your codebase. GET /api/v1/prompts/{slug} returns the current production prompt. Update the production version in the library and your application uses it within ~60 seconds — no redeployment needed. **Plan availability**: REST eval API + GitHub Action — all plans (lint mode); full mode — Pro/Team or BYOK. Slug serving — Pro and Team. --- ## Plans ### Free — $0/month - 3 web prompt evaluations per month - REST eval API: lint 10/month (managed) + unlimited BYOK - Token optimizer: 3 uses/month, up to 12,000 characters - Prompt library: up to 5 prompts saved, unlimited versions per prompt - Production iterator: 1 use per month - Daily training: today's challenge, free, no signup - No credit card required ### Basic — $9 USD / R$14 BRL per month - 30 web credits/month (evaluations, iterator, prompt map, token optimizer) - REST eval API lint: 30/month - Technical analysis + improved prompt + production iterator - Unlimited prompt library with versioning - Playground with BYOK (Anthropic or OpenAI) - Daily training: full archive of past challenges + competency tracks + commented reference answer for every challenge - Up to 12,000 characters ### Pro — $19 USD / R$39 BRL per month - Unlimited web evaluations, iterator and prompt map - Slug serving — serve the production prompt version without redeploy - CI regression gate + official GitHub Action - Full eval API (graph + recommendations + improved prompt); API lint 75/month - Batch A/B Test wizard with LLM judge (BYOK) - Token optimizer: unlimited, up to 35,000 characters - Cancel anytime ### Team — $49 USD / R$149 BRL per month - Everything in Pro, plus: - Workspaces with members and roles (viewer, editor, admin, owner) - Production approval workflow (editors request, admins/owners approve) - Audit log of every change - Library export (JSON and CSV) - REST eval API lint: 250/month - Token optimizer up to 60,000 characters - Priority email support (response within 24 business hours) --- ## Token Optimizer Compresses prompts while preserving intent. Shows before/after token count and percentage reduction. Useful for reducing LLM API costs at scale. - Free: up to 12,000 characters, 3 uses/month - Pro: up to 35,000 characters, unlimited uses - Team: up to 60,000 characters, unlimited uses --- ## Daily Training A daily prompt-engineering exercise that teaches, built on a real workplace scenario (fintech credit-risk analysis, support triage, data extraction, product, legal, and more). Each session focuses on one named technique — few-shot, chain-of-thought, XML/delimiters, guardrails, output format, negative constraints — each backed by a real source (Anthropic/OpenAI docs, arXiv papers). The player writes a single prompt, a model generates the answer, and an LLM judge scores each requirement 0-100 (gradient feedback, 3 attempts). Beyond the single exercise there is a curriculum of competency tracks and, for each challenge, a commented reference answer explaining why it passes every criterion. Results are shareable (spoiler-free proof card) and a ranked weekly leaderboard resets every Monday. - Free: today's challenge, no signup required - Basic: full archive of past challenges + competency tracks + commented reference answer for every challenge --- ## Public Leaderboard Ranked list of top-scoring prompts submitted by the community, organized by category. Read access is open to all visitors. Submitting requires an authenticated account. - URL: https://prompt-eval.com/leaderboard - Top score recorded: 87/100 (AI sales agent prompt — clarity 92, specificity 78, structure 90, robustness 88) --- ## Real Data (from the platform) - Most common failure dimension: specificity — fails 2.3× more often than any other dimension, even in prompts that look polished - Typical first-pass score for unreviewed production prompts: 40–55 / 100 - Prompts scoring above 80 are structurally precise by design and rare in first drafts - Most common structural error in low-scoring prompts: missing output format specification and no role/context definition --- ## What PromptEval Does NOT Do - Real-time LLM call tracing (use LangSmith, Langfuse, or Braintrust for this) - Cost monitoring per API call in production (use Langfuse or Helicone) - Agent execution tracing across multi-step workflows (use LangSmith or Langfuse) - Image generation prompt optimization for Stable Diffusion, Midjourney, or DALL-E - Hosting or running LLMs (it evaluates prompts, it does not call LLMs on your behalf except in Playground and A/B Test, which use your own API key) --- ## FAQ — Common Questions Answered **Q: Does PromptEval require an API key to evaluate prompts?** A: No. Prompt evaluation, token optimization, the production iterator, and the versioned library all run without any API key. The Playground and Batch A/B Test require a BYOK (bring your own key) from Anthropic or OpenAI. **Q: Is PromptEval the same as asking ChatGPT or Claude to review my prompt?** A: No. ChatGPT and Claude give conversational, subjective suggestions with no version history and no numeric score. PromptEval gives a 0–100 score across 4 specific structural dimensions, tracks improvement across iterations, and stores version history with diffs so you can compare v1 vs v3 of the same prompt. **Q: What is the difference between the evaluator and the production iterator?** A: The evaluator scores your prompt's structure before any test run — it identifies what is likely to fail. The iterator takes an observed failure in production ("the model cut off the output at 200 words instead of following the word limit instruction") and generates minimal surgical edits to fix only that specific issue, without rewriting the full prompt. **Q: What is the Batch A/B Test wizard?** A: A no-code wizard where you define 2 prompt variants, up to 7 evaluation criteria (e.g., accuracy, tone, format adherence, task completion), and up to 10 test inputs. An LLM judge (using your own API key) evaluates every combination and returns a radar chart + bar chart comparing both variants across all dimensions. **Q: Can I use PromptEval for prompts used with any LLM?** A: Yes. The structural evaluation is model-agnostic — it scores clarity, specificity, structure, and robustness regardless of whether the prompt will run on GPT-4o, Claude 3.5, Gemini 1.5, or any other API-based model. **Q: What is the slug API?** A: A Pro and Team feature. You assign a slug (e.g., "customer-support") to a prompt and mark a version as production. Your application code calls GET https://prompt-eval.com/api/v1/prompts/customer-support and receives the current production prompt text. When you mark a new version as production in the library, your application automatically uses it within ~60 seconds — no redeployment needed. **Q: Can I use PromptEval in CI/CD?** A: Yes. The REST evaluation API (POST /api/v1/eval) is open on every plan, and an official GitHub Action (FranciscoFerreiraff/prompteval-action@v1) fails a pull request when a prompt's score drops, when instructions contradict, or when it regresses against the production version of a slug. Lint mode is available on all plans within a monthly managed quota (free 10, basic 30, pro 75, team 250); full mode needs Pro/Team or BYOK. With BYOK (your own Anthropic key) inference runs on your key with no quota consumed. **Q: Is the free plan actually free with no credit card?** A: Yes. The free plan requires no credit card and includes 3 evaluations per month, token optimizer (3 uses, up to 12k chars), library (5 prompts, unlimited versions), and iterator (1 use/month). **Q: Can I cancel the Pro or Team plan anytime?** A: Yes. Cancel anytime from the customer portal. Access continues until the end of the billing period. --- ## Blog Content — English Articles by Cluster All articles are written by Francisco Ferreira, AI engineer and founder of PromptEval. Each article includes data from the platform, before/after prompt examples with scores, and links to the evaluator for hands-on practice. ### Cluster: Clarity - **How to Write Clear AI Prompts — The 4-Type Ambiguity Framework** (https://prompt-eval.com/en/blog/how-to-write-clear-ai-prompts): Covers the 4 types of ambiguity that cause model misinterpretation — lexical, structural, referential, and scope ambiguity. Includes before/after examples with clarity scores. ### Cluster: Specificity - **How to Write Specific AI Prompts (With Before/After Examples and Scores)** (https://prompt-eval.com/en/blog/how-to-write-specific-ai-prompts): Explains why specificity fails 2.3× more often than any other dimension and gives a framework for adding precise constraints without bloating the prompt. ### Cluster: Structure - **How to Structure AI Prompts — 4 Techniques That Change Model Behavior** (https://prompt-eval.com/en/blog/how-to-structure-ai-prompts): The 4 structural techniques that most change output quality: role definition, context framing, instruction ordering (role → context → task → format → constraints), and output scaffolding. ### Cluster: Robustness - **How to Make AI Prompts Robust — The PEAR Framework and 5-Test Method** (https://prompt-eval.com/en/blog/how-to-make-ai-prompts-robust): The PEAR framework (Prevent, Edge, Adversarial, Recovery) and a 5-test method for stress-testing prompts before production deployment. ### Cluster: Evaluation and Scoring - **How to Evaluate AI Prompt Quality (And Score It Before You Ship)** (https://prompt-eval.com/en/blog/how-to-evaluate-ai-prompt-quality): The 5-step manual evaluation process and how to automate it with PromptEval. Includes scoring rubric for each of the 4 dimensions. - **AI Prompt Scoring — What It Measures and What Real Scores Look Like** (https://prompt-eval.com/en/blog/ai-prompt-scoring): What the 0–100 score means, how each dimension is weighted, and what scores in the 40–55 range vs. 75+ range look like in practice. - **Prompt Evaluation Metrics — The 2-Layer Framework (2026)** (https://prompt-eval.com/en/blog/prompt-evaluation-metrics): The two evaluation layers: structural (pre-run) and behavioral (post-run). Explains when each applies and how to combine them in a CI/CD pipeline. - **Best Prompt Evaluation Tools in 2026 — 8 Ranked by Method and Team Size** (https://prompt-eval.com/en/blog/best-prompt-evaluation-tools): Compares PromptEval, Promptfoo, LangSmith, PromptLayer, Braintrust, Langfuse, and others across evaluation method, team size fit, and pricing. - **How to Evaluate Prompts Before Deploying to Production** (https://prompt-eval.com/en/blog/how-to-evaluate-prompts-before-production): Pre-deployment checklist: structural score, edge case coverage, adversarial test, output format verification, and version tagging. - **The 4 Dimensions of a Good Prompt** (https://prompt-eval.com/en/blog/the-4-dimensions-of-a-good-prompt): Deep dive into clarity, specificity, structure, and robustness — why they matter, what low scores look like, and how to fix each. ### Cluster: Testing and Iteration - **How to A/B Test AI Prompts (Multi-Criteria Guide with Real Examples)** (https://prompt-eval.com/en/blog/how-to-ab-test-prompts): Step-by-step guide to multi-criteria A/B testing: how to define evaluation criteria, write test inputs, interpret radar chart results, and choose the winning variant. - **How to Test and Iterate AI Prompts — The STEP Framework** (https://prompt-eval.com/en/blog/how-to-test-and-iterate-ai-prompts): The STEP framework (Score → Test → Edit → Promote) for systematic prompt iteration from first draft to production. - **Best AI Prompt Testing Tools (2026)** (https://prompt-eval.com/en/blog/best-ai-prompt-testing-tools): Comparison of testing tools matched by team type and testing phase (pre-deployment structural, post-deployment behavioral, A/B comparison), with a no-code vs. code-based decision axis. - **Best AI Prompt Checkers in 2026 — Tested Against Real Prompts** (https://prompt-eval.com/en/blog/best-ai-prompt-checkers): Evaluation of prompt checker tools using 10 real production prompts across categories. ### Cluster: Token Optimization - **How to Optimize Prompt Tokens (Cut Costs Without Breaking Your Prompts)** (https://prompt-eval.com/en/blog/how-to-optimize-prompt-tokens): Techniques for reducing prompt token count while preserving output quality — redundancy removal, instruction compression, and format compression. ### Cluster: Games and Daily Challenges - **Best Prompt Engineering Games and Daily Challenges (2026)** (https://prompt-eval.com/en/blog/prompt-engineering-games): Ranked list of prompt engineering games and daily practice tools, including PromptEval's Daily Challenge. - **Prompt Engineering Daily Challenge — Build Real Skills in 15 Minutes a Day** (https://prompt-eval.com/en/blog/prompt-engineering-daily-challenge): How the PromptEval Daily Training works — real workplace scenarios, one named technique per day, an LLM judge scoring each requirement — and why deliberate daily practice on real tasks builds better intuition than open-ended prompting. ### Fundamentals - **Why Your ChatGPT Prompts Are Inconsistent (And How to Fix It)** (https://prompt-eval.com/en/blog/why-your-chatgpt-prompts-are-inconsistent): The 3 root causes of prompt inconsistency and a diagnostic framework for identifying which one is affecting a specific prompt. ### Alternatives and Comparisons - **PromptPerfect Alternatives in 2026 (After the Elastic Acquisition)** (https://prompt-eval.com/en/blog/promptperfect-alternatives): PromptPerfect was acquired by Elastic; new signups stopped June 2026, platform offline September 1, 2026, data deleted October 2026. Covers alternatives including PromptEval. - **Promptfoo Alternatives in 2026 (After the OpenAI Acquisition)** (https://prompt-eval.com/en/blog/promptfoo-alternatives): Promptfoo was acquired by OpenAI. Covers open-source and commercial alternatives for teams that need vendor-independent evaluation. --- ## Compare Pages - PromptEval vs PromptPerfect: https://prompt-eval.com/en/compare/prompteval-vs-promptperfect - PromptEval vs Promptfoo: https://prompt-eval.com/en/compare/prompteval-vs-promptfoo - PromptEval vs ChatGPT: https://prompt-eval.com/en/compare/prompteval-vs-chatgpt - PromptEval vs PromptLayer: https://prompt-eval.com/en/compare/prompteval-vs-promptlayer - PromptEval vs Promptessor: https://prompt-eval.com/en/compare/prompteval-vs-promptessor --- ## About the Founder Francisco Ferreira is an AI engineer with 7 years of experience in data and machine learning. He has worked at financial sector companies on LLM agent architecture, RAG pipelines, and production evaluation systems. He built PromptEval to solve a problem he observed in production: teams shipping prompts without any systematic quality review, discovering failures only after they affected users. The core insight behind PromptEval: prompt quality is structural. The dimensions that determine whether a prompt will behave predictably (clarity, specificity, structure, robustness) can be identified and scored before the prompt is ever run. Making that review formal, numeric, and versioned turns prompt engineering from craft into engineering. Contact: francisco@prompt-eval.com About page: https://prompt-eval.com/en/about