P

AI Security Testing Tools

Promptfoo: AI Evaluation and Red-Teaming Software

promptfoo.dev · checked 15 Oct 2026

Open-source tooling for evaluating and red-teaming AI applications.

How it plugs in

—Public API
YesOpen source
YesFree tier

Signal on AI Security Testing Tools7.7#1 of 29

Promptfoo is a local-first CLI and library for testing prompts and AI applications. It places model outputs side by side, calculates accuracy, safety, and performance, and searches for problems including prompt injection and data exposure. Teams can work locally or self-host, automate checks in CI/CD, test browsers, and connect more than 60 model providers.

Promptfoo home page

Review

In a line

Promptfoo is an open-source CLI and library for evaluating and red-teaming applications built with large language models. Developers and enterprises use it while building AI software.

What it does

Prompts can be exercised against multiple model providers, with responses shown together in a web viewer and scored through assertion rules. Accuracy, safety, and performance are among the metrics it calculates.

Its red-team mode imitates user behavior to investigate prompt injection, jailbreaks, data exposure, business-rule breaches, and unsafe agent-tool operation. The project documents 157 plugins across security, compliance, datasets, trust and safety, brand, and custom testing categories.

Optional features include human review workflows, custom metrics, safety evaluations, model audits, code scanning, and browser interactions through Playwright with Chromium, Edge, Firefox, or WebKit.

How it connects

Promptfoo's catalog lists more than 60 providers, among them OpenAI, Anthropic, Google, and local systems such as Ollama. It also covers HTTP, JavaScript, Python, Ruby, shell, OpenAI Agents, Vertex, and other services.

Developers may create providers in JavaScript, TypeScript, Python, Ruby, or script files. Through templates, the HTTP provider can call arbitrary endpoints, reshape replies, authenticate, and process tool-calling formats.

Integrations include GitHub Actions, GitLab CI, Jenkins, Azure Pipelines, CircleCI, Bitbucket Pipelines, Splunk, Burp Suite, Google Sheets, and MCP. A Node API exposes evaluation, provider, assertion, cache, guardrail, and TypeScript interfaces, with stable and experimental areas.

The project uses the MIT license. Configuration files and provider credentials allow evaluations and security checks to run in CI/CD. A free viewer at promptfoo.app offers private organization links and self-hosted sharing endpoints.

Good fit

Promptfoo suits teams that need repeatable checks for AI prompts, model responses, and security risks. It works for local development, automated pipeline checks, and enterprise deployments.

Community users can run it locally or host it themselves. Enterprise and On-Premise options add managed or customer-infrastructure deployment, collaboration, monitoring, access controls, and support services.

Watch for

Running Promptfoo requires Node.js 22.22.0 or newer; its documentation points users toward Node.js 24 LTS. Windows, macOS, and Linux command-line use is available, with npm, npx, or Homebrew installation.

The Community plan allocates 10k red-team probes each month. In the pricing FAQ, a probe means one request made to the target during red-team testing.

Custom code, providers, transforms, hooks, plugins, and templates execute with the user's permissions and without sandboxing. Review those components before use. GDPR tests cover privacy and related risks, but they are not legal certification.

Pricing

Community has no charge and remains free indefinitely, offering 10k red-team probes per month, local or self-hosted operation, and community support.

Enterprise pricing is provided on request. This tier adds custom red-team limits, team collaboration, continuous monitoring, SSO and role-based access, API access, managed cloud deployment, priority support, and service-level guarantees.

On-Premise pricing is provided on request. It covers enterprise features on customer infrastructure, complete data isolation, a dedicated runner, and a deployment engineer.

Verdict

Choose Promptfoo if you want open-source, scriptable checks for prompts and AI application security, especially when your workflow already uses CI/CD or self-hosted systems. Its broad provider list and custom extension options cover varied model setups.

The free Community tier is a practical starting point, but review the Node.js requirement, command-line orientation, monthly probe allowance, and unsandboxed custom code. Buyers seeking managed cloud hosting, formal support commitments, or isolated enterprise infrastructure should request Enterprise or On-Premise pricing.

Our take

Promptfoo suits teams comparing model responses and conducting repeatable security checks while retaining local control. Its MIT license, broad provider coverage, CI/CD integrations, browser testing, and extensible APIs support varied AI projects. Red-team use on Community is capped at 10k probes monthly, and Node.js 22.22.0 or newer is needed. Managed hosting, collaboration controls, monitoring, or service guarantees require a custom-priced tier. Unsandboxed custom code and plugins remain the user's security responsibility.

  • Open-source MIT-licensed project
  • Supports more than 60 model providers
  • Free local or self-hosted Community plan
  • Evaluations and security checks can run in CI/CD
  • Includes browser testing through Playwright
  • Community plan has 10k red-team probes per month
  • Requires Node.js 22.22.0 or newer
  • Custom code runs without sandboxing
  • Enterprise and On-Premise prices are by request
  • Pick it ifTeams evaluating prompts across several AI providers; Organizations running AI security checks in CI/CD; Users needing local or self-hosted deployment
  • Skip it ifYou need a fully managed service without configuring Node.js; You require sandboxed execution for custom providers and plugins; You need pricing without contacting sales for enterprise features

Pricing

Community remains free with local or self-hosted use, 10k monthly red-team probes, and community support. Enterprise and On-Premise pricing is available on request, with added deployment, collaboration, monitoring, support, and infrastructure features.

  • CommunityFree Forever · 10k red-team probes/month · local or self-hosted · community supportFree
  • EnterpriseCustom · custom red-team limits · team collaboration · continuous monitoring · SSO/RBAC · API access · managed cloud · priority support and SLAOn request
  • On-PremiseCustom · enterprise features · customer infrastructure · complete data isolation · dedicated runner · deployment engineerOn request

As promptfoo.dev lists them, checked 15 Oct 2026.

Questions

Is Promptfoo open source?

Yes. The project uses the MIT license and supports local or self-hosted operation.

Which model providers does Promptfoo support?

Its guide lists more than 60 providers, including OpenAI, Anthropic, Google, and local models such as Ollama.

What does a red-team probe mean?

For pricing purposes, a probe is one request sent to the target during red-team testing.

Does Promptfoo have a free plan?

Yes. The Community plan stays free and includes 10k monthly red-team probes, local or self-hosted use, and community support.

What does Promptfoo require to run?

Node.js 22.22.0 or later is needed, while Node.js 24 LTS is recommended. Command-line use works on Windows, macOS, and Linux.

Can Promptfoo run in CI/CD pipelines?

Yes. With configuration files and provider credentials, it can perform evaluations and security checks in CI/CD.

Is Promptfoo a legal GDPR certification tool?

No. Its GDPR preset checks privacy and related risks, but the documentation does not present it as legal certification.