Project Moonshot

Input—per 1M tokens

Output—per 1M tokens

Context—tokens

WeightsClosed

About

Project Moonshot is ranked #5 of 28 in AI security testing tools on Inferse. It runs on API, Linux, macOS, Self-hosted, Web, Windows. There is a free plan.

Compared on AI security testing tools

Free plan: Yesaiverifyfoundation.sg
Prompt injection tests: Yesaiverifyfoundation.sg
Jailbreak tests: Yesaiverifyfoundation.sg
Data leakage tests: Yesaiverifyfoundation.sg
Unsafe output tests: Yesaiverifyfoundation.sg
Custom test cases: Yesaiverifyfoundation.sg
Deployment mode: self_hostedaiverifyfoundation.sg

Facts

Purpose: Project Moonshot is an open-source toolkit for assessing the safety and reliability of large language models and applications through benchmark testing and red teaming.aiverifyfoundation.sg · 29 Sept 2026
Benchmarking: It runs benchmark tests on LLMs and applications using open-source datasets and metrics for performance and trust and safety risks.github.com · 29 Sept 2026
Red teaming: It generates adversarial prompts using algorithmic strategies or generative LLMs to probe potential vulnerabilities and misuse.github.com · 29 Sept 2026
Custom tests: Users can build benchmark tests using custom datasets, optional prompt templates, evaluation metrics, and grading scales.github.com · 29 Sept 2026
Reporting: It provides interactive HTML reports and downloadable raw JSON test results.github.com · 29 Sept 2026
Interfaces: Moonshot can be used through a Web UI, an interactive command-line interface, library APIs, and Web APIs.github.com · 29 Sept 2026
Integrations: Users can configure connections to their LLMs and create custom connector endpoints through the Web UI or CLI guides.aiverify-foundation.github.io · 29 Sept 2026
IMDA alignment: The toolkit implements benchmarks recommended in IMDA’s Starter Kit for safety testing LLM-based applications.aiverifyfoundation.sg · 29 Sept 2026
Requirements: The repository specifies Python 3.11 and, for the Web UI, Node.js 20.11.1 LTS or later; test assets are also required to run tests.github.com · 29 Sept 2026
License and maturity: The repository identifies Moonshot as beta software released under the Apache Software License 2.0.github.com · 29 Sept 2026
Intended users: The project describes its users as AI developers and compliance teams evaluating LLM-based applications and LLMs.github.com · 29 Sept 2026
Maker: AI Verify Foundation is a not-for-profit wholly owned subsidiary of Singapore’s Infocommunications Media Development Authority.aiverifyfoundation.sg · 29 Sept 2026
Purpose: Project Moonshot is an open-source toolkit for testing the safety and reliability of LLMs and LLM applications through benchmarking and red teaming.aiverifyfoundation.sg · 30 Sept 2026
Benchmarking: Its benchmarks cover capability, quality, and trust and safety, including measures such as accuracy, bias, toxicity, and hallucination.github.com · 30 Sept 2026
Red teaming: The toolkit supports adversarial testing with prompt templates, context strategies, and automated attack modules.aiverify-foundation.github.io · 30 Sept 2026
Interfaces: Moonshot can be used through a web UI, an interactive command-line interface, library APIs, and web APIs.github.com · 30 Sept 2026
Provider connections: The documentation names OpenAI, Anthropic, Together, and Hugging Face as model providers Moonshot can connect to with an API key.aiverify-foundation.github.io · 30 Sept 2026
Custom connectors: Users can create model connectors for other models or their own LLM applications hosted on custom servers.aiverify-foundation.github.io · 30 Sept 2026
Custom evaluations: Users can create recipes using their own datasets, optional prompt templates, evaluation metrics, and grading scales.github.com · 30 Sept 2026
Reports: Moonshot provides interactive HTML reports and downloadable raw JSON results for programmatic analysis.github.com · 30 Sept 2026
Installation: The maker's instructions install Moonshot with pip and run the web UI locally at localhost:3000.aiverify-foundation.github.io · 30 Sept 2026
Requirements: Moonshot requires Python 3.11, and its web UI requires Node.js 20.11.1 LTS or above and npm 10.8.0 or above.aiverify-foundation.github.io · 30 Sept 2026
Compatibility note: The installation guide recommends Chrome for the best web UI experience and says x86 Macs may encounter installation difficulties with moonshot-data dependencies.aiverify-foundation.github.io · 30 Sept 2026
License and status: The GitHub repository identifies the project as beta and licenses it under Apache License 2.0.github.com · 30 Sept 2026
Support: The Moonshot FAQ directs users who need more help to raise an issue on GitHub.aiverify-foundation.github.io · 30 Sept 2026
Intended users: The maker describes Moonshot as a tool for AI developers, compliance teams, and AI system owners evaluating LLMs and LLM applications.aiverify-foundation.github.io · 30 Sept 2026