Lighteval

Lighteval alternatives

The platforms standing nearest to Lighteval on LLM Evaluation Tools.

Nearest on its board

  1. 28
    KB
    kube-benchgithub.com
    4.0 APIOpenFree —
  2. 27
    OE
    OpenAI Evalsgithub.com
    4.8 APIOpenFree Free tier
  3. 26
    D
    DecodingTrustdecodingtrust.github.io

    DecodingTrust is a research project for assessing trustworthiness in GPT models and helping researchers and practitioners understand their capabilities…

    5.0 APIOpenFree —
  4. 25
    W
    WebArenawebarena.dev

    WebArena is a standalone, self-hostable web environment and benchmark for building autonomous agents.

    5.0 APIOpenFree —
  5. 24
    L
    LiveBenchlivebench.ai

    LiveBench is an LLM benchmark designed to limit test set contamination and use objective evaluation.

    5.5 APIOpenFree —
  6. 23
    IA
    Inspect AIinspect.aisi.org.uk

    Inspect is an open-source framework for evaluating large language models across coding, agentic tasks, reasoning, knowledge, behavior, and multimodal…

    5.6 APIOpenFree Free tier
  7. 22
    R
    RAGCheckergithub.com

    RAGChecker is an automatic evaluation framework for assessing and diagnosing retrieval-augmented generation systems.

    5.7 APIOpenFree Free tier
  8. 21
    PA
    Parea AIparea.ai

    Parea describes itself as a platform for experiment tracking, observability, and human annotation to help teams ship LLM applications to production.

    5.8 APIOpenFree Free tier · from $150/mo
  9. 20
    LE
    LM Evaluation Harnessgithub.com

    LM Evaluation Harness is a unified framework for evaluating generative language models across evaluation tasks.

    5.8 APIOpenFree Free tier
  10. 19
    H
    HarmBenchgithub.com

    HarmBench is an open-source framework for evaluating automated red teaming methods and LLM attacks and defenses.

    5.8 APIOpenFree Free tier
  11. 18
    E
    EvalPlusgithub.com

    EvalPlus is a framework for rigorously evaluating large language models on code related tasks.

    5.8 APIOpenFree Free tier
  12. 17
    CA
    Confident AIconfident-ai.com

    Confident AI is an AI Quality platform that provides development evaluations and production observability for reliable AI applications.

    5.9 APIOpenFree Free tier · from $200/mo
  13. 16
    P
    PyRITmicrosoft.github.io

    PyRIT is an open-source framework for security professionals and engineers to identify risks in generative AI systems.

    6.0 APIOpenFree Free tier
  14. 15
    O
    OpenCompassgithub.com

    OpenCompass is an LLM evaluation platform for assessing models across datasets covering knowledge, reasoning, coding, science, language, long context, and…

    6.0 APIOpenFree Free tier
  15. 14
    MA
    Maxim AIgetmaxim.ai

    Maxim is an end-to-end evaluation and observability platform for simulating, evaluating, and monitoring AI agents.

    6.0 APIOpenFree Free tier · from $29/mo
  16. 13
    G
    Galileogalileo.ai

    Galileo is an observability, evaluation, and guardrail platform for GenAI and agentic applications.

    6.0 APIOpenFree Free tier · from $100/mo
  17. 12
    A
    ARESgithub.com

    ARES is an automated framework for evaluating retrieval-augmented generation (RAG) systems.

    6.0 APIOpenFree Free tier
  18. 11
    A
    AgentBenchgithub.com

    AgentBench is a benchmark for evaluating the reasoning and decision-making abilities of LLMs acting as agents.

    6.0 APIOpenFree Free tier
  19. 10
    R
    Ragasragas.io

    Ragas is a library for systematically evaluating AI and large language model applications.

    6.0 APIOpenFree Free tier
  20. 9
    G
    garakgarak.ai

    garak is an LLM vulnerability scanner that probes language models and systems using them for security weaknesses.

    6.1 APIOpenFree Free tier