ai-benchmark

Windows Agent Arena (WAA) 🪟 is a scalable OS platform for testing and benchmarking of multi-modal AI agents.

An agent benchmark with tasks in a simulated software company.

Frontier Models playing the board game Diplomacy.

Track 20,710+ AI benchmark, eval, dataset, and data-quality records from 39 public sources, with linked evidence and daily updates.

JevBench by Benchmark Heaven: benchmark and leaderboard for AI decision models, measuring typed decision accuracy, calibration, latency and cost.

Ranking LLMs on agentic tasks

The definitive benchmark for AI agents on OpenClaw. 45 tasks across 4 tiers. Powered by MyClaw.ai

开源劳动者AI研究与评测项目：建设劳动知识、公开评测基准与共同治理机制｜Open worker-centered AI research and evaluation project.

🤖 A curated list of resources for testing AI agents - frameworks, methodologies, benchmarks, tools, and best practices for ensuring reliable, safe, and effective autonomous AI systems

llmBench is a high-depth benchmarking tool designed to measure the raw performance of local LLM runtimes (Ollama, llama.cpp) while providing deep hardware intelligence.

A visual benchmark testing whether leading coding agents repeat the same design patterns across 100 neutral website briefs.

📊 Daily auto-updated snapshots of all Arena AI (LMSYS Chatbot Arena) leaderboards — LLM, Vision, Code, Video, Image & more. Structured JSON with historical tracking.

Benchmarking and evaluation resources for trust, robustness, and reliability analysis in graph-based AI systems.

MindTrial: Evaluate and compare AI language models (LLMs) on text-based tasks with optional file/image attachments and tool use. Supports multiple providers (OpenAI, Google, Anthropic, DeepSeek, Mistral AI, xAI, Alibaba, Moonshot AI, OpenRouter), custom tasks in YAML, and HTML/CSV/JSON reports.

Benchmark Heaven: AI model benchmarks with sources and modeled costs, plus the JevBench and ImageJevBench decision model leaderboards.

Precision-aware retrieval benchmark for LLM memory systems.

A curated list of evaluation tools, benchmark datasets, leaderboards, frameworks, and resources for assessing model performance.

GDB: GraphicDesignBench - A real-world benchmark for evaluating AI on graphic design tasks

The document extraction benchmark by Flexprice

Benchmark abierto en español de modelos de IA para negocios y agentes, con juez independiente (Phi-4). Calidad, costo, velocidad, contexto largo, trabajo agéntico y fuga de credenciales, cada uno por separado. Calculadora interactiva con tus propios pesos.