[harishk30]

WorldBench is a visually diverse multimodal reasoning benchmark with 2,000 image-based multiple-choice questions spanning living things, objects, scenes, the digital world, academics, documents, and agents. This adds WorldBench as an installable Verifiers v1 taskset. See the paper and official dataset.

The environment streams a pinned revision of the Hugging Face dataset, implements the benchmark prompt and answer parser, preserves category/domain metadata, and scores exact-match accuracy with an answer-parsing metric. Supported source images are preserved within a 10 MB encoded-payload and 8000-pixel dimension limit; unsupported formats are normalized to PNG, with resizing when necessary.

Existing evaluation results

The original README reports the following direct evaluations over all 2,000 examples, with one rollout per example, NullHarness, and a 32,768-token output cap:

Model	Average accuracy
Kimi K3	59.95%
Kimi-K2.5	53.60%
Qwen3.5-VL-35B-A3B	48.30%

Per-domain results and comparisons with paper-reported scores are included in the README. These are the results already recorded in the original contribution, not newly run evaluations for this port.

Note

Add WorldBench residency environment with Verifiers taskset

Adds the worldbench package, which streams the pinned zlab-princeton/WorldBench training split and yields one multimodal A-D task per row via WorldBenchTaskset.load in taskset.py.

• format_prompt builds the question and A-D options; parse_answer in formatting.py accepts only a final ANSWER: line with a letter A-D.
• bounded_image_data_url in images.py encodes images as data URLs under a 10 MB payload limit and 8,000-pixel max dimension, resizing or re-encoding (PNG/JPEG/WebP) as needed.
• Each task scores answer_parsed (0/1) and rewards 1.0 only when the parsed answer matches the stored expected answer.
• Declares Python 3.11 plus Datasets, Pillow, and Verifiers dependencies in pyproject.toml; usage is documented in the README.

^{Macroscope summarized ae7ed31.}