From 01cd08aeec4ba7fb8cc20f44796ccb0ef1505e5d Mon Sep 17 00:00:00 2001 From: Sebastian Date: Fri, 28 Aug 2026 22:51:01 +0200 Subject: [PATCH 1/3] chore: add scripts and configs layout --- .pre-commit-config.yaml | 2 +- AGENTS.md | 10 +++++++++- README.md | 2 ++ configs/README.md | 8 ++++++++ scripts/README.md | 11 +++++++++++ 5 files changed, 31 insertions(+), 2 deletions(-) create mode 100644 configs/README.md create mode 100644 scripts/README.md diff --git a/.pre-commit-config.yaml b/.pre-commit-config.yaml index e83c2c9..8da1cb9 100644 --- a/.pre-commit-config.yaml +++ b/.pre-commit-config.yaml @@ -10,7 +10,7 @@ repos: rev: v0.23.1 hooks: - id: markdownlint-cli2 - files: ^(?:[^/]+\.md|\.github/.*\.md|environments/[^/]+/README\.md|generators/.*\.md)$ + files: ^(?:[^/]+\.md|\.github/.*\.md|environments/[^/]+/README\.md|(?:configs|generators|scripts)/.*\.md)$ - repo: local hooks: diff --git a/AGENTS.md b/AGENTS.md index e05a9cd..11bcc00 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -55,6 +55,8 @@ Use this guidance when contributing to the `residency-environments` repository i - During development, install environments (`/environments`) from the project's root directory using editable, local installs as `uv pip install -e ./environments/`. DO NOT install from within the environment directories. - Every directory under `environments/` is a Verifiers v1 environment; no naming suffix is required. - For synthetically generated environments, the data generation and validation code lives in a matching `generators//` directory (see `generators/README.md`). +- Environment-specific support code that is not part of the installable package lives in `scripts//` (see `scripts/README.md`). This includes sandbox image definitions and build helpers. +- Reusable configurations for testing and evaluating an environment live in `configs//` (see `configs/README.md`). Never commit credentials or other secrets in configuration files. - Every environment must declare its full-eval convention in `pyproject.toml` under `[tool.verifiers.eval]` (`num_examples`, `rollouts_per_example`); automated eval runs read it when no sample count is given. Infinite tasksets fall back to 50 examples. - To check an environment implementation, use the v1 `eval` CLI. Start with a single example and two rollouts so tasksets using group rewards can compare outputs. @@ -81,7 +83,9 @@ Tasksets go in *environments*. Questions/task descriptions, and possibly gold standard answers go into a HuggingFace dataset. If there's large amounts of data that the model has to work with, which takes time to setup in a sandbox, ship it in sandbox images and reference them in the Huggingface dataset for every prompt, so that each task starts with the right sandbox. -If the data is generated synthetically, the scripts belong under *generators*. +If the data is generated synthetically, the generation and validation code belongs under *generators*. + +Supporting development code, such as sandbox image definitions and build helpers, belongs under *scripts*. Reusable test and evaluation settings belong under *configs*. The structure is always: @@ -89,6 +93,10 @@ The structure is always: - `/` - generators - `/` # if data is synthetically generated +- scripts + - `/` # if the environment needs supporting development code +- configs + - `/` # if reusable test or evaluation configurations exist ### On tools diff --git a/README.md b/README.md index 33b5a47..bf698b0 100644 --- a/README.md +++ b/README.md @@ -3,6 +3,8 @@ RL environments built by the Prime Intellect **RL Residency**. Each directory in `environments/` is an installable Verifiers v1 taskset. +Supporting data generators, development scripts, and reusable test configurations +live in `generators/`, `scripts/`, and `configs/`, respectively. ## Setup diff --git a/configs/README.md b/configs/README.md new file mode 100644 index 0000000..c135042 --- /dev/null +++ b/configs/README.md @@ -0,0 +1,8 @@ +# configs + +Reusable configuration files for testing and evaluating environments. + +Keep environment-specific configurations in `configs//`. Configurations +should be portable, contain no credentials or other secrets, and describe the +settings needed to reproduce the intended test or evaluation. Evaluation outputs +do not belong here. diff --git a/scripts/README.md b/scripts/README.md new file mode 100644 index 0000000..0699f3c --- /dev/null +++ b/scripts/README.md @@ -0,0 +1,11 @@ +# scripts + +Supporting code and build files for developing and operating environments. + +Keep environment-specific files in `scripts//`. This includes sandbox +image definitions and build helpers, setup utilities, and other development code +that is not part of the installable environment package. Synthetic data generation +and validation code remains in `generators//`. + +Repository-wide utilities, such as `install.sh`, may remain directly under +`scripts/`. From e6bb2b7956ebd42e7813675bdfdc9c838f7401c5 Mon Sep 17 00:00:00 2001 From: Sebastian Date: Fri, 28 Aug 2026 22:55:15 +0200 Subject: [PATCH 2/3] chore: align Markdown validation --- .markdownlint-cli2.yaml | 8 ++++++++ 1 file changed, 8 insertions(+) diff --git a/.markdownlint-cli2.yaml b/.markdownlint-cli2.yaml index 7148be7..5fd8e37 100644 --- a/.markdownlint-cli2.yaml +++ b/.markdownlint-cli2.yaml @@ -4,12 +4,20 @@ config: allowed_elements: - br - details + - h3 - img - p - summary MD060: style: compact overrides: + - filter: + - README.md + config: + MD029: false + MD036: false + MD041: false + combine: merge - filter: - CLAUDE.md config: From 710beab495e028d60140375bb7a260c615e79634 Mon Sep 17 00:00:00 2001 From: Sebastian Date: Fri, 28 Aug 2026 23:12:23 +0200 Subject: [PATCH 3/3] docs: keep layout prose unwrapped --- README.md | 4 ++-- configs/README.md | 5 +---- scripts/README.md | 8 ++------ 3 files changed, 5 insertions(+), 12 deletions(-) diff --git a/README.md b/README.md index bf698b0..567c6ff 100644 --- a/README.md +++ b/README.md @@ -3,8 +3,8 @@ RL environments built by the Prime Intellect **RL Residency**. Each directory in `environments/` is an installable Verifiers v1 taskset. -Supporting data generators, development scripts, and reusable test configurations -live in `generators/`, `scripts/`, and `configs/`, respectively. + +Supporting data generators, development scripts, and reusable test configurations live in `generators/`, `scripts/`, and `configs/`, respectively. ## Setup diff --git a/configs/README.md b/configs/README.md index c135042..81884b1 100644 --- a/configs/README.md +++ b/configs/README.md @@ -2,7 +2,4 @@ Reusable configuration files for testing and evaluating environments. -Keep environment-specific configurations in `configs//`. Configurations -should be portable, contain no credentials or other secrets, and describe the -settings needed to reproduce the intended test or evaluation. Evaluation outputs -do not belong here. +Keep environment-specific configurations in `configs//`. Configurations should be portable, contain no credentials or other secrets, and describe the settings needed to reproduce the intended test or evaluation. Evaluation outputs do not belong here. diff --git a/scripts/README.md b/scripts/README.md index 0699f3c..9bf9d83 100644 --- a/scripts/README.md +++ b/scripts/README.md @@ -2,10 +2,6 @@ Supporting code and build files for developing and operating environments. -Keep environment-specific files in `scripts//`. This includes sandbox -image definitions and build helpers, setup utilities, and other development code -that is not part of the installable environment package. Synthetic data generation -and validation code remains in `generators//`. +Keep environment-specific files in `scripts//`. This includes sandbox image definitions and build helpers, setup utilities, and other development code that is not part of the installable environment package. Synthetic data generation and validation code remains in `generators//`. -Repository-wide utilities, such as `install.sh`, may remain directly under -`scripts/`. +Repository-wide utilities, such as `install.sh`, may remain directly under `scripts/`.