Skip to content

Commit 11c72b2

Browse files
authored
Fix Eleusis for current Verifiers chat program source (#23)
## Summary - Use Verifiers' `CHAT_PROGRAM_SOURCE` in the Eleusis null harness. - Pin Verifiers to commit `7eaa262` (`0.3.2.dev128`), which includes the rename, so installs used for training resolve the compatible source. - Allow the direct Git dependency in Hatchling metadata and document the pin. `uv sync --all-extras` completed in `environments/eleusis` with the pinned Verifiers commit. <!-- Macroscope's pull request summary starts here --> <!-- Macroscope will only edit the content between these invisible markers, and the markers themselves will not be visible in the GitHub rendered markdown. --> <!-- If you delete either of the start / end markers from your PR's description, Macroscope will append its summary at the bottom of the description. --> > [!NOTE] > ### Fix Eleusis harness to use `CHAT_PROGRAM_SOURCE` and pin Verifiers to commit `7eaa262` > - Replaces the Null harness `PROGRAM_SOURCE` with the shared `CHAT_PROGRAM_SOURCE` from launch utilities in [harness.py](https://github.com/PrimeIntellect-ai/residency-environments/pull/23/files#diff-30c0abedb7aff32c08b6b8a66e8de2362647d919455b5102dadb05c04a5a4817), assigning it to `ELEUSIS_PROGRAM_SOURCE` > - Switches the Verifiers dependency in [pyproject.toml](https://github.com/PrimeIntellect-ai/residency-environments/pull/23/files#diff-1cac076d0da70b40f95a6ecf61cd5ba94a1a9eeef4494fd1367da70a76936e0b) from a `>=0.3.0,<0.4` version range to a direct Git dependency pinned to commit `7eaa262`, enabling Hatch's `allow-direct-references` > - Documents the pinned commit in [README.md](https://github.com/PrimeIntellect-ai/residency-environments/pull/23/files#diff-0ae3cc3192a10f625bab934b96494d674260c695d6b74043a9a4f2fadb1c5062) so the chat harness matches the training version > - Behavioral Change: installing the Eleusis project now resolves Verifiers from the pinned Git commit instead of the published package version range > > <!-- Macroscope's review summary starts here --> > > <sup><a href="https://app.macroscope.com">Macroscope</a> summarized 8de768b.</sup> > <!-- Macroscope's review summary ends here --> > <!-- Macroscope's pull request summary ends here -->
1 parent 01d9f5f commit 11c72b2

3 files changed

Lines changed: 10 additions & 3 deletions

File tree

‎environments/eleusis/README.md‎

Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -39,6 +39,9 @@ Unsolved episodes receive zero.
3939

4040
## Run
4141

42+
Eleusis pins Verifiers to commit `7eaa262` so the chat harness matches the
43+
version used for training.
44+
4245
```bash
4346
uv run eval eleusis -m <model>
4447
```

‎environments/eleusis/eleusis/harness.py‎

Lines changed: 3 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -6,9 +6,10 @@
66
injected messages or recovery turns.
77
"""
88

9-
from verifiers.v1.harnesses.null.harness import PROGRAM_SOURCE, NullHarness
9+
from verifiers.v1.harnesses.null.harness import NullHarness
10+
from verifiers.v1.harnesses.utils.launch import CHAT_PROGRAM_SOURCE
1011

11-
ELEUSIS_PROGRAM_SOURCE = PROGRAM_SOURCE
12+
ELEUSIS_PROGRAM_SOURCE = CHAT_PROGRAM_SOURCE
1213

1314

1415
class EleusisHarness(NullHarness):

‎environments/eleusis/pyproject.toml‎

Lines changed: 4 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -5,14 +5,17 @@ description = "A long-horizon inductive-reasoning card game environment with nat
55
tags = ["eleusis", "card-game", "multiturn", "tool-use", "inductive-reasoning", "eval", "verifiers"]
66
requires-python = ">=3.11,<3.14"
77
dependencies = [
8-
"verifiers>=0.3.0,<0.4",
8+
"verifiers @ git+https://github.com/PrimeIntellect-ai/verifiers.git@7eaa262",
99
"datasets>=3.0",
1010
]
1111

1212
[build-system]
1313
requires = ["hatchling"]
1414
build-backend = "hatchling.build"
1515

16+
[tool.hatch.metadata]
17+
allow-direct-references = true
18+
1619
[tool.hatch.build.targets.wheel]
1720
packages = ["eleusis"]
1821

0 commit comments

Comments
 (0)