Feat/pettingzoo parallel env by kydrahul · Pull Request #701 · Farama-Foundation/HighwayEnv

kydrahul · 2026-07-04T14:30:14Z

Description

Adds PettingZoo ParallelEnv support for HighwayEnv's multi-agent intersection environments, implementing true per-agent termination semantics (crashed/arrived agents are removed from self.agents while the rest continue — following PettingZoo's convention, since Gymnasium itself has no "removing agent" mechanism).

This is the initial PR as discussed — the goal here is to get PettingZoo integration working end-to-end. A follow-up PR will expand the regression test suite, add documentation, and cover additional QoL items.

Dependencies added: pettingzoo>=1.24.0, added as an optional [multi-agent] extra so single-agent users see zero change to their install.

Fixes #677

Type of change

New feature (non-breaking change which adds functionality)
New environment or environment variant
Documentation only change (no code changed)
Bug fix (non-breaking change which fixes an issue)
Performance improvement (non-breaking change that improves speed, memory, or resource usage)
Breaking change (fix or feature that would cause existing functionality to not work as expected)
Compatibility fix or other chore tasks
This change requires a documentation update

Summary of Changes

pyproject.toml: Add [multi-agent] optional extra (pettingzoo>=1.24.0)
highway_env/envs/common/abstract.py: Deprecate MultiAgentWrapper with a DeprecationWarning pointing to the new API (old code kept, not removed)
highway_env/envs/common/pettingzoo_env.py: New — HighwayParallelEnv, a PettingZoo ParallelEnv wrapper around any AbstractEnv configured for multi-agent control
highway_env/envs/intersection_pz_env.py: New — IntersectionParallelEnv and ConnectedLaneIntersectionParallelEnv, concrete user-facing subclasses
highway_env/__init__.py: Register intersection-multi-agent-v3 / v4 (only when pettingzoo is installed; skipped silently otherwise)
highway_env/envs/__init__.py: Export the new PZ classes at package level
tests/envs/test_gym.py: Exclude PZ ParallelEnv IDs from the existing gym.make() API test, since these are PettingZoo ParallelEnv instances, not gymnasium.Env subclasses, and gym.make() correctly rejects them
tests/envs/test_pettingzoo.py: New — test suite covering PettingZoo API compliance (parallel_api_test), per-agent termination semantics, seed reproducibility, and the deprecation warning

Testing

parallel_api_test passes for both IntersectionParallelEnv and ConnectedLaneIntersectionParallelEnv
Per-agent termination verified (terminated/truncated agents are correctly removed from self.agents)
Seed reproducibility verified across independent env instances
MultiAgentWrapper deprecation warning verified
Full existing test suite passes with no regressions: 146 passed, 8 skipped
New PZ test suite: 9 passed
Verified graceful skip of PZ tests when pettingzoo is not installed

Breaking Changes

None. Existing intersection-multi-agent-v0/v1/v2 IDs continue to work exactly as before. MultiAgentWrapper is deprecated (raises a warning) but not removed.

Checklist

I have run the pre-commit checks with pre-commit run --all-files
I have commented my code, particularly in hard-to-understand areas
I have made corresponding changes to the documentation (deferred to follow-up PR, per discussion)
My changes generate no new warnings
I have added tests that prove my fix is effective / feature works
New and existing unit tests pass locally with my changes
If my code may add latency, I've tested locally and provided details above (not applicable)

Trenza1ore

Thanks for the quick initial draft! I've marked out some initial places where changes are needed. PettingZoo support will not be part of v1.12 feature due to amount of work still required & changes required for upstream PettingZoo package. So take your time, no rush. 😉

Update:
pettingzoo>=1.27 has a Gymnasium-like registry

Trenza1ore · 2026-07-04T14:37:16Z

+            id="intersection-multi-agent-v3",
+            entry_point="highway_env.envs.intersection_pz_env:IntersectionParallelEnv",
+        )
+    if "intersection-multi-agent-v4" not in registry:


Like the non-PettingZoo registration function above. It's cleaner to just check for whether "intersection-multi-agent-v3" is in registry and:

return if yes

register all pettingzoo envs

Note that this is dependent on the comment above, we shouldn't register PettingZoo environments into Gymnasium's registry.

I completely removed the _register_pettingzoo_envs() function since we're not registering PZ envs in Gymnasium at all.

Trenza1ore · 2026-07-04T14:42:00Z

        RecordConstructorArgs.__init__(self)

-    def step(self, action):
+    def step(self, action):  # unchanged


This comment is pointless and potentially confusing

removed the # unchanged comment.

Trenza1ore · 2026-07-04T14:49:28Z

+    except ImportError:
+        return  # pettingzoo not installed; skip silently
+    if "intersection-multi-agent-v3" not in registry:
+        register(


We shouldn't use register here as these are PettingZoo environments not Gymnasium ones. I'll talk with PettingZoo maintainers and see if they'd be happy to support a PettingZoo registry & potentially a pz.make.

removed all gymnasium.register() calls for PZ environments entirely. Users will instantiate via direct import.

Trenza1ore · 2026-07-04T14:54:44Z

+    from highway_env.envs.common.pettingzoo_env import HighwayParallelEnv  # noqa: F401
+    from highway_env.envs.intersection_pz_env import (  # noqa: F401
+        ConnectedLaneIntersectionParallelEnv,
+        IntersectionParallelEnv,
+    )
+
+    __all__ += [
+        "HighwayParallelEnv",
+        "IntersectionParallelEnv",
+        "ConnectedLaneIntersectionParallelEnv",
+    ]


This is not an especially good pattern as:

IDEs, language servers, etc. don't like mutating __all__ dynamically

The noqa comments are not necessarily respected by all linters & type checkers

Please let me consider it for a bit.

Removed the dynamic all mutation block entirely. PZ classes will just be accessed directly via their own module

Trenza1ore · 2026-07-04T14:59:54Z

+    Parameters
+    ----------
+    env:
+        A fully-initialised :class:`AbstractEnv` with ``MultiAgentObservation``
+        and ``MultiAgentAction`` configured.


This appears to be an incomplete piece of numpy-style docstring (with Returns cutoff), we don't use numpy-style within the repo, please use pep287 style docstring instead (the :param env: ... kind). I see that the docstring above this part seems to be pep287 style as well.

Converted all docstrings in this file to PEP 287 (:param:, :returns: style) and trimmed the redundant prose

Trenza1ore · 2026-07-04T15:07:02Z

+        Returns
+        -------
+        observations:
+            ``{agent_name: obs_array}`` for all agents in :attr:`possible_agents`.
+        infos:
+            ``{agent_name: info_dict}`` (shared info dict for now).


Similar to a previous comment above, Parameters is missing and please don't use numpy-style

Converted all of these to PEP 287 format and trimmed the redundant text.

Trenza1ore · 2026-07-04T15:07:32Z

+        Returns
+        -------
+        observations, rewards, terminations, truncations, infos:
+            All keyed by agent name, covering only the agents that were
+            *active at the start of this step* (i.e. the pre-step
+            :attr:`agents` list).


Same as above

Trenza1ore · 2026-07-04T15:09:54Z

-    if isinstance(env_spec.entry_point, str) and "highway_env" in env_spec.entry_point
+    if isinstance(env_spec.entry_point, str)
+    and "highway_env" in env_spec.entry_point
+    and "intersection_pz_env" not in env_spec.entry_point


PettingZoo environments shouldn't be included in Gymnasium registry as previously mentioned.

Since PZ envs are no longer in the registry at all, this filter is no longer needed. Reverted it back to upstream

Trenza1ore · 2026-07-04T15:13:48Z

+        agents_rewards: tuple = info.get(
+            "agents_rewards", (_reward,) * len(self.possible_agents)
+        )
+        agents_terminated: tuple = info.get(
+            "agents_terminated", (_terminated,) * len(self.possible_agents)
+        )


The use of default in this way is not preferable as they are always evaluated eagerly at runtime (also I believe erroring is fine as if the info doesn't contain these keys something is wrong with the environment? Correct me if I'm wrong as I haven't looked *too* closely)

Switched .get() to direct dictionary access (e.g., info["agents_rewards"]). If the base environment ever forgets to provide those keys, this will fail fast with a KeyError rather than silently swallowing the bug.

Trenza1ore · 2026-07-04T15:19:31Z

+        if hasattr(space, "n"):
+            return min(1, space.n - 1)
+        # Continuous: return zeros
+        import numpy as np


Import NumPy at top-level

moved to the top of the file.

jkterry1 · 2026-09-05T15:29:08Z

@kydrahul would you be kind enough to make the changes requested in review and fix merge conflicts?

…elEnv

…rings, cleanup

jkterry1 · 2026-09-05T18:43:12Z

@kydrahul would you be so kind as to also rebase off main?

Trenza1ore requested changes Jul 4, 2026

View reviewed changes

kydrahul added 10 commits September 5, 2026 22:34

build: add pettingzoo as [multi-agent] optional dependency

8caa402

deprecate: add DeprecationWarning to MultiAgentWrapper

a18c686

feat: add HighwayParallelEnv base wrapper (pettingzoo_env.py)

9fcd12e

feat: add IntersectionParallelEnv and ConnectedLaneIntersectionParall…

d2dc6c7

…elEnv

feat: register intersection-multi-agent-v3/v4 in __init__.py

f8efa8e

fix: exclude PettingZoo ParallelEnv IDs from gym.make() API test

ed0a7c7

feat: export PettingZoo env classes in highway_env/envs/__init__.py

0cab5bb

test: add PettingZoo parallel env test suite

9ac3e55

style: apply pre-commit formatting and lint fixes

7cd3246

fix: address review feedback — remove gym registration, PEP 287 docst…

26b3452

…rings, cleanup

kydrahul force-pushed the feat/pettingzoo-parallel-env branch from ba5f9f7 to 26b3452 Compare September 5, 2026 17:59

ci: run tests with all extras to fix PettingZoo coverage

a3d7eba

Conversation

kydrahul commented Jul 4, 2026

Description

Type of change

Summary of Changes

Testing

Breaking Changes

Checklist

Uh oh!

Trenza1ore left a comment • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

jkterry1 commented Sep 5, 2026

Uh oh!

jkterry1 commented Sep 5, 2026

Uh oh!

This branch has not been deployed

Reviewers

Uh oh!

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

Trenza1ore left a comment •

edited

Loading