castArrayFilters only looks at the first key: $or/$and after a field key is cast as subdocuments, fields after a leading $or and $nor are not cast #16637

Prerequisites

• I have written a descriptive issue title
• I have searched existing issues to ensure the bug has not already been reported

Mongoose version

9.11.4 (master at f59e13a); the same code is in 6.13.x and 8.24.x

Node.js version

v24.21.0

MongoDB server version

8.0

Typescript version (if applicable)

No response

Description

castArrayFilters decides how to cast an array filter document by looking only at its first key. When $and/$or is first, it recurses into it (gh-10696). When the first key is a field, every key, including any logical operator, is cast as a field of the filtered element. That has three consequences:

1. A $or/$and after a field key ({ 'el.deleted': false, $or: [...] }) is cast with the document array's own schematype. Each branch is replaced by a new subdocument ({ _id: <new ObjectId> }) instead of a cast query. The branches no longer name the filter's identifier, so the server refuses the update.
2. Field keys after a leading $or ({ $or: [...], 'el.deleted': 'true' }) are never cast.
3. $nor is never cast, wherever it appears.

The server treats all three shapes as valid, and key order in a filter object is usually incidental. So the same filter works or breaks depending on how it was spread together.

We hit (1) in production. A bulk updateMany stamped an error state on array elements with arrayFilters: [{ 'el.deleted': { $ne: true }, $or: [{ 'el.listingId': null }, { 'el.listingId': { $exists: false } }] }]. It stopped updating any element for about three months before we traced it to the cast.

Steps to Reproduce

const mongoose = require('mongoose');
const castArrayFilters = require('mongoose/lib/helpers/update/castArrayFilters');

const Test = mongoose.model('Test', new mongoose.Schema({ arr: [{ id: Number, deleted: Boolean }] }));
const show = (arrayFilters) => {
  const q = Test.updateOne({}, { $set: { 'arr.$[el].id': 1 } }, { arrayFilters });
  castArrayFilters(q);
  console.log(JSON.stringify(q.options.arrayFilters));
};

show([{ 'el.deleted': 'false', $or: [{ 'el.id': '12' }, { 'el.id': { $exists: false } }] }]);
show([{ $or: [{ 'el.id': '12' }], 'el.deleted': 'true' }]);
show([{ $nor: [{ 'el.id': '12' }] }]);

Output on master:

[{"el.deleted":false,"$or":[{"_id":"6adbe876455a13e2680a505b"},{"_id":"6adbe876455a13e2680a505c"}]}]
[{"$or":[{"el.id":12}],"el.deleted":"true"}]
[{"$nor":[{"el.id":"12"}]}]

Expected Behavior

Each logical key ($and, $or, $nor) is recursed into wherever it appears in the filter, and every field key is cast against the element schema:

[{"el.deleted":false,"$or":[{"el.id":12},{"el.id":{"$exists":false}}]}]
[{"$or":[{"el.id":12}],"el.deleted":true}]
[{"$nor":[{"el.id":12}]}]

I have a PR with a fix and tests, and will link it here.