Skip to content

feat: mxcli test call — JSON runtime executor against an already-running app - #1789

Draft
MendixMau wants to merge 3 commits into
mendixlabs:mainfrom
MendixMau:feature/1788-test-call
Draft

MendixMau wants to merge 3 commits into
mendixlabs:mainfrom
MendixMau:feature/1788-test-call

Conversation

@MendixMau

@MendixMau MendixMau commented Nov 9, 2026 •

Copy link
Copy Markdown
Contributor

Closes #1788

Why

A single-flow logic edit currently costs a full mxcli test cycle (regenerate test MDL, mxbuild, boot/reload the runtime, run the suite) even though the actual flow-plus-assertion cost, once the app is already running, is a few milliseconds. This adds a second, opt-in route for that narrower case: checking one or a few flows against a live app without touching test MDL files at all.

What changed

New command mxcli test call <cases.json|-> [--repeat N], requires an app already booted with mxcli run --local --test-endpoint:

  • cmd/mxcli/cmd_test_call.go — CLI wiring.
  • cmd/mxcli/testrunner/call.go — case parsing (ParseCallCases, single object or array), stale-model check (checkModelNotStale).
  • cmd/mxcli/testrunner/endpoint.go / endpoint_call.go — new call route on the existing test endpoint.

Case shape: {flow, params, setup, expect, throws, rollback}.

  • params/setup build objects via $new (create), $find (lookup), $ref (reuse a $named object from setup), nested associations inline.
  • expect: [{path, op, value}], evaluated before rollback. path supports $result, $<param>, association hops (e.g. /Member/Member), and count:Entity[xpath] for row counts. op is = != < <= > >= contains isnull notnull.
  • throws asserts the flow raised.

This reuses the existing --test-endpoint transaction-and-rollback machinery (Core.microflowCall) and the same known.contains(flow) microflow allowlist the pre-existing run route already enforces — it is not a new security surface, just a second entry point into the same gated mechanism.

The all-or-nothing diff is purely additive: 5 files, 744 insertions, 0 deletions — no existing code path is touched.

How tested

Unit:

go test ./cmd/mxcli/testrunner/...

→ ok github.com/mendixlabs/mxcli/cmd/mxcli/testrunner 13.442s, all green, including 2 new tests (TestParseCallCases..., TestCheckModelNotStale...) plus the full pre-existing suite (watch-loop, stale-flow, mtime tests) unaffected.

Real app (~/mta-lab/w/rt1, E1 procurement domain, 8 pre-existing cases — 5x invoice pay/approve/dispute, 2x price/qty matching, 1x ERP posting), each step's 1-minute load average from uptime noted:

step result load
clean baseline 8/8 PASS 26.3
wrong @expect (mutated expected value on case 120-1) 1 FAIL (the mutated case), 4 PASS — stays red 21.8
seed pay-guard bug (ACT_Invoice_Pay spliced to skip a status check) + mxcli run restart case 120-5 flips to FAIL: actual: PAID where APPROVED was expected 21.3-23.7
revert bug + restart 5/5 PASS again 16.6-20.8
rollback-leaves-0-rows: call A creates+counts an Invoice (count=1, pre-rollback); independent call B counts the same row count=0 in call B — confirms no cross-call leakage 26.8
stale model: edit model without rebuilding, then call refused: Error: the model changed after the running app was built (...) a call now would test the OLD model and could report a stale green 25.4
broken microflow (references a dropped entity) + restart build fails [CE1613]; run restart reports failed: initial build failed; call never reachable — never a false green 23.2
cleanup + restart 8/8 PASS restored 21.3
warm timing, --repeat 20, 8 cases/round round 1: 19.2 ms/case → round 14-20: 7.3-8.4 ms/case 21.6-22.8
re-confirmation under load < 15 (separate --repeat 20 run) round 1: 10.3 ms/case → round 13-20: 3.5-4.9 ms/case — same convergence shape, even faster 9.5-11.1

Repro commands:

mxcli run --local --test-endpoint --detach --db-type hsqldb --app-port 8301 --admin-port 8311 --serve-port 7901 -p App.mpr
mxcli test call ~/mta-lab/speedlab/cases/e1-all.json -p App.mpr
mxcli test call ~/mta-lab/speedlab/cases/e1-all.json -p App.mpr --repeat 20

Regression evidence

Diff touches only the 5 new/extended files listed above — no existing run/test code path is modified, confirmed via git diff --stat origin/main...HEAD. Full cmd/mxcli/testrunner suite green before and after on this branch.

Known limitation

Additive and opt-in: does not replace file-based mxcli test (no test discovery, no watch mode of its own — see mxcli test --watch, #1786, for that loop). Requires an app already booted with --test-endpoint.

Checklist

🤖 Generated with Claude Code

…he test endpoint's new call route (experiment)

(cherry picked from commit 788952fac01c7326dd349a68563ca4fe426229b1)
…del refusal

(cherry picked from commit 0bbbd586127b3c2a0ce6df9b0b5eed6c8efced79)
Covers the pure-Go, runtime-independent logic: ParseCallCases (single
object vs array form, missing-flow rejection, invalid JSON) and
checkModelNotStale (no deployment, fresh model, and the core safety
property — a project/javasource file edited after the build is refused
rather than silently tested against the old model).

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>

(refs mendixlabs#1788)

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

feat: mxcli test call — JSON runtime executor against an already-running app

1 participant