Testing
Fast unit tests by default, quick release checks in one command, and Playground end-to-end suites that call the real AXe CLI on iPhone, iPad and iPhone Duo simulators and physical devices.
Unit tests
make test
swift test --filter DrawTestsUnit tests need no simulator. They cover command validation, agent admission and output, display discovery, the device JSON shapes and the orientation rules.
Release checks
make release-checkRuns everything short of the end-to-end suites and reports each check: released changelog sections unchanged, the script tests, the two skill copies identical, the skill lint, no open items in the release readiness log, the packaging dry run and the unit tests. Every check runs even when one fails. RELEASE_CHECK_SKIP="skill lint,packaging" leaves checks out by name.
Playground end-to-end suite
The Playground is a normal app. Swift Testing calls the real AXe CLI and checks the app's state, accessibility, contacts and callbacks after real input. The app has no injected test state.
DEVELOPER_DIR=/Applications/Xcode.app/Contents/Developer make e2emake e2e uses dedicated iPhone 18 Pro, iPad Air and iPhone Duo simulators on the newest runtime the selected Xcode supports, creating any that are missing, and writes results to e2e-results/<UTC time>/. It needs Xcode 27.1 or later for the Playground's Duo hinge view, and exits 2 without an iPhone Duo runtime. Set AXE_E2E_IPHONE_UDID, AXE_E2E_IPAD_UDID or AXE_E2E_DUO_UDID to use other simulators.
Run the whole matrix once at the end of a significant piece of work. For a fix, run only the affected groups.
Focused checks
Run one group on one device:
python3 scripts/test-playground-matrix.py --only duo --suite keyboard-hardware \
--axe /absolute/path/to/axe --iphone IPHONE_UUID --duo DUO_UUID \
--developer-dir /Applications/Xcode.app/Contents/Developer \
--output e2e-results/NEW_RUN--only takes iphone, ipad or duo; --suite groups include core-controls, agent-controls, keyboard-hardware, media and hinge. Add --group <TestSuite>, repeatable, to run only named test suites of that group. A focused run proves only what it ran, on that device.
Physical devices
DEVELOPER_DIR=/Applications/Xcode.app/Contents/Developer make e2e-devicesThe device suite is opt-in. It exits 1 before building when no physical iPhone or iPad is paired, installs the Playground on each device with automatic signing and reads every result back from the device.
Validate a release payload
To check a release binary under another Xcode, build the test bundle with that Xcode, then:
AXE_BIN_PATH=/path/to/release/axe ./test-runner.sh --tests-onlyBenchmarks
./benchmarks/run setup
./benchmarks/run doctor
./benchmarks/run --tools axe --scenarios weatherThe benchmarks compare AXe with its best accepted baseline on Weather, Contacts and Reminders tasks, recording calls, time and returned tokens.