Skip to content

Add iOS Simulator AX snapshot fallback for XCTest snapshot failures #701

@thymikee

Description

@thymikee

Problem

Some iOS Simulator app screens are visible and interactable, but XCTest cannot serialize or enumerate their accessibility tree reliably.

Observed failure mode:

  • full XCTest snapshot can fail with kAXErrorIllegalArgument
  • compact interactive XCTest snapshot can degrade to only the synthetic application root
  • retrying the same XCTest APIs tends to make the command slower rather than recover semantic UI nodes

The current bounded XCTest behavior is correct for avoiding hangs and false confidence, but it still leaves simulator sessions without semantic snapshot coverage for this class of apps.

Proposal

Add an iOS Simulator-only AX snapshot backend/fallback that does not depend on XCUIElement.snapshot().

Expected shape:

  • read the Simulator accessibility surface through host-side Simulator/macOS accessibility APIs, similar in role to idb's simulator accessibility commands
  • normalize output into the existing SnapshotNode model
  • keep XCTest as the default runner for actions, physical devices, and normal working snapshots
  • use the AX backend as a simulator fallback when XCTest reports IOS_AX_SNAPSHOT_FAILED or compact interactive output is sparse/root-only
  • make fallback choice observable in diagnostics and snapshot warnings

Requirements

  • simulator-only; do not imply physical-device support
  • deterministic Simulator window/device/app-root selection
  • bounded traversal with per-node failure tolerance
  • stable frame normalization into simulated device coordinates
  • role/label/value/identifier/selected/enabled/hittable mapping compatible with existing selectors where possible
  • clear degraded/confidence metadata when root selection or traversal is partial
  • no silent fallback that hides XCTest failures without diagnostics

Validation

Add coverage for:

  • Settings app snapshot shape and actionable rows
  • React Navigation example snapshots and common interactions
  • a fixture or repro screen where XCTest snapshot fails or compact output is root-only
  • modals, keyboard, scroll views, and multiple Simulator windows/devices
  • fallback diagnostics and no-fallback behavior on physical devices

Notes

This should be a separate follow-up to the current bounded XCTest work. The XCTest implementation should remain honest and fast when it cannot inspect a tree; this fallback is for regaining simulator semantic coverage through a different backend.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type
    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions