Skip to content

Add support for accessibility navigation gestures (TalkBack)#2144

Merged
sds100 merged 5 commits into
developfrom
claude/issue-262-talkback-gestures
May 29, 2026
Merged

Add support for accessibility navigation gestures (TalkBack)#2144
sds100 merged 5 commits into
developfrom
claude/issue-262-talkback-gestures

Conversation

@keymapper-ai
Copy link
Copy Markdown
Collaborator

Closes #262

Summary

  • Adds a new "TalkBack gesture" action (ActionId.TALKBACK_GESTURE) that lets users trigger TalkBack navigation gestures via a key mapping.
  • Supports all 25 gestures listed in the issue: 1-finger swipes (4), 1-finger angular/chained swipes (5), 2-finger gestures (4), 3-finger gestures (5), and 4-finger gestures (7).
  • The action is editable: tapping it opens a picker dialog showing each gesture's TalkBack action label and its default gesture name (e.g. "Move reading control up or backwards (Swipe up)").
  • Gestures are dispatched via the accessibility service's dispatchGesture() API using computed screen-center coordinates.
  • Error checking shows a warning if the TalkBack app is not installed or disabled.

Files changed

File Change
TalkBackGestureType.kt New enum of 25 gesture types
TalkBackGestureStrings.kt String-resource lookup helpers for action labels and gesture names
ActionId.kt New TALKBACK_GESTURE entry
ActionData.kt New ActionData.TalkBackGesture(gesture) data class
ActionEntity.kt New EXTRA_TALKBACK_GESTURE_TYPE extra constant
ActionDataEntityMapper.kt Bidirectional entity ↔ data mapping
ActionUtils.kt Category (INTERFACE), title, icon (Icons.Outlined.Accessibility), editability
ActionUiHelper.kt Human-readable instance title (e.g. "TalkBack: Move reading control up or backwards")
ActionErrorSnapshot.kt Error if TalkBack package not installed/enabled
CreateActionDelegate.kt Picker dialog to choose gesture at action-creation time
PerformActionsUseCase.kt Dispatches gesture at execution time
IAccessibilityService.kt New performTalkBackGesture() method in interface
BaseAccessibilityService.kt Full gesture dispatch implementation using dispatchGesture()
strings.xml Action title, formatted title, picker title, and 50 label strings (25 action labels + 25 gesture names)
CHANGELOG.md Unreleased entry

Known limitations

  • Samsung TalkBack: Samsung ships its own TalkBack variant under a different package name. The error check only covers the standard Google TalkBack package (com.google.android.marvin.talkback).
  • Gesture timing: Tap durations and swipe speeds are approximations computed from screen density. They work for standard TalkBack configurations but may need tuning for non-default TalkBack gesture sensitivity settings.
  • Multi-finger tap stroke count: Android limits GestureDescription to 10 strokes. Four-finger triple-tap would require 12 strokes and is not included in the gesture set for this reason. The implemented set stays within this limit.
  • Coordinate-based dispatch: Gestures are dispatched relative to the screen center and are not aware of focused elements, so they simulate physical gestures rather than semantic TalkBack commands.

Generated by Claude Code

claude and others added 5 commits May 24, 2026 00:38
@sds100 sds100 self-requested a review May 29, 2026 14:52
@sds100 sds100 merged commit 0f5436c into develop May 29, 2026
4 checks passed
@sds100 sds100 deleted the claude/issue-262-talkback-gestures branch May 29, 2026 15:17
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Add support for accessibility navigation gestures (talkback)

3 participants