diff --git a/docs/articles/uiaccessibility.md b/docs/articles/uiaccessibility.md index 4eaf8ba86..334b3376d 100644 --- a/docs/articles/uiaccessibility.md +++ b/docs/articles/uiaccessibility.md @@ -75,9 +75,26 @@ need to be materialized as `UIWidget` objects merely for accessibility. While a cell is being edited, its editor widget is exposed as the only child of the virtual row (lists and trees) or cell (tables) it edits, and keyboard focus is reported on the editor. -Text editing support currently covers `UITextInput` and `UITextEdit`, including caret and selection -queries/actions and native change notifications. `UICodeEditor` and an end-to-end screen-reader -workflow in ecode are not part of this initial foundation. +Text editing support covers `UITextInput`, `UITextEdit` and `UICodeEditor`, including caret and +selection queries/actions and native change notifications. A code editor with a file is named after +that file unless it has a label. Setting its text through an assistive client replaces the document +contents as one undoable edit, so the file path and undo history are kept. + +Text queries read only the range a client asks for, through a line index that is rebuilt once per +edit, so moving the caret in a large file does not copy the file. Edits reach Linux clients as exact +insert and delete records taken from the document. A wholesale replacement (a reset, a reload, or +many edits in one frame, such as replace-all) is reported once as "everything removed, everything +inserted". Windows and macOS need UTF-16 offsets, so they convert the text once per edit and reuse +that copy for every query until the next edit. + +Controls whose content is their own name, value and state (buttons, check boxes, radio buttons, +labels, tabs, menu items, sliders, spin boxes, progress bars and images) do not expose their +internal widgets, so a button's text is read once, as its name, and not again as a child label. + +Menu items expose their keyboard shortcut (AT-SPI key binding, UIA `AcceleratorKey`, macOS menu +command character and modifiers). Any widget inside a `UIScrollView` offers a scroll-into-view +action, which scrolls every enclosing scroll view the least distance needed. The same behavior is +available to applications as `UIScrollView::scrollIntoView( node )`. ## Accessible names and descriptions @@ -90,9 +107,26 @@ Use `aria-label` when a widget's visible content does not provide a useful name. + aria-description="Multiline editor. Press Control Tab to move to the next control." /> ``` +When the name is visible elsewhere, point at it instead of repeating it. `aria-labelledby` names a +widget after the widget with that id, and `aria-describedby` uses another widget's text as the +description. Both ids are resolved from the scene root. When the referenced widget's text changes, +or it is added, removed or given another id, assistive clients are told that the dependent widget's +name or description changed: + +```xml + + + +``` + +Names resolve in this order: `aria-labelledby`, `aria-label`, the widget's own text (button text, +label text, window title, code editor file name), then its tooltip. An icon-only button with a +tooltip is therefore named by the tooltip. A tooltip that did not become the name is used as the +description when no other description is set. + Use `aria-hidden="true"` for decorative or redundant widgets that should not appear in the native accessibility hierarchy: @@ -107,8 +141,12 @@ These properties are also available through C++: widget->setAccessibilityLabel( "Project name" ); widget->setAccessibilityDescription( "Enter the project display name." ); widget->setAccessibilityHidden( false ); +widget->setAccessibilityLabelledBy( "user-label" ); +widget->setAccessibilityDescribedBy( "user-hint" ); ``` +Setting a property to its current value does nothing, so re-applying styles is free. + Prefer a concise label that identifies the control. A description should add information rather than repeat the label or visible text. @@ -147,7 +185,35 @@ Standard widgets notify the accessibility manager when relevant UI state changes - model and child hierarchy changes. Bounds are queried on demand. Routine position and size changes do not emit per-widget native -layout notifications; this avoids event floods while scrolling or running layout. +layout notifications; this avoids event floods while scrolling or running layout. A window resize is +announced once, on the window root. + +### Announcements and live regions + +Use an announcement for an event the user should hear without moving focus, such as a finished +build: + +```cpp +sceneNode->announceForAccessibility( "Build finished" ); +sceneNode->announceForAccessibility( "Build failed", AccessibilityLive::Assertive ); +``` + +`Polite` waits until the screen reader finishes speaking; `Assertive` interrupts it. Announcements +use UIA notifications (Windows 10 1709 and later), the AT-SPI `object:announcement` signal +(at-spi2-core 2.46 and later, read by Orca) and `NSAccessibilityAnnouncementRequestedNotification` +on macOS. Older systems ignore them. + +For text that changes in place, such as a status bar, mark it as a live region instead: + +```xml + +``` + +Name and value changes of a widget inside a live region (`polite` or `assertive`) are announced, +unless the widget or one of its ancestors is `aria-hidden`. +Several changes to the same widget in one frame produce one announcement with the final text, so a +progress label updated every frame does not flood the screen reader. Both APIs are free while no +assistive client is active. Applications using the standard widget APIs receive these notifications automatically. A custom control that changes semantic state outside the standard paths may notify its scene accessibility @@ -162,9 +228,9 @@ destroyed. Accessible controls should be keyboard focusable and have a logical focus order. Standard Tab and Shift+Tab navigation moves between controls. -Multiline text editors reserve Tab for inserting indentation. Use Mod+Tab to move to the next -focusable widget and Mod+Shift+Tab to move to the previous one. `Mod` means Command on macOS and -Control on other desktop platforms. +Multiline text editors reserve Tab for inserting indentation. Use Control+Tab to move to the next +focusable widget and Control+Shift+Tab to move to the previous one. This matches AppKit on macOS, +where Command+Tab is reserved for switching applications. Screen readers may provide an additional browse or navigation mode. Their commands and speech settings are controlled by the screen reader and are separate from eepp's keyboard focus handling. @@ -172,8 +238,14 @@ settings are controlled by the screen reader and are separate from eepp's keyboa ## Testing an application The `eepp-ui-accessibility` example contains representative controls, model-backed views, dynamic -metadata, and multiple native windows. Build and run the optimized release target before using a -platform accessibility tool. +metadata, a live region, and multiple native windows. Build and run the optimized release target +before using a platform accessibility tool. + +The widget inspector (`UIWidgetInspector`) has an Accessibility tab with the selected widget's +resolved role, name, description, value, shortcut, relations, states and actions, and an Audit tab +that lists focusable controls without an accessible name and `aria-labelledby` / +`aria-describedby` ids that match nothing. Selecting an issue selects the widget in the tree. The +same check is available to tests as `AccessibilityWidgetResolver::audit( root )`. ### Linux @@ -222,8 +294,13 @@ and destruction. ## Performance behavior Native accessibility backends are query-centered and create provider objects lazily. When no -native client has interacted with the application, widget notifications take a cached inactive -fast path and avoid resolving accessibility data or walking the widget hierarchy. +native client has interacted with the application, widget notifications take an inline inactive +fast path: one atomic load, no call, no allocation, and no accessibility data resolved. Editing +text, building widgets (spin boxes included), deleting widget hierarchies and updating models +allocate no more than they would without accessibility, as measured by C++ `new` on the UI thread +(see the architecture guide's benchmark notes). When the last client disconnects, rows it +queried are released, so model updates stop paying for them; a client connecting later starts +fresh. Model child counts and individual child queries do not instantiate every accessible row. Queried rows retain identities across insertion and removal, and model sibling indexes are resolved @@ -245,7 +322,10 @@ must not depend on rendering, and native queries or notifications must never sta The first implementation focuses on the semantics used by standard eepp desktop widgets. Native platform APIs contain additional advanced interfaces that are not yet exposed uniformly, including -some rich-document formatting, complex table metadata, and container-specific scrolling APIs. +some rich-document formatting, complex table metadata, and container scroll patterns (scroll +position and paging, as opposed to scrolling an element into view). HTML content (`UIRichText`, +headings, links, lists) is not yet exposed as semantic elements. Relations are used to compute names +and descriptions but are not published as native relation sets. Do not advertise a native pattern or capability unless its required behavior is implemented. Assistive technologies can still navigate the semantic hierarchy and operate the supported diff --git a/docs/articles/uiaccessibilityarchitecture.md b/docs/articles/uiaccessibilityarchitecture.md index 07b66cb6a..9b3000374 100644 --- a/docs/articles/uiaccessibilityarchitecture.md +++ b/docs/articles/uiaccessibilityarchitecture.md @@ -135,10 +135,47 @@ materializing a whole child list. view) or drops it (implementation children, detached widgets). 4. `AccessibilityManager::notify()` invalidates caches, appends the event to the frame's queue and hands it to the backend immediately through `AccessibilityBackend::onEvent()`. -5. The backend emits the native notification. AT-SPI emits D-Bus signals right away, except text - changes, which are diffed once per frame. UIA and macOS queue the event and raise it from their - next `update()`, on the UI thread and outside any widget or model callback. The manager's queue - is cleared at the end of every `AccessibilityManager::update()`. +5. The backend emits the native notification. AT-SPI emits D-Bus signals right away (caret and + selection changes are compared once per frame). UIA and macOS queue the event and raise it from + their next `update()`, on the UI thread and outside any widget or model callback. The manager's + queue is cleared at the end of every `AccessibilityManager::update()`. + +Text edits take an extra step before their `ValueChanged`. `UICodeEditor` and `UITextInput` pass +the document's `DocumentContentChange` (range and inserted text) to +`UIWidget::notifyAccessibilityTextChanged()`. The removed text is not part of the change record, +so consumers that queue records (LSP clients) never copy it: the manager reads it, while the +document is still notifying, from `TextDocument::getNotifiedRemovedText()`. `AccessibilityManager::onTextChanged()` turns the +first four per element and frame into exact `AccessibilityTextChange` records (code-point offset, +removed, inserted). After that, and for resets, loads, reloads and document swaps, it sends one +whole-text change for the rest of the frame, so a replace-all costs one notification. +`AccessibilityBackend::onTextChanged()` receives them; AT-SPI turns exact ones into `text-changed` +signals immediately and a whole-text one into delete-all plus insert-all at its next update. A +backend returns false for a change it ignores, and an ignored change does not count toward the +frame's four. AT-SPI's `SetTextContents` reports its replacement as one minimal diff, so it +suppresses the element's records through `AccessibilityManager::setSuppressedTextChanges()` while +the action runs. Node refs are manager-local, so an editor in another window sharing the document +still reports the change. + +Announcements take a parallel path. `UISceneNode::announceForAccessibility()` and live regions (a +`NameChanged` or `ValueChanged` from a widget with an `aria-live` ancestor and no `aria-hidden` +one) append to `AccessibilityManager`'s announcement queue. A live-region entry remembers its +source widget, and a later change from the same widget in the same frame replaces its text instead +of queueing another entry. `AccessibilityManager::update()` hands the queue to +`AccessibilityBackend::announce()` after the backend's own `update()`, so the frame's focus and +state changes reach the client first. Live-region entries are checked again at delivery and +dropped when their source was destroyed, hidden or made `aria-hidden` in the meantime. + +Names and descriptions that come from `aria-labelledby` / `aria-describedby` depend on another +widget. When a client reads such a widget's info, the manager remembers the referenced id (ids, +not pointers, so no entry can dangle). A name change of a widget with a remembered id, an id +change, or the arrival, removal or deletion of a subtree containing one, makes the manager scan +the scene for dependents and notify `NameChanged` or `DescriptionChanged` for them. While no +client has read a dependent, the check is one lookup in an empty set. `UIWidget::setId()` asks +`AccessibilityManager::isRelationTarget()` (an allocation-free hash lookup) before renaming, and +keeps a copy of the old id only when it is remembered. Dependents are notified after the id has +changed, for the old id and then the new one (which may complete a relation that pointed at a +missing widget), so a backend that reads the node from inside the notification sees the new +relation. ## Rules the implementation relies on @@ -148,17 +185,46 @@ These are the invariants that past bugs came from. Keep them when changing the c Applications without an assistive client must pay close to nothing. -- `UIWidget::notifyAccessibilityEvent()` first checks `UISceneNode::hasActiveAccessibilityClients()`, - a cached bit (`AccessibilityClientActive`). Nothing else runs while it is clear. +- `UIWidget::notifyAccessibilityEvent()`, `notifyAccessibilityTextChanged()` and + `notifyAccessibilityWholeTextChanged()` are inline. They test one process-wide counter of root + scenes with an active client (`UIWidget::isAccessibilityActive()`, a relaxed atomic load) and + only then call the out-of-line `deliver*()` implementation, which checks the scene's own bit + (`UISceneNode::hasActiveAccessibilityClients()`, `AccessibilityClientActive`). Callers that + compute a payload before notifying test `isAccessibilityActive()` first. Pick the event by + overriding, not by checking the type at the call site: `UITextView::onTextChanged()` reports + `NameChanged` and `UITextInput` overrides it to report `ValueChanged`. Release code for a dormant + `UITextView::onTextChanged()` is one load, one test and a jump. - The bit is set when a backend reports a client (`hasActiveNativeClients()`, refreshed in - `UISceneNode::update()`) or when a native query arrives (`onNativeClientObserved()`). + `UISceneNode::update()`) or when a native query arrives (`onNativeClientObserved()`). Every + change goes through `UISceneNode::setAccessibilityClientActive()`, which keeps the process-wide + counter in sync and, when the last client goes away, calls + `AccessibilityManager::onClientsDisconnected()` (see [rule 6](#6-what-survives-a-disconnection)). - Backends decide what "active" means. AT-SPI tracks client bus names and clears the bit when they disconnect. UIA requires window-specific activity. macOS has no disconnect signal, so the bit stays set after the first query. +- Without a client, `AccessibilityManager::notify()` queues nothing: only identity invalidation + (`Destroyed` without a related node) still reaches the backend, which detaches native wrappers + on it. Removing a widget a disconnected client had seen invalidates its identity without + resolving or registering its parent to announce the removal. While only the scene root has an + identity, deleting any other widget returns before walking its ancestors, unless a client is + active and has read a relation (`aria-labelledby`, `aria-describedby`) whose target may be in + the deleted subtree. +- Lifecycle bookkeeping never creates a manager: destruction, scene and host changes use + `UISceneNode::getExistingAccessibilityManager()`, and the const `getAccessibilityManager()` does + not create one at any nesting level. - Do not add per-frame or per-layout work that runs before this check. Bounds are queried on demand; there is deliberately no per-widget bounds notification. +- `UISceneNode::announceForAccessibility()` returns before doing anything while no client is + active, but a caller that formats its message first still pays for that formatting. -`benchmark_accessibility_inactive.py` guards this path in CI. +`benchmark_accessibility_inactive.py` compares the enabled and environment-disabled backends of the +same build: both run the same compiled-in hooks, so it cannot show costs they share. +`benchmark_accessibility_dormant.py` measures those directly and can compare against the same +benchmark built against another tree (`--compare LABEL=PATH`), such as an exported develop +revision. Its allocation counts and bytes cover ordinary C++ `new`/`new[]` on the measuring thread +only: not `malloc`, other threads or background work such as AT-SPI initialization. "No more +allocations than without accessibility" means exactly that measurement. A run passes only when +every sample exits cleanly and reports every expected measurement, guarded scenarios included. ### 2. Teardown happens once, top-down @@ -187,7 +253,8 @@ sequence that happened. Only adjacent query-again hints (`ModelChanged`, `Childr events: dropping the second half of add/remove/add or on/off/on leaves clients with stale state. Backends that need batching do it themselves, per frame: -- AT-SPI diffs each changed text once. +- AT-SPI compares each text's caret and selection once, and reports a whole-text replacement + once. - UIA raises one focus change, for the final keyboard focus. - macOS keeps only the last focus change, and posts every other non-structural notification at most once per (event, element, related element). AppKit notifications carry no state, so a @@ -213,7 +280,28 @@ The manager, the resolver and every widget are UI-thread only. - Never walk the whole widget tree or the whole model per event or per query. Prefer the event dispatcher's focus node, `getIndexInParent()` and source lookups over scanning children. - Building `AccessibilityNodeInfo` with the value copies the text of a text widget; request it only - where the value is needed. + where the value is needed. Text interfaces use `getTextLength()`, `getTextRange()` and + `getTextLineBounds()`, which read only what they return. Offsets convert through a line index of + the document (`lineStarts()` in the resolver), keyed by `getTextRevision()`: the document's UUID + and modification id, which every content change bumps. Backends that need a converted copy + (UTF-16 for UIA and AppKit) cache it per revision. +- Leaf controls (`AccessibilityWidgetResolver::isLeafRole()`) have no exposed children. Events and + focus from inside one are reported on the control (`getLeafOwner()`), so the inner widgets never + get identities. + +### 6. What survives a disconnection + +When a scene's last client disconnects, `onClientsDisconnected()` resets every model-view source: +queried rows are dropped together with their persistent model registrations, and backends detach +their native wrappers (`onSourceInvalidated()`). Without that, the model would keep updating one +persistent handle per queried row on every insert, delete and move for as long as it lives. +Identities are never reused, so a reconnecting client starts from fresh row identities and a stale +one stays invalid. Widget identities and metadata (labels, relations, live regions) are kept. + +The model only drops a persistent handle when all its registrations are released +(`PersistentModelIndex::release()`); handles other consumers registered are unaffected. A queried +row's registration is released when its node is dropped, so pruned, reset and destroyed sources do +not leave handles behind either. ## Backends @@ -223,11 +311,17 @@ One `AtSpi::AtSpiApplication` is shared by all windows of the process and owns a connection to the accessibility bus. libdbus is loaded at runtime (`DBusLibrary`), so there is no build-time D-Bus dependency. Object paths encode `/org/eepp/a11y///`. +A peer other than the registry that calls the application becomes a client. Its disconnection is +watched with a `NameOwnerChanged` match on its own unique name, not a bus-wide match that would +wake the application for every program joining or leaving the bus. A non-blocking `NameHasOwner` +check sent after the match covers a client that left before the match was installed: the bus +processes both requests in order. + | File | Contents | | --- | --- | | `accessibilitybackendatspi.hpp` | `DBusLibrary` and the `AtSpiApplication` declaration, with members grouped by implementing file. | | `accessibilitybackendatspi.cpp` | Connection setup and registry embedding, I/O thread, per-frame dispatch, client tracking, `AtSpiAccessibilityBackend`. | -| `accessibilitybackendatspievents.cpp` | Manager events to `org.a11y.atspi.Event.*` signals; text snapshots and once-per-frame text diffs (`text-changed`, `text-caret-moved`, `text-selection-changed`). | +| `accessibilitybackendatspievents.cpp` | Manager events to `org.a11y.atspi.Event.*` signals; text change records to `text-changed`, and once-per-frame caret and selection comparison (`text-caret-moved`, `text-selection-changed`). Keeps no text contents. | | `accessibilitybackendatspitree.cpp` | Path/ref conversion, navigation, role and state mapping, action names, D-Bus marshalling helpers. | | `accessibilitybackendatspimethods.cpp` | Incoming calls: `handleMessage()` routes to one handler per interface (`Accessible`, `Component`, `Action`, `Text`, `EditableText`, `Application`, `Properties`, `Cache`). | @@ -267,6 +361,11 @@ every backend: `role()` and `roleName()` in `accessibilitybackendatspitree.cpp`, AT-SPI `appendStates()` / `actionName()`, UIA patterns in the provider's `QueryInterface()` and `GetPatternProvider()`, macOS action names and attributes. +**Announce something.** Call `UISceneNode::announceForAccessibility()`, or mark the widget that +changes with `aria-live`. A backend implements `AccessibilityBackend::announce()`; AT-SPI emits +`Announcement` on the root object, UIA calls `UiaRaiseNotificationEvent` (looked up at runtime), +macOS posts `NSAccessibilityAnnouncementRequestedNotification` on the window. + **Answer a new AT-SPI method.** Add it to the matching `handle()` in `accessibilitybackendatspimethods.cpp`. Return `1` for "not handled" so the dispatcher replies with `UnknownMethod`. @@ -284,22 +383,42 @@ or wrong. | macOS backend semantics | `src/tests/unit_tests/accessibility_macos_tests.mm` | macOS unit tests. | | AT-SPI end to end | `projects/scripts/test_atspi.py` (default, `--multi-window`, `--close-primary`) and `benchmark_atspi.py` | Linux CI, inside `dbus-run-session` and Xvfb. | | Inactive overhead | `projects/scripts/benchmark_accessibility_inactive.py` | Linux CI. | +| Dormant allocations and costs | `src/benchmarks/accessibility_dormant_benchmark.cpp` (`eepp-benchmarks --filter='AccessibilityDormant.*'`), summarized by `projects/scripts/benchmark_accessibility_dormant.py [--compare LABEL=PATH]` | Locally, Release builds. Allocation counts are deterministic and asserted where they guard a fix; times are indicative. | | UIA end to end | `src/tests/windows_accessibility/main.cpp` (`eepp-windows-accessibility-tests`) | Windows CI. Under Wine it builds and finds the window, but Wine's UIA client lacks several APIs, so most checks report `E_NOTIMPL`. | | NSAccessibility end to end | `projects/scripts/test_macos_accessibility.swift`; `benchmark_macos_accessibility.swift` locally | macOS CI. | Useful tools: -- the runtime inspector's *Accessibility* tab shows exactly what a widget projects; +- the runtime inspector's *Accessibility* tab shows exactly what a widget projects, and its *Audit* + tab (`AccessibilityWidgetResolver::audit()`) lists unnamed focusable controls and broken + `aria-labelledby` / `aria-describedby` ids; - `EEPP_DISABLE_ACCESSIBILITY=1` runs with the null backend, the baseline for performance and behavior comparisons; - Accerciser (Linux), Accessibility Inspector (macOS) and Inspect / Accessibility Insights (Windows) show the native tree; - to run the AT-SPI scripts without touching your desktop session, wrap them in - `dbus-run-session` as CI does. + `dbus-run-session` as CI does; +- `EEPP_ACCESSIBILITY_TRACE=1` prints client activity transitions, and AT-SPI client connections + and disconnections, to stderr. It shows what activates an application that has no screen reader + running. ## Known limitations - Enumerating all children of a model view is O(rows). - macOS cannot detect that a client went away, so its active bit stays set after the first query. -- `UICodeEditor` is not exposed; only `UITextInput` and `UITextEdit` provide text. + Any process that queries one of the application's accessibility elements (not only screen + readers) activates it for the rest of its life. +- Windows refreshes `UiaClientsAreListening()` on every scene update; a cheaper cadence needs + native validation that listener changes are still observed promptly. +- Linux connects to the accessibility bus at startup, whether or not a client will ever attach, + so applications are discoverable by tools started later and by clients that never set the + `org.a11y.Status` flags. Gating the connection on those flags (as Qt does when no X11 root + window atom publishes the bus) would trade that discovery for startup work; it is a policy + choice, not a transparent optimization. Initialization runs off the UI thread: on the scene's + thread pool, or on a detached thread when the scene has none. Either way the task shares + ownership of the AT-SPI application, so tearing the backend down never waits for pending D-Bus + calls or their timeouts; whichever owner releases it last destroys it. +- The line index is rebuilt in full after each edit (about 3 ms for a 100,000-line document), + paid only while a client is querying that document. +- HTML content (`UIRichText`, headings, links, lists) is not exposed as semantic elements. - Only one cell editor (the view's active editor) is exposed inside a model view. diff --git a/include/eepp/system/uuid.hpp b/include/eepp/system/uuid.hpp index 927206967..acca29248 100644 --- a/include/eepp/system/uuid.hpp +++ b/include/eepp/system/uuid.hpp @@ -31,6 +31,10 @@ class EE_API UUID { bool isInitialized() const; + uint64_t high() const { return mHigh; } + + uint64_t low() const { return mLow; } + bool operator==( const UUID& other ) const { return mHigh == other.mHigh && mLow == other.mLow; } diff --git a/include/eepp/ui/accessibility/accessibility.hpp b/include/eepp/ui/accessibility/accessibility.hpp index 69e67d281..a2b54d85c 100644 --- a/include/eepp/ui/accessibility/accessibility.hpp +++ b/include/eepp/ui/accessibility/accessibility.hpp @@ -101,6 +101,10 @@ constexpr AccessibilityActions accessibilityActionMask( AccessibilityAction acti return 1u << static_cast( action ); } +/** Politeness of an announcement or live region: Polite waits for the client to finish speaking, + * Assertive interrupts it. */ +enum class AccessibilityLive : Uint8 { Off, Polite, Assertive }; + enum class AccessibilityEvent : Uint8 { Created, Destroyed, @@ -117,6 +121,20 @@ enum class AccessibilityEvent : Uint8 { BoundsChanged, }; +/** A keyboard shortcut, both as displayed and decomposed for native APIs that want their own + * format (GTK accelerators on AT-SPI, command characters on macOS). */ +struct AccessibilityShortcut { + enum Modifier : Uint8 { Control = 1 << 0, Shift = 1 << 1, Alt = 1 << 2, Meta = 1 << 3 }; + + /** As displayed to the user (UTF-8), for example "Ctrl+S". */ + std::string text; + /** The platform key name (UTF-8), for example "S", "F5" or "Return". */ + std::string key; + Uint8 modifiers{ 0 }; + + bool empty() const { return text.empty(); } +}; + struct AccessibilityNodeRef { AccessibilitySourceId source{ 0 }; Uint64 id{ 0 }; @@ -145,11 +163,41 @@ struct AccessibilityTextInfo { bool valid{ false }; }; +/** Identifies one version of an element's text. Equal revisions have equal text, so backends can + * cache converted copies (UTF-16 for UIA and AppKit) instead of rebuilding them per query. */ +struct AccessibilityTextRevision { + /** The document's UUID halves: unlike its address, never reused by another document. */ + Uint64 documentHigh{ 0 }; + Uint64 documentLow{ 0 }; + Uint64 modification{ 0 }; + + bool isValid() const { return documentHigh != 0 || documentLow != 0; } + + bool operator==( const AccessibilityTextRevision& other ) const { + return documentHigh == other.documentHigh && documentLow == other.documentLow && + modification == other.modification; + } + + bool operator!=( const AccessibilityTextRevision& other ) const { return !( *this == other ); } +}; + +/** One edit, in code points: at `offset`, `removed` was replaced by `inserted`. An offset of -1 + * means the whole text was replaced (a reset, a reload, or too many edits in one frame). */ +struct AccessibilityTextChange { + Int32 offset{ -1 }; + String removed; + String inserted; + + bool isWholeText() const { return offset < 0; } +}; + struct AccessibilityNodeInfo { AccessibilityRole role{ AccessibilityRole::None }; String name; String description; String value; + /** Keyboard shortcut that activates the element. */ + AccessibilityShortcut shortcut; AccessibilityState states{ AccessibilityState::None }; AccessibilityActions actions{ 0 }; AccessibilityRangeInfo range; @@ -158,6 +206,14 @@ struct AccessibilityNodeInfo { bool boundsValid{ false }; }; +struct AccessibilityAnnouncement { + String message; + AccessibilityLive priority{ AccessibilityLive::Polite }; + /** The live-region widget that produced it; invalid for an explicit announcement. A newer + * change from the same widget replaces its pending announcement. */ + AccessibilityNodeRef source; +}; + struct AccessibilityActionRequest { AccessibilityAction action{ AccessibilityAction::Focus }; String value; diff --git a/include/eepp/ui/accessibility/accessibilitymanager.hpp b/include/eepp/ui/accessibility/accessibilitymanager.hpp index dd6f2feb0..9fc9a9fb1 100644 --- a/include/eepp/ui/accessibility/accessibilitymanager.hpp +++ b/include/eepp/ui/accessibility/accessibilitymanager.hpp @@ -9,6 +9,10 @@ namespace EE { namespace Scene { class Node; }} // namespace EE::Scene +namespace EE { namespace UI { namespace Doc { +struct DocumentContentChange; +}}} // namespace EE::UI::Doc + namespace EE { namespace UI { class UISceneNode; @@ -41,6 +45,29 @@ class EE_API AccessibilityManager { AccessibilityTextInfo getTextInfo( AccessibilityNodeRef ref ) const; + /** Ranged text access in code points. Prefer these over getNodeInfo()'s value for text + * elements: they never copy more than the requested range. */ + Int32 getTextLength( AccessibilityNodeRef ref ) const; + + String getTextRange( AccessibilityNodeRef ref, Int32 start, Int32 end ) const; + + /** The line containing `offset`: [start, end), end past its newline. */ + bool getTextLineBounds( AccessibilityNodeRef ref, Int32 offset, Int32& start, + Int32& end ) const; + + AccessibilityTextRevision getTextRevision( AccessibilityNodeRef ref ) const; + + /** Reports an edit of a text widget's document, or nullptr when its whole text was replaced. + * The first few edits per widget and frame are forwarded exactly; later ones collapse into a + * single whole-text change, so a replace-all costs one notification. */ + void onTextChanged( UIWidget* widget, const Doc::DocumentContentChange* change ); + + /** While set, text changes of this element are not forwarded: the caller reports them itself + * (AT-SPI SetTextContents sends one minimal diff). Node refs are local to this manager, so + * another window's element sharing the document still reports its changes. Pass an invalid + * ref to clear. */ + void setSuppressedTextChanges( AccessibilityNodeRef ref ); + size_t getChildCount( AccessibilityNodeRef ref ); AccessibilityNodeRef getChild( AccessibilityNodeRef ref, size_t index ); @@ -64,8 +91,17 @@ class EE_API AccessibilityManager { /** Updates the scene's cached fast-path flag after an actual native accessibility query. */ void onNativeClientObserved(); + /** The scene's last client went away: releases what only clients needed, such as queried + * model rows and their persistent model registrations. Identities are never reused, so a + * reconnecting client cannot reach a stale row. */ + void onClientsDisconnected(); + void update(); + /** Replaces the platform backend, which update() otherwise creates on first use. Intended + * for tests that observe what the manager delivers. */ + void setBackend( std::unique_ptr backend ); + UISceneNode* getSceneNode() const; void notify( AccessibilityNodeRef ref, AccessibilityEvent event, @@ -75,6 +111,30 @@ class EE_API AccessibilityManager { void clearPendingEvents(); + /** Asks assistive clients to speak a message without moving focus. Delivered on the next + * update(); a no-op while no client is active. */ + void announce( const String& message, AccessibilityLive priority = AccessibilityLive::Polite ); + + /** Queues the current text of a widget inside a live region, replacing any announcement the + * same widget queued earlier in this frame. */ + void onLiveRegionChanged( UIWidget* widget, AccessibilityLive priority ); + + const std::vector& getPendingAnnouncements() const; + + /** The widget with this id changed its name, appeared or went away. Widgets that a client + * read and that are named or described by it (aria-labelledby, aria-describedby) report + * NameChanged or DescriptionChanged. Dependents inside `excluded` are skipped. */ + void onRelationTargetChanged( const std::string& id, const Scene::Node* excluded = nullptr ); + + /** Whether an active client may hold a name or description resolved through the id with this + * hash, so its dependents must be told when that id goes away. Allocation-free; it matches + * exactly what onRelationTargetChanged() would act on. */ + bool isRelationTarget( String::HashType idHash ) const; + + /** A subtree appeared or is going away: notifies the dependents of each relation target in + * it, except those inside the subtree when it is going away. */ + void onRelationSubtreeChanged( const Scene::Node* subtree, bool removed ); + void onWidgetParentChange( UIWidget* widget ); void onWidgetRemovedFromParent( UIWidget* widget ); @@ -98,6 +158,15 @@ class EE_API AccessibilityManager { UnorderedMap> mSources; UnorderedMap mWidgetSources; std::vector mPendingEvents; + std::vector mPendingAnnouncements; + /** Edits forwarded this frame per text element; reset by update(). */ + std::vector> mTextChangeCounts; + static constexpr Uint32 MaxExactTextChangesPerFrame = 4; + AccessibilityNodeRef mSuppressedTextRef; + /** Ids referenced by aria-labelledby / aria-describedby of widgets a client has read. Only + * changes to these widgets can make a client's cached name or description stale. Ids, not + * widget pointers: entries never dangle, and a stale one costs one scene scan. */ + mutable UnorderedSet mRelationTargets; struct ChildrenCacheEntry { std::vector children; AccessibilityNodeRef parent; @@ -117,6 +186,9 @@ class EE_API AccessibilityManager { /** Returns whether any identity or model source in the subtree was released. */ bool removeSubtreeIdentities( Scene::Node* node ); + + /** Whether a client can receive events: without one, only invalidation runs. */ + bool hasActiveClients() const; }; }} // namespace EE::UI diff --git a/include/eepp/ui/accessibility/accessibilitywidgetresolver.hpp b/include/eepp/ui/accessibility/accessibilitywidgetresolver.hpp index 33cb31d11..3b0e12d72 100644 --- a/include/eepp/ui/accessibility/accessibilitywidgetresolver.hpp +++ b/include/eepp/ui/accessibility/accessibilitywidgetresolver.hpp @@ -7,21 +7,48 @@ namespace EE { namespace Scene { class Node; }} // namespace EE::Scene +namespace EE { namespace UI { namespace Doc { +class TextDocument; +class TextPosition; +}}} // namespace EE::UI::Doc + namespace EE { namespace UI { class UIWidget; +struct AccessibilityIssue { + const UIWidget* widget{ nullptr }; + String message; +}; + /** Resolves standard widget semantics directly from the live widget instance. */ class EE_API AccessibilityWidgetResolver { public: static AccessibilityRole getRole( const UIWidget* widget ); + /** A readable role name for tools and logs, for example "CheckBox". */ + static const char* getRoleName( AccessibilityRole role ); + + /** Controls whose content is fully described by their own name, value and state. Their + * implementation children (a button's text view, a menu item's shortcut label) are not exposed. + */ + static bool isLeafRole( AccessibilityRole role ); + + /** The outermost leaf control containing the node, or nullptr when no ancestor is a leaf. */ + static UIWidget* getLeafOwner( const Scene::Node* node ); + + /** Whether the widget or any ancestor is hidden from accessibility (aria-hidden). */ + static bool isHiddenFromAccessibility( const UIWidget* widget ); + static String getName( const UIWidget* widget ); static String getDescription( const UIWidget* widget ); static String getValue( const UIWidget* widget ); + /** The keyboard shortcut of a menu item; empty for every other widget. */ + static AccessibilityShortcut getShortcut( const UIWidget* widget ); + static AccessibilityRangeInfo getRange( const UIWidget* widget ); static AccessibilityTextInfo getText( const UIWidget* widget ); @@ -29,6 +56,24 @@ class EE_API AccessibilityWidgetResolver { /** Metadata-only queries avoid scanning document lines to calculate offsets. */ static AccessibilityTextInfo getText( const UIWidget* widget, bool includeOffsets ); + /** The document behind a text element; nullptr for other widgets and for password inputs, + * whose text is never exposed. */ + static const Doc::TextDocument* getTextDocument( const UIWidget* widget ); + + /** Ranged text access, in code points. None of these copy more than the requested range. */ + static Int32 getTextLength( const UIWidget* widget ); + + static String getTextRange( const UIWidget* widget, Int32 start, Int32 end ); + + /** The line containing `offset`: [start, end), end past its newline. */ + static bool getTextLineBounds( const UIWidget* widget, Int32 offset, Int32& start, Int32& end ); + + static AccessibilityTextRevision getTextRevision( const UIWidget* widget ); + + /** The code-point offset of a document position. */ + static Int32 getTextOffset( const Doc::TextDocument& document, + const Doc::TextPosition& position ); + static AccessibilityState getState( const UIWidget* widget ); static AccessibilityActions getActions( const UIWidget* widget ); @@ -45,6 +90,16 @@ class EE_API AccessibilityWidgetResolver { static UIWidget* getEventTarget( UIWidget* widget, AccessibilityEvent event ); static bool performAction( UIWidget* widget, const AccessibilityActionRequest& request ); + + /** Lists common authoring problems in a widget tree: controls without an accessible name and + * aria-labelledby / aria-describedby ids that resolve to nothing. Hidden and invisible + * subtrees are skipped, as are the implementation children of model views. */ + static std::vector audit( const UIWidget* root ); + + private: + /** The widget's own name: its label, else its native text, else its tooltip. Relations are + * not followed, so a labelled-by target that is itself labelled by another cannot loop. */ + static String getOwnName( const UIWidget* widget ); }; }} // namespace EE::UI diff --git a/include/eepp/ui/css/propertyids.hpp b/include/eepp/ui/css/propertyids.hpp index 24d496b36..b122e0c2d 100644 --- a/include/eepp/ui/css/propertyids.hpp +++ b/include/eepp/ui/css/propertyids.hpp @@ -59,6 +59,9 @@ enum class PropertyId : Uint16 { AccessibilityLabel, AccessibilityDescription, AccessibilityHidden, + AccessibilityLabelledBy, + AccessibilityDescribedBy, + AccessibilityLive, LayoutWeight, LayoutGravity, LayoutWidth, diff --git a/include/eepp/ui/doc/textdocument.hpp b/include/eepp/ui/doc/textdocument.hpp index 20e682f13..fcb860471 100644 --- a/include/eepp/ui/doc/textdocument.hpp +++ b/include/eepp/ui/doc/textdocument.hpp @@ -775,6 +775,12 @@ class EE_API TextDocument { std::string toUtf8String(); + /** The text removed by the edit being notified: valid only during + * Client::onDocumentTextChanged(), and empty for insertions. Clients that need it later must + * copy it. Change records do not carry it, so consumers that queue them (such as LSP clients) + * never copy text they do not use. */ + const String& getNotifiedRemovedText() const; + protected: friend class TextUndoStack; friend class FoldRangeService; @@ -794,6 +800,8 @@ class EE_API TextDocument { Mutex mClientsMutex; mutable Mutex mLinesMutex; mutable std::shared_ptr mDocumentMutex; + /** The removed text of the change being notified; see getNotifiedRemovedText(). */ + const String* mNotifiedRemovedText{ nullptr }; TextFormat::Encoding mEncoding{ TextFormat::Encoding::UTF8 }; TextFormat::LineEnding mLineEnding{ TextFormat::LineEnding::LF }; std::atomic mLoading{ false }; @@ -855,7 +863,7 @@ class EE_API TextDocument { void notifyDocumentReset(); - void notifyTextChanged( const DocumentContentChange& ); + void notifyTextChanged( const DocumentContentChange&, const String* removed = nullptr ); void notifyCursorChanged( TextPosition selection = TextPosition() ); diff --git a/include/eepp/ui/models/model.hpp b/include/eepp/ui/models/model.hpp index 52ef891ca..e668f634a 100644 --- a/include/eepp/ui/models/model.hpp +++ b/include/eepp/ui/models/model.hpp @@ -134,6 +134,9 @@ class EE_API Model { std::weak_ptr registerPersistentIndex( ModelIndex const& ); + /** Releases one registration of the handle; the handle is dropped when none remain. */ + void releasePersistentIndex( const std::shared_ptr& handle ); + void beginInsertRows( ModelIndex const& parent, int first, int last ); void beginInsertColumns( ModelIndex const& parent, int first, int last ); void beginMoveRows( ModelIndex const& sourceParent, int first, int last, diff --git a/include/eepp/ui/models/persistentmodelindex.hpp b/include/eepp/ui/models/persistentmodelindex.hpp index 06c5dbd38..9b19cdf80 100644 --- a/include/eepp/ui/models/persistentmodelindex.hpp +++ b/include/eepp/ui/models/persistentmodelindex.hpp @@ -16,6 +16,9 @@ class PersistentHandle { PersistentHandle( ModelIndex const& index ) : mIndex( index ) {} ModelIndex mIndex; + /** PersistentModelIndex constructions that registered this handle and have not released it. + * The model keeps the handle while any registration remains. */ + Uint32 mRegistrations{ 0 }; }; class EE_API PersistentModelIndex { @@ -50,6 +53,12 @@ class EE_API PersistentModelIndex { bool operator==( ModelIndex const& ) const; bool operator!=( ModelIndex const& ) const; + /** Gives back the registration made when this index was constructed from a ModelIndex, and + * detaches it. The model drops the handle, and stops updating it on every row operation, + * once all its registrations are released. Indexes that never release keep the handle + * alive: call this at most once per construction, never on a copy. */ + void release(); + private: std::weak_ptr mHandle; }; diff --git a/include/eepp/ui/uiscenenode.hpp b/include/eepp/ui/uiscenenode.hpp index f00ef2b8b..9349d533d 100644 --- a/include/eepp/ui/uiscenenode.hpp +++ b/include/eepp/ui/uiscenenode.hpp @@ -526,6 +526,11 @@ class EE_API UISceneNode : public SceneNode { AccessibilityPolicy getAccessibilityPolicy() const; + /** Asks assistive clients to speak a message (for example "Build finished") without moving + * focus. Costs nothing while no client is active. */ + void announceForAccessibility( const String& message, + AccessibilityLive priority = AccessibilityLive::Polite ); + bool hasActiveAccessibilityClients() const { return !( mNodeFlags & NODE_FLAG_CLOSE ) && ( mHostUISceneNode ? mHostUISceneNode->hasActiveAccessibilityClients() @@ -1329,6 +1334,15 @@ class EE_API UISceneNode : public SceneNode { void onWidgetAccessibilitySourceDelete( UIWidget* widget ); + /** The manager this scene's widgets report to, or nullptr when none exists yet. Never creates + * one: lifecycle bookkeeping (destruction, scene and host changes) must not initialize + * accessibility. */ + AccessibilityManager* getExistingAccessibilityManager() const; + + /** Sets this root scene's client-active state and keeps the process-wide count that widget + * notifications test inline in sync with it. */ + void setAccessibilityClientActive( bool active ); + /** * @brief Recursively resets tooltips for a node and its children. * diff --git a/include/eepp/ui/uiscrollview.hpp b/include/eepp/ui/uiscrollview.hpp index aaeef1a37..37d06da97 100644 --- a/include/eepp/ui/uiscrollview.hpp +++ b/include/eepp/ui/uiscrollview.hpp @@ -57,6 +57,11 @@ class EE_API UIScrollView : public UITouchDraggableWidget { bool areDefaultKeybindingsEnabled() const { return mDefaultKeybindings; } + /** Scrolls the least distance that makes a descendant of the content fully visible (or its + * top-left corner, when it is larger than the viewport). Returns false when the node is not + * part of this view's content. */ + bool scrollIntoView( const Node* node ); + protected: ScrollViewType mViewType; ScrollBarMode mVScrollMode; diff --git a/include/eepp/ui/uitextedit.hpp b/include/eepp/ui/uitextedit.hpp index e349f2acb..085352eeb 100644 --- a/include/eepp/ui/uitextedit.hpp +++ b/include/eepp/ui/uitextedit.hpp @@ -88,8 +88,6 @@ class EE_API UITextEdit : public UICodeEditor { const TextPosition& cursor ); virtual void onTextHintsChanged(); - - virtual void onDocumentSelectionChange( const Doc::TextRange& ); }; }} // namespace EE::UI diff --git a/include/eepp/ui/uitextinput.hpp b/include/eepp/ui/uitextinput.hpp index 60bf27073..9531d9b9e 100644 --- a/include/eepp/ui/uitextinput.hpp +++ b/include/eepp/ui/uitextinput.hpp @@ -205,6 +205,8 @@ class EE_API UITextInput : public UITextView, public TextDocument::Client { virtual void onFontColorChanged(); + virtual void onTextChanged(); + void onThemeLoaded(); virtual void onCursorPosChange(); @@ -245,7 +247,7 @@ class EE_API UITextInput : public UITextView, public TextDocument::Client { virtual void onDocumentMoved( TextDocument* ); - virtual void onDocumentReset( TextDocument* ) {} + virtual void onDocumentReset( TextDocument* ) { notifyAccessibilityTextChanged( nullptr ); } void onDocumentClosed( TextDocument* ) {}; diff --git a/include/eepp/ui/uiwidget.hpp b/include/eepp/ui/uiwidget.hpp index 67a52b430..1faa3d40f 100644 --- a/include/eepp/ui/uiwidget.hpp +++ b/include/eepp/ui/uiwidget.hpp @@ -2,6 +2,7 @@ #define EE_UIUIWIDGET_HPP #include +#include #include #include #include @@ -26,6 +27,9 @@ class UITooltip; class UIStyle; class AccessibilityWidgetResolver; struct AccessibilityProperties; +namespace Doc { +struct DocumentContentChange; +} class UIWidget; class UITextSelectionController; @@ -108,6 +112,23 @@ class EE_API UIWidget : public UINode { UIWidget* setAccessibilityHidden( bool hidden ); + /** Names this widget after the widget with the given id (aria-labelledby). It takes + * precedence over the accessibility label. The id is resolved from the scene root. */ + UIWidget* setAccessibilityLabelledBy( const std::string& id ); + + const std::string& getAccessibilityLabelledBy() const; + + /** Describes this widget with the text of the widget with the given id (aria-describedby). */ + UIWidget* setAccessibilityDescribedBy( const std::string& id ); + + const std::string& getAccessibilityDescribedBy() const; + + /** Marks this widget as a live region (aria-live): name and value changes inside it are + * announced to assistive clients without moving focus. */ + UIWidget* setAccessibilityLive( AccessibilityLive live ); + + AccessibilityLive getAccessibilityLive() const; + virtual UITextSelectionController* getTextSelectionController(); virtual const UITextSelectionController* getTextSelectionController() const; @@ -285,7 +306,7 @@ class EE_API UIWidget : public UINode { * * @return The tooltip text string. */ - String getTooltipText(); + String getTooltipText() const; /** * @brief Updates the distances to parent borders for anchoring. @@ -1967,9 +1988,41 @@ class EE_API UIWidget : public UINode { String resolveAccessibilityDescription() const; - void notifyAccessibilityEvent( AccessibilityEvent event ); + /** Whether any scene in the process has an active accessibility client. Notifications test + * it inline, so while no client is attached anywhere they cost one relaxed load and no call. + * Callers that compute an event or payload test it first. */ + static bool isAccessibilityActive() { + return sAccessibilityActiveScenes.load( std::memory_order_relaxed ) != 0; + } + + void notifyAccessibilityEvent( AccessibilityEvent event ) { + if ( isAccessibilityActive() ) + deliverAccessibilityEvent( event ); + } + + /** Reports an edit of this text widget's document; nullptr when the whole text changed. */ + void notifyAccessibilityTextChanged( const Doc::DocumentContentChange* change ) { + if ( isAccessibilityActive() ) + deliverAccessibilityTextChanged( change ); + } + + /** A reset, load or document swap: the whole text changed, and with it the value. */ + void notifyAccessibilityWholeTextChanged() { + if ( isAccessibilityActive() ) + deliverAccessibilityWholeTextChanged(); + } void detachAccessibilitySource(); + + private: + /** Root scenes whose accessibility client is active; maintained by UISceneNode. */ + static std::atomic sAccessibilityActiveScenes; + + void deliverAccessibilityEvent( AccessibilityEvent event ); + + void deliverAccessibilityTextChanged( const Doc::DocumentContentChange* change ); + + void deliverAccessibilityWholeTextChanged(); }; }} // namespace EE::UI diff --git a/projects/scripts/benchmark_accessibility_dormant.py b/projects/scripts/benchmark_accessibility_dormant.py new file mode 100644 index 000000000..fdfdfa4cf --- /dev/null +++ b/projects/scripts/benchmark_accessibility_dormant.py @@ -0,0 +1,164 @@ +#!/usr/bin/env python3 + +"""Dormant-path accessibility benchmark: allocations, bytes and time with no client attached. + +Runs the AccessibilityDormant benchmarks from eepp-benchmarks (a Release build) several times and +reports the median of each measurement. Each --compare LABEL=PATH runs the same benchmark source +built against another tree as well (for example an exported develop or merge-base revision, +compiled with -DEE_BENCH_NO_ACCESSIBILITY), and prints the current build's differences from it. +Allocation counts and bytes cover ordinary C++ new/new[] on the measuring thread only (not malloc, +other threads or background work) and are deterministic; times are indicative only. A run passes +only when every sample exits cleanly and reports every expected measurement, guarded scenarios +included. A compared build whose run fails still reports its measurements. +""" + +import argparse +import json +import re +import statistics +import subprocess +import sys + +LINE = re.compile(r"^ACCESSIBILITY_DORMANT (\S+) ?(.*)$") +FIELD = re.compile(r"(\w+)=([-\d.]+)") + +# Every build must report these; a run missing any of them did not measure what it claims. +COMMON = { + "typing": ("allocations", "bytes", "ms"), + "deleting_characters": ("allocations", "bytes", "ms"), + "large_deletion": ("allocations", "bytes", "ms"), + "large_deletion_removed_text_bytes": ("value", "allocated_bytes_per_removed_byte"), + "spinbox_construction": ("allocations", "bytes", "ms"), + "spinbox_destruction": ("allocations", "bytes", "ms"), + "deep_hierarchy_lifecycle": ("allocations", "bytes", "ms"), + "nested_scene_lifecycle": ("allocations", "bytes", "ms"), + "list_scroll_updates": ("allocations", "bytes", "ms"), + "list_model_mutations": ("allocations", "bytes", "ms"), + "widget_notifications": ("allocations", "bytes", "ms"), +} +# Only builds with accessibility support (not EE_BENCH_NO_ACCESSIBILITY) report these, including +# the guarded after-disconnect and unreferenced id scenarios. +ACCESSIBILITY = { + "first_update_without_thread_pool": ("allocations", "bytes", "ms"), + "later_update_without_thread_pool": ("allocations", "bytes", "ms"), + "readiness_ms": ("value", "backend_available"), + "model_mutations_never_queried": ("allocations", "bytes", "ms"), + "model_mutations_after_disconnect": ("allocations", "bytes", "ms"), + "set_id_no_client": ("allocations", "bytes", "ms"), + "set_id_active_unreferenced": ("allocations", "bytes", "ms"), + "set_id_active_referenced": ("allocations", "bytes", "ms"), +} + + +def parse(stdout): + sample = {} + for line in stdout.splitlines(): + match = LINE.match(line) + if not match: + continue + name, rest = match.groups() + fields = {} + # Single-value lines ("name=value ...") carry the measurement in the name itself. + if "=" in name: + name, value = name.split("=", 1) + fields["value"] = float(value) + for key, value in FIELD.findall(rest): + fields[key] = float(value) + sample[name] = fields + return sample + + +def missing(sample, expected): + return sorted( + f"{name}.{key}" if name in sample else name + for name, keys in expected.items() + for key in (keys if name in sample else keys[:1]) + if name not in sample or key not in sample[name] + ) + + +def run(executable, samples, timeout, expected): + """Returns the median of every measurement and the reasons the run did not pass, if any.""" + results = {} + failures = [] + for index in range(samples): + completed = subprocess.run( + [executable, "--filter=AccessibilityDormant.*"], + capture_output=True, + text=True, + timeout=timeout, + ) + if completed.returncode != 0: + failures.append(f"sample {index + 1}: exit code {completed.returncode}") + sample = parse(completed.stdout) + absent = missing(sample, expected) + if absent: + failures.append(f"sample {index + 1}: missing {', '.join(absent)}") + for name, fields in sample.items(): + for key, value in fields.items(): + results.setdefault(name, {}).setdefault(key, []).append(value) + medians = { + name: {key: statistics.median(values) for key, values in fields.items()} + for name, fields in results.items() + } + return medians, failures + + +def main(): + parser = argparse.ArgumentParser(description=__doc__.splitlines()[0]) + parser.add_argument("--executable", default="bin/benchmarks/eepp-benchmarks") + parser.add_argument("--compare", action="append", default=[], metavar="LABEL=PATH", + help="the same benchmark built against another tree") + parser.add_argument("--samples", type=int, default=3) + parser.add_argument("--timeout", type=float, default=300) + parser.add_argument("--json", action="store_true") + args = parser.parse_args() + + if args.samples < 1: + parser.error("--samples must be at least 1") + builds = [("current", args.executable)] + for entry in args.compare: + label, _, path = entry.partition("=") + if not label or not path: + parser.error(f"--compare expects LABEL=PATH, got {entry!r}") + if any(label == existing for existing, _ in builds): + # Results are keyed by label: a repeated one would replace another build's results. + reason = "is reserved for --executable" if label == "current" else "is already used" + parser.error(f"--compare label {label!r} {reason}") + builds.append((label, path)) + output = {} + failures = {} + for label, path in builds: + # The current build must measure everything; a compared tree may lack accessibility. + expected = {**COMMON, **ACCESSIBILITY} if label == "current" else COMMON + output[label], failures[label] = run(path, args.samples, args.timeout, expected) + passed = {label: not failures[label] for label in failures} + if args.json: + print(json.dumps({"results": output, "guards_passed": passed, "failures": failures}, + indent=1, sort_keys=True)) + sys.exit(0 if passed["current"] else 1) + + current = output["current"] + labels = [label for label, _ in builds] + for key in ("allocations", "bytes", "ms"): + print(f"\n{key} (median of {args.samples})") + print(f"{'measurement':42}" + "".join(f" {label:>16}" for label in labels)) + for name in sorted(current): + if key not in current[name]: + continue + row = f"{name:42}" + for label in labels: + value = output[label].get(name, {}).get(key) + if value is None: + row += f" {'-':>16}" + else: + row += f" {value:>16.3f}" if key == "ms" else f" {value:>16.0f}" + print(row) + for label in labels: + for failure in failures[label]: + print(f"\n{label}: {failure} (measurements above are still reported)") + sys.exit(0 if passed["current"] else 1) + + +if __name__ == "__main__": + main() diff --git a/projects/scripts/test_atspi.py b/projects/scripts/test_atspi.py index afd5f6e28..f6f41dc54 100644 --- a/projects/scripts/test_atspi.py +++ b/projects/scripts/test_atspi.py @@ -90,7 +90,7 @@ def start_discovery_listener(): text_events = [] def on_children_changed(event): - if event.type.startswith("object:text-"): + if event.type.startswith("object:text-") or event.type == "object:announcement": text_events.append((event.type, event.detail1, event.detail2, event.any_data)) return application = event.any_data @@ -105,7 +105,8 @@ def start_discovery_listener(): listener.register("object:children-changed:add"), "could not register the AT-SPI application discovery listener", ) - for event in ("object:text-changed", "object:text-caret-moved", "object:text-selection-changed"): + for event in ("object:text-changed", "object:text-caret-moved", "object:text-selection-changed", + "object:announcement"): check(listener.register(event), f"could not register {event}") return listener, discovered_process_ids, text_events @@ -201,6 +202,44 @@ def validate(application, multi_window=False, close_primary=False, process_id=No line = Atspi.Text.get_string_at_offset(description, 8, Atspi.TextGranularity.LINE) check((line.content, line.start_offset, line.end_offset) == ("second\n", 6, 13), f"line granularity returned the wrong range: {line}") + last_line = Atspi.Text.get_string_at_offset(description, 15, Atspi.TextGranularity.LINE) + check((last_line.content, last_line.start_offset, last_line.end_offset) == ("third", 13, 18), + f"last line must exclude the final newline: {last_line}") + word = Atspi.Text.get_string_at_offset(description, 8, Atspi.TextGranularity.WORD) + check((word.content, word.start_offset, word.end_offset) == ("second", 6, 12), + f"word granularity returned the wrong range: {word}") + check(description.get_character_count() == 18, "multiline character count must equal 18") + check(Atspi.Text.get_character_at_offset(description, 6) == ord("s"), + "character at offset returned the wrong character") + + if text_events is not None: + # An application edit arrives as one exact insertion, not as a diff of the whole text. + append = find_named(nodes, "Append description line") + del text_events[:] + check(append.do_action(action_index(append, "click")), "append action failed") + def appended(): + return [event for event in text_events if event[0] == "object:text-changed:insert"] + deadline = time.monotonic() + 2 + while time.monotonic() < deadline and not appended(): + pause(0.01) + check(appended() == [("object:text-changed:insert", 18, 9, "\nappended")], + f"application edit was not reported exactly: {text_events}") + check(Atspi.Text.get_text(description, 18, -1) == "\nappended", "appended text query failed") + + # Replacing a field's text reports the old contents removed and the new ones inserted. + reset = find_named(nodes, "Reset project name") + del text_events[:] + check(reset.do_action(action_index(reset, "click")), "reset action failed") + def replaced(): + return [(event[0], event[1], event[2]) for event in text_events + if event[0].startswith("object:text-changed")] + deadline = time.monotonic() + 2 + while time.monotonic() < deadline and len(replaced()) < 2: + pause(0.01) + check(replaced() == [("object:text-changed:delete", 0, 5), + ("object:text-changed:insert", 0, 4)], + f"text replacement was not reported as delete and insert: {text_events}") + check(Atspi.Text.get_text(project_name, 0, -1) == "eepp", "reset text query failed") checkbox = find_named(nodes, "Enable autosave") check(checkbox.get_role_name() == "check box", "autosave must be a check box") @@ -239,6 +278,22 @@ def validate(application, multi_window=False, close_primary=False, process_id=No find_named(nodes, "Colors") find_named(nodes, "Projects") + + # An icon-only button is named by its tooltip; the status line it updates is a live region. + refresh = find_named(nodes, "Refresh status") + check(refresh.get_role() == Atspi.Role.PUSH_BUTTON, + "icon-only button must be named by its tooltip") + check(refresh.do_action(action_index(refresh, "click")), "refresh action failed") + if text_events is not None: + def announced(): + return any(event[0] == "object:announcement" and event[3] == "Status refreshed" + for event in text_events) + deadline = time.monotonic() + 2 + while time.monotonic() < deadline and not announced(): + pause(0.01) + check(announced(), "live region change was not announced: " + f"{[event for event in text_events if event[0] == 'object:announcement']}") + if multi_window: find_named(nodes, "Secondary accessibility window") open_window = find_named(nodes, "Open accessibility window") diff --git a/src/benchmarks/accessibility_dormant_benchmark.cpp b/src/benchmarks/accessibility_dormant_benchmark.cpp new file mode 100644 index 000000000..d39b1617c --- /dev/null +++ b/src/benchmarks/accessibility_dormant_benchmark.cpp @@ -0,0 +1,539 @@ +#include "../tests/unit_tests/utest.hpp" + +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include + +// Define EE_BENCH_NO_ACCESSIBILITY when building against a tree without accessibility support. +#if !defined( EE_BENCH_NO_ACCESSIBILITY ) && \ + __has_include( ) +#define EE_BENCH_HAS_ACCESSIBILITY 1 +#include "../eepp/ui/accessibility/accessibilitybackend.hpp" +#include +#endif + +#include +#include +#include +#include + +using namespace EE; +using namespace EE::Scene; +using namespace EE::System; +using namespace EE::UI; +using namespace EE::UI::Doc; +using namespace EE::UI::Models; +using namespace EE::Window; + +// Dormant-path benchmarks: no accessibility client is attached. The same source builds against a +// tree without accessibility support (for a baseline comparison) and against the branch. +// +// Allocations are counted by replacing the global operator new in this benchmark binary only. +// Only the measuring thread counts: background threads (native accessibility I/O, worker pools) +// would otherwise add nondeterministic noise, and the UI thread's cost is what is measured. + +namespace { + +thread_local bool tCountAllocations{ false }; +thread_local Uint64 tAllocations{ 0 }; +thread_local Uint64 tAllocatedBytes{ 0 }; + +void* countedAllocate( std::size_t size ) { + if ( tCountAllocations ) { + ++tAllocations; + tAllocatedBytes += size; + } + if ( void* memory = std::malloc( size ? size : 1 ) ) + return memory; + throw std::bad_alloc(); +} + +} // namespace + +void* operator new( std::size_t size ) { + return countedAllocate( size ); +} + +void* operator new[]( std::size_t size ) { + return countedAllocate( size ); +} + +void operator delete( void* memory ) noexcept { + std::free( memory ); +} + +void operator delete[]( void* memory ) noexcept { + std::free( memory ); +} + +void operator delete( void* memory, std::size_t ) noexcept { + std::free( memory ); +} + +void operator delete[]( void* memory, std::size_t ) noexcept { + std::free( memory ); +} + +namespace { + +struct Measurement { + Uint64 allocations{ 0 }; + Uint64 bytes{ 0 }; + double milliseconds{ 0 }; +}; + +/** Counts the operator new calls and bytes `work` makes on this thread, and its wall time. */ +template Measurement measure( const Work& work ) { + tAllocations = 0; + tAllocatedBytes = 0; + tCountAllocations = true; + Clock clock; + work(); + Measurement result; + result.milliseconds = clock.getElapsedTime().asMilliseconds(); + tCountAllocations = false; + result.allocations = tAllocations; + result.bytes = tAllocatedBytes; + return result; +} + +void report( const char* name, const Measurement& measurement, Uint64 operations ) { + std::printf( "ACCESSIBILITY_DORMANT %s allocations=%llu bytes=%llu ms=%.3f " + "allocations_per_op=%.2f bytes_per_op=%.1f\n", + name, static_cast( measurement.allocations ), + static_cast( measurement.bytes ), measurement.milliseconds, + operations ? static_cast( measurement.allocations ) / operations : 0.0, + operations ? static_cast( measurement.bytes ) / operations : 0.0 ); + std::fflush( stdout ); +} + +UIApplication::Settings benchmarkSettings() { + // The default policy: native accessibility is available, but no client is attached. + return UIApplication::Settings( Sys::getProcessPath() + ".." + FileSystem::getOSSlash(), 1 ); +} + +WindowSettings benchmarkWindow() { + return WindowSettings( 640, 480, "Accessibility dormant benchmark", + WindowStyle::Default | WindowStyle::Hidden, WindowBackend::Default, 32, + {}, 1, false, true ); +} + +/** Retains every change record the way an asynchronous consumer (such as an LSP client) does: + * wraps it in a vector, queues a copy and copies it out again when draining. */ +class RetainingConsumer : public TextDocument::Client { + public: + struct Record { + Uint64 version; + std::vector changes; + }; + + void onDocumentTextChanged( const DocumentContentChange& change ) override { + std::vector changes{ change }; + mQueue.push_back( { ++mVersion, changes } ); + } + + void drain() { + while ( !mQueue.empty() ) { + Record record = mQueue.front(); + mQueue.pop_front(); + mDrained += record.changes.size(); + } + } + + void onDocumentUndoRedo( const TextDocument::UndoRedo& ) override {} + + void onDocumentCursorChange( const TextPosition& ) override {} + + void onDocumentSelectionChange( const TextRange& ) override {} + + void onDocumentLineCountChange( const size_t&, const size_t& ) override {} + + void onDocumentLineChanged( const Int64& ) override {} + + void onDocumentSaved( TextDocument* ) override {} + + void onDocumentClosed( TextDocument* ) override {} + + void onDocumentDirtyOnFileSystem( TextDocument* ) override {} + + void onDocumentMoved( TextDocument* ) override {} + + void onDocumentReset( TextDocument* ) override {} + + Type getTextDocumentClientType() override { return Type::Auxiliary; } + + Uint64 drained() const { return mDrained; } + + private: + std::deque mQueue; + Uint64 mVersion{ 0 }; + Uint64 mDrained{ 0 }; +}; + +class RowsModel : public Model { + public: + explicit RowsModel( size_t rows ) : mRows( rows ) {} + + size_t rowCount( const ModelIndex& ) const override { return mRows; } + + size_t columnCount( const ModelIndex& ) const override { return 1; } + + Variant data( const ModelIndex& index, ModelRole role = ModelRole::Display ) const override { + return role == ModelRole::Display ? Variant( String::format( "Row %d", index.row() ) ) + : Variant(); + } + + void insertRow( int row ) { + beginInsertRows( {}, row, row ); + ++mRows; + endInsertRows(); + } + + void removeRow( int row ) { + if ( beginDeleteRows( {}, row, row ) ) { + --mRows; + endDeleteRows(); + } + } + + private: + size_t mRows; +}; + +} // namespace + +UTEST( AccessibilityDormant, LibraryAllocationsAreCounted ) { + // Global operator new interposition must reach allocations made inside libeepp. + const auto measurement = measure( [] { + volatile size_t size = String::toLower( std::string( 256, 'A' ) ).size(); + (void)size; + } ); + EXPECT_GT( measurement.allocations, 0u ); +} + +UTEST( AccessibilityDormant, TypingWithRetainingConsumer ) { + UIApplication app( benchmarkWindow(), benchmarkSettings() ); + ASSERT_NE( app.getUI(), nullptr ); + auto* editor = UICodeEditor::New(); + editor->setParent( app.getUI()->getRoot() ); + RetainingConsumer consumer; + auto& document = editor->getDocument(); + document.registerClient( &consumer ); + app.getUI()->update( Time::Zero ); + static constexpr int Characters = 5000; + const auto typing = measure( [&] { + for ( int i = 0; i < Characters; ++i ) + document.textInput( "a" ); + consumer.drain(); + } ); + report( "typing", typing, Characters ); + const auto deleting = measure( [&] { + for ( int i = 0; i < Characters; ++i ) + document.deleteToPreviousChar(); + consumer.drain(); + } ); + report( "deleting_characters", deleting, Characters ); + document.unregisterClient( &consumer ); + EXPECT_EQ( consumer.drained(), static_cast( Characters * 2 ) ); +} + +UTEST( AccessibilityDormant, LargeDeletionWithRetainingConsumer ) { + UIApplication app( benchmarkWindow(), benchmarkSettings() ); + ASSERT_NE( app.getUI(), nullptr ); + auto* editor = UICodeEditor::New(); + editor->setParent( app.getUI()->getRoot() ); + auto& document = editor->getDocument(); + std::string contents; + static constexpr int Lines = 100000; + contents.reserve( Lines * 32 ); + for ( int line = 0; line < Lines; ++line ) + contents += "line " + std::to_string( line ) + " with some text\n"; + document.textInput( String::fromUtf8( contents ) ); + app.getUI()->update( Time::Zero ); + RetainingConsumer consumer; + document.registerClient( &consumer ); + const Uint64 removedBytes = contents.size() * sizeof( String::StringBaseType ); + const auto deletion = measure( [&] { + document.selectAll(); + document.deleteSelection(); + consumer.drain(); + } ); + report( "large_deletion", deletion, 1 ); + std::printf( "ACCESSIBILITY_DORMANT large_deletion_removed_text_bytes=%llu " + "allocated_bytes_per_removed_byte=%.2f\n", + static_cast( removedBytes ), + static_cast( deletion.bytes ) / removedBytes ); + document.unregisterClient( &consumer ); + EXPECT_TRUE( document.isEmpty() ); + // The removed text is extracted once for undo; change records must not carry it into + // consumers that queue them (6 bytes per removed byte when they did, 2 without). + EXPECT_LT( static_cast( deletion.bytes ) / removedBytes, 3.0 ); +} + +UTEST( AccessibilityDormant, SpinBoxConstruction ) { + UIApplication app( benchmarkWindow(), benchmarkSettings() ); + ASSERT_NE( app.getUI(), nullptr ); + static constexpr int SpinBoxes = 1000; + std::vector spinBoxes; + spinBoxes.reserve( SpinBoxes ); + const auto construction = measure( [&] { + for ( int i = 0; i < SpinBoxes; ++i ) { + auto* spinBox = UISpinBox::New(); + spinBox->setParent( app.getUI()->getRoot() ); + spinBoxes.push_back( spinBox ); + } + } ); + report( "spinbox_construction", construction, SpinBoxes ); + const auto destruction = measure( [&] { + for ( auto* spinBox : spinBoxes ) + eeDelete( spinBox ); + } ); + report( "spinbox_destruction", destruction, SpinBoxes ); +} + +UTEST( AccessibilityDormant, DeepHierarchyLifecycle ) { + UIApplication app( benchmarkWindow(), benchmarkSettings() ); + ASSERT_NE( app.getUI(), nullptr ); + app.getUI()->update( Time::Zero ); + static constexpr int Chains = 200; + static constexpr int Depth = 64; + const auto lifecycle = measure( [&] { + for ( int chain = 0; chain < Chains; ++chain ) { + UIWidget* top = UIWidget::New(); + top->setParent( app.getUI()->getRoot() ); + UIWidget* parent = top; + for ( int depth = 1; depth < Depth; ++depth ) { + auto* child = UIWidget::New(); + child->setParent( parent ); + parent = child; + } + // Delete from the deepest widget up: each deletion is a top-level one. + while ( parent != top ) { + auto* next = parent->getParent()->asType(); + eeDelete( parent ); + parent = next; + } + eeDelete( top ); + } + } ); + report( "deep_hierarchy_lifecycle", lifecycle, Chains * Depth ); +} + +UTEST( AccessibilityDormant, NestedSceneLifecycle ) { + UIApplication app( benchmarkWindow(), benchmarkSettings() ); + ASSERT_NE( app.getUI(), nullptr ); + app.getUI()->update( Time::Zero ); + static constexpr int Scenes = 100; + const auto lifecycle = measure( [&] { + for ( int i = 0; i < Scenes; ++i ) { + auto* nested = UISceneNode::New( app.getWindow() ); + nested->setParent( app.getUI()->getRoot() ); + for ( int child = 0; child < 20; ++child ) + UIWidget::New()->setParent( nested->getRoot() ); + eeDelete( nested ); + } + } ); + report( "nested_scene_lifecycle", lifecycle, Scenes ); +} + +UTEST( AccessibilityDormant, RecycledListCells ) { + UIApplication app( benchmarkWindow(), benchmarkSettings() ); + ASSERT_NE( app.getUI(), nullptr ); + auto* list = UIListView::New(); + list->setParent( app.getUI()->getRoot() ); + list->setPixelsSize( 400, 400 ); + auto model = std::make_shared( 2000 ); + list->setModel( model ); + app.getUI()->update( Time::Zero ); + static constexpr int Updates = 2000; + const auto scrolling = measure( [&] { + for ( int i = 0; i < Updates; ++i ) { + list->getVerticalScrollBar()->setValue( static_cast( ( i * 37 ) % 1000 ) / + 1000.f ); + app.getUI()->update( Time::Zero ); + } + } ); + report( "list_scroll_updates", scrolling, Updates ); + const auto mutations = measure( [&] { + for ( int i = 0; i < Updates; ++i ) { + model->insertRow( i % 100 ); + model->removeRow( ( i * 7 ) % 100 ); + } + } ); + report( "list_model_mutations", mutations, Updates ); +} + +UTEST( AccessibilityDormant, WidgetNotifications ) { + UIApplication app( benchmarkWindow(), benchmarkSettings() ); + ASSERT_NE( app.getUI(), nullptr ); + static constexpr int Widgets = 200; + static constexpr int Rounds = 50; + std::vector labels; + std::vector bars; + for ( int i = 0; i < Widgets; ++i ) { + auto* label = UITextView::New(); + label->setParent( app.getUI()->getRoot() ); + labels.push_back( label ); + auto* bar = UIProgressBar::New(); + bar->setParent( app.getUI()->getRoot() ); + bars.push_back( bar ); + } + app.getUI()->update( Time::Zero ); + const String first( "First label" ); + const String second( "Second label" ); + const auto notifications = measure( [&] { + for ( int round = 0; round < Rounds; ++round ) { + for ( int i = 0; i < Widgets; ++i ) { + labels[i]->setText( round % 2 ? first : second ); + bars[i]->setProgress( static_cast( round % 100 ) ); + labels[i]->setVisible( round % 2 == 0 ); + labels[i]->setEnabled( round % 2 == 0 ); + } + } + } ); + report( "widget_notifications", notifications, Widgets * Rounds ); +} + +#ifdef EE_BENCH_HAS_ACCESSIBILITY + +namespace { + +/** A backend whose client can attach and disconnect on demand. */ +class SwitchableBackend : public AccessibilityBackend { + public: + bool active{ false }; + + bool isAvailable() const override { return true; } + + bool hasActiveClients() const override { return active; } + + void onEvent( const AccessibilityPendingEvent& ) override {} +}; + +} // namespace + +UTEST( AccessibilityDormant, InitializationWithoutThreadPool ) { + // Without a scene thread pool the native backend initializes on the UI thread during the + // first scene update. That update is measured against a later one. + UIApplication app( benchmarkWindow(), benchmarkSettings() ); + ASSERT_NE( app.getUI(), nullptr ); + ASSERT_FALSE( app.getUI()->hasThreadPool() ); + const auto first = measure( [&] { app.getUI()->update( Time::Zero ); } ); + report( "first_update_without_thread_pool", first, 1 ); + const auto later = measure( [&] { app.getUI()->update( Time::Zero ); } ); + report( "later_update_without_thread_pool", later, 1 ); + // Readiness (the backend finished connecting) is separate from UI-thread cost. + auto* manager = app.getUI()->getAccessibilityManager(); + Clock readiness; + while ( !manager->isBackendInitializationComplete() && + readiness.getElapsedTime() < Seconds( 5 ) ) + Sys::sleep( Microseconds( 100 ) ); + std::printf( "ACCESSIBILITY_DORMANT readiness_ms=%.3f backend_available=%d\n", + readiness.getElapsedTime().asMilliseconds(), + manager->isBackendAvailable() ? 1 : 0 ); +} + +UTEST( AccessibilityDormant, ModelUpdatesAfterClientDisconnects ) { + UIApplication app( benchmarkWindow(), benchmarkSettings() ); + ASSERT_NE( app.getUI(), nullptr ); + auto* manager = app.getUI()->getAccessibilityManager(); + auto backend = std::make_unique(); + auto* switchable = backend.get(); + manager->setBackend( std::move( backend ) ); + auto* list = UIListView::New(); + list->setParent( app.getUI()->getRoot() ); + list->setPixelsSize( 400, 400 ); + auto model = std::make_shared( 2000 ); + list->setModel( model ); + app.getUI()->update( Time::Zero ); + static constexpr int Updates = 2000; + auto mutate = [&] { + for ( int i = 0; i < Updates; ++i ) { + model->insertRow( i % 100 ); + model->removeRow( ( i * 7 ) % 100 ); + } + }; + const auto neverQueried = measure( mutate ); + report( "model_mutations_never_queried", neverQueried, Updates ); + + // A client reads every row, then disconnects. + switchable->active = true; + app.getUI()->update( Time::Zero ); + const auto listRef = manager->getNodeRef( list ); + const auto rows = manager->getChildCount( listRef ); + for ( size_t row = 0; row < rows; ++row ) + manager->getNodeInfo( manager->getChild( listRef, row ) ); + switchable->active = false; + app.getUI()->update( Time::Zero ); + const auto afterDisconnect = measure( mutate ); + report( "model_mutations_after_disconnect", afterDisconnect, Updates ); + // A disconnected client's rows must not keep the model updating persistent handles. + EXPECT_LE( afterDisconnect.allocations, neverQueried.allocations ); +} + +UTEST( AccessibilityDormant, IdChangesWithAnActiveClient ) { + UIApplication app( benchmarkWindow(), benchmarkSettings() ); + ASSERT_NE( app.getUI(), nullptr ); + auto* scene = app.getUI(); + auto* manager = scene->getAccessibilityManager(); + auto backend = std::make_unique(); + auto* switchable = backend.get(); + manager->setBackend( std::move( backend ) ); + // Equal lengths past any small-string buffer: renaming reuses the id's capacity, so only a + // copy of the previous id would allocate. + const std::string first( 64, 'a' ); + const std::string second( 64, 'b' ); + const std::string target( 64, 't' ); + const std::string moved( 64, 'm' ); + auto* label = UITextView::New(); + label->setParent( scene->getRoot() ); + label->setId( target ); + label->setText( "Label" ); + auto* input = UITextInput::New(); + input->setParent( scene->getRoot() ); + input->setAccessibilityLabelledBy( target ); + auto* renamed = UIWidget::New(); + renamed->setParent( scene->getRoot() ); + renamed->setId( first ); + scene->update( Time::Zero ); + static constexpr int Renames = 10000; + auto rename = [&]( UIWidget* widget, const std::string& a, const std::string& b ) { + for ( int i = 0; i < Renames; ++i ) + widget->setId( i % 2 ? a : b ); + }; + rename( renamed, first, second ); + const auto noClient = measure( [&] { rename( renamed, first, second ); } ); + report( "set_id_no_client", noClient, Renames ); + + // A client reads the input, so the manager remembers the label's id as a relation target. + switchable->active = true; + scene->update( Time::Zero ); + manager->getNodeInfo( manager->getNodeRef( input ) ); + const auto unreferenced = measure( [&] { rename( renamed, first, second ); } ); + report( "set_id_active_unreferenced", unreferenced, Renames ); + // An id no client depends on is never copied. + EXPECT_LE( unreferenced.allocations, noClient.allocations ); + + // Renaming the target itself keeps the old id to notify the input; reported, not guarded. + const auto referenced = measure( [&] { + for ( int i = 0; i < Renames; ++i ) { + label->setId( i % 2 ? target : moved ); + manager->clearPendingEvents(); + } + } ); + report( "set_id_active_referenced", referenced, Renames ); +} + +#endif diff --git a/src/eepp/ui/accessibility/accessibilitybackend.cpp b/src/eepp/ui/accessibility/accessibilitybackend.cpp index bdec2cb70..e87293ee9 100644 --- a/src/eepp/ui/accessibility/accessibilitybackend.cpp +++ b/src/eepp/ui/accessibility/accessibilitybackend.cpp @@ -1,7 +1,14 @@ #include "accessibilitybackend.hpp" +#include + namespace EE { namespace UI { +bool isAccessibilityTraceEnabled() { + static const bool enabled = std::getenv( "EEPP_ACCESSIBILITY_TRACE" ) != nullptr; + return enabled; +} + class NullAccessibilityBackend final : public AccessibilityBackend { public: bool isAvailable() const { return false; } diff --git a/src/eepp/ui/accessibility/accessibilitybackend.hpp b/src/eepp/ui/accessibility/accessibilitybackend.hpp index 88b8f5064..901983567 100644 --- a/src/eepp/ui/accessibility/accessibilitybackend.hpp +++ b/src/eepp/ui/accessibility/accessibilitybackend.hpp @@ -16,6 +16,10 @@ inline bool hasAction( AccessibilityActions actions, AccessibilityAction action return ( actions & accessibilityActionMask( action ) ) != 0; } +/** Opt-in diagnostics (EEPP_ACCESSIBILITY_TRACE set): client activity transitions and native + * client connections, printed to stderr. Only rare transitions consult it. */ +bool isAccessibilityTraceEnabled(); + class AccessibilityBackend { public: virtual ~AccessibilityBackend() = default; @@ -30,6 +34,18 @@ class AccessibilityBackend { virtual void onEvent( const AccessibilityPendingEvent& event ) = 0; + /** An edit of a text element, delivered synchronously before its ValueChanged event. + * Returns false when the change was ignored, so it does not count toward the frame's budget + * of exact changes. Changes the manager suppresses never arrive here. */ + virtual bool onTextChanged( AccessibilityNodeRef /*ref*/, + const AccessibilityTextChange& /*change*/ ) { + return false; + } + + /** Speaks a message through the platform announcement API. Called from update() on the UI + * thread, after the frame's events were delivered. */ + virtual void announce( const String& /*message*/, AccessibilityLive /*priority*/ ) {} + /** Evicts native wrappers for a model source whose identity is no longer valid. */ virtual void onSourceInvalidated( AccessibilitySourceId ) {} diff --git a/src/eepp/ui/accessibility/accessibilitybackendatspi.cpp b/src/eepp/ui/accessibility/accessibilitybackendatspi.cpp index 30f158576..4b53d7b54 100644 --- a/src/eepp/ui/accessibility/accessibilitybackendatspi.cpp +++ b/src/eepp/ui/accessibility/accessibilitybackendatspi.cpp @@ -4,6 +4,7 @@ #include #include +#include #include #include #include @@ -48,11 +49,10 @@ void AtSpiApplication::initialize() { } mDBus.messageUnref( reply ); if ( mConnection ) { + // Client disconnections are watched per client (watchClient()), not with a bus-wide + // NameOwnerChanged match that would wake us for every application on the bus. mDBus.connectionAddFilter( mConnection, &AtSpiApplication::clientDisconnected, this, nullptr ); - mDBus.busAddMatch( - mConnection, "type='signal',interface='org.freedesktop.DBus',member='NameOwnerChanged'", - nullptr ); registerApplication(); if ( mConnection && mDBus.connectionGetUnixFd( mConnection, &mConnectionFd ) && pipe( mWakePipe ) == 0 ) { @@ -114,12 +114,14 @@ void AtSpiApplication::unregisterManager( AccessibilityManager& manager ) { if ( it->second != &manager ) continue; const Int32 index = static_cast( it - mManagers.begin() ); - const auto prefix = std::string( NodePathPrefix ) + String::toString( it->first ) + "/"; - for ( auto text = mTextSnapshots.begin(); text != mTextSnapshots.end(); ) { - if ( text->first.compare( 0, prefix.size(), prefix ) == 0 ) - text = mTextSnapshots.erase( text ); - else - ++text; + if ( !mTextStates.empty() ) { + const auto prefix = std::string( NodePathPrefix ) + String::toString( it->first ) + "/"; + for ( auto text = mTextStates.begin(); text != mTextStates.end(); ) { + if ( text->first.compare( 0, prefix.size(), prefix ) == 0 ) + text = mTextStates.erase( text ); + else + ++text; + } } if ( isAvailable() && hasActiveClients() ) sendWindowChanged( manager, false, index ); @@ -232,18 +234,87 @@ void AtSpiApplication::activate( DBusMessage* message ) { const char* sender = mDBus.messageGetSender( message ); // Registry bookkeeping is not evidence of an assistive client. Its bus connection // normally outlives every screen reader and would otherwise pin the active flag forever. - if ( !sender || mRegistryBusName == sender ) + if ( !sender || mRegistryBusName == sender || !mClients.emplace( sender ).second ) return; - mClients.emplace( sender ); - mHasActiveClients.store( !mClients.empty(), std::memory_order_release ); + mHasActiveClients.store( true, std::memory_order_release ); + if ( isAccessibilityTraceEnabled() ) { + const char* member = mDBus.messageGetMember( message ); + std::fprintf( stderr, "eepp accessibility: AT-SPI client %s connected (first call %s)\n", + sender, member ? member : "?" ); + } + watchClient( sender ); +} + +namespace { + +std::string clientMatchRule( const char* name ) { + return std::string( "type='signal',sender='org.freedesktop.DBus'," + "interface='org.freedesktop.DBus',member='NameOwnerChanged',arg0='" ) + + name + "'"; +} + +} // namespace + +void AtSpiApplication::watchClient( const char* name ) { + // Neither request blocks: without an error argument libdbus queues the AddMatch instead of + // waiting for its reply, and the NameHasOwner reply arrives through clientDisconnected(). + // The bus daemon handles one connection's requests in order, so the check runs after the + // match is installed: a client that already left is reported by the check, and one that + // leaves later by the match. + mDBus.busAddMatch( mConnection, clientMatchRule( name ).c_str(), nullptr ); + DBusMessage* check = mDBus.messageNewMethodCall( + "org.freedesktop.DBus", "/org/freedesktop/DBus", "org.freedesktop.DBus", "NameHasOwner" ); + if ( !check ) + return; + DBusMessageIter iter; + mDBus.messageIterInitAppend( check, &iter ); + appendBasic( iter, 's', &name ); + Uint32 serial = 0; + if ( mDBus.connectionSend( mConnection, check, &serial ) ) + mClientChecks.emplace( serial, name ); + mDBus.messageUnref( check ); + if ( !mOutgoingPending ) { + mOutgoingPending = true; + Window::Input::wakeUpEventLoop(); + } +} + +void AtSpiApplication::clientGone( const char* name ) { + auto found = mClients.find( name ); + if ( found == mClients.end() ) + return; + if ( isAccessibilityTraceEnabled() ) + std::fprintf( stderr, "eepp accessibility: AT-SPI client %s disconnected\n", name ); + mDBus.busRemoveMatch( mConnection, clientMatchRule( name ).c_str(), nullptr ); + mClients.erase( found ); + const bool active = !mClients.empty(); + if ( !active ) { + mTextStates.clear(); + mPendingTexts.clear(); + } + mHasActiveClients.store( active, std::memory_order_release ); } DBusHandlerResult AtSpiApplication::clientDisconnected( DBusConnection*, DBusMessage* message, void* userData ) { auto* application = static_cast( userData ); - const char* interface = application->mDBus.messageGetInterface( message ); - const char* member = application->mDBus.messageGetMember( message ); - const char* sender = application->mDBus.messageGetSender( message ); + auto& dbus = application->mDBus; + const int type = dbus.messageGetType( message ); + if ( type == DBusMessageTypeMethodReturn || type == DBusMessageTypeError ) { + // The reply to a watchClient() NameHasOwner check. + auto check = application->mClientChecks.find( dbus.messageGetReplySerial( message ) ); + if ( check == application->mClientChecks.end() ) + return 1; + Uint32 hasOwner = 1; + if ( type == DBusMessageTypeMethodReturn && + dbus.messageGetArgs( message, nullptr, 'b', &hasOwner, 0 ) && !hasOwner ) + application->clientGone( check->second.c_str() ); + application->mClientChecks.erase( check ); + return 0; + } + const char* interface = dbus.messageGetInterface( message ); + const char* member = dbus.messageGetMember( message ); + const char* sender = dbus.messageGetSender( message ); if ( !interface || !member || std::strcmp( interface, "org.freedesktop.DBus" ) != 0 || std::strcmp( member, "NameOwnerChanged" ) != 0 || !sender || std::strcmp( sender, "org.freedesktop.DBus" ) != 0 ) @@ -251,17 +322,9 @@ DBusHandlerResult AtSpiApplication::clientDisconnected( DBusConnection*, DBusMes const char* name = nullptr; const char* oldOwner = nullptr; const char* newOwner = nullptr; - if ( application->mDBus.messageGetArgs( message, nullptr, 's', &name, 's', &oldOwner, 's', - &newOwner, 0 ) && - name && newOwner && !*newOwner ) { - application->mClients.erase( name ); - const bool active = !application->mClients.empty(); - if ( !active ) { - application->mTextSnapshots.clear(); - application->mPendingTexts.clear(); - } - application->mHasActiveClients.store( active, std::memory_order_release ); - } + if ( dbus.messageGetArgs( message, nullptr, 's', &name, 's', &oldOwner, 's', &newOwner, 0 ) && + name && newOwner && !*newOwner ) + application->clientGone( name ); return 1; } @@ -353,7 +416,11 @@ class AtSpiAccessibilityBackend final : public AccessibilityBackend { scene->getThreadPool()->run( [application = mApplication] { application->initialize(); } ); } else { - mApplication->initialize(); + // Without a pool, run the D-Bus round trips on a detached thread that shares + // ownership: tearing the backend down never waits for their timeouts, and when + // the thread holds the last reference the application is destroyed there, which + // joins only the I/O thread. + std::thread( [application = mApplication] { application->initialize(); } ).detach(); } } else { mApplication->registerManager( manager ); @@ -383,6 +450,15 @@ class AtSpiAccessibilityBackend final : public AccessibilityBackend { mApplication->onEvent( mManager, event ); } + bool onTextChanged( AccessibilityNodeRef ref, const AccessibilityTextChange& change ) override { + return mApplication && mApplication->onTextChanged( mManager, ref, change ); + } + + void announce( const String& message, AccessibilityLive priority ) override { + if ( mApplication ) + mApplication->announce( mManager, message, priority ); + } + private: static std::weak_ptr& application() { static std::weak_ptr instance; diff --git a/src/eepp/ui/accessibility/accessibilitybackendatspi.hpp b/src/eepp/ui/accessibility/accessibilitybackendatspi.hpp index cd9de16dd..f26ff4429 100644 --- a/src/eepp/ui/accessibility/accessibilitybackendatspi.hpp +++ b/src/eepp/ui/accessibility/accessibilitybackendatspi.hpp @@ -31,6 +31,7 @@ namespace EE { namespace UI { namespace AtSpi { +constexpr int DBusMessageTypeMethodReturn = 2; constexpr int DBusMessageTypeError = 3; struct DBusConnection; @@ -75,6 +76,7 @@ class DBusLibrary { using BusRegister = int ( * )( DBusConnection*, DBusError* ); using BusGetUniqueName = const char* ( * )( DBusConnection* ); using BusAddMatch = void ( * )( DBusConnection*, const char*, DBusError* ); + using BusRemoveMatch = BusAddMatch; using ConnectionAddFilter = int ( * )( DBusConnection*, DBusMessageFunction, void*, void ( * )( void* ) ); using ConnectionOpenPrivate = DBusConnection* ( * )( const char*, DBusError* ); @@ -96,6 +98,7 @@ class DBusLibrary { using MessageGetArgs = int ( * )( DBusMessage*, DBusError*, int, ... ); using MessageGetString = const char* ( * )( DBusMessage* ); using MessageGetType = int ( * )( DBusMessage* ); + using MessageGetReplySerial = Uint32 ( * )( DBusMessage* ); using MessageNewMethodReturn = DBusMessage* ( * )( DBusMessage* ); using MessageNewError = DBusMessage* ( * )( DBusMessage*, const char*, const char* ); using MessageNewSignal = DBusMessage* ( * )( const char*, const char*, const char* ); @@ -124,6 +127,7 @@ class DBusLibrary { loadSymbol( busRegister, "dbus_bus_register" ) && loadSymbol( busGetUniqueName, "dbus_bus_get_unique_name" ) && loadSymbol( busAddMatch, "dbus_bus_add_match" ) && + loadSymbol( busRemoveMatch, "dbus_bus_remove_match" ) && loadSymbol( connectionAddFilter, "dbus_connection_add_filter" ) && loadSymbol( connectionOpenPrivate, "dbus_connection_open_private" ) && loadSymbol( connectionClose, "dbus_connection_close" ) && @@ -146,6 +150,7 @@ class DBusLibrary { loadSymbol( messageGetInterface, "dbus_message_get_interface" ) && loadSymbol( messageGetMember, "dbus_message_get_member" ) && loadSymbol( messageGetType, "dbus_message_get_type" ) && + loadSymbol( messageGetReplySerial, "dbus_message_get_reply_serial" ) && loadSymbol( messageGetSender, "dbus_message_get_sender" ) && loadSymbol( messageGetArgs, "dbus_message_get_args" ) && loadSymbol( messageIterInitAppend, "dbus_message_iter_init_append" ) && @@ -164,6 +169,7 @@ class DBusLibrary { BusRegister busRegister{}; BusGetUniqueName busGetUniqueName{}; BusAddMatch busAddMatch{}; + BusRemoveMatch busRemoveMatch{}; ConnectionAddFilter connectionAddFilter{}; ConnectionOpenPrivate connectionOpenPrivate{}; ConnectionClose connectionClose{}; @@ -186,6 +192,7 @@ class DBusLibrary { MessageGetString messageGetMember{}; MessageGetString messageGetSender{}; MessageGetType messageGetType{}; + MessageGetReplySerial messageGetReplySerial{}; MessageGetArgs messageGetArgs{}; MessageIterInitAppend messageIterInitAppend{}; MessageIterInit messageIterInit{}; @@ -233,6 +240,12 @@ class AtSpiApplication final : public std::enable_shared_from_this mTextSnapshots; + UnorderedMap mTextStates; UnorderedSet mClients; + /** NameHasOwner checks sent when a client was first seen, by reply serial. */ + UnorderedMap mClientChecks; struct PendingText { AccessibilityManager* manager; AccessibilityNodeRef ref; - bool valueChanged; + bool wholeText; }; std::vector mPendingTexts; @@ -298,6 +316,11 @@ class AtSpiApplication final : public std::enable_shared_from_thissecond = TextSnapshot{ info.value, info.text }; + ScopedManager scopedManager( *this, &manager ); + // object:announcement (at-spi2-core 2.46): detail1 is the AtspiLive politeness, 1 polite and + // 2 assertive. Clients without support ignore the signal. + sendObjectEvent( manager.getRoot(), "Announcement", "", + priority == AccessibilityLive::Assertive ? 2 : 1, 0, message ); } -bool AtSpiApplication::hasPendingText( AccessibilityNodeRef ref ) const { - for ( const auto& text : mPendingTexts ) { - if ( text.manager == mManager && text.ref == ref ) - return true; +AtSpiApplication::TextState& AtSpiApplication::rememberText( AccessibilityNodeRef ref, + const AccessibilityTextInfo& text ) { + auto path = pathFromRef( ref ); + auto found = mTextStates.find( path ); + if ( found != mTextStates.end() ) + return found->second; + // Counting characters scans line sizes only; the contents are never copied. + return mTextStates + .emplace( std::move( path ), + TextState{ text, mManager ? mManager->getTextLength( ref ) : -1 } ) + .first->second; +} + +bool AtSpiApplication::onTextChanged( AccessibilityManager& manager, AccessibilityNodeRef ref, + const AccessibilityTextChange& change ) { + if ( !isAvailable() || !hasActiveClients() ) + return false; + if ( change.isWholeText() ) { + queueText( manager, ref, true ); + return true; } - return false; + ScopedManager scopedManager( *this, &manager ); + if ( !change.removed.empty() ) + sendObjectEvent( ref, "TextChanged", "delete", change.offset, + static_cast( change.removed.size() ), change.removed ); + if ( !change.inserted.empty() ) + sendObjectEvent( ref, "TextChanged", "insert", change.offset, + static_cast( change.inserted.size() ), change.inserted ); + auto found = mTextStates.find( pathFromRef( ref ) ); + if ( found != mTextStates.end() && found->second.length >= 0 ) + found->second.length += static_cast( change.inserted.size() ) - + static_cast( change.removed.size() ); + return true; } void AtSpiApplication::queueText( AccessibilityManager& manager, AccessibilityNodeRef ref, - bool valueChanged ) { + bool wholeText ) { for ( auto& text : mPendingTexts ) { if ( text.manager == &manager && text.ref == ref ) { - text.valueChanged |= valueChanged; + text.wholeText |= wholeText; return; } } - mPendingTexts.push_back( { &manager, ref, valueChanged } ); + mPendingTexts.push_back( { &manager, ref, wholeText } ); Window::Input::wakeUpEventLoop(); } @@ -183,10 +210,9 @@ void AtSpiApplication::flushPendingTexts() { if ( !text.manager->isValid( text.ref ) ) continue; ScopedManager scopedManager( *this, text.manager ); - if ( text.valueChanged ) - sendTextChanged( text.ref ); - else - sendTextSelection( text.ref, text.manager->getTextInfo( text.ref ) ); + if ( text.wholeText ) + sendTextReplaced( text.ref ); + sendTextSelection( text.ref, text.manager->getTextInfo( text.ref ) ); } } @@ -214,13 +240,13 @@ void AtSpiApplication::sendObjectEvent( AccessibilityNodeRef ref, const char* si void AtSpiApplication::sendTextSelection( AccessibilityNodeRef ref, const AccessibilityTextInfo& text ) { - auto found = mTextSnapshots.find( pathFromRef( ref ) ); - if ( found == mTextSnapshots.end() ) { - rememberText( ref, getNodeInfo( ref ) ); - found = mTextSnapshots.find( pathFromRef( ref ) ); - } - if ( found == mTextSnapshots.end() ) + auto path = pathFromRef( ref ); + auto found = mTextStates.find( path ); + if ( found == mTextStates.end() ) { + // A client that never looked at this text has no caret position to compare against. + rememberText( ref, text ); return; + } auto& previous = found->second.text; if ( previous.caretOffset != text.caretOffset ) sendObjectEvent( ref, "TextCaretMoved", "", text.caretOffset ); @@ -230,36 +256,44 @@ void AtSpiApplication::sendTextSelection( AccessibilityNodeRef ref, previous = text; } -void AtSpiApplication::sendTextChanged( AccessibilityNodeRef ref ) { - auto info = getNodeInfo( ref ); - auto path = pathFromRef( ref ); - auto found = mTextSnapshots.find( path ); - if ( found == mTextSnapshots.end() ) { - // A client which has not queried or focused this text has no old contents to diff. - rememberText( ref, info ); +void AtSpiApplication::sendTextReplaced( AccessibilityNodeRef ref ) { + // Short texts (form fields) carry their new contents; a document's would only flood speech. + static constexpr Int32 MaxReplacedTextReported = 4096; + const Int32 length = mManager ? mManager->getTextLength( ref ) : 0; + auto found = mTextStates.find( pathFromRef( ref ) ); + if ( found == mTextStates.end() ) return; - } - auto& previous = found->second.value; + if ( found->second.length > 0 ) + sendObjectEvent( ref, "TextChanged", "delete", 0, found->second.length ); + if ( length > 0 ) + sendObjectEvent( ref, "TextChanged", "insert", 0, length, + length <= MaxReplacedTextReported + ? mManager->getTextRange( ref, 0, length ) + : String() ); + found->second.length = length; +} + +void AtSpiApplication::sendTextDiff( AccessibilityNodeRef ref, const String& previous, + const String& current ) { size_t prefix = 0; - while ( prefix < previous.size() && prefix < info.value.size() && - previous[prefix] == info.value[prefix] ) + while ( prefix < previous.size() && prefix < current.size() && + previous[prefix] == current[prefix] ) ++prefix; size_t suffix = 0; - while ( suffix < previous.size() - prefix && suffix < info.value.size() - prefix && - previous[previous.size() - suffix - 1] == info.value[info.value.size() - suffix - 1] ) + while ( suffix < previous.size() - prefix && suffix < current.size() - prefix && + previous[previous.size() - suffix - 1] == current[current.size() - suffix - 1] ) ++suffix; const size_t removed = previous.size() - prefix - suffix; - const size_t inserted = info.value.size() - prefix - suffix; - if ( removed ) { + const size_t inserted = current.size() - prefix - suffix; + if ( removed ) sendObjectEvent( ref, "TextChanged", "delete", prefix, removed, previous.substr( prefix, removed ) ); - } - if ( inserted ) { + if ( inserted ) sendObjectEvent( ref, "TextChanged", "insert", prefix, inserted, - info.value.substr( prefix, inserted ) ); - } - previous = std::move( info.value ); - sendTextSelection( ref, info.text ); + current.substr( prefix, inserted ) ); + auto found = mTextStates.find( pathFromRef( ref ) ); + if ( found != mTextStates.end() ) + found->second.length = static_cast( current.size() ); } void AtSpiApplication::sendWindowChanged( AccessibilityManager& manager, bool added, Int32 index ) { diff --git a/src/eepp/ui/accessibility/accessibilitybackendatspimethods.cpp b/src/eepp/ui/accessibility/accessibilitybackendatspimethods.cpp index e76e149cc..22748e04a 100644 --- a/src/eepp/ui/accessibility/accessibilitybackendatspimethods.cpp +++ b/src/eepp/ui/accessibility/accessibilitybackendatspimethods.cpp @@ -303,10 +303,12 @@ DBusHandlerResult AtSpiApplication::handleAction( DBusMessage* request, Accessib auto action = actionAt( actions, index ); const char* name = actionName( action ); const char* empty = ""; + const std::string bindingStorage = actionKeyBinding( info, action ); + const char* binding = bindingStorage.c_str(); mDBus.messageIterOpenContainer( &array, 'r', nullptr, &structure ); appendBasic( structure, 's', &name ); appendBasic( structure, 's', &empty ); - appendBasic( structure, 's', &empty ); + appendBasic( structure, 's', &binding ); mDBus.messageIterCloseContainer( &array, &structure ); } mDBus.messageIterCloseContainer( &iter, &array ); @@ -324,8 +326,11 @@ DBusHandlerResult AtSpiApplication::handleAction( DBusMessage* request, Accessib std::strcmp( member, "GetLocalizedName" ) == 0 ) { const char* name = actionName( action ); sendBasic( request, 's', &name ); - } else if ( std::strcmp( member, "GetDescription" ) == 0 || - std::strcmp( member, "GetKeyBinding" ) == 0 ) { + } else if ( std::strcmp( member, "GetKeyBinding" ) == 0 ) { + const std::string bindingStorage = actionKeyBinding( info, action ); + const char* binding = bindingStorage.c_str(); + sendBasic( request, 's', &binding ); + } else if ( std::strcmp( member, "GetDescription" ) == 0 ) { const char* empty = ""; sendBasic( request, 's', &empty ); } else { @@ -337,64 +342,82 @@ DBusHandlerResult AtSpiApplication::handleAction( DBusMessage* request, Accessib DBusHandlerResult AtSpiApplication::handleText( DBusMessage* request, AccessibilityNodeRef ref, const char* member ) { - auto info = getNodeInfo( ref, true ); - if ( !info.text.valid ) + // Every method reads only the range it reports: copying a large document per query would + // make each caret movement cost the whole file. + auto info = getNodeInfo( ref, false ); + if ( !info.text.valid || !mManager ) return 1; - rememberText( ref, info, std::strcmp( member, "GetText" ) == 0 ); - const Int32 characterCount = static_cast( info.value.size() ); + info.text = mManager->getTextInfo( ref ); + // Answer with the current length, but keep the remembered one: it is what change events + // told the client, and a queued whole-text replacement must delete exactly that. + rememberText( ref, info.text ); + const Int32 characterCount = mManager->getTextLength( ref ); if ( std::strcmp( member, "GetText" ) == 0 ) { Int32 start = 0; Int32 end = -1; mDBus.messageGetArgs( request, nullptr, 'i', &start, 'i', &end, 0 ); start = std::max( 0, std::min( start, characterCount ) ); end = end < 0 ? characterCount : std::max( start, std::min( end, characterCount ) ); - std::string textStorage = info.value.substr( start, end - start ).toUtf8(); + std::string textStorage = mManager->getTextRange( ref, start, end ).toUtf8(); const char* text = textStorage.c_str(); sendBasic( request, 's', &text ); } else if ( std::strcmp( member, "GetStringAtOffset" ) == 0 ) { Int32 offset = 0; Uint32 granularity = 0; mDBus.messageGetArgs( request, nullptr, 'i', &offset, 'u', &granularity, 0 ); - Int32 start = std::max( 0, std::min( offset, characterCount ) ); - Int32 end = start; - if ( granularity == 0 && start < characterCount ) { - end = start + 1; - } else if ( granularity == 1 ) { - auto isSpace = [&info]( Int32 index ) { - auto character = info.value[index]; - return character == ' ' || character == '\t' || character == '\n' || - character == '\r'; - }; - while ( start > 0 && !isSpace( start - 1 ) ) - --start; - end = std::max( 0, std::min( offset, characterCount ) ); - while ( end < characterCount && !isSpace( end ) ) - ++end; - } else if ( granularity == 3 || granularity == 4 ) { - while ( start > 0 && info.value[start - 1] != '\n' ) - --start; - while ( end < characterCount && info.value[end] != '\n' ) - ++end; - if ( end < characterCount ) - ++end; + offset = std::max( 0, std::min( offset, characterCount ) ); + Int32 start = offset; + Int32 end = offset; + String text; + if ( granularity == 0 ) { + if ( offset < characterCount ) + end = offset + 1; + text = mManager->getTextRange( ref, start, end ); + } else if ( granularity == 1 || granularity == 3 || granularity == 4 ) { + // Lines and paragraphs are the same here. Words never cross a newline, so both are + // found within the line containing the offset. + Int32 lineStart = 0; + Int32 lineEnd = 0; + mManager->getTextLineBounds( ref, offset, lineStart, lineEnd ); + const String line = mManager->getTextRange( ref, lineStart, lineEnd ); + if ( granularity == 1 ) { + auto isSpace = [&line, lineStart]( Int32 index ) { + auto character = line[index - lineStart]; + return character == ' ' || character == '\t' || character == '\n' || + character == '\r'; + }; + while ( start > lineStart && !isSpace( start - 1 ) ) + --start; + while ( end < lineEnd && !isSpace( end ) ) + ++end; + text = line.substr( start - lineStart, end - start ); + } else { + start = lineStart; + end = lineEnd; + text = line; + } } else { start = 0; end = characterCount; + text = mManager->getTextRange( ref, start, end ); } - std::string textStorage = info.value.substr( start, end - start ).toUtf8(); - const char* text = textStorage.c_str(); + std::string textStorage = text.toUtf8(); + const char* textValue = textStorage.c_str(); DBusMessage* reply = mDBus.messageNewMethodReturn( request ); DBusMessageIter iter; mDBus.messageIterInitAppend( reply, &iter ); - appendBasic( iter, 's', &text ); + appendBasic( iter, 's', &textValue ); appendBasic( iter, 'i', &start ); appendBasic( iter, 'i', &end ); send( reply ); } else if ( std::strcmp( member, "GetCharacterAtOffset" ) == 0 ) { Int32 offset = 0; mDBus.messageGetArgs( request, nullptr, 'i', &offset, 0 ); - Int32 character = offset >= 0 && offset < characterCount ? info.value[offset] : 0; - sendBasic( request, 'i', &character ); + const String character = offset >= 0 && offset < characterCount + ? mManager->getTextRange( ref, offset, offset + 1 ) + : String(); + Int32 value = character.empty() ? 0 : static_cast( character[0] ); + sendBasic( request, 'i', &value ); } else if ( std::strcmp( member, "GetAttributes" ) == 0 ) { DBusMessage* reply = mDBus.messageNewMethodReturn( request ); DBusMessageIter iter; @@ -464,18 +487,23 @@ DBusHandlerResult AtSpiApplication::handleText( DBusMessage* request, Accessibil DBusHandlerResult AtSpiApplication::handleEditableText( DBusMessage* request, AccessibilityNodeRef ref, const char* member ) { - auto info = getNodeInfo( ref, true ); + auto info = getNodeInfo( ref, false ); if ( !info.text.valid || !hasState( info.states, AccessibilityState::Editable ) || - std::strcmp( member, "SetTextContents" ) != 0 ) + std::strcmp( member, "SetTextContents" ) != 0 || !mManager ) return 1; const char* contents = nullptr; if ( !mDBus.messageGetArgs( request, nullptr, 's', &contents, 0 ) ) return 1; - rememberText( ref, info, true ); - int success = - mManager && mManager->performAction( - ref, { AccessibilityAction::SetText, - String::fromUtf8( std::string_view( contents ? contents : "" ) ) } ); + // The client sent the whole new text, so copying the old one costs no more than the request. + // Report the replacement as its minimal diff instead of the reset and insert it performs. + rememberText( ref, mManager->getTextInfo( ref ) ); + const String previous = mManager->getTextRange( ref, 0, mManager->getTextLength( ref ) ); + const String current = String::fromUtf8( std::string_view( contents ? contents : "" ) ); + mManager->setSuppressedTextChanges( ref ); + int success = mManager->performAction( ref, { AccessibilityAction::SetText, current } ); + mManager->setSuppressedTextChanges( {} ); + if ( success ) + sendTextDiff( ref, previous, current ); sendBasic( request, 'b', &success ); return 0; } @@ -487,10 +515,8 @@ DBusHandlerResult AtSpiApplication::handlePropertiesGet( DBusMessage* request, if ( !mDBus.messageGetArgs( request, nullptr, 's', &requestedInterface, 's', &property, 0 ) || !requestedInterface || !property ) return 1; - const bool includeValue = ( std::strcmp( requestedInterface, "org.a11y.atspi.Text" ) == 0 && - std::strcmp( property, "CharacterCount" ) == 0 ) || - ( std::strcmp( requestedInterface, "org.a11y.atspi.Value" ) == 0 && - std::strcmp( property, "CurrentValue" ) == 0 ); + const bool includeValue = std::strcmp( requestedInterface, "org.a11y.atspi.Value" ) == 0 && + std::strcmp( property, "CurrentValue" ) == 0; const bool needsInfo = std::strcmp( requestedInterface, "org.a11y.atspi.Action" ) == 0 || std::strcmp( requestedInterface, "org.a11y.atspi.Text" ) == 0 || std::strcmp( requestedInterface, "org.a11y.atspi.Value" ) == 0 || @@ -515,7 +541,7 @@ DBusHandlerResult AtSpiApplication::handlePropertiesGet( DBusMessage* request, ( std::strcmp( property, "CharacterCount" ) == 0 || std::strcmp( property, "CaretOffset" ) == 0 ) ) { Int32 value = std::strcmp( property, "CharacterCount" ) == 0 - ? static_cast( info.value.size() ) + ? ( mManager ? mManager->getTextLength( ref ) : 0 ) : info.text.caretOffset; mDBus.messageIterOpenContainer( &iter, 'v', "i", &variant ); appendBasic( variant, 'i', &value ); diff --git a/src/eepp/ui/accessibility/accessibilitybackendatspitree.cpp b/src/eepp/ui/accessibility/accessibilitybackendatspitree.cpp index da774f674..682089aea 100644 --- a/src/eepp/ui/accessibility/accessibilitybackendatspitree.cpp +++ b/src/eepp/ui/accessibility/accessibilitybackendatspitree.cpp @@ -270,6 +270,28 @@ const char* AtSpiApplication::actionName( AccessibilityAction action ) const { return ""; } +std::string AtSpiApplication::actionKeyBinding( const AccessibilityNodeInfo& info, + AccessibilityAction action ) const { + if ( info.shortcut.key.empty() || + ( action != AccessibilityAction::Press && action != AccessibilityAction::Toggle && + action != AccessibilityAction::Select ) ) + return {}; + // Only the accelerator part is known; it uses GTK accelerator syntax ("s"). + std::string binding( ";;" ); + const auto modifiers = info.shortcut.modifiers; + if ( modifiers & AccessibilityShortcut::Control ) + binding += ""; + if ( modifiers & AccessibilityShortcut::Shift ) + binding += ""; + if ( modifiers & AccessibilityShortcut::Alt ) + binding += ""; + if ( modifiers & AccessibilityShortcut::Meta ) + binding += ""; + const auto& key = info.shortcut.key; + binding += key.size() == 1 ? String::toLower( key ) : key; + return binding; +} + bool AtSpiApplication::readPropertySet( DBusMessage* request, const char*& interface, const char*& property, DBusMessageIter& variant ) { DBusMessageIter iter; diff --git a/src/eepp/ui/accessibility/accessibilitybackendmacos.mm b/src/eepp/ui/accessibility/accessibilitybackendmacos.mm index eae7c0f70..dc5cbe972 100644 --- a/src/eepp/ui/accessibility/accessibilitybackendmacos.mm +++ b/src/eepp/ui/accessibility/accessibilitybackendmacos.mm @@ -133,12 +133,10 @@ static NSAccessibilitySubrole nativeSubrole( const AccessibilityNodeInfo& info ) } } -static std::u16string utf16String( const String& string ) { - return string.toUtf16(); -} +// AppKit text offsets are UTF-16 code units; eepp's are code points. The helpers below work on a +// UTF-16 copy that MacAccessibilityState::textFor() caches per text revision. -static NSUInteger codePointToUTF16( const String& string, Int32 offset ) { - const auto text = utf16String( string ); +static NSUInteger codePointToUTF16( const std::u16string& text, Int32 offset ) { NSUInteger codePoints = 0; NSUInteger utf16Offset = 0; const NSUInteger wanted = offset > 0 ? static_cast( offset ) : 0; @@ -154,8 +152,7 @@ static NSUInteger codePointToUTF16( const String& string, Int32 offset ) { return utf16Offset; } -static Int32 utf16ToCodePoint( const String& string, NSUInteger offset ) { - const auto text = utf16String( string ); +static Int32 utf16ToCodePoint( const std::u16string& text, NSUInteger offset ) { NSUInteger index = 0; Int32 codePoints = 0; while ( index < text.size() && index < offset ) { @@ -170,11 +167,15 @@ static Int32 utf16ToCodePoint( const String& string, NSUInteger offset ) { return codePoints; } -static NSRange textRange( const AccessibilityNodeInfo& info, const String& value ) { - if ( !info.text.valid ) +static Int32 utf16ToCodePoint( const String& string, NSUInteger offset ) { + return utf16ToCodePoint( string.toUtf16(), offset ); +} + +static NSRange textRange( const AccessibilityTextInfo& info, const std::u16string& text ) { + if ( !info.valid ) return NSMakeRange( 0, 0 ); - const NSUInteger start = codePointToUTF16( value, info.text.selectionStart ); - const NSUInteger end = codePointToUTF16( value, info.text.selectionEnd ); + const NSUInteger start = codePointToUTF16( text, info.selectionStart ); + const NSUInteger end = codePointToUTF16( text, info.selectionEnd ); return NSMakeRange( start, end >= start ? end - start : 0 ); } @@ -182,6 +183,23 @@ static bool validRange( NSRange range, NSString* string ) { return range.location <= string.length && range.length <= string.length - range.location; } +static bool validRange( NSRange range, const std::u16string& text ) { + return range.location <= text.size() && range.length <= text.size() - range.location; +} + +/** Builds an NSString from part of the cached UTF-16 text, copying only that part. */ +static NSString* toNSString( const std::u16string& text, NSRange range ) { + return validRange( range, text ) + ? [NSString stringWithCharacters:reinterpret_cast( text.data() + + range.location ) + length:range.length] + : nil; +} + +static NSString* toNSString( const std::u16string& text ) { + return toNSString( text, NSMakeRange( 0, text.size() ) ); +} + static void addAccessibilityChild( NSView* view, id child ) { if ( !view || !child ) return; @@ -285,6 +303,15 @@ class MacAccessibilityState : public std::enable_shared_from_thisgetTextInfo( ref ) : AccessibilityTextInfo(); + } + + /** A text element's contents in UTF-16, cached until its text revision changes; nullptr for + * other elements and for protected text. */ + std::shared_ptr textFor( AccessibilityNodeRef ref ); AccessibilityNodeRef hitTest( NSPoint point ); AccessibilityNodeRef focused() const; bool perform( AccessibilityNodeRef ref, AccessibilityAction action, const String& value = {} ); @@ -315,6 +342,15 @@ class MacAccessibilityState : public std::enable_shared_from_this text; + }; + /** A handful of entries covers the focused editor and the fields around it. */ + static constexpr size_t TextCacheSize = 4; + std::vector mTextCache; + size_t mNextTextCacheEntry{ 0 }; NSWindow* mNativeWindow{ nil }; std::string mWindowIdentifier; AccessibilityNodeRef mRoot; @@ -435,6 +471,36 @@ AccessibilityNodeInfo MacAccessibilityState::infoFor( AccessibilityNodeRef ref, : AccessibilityNodeInfo(); } +std::shared_ptr MacAccessibilityState::textFor( AccessibilityNodeRef ref ) { + if ( !isValid( ref ) ) + return nullptr; + const auto revision = mManager->getTextRevision( ref ); + if ( !revision.isValid() ) + return nullptr; + for ( const auto& entry : mTextCache ) { + if ( entry.ref == ref && entry.revision == revision ) + return entry.text; + } + CachedText entry{ + ref, revision, + std::make_shared( + mManager->getTextRange( ref, 0, mManager->getTextLength( ref ) ).toUtf16() ) }; + for ( auto& cached : mTextCache ) { + if ( cached.ref == ref ) { + cached = std::move( entry ); + return cached.text; + } + } + if ( mTextCache.size() < TextCacheSize ) { + mTextCache.push_back( std::move( entry ) ); + return mTextCache.back().text; + } + auto& replaced = mTextCache[mNextTextCacheEntry]; + mNextTextCacheEntry = ( mNextTextCacheEntry + 1 ) % TextCacheSize; + replaced = std::move( entry ); + return replaced.text; +} + NSArray* MacAccessibilityState::childrenFor( AccessibilityNodeRef ref ) { if ( !isValid( ref ) ) return nil; @@ -773,6 +839,17 @@ bool MacAccessibilityState::perform( AccessibilityNodeRef ref, AccessibilityActi return state ? state->infoFor( _ref, false ) : AccessibilityNodeInfo(); } +/** The cached UTF-16 contents of a text element; nullptr otherwise. */ +- (std::shared_ptr)textContents { + auto state = [self accessibilityState]; + return state ? state->textFor( _ref ) : nullptr; +} + +- (AccessibilityTextInfo)textSelection { + auto state = [self accessibilityState]; + return state ? state->textInfoFor( _ref ) : AccessibilityTextInfo(); +} + - (BOOL)isAccessibilityElement { auto state = [self accessibilityState]; if ( !state ) @@ -859,6 +936,9 @@ bool MacAccessibilityState::perform( AccessibilityNodeRef ref, AccessibilityActi } - (id)accessibilityValue { + // A text element's value comes from the cached contents, not a fresh copy per query. + if ( auto text = [self textContents] ) + return text->empty() ? nil : toNSString( *text ); const auto info = [self nodeInfo]; if ( info.role == AccessibilityRole::Label || info.role == AccessibilityRole::Text ) return info.name.empty() ? nil : toNSString( info.name ); @@ -1147,13 +1227,13 @@ bool MacAccessibilityState::perform( AccessibilityNodeRef ref, AccessibilityActi auto state = [self accessibilityState]; if ( !state ) return; - const auto info = state->infoFor( _ref, true ); - NSString* nativeText = toNSString( info.value ); - if ( !info.text.valid || !validRange( range, nativeText ) || + const auto info = state->infoFor( _ref, false ); + auto text = state->textFor( _ref ); + if ( !text || !validRange( range, *text ) || !hasAction( info.actions, AccessibilityAction::SetTextSelection ) ) return; - const Int32 start = utf16ToCodePoint( info.value, range.location ); - const Int32 end = utf16ToCodePoint( info.value, NSMaxRange( range ) ); + const Int32 start = utf16ToCodePoint( *text, range.location ); + const Int32 end = utf16ToCodePoint( *text, NSMaxRange( range ) ); state->perform( _ref, AccessibilityAction::SetTextSelection, String( std::to_string( start ) + ":" + std::to_string( end ) ) ); } @@ -1167,11 +1247,12 @@ bool MacAccessibilityState::perform( AccessibilityNodeRef ref, AccessibilityActi auto state = [self accessibilityState]; if ( !state || !replacement ) return; - const auto info = state->infoFor( _ref, true ); - if ( !info.text.valid || !hasAction( info.actions, AccessibilityAction::SetText ) ) + const auto info = state->infoFor( _ref, false ); + auto text = state->textFor( _ref ); + if ( !text || !hasAction( info.actions, AccessibilityAction::SetText ) ) return; - NSMutableString* updated = [toNSString( info.value ) mutableCopy]; - const NSRange selected = textRange( info, info.value ); + NSMutableString* updated = [toNSString( *text ) mutableCopy]; + const NSRange selected = textRange( state->textInfoFor( _ref ), *text ); if ( !validRange( selected, updated ) ) { EE_OBJC_RELEASE( updated ); return; @@ -1224,6 +1305,26 @@ bool MacAccessibilityState::perform( AccessibilityNodeRef ref, AccessibilityActi return self.accessibilityLabel; if ( [attribute isEqualToString:NSAccessibilityHelpAttribute] ) return self.accessibilityHelp; + if ( [attribute isEqualToString:NSAccessibilityMenuItemCmdCharAttribute] ) { + const auto shortcut = [self nodeInfoWithoutValue].shortcut; + if ( String::utf8Length( shortcut.key ) != 1 ) + return nil; + return toNSString( shortcut.key ).uppercaseString; + } + if ( [attribute isEqualToString:NSAccessibilityMenuItemCmdModifiersAttribute] ) { + const auto modifiers = [self nodeInfoWithoutValue].shortcut.modifiers; + // kAXMenuItemModifier*: Shift 1, Option 2, Control 4, NoCommand 8 (Command is implied). + NSInteger flags = 0; + if ( modifiers & AccessibilityShortcut::Shift ) + flags |= 1; + if ( modifiers & AccessibilityShortcut::Alt ) + flags |= 2; + if ( modifiers & AccessibilityShortcut::Control ) + flags |= 4; + if ( !( modifiers & AccessibilityShortcut::Meta ) ) + flags |= 8; + return @( flags ); + } if ( [attribute isEqualToString:NSAccessibilityTitleAttribute] ) return self.accessibilityTitle; if ( [attribute isEqualToString:NSAccessibilityValueAttribute] ) @@ -1276,21 +1377,18 @@ bool MacAccessibilityState::perform( AccessibilityNodeRef ref, AccessibilityActi return [NSValue valueWithPoint:self.accessibilityFrame.origin]; if ( [attribute isEqualToString:NSAccessibilitySizeAttribute] ) return [NSValue valueWithSize:self.accessibilityFrame.size]; - const auto info = state->infoFor( _ref, true ); - if ( info.text.valid ) { - const String& text = info.value; - NSString* string = toNSString( text ); - const NSRange selected = textRange( info, text ); + if ( auto text = state->textFor( _ref ) ) { + const NSRange selected = textRange( state->textInfoFor( _ref ), *text ); if ( [attribute isEqualToString:NSAccessibilitySelectedTextAttribute] ) - return [string substringWithRange:selected]; + return toNSString( *text, selected ); if ( [attribute isEqualToString:NSAccessibilitySelectedTextRangeAttribute] ) return [NSValue valueWithRange:selected]; if ( [attribute isEqualToString:NSAccessibilitySelectedTextRangesAttribute] ) return @[ [NSValue valueWithRange:selected] ]; if ( [attribute isEqualToString:NSAccessibilityNumberOfCharactersAttribute] ) - return @( string.length ); + return @( text->size() ); if ( [attribute isEqualToString:NSAccessibilityVisibleCharacterRangeAttribute] ) - return [NSValue valueWithRange:NSMakeRange( 0, string.length )]; + return [NSValue valueWithRange:NSMakeRange( 0, text->size() )]; if ( [attribute isEqualToString:NSAccessibilityInsertionPointLineNumberAttribute] ) return @( self.accessibilityInsertionPointLineNumber ); } @@ -1324,6 +1422,10 @@ bool MacAccessibilityState::perform( AccessibilityNodeRef ref, AccessibilityActi [names addObject:NSAccessibilityDescriptionAttribute]; if ( !info.description.empty() ) [names addObject:NSAccessibilityHelpAttribute]; + if ( String::utf8Length( info.shortcut.key ) == 1 ) + [names addObjectsFromArray:@[ + NSAccessibilityMenuItemCmdCharAttribute, NSAccessibilityMenuItemCmdModifiersAttribute + ]]; const bool hasBooleanValue = info.role == AccessibilityRole::CheckBox || info.role == AccessibilityRole::RadioButton || info.role == AccessibilityRole::CheckMenuItem || @@ -1411,34 +1513,29 @@ bool MacAccessibilityState::perform( AccessibilityNodeRef ref, AccessibilityActi } - (NSString*)accessibilitySelectedText { - const auto info = [self nodeInfo]; - if ( !info.text.valid ) - return nil; - NSString* string = toNSString( info.value ); - const NSRange selected = textRange( info, info.value ); - return validRange( selected, string ) ? [string substringWithRange:selected] : nil; + auto text = [self textContents]; + return text ? toNSString( *text, textRange( [self textSelection], *text ) ) : nil; } - (NSArray*)accessibilitySelectedTextRanges { - const auto info = [self nodeInfo]; - return info.text.valid ? @[ [NSValue valueWithRange:textRange( info, info.value )] ] : nil; + auto text = [self textContents]; + return text ? @[ [NSValue valueWithRange:textRange( [self textSelection], *text )] ] : nil; } - (NSInteger)accessibilityNumberOfCharacters { - const auto info = [self nodeInfo]; - return info.text.valid ? static_cast( toNSString( info.value ).length ) : 0; + auto text = [self textContents]; + return text ? static_cast( text->size() ) : 0; } - (NSInteger)accessibilityInsertionPointLineNumber { - const auto info = [self nodeInfo]; - return info.text.valid - ? [self accessibilityLineForIndex:static_cast( codePointToUTF16( - info.value, info.text.caretOffset ) )] - : NSNotFound; + auto text = [self textContents]; + return text ? [self accessibilityLineForIndex:static_cast( codePointToUTF16( + *text, [self textSelection].caretOffset ) )] + : NSNotFound; } - (NSArray*)accessibilityParameterizedAttributeNames { - if ( ![self nodeInfo].text.valid ) + if ( ![self textContents] ) return @[]; return @[ NSAccessibilityStringForRangeParameterizedAttribute, @@ -1452,10 +1549,9 @@ bool MacAccessibilityState::perform( AccessibilityNodeRef ref, AccessibilityActi - (id)accessibilityAttributeValue:(NSAccessibilityParameterizedAttributeName)attribute forParameter:(id)parameter { - const auto info = [self nodeInfo]; - if ( !info.text.valid ) + auto text = [self textContents]; + if ( !text ) return nil; - NSString* string = toNSString( info.value ); if ( [attribute isEqualToString:NSAccessibilityStringForRangeParameterizedAttribute] || [attribute isEqualToString:NSAccessibilityAttributedStringForRangeParameterizedAttribute] || @@ -1464,22 +1560,21 @@ bool MacAccessibilityState::perform( AccessibilityNodeRef ref, AccessibilityActi return nil; NSRange range; [parameter getValue:&range]; - if ( !validRange( range, string ) ) + if ( !validRange( range, *text ) ) return nil; if ( [attribute isEqualToString:NSAccessibilityStringForRangeParameterizedAttribute] ) - return [string substringWithRange:range]; + return toNSString( *text, range ); if ( [attribute isEqualToString:NSAccessibilityAttributedStringForRangeParameterizedAttribute] ) return EE_OBJC_AUTORELEASE( - [[NSAttributedString alloc] initWithString:[string substringWithRange:range]] ); + [[NSAttributedString alloc] initWithString:toNSString( *text, range )] ); return [NSValue valueWithRect:self.accessibilityFrame]; } if ( [attribute isEqualToString:NSAccessibilityRangeForIndexParameterizedAttribute] && [parameter respondsToSelector:@selector( unsignedIntegerValue )] ) { - const NSUInteger index = [parameter unsignedIntegerValue]; - if ( index >= string.length ) - return nil; - return [NSValue valueWithRange:[string rangeOfComposedCharacterSequenceAtIndex:index]]; + const NSRange range = [self + accessibilityRangeForIndex:static_cast( [parameter unsignedIntegerValue] )]; + return range.location == NSNotFound ? nil : [NSValue valueWithRange:range]; } if ( [attribute isEqualToString:NSAccessibilityLineForIndexParameterizedAttribute] && [parameter respondsToSelector:@selector( integerValue )] ) @@ -1491,30 +1586,35 @@ bool MacAccessibilityState::perform( AccessibilityNodeRef ref, AccessibilityActi } - (NSRange)accessibilitySelectedTextRange { - const auto info = [self nodeInfo]; - return textRange( info, info.value ); + auto text = [self textContents]; + return text ? textRange( [self textSelection], *text ) : NSMakeRange( 0, 0 ); } - (NSRange)accessibilityVisibleCharacterRange { - const auto info = [self nodeInfo]; - return info.text.valid ? NSMakeRange( 0, toNSString( info.value ).length ) - : NSMakeRange( 0, 0 ); + auto text = [self textContents]; + return text ? NSMakeRange( 0, text->size() ) : NSMakeRange( 0, 0 ); } - (NSRange)accessibilityRangeForIndex:(NSInteger)index { - const auto info = [self nodeInfo]; - NSString* string = toNSString( info.value ); - return index >= 0 && static_cast( index ) < string.length - ? [string rangeOfComposedCharacterSequenceAtIndex:static_cast( index )] - : NSMakeRange( NSNotFound, 0 ); + auto text = [self textContents]; + if ( !text || index < 0 || static_cast( index ) >= text->size() ) + return NSMakeRange( NSNotFound, 0 ); + // A composed sequence can be arbitrarily long (a base followed by many combining marks), so + // query the whole cached text. Wrapping it without copying keeps the query free of allocations + // proportional to the document; the cache entry outlives the string, which is released here. + NSString* string = [[NSString alloc] + initWithCharactersNoCopy:const_cast( + reinterpret_cast( text->data() ) ) + length:text->size() + freeWhenDone:NO]; + const NSRange range = [string rangeOfComposedCharacterSequenceAtIndex:index]; + EE_OBJC_RELEASE( string ); + return range; } - (NSString*)accessibilityStringForRange:(NSRange)range { - const auto info = [self nodeInfo]; - if ( !info.text.valid ) - return nil; - NSString* string = toNSString( info.value ); - return validRange( range, string ) ? [string substringWithRange:range] : nil; + auto text = [self textContents]; + return text ? toNSString( *text, range ) : nil; } - (NSAttributedString*)accessibilityAttributedStringForRange:(NSRange)range { @@ -1523,38 +1623,27 @@ bool MacAccessibilityState::perform( AccessibilityNodeRef ref, AccessibilityActi } - (NSInteger)accessibilityLineForIndex:(NSInteger)index { - const auto info = [self nodeInfo]; - NSString* string = toNSString( info.value ); - if ( !info.text.valid || index < 0 || static_cast( index ) > string.length ) + auto text = [self textContents]; + if ( !text || index < 0 || static_cast( index ) > text->size() ) return NSNotFound; - NSInteger line = 0; - for ( NSInteger offset = 0; offset < index; ++offset ) { - if ( [string characterAtIndex:static_cast( offset )] == '\n' ) - ++line; - } - return line; + return static_cast( std::count( text->begin(), text->begin() + index, u'\n' ) ); } - (NSRange)accessibilityRangeForLine:(NSInteger)wantedLine { - const auto info = [self nodeInfo]; - NSString* string = toNSString( info.value ); - if ( !info.text.valid || wantedLine < 0 ) + auto text = [self textContents]; + if ( !text || wantedLine < 0 ) return NSMakeRange( NSNotFound, 0 ); - NSUInteger start = 0; - for ( NSInteger line = 0; start <= string.length; ++line ) { - NSUInteger lineStart = 0; - NSUInteger lineEnd = 0; - [string getLineStart:&lineStart - end:&lineEnd - contentsEnd:nil - forRange:NSMakeRange( start, 0 )]; - if ( line == wantedLine ) - return NSMakeRange( lineStart, lineEnd - lineStart ); - if ( lineEnd <= start ) - break; - start = lineEnd; + // Lines end after their newline; text ending in a newline has an empty last line. + size_t start = 0; + for ( NSInteger line = 0; line < wantedLine; ++line ) { + const size_t newline = text->find( u'\n', start ); + if ( newline == std::u16string::npos ) + return NSMakeRange( NSNotFound, 0 ); + start = newline + 1; } - return NSMakeRange( NSNotFound, 0 ); + const size_t newline = text->find( u'\n', start ); + const size_t end = newline == std::u16string::npos ? text->size() : newline + 1; + return NSMakeRange( start, end - start ); } - (NSRect)accessibilityFrameForRange:(NSRange)range { @@ -1871,6 +1960,21 @@ class MacAccessibilityBackend final : public AccessibilityBackend { mPendingEvents.emplace_back( event ); } + void announce( const String& message, AccessibilityLive priority ) override { + // Called from the manager's update() on the main thread, outside eepp locks. + NSWindow* window = mState && mState->hasActiveClient() ? mState->nativeWindow() : nil; + if ( !window ) + return; + NSDictionary* userInfo = @{ + NSAccessibilityAnnouncementKey : toNSString( message ), + NSAccessibilityPriorityKey : + @( priority == AccessibilityLive::Assertive ? NSAccessibilityPriorityHigh + : NSAccessibilityPriorityMedium ) + }; + NSAccessibilityPostNotificationWithUserInfo( + window, NSAccessibilityAnnouncementRequestedNotification, userInfo ); + } + void onSourceInvalidated( AccessibilitySourceId source ) override { // AppKit drops UIElementDestroyed once the source no longer resolves. The manager calls // this on the main thread immediately before reset(), outside model locks, so destruction diff --git a/src/eepp/ui/accessibility/accessibilitybackenduia.cpp b/src/eepp/ui/accessibility/accessibilitybackenduia.cpp index 41a842a98..f03211037 100644 --- a/src/eepp/ui/accessibility/accessibilitybackenduia.cpp +++ b/src/eepp/ui/accessibility/accessibilitybackenduia.cpp @@ -42,6 +42,7 @@ void UIAutomationProviderContext::invalidateProvider( AccessibilityNodeRef ref ) { std::lock_guard lock( mProviderMutex ); mInvalidatedProviders.emplace_back( providerToDetach ); + mHasInvalidatedProviders.store( true, std::memory_order_release ); } } @@ -62,13 +63,19 @@ void UIAutomationProviderContext::invalidateSource( AccessibilitySourceId source } std::lock_guard lock( mProviderMutex ); mInvalidatedProviders.insert( mInvalidatedProviders.end(), providers.begin(), providers.end() ); + mHasInvalidatedProviders.store( true, std::memory_order_release ); } void UIAutomationProviderContext::disconnectInvalidatedProviders() { + // Producers set the flag under the lock after appending, so a provider queued after this + // check is drained by the next update. + if ( !mHasInvalidatedProviders.load( std::memory_order_acquire ) ) + return; std::vector providers; { std::lock_guard lock( mProviderMutex ); providers.swap( mInvalidatedProviders ); + mHasInvalidatedProviders.store( false, std::memory_order_relaxed ); } // Outgoing COM calls can re-enter an STA. Run these only after widget destruction finishes; // the detached providers already reject queries and retain their original cache reference. @@ -92,6 +99,7 @@ void UIAutomationProviderContext::pruneSource( AccessibilitySourceId sourceId ) } it->second->detach(); mInvalidatedProviders.emplace_back( it->second ); + mHasInvalidatedProviders.store( true, std::memory_order_release ); it = source->second.erase( it ); } if ( source->second.empty() ) @@ -152,6 +160,8 @@ class UIAutomationAccessibilityBackend final : public AccessibilityBackend { void onEvent( const AccessibilityPendingEvent& event ) override; + void announce( const String& message, AccessibilityLive priority ) override; + void onSourceInvalidated( AccessibilitySourceId sourceId ) override { if ( mContext ) mContext->invalidateSource( sourceId ); @@ -301,6 +311,48 @@ void UIAutomationAccessibilityBackend::onEvent( const AccessibilityPendingEvent& mPendingEvents.emplace_back( event ); } +namespace { + +// UiaRaiseNotificationEvent exists from Windows 10 1709. Resolve it at runtime so the library +// still loads on older systems and with SDK headers that predate it. +using RaiseNotificationEventFunction = HRESULT( WINAPI* )( IRawElementProviderSimple*, int, int, + BSTR, BSTR ); + +RaiseNotificationEventFunction raiseNotificationEventFunction() { + static const auto function = []() -> RaiseNotificationEventFunction { + HMODULE module = GetModuleHandleW( L"uiautomationcore.dll" ); + return module ? reinterpret_cast( reinterpret_cast( + GetProcAddress( module, "UiaRaiseNotificationEvent" ) ) ) + : nullptr; + }(); + return function; +} + +} // namespace + +void UIAutomationAccessibilityBackend::announce( const String& message, + AccessibilityLive priority ) { + auto raise = raiseNotificationEventFunction(); + if ( !raise || !mContext || !mContext->hasActiveClients() ) + return; + UIAutomationProvider* root = mContext->provider( mManager.getRoot() ); + if ( !root ) + return; + // NotificationKind_Other = 4; NotificationProcessing_ImportantAll = 0 for assertive and + // NotificationProcessing_MostRecent = 3 for polite, so a stream of updates speaks the latest. + constexpr int NotificationKindOther = 4; + const int processing = priority == AccessibilityLive::Assertive ? 0 : 3; + const auto wide = message.toWideString(); + BSTR display = SysAllocStringLen( wide.data(), static_cast( wide.size() ) ); + BSTR activity = SysAllocString( L"eepp.announcement" ); + if ( display && activity ) + raise( static_cast( root ), NotificationKindOther, processing, + display, activity ); + SysFreeString( display ); + SysFreeString( activity ); + root->Release(); +} + void UIAutomationAccessibilityBackend::raiseEvent( const AccessibilityPendingEvent& event ) { if ( !mContext || !mContext->hasActiveClients() ) return; diff --git a/src/eepp/ui/accessibility/accessibilitybackenduia.hpp b/src/eepp/ui/accessibility/accessibilitybackenduia.hpp index 793a4caf6..af60d1293 100644 --- a/src/eepp/ui/accessibility/accessibilitybackenduia.hpp +++ b/src/eepp/ui/accessibility/accessibilitybackenduia.hpp @@ -59,6 +59,15 @@ inline UINT accessibilityDispatchMessage() { class UIAutomationProvider; +/** A text element's contents at one revision, converted once and shared by every range query + * until the text changes. Immutable, so UIA threads may read it while the UI thread replaces it. */ +struct TextContents { + /** UTF-16 of the exposed text, the base for code-point offsets. */ + std::u16string source; + /** What UIA sees: `source` with CRLF line endings. */ + std::u16string text; +}; + class UIAutomationProviderContext final : public std::enable_shared_from_this { private: @@ -266,6 +275,10 @@ class UIAutomationProviderContext final void detach(); + /** UI thread only: the element's contents at its current revision, from a small cache. */ + std::shared_ptr textContents( AccessibilityManager& manager, + AccessibilityNodeRef ref ); + private: void removePendingRequest( PendingRequest* request ) { std::lock_guard lock( mPendingMutex ); @@ -286,6 +299,9 @@ class UIAutomationProviderContext final std::atomic mAlive{ true }; std::atomic mClientsListening{ false }; std::atomic mClientObserved{ false }; + /** Set, under mProviderMutex, whenever mInvalidatedProviders gains entries: the per-update + * drain skips the lock while there is nothing to release. */ + std::atomic mHasInvalidatedProviders{ false }; std::atomic mAdvisedEventCount{ 0 }; std::mutex mProviderMutex; std::unordered_map> @@ -293,6 +309,15 @@ class UIAutomationProviderContext final std::vector mInvalidatedProviders; std::mutex mPendingMutex; std::vector> mPendingRequests; + struct CachedTextContents { + AccessibilityNodeRef ref; + AccessibilityTextRevision revision; + std::shared_ptr contents; + }; + /** A handful of entries covers the focused editor and the fields around it. */ + static constexpr size_t TextCacheSize = 4; + std::vector mTextCache; + size_t mNextTextCacheEntry{ 0 }; }; struct NavigationResult { @@ -307,13 +332,16 @@ struct GeometryResult { }; struct TextSnapshot { - String source; - std::u16string text; + std::shared_ptr contents; AccessibilityActions actions{}; int selectionStart{}; int selectionEnd{}; Math::Rectf bounds; bool boundsValid{ false }; + + const std::u16string& source() const { return contents->source; } + + const std::u16string& text() const { return contents->text; } }; // Role, VARIANT and runtime id helpers: accessibilitybackenduiaprovider.cpp @@ -334,10 +362,16 @@ SAFEARRAY* safeArrayFromInts( const std::vector& values ); size_t nextSourceCodePoint( const std::u16string& text, size_t index ); +size_t codePointToUTF16( const std::u16string& source, Int32 offset ); + size_t codePointToUTF16( const String& string, Int32 offset ); +Int32 utf16ToCodePoint( const std::u16string& source, size_t offset ); + Int32 utf16ToCodePoint( const String& string, size_t offset ); +std::u16string normalizedUTF16( const std::u16string& source ); + std::u16string normalizedUTF16( const String& string ); HRESULT textSnapshot( const std::shared_ptr& context, diff --git a/src/eepp/ui/accessibility/accessibilitybackenduiaprovider.cpp b/src/eepp/ui/accessibility/accessibilitybackenduiaprovider.cpp index eec2102bf..9b8b1d553 100644 --- a/src/eepp/ui/accessibility/accessibilitybackenduiaprovider.cpp +++ b/src/eepp/ui/accessibility/accessibilitybackenduiaprovider.cpp @@ -241,6 +241,10 @@ HRESULT STDMETHODCALLTYPE UIAutomationProvider::GetPropertyValue( PROPERTYID pro return setBstr( value, nodeInfo.name ); if ( propertyId == UIA_HelpTextPropertyId ) return setBstr( value, nodeInfo.description ); + if ( propertyId == UIA_AcceleratorKeyPropertyId ) + return nodeInfo.shortcut.empty() + ? S_OK + : setBstr( value, String::fromUtf8( nodeInfo.shortcut.text ) ); if ( propertyId == UIA_ControlTypePropertyId ) { value->vt = VT_I4; value->lVal = controlType( nodeInfo.role ); @@ -757,7 +761,7 @@ HRESULT STDMETHODCALLTYPE UIAutomationProvider::GetVisibleRanges( SAFEARRAY** ra if ( !array ) return E_OUTOFMEMORY; ITextRangeProvider* range = - new UIAutomationTextRange( mContext, mRef, 0, static_cast( snapshot.text.size() ) ); + new UIAutomationTextRange( mContext, mRef, 0, static_cast( snapshot.text().size() ) ); LONG index = 0; IUnknown* unknown = range; const HRESULT putResult = SafeArrayPutElement( array, &index, unknown ); @@ -796,8 +800,8 @@ HRESULT STDMETHODCALLTYPE UIAutomationProvider::RangeFromPoint( UiaPoint point, const double width = snapshot.bounds.getWidth(); double ratio = width > 0 ? ( point.x - left ) / width : 0; ratio = std::max( 0.0, std::min( ratio, 1.0 ) ); - size_t offset = static_cast( ratio * snapshot.text.size() ); - if ( offset < snapshot.text.size() && isLowSurrogate( snapshot.text[offset] ) ) + size_t offset = static_cast( ratio * snapshot.text().size() ); + if ( offset < snapshot.text().size() && isLowSurrogate( snapshot.text()[offset] ) ) --offset; *range = new UIAutomationTextRange( mContext, mRef, static_cast( offset ), static_cast( offset ) ); @@ -813,7 +817,7 @@ HRESULT STDMETHODCALLTYPE UIAutomationProvider::get_DocumentRange( ITextRangePro if ( FAILED( result ) ) return result; *range = - new UIAutomationTextRange( mContext, mRef, 0, static_cast( snapshot.text.size() ) ); + new UIAutomationTextRange( mContext, mRef, 0, static_cast( snapshot.text().size() ) ); return S_OK; } diff --git a/src/eepp/ui/accessibility/accessibilitybackenduiatext.cpp b/src/eepp/ui/accessibility/accessibilitybackenduiatext.cpp index a97833a09..bba101f2d 100644 --- a/src/eepp/ui/accessibility/accessibilitybackenduiatext.cpp +++ b/src/eepp/ui/accessibility/accessibilitybackenduiatext.cpp @@ -15,7 +15,10 @@ size_t nextSourceCodePoint( const std::u16string& text, size_t index ) { } size_t codePointToUTF16( const String& string, Int32 offset ) { - const auto source = string.toUtf16(); + return codePointToUTF16( string.toUtf16(), offset ); +} + +size_t codePointToUTF16( const std::u16string& source, Int32 offset ) { const size_t wanted = offset > 0 ? static_cast( offset ) : 0; size_t sourceIndex = 0; size_t displayIndex = 0; @@ -31,7 +34,10 @@ size_t codePointToUTF16( const String& string, Int32 offset ) { } Int32 utf16ToCodePoint( const String& string, size_t offset ) { - const auto source = string.toUtf16(); + return utf16ToCodePoint( string.toUtf16(), offset ); +} + +Int32 utf16ToCodePoint( const std::u16string& source, size_t offset ) { size_t sourceIndex = 0; size_t displayIndex = 0; Int32 codePoints = 0; @@ -51,7 +57,10 @@ Int32 utf16ToCodePoint( const String& string, size_t offset ) { } std::u16string normalizedUTF16( const String& string ) { - const auto source = string.toUtf16(); + return normalizedUTF16( string.toUtf16() ); +} + +std::u16string normalizedUTF16( const std::u16string& source ) { std::u16string result; result.reserve( source.size() ); for ( size_t index = 0; index < source.size(); ++index ) { @@ -62,6 +71,36 @@ std::u16string normalizedUTF16( const String& string ) { return result; } +std::shared_ptr +UIAutomationProviderContext::textContents( AccessibilityManager& manager, + AccessibilityNodeRef ref ) { + const auto revision = manager.getTextRevision( ref ); + if ( !revision.isValid() ) + return nullptr; + for ( const auto& entry : mTextCache ) { + if ( entry.ref == ref && entry.revision == revision ) + return entry.contents; + } + auto contents = std::make_shared(); + contents->source = manager.getTextRange( ref, 0, manager.getTextLength( ref ) ).toUtf16(); + contents->text = normalizedUTF16( contents->source ); + CachedTextContents entry{ ref, revision, std::move( contents ) }; + for ( auto& cached : mTextCache ) { + if ( cached.ref == ref ) { + cached = std::move( entry ); + return cached.contents; + } + } + if ( mTextCache.size() < TextCacheSize ) { + mTextCache.push_back( std::move( entry ) ); + return mTextCache.back().contents; + } + auto& replaced = mTextCache[mNextTextCacheEntry]; + mNextTextCacheEntry = ( mNextTextCacheEntry + 1 ) % TextCacheSize; + replaced = std::move( entry ); + return replaced.contents; +} + HRESULT textSnapshot( const std::shared_ptr& context, AccessibilityNodeRef ref, TextSnapshot& snapshot ) { if ( !context || !context->isAlive() ) @@ -70,16 +109,18 @@ HRESULT textSnapshot( const std::shared_ptr& contex snapshot, [=]( AccessibilityManager& manager, TextSnapshot& output ) -> HRESULT { if ( !manager.isValid( ref ) ) return UIA_E_ELEMENTNOTAVAILABLE; - const auto info = manager.getNodeInfo( ref ); + const auto info = manager.getNodeInfo( ref, false, true ); if ( !info.text.valid || hasState( info.states, AccessibilityState::Protected ) ) return UIA_E_INVALIDOPERATION; - output.source = info.value; - output.text = normalizedUTF16( info.value ); + // The contents are copied only when the text changed since the last query. + output.contents = context->textContents( manager, ref ); + if ( !output.contents ) + return UIA_E_INVALIDOPERATION; output.actions = info.actions; output.selectionStart = static_cast( codePointToUTF16( - info.value, std::min( info.text.selectionStart, info.text.selectionEnd ) ) ); + output.source(), std::min( info.text.selectionStart, info.text.selectionEnd ) ) ); output.selectionEnd = static_cast( codePointToUTF16( - info.value, std::max( info.text.selectionStart, info.text.selectionEnd ) ) ); + output.source(), std::max( info.text.selectionStart, info.text.selectionEnd ) ) ); output.bounds = info.bounds; output.boundsValid = info.boundsValid; return S_OK; @@ -234,7 +275,7 @@ HRESULT STDMETHODCALLTYPE UIAutomationTextRange::Clone( ITextRangeProvider** ran const HRESULT result = currentSnapshot( snapshot ); if ( FAILED( result ) ) return result; - auto endpoints = normalizedEndpoints( snapshot.text.size() ); + auto endpoints = normalizedEndpoints( snapshot.text().size() ); *range = new UIAutomationTextRange( mContext, mRef, endpoints.first, endpoints.second ); return S_OK; } @@ -274,8 +315,8 @@ HRESULT STDMETHODCALLTYPE UIAutomationTextRange::ExpandToEnclosingUnit( TextUnit const HRESULT result = currentSnapshot( snapshot ); if ( FAILED( result ) ) return result; - const auto current = normalizedEndpoints( snapshot.text.size() ); - const auto expanded = enclosingTextUnit( snapshot.text, current.first, unit ); + const auto current = normalizedEndpoints( snapshot.text().size() ); + const auto expanded = enclosingTextUnit( snapshot.text(), current.first, unit ); setEndpoints( static_cast( expanded.first ), static_cast( expanded.second ) ); return S_OK; } @@ -299,7 +340,7 @@ HRESULT STDMETHODCALLTYPE UIAutomationTextRange::FindText( BSTR text, BOOL backw const HRESULT result = currentSnapshot( snapshot ); if ( FAILED( result ) ) return result; - const auto current = normalizedEndpoints( snapshot.text.size() ); + const auto current = normalizedEndpoints( snapshot.text().size() ); const size_t rangeStart = static_cast( current.first ); const size_t rangeEnd = static_cast( current.second ); std::u16string needle( reinterpret_cast( text ), SysStringLen( text ) ); @@ -314,7 +355,7 @@ HRESULT STDMETHODCALLTYPE UIAutomationTextRange::FindText( BSTR text, BOOL backw if ( backward ) { for ( size_t candidate = rangeEnd; candidate-- > rangeStart; ) { if ( candidate + needle.size() <= rangeEnd && - std::equal( needle.begin(), needle.end(), snapshot.text.begin() + candidate, + std::equal( needle.begin(), needle.end(), snapshot.text().begin() + candidate, equal ) ) { found = candidate; break; @@ -322,7 +363,7 @@ HRESULT STDMETHODCALLTYPE UIAutomationTextRange::FindText( BSTR text, BOOL backw } } else { for ( size_t candidate = rangeStart; candidate + needle.size() <= rangeEnd; ++candidate ) { - if ( std::equal( needle.begin(), needle.end(), snapshot.text.begin() + candidate, + if ( std::equal( needle.begin(), needle.end(), snapshot.text().begin() + candidate, equal ) ) { found = candidate; break; @@ -353,7 +394,7 @@ HRESULT STDMETHODCALLTYPE UIAutomationTextRange::GetBoundingRectangles( SAFEARRA const HRESULT result = currentSnapshot( snapshot ); if ( FAILED( result ) ) return result; - const auto current = normalizedEndpoints( snapshot.text.size() ); + const auto current = normalizedEndpoints( snapshot.text().size() ); const bool hasRectangle = current.first != current.second && snapshot.boundsValid && IsWindowVisible( mContext->window() ) && !IsIconic( mContext->window() ); @@ -386,13 +427,13 @@ HRESULT STDMETHODCALLTYPE UIAutomationTextRange::GetText( int maxLength, BSTR* t const HRESULT result = currentSnapshot( snapshot ); if ( FAILED( result ) ) return result; - const auto current = normalizedEndpoints( snapshot.text.size() ); + const auto current = normalizedEndpoints( snapshot.text().size() ); size_t length = current.second - current.first; if ( maxLength >= 0 ) length = std::min( length, static_cast( maxLength ) ); - *text = - SysAllocStringLen( reinterpret_cast( snapshot.text.data() + current.first ), - static_cast( length ) ); + *text = SysAllocStringLen( + reinterpret_cast( snapshot.text().data() + current.first ), + static_cast( length ) ); return *text ? S_OK : E_OUTOFMEMORY; } @@ -404,13 +445,13 @@ HRESULT STDMETHODCALLTYPE UIAutomationTextRange::Move( TextUnit unit, int count, const HRESULT result = currentSnapshot( snapshot ); if ( FAILED( result ) ) return result; - const auto current = normalizedEndpoints( snapshot.text.size() ); + const auto current = normalizedEndpoints( snapshot.text().size() ); int actual{}; - const size_t start = moveTextEndpoint( snapshot.text, current.first, unit, count, actual ); + const size_t start = moveTextEndpoint( snapshot.text(), current.first, unit, count, actual ); if ( current.first == current.second ) setEndpoints( static_cast( start ), static_cast( start ) ); else { - const auto expanded = enclosingTextUnit( snapshot.text, start, unit ); + const auto expanded = enclosingTextUnit( snapshot.text(), start, unit ); setEndpoints( static_cast( expanded.first ), static_cast( expanded.second ) ); } *moved = actual; @@ -426,11 +467,12 @@ HRESULT STDMETHODCALLTYPE UIAutomationTextRange::MoveEndpointByUnit( const HRESULT result = currentSnapshot( snapshot ); if ( FAILED( result ) ) return result; - auto current = normalizedEndpoints( snapshot.text.size() ); + auto current = normalizedEndpoints( snapshot.text().size() ); int actual{}; const size_t target = moveTextEndpoint( - snapshot.text, endpoint == TextPatternRangeEndpoint_Start ? current.first : current.second, - unit, count, actual ); + snapshot.text(), + endpoint == TextPatternRangeEndpoint_Start ? current.first : current.second, unit, count, + actual ); if ( endpoint == TextPatternRangeEndpoint_Start ) { current.first = static_cast( target ); if ( current.first > current.second ) @@ -477,9 +519,9 @@ HRESULT STDMETHODCALLTYPE UIAutomationTextRange::Select() { return result; if ( !hasAction( snapshot.actions, AccessibilityAction::SetTextSelection ) ) return UIA_E_INVALIDOPERATION; - const auto current = normalizedEndpoints( snapshot.text.size() ); - const Int32 start = utf16ToCodePoint( snapshot.source, current.first ); - const Int32 end = utf16ToCodePoint( snapshot.source, current.second ); + const auto current = normalizedEndpoints( snapshot.text().size() ); + const Int32 start = utf16ToCodePoint( snapshot.source(), current.first ); + const Int32 end = utf16ToCodePoint( snapshot.source(), current.second ); bool ignored{}; return invoke( ignored, [this, start, end]( AccessibilityManager& manager, bool& ) -> HRESULT { if ( !manager.isValid( mRef ) ) diff --git a/src/eepp/ui/accessibility/accessibilitymanager.cpp b/src/eepp/ui/accessibility/accessibilitymanager.cpp index 925fed17b..b7cfbf9cc 100644 --- a/src/eepp/ui/accessibility/accessibilitymanager.cpp +++ b/src/eepp/ui/accessibility/accessibilitymanager.cpp @@ -5,15 +5,26 @@ #include #include #include +#include #include #include +#include #include namespace EE { namespace UI { namespace { +/** Depth-first walk; `visitor` returns false to skip a node's children. */ +template +void visitNodes( NodeType* node, const Visitor& visitor ) { + if ( !visitor( node ) ) + return; + for ( auto* child = node->getFirstChild(); child; child = child->getNextNode() ) + visitNodes( static_cast( child ), visitor ); +} + bool isModelViewImplementationChild( const Node* node ) { return AccessibilityWidgetResolver::getOwningModelView( node ) != nullptr; } @@ -191,8 +202,14 @@ AccessibilityNodeInfo AccessibilityManager::getNodeInfo( AccessibilityNodeRef re AccessibilityNodeInfo info; if ( auto widget = resolve( ref ) ) { info.role = widget->getAccessibilityRole(); + // A client now holds a name or description that depends on another widget. + if ( !widget->getAccessibilityLabelledBy().empty() ) + mRelationTargets.insert( String::hash( widget->getAccessibilityLabelledBy() ) ); + if ( !widget->getAccessibilityDescribedBy().empty() ) + mRelationTargets.insert( String::hash( widget->getAccessibilityDescribedBy() ) ); info.name = widget->getAccessibilityName(); info.description = widget->getAccessibilityDescription(); + info.shortcut = AccessibilityWidgetResolver::getShortcut( widget ); if ( includeValue ) info.value = widget->getAccessibilityValue(); info.range = widget->getAccessibilityRange(); @@ -240,6 +257,71 @@ AccessibilityTextInfo AccessibilityManager::getTextInfo( AccessibilityNodeRef re return {}; } +Int32 AccessibilityManager::getTextLength( AccessibilityNodeRef ref ) const { + auto* widget = resolve( ref ); + return widget ? AccessibilityWidgetResolver::getTextLength( widget ) : 0; +} + +String AccessibilityManager::getTextRange( AccessibilityNodeRef ref, Int32 start, + Int32 end ) const { + auto* widget = resolve( ref ); + return widget ? AccessibilityWidgetResolver::getTextRange( widget, start, end ) : String(); +} + +bool AccessibilityManager::getTextLineBounds( AccessibilityNodeRef ref, Int32 offset, Int32& start, + Int32& end ) const { + auto* widget = resolve( ref ); + return widget && AccessibilityWidgetResolver::getTextLineBounds( widget, offset, start, end ); +} + +AccessibilityTextRevision AccessibilityManager::getTextRevision( AccessibilityNodeRef ref ) const { + auto* widget = resolve( ref ); + return widget ? AccessibilityWidgetResolver::getTextRevision( widget ) + : AccessibilityTextRevision{}; +} + +void AccessibilityManager::onTextChanged( UIWidget* widget, + const Doc::DocumentContentChange* change ) { + if ( !mBackend || !mScene || !mScene->hasActiveAccessibilityClients() || !widget || + !widget->isAccessibilityElement() || + AccessibilityWidgetResolver::getEventTarget( widget, AccessibilityEvent::ValueChanged ) != + widget ) + return; + const auto* document = AccessibilityWidgetResolver::getTextDocument( widget ); + if ( !document ) + return; + const auto ref = getNodeRef( widget ); + if ( ref == mSuppressedTextRef ) + return; + auto counter = std::find_if( mTextChangeCounts.begin(), mTextChangeCounts.end(), + [&ref]( const auto& entry ) { return entry.first == ref; } ); + if ( counter == mTextChangeCounts.end() ) { + mTextChangeCounts.emplace_back( ref, 0 ); + counter = mTextChangeCounts.end() - 1; + } + // Once this frame's edits collapsed into a whole-text change, that change covers the rest. + if ( counter->second > MaxExactTextChangesPerFrame ) + return; + const auto previousCount = counter->second; + AccessibilityTextChange textChange; + if ( change && ++counter->second <= MaxExactTextChangesPerFrame ) { + textChange.offset = AccessibilityWidgetResolver::getTextOffset( + *document, change->range.normalized().start() ); + textChange.removed = document->getNotifiedRemovedText(); + textChange.inserted = change->text; + } else { + counter->second = MaxExactTextChangesPerFrame + 1; + } + // An ignored change (no client listening, for example) must not consume the budget, or + // later edits would be dropped while nothing covers them. + if ( !mBackend->onTextChanged( ref, textChange ) ) + counter->second = previousCount; +} + +void AccessibilityManager::setSuppressedTextChanges( AccessibilityNodeRef ref ) { + mSuppressedTextRef = ref; +} + Int32 AccessibilityManager::getIndexInParent( AccessibilityNodeRef ref ) { if ( auto* source = resolveSource( ref ) ) { const auto index = source->getIndexInParent( ref.id ); @@ -318,6 +400,8 @@ AccessibilityManager::getChildren( AccessibilityNodeRef ref ) { cache.children.emplace_back( source->getRootChild( i ) ); return cache.children; } + if ( AccessibilityWidgetResolver::isLeafRole( widget->getAccessibilityRole() ) ) + return cache.children; cache.children.reserve( countSemanticChildren( widget ) ); collectSemanticChildren( widget, cache.children, *this ); return cache.children; @@ -374,7 +458,19 @@ bool AccessibilityManager::hasActiveNativeClients() const { void AccessibilityManager::onNativeClientObserved() { if ( mScene ) - mScene->mAccessibilityState |= UISceneNode::AccessibilityClientActive; + mScene->setAccessibilityClientActive( true ); +} + +void AccessibilityManager::onClientsDisconnected() { + // Queried rows hold persistent model registrations, which the model updates on every row + // insert, delete and move. Without a client nothing reads them. + for ( auto& source : mSources ) { + if ( mBackend ) + mBackend->onSourceInvalidated( source.first ); + source.second->reset(); + } + mRelationTargets.clear(); + invalidateChildren(); } void AccessibilityManager::update() { @@ -385,10 +481,27 @@ void AccessibilityManager::update() { mBackend = disabled ? createNullAccessibilityBackend() : createAccessibilityBackend( *this ); } + mTextChangeCounts.clear(); if ( mBackend ) { mBackend->update(); mPendingEvents.clear(); + // Announcements follow the frame's events, so focus and state reach the client first. + for ( const auto& announcement : mPendingAnnouncements ) { + // A live region may have been hidden or removed since its change was queued. + if ( announcement.source.isValid() ) { + const auto* source = resolve( announcement.source ); + if ( !source || !source->hasVisibility() || + AccessibilityWidgetResolver::isHiddenFromAccessibility( source ) ) + continue; + } + mBackend->announce( announcement.message, announcement.priority ); + } } + mPendingAnnouncements.clear(); +} + +void AccessibilityManager::setBackend( std::unique_ptr backend ) { + mBackend = std::move( backend ); } UISceneNode* AccessibilityManager::getSceneNode() const { @@ -424,6 +537,14 @@ void AccessibilityManager::notify( AccessibilityNodeRef ref, AccessibilityEvent } } } + // Without a client nothing can be delivered: queueing would only grow the vector until the + // next update() discards it. Identity invalidation (Destroyed without a related node) still + // reaches the backend, which detaches native wrappers and per-node state on it. + if ( !hasActiveClients() ) { + if ( mBackend && event == AccessibilityEvent::Destroyed && !related.isValid() ) + mBackend->onEvent( { ref, related, index, event } ); + return; + } // Only adjacent query-again hints may be coalesced. They carry no historical state; source // invalidation above must still run for every mutation. State transitions and structural // sequences such as add/remove/add must always retain their original dispatch order. @@ -446,10 +567,98 @@ const std::vector& AccessibilityManager::getPendingEv void AccessibilityManager::clearPendingEvents() { mPendingEvents.clear(); + mPendingAnnouncements.clear(); +} + +void AccessibilityManager::announce( const String& message, AccessibilityLive priority ) { + if ( message.empty() || priority == AccessibilityLive::Off || !mScene || + !mScene->hasActiveAccessibilityClients() ) + return; + mPendingAnnouncements.push_back( { message, priority, {} } ); +} + +void AccessibilityManager::onLiveRegionChanged( UIWidget* widget, AccessibilityLive priority ) { + if ( !widget || priority == AccessibilityLive::Off || !widget->hasVisibility() ) + return; + String message = widget->getAccessibilityValue(); + if ( message.empty() ) + message = widget->getAccessibilityName(); + const auto ref = getNodeRef( widget ); + auto pending = std::find_if( mPendingAnnouncements.begin(), mPendingAnnouncements.end(), + [&ref]( const auto& item ) { return item.source == ref; } ); + if ( pending != mPendingAnnouncements.end() ) { + if ( message.empty() ) { + mPendingAnnouncements.erase( pending ); + } else { + pending->message = std::move( message ); + pending->priority = priority; + } + } else if ( !message.empty() ) { + mPendingAnnouncements.push_back( { std::move( message ), priority, ref } ); + } +} + +const std::vector& +AccessibilityManager::getPendingAnnouncements() const { + return mPendingAnnouncements; +} + +bool AccessibilityManager::isRelationTarget( String::HashType idHash ) const { + return !mRelationTargets.empty() && mScene && mScene->hasActiveAccessibilityClients() && + mRelationTargets.find( idHash ) != mRelationTargets.end(); +} + +void AccessibilityManager::onRelationTargetChanged( const std::string& id, + const Scene::Node* excluded ) { + if ( id.empty() || !isRelationTarget( String::hash( id ) ) ) + return; + auto notifyDependent = [this]( UIWidget* widget, AccessibilityEvent event ) { + if ( widget->isAccessibilityHidden() ) + return; + auto* target = AccessibilityWidgetResolver::getEventTarget( widget, event ); + if ( target && target->isAccessibilityElement() ) + notify( getNodeRef( target ), event ); + }; + // Relations are rare and resolved by id from the scene root, so scan the scene. + if ( mScene->getRoot() ) + visitNodes( static_cast( mScene->getRoot() ), [&]( Scene::Node* node ) { + if ( node == excluded ) + return false; + if ( node->isWidget() ) { + auto* widget = node->asType(); + if ( widget->getAccessibilityLabelledBy() == id ) + notifyDependent( widget, AccessibilityEvent::NameChanged ); + if ( widget->getAccessibilityDescribedBy() == id ) + notifyDependent( widget, AccessibilityEvent::DescriptionChanged ); + } + return true; + } ); +} + +void AccessibilityManager::onRelationSubtreeChanged( const Scene::Node* subtree, bool removed ) { + if ( !subtree || mRelationTargets.empty() || !mScene || + !mScene->hasActiveAccessibilityClients() ) + return; + std::vector ids; + visitNodes( subtree, [&]( const Scene::Node* node ) { + if ( !node->getId().empty() && + mRelationTargets.find( node->getIdHash() ) != mRelationTargets.end() ) + ids.push_back( node->getId() ); + return true; + } ); + for ( const auto& id : ids ) + onRelationTargetChanged( id, removed ? subtree : nullptr ); } void AccessibilityManager::onWidgetParentChange( UIWidget* widget ) { - if ( !widget || !widget->isAccessibilityElement() || widget->isAccessibilityHidden() ) + if ( !widget ) + return; + // An arriving label renames the controls it labels, whatever its own exposure. + onRelationSubtreeChanged( widget, false ); + if ( !widget->isAccessibilityElement() || widget->isAccessibilityHidden() ) + return; + // Children of a leaf control (a button's text view) are not part of the tree. + if ( AccessibilityWidgetResolver::getLeafOwner( widget ) ) return; // Model sources, not recycled cell widgets, own the accessible children of a view. if ( isModelViewImplementationChild( widget ) ) @@ -471,6 +680,7 @@ void AccessibilityManager::onWidgetParentChange( UIWidget* widget ) { } void AccessibilityManager::onWidgetRemovedFromParent( UIWidget* widget ) { + onRelationSubtreeChanged( widget, true ); if ( isModelViewImplementationChild( widget ) ) return; auto found = mWidgetIds.find( widget ); @@ -504,21 +714,26 @@ bool AccessibilityManager::onWidgetAccessibilitySourceDelete( UIWidget* widget ) void AccessibilityManager::onWidgetDelete( UIWidget* widget ) { if ( mWidgetIds.empty() && mWidgetSources.empty() ) return; + // Native roots can exist without any client or queried descendants. Keep that dormant case + // constant-time, before any ancestor walk: the scene root is never inside another widget's + // subtree, so a deleted non-root widget holds nothing to invalidate. It may still be a + // relation target of the root a client has read, which must be renamed. + if ( mWidgetSources.empty() && mWidgetIds.size() == 1 && widget != mScene->getRoot() && + mWidgetIds.find( mScene->getRoot() ) != mWidgetIds.end() && + ( mRelationTargets.empty() || !hasActiveClients() ) ) + return; for ( auto* parent = widget->getParent(); parent; parent = parent->getParent() ) { if ( parent->isDestroying() ) return; } + // The subtree is still intact here; its children are destroyed after this widget. + onRelationSubtreeChanged( widget, true ); if ( isModelViewImplementationChild( widget ) ) { removeSubtreeIdentities( widget ); return; } auto found = mWidgetIds.find( widget ); if ( found == mWidgetIds.end() ) { - // Native roots can exist without any client or queried descendants. Keep that dormant - // case constant-time, including destruction of large unqueried widget hierarchies. - if ( mWidgetSources.empty() && mWidgetIds.size() == 1 && - mWidgetIds.find( mScene->getRoot() ) != mWidgetIds.end() ) - return; // Only a subtree holding identities a client may have cached changes the tree it knows. if ( !removeSubtreeIdentities( widget ) ) return; @@ -531,14 +746,20 @@ void AccessibilityManager::onWidgetDelete( UIWidget* widget ) { return; } AccessibilityNodeRef ref{ WidgetSource, found->second }; - auto parent = getParent( ref ); - if ( parent.isValid() ) { + if ( hasActiveClients() ) { + auto parent = getParent( ref ); + if ( parent.isValid() ) { + invalidateChildren(); + Int32 index = -1; + size_t currentIndex = 0; + if ( auto parentWidget = resolve( parent ) ) + semanticIndexOf( parentWidget, widget, currentIndex, index ); + notify( parent, AccessibilityEvent::Destroyed, ref, index ); + } + } else { + // No client to tell: do not register the parent or compute its index just to announce + // this removal. Cached children still drop the widget. invalidateChildren(); - Int32 index = -1; - size_t currentIndex = 0; - if ( auto parentWidget = resolve( parent ) ) - semanticIndexOf( parentWidget, widget, currentIndex, index ); - notify( parent, AccessibilityEvent::Destroyed, ref, index ); } // getParent() and native destruction notifications can register more widgets, invalidating // dense-map iterators. Keep the copied identity and erase by key instead. @@ -549,7 +770,7 @@ void AccessibilityManager::onSubtreeRemoved( Scene::Node* node ) { if ( !node || !removeSubtreeIdentities( node ) ) return; invalidateChildren(); - if ( node->isWidget() ) { + if ( node->isWidget() && hasActiveClients() ) { auto parent = getWidgetParent( node->asType() ); if ( parent.isValid() ) notify( parent, AccessibilityEvent::ChildrenChanged ); @@ -578,6 +799,10 @@ bool AccessibilityManager::removeSubtreeIdentities( Scene::Node* node ) { return true; } +bool AccessibilityManager::hasActiveClients() const { + return mScene && mScene->hasActiveAccessibilityClients(); +} + void AccessibilityManager::invalidateChildren() { for ( auto& cache : mChildrenCache ) { cache.parent = {}; diff --git a/src/eepp/ui/accessibility/accessibilitymodelviewsource.cpp b/src/eepp/ui/accessibility/accessibilitymodelviewsource.cpp index 372f320ed..d9a5e23c7 100644 --- a/src/eepp/ui/accessibility/accessibilitymodelviewsource.cpp +++ b/src/eepp/ui/accessibility/accessibilitymodelviewsource.cpp @@ -250,7 +250,29 @@ class AccessibilityModelViewSource final : public AccessibilitySource { } private: + /** A queried row or cell. It owns the persistent registration made for it, so dropping + * the node lets the model drop the handle instead of updating it on every row operation. */ struct NodeInfo { + NodeInfo( const ModelIndex& modelIndex, bool cell ) : index( modelIndex ), cell( cell ) {} + + NodeInfo( NodeInfo&& ) = default; + + NodeInfo& operator=( NodeInfo&& other ) { + if ( this != &other ) { + index.release(); + index = std::move( other.index ); + cell = other.cell; + } + return *this; + } + + NodeInfo( const NodeInfo& ) = delete; + + NodeInfo& operator=( const NodeInfo& ) = delete; + + // A moved-from index holds no handle, so releasing it does nothing. + ~NodeInfo() { index.release(); } + Models::PersistentModelIndex index; bool cell{ false }; }; @@ -284,7 +306,7 @@ class AccessibilityModelViewSource final : public AccessibilitySource { return { mSourceId, found->second }; Uint64 id = mNextId++; ids.emplace( index, id ); - mNodes.emplace( id, NodeInfo{ Models::PersistentModelIndex( index ), cell } ); + mNodes.emplace( id, NodeInfo( index, cell ) ); return { mSourceId, id }; } diff --git a/src/eepp/ui/accessibility/accessibilitywidgetresolver.cpp b/src/eepp/ui/accessibility/accessibilitywidgetresolver.cpp index 1142afad6..4c874c6d8 100644 --- a/src/eepp/ui/accessibility/accessibilitywidgetresolver.cpp +++ b/src/eepp/ui/accessibility/accessibilitywidgetresolver.cpp @@ -3,6 +3,7 @@ #include #include #include +#include #include #include #include @@ -14,6 +15,7 @@ #include #include #include +#include #include #include #include @@ -26,6 +28,7 @@ #include #include #include +#include #include #include @@ -77,18 +80,90 @@ AccessibilityActions baseActions( const UIWidget* widget ) { return widget->isTabFocusable() ? accessibilityActionMask( AccessibilityAction::Focus ) : 0; } +/** The nearest scroll view whose scrolled content contains the widget. */ +UIScrollView* scrollViewFor( const UIWidget* widget ) { + for ( auto* parent = widget->getParent(); parent; parent = parent->getParent() ) { + if ( parent->isType( UI_TYPE_SCROLLVIEW ) ) { + auto* view = parent->asType(); + if ( view->getScrollView() && view->getScrollView()->isParentOf( widget ) ) + return view; + } + } + return nullptr; +} + +/** Start offsets of every line of one document version, plus the total length. Offset and + * position conversions become an index lookup or a binary search instead of a walk over every + * line (each line's size takes the document mutex). Rebuilt once per text revision. */ +class LineIndex { + public: + const std::vector& starts( const Doc::TextDocument& document ) { + if ( document.getUUID() != mDocument || document.getModificationId() != mModification || + mStarts.empty() ) { + mDocument = document.getUUID(); + mModification = document.getModificationId(); + const size_t lines = document.linesCount(); + mStarts.resize( lines + 1 ); + mStarts[0] = 0; + for ( size_t line = 0; line < lines; ++line ) + mStarts[line + 1] = + mStarts[line] + static_cast( document.line( line ).size() ); + } + return mStarts; + } + + bool matches( const Doc::TextDocument& document ) const { + return document.getUUID() == mDocument && document.getModificationId() == mModification && + !mStarts.empty(); + } + + private: + System::UUID mDocument{ 0, 0 }; + Uint64 mModification{ 0 }; + std::vector mStarts; +}; + +/** UI thread only. Two entries let a client alternate between an editor and a field. */ +const std::vector& lineStarts( const Doc::TextDocument& document ) { + static LineIndex indexes[2]; + static size_t next = 0; + for ( auto& index : indexes ) { + if ( index.matches( document ) ) + return index.starts( document ); + } + auto& index = indexes[next]; + next = ( next + 1 ) % 2; + return index.starts( document ); +} + Int32 documentOffset( const Doc::TextDocument& document, const Doc::TextPosition& position ) { if ( !position.isValid() ) return 0; - Int64 offset = 0; - for ( Int64 line = 0; - line < position.line() && line < static_cast( document.linesCount() ); ++line ) - offset += static_cast( document.line( static_cast( line ) ).size() ); - offset += position.column(); - return static_cast( std::max( 0, offset ) ); + const auto& starts = lineStarts( document ); + const size_t line = + std::min( static_cast( std::max( 0, position.line() ) ), starts.size() - 1 ); + return static_cast( std::max( 0, starts[line] + position.column() ) ); +} + +Doc::TextPosition documentPosition( const Doc::TextDocument& document, Int32 offset ) { + const auto& starts = lineStarts( document ); + if ( starts.size() < 2 ) + return { 0, 0 }; + const Int64 target = std::max( 0, offset ); + // The last line whose start is at or before the offset; past the end clamps to the last line. + const size_t lines = starts.size() - 1; + size_t line = static_cast( std::upper_bound( starts.begin(), starts.end(), target ) - + starts.begin() ) - + 1; + line = std::min( line, lines - 1 ); + const Int64 length = starts[line + 1] - starts[line]; + return { static_cast( line ), std::min( target - starts[line], length ) }; } AccessibilityTextInfo getDocumentText( const Doc::TextDocument& document ) { + // A document loading on a worker thread is still being filled in. + if ( document.isLoading() ) + return { 0, 0, 0, true }; auto selection = document.getSelection( true ); return { documentOffset( document, document.getSelection().end() ), documentOffset( document, selection.start() ), @@ -101,19 +176,6 @@ bool parseSelection( const String& value, Int32& start, Int32& end ) { String::fromString( end, parts[1].toUtf8() ); } -Doc::TextPosition documentPosition( const Doc::TextDocument& document, Int32 offset ) { - Int64 remaining = std::max( 0, offset ); - for ( size_t line = 0; line < document.linesCount(); ++line ) { - const Int64 length = static_cast( document.line( line ).size() ); - if ( line + 1 == document.linesCount() ) - return { static_cast( line ), std::min( remaining, length ) }; - if ( remaining < length ) - return { static_cast( line ), remaining }; - remaining -= length; - } - return { 0, 0 }; -} - } // namespace AccessibilityRole AccessibilityWidgetResolver::getRole( const UIWidget* widget ) { @@ -146,7 +208,7 @@ AccessibilityRole AccessibilityWidgetResolver::getRole( const UIWidget* widget ) role = AccessibilityRole::TabList; else if ( widget->isType( UI_TYPE_COMBOBOX ) ) role = AccessibilityRole::ComboBox; - else if ( widget->isType( UI_TYPE_TEXTEDIT ) ) + else if ( widget->isType( UI_TYPE_CODEEDITOR ) ) role = AccessibilityRole::TextBox; else if ( widget->isType( UI_TYPE_TEXTINPUT ) ) role = AccessibilityRole::TextBox; @@ -165,7 +227,45 @@ AccessibilityRole AccessibilityWidgetResolver::getRole( const UIWidget* widget ) return widget->resolveAccessibilityRole( role ); } -String AccessibilityWidgetResolver::getName( const UIWidget* widget ) { +bool AccessibilityWidgetResolver::isLeafRole( AccessibilityRole role ) { + switch ( role ) { + case AccessibilityRole::Button: + case AccessibilityRole::CheckBox: + case AccessibilityRole::RadioButton: + case AccessibilityRole::Label: + case AccessibilityRole::Text: + case AccessibilityRole::Image: + case AccessibilityRole::Slider: + case AccessibilityRole::SpinButton: + case AccessibilityRole::ProgressBar: + case AccessibilityRole::Tab: + case AccessibilityRole::MenuItem: + case AccessibilityRole::CheckMenuItem: + case AccessibilityRole::RadioMenuItem: + return true; + default: + return false; + } +} + +UIWidget* AccessibilityWidgetResolver::getLeafOwner( const Node* node ) { + UIWidget* owner = nullptr; + for ( auto* parent = node->getParent(); parent; parent = parent->getParent() ) { + if ( parent->isWidget() && isLeafRole( getRole( parent->asType() ) ) ) + owner = parent->asType(); + } + return owner; +} + +bool AccessibilityWidgetResolver::isHiddenFromAccessibility( const UIWidget* widget ) { + for ( const Node* node = widget; node; node = node->getParent() ) { + if ( node->isWidget() && static_cast( node )->isAccessibilityHidden() ) + return true; + } + return false; +} + +String AccessibilityWidgetResolver::getOwnName( const UIWidget* widget ) { String name; if ( isRoot( widget ) && widget->getUISceneNode()->getWindow() ) name = String::fromUtf8( widget->getUISceneNode()->getWindow()->getTitle() ); @@ -174,18 +274,147 @@ String AccessibilityWidgetResolver::getName( const UIWidget* widget ) { else if ( widget->isType( UI_TYPE_TEXTVIEW ) && !widget->isType( UI_TYPE_TEXTINPUT ) && !widget->isType( UI_TYPE_TEXTEDIT ) ) name = static_cast( widget )->getText(); - return widget->resolveAccessibilityName( name ); + else if ( widget->isType( UI_TYPE_CODEEDITOR ) && + static_cast( widget )->getDocument().hasFilepath() ) + name = String::fromUtf8( + static_cast( widget )->getDocument().getFilename() ); + name = widget->resolveAccessibilityName( name ); + // Icon-only buttons usually carry their meaning in the tooltip. + return name.empty() ? widget->getTooltipText() : name; +} + +namespace { + +/** Resolves an aria-labelledby / aria-describedby id from the widget's scene root. */ +const UIWidget* findRelated( const UIWidget* widget, const std::string& id ) { + if ( id.empty() || !widget->getUISceneNode() || !widget->getUISceneNode()->getRoot() ) + return nullptr; + Node* node = widget->getUISceneNode()->getRoot()->find( id ); + return node && node != widget && node->isWidget() ? node->asType() : nullptr; +} + +} // namespace + +const char* AccessibilityWidgetResolver::getRoleName( AccessibilityRole role ) { + switch ( role ) { + case AccessibilityRole::None: + return "None"; + case AccessibilityRole::Application: + return "Application"; + case AccessibilityRole::Window: + return "Window"; + case AccessibilityRole::Dialog: + return "Dialog"; + case AccessibilityRole::Group: + return "Group"; + case AccessibilityRole::Button: + return "Button"; + case AccessibilityRole::CheckBox: + return "CheckBox"; + case AccessibilityRole::RadioButton: + return "RadioButton"; + case AccessibilityRole::Label: + return "Label"; + case AccessibilityRole::Text: + return "Text"; + case AccessibilityRole::TextBox: + return "TextBox"; + case AccessibilityRole::Image: + return "Image"; + case AccessibilityRole::ComboBox: + return "ComboBox"; + case AccessibilityRole::Slider: + return "Slider"; + case AccessibilityRole::SpinButton: + return "SpinButton"; + case AccessibilityRole::ProgressBar: + return "ProgressBar"; + case AccessibilityRole::TabList: + return "TabList"; + case AccessibilityRole::Tab: + return "Tab"; + case AccessibilityRole::TabPanel: + return "TabPanel"; + case AccessibilityRole::MenuBar: + return "MenuBar"; + case AccessibilityRole::Menu: + return "Menu"; + case AccessibilityRole::MenuItem: + return "MenuItem"; + case AccessibilityRole::CheckMenuItem: + return "CheckMenuItem"; + case AccessibilityRole::RadioMenuItem: + return "RadioMenuItem"; + case AccessibilityRole::List: + return "List"; + case AccessibilityRole::ListItem: + return "ListItem"; + case AccessibilityRole::Table: + return "Table"; + case AccessibilityRole::Row: + return "Row"; + case AccessibilityRole::Cell: + return "Cell"; + case AccessibilityRole::Tree: + return "Tree"; + case AccessibilityRole::TreeItem: + return "TreeItem"; + } + return "Unknown"; +} + +String AccessibilityWidgetResolver::getName( const UIWidget* widget ) { + if ( const auto* label = findRelated( widget, widget->getAccessibilityLabelledBy() ) ) { + String name = getOwnName( label ); + if ( !name.empty() ) + return name; + } + return getOwnName( widget ); } String AccessibilityWidgetResolver::getDescription( const UIWidget* widget ) { - return widget->resolveAccessibilityDescription(); + if ( const auto* description = findRelated( widget, widget->getAccessibilityDescribedBy() ) ) { + String text = getOwnName( description ); + if ( !text.empty() ) + return text; + } + String description = widget->resolveAccessibilityDescription(); + if ( !description.empty() ) + return description; + // A tooltip that did not become the name still explains the control. + const String tooltip = widget->getTooltipText(); + return tooltip.empty() || tooltip == getOwnName( widget ) ? String() : tooltip; +} + +AccessibilityShortcut AccessibilityWidgetResolver::getShortcut( const UIWidget* widget ) { + AccessibilityShortcut result; + if ( !widget->isType( UI_TYPE_MENUITEM ) ) + return result; + const auto* item = static_cast( widget ); + const auto* view = item->getShortcutView(); + if ( !view || view->getText().empty() ) + return result; + result.text = view->getText().toUtf8(); + const auto& shortcut = item->getShortcut(); + if ( shortcut.key == KEY_UNKNOWN || !widget->getInput() ) + return result; + result.key = widget->getInput()->getKeyName( shortcut.key ); + if ( shortcut.mod & KEYMOD_CTRL ) + result.modifiers |= AccessibilityShortcut::Control; + if ( shortcut.mod & KEYMOD_SHIFT ) + result.modifiers |= AccessibilityShortcut::Shift; + if ( shortcut.mod & KEYMOD_ALT ) + result.modifiers |= AccessibilityShortcut::Alt; + if ( shortcut.mod & KEYMOD_META ) + result.modifiers |= AccessibilityShortcut::Meta; + return result; } String AccessibilityWidgetResolver::getValue( const UIWidget* widget ) { if ( widget->isType( UI_TYPE_COMBOBOX ) ) return static_cast( widget )->getDropDownList()->getText(); - if ( widget->isType( UI_TYPE_TEXTEDIT ) ) - return static_cast( widget )->getText(); + if ( widget->isType( UI_TYPE_CODEEDITOR ) ) + return static_cast( widget )->getDocument().getText(); if ( widget->isType( UI_TYPE_TEXTINPUT ) ) { auto input = static_cast( widget ); return input->getMode() == UITextInput::TextInputMode::Password ? String() @@ -221,11 +450,68 @@ AccessibilityTextInfo AccessibilityWidgetResolver::getText( const UIWidget* widg return getText( widget, true ); } +const Doc::TextDocument* AccessibilityWidgetResolver::getTextDocument( const UIWidget* widget ) { + const Doc::TextDocument* document = nullptr; + if ( widget->isType( UI_TYPE_CODEEDITOR ) ) { + document = &static_cast( widget )->getDocument(); + } else if ( widget->isType( UI_TYPE_TEXTINPUT ) ) { + const auto* input = static_cast( widget ); + if ( input->getMode() != UITextInput::TextInputMode::Password ) + document = &input->getDocument(); + } + // A document loading on a worker thread is still being filled in. + return document && !document->isLoading() ? document : nullptr; +} + +Int32 AccessibilityWidgetResolver::getTextLength( const UIWidget* widget ) { + const auto* document = getTextDocument( widget ); + if ( !document ) + return 0; + // The exposed text stops before the last line's newline, like TextDocument::getText(). + return static_cast( std::max( 0, lineStarts( *document ).back() - 1 ) ); +} + +String AccessibilityWidgetResolver::getTextRange( const UIWidget* widget, Int32 start, Int32 end ) { + const auto* document = getTextDocument( widget ); + if ( !document || end <= start ) + return {}; + return document->getText( + { documentPosition( *document, start ), documentPosition( *document, end ) } ); +} + +bool AccessibilityWidgetResolver::getTextLineBounds( const UIWidget* widget, Int32 offset, + Int32& start, Int32& end ) { + const auto* document = getTextDocument( widget ); + if ( !document || document->linesCount() == 0 ) + return false; + const auto position = documentPosition( *document, offset ); + start = static_cast( std::max( 0, offset - position.column() ) ); + const Int64 lineLength = static_cast( document->line( position.line() ).size() ); + const bool lastLine = position.line() + 1 == static_cast( document->linesCount() ); + // The last line's newline is not part of the exposed text. + end = + start + static_cast( lastLine ? std::max( 0, lineLength - 1 ) : lineLength ); + return true; +} + +AccessibilityTextRevision AccessibilityWidgetResolver::getTextRevision( const UIWidget* widget ) { + const auto* document = getTextDocument( widget ); + return document + ? AccessibilityTextRevision{ document->getUUID().high(), document->getUUID().low(), + document->getModificationId() } + : AccessibilityTextRevision{}; +} + +Int32 AccessibilityWidgetResolver::getTextOffset( const Doc::TextDocument& document, + const Doc::TextPosition& position ) { + return documentOffset( document, position ); +} + AccessibilityTextInfo AccessibilityWidgetResolver::getText( const UIWidget* widget, bool includeOffsets ) { - if ( widget->isType( UI_TYPE_TEXTEDIT ) ) { + if ( widget->isType( UI_TYPE_CODEEDITOR ) ) { return includeOffsets - ? getDocumentText( static_cast( widget )->getDocument() ) + ? getDocumentText( static_cast( widget )->getDocument() ) : AccessibilityTextInfo{ 0, 0, 0, true }; } if ( widget->isType( UI_TYPE_TEXTINPUT ) ) { @@ -261,9 +547,9 @@ AccessibilityState AccessibilityWidgetResolver::getState( const UIWidget* widget if ( widget->isType( UI_TYPE_COMBOBOX ) && isComboBoxExpanded( static_cast( widget ) ) ) state |= AccessibilityState::Expanded; - if ( widget->isType( UI_TYPE_TEXTEDIT ) ) { + if ( widget->isType( UI_TYPE_CODEEDITOR ) ) { state |= AccessibilityState::MultiLine; - state |= static_cast( widget )->isLocked() + state |= static_cast( widget )->isLocked() ? AccessibilityState::ReadOnly : AccessibilityState::Editable; } else if ( widget->isType( UI_TYPE_TEXTINPUT ) ) { @@ -278,11 +564,14 @@ AccessibilityState AccessibilityWidgetResolver::getState( const UIWidget* widget } AccessibilityActions AccessibilityWidgetResolver::getActions( const UIWidget* widget ) { + // Anything inside a scroll view, labels included, can be brought into view. + const AccessibilityActions scroll = + scrollViewFor( widget ) ? accessibilityActionMask( AccessibilityAction::ScrollTo ) : 0; if ( isRoot( widget ) || ( widget->isType( UI_TYPE_TEXTVIEW ) && !widget->isType( UI_TYPE_CHECKBOX ) && !widget->isType( UI_TYPE_RADIOBUTTON ) && !widget->isType( UI_TYPE_TEXTINPUT ) ) ) - return 0; - auto actions = baseActions( widget ); + return scroll; + auto actions = baseActions( widget ) | scroll; if ( widget->isType( UI_TYPE_MENUCHECKBOX ) ) { actions |= accessibilityActionMask( AccessibilityAction::Toggle ); } else if ( widget->isType( UI_TYPE_MENURADIOBUTTON ) ) { @@ -298,8 +587,8 @@ AccessibilityActions AccessibilityWidgetResolver::getActions( const UIWidget* wi actions |= accessibilityActionMask( AccessibilityAction::Toggle ); if ( widget->isType( UI_TYPE_RADIOBUTTON ) ) actions |= accessibilityActionMask( AccessibilityAction::Select ); - if ( widget->isType( UI_TYPE_TEXTEDIT ) && - !static_cast( widget )->isLocked() ) + if ( widget->isType( UI_TYPE_CODEEDITOR ) && + !static_cast( widget )->isLocked() ) actions |= accessibilityActionMask( AccessibilityAction::SetText ); else if ( widget->isType( UI_TYPE_TEXTINPUT ) && static_cast( widget )->isEditingAllowed() ) @@ -308,8 +597,8 @@ AccessibilityActions AccessibilityWidgetResolver::getActions( const UIWidget* wi static_cast( widget )->isEditingAllowed() && static_cast( widget )->getMode() != UITextInput::TextInputMode::Password ) || - ( widget->isType( UI_TYPE_TEXTEDIT ) && - !static_cast( widget )->isLocked() ) ) + ( widget->isType( UI_TYPE_CODEEDITOR ) && + !static_cast( widget )->isLocked() ) ) actions |= accessibilityActionMask( AccessibilityAction::SetTextSelection ); if ( widget->isType( UI_TYPE_COMBOBOX ) ) { actions |= @@ -356,6 +645,9 @@ UIWidget* AccessibilityWidgetResolver::getFocusOwner( Node* focused, const UIWid element = widget; } else if ( !element && widget != sceneRoot && widget->isAccessibilityElement() ) element = widget; + else if ( element && widget != element && isLeafRole( getRole( widget ) ) ) + // A leaf control does not expose its children; it owns their focus. + element = widget; } return element; } @@ -374,6 +666,11 @@ UIWidget* AccessibilityWidgetResolver::getEventTarget( UIWidget* widget, return nullptr; if ( auto* view = getOwningModelView( widget ) ) return event == AccessibilityEvent::FocusChanged ? view : nullptr; + if ( auto* leaf = getLeafOwner( widget ) ) { + // A leaf control's text view changing its text is the control's name changing. Structural + // events stay invisible: the control has no exposed children. + return event == AccessibilityEvent::ChildrenChanged ? nullptr : leaf; + } if ( event == AccessibilityEvent::StateChanged && widget->isType( UI_TYPE_DROPDOWN ) ) { auto* parent = widget->getParent(); if ( parent && parent->isType( UI_TYPE_COMBOBOX ) ) @@ -382,6 +679,48 @@ UIWidget* AccessibilityWidgetResolver::getEventTarget( UIWidget* widget, return widget; } +namespace { + +void auditWidget( const UIWidget* widget, std::vector& issues ) { + // Recycled model-view cells are exposed through the view's virtual rows, not as widgets. + if ( widget->isAccessibilityHidden() || !widget->isVisible() || + AccessibilityWidgetResolver::getOwningModelView( widget ) ) + return; + const auto role = AccessibilityWidgetResolver::getRole( widget ); + if ( role != AccessibilityRole::None && role != AccessibilityRole::Application && + role != AccessibilityRole::Label && widget->isTabFocusable() && + AccessibilityWidgetResolver::getName( widget ).empty() && + AccessibilityWidgetResolver::getValue( widget ).empty() ) + issues.push_back( { widget, "Focusable control has no accessible name (set aria-label, " + "aria-labelledby or a tooltip)." } ); + if ( !widget->getAccessibilityLabelledBy().empty() && + !findRelated( widget, widget->getAccessibilityLabelledBy() ) ) + issues.push_back( { widget, "aria-labelledby \"" + + String( widget->getAccessibilityLabelledBy() ) + + "\" does not match any widget." } ); + if ( !widget->getAccessibilityDescribedBy().empty() && + !findRelated( widget, widget->getAccessibilityDescribedBy() ) ) + issues.push_back( { widget, "aria-describedby \"" + + String( widget->getAccessibilityDescribedBy() ) + + "\" does not match any widget." } ); + // A leaf control's children are implementation details that no client can reach. + if ( AccessibilityWidgetResolver::isLeafRole( role ) ) + return; + for ( const Node* child = widget->getFirstChild(); child; child = child->getNextNode() ) { + if ( child->isWidget() ) + auditWidget( static_cast( child ), issues ); + } +} + +} // namespace + +std::vector AccessibilityWidgetResolver::audit( const UIWidget* root ) { + std::vector issues; + if ( root ) + auditWidget( root, issues ); + return issues; +} + bool AccessibilityWidgetResolver::performAction( UIWidget* widget, const AccessibilityActionRequest& request ) { if ( !( getActions( widget ) & accessibilityActionMask( request.action ) ) ) @@ -390,6 +729,13 @@ bool AccessibilityWidgetResolver::performAction( UIWidget* widget, widget->setFocus( NodeFocusReason::Unknown ); return true; } + if ( request.action == AccessibilityAction::ScrollTo ) { + // Bring the widget into view through every enclosing scroll view, innermost first. + bool scrolled = false; + for ( auto* view = scrollViewFor( widget ); view; view = scrollViewFor( view ) ) + scrolled |= view->scrollIntoView( widget ); + return scrolled; + } if ( request.action == AccessibilityAction::Toggle && widget->isType( UI_TYPE_MENUCHECKBOX ) ) { static_cast( widget )->activate(); return true; @@ -436,14 +782,22 @@ bool AccessibilityWidgetResolver::performAction( UIWidget* widget, static_cast( widget )->setText( request.value ); return true; } + if ( request.action == AccessibilityAction::SetText && widget->isType( UI_TYPE_CODEEDITOR ) ) { + // A code editor keeps its document (file path, undo history): replace its contents as + // one undoable edit instead of resetting it. + auto& document = static_cast( widget )->getDocument(); + document.selectAll(); + document.replaceSelection( request.value ); + return true; + } if ( request.action == AccessibilityAction::SetTextSelection && - ( widget->isType( UI_TYPE_TEXTINPUT ) || widget->isType( UI_TYPE_TEXTEDIT ) ) ) { + ( widget->isType( UI_TYPE_TEXTINPUT ) || widget->isType( UI_TYPE_CODEEDITOR ) ) ) { Int32 start = 0; Int32 end = 0; if ( !parseSelection( request.value, start, end ) ) return false; - auto& document = widget->isType( UI_TYPE_TEXTEDIT ) - ? static_cast( widget )->getDocument() + auto& document = widget->isType( UI_TYPE_CODEEDITOR ) + ? static_cast( widget )->getDocument() : static_cast( widget )->getDocument(); document.setSelection( documentPosition( document, start ), documentPosition( document, end ) ); diff --git a/src/eepp/ui/css/stylesheetspecification.cpp b/src/eepp/ui/css/stylesheetspecification.cpp index cea06ccc2..d7a5e2bc9 100644 --- a/src/eepp/ui/css/stylesheetspecification.cpp +++ b/src/eepp/ui/css/stylesheetspecification.cpp @@ -198,6 +198,12 @@ void StyleSheetSpecification::registerDefaultProperties() { .setType( PropertyType::String ); registerProperty( PropertyId::AccessibilityHidden, "aria-hidden", "false" ) .setType( PropertyType::Bool ); + registerProperty( PropertyId::AccessibilityLabelledBy, "aria-labelledby", "" ) + .setType( PropertyType::String ); + registerProperty( PropertyId::AccessibilityDescribedBy, "aria-describedby", "" ) + .setType( PropertyType::String ); + registerProperty( PropertyId::AccessibilityLive, "aria-live", "off" ) + .setType( PropertyType::String ); registerProperty( PropertyId::LayoutWeight, "layout-weight", "" ) .addAlias( "layout_weight" ) .addAlias( "lw8" ) diff --git a/src/eepp/ui/doc/textdocument.cpp b/src/eepp/ui/doc/textdocument.cpp index 1afe826d2..380645030 100644 --- a/src/eepp/ui/doc/textdocument.cpp +++ b/src/eepp/ui/doc/textdocument.cpp @@ -396,6 +396,9 @@ void TextDocument::reset() { mLines.clear(); mLines.emplace_back( String( "\n" ), mDocumentMutex ); } + // Every content change bumps the modification id, wholesale replacements included, so it can + // identify a version of the text (LSP responses, accessibility text caches). + mModificationId++; { Lock l( mSyntaxDefinitionMutex ); @@ -519,6 +522,7 @@ TextDocument::LoadStatus TextDocument::loadFromStream( IOStream& file, std::stri { Lock l( mLinesMutex ); mLines.clear(); + mModificationId++; } MD5::Context md5Ctx; @@ -1727,8 +1731,9 @@ size_t TextDocument::remove( const size_t& cursorIdx, TextRange range, } TextRange originalRange = range; + String removedText( getText( range ) ); mUndoStack.pushSelection( undoStack, cursorIdx, mSelection, time ); - mUndoStack.pushInsert( undoStack, getText( range ), cursorIdx, range.start(), time ); + mUndoStack.pushInsert( undoStack, removedText, cursorIdx, range.start(), time ); Int64 linesRemoved = 0; bool deletedAcrossNewLine = false; @@ -1853,7 +1858,7 @@ size_t TextDocument::remove( const size_t& cursorIdx, TextRange range, -linesRemoved ); } - notifyTextChanged( { originalRange, "" } ); + notifyTextChanged( { originalRange, "" }, &removedText ); notifyLineChanged( range.start().line() ); return linesRemoved; @@ -3965,6 +3970,7 @@ std::vector TextDocument::getLines() const { void TextDocument::setLines( std::vector&& lines ) { mLines = std::move( lines ); + mModificationId++; } std::string TextDocument::serializeUndoRedo( bool inverted ) { @@ -4396,11 +4402,21 @@ void TextDocument::setNonWordChars( const String& nonWordChars ) { mNonWordChars = nonWordChars; } -void TextDocument::notifyTextChanged( const DocumentContentChange& change ) { +void TextDocument::notifyTextChanged( const DocumentContentChange& change, const String* removed ) { Lock l( mClientsMutex ); + // Set under the clients lock, which serializes notifications. A client can edit the document + // from its callback: restore the outer edit's removed text once that nested one returns. + const String* outerRemoved = mNotifiedRemovedText; + mNotifiedRemovedText = removed; for ( auto& client : mClients ) { client->onDocumentTextChanged( change ); } + mNotifiedRemovedText = outerRemoved; +} + +const String& TextDocument::getNotifiedRemovedText() const { + static const String empty; + return mNotifiedRemovedText ? *mNotifiedRemovedText : empty; } void TextDocument::notifyCursorChanged( TextPosition selection ) { diff --git a/src/eepp/ui/models/model.cpp b/src/eepp/ui/models/model.cpp index f1ff6df36..b316a505e 100644 --- a/src/eepp/ui/models/model.cpp +++ b/src/eepp/ui/models/model.cpp @@ -181,17 +181,27 @@ std::weak_ptr Model::registerPersistentIndex( ModelIndex const auto it = mPersistentHandles.find( index ); // Easy modo: we already have a handle for this model index. if ( it != mPersistentHandles.end() ) { + ++it->second->mRegistrations; return it->second; } // Hard modo: create a new persistent handle. auto handle = std::make_shared( index ); + handle->mRegistrations = 1; std::weak_ptr weak_handle = handle; mPersistentHandles[index] = std::move( handle ); return weak_handle; } +void Model::releasePersistentIndex( const std::shared_ptr& handle ) { + if ( !handle || handle->mRegistrations == 0 || --handle->mRegistrations > 0 ) + return; + auto it = mPersistentHandles.find( handle->mIndex ); + if ( it != mPersistentHandles.end() && it->second == handle ) + mPersistentHandles.erase( it ); +} + template void Model::saveDeletedIndices( ModelIndex const& parent, int first, int last ) { std::vector deletedIndices; @@ -308,7 +318,7 @@ void Model::handleInsert( Operation const& operation ) { int newColumn = isRow ? entry.first.column() : entry.first.column() + offset; indexChanges.emplace_back( entry.first, createIndex( newRow, newColumn, entry.first.internalData(), - entry.first.internalId() ) ); + entry.first.internalId() ) ); } } @@ -337,7 +347,7 @@ void Model::handleDelete( Operation const& operation ) { int newColumn = isRow ? entry.first.column() : entry.first.column() - offset; indexChanges.emplace_back( entry.first, createIndex( newRow, newColumn, entry.first.internalData(), - entry.first.internalId() ) ); + entry.first.internalId() ) ); } } diff --git a/src/eepp/ui/models/persistentmodelindex.cpp b/src/eepp/ui/models/persistentmodelindex.cpp index cdef8cc3a..22ba76d51 100644 --- a/src/eepp/ui/models/persistentmodelindex.cpp +++ b/src/eepp/ui/models/persistentmodelindex.cpp @@ -10,6 +10,14 @@ PersistentModelIndex::PersistentModelIndex( ModelIndex const& index ) { mHandle = model->registerPersistentIndex( index ); } +void PersistentModelIndex::release() { + auto handle = mHandle.lock(); + mHandle.reset(); + // An expired handle was already dropped with its row or its model. + if ( handle && handle->mIndex.model() ) + const_cast( handle->mIndex.model() )->releasePersistentIndex( handle ); +} + int PersistentModelIndex::row() const { if ( !hasValidHandle() ) return -1; diff --git a/src/eepp/ui/tools/uiwidgetinspector.cpp b/src/eepp/ui/tools/uiwidgetinspector.cpp index 68b5f0c25..755189359 100644 --- a/src/eepp/ui/tools/uiwidgetinspector.cpp +++ b/src/eepp/ui/tools/uiwidgetinspector.cpp @@ -1,5 +1,6 @@ #include #include +#include #include #include #include @@ -109,15 +110,23 @@ class AccessibilityPropertiesModel final : public Model { { AccessibilityAction::SetText, "SetText" }, { AccessibilityAction::Expand, "Expand" }, { AccessibilityAction::Collapse, "Collapse" }, + { AccessibilityAction::ScrollTo, "ScrollTo" }, + { AccessibilityAction::SetTextSelection, "SetTextSelection" }, }; for ( const auto& action : ActionNames ) appendAccessibilityFlag( actions, action.second, accessibilityActionMask( action.first ), info.actions ); + static constexpr const char* LiveNames[] = { "Off", "Polite", "Assertive" }; mData = { - { "Role", mWidget->getElementTag() }, + { "Role", std::string( AccessibilityWidgetResolver::getRoleName( info.role ) ) + + " (" + mWidget->getElementTag() + ")" }, { "Name", info.name.toUtf8() }, { "Description", info.description.toUtf8() }, { "Value", info.value.toUtf8() }, + { "Shortcut", info.shortcut.text }, + { "Labelled By", mWidget->getAccessibilityLabelledBy() }, + { "Described By", mWidget->getAccessibilityDescribedBy() }, + { "Live", LiveNames[static_cast( mWidget->getAccessibilityLive() )] }, { "States", states }, { "Actions", actions }, { "Source", String::toString( ref.source ) }, @@ -141,6 +150,68 @@ class AccessibilityPropertiesModel final : public Model { std::vector> mData; }; +/** Lists the problems found by AccessibilityWidgetResolver::audit(). */ +class AccessibilityAuditModel final : public Model { + public: + static std::shared_ptr create() { + return std::make_shared(); + } + + size_t rowCount( const ModelIndex& ) const override { return mIssues.size(); } + + size_t columnCount( const ModelIndex& ) const override { return 2; } + + std::string columnName( const size_t& column ) const override { + return column == 0 ? "Widget" : "Issue"; + } + + Variant data( const ModelIndex& index, ModelRole role ) const override { + if ( role != ModelRole::Display || index.row() < 0 || + index.row() >= static_cast( mIssues.size() ) ) + return {}; + const auto& issue = mIssues[index.row()]; + return index.column() == 0 ? Variant( issue.label ) : Variant( issue.message ); + } + + void run( const UIWidget* root ) { + mIssues.clear(); + for ( auto& issue : AccessibilityWidgetResolver::audit( root ) ) { + std::string label( issue.widget->getElementTag() ); + if ( !issue.widget->getId().empty() ) + label += "#" + issue.widget->getId(); + mIssues.push_back( { issue.widget, std::move( label ), issue.message.toUtf8() } ); + } + if ( mIssues.empty() ) + mIssues.push_back( { nullptr, "", "No issues found." } ); + invalidate(); + } + + /** The audited widget of a row. The pointer is only compared, never dereferenced, until it is + * found again in the live tree. */ + const UIWidget* widgetAt( Int64 row ) const { + return row >= 0 && row < static_cast( mIssues.size() ) ? mIssues[row].widget + : nullptr; + } + + private: + struct Row { + const UIWidget* widget; + std::string label; + std::string message; + }; + std::vector mIssues; +}; + +bool containsNode( const Node* root, const Node* target ) { + if ( root == target ) + return true; + for ( const Node* child = root->getFirstChild(); child; child = child->getNextNode() ) { + if ( containsNode( child, target ) ) + return true; + } + return false; +} + } // namespace struct UIWidgetInspector::PickHighlightOverState { @@ -193,9 +264,14 @@ UIWindow* UIWidgetInspector::create( UISceneNode* sceneNode, const Float& menuIc + + + + + @@ -246,6 +322,25 @@ UIWindow* UIWidgetInspector::create( UISceneNode* sceneNode, const Float& menuIc auto accessibilityModel = AccessibilityPropertiesModel::create(); accessibilityView->setModel( accessibilityModel ); + UITableView* auditView = cont->find( "widget_inspector_audit_issues" ); + auditView->setAutoColumnsWidth( true ); + auditView->setHeadersVisible( true ); + auto auditModel = AccessibilityAuditModel::create(); + auditView->setModel( auditModel ); + cont->find( "widget_inspector_audit_run" ) + ->on( Event::MouseClick, [sceneNode, auditModel]( const Event* event ) { + if ( event->asMouseEvent()->getFlags() & EE_BUTTON_LMASK ) + auditModel->run( sceneNode->getRoot() ); + } ); + auditView->setOnSelection( [sceneNode, nodeTree, auditModel]( const ModelIndex& index ) { + const UIWidget* widget = auditModel->widgetAt( index.row() ); + // The tree may have changed since the audit ran: select only widgets still in it. + if ( !widget || !containsNode( sceneNode->getRoot(), widget ) ) + return; + auto* treeModel = static_cast( nodeTree->getModel() ); + nodeTree->setSelection( treeModel->getModelIndex( widget ) ); + } ); + UITableView* computedView = cont->find( "widget_inspector_computed" ); computedView->setAutoColumnsWidth( true ); computedView->setHeadersVisible( true ); diff --git a/src/eepp/ui/uicodeeditor.cpp b/src/eepp/ui/uicodeeditor.cpp index 12a340e4a..c5814fc1f 100644 --- a/src/eepp/ui/uicodeeditor.cpp +++ b/src/eepp/ui/uicodeeditor.cpp @@ -272,7 +272,7 @@ Uint32 UICodeEditor::getType() const { } bool UICodeEditor::isType( const Uint32& type ) const { - return type == getType() || UITouchDraggableWidget::isType( type ); + return type == UICodeEditor::getType() || UITouchDraggableWidget::isType( type ); } void UICodeEditor::setTheme( UITheme* Theme ) { @@ -718,6 +718,7 @@ void UICodeEditor::invalidateLineWrapMaxWidth( bool force ) { void UICodeEditor::onDocumentReloaded( TextDocument* ) { DocEvent event( this, mDoc.get(), Event::OnDocumentReloaded ); sendEvent( &event ); + notifyAccessibilityWholeTextChanged(); invalidateDraw(); invalidateLongestLineWidth(); invalidateLineWrapMaxWidth( true ); @@ -728,6 +729,7 @@ void UICodeEditor::onDocumentReloaded( TextDocument* ) { void UICodeEditor::onDocumentLoaded() { DocEvent event( this, mDoc.get(), Event::OnDocumentLoaded ); sendEvent( &event ); + notifyAccessibilityWholeTextChanged(); invalidateEditor(); invalidateDraw(); invalidateLongestLineWidth(); @@ -739,6 +741,7 @@ void UICodeEditor::onDocumentLoaded() { void UICodeEditor::onDocumentReset( TextDocument* ) { DocEvent event( this, mDoc.get(), Event::OnDocumentReset ); sendEvent( &event ); + notifyAccessibilityWholeTextChanged(); mDocView.clear(); mLinesWidthCache.clear(); mColorBoxesCache.clear(); @@ -753,6 +756,7 @@ void UICodeEditor::onDocumentReset( TextDocument* ) { void UICodeEditor::onDocumentChanged( URI oldDocURI ) { if ( mFindReplace ) mFindReplace->setDoc( mDoc ); + notifyAccessibilityWholeTextChanged(); DocChangedEvent event( this, mDoc.get(), Event::OnDocumentChanged, oldDocURI ); sendEvent( &event ); } @@ -2422,8 +2426,8 @@ void UICodeEditor::onDocumentTextChanged( const DocumentContentChange& change ) findRegionsDelayed(); checkMatchingBrackets(); sendCommonEvent( Event::OnTextChanged ); - if ( isType( UI_TYPE_TEXTEDIT ) ) - notifyAccessibilityEvent( AccessibilityEvent::ValueChanged ); + notifyAccessibilityTextChanged( &change ); + notifyAccessibilityEvent( AccessibilityEvent::ValueChanged ); } void UICodeEditor::onDocumentCursorChange( const Doc::TextPosition& ) { @@ -2444,6 +2448,7 @@ void UICodeEditor::onDocumentSelectionChange( const Doc::TextRange& ) { resetCursor(); invalidateDraw(); sendCommonEvent( Event::OnSelectionChanged ); + notifyAccessibilityEvent( AccessibilityEvent::SelectionChanged ); } void UICodeEditor::onDocumentLineCountChange( const size_t& lastCount, const size_t& newCount ) { diff --git a/src/eepp/ui/uinode.cpp b/src/eepp/ui/uinode.cpp index 36f4a4a9f..9d0711221 100644 --- a/src/eepp/ui/uinode.cpp +++ b/src/eepp/ui/uinode.cpp @@ -1800,6 +1800,8 @@ Uint32 UINode::onFocusLoss() { } void UINode::onSceneChange() { + // The const accessor never creates a manager: moving between scenes must not initialize + // accessibility. const auto* previousManager = mUISceneNode && isWidget() ? static_cast( mUISceneNode )->getAccessibilityManager() diff --git a/src/eepp/ui/uiscenenode.cpp b/src/eepp/ui/uiscenenode.cpp index b82df93f9..f593efdc8 100644 --- a/src/eepp/ui/uiscenenode.cpp +++ b/src/eepp/ui/uiscenenode.cpp @@ -1,4 +1,6 @@ +#include "accessibility/accessibilitybackend.hpp" #include +#include #include #include #include @@ -220,11 +222,11 @@ UISceneNode::~UISceneNode() { // Descendant destructors must not rediscover or register a half-destroyed ancestor. if ( mHostUISceneNode ) { // This scene is already closing, so ask the host for the manager it shares. - if ( auto* manager = mHostUISceneNode->getAccessibilityManager() ) + if ( auto* manager = mHostUISceneNode->getExistingAccessibilityManager() ) manager->onSubtreeRemoved( mRoot ); } mAccessibilityManager.reset(); - mAccessibilityState &= ~AccessibilityClientActive; + setAccessibilityClientActive( false ); if ( mAsyncResourceLoadState ) { mAsyncResourceLoadState->owner.store( nullptr, std::memory_order_release ); mAsyncResourceLoadState->alive.store( false, std::memory_order_release ); @@ -509,14 +511,12 @@ void UISceneNode::updateHostUISceneNode() { if ( mHostUISceneNode == hostScene ) return; - auto* previousManager = static_cast( this )->getAccessibilityManager(); - auto* nextManager = - hostScene ? static_cast( hostScene )->getAccessibilityManager() - : nullptr; + auto* previousManager = getExistingAccessibilityManager(); + auto* nextManager = hostScene ? hostScene->getExistingAccessibilityManager() : nullptr; if ( previousManager && previousManager != nextManager && mRoot ) { // Embedded descendants keep this local scene pointer while its host changes. Invalidate // their old-owner identities before updating the host used by getAccessibilityManager(). - const_cast( previousManager )->onSubtreeRemoved( mRoot ); + previousManager->onSubtreeRemoved( mRoot ); } if ( mHostUISceneNode ) @@ -526,7 +526,7 @@ void UISceneNode::updateHostUISceneNode() { if ( mHostUISceneNode ) { mAccessibilityManager.reset(); - mAccessibilityState &= AccessibilityPolicyMask; + setAccessibilityClientActive( false ); mHostUISceneNode->registerChildUISceneNode( this ); } } @@ -1291,10 +1291,7 @@ void UISceneNode::update( const Time& elapsed ) { mAccessibilityManager = std::make_unique( this ); if ( mAccessibilityManager ) { mAccessibilityManager->update(); - if ( mAccessibilityManager->hasActiveNativeClients() ) - mAccessibilityState |= AccessibilityClientActive; - else - mAccessibilityState &= ~AccessibilityClientActive; + setAccessibilityClientActive( mAccessibilityManager->hasActiveNativeClients() ); } drainAsyncResourceMainThreadQueue(); @@ -1348,10 +1345,9 @@ void UISceneNode::update( const Time& elapsed ) { } void UISceneNode::onWidgetDelete( Node* node ) { - auto* manager = static_cast( this )->getAccessibilityManager(); - if ( manager && node->isWidget() && !node->isDestroying() ) { - const_cast( manager )->onWidgetDelete( node->asType() ); - } + auto* manager = getExistingAccessibilityManager(); + if ( manager && node->isWidget() && !node->isDestroying() ) + manager->onWidgetDelete( node->asType() ); if ( node->isWidget() ) { UIWidget* widget = node->asType(); @@ -1409,10 +1405,31 @@ AccessibilityManager* UISceneNode::getAccessibilityManager() { } const AccessibilityManager* UISceneNode::getAccessibilityManager() const { + return getExistingAccessibilityManager(); +} + +void UISceneNode::setAccessibilityClientActive( bool active ) { + if ( ( ( mAccessibilityState & AccessibilityClientActive ) != 0 ) == active ) + return; + if ( active ) { + mAccessibilityState |= AccessibilityClientActive; + UIWidget::sAccessibilityActiveScenes.fetch_add( 1, std::memory_order_relaxed ); + } else { + mAccessibilityState &= ~AccessibilityClientActive; + UIWidget::sAccessibilityActiveScenes.fetch_sub( 1, std::memory_order_relaxed ); + if ( mAccessibilityManager ) + mAccessibilityManager->onClientsDisconnected(); + } + if ( isAccessibilityTraceEnabled() ) + std::fprintf( stderr, "eepp accessibility: scene %p client %s\n", + static_cast( this ), active ? "active" : "inactive" ); +} + +AccessibilityManager* UISceneNode::getExistingAccessibilityManager() const { // Match the mutable accessor: a closing scene's widgets must not reach any manager. if ( mNodeFlags & NODE_FLAG_CLOSE ) return nullptr; - return mHostUISceneNode ? mHostUISceneNode->getAccessibilityManager() + return mHostUISceneNode ? mHostUISceneNode->getExistingAccessibilityManager() : mAccessibilityManager.get(); } @@ -1424,10 +1441,18 @@ UISceneNode* UISceneNode::setAccessibilityPolicy( AccessibilityPolicy policy ) { if ( getAccessibilityPolicy() == policy ) return this; mAccessibilityManager.reset(); + setAccessibilityClientActive( false ); mAccessibilityState = static_cast( policy ); return this; } +void UISceneNode::announceForAccessibility( const String& message, AccessibilityLive priority ) { + if ( !hasActiveAccessibilityClients() ) + return; + if ( auto* manager = getAccessibilityManager() ) + manager->announce( message, priority ); +} + AccessibilityPolicy UISceneNode::getAccessibilityPolicy() const { return mHostUISceneNode ? mHostUISceneNode->getAccessibilityPolicy() diff --git a/src/eepp/ui/uiscrollview.cpp b/src/eepp/ui/uiscrollview.cpp index bdb4b8270..75c868b66 100644 --- a/src/eepp/ui/uiscrollview.cpp +++ b/src/eepp/ui/uiscrollview.cpp @@ -490,6 +490,30 @@ void UIScrollView::setScrollControllerPosition( const Vector2f& position ) { mVScroll->setValue( maxPosition.y > 0.f ? position.y / maxPosition.y : 0.f ); } +bool UIScrollView::scrollIntoView( const Node* node ) { + if ( !node || !mScrollView || !mScrollView->isParentOf( node ) ) + return false; + const Vector2f maxPosition( getScrollControllerMaxPosition() ); + if ( maxPosition.x <= 0.f && maxPosition.y <= 0.f ) + return true; + const Vector2f start( + mScrollView->convertToNodeSpace( node->convertToWorldSpace( Vector2f::Zero ) ) ); + const Sizef size( node->getPixelsSize() ); + const Sizef viewport( mContainer->getPixelsSize() ); + const auto fit = []( Float current, Float begin, Float length, Float view ) { + if ( begin < current || length > view ) + return begin; + return begin + length > current + view ? begin + length - view : current; + }; + Vector2f position( getScrollControllerPosition() ); + position.x = eeclamp( fit( position.x, start.x, size.getWidth(), viewport.getWidth() ), 0.f, + maxPosition.x ); + position.y = eeclamp( fit( position.y, start.y, size.getHeight(), viewport.getHeight() ), 0.f, + maxPosition.y ); + setScrollControllerPosition( position ); + return true; +} + void UIScrollView::setEnableDefaultKeybindings( bool enable ) { mDefaultKeybindings = enable; } diff --git a/src/eepp/ui/uispinbox.cpp b/src/eepp/ui/uispinbox.cpp index 9b6da6d71..20961179c 100644 --- a/src/eepp/ui/uispinbox.cpp +++ b/src/eepp/ui/uispinbox.cpp @@ -23,7 +23,6 @@ UISpinBox::UISpinBox() : mInput->setEnabled( true ); mInput->setParent( this ); mInput->setFlags( UI_SCROLLABLE ); - mInput->setAccessibilityHidden( true ); auto cb = [this]( const Event* ) { adjustChildren(); }; diff --git a/src/eepp/ui/uitextedit.cpp b/src/eepp/ui/uitextedit.cpp index 0cb181c23..5ce781b7d 100644 --- a/src/eepp/ui/uitextedit.cpp +++ b/src/eepp/ui/uitextedit.cpp @@ -86,11 +86,6 @@ void UITextEdit::setText( const String& text ) { invalidateLongestLineWidth(); } -void UITextEdit::onDocumentSelectionChange( const Doc::TextRange& range ) { - UICodeEditor::onDocumentSelectionChange( range ); - notifyAccessibilityEvent( AccessibilityEvent::SelectionChanged ); -} - void UITextEdit::setWordWrap( bool enabled ) { if ( enabled ) setFlags( UI_WORD_WRAP ); diff --git a/src/eepp/ui/uitextinput.cpp b/src/eepp/ui/uitextinput.cpp index b0effdcd8..38fa2c455 100644 --- a/src/eepp/ui/uitextinput.cpp +++ b/src/eepp/ui/uitextinput.cpp @@ -417,6 +417,14 @@ void UITextInput::onFontColorChanged() { mPassCache->setFillColor( mTextCache.getFillColor() ); } +void UITextInput::onTextChanged() { + // An input's text is its value, not its name. + sendCommonEvent( Event::OnTextChanged ); + sendCommonEvent( Event::OnValueChange ); + notifyAccessibilityEvent( AccessibilityEvent::ValueChanged ); + invalidateDraw(); +} + Text& UITextInput::getVisibleTextCache() { if ( mMode == TextInputMode::Password && mPassCache ) return *mPassCache; @@ -524,7 +532,9 @@ Int32 UITextInput::selCurEnd() { return mDoc.getSelection().end().column(); } -void UITextInput::onDocumentTextChanged( const DocumentContentChange& ) { +void UITextInput::onDocumentTextChanged( const DocumentContentChange& change ) { + // The edit record must precede the ValueChanged that UITextView::setText() reports below. + notifyAccessibilityTextChanged( &change ); Vector2f offSet = mRealAlignOffset; const String& text = mDoc.line( 0 ).getText(); diff --git a/src/eepp/ui/uitextview.cpp b/src/eepp/ui/uitextview.cpp index 42e67476c..a8153e77c 100644 --- a/src/eepp/ui/uitextview.cpp +++ b/src/eepp/ui/uitextview.cpp @@ -515,8 +515,7 @@ void UITextView::onSizeChange() { void UITextView::onTextChanged() { sendCommonEvent( Event::OnTextChanged ); sendCommonEvent( Event::OnValueChange ); - notifyAccessibilityEvent( isType( UI_TYPE_TEXTINPUT ) ? AccessibilityEvent::ValueChanged - : AccessibilityEvent::NameChanged ); + notifyAccessibilityEvent( AccessibilityEvent::NameChanged ); invalidateDraw(); } diff --git a/src/eepp/ui/uiwidget.cpp b/src/eepp/ui/uiwidget.cpp index 666010d85..ffa845498 100644 --- a/src/eepp/ui/uiwidget.cpp +++ b/src/eepp/ui/uiwidget.cpp @@ -29,10 +29,17 @@ using namespace EE::Window; namespace EE { namespace UI { +std::atomic UIWidget::sAccessibilityActiveScenes{ 0 }; + +// Allocated only for widgets that set accessibility metadata. The one-byte fields are grouped +// after the strings: 136 bytes instead of 144 with the role first (x86_64 libstdc++). struct AccessibilityProperties { - AccessibilityRole role{ AccessibilityRole::None }; String label; String description; + std::string labelledBy; + std::string describedBy; + AccessibilityRole role{ AccessibilityRole::None }; + AccessibilityLive live{ AccessibilityLive::Off }; bool roleSet{ false }; bool hidden{ false }; }; @@ -116,9 +123,8 @@ UIWidget::~UIWidget() { // Invalidate the entire subtree before OnClose handlers can delete children themselves. // Dirty-style cleanup still runs after those callbacks, as it did before accessibility. if ( !SceneManager::instance()->isShuttingDown() && mUISceneNode ) { - if ( auto* manager = - static_cast( mUISceneNode )->getAccessibilityManager() ) - const_cast( manager )->onWidgetDelete( this ); + if ( auto* manager = mUISceneNode->getExistingAccessibilityManager() ) + manager->onWidgetDelete( this ); } mNodeFlags |= NODE_FLAG_DESTROYING; onClose(); @@ -233,13 +239,50 @@ UIWidget* UIWidget::setAccessibilityHidden( bool hidden ) { return this; } +UIWidget* UIWidget::setAccessibilityLabelledBy( const std::string& id ) { + if ( getAccessibilityLabelledBy() == id ) + return this; + ensureAccessibilityProperties().labelledBy = id; + notifyAccessibilityEvent( AccessibilityEvent::NameChanged ); + return this; +} + +const std::string& UIWidget::getAccessibilityLabelledBy() const { + static const std::string empty; + return mAccessibilityProperties ? mAccessibilityProperties->labelledBy : empty; +} + +UIWidget* UIWidget::setAccessibilityDescribedBy( const std::string& id ) { + if ( getAccessibilityDescribedBy() == id ) + return this; + ensureAccessibilityProperties().describedBy = id; + notifyAccessibilityEvent( AccessibilityEvent::DescriptionChanged ); + return this; +} + +const std::string& UIWidget::getAccessibilityDescribedBy() const { + static const std::string empty; + return mAccessibilityProperties ? mAccessibilityProperties->describedBy : empty; +} + +UIWidget* UIWidget::setAccessibilityLive( AccessibilityLive live ) { + if ( getAccessibilityLive() == live ) + return this; + ensureAccessibilityProperties().live = live; + return this; +} + +AccessibilityLive UIWidget::getAccessibilityLive() const { + return mAccessibilityProperties ? mAccessibilityProperties->live : AccessibilityLive::Off; +} + AccessibilityProperties& UIWidget::ensureAccessibilityProperties() { if ( !mAccessibilityProperties ) mAccessibilityProperties = eeNew( AccessibilityProperties, () ); return *mAccessibilityProperties; } -void UIWidget::notifyAccessibilityEvent( AccessibilityEvent event ) { +void UIWidget::deliverAccessibilityEvent( AccessibilityEvent event ) { if ( !mUISceneNode || !mUISceneNode->hasActiveAccessibilityClients() ) return; if ( isDestroying() ) @@ -255,6 +298,51 @@ void UIWidget::notifyAccessibilityEvent( AccessibilityEvent event ) { if ( !target ) target = mUISceneNode->getRoot(); manager->notify( manager->getNodeRef( target ), event ); + if ( event == AccessibilityEvent::NameChanged || event == AccessibilityEvent::ValueChanged ) { + // The nearest live region speaks, unless the change is hidden from accessibility by the + // widget itself or any ancestor, including ancestors of the live region. + auto live = AccessibilityLive::Off; + for ( const Node* node = this; node; node = node->getParent() ) { + if ( !node->isWidget() ) + continue; + const auto* widget = static_cast( node ); + if ( widget->isAccessibilityHidden() ) { + live = AccessibilityLive::Off; + break; + } + if ( live == AccessibilityLive::Off ) + live = widget->getAccessibilityLive(); + } + if ( live != AccessibilityLive::Off ) + manager->onLiveRegionChanged( target, live ); + } + // Widgets labelled or described by this one (or by the control it belongs to) change too, + // even when this one is hidden: a hidden label still names its controls. + if ( event == AccessibilityEvent::NameChanged ) { + for ( const Node* node = this; node; node = node->getParent() ) { + manager->onRelationTargetChanged( node->getId() ); + if ( node == target ) + break; + } + } +} + +void UIWidget::deliverAccessibilityTextChanged( const Doc::DocumentContentChange* change ) { + // Documents also load and reset on worker threads; the manager is UI-thread only. + if ( !mUISceneNode || !mUISceneNode->hasActiveAccessibilityClients() || isDestroying() || + !Engine::isMainThread() ) + return; + if ( auto* manager = mUISceneNode->getAccessibilityManager() ) + manager->onTextChanged( this, change ); +} + +void UIWidget::deliverAccessibilityWholeTextChanged() { + if ( !mUISceneNode || !mUISceneNode->hasActiveAccessibilityClients() ) + return; + ensureMainThread( [this] { + notifyAccessibilityTextChanged( nullptr ); + notifyAccessibilityEvent( AccessibilityEvent::ValueChanged ); + } ); } void UIWidget::detachAccessibilitySource() { @@ -699,9 +787,14 @@ Uint32 UIWidget::onMouseLeave( const Vector2i& Pos, const Uint32& Flags ) { } UIWidget* UIWidget::setTooltipText( const String& text ) { + if ( mTooltipText == text ) + return this; mTooltipText = text; if ( mTooltip ) mTooltip->setText( text ); + // The tooltip is the fallback accessible name, or the description of a named widget. + notifyAccessibilityEvent( AccessibilityEvent::NameChanged ); + notifyAccessibilityEvent( AccessibilityEvent::DescriptionChanged ); return this; } @@ -711,7 +804,7 @@ UIWidget* UIWidget::setTooltipTextIfNotEmpty( const String& text ) { return this; } -String UIWidget::getTooltipText() { +String UIWidget::getTooltipText() const { return mTooltipText; } @@ -762,7 +855,25 @@ UINode* UIWidget::setThemeSkin( UITheme* Theme, const std::string& skinName ) { } Node* UIWidget::setId( const std::string& id ) { - Node::setId( id ); + // Relations resolve by id: a renamed widget stops labelling one set of controls and starts + // labelling another. The activity test comes first; the comparison only serves that + // notification. + if ( isAccessibilityActive() && mUISceneNode && mUISceneNode->hasActiveAccessibilityClients() && + id != getId() ) { + auto* manager = mUISceneNode->getAccessibilityManager(); + // Dependents are notified after the change, so they resolve the new relation. Keep the + // old id for that only when a client may have read a relation through it. + std::string previousId; + if ( manager->isRelationTarget( getIdHash() ) ) + previousId = getId(); + Node::setId( id ); + if ( !previousId.empty() ) + manager->onRelationTargetChanged( previousId ); + // The new id may complete a relation that pointed at a missing target. + manager->onRelationTargetChanged( id ); + } else { + Node::setId( id ); + } if ( !isSceneNodeLoading() && !isLoadingState() ) { getUISceneNode()->invalidateStyle( this ); @@ -1698,7 +1809,7 @@ void UIWidget::onParentChange() { getUISceneNode()->invalidateStyle( this, true ); getUISceneNode()->invalidateStyleState( this, true, true ); } - if ( mUISceneNode && mUISceneNode->hasActiveAccessibilityClients() ) + if ( isAccessibilityActive() && mUISceneNode && mUISceneNode->hasActiveAccessibilityClients() ) mUISceneNode->getAccessibilityManager()->onWidgetParentChange( this ); } @@ -1947,6 +2058,9 @@ std::vector UIWidget::getPropertiesImplemented() const { PropertyId::AccessibilityLabel, PropertyId::AccessibilityDescription, PropertyId::AccessibilityHidden, + PropertyId::AccessibilityLabelledBy, + PropertyId::AccessibilityDescribedBy, + PropertyId::AccessibilityLive, PropertyId::Theme, PropertyId::Skin, PropertyId::Flags, @@ -2530,6 +2644,19 @@ bool UIWidget::applyProperty( const StyleSheetProperty& attribute ) { case PropertyId::AccessibilityHidden: setAccessibilityHidden( attribute.asBool() ); break; + // The setters compare before storing, so an unchanged id is never copied. + case PropertyId::AccessibilityLabelledBy: + setAccessibilityLabelledBy( attribute.value() ); + break; + case PropertyId::AccessibilityDescribedBy: + setAccessibilityDescribedBy( attribute.value() ); + break; + case PropertyId::AccessibilityLive: + setAccessibilityLive( + String::iequals( attribute.value(), "assertive" ) ? AccessibilityLive::Assertive + : String::iequals( attribute.value(), "polite" ) ? AccessibilityLive::Polite + : AccessibilityLive::Off ); + break; case PropertyId::LayoutWeight: setLayoutWeight( attribute.asFloat() ); break; diff --git a/src/examples/ui_accessibility/ui_accessibility.cpp b/src/examples/ui_accessibility/ui_accessibility.cpp index c8557435b..053f71566 100644 --- a/src/examples/ui_accessibility/ui_accessibility.cpp +++ b/src/examples/ui_accessibility/ui_accessibility.cpp @@ -41,7 +41,7 @@ EE_MAIN_FUNC int main( int argc, char** argv ) { - + @@ -74,8 +74,13 @@ EE_MAIN_FUNC int main( int argc, char** argv ) { + - + + + + + )xml" ); content->setAccessibilityLabel( "Accessibility settings" ); @@ -133,6 +138,21 @@ EE_MAIN_FUNC int main( int argc, char** argv ) { } content->find( "save" )->onClick( [status]( auto ) { status->setText( "Settings saved" ); } ); + // Icon-only: its accessible name comes from the tooltip. The status line is a live region. + content->find( "refresh-status" )->onClick( [status]( auto ) { + status->setText( "Status refreshed" ); + } ); + // An application-side edit is reported as an exact change record; a replaced text is + // reported as a whole-text change. + auto description = content->find( "description" ); + content->find( "append-description" )->onClick( [description]( auto ) { + auto& document = description->getDocument(); + document.insert( 0, document.endOfDoc(), "\nappended" ); + } ); + auto projectName = content->find( "project-name" ); + content->find( "reset-project-name" )->onClick( [projectName]( auto ) { + projectName->setText( "eepp" ); + } ); content->find( "inspect" )->onClick( [scene]( auto ) { EE::UI::Tools::UIWidgetInspector::create( scene ); } ); diff --git a/src/tests/unit_tests/accessibility_tests.cpp b/src/tests/unit_tests/accessibility_tests.cpp index 03671df8c..1eef4141c 100644 --- a/src/tests/unit_tests/accessibility_tests.cpp +++ b/src/tests/unit_tests/accessibility_tests.cpp @@ -1,27 +1,36 @@ #include "utest.h" +#include "../../eepp/ui/accessibility/accessibilitybackend.hpp" + #include #include #include #include #include #include +#include #include +#include #include #include #include +#include #include +#include #include #include +#include #include #include #include +#include #include #include #include #include #include #include +#include #include #include @@ -307,6 +316,8 @@ UTEST( Accessibility, RepeatedStateAndStructuralEventsKeepTheirOrder ) { ASSERT_NE( app.getUI(), nullptr ); auto* manager = app.getUI()->getAccessibilityManager(); manager->update(); + // Events are recorded only for a client that can receive them. + manager->onNativeClientObserved(); auto root = manager->getRoot(); auto* checkbox = UICheckBox::New(); checkbox->setParent( app.getUI()->getRoot() ); @@ -671,6 +682,8 @@ UTEST( Accessibility, LiveProjectionIdentityActionsAndInvalidation ) { if ( !scene ) UTEST_SKIP( "UIApplication initialization failed: a usable graphical display is required" ); auto manager = scene->getAccessibilityManager(); + // Events are recorded only for a client that can receive them. + manager->onNativeClientObserved(); UIWidget* ignoredContainer = UIWidget::New(); ignoredContainer->setParent( scene->getRoot() ); @@ -957,9 +970,10 @@ UTEST( Accessibility, LiveProjectionIdentityActionsAndInvalidation ) { } EXPECT_TRUE( removedEventFound ); - manager->clearPendingEvents(); button->setAccessibilityLabel( "Save project" ); EXPECT_TRUE( manager->getNodeInfo( buttonRef ).name == String( "Save project" ) ); + // Repeated state notifications are never coalesced. + manager->clearPendingEvents(); manager->notify( buttonRef, AccessibilityEvent::NameChanged ); manager->notify( buttonRef, AccessibilityEvent::NameChanged ); EXPECT_EQ( manager->getPendingEvents().size(), 2u ); @@ -1072,3 +1086,1011 @@ UTEST( Accessibility, TableCellsFollowVisibleColumnOrder ) { EXPECT_TRUE( manager->getChild( rowRef, 0 ) == nameRef ); EXPECT_TRUE( manager->getChild( rowRef, 1 ) == valueRef ); } + +UTEST( Accessibility, CodeEditorIsExposedAsEditableText ) { + UIApplication app( accessibilityTestWindow(), accessibilityTestSettings() ); + ASSERT_NE( app.getUI(), nullptr ); + auto* scene = app.getUI(); + auto* manager = scene->getAccessibilityManager(); + auto* editor = UICodeEditor::New(); + editor->setParent( scene->getRoot() ); + editor->getDocument().textInput( "int main" ); + auto ref = manager->getNodeRef( editor ); + auto info = manager->getNodeInfo( ref ); + EXPECT_EQ( info.role, AccessibilityRole::TextBox ); + EXPECT_TRUE( info.value == String( "int main" ) ); + EXPECT_TRUE( info.text.valid ); + EXPECT_TRUE( ( static_cast( info.states ) & + static_cast( AccessibilityState::MultiLine ) ) != 0 ); + // SetText edits the document in place, so it stays undoable. + EXPECT_TRUE( manager->performAction( ref, { AccessibilityAction::SetText, "return 0;" } ) ); + EXPECT_TRUE( editor->getDocument().getText() == String( "return 0;" ) ); + EXPECT_TRUE( editor->getDocument().hasUndo() ); + EXPECT_TRUE( manager->performAction( ref, { AccessibilityAction::SetTextSelection, "0:6" } ) ); + EXPECT_EQ( manager->getNodeInfo( ref ).text.selectionEnd, 6 ); + + manager->onNativeClientObserved(); + manager->clearPendingEvents(); + editor->getDocument().textInput( "x" ); + bool valueChanged = false; + for ( const auto& event : manager->getPendingEvents() ) + valueChanged |= event.ref == ref && event.type == AccessibilityEvent::ValueChanged; + EXPECT_TRUE( valueChanged ); + + editor->setLocked( true ); + info = manager->getNodeInfo( ref ); + EXPECT_TRUE( ( static_cast( info.states ) & + static_cast( AccessibilityState::ReadOnly ) ) != 0 ); + EXPECT_FALSE( info.actions & accessibilityActionMask( AccessibilityAction::SetText ) ); +} + +UTEST( Accessibility, TooltipNamesIconOnlyControls ) { + UIApplication app( accessibilityTestWindow(), accessibilityTestSettings() ); + ASSERT_NE( app.getUI(), nullptr ); + auto* scene = app.getUI(); + auto* manager = scene->getAccessibilityManager(); + auto* iconButton = UIPushButton::New(); + iconButton->setParent( scene->getRoot() ); + iconButton->setTooltipText( "Save" ); + auto iconRef = manager->getNodeRef( iconButton ); + EXPECT_TRUE( manager->getNodeInfo( iconRef ).name == String( "Save" ) ); + EXPECT_TRUE( manager->getNodeInfo( iconRef ).description.empty() ); + + auto* textButton = UIPushButton::New(); + textButton->setParent( scene->getRoot() ); + textButton->setText( "Build" ); + textButton->setTooltipText( "Builds the project" ); + auto textRef = manager->getNodeRef( textButton ); + EXPECT_TRUE( manager->getNodeInfo( textRef ).name == String( "Build" ) ); + EXPECT_TRUE( manager->getNodeInfo( textRef ).description == String( "Builds the project" ) ); + + manager->onNativeClientObserved(); + manager->clearPendingEvents(); + iconButton->setTooltipText( "Save all" ); + iconButton->setTooltipText( "Save all" ); + size_t nameChanges = 0; + for ( const auto& event : manager->getPendingEvents() ) + nameChanges += event.ref == iconRef && event.type == AccessibilityEvent::NameChanged; + EXPECT_EQ( nameChanges, 1u ); +} + +UTEST( Accessibility, LabelledByAndDescribedByResolveFromTheScene ) { + UIApplication app( accessibilityTestWindow(), accessibilityTestSettings() ); + ASSERT_NE( app.getUI(), nullptr ); + auto* scene = app.getUI(); + auto* manager = scene->getAccessibilityManager(); + auto* label = UITextView::New(); + label->setParent( scene->getRoot() ); + label->setId( "user-label" ); + label->setText( "User name" ); + auto* hint = UITextView::New(); + hint->setParent( scene->getRoot() ); + hint->setId( "user-hint" ); + hint->setText( "Your login" ); + auto* input = UITextInput::New(); + input->setParent( scene->getRoot() ); + input->applyProperty( CSS::StyleSheetProperty( "aria-labelledby", "user-label" ) ); + input->applyProperty( CSS::StyleSheetProperty( "aria-describedby", "user-hint" ) ); + auto ref = manager->getNodeRef( input ); + EXPECT_TRUE( manager->getNodeInfo( ref ).name == String( "User name" ) ); + EXPECT_TRUE( manager->getNodeInfo( ref ).description == String( "Your login" ) ); + // A missing target falls back to the widget's own name. + input->setAccessibilityLabelledBy( "missing" ); + input->setAccessibilityLabel( "Login" ); + EXPECT_TRUE( manager->getNodeInfo( ref ).name == String( "Login" ) ); + // Mutual references cannot recurse. + label->setAccessibilityLabelledBy( "loop-input" ); + input->setId( "loop-input" ); + input->setAccessibilityLabelledBy( "user-label" ); + EXPECT_TRUE( manager->getNodeInfo( ref ).name == String( "User name" ) ); +} + +UTEST( Accessibility, AnnouncementsAndLiveRegionsCoalescePerFrame ) { + UIApplication app( accessibilityTestWindow(), accessibilityTestSettings() ); + ASSERT_NE( app.getUI(), nullptr ); + auto* scene = app.getUI(); + auto* manager = scene->getAccessibilityManager(); + // Dormant: nothing is queued without a client. + scene->announceForAccessibility( "Build finished" ); + EXPECT_EQ( manager->getPendingAnnouncements().size(), 0u ); + + manager->onNativeClientObserved(); + scene->announceForAccessibility( "Build finished" ); + scene->announceForAccessibility( "" ); + scene->announceForAccessibility( "Ignored", AccessibilityLive::Off ); + ASSERT_EQ( manager->getPendingAnnouncements().size(), 1u ); + EXPECT_TRUE( manager->getPendingAnnouncements()[0].message == String( "Build finished" ) ); + manager->clearPendingEvents(); + + auto* status = UILinearLayout::NewVertical(); + status->setParent( scene->getRoot() ); + status->applyProperty( CSS::StyleSheetProperty( "aria-live", "assertive" ) ); + auto* text = UITextView::New(); + text->setParent( status ); + text->setText( "Indexing 1%" ); + text->setText( "Indexing 2%" ); + ASSERT_EQ( manager->getPendingAnnouncements().size(), 1u ); + EXPECT_TRUE( manager->getPendingAnnouncements()[0].message == String( "Indexing 2%" ) ); + EXPECT_EQ( manager->getPendingAnnouncements()[0].priority, AccessibilityLive::Assertive ); + + // Widgets outside a live region stay silent. + manager->clearPendingEvents(); + auto* plain = UITextView::New(); + plain->setParent( scene->getRoot() ); + plain->setText( "Not live" ); + EXPECT_EQ( manager->getPendingAnnouncements().size(), 0u ); + + // Content hidden from accessibility stays silent, whether the hiding is on the changed + // widget or on an ancestor of the live region. + auto* hiddenText = UITextView::New(); + hiddenText->setParent( status ); + hiddenText->applyProperty( CSS::StyleSheetProperty( "aria-hidden", "true" ) ); + hiddenText->setText( "Hidden detail" ); + EXPECT_EQ( manager->getPendingAnnouncements().size(), 0u ); + auto* hiddenHost = UILinearLayout::NewVertical(); + hiddenHost->setParent( scene->getRoot() ); + hiddenHost->applyProperty( CSS::StyleSheetProperty( "aria-hidden", "true" ) ); + status->setParent( hiddenHost ); + manager->clearPendingEvents(); + text->setText( "Indexing 3%" ); + EXPECT_EQ( manager->getPendingAnnouncements().size(), 0u ); +} + +UTEST( Accessibility, MenuItemExposesItsShortcut ) { + UIApplication app( accessibilityTestWindow(), accessibilityTestSettings() ); + ASSERT_NE( app.getUI(), nullptr ); + auto* scene = app.getUI(); + auto* manager = scene->getAccessibilityManager(); + auto* menu = UIPopUpMenu::New(); + menu->setParent( scene->getRoot() ); + auto* save = menu->add( "Save", {}, "ctrl+shift+s" ); + auto* plain = menu->add( "About" ); + auto info = manager->getNodeInfo( manager->getNodeRef( save ) ); + EXPECT_FALSE( info.shortcut.empty() ); + EXPECT_TRUE( String::toLower( info.shortcut.key ) == "s" ); + EXPECT_EQ( info.shortcut.modifiers, + AccessibilityShortcut::Control | AccessibilityShortcut::Shift ); + EXPECT_TRUE( manager->getNodeInfo( manager->getNodeRef( plain ) ).shortcut.empty() ); +} + +UTEST( Accessibility, ScrollToBringsScrollViewContentIntoView ) { + UIApplication app( accessibilityTestWindow(), accessibilityTestSettings() ); + ASSERT_NE( app.getUI(), nullptr ); + auto* scene = app.getUI(); + auto* manager = scene->getAccessibilityManager(); + auto* scrollView = UIScrollView::New(); + scrollView->setParent( scene->getRoot() ); + scrollView->setPixelsSize( 100, 100 ); + auto* content = UILinearLayout::NewVertical(); + content->setParent( scrollView ); + content->setLayoutSizePolicy( SizePolicy::Fixed, SizePolicy::WrapContent ); + content->setPixelsSize( 80, 0 ); + UIPushButton* last = nullptr; + for ( int i = 0; i < 20; ++i ) { + last = UIPushButton::New(); + last->setParent( content ); + last->setText( "Item " + String::toString( i ) ); + last->setLayoutSizePolicy( SizePolicy::Fixed, SizePolicy::Fixed ); + last->setPixelsSize( 80, 30 ); + } + scene->update( Time::Zero ); + auto ref = manager->getNodeRef( last ); + auto info = manager->getNodeInfo( ref ); + ASSERT_TRUE( info.actions & accessibilityActionMask( AccessibilityAction::ScrollTo ) ); + const auto viewBounds = scrollView->getWorldBounds(); + EXPECT_FALSE( viewBounds.contains( last->getWorldBounds() ) ); + EXPECT_TRUE( manager->performAction( ref, { AccessibilityAction::ScrollTo, {} } ) ); + EXPECT_GT( scrollView->getVerticalScrollBar()->getValue(), 0.f ); + const auto bounds = last->getWorldBounds(); + EXPECT_GE( bounds.Top, viewBounds.Top - 1.f ); + EXPECT_LE( bounds.Bottom, viewBounds.Bottom + 1.f ); + // Widgets outside any scroll view do not offer the action. + auto* outside = UIPushButton::New(); + outside->setParent( scene->getRoot() ); + EXPECT_FALSE( manager->getNodeInfo( manager->getNodeRef( outside ) ).actions & + accessibilityActionMask( AccessibilityAction::ScrollTo ) ); +} + +UTEST( Accessibility, AuditReportsUnnamedControlsAndBrokenRelations ) { + UIApplication app( accessibilityTestWindow(), accessibilityTestSettings() ); + ASSERT_NE( app.getUI(), nullptr ); + auto* scene = app.getUI(); + auto* unnamed = UIPushButton::New(); + unnamed->setParent( scene->getRoot() ); + auto* tooltipNamed = UIPushButton::New(); + tooltipNamed->setParent( scene->getRoot() ); + tooltipNamed->setTooltipText( "Open" ); + auto* broken = UITextInput::New(); + broken->setParent( scene->getRoot() ); + broken->setAccessibilityLabelledBy( "nowhere" ); + auto* hidden = UIPushButton::New(); + hidden->setParent( scene->getRoot() ); + hidden->setAccessibilityHidden( true ); + + auto issues = AccessibilityWidgetResolver::audit( scene->getRoot() ); + bool unnamedReported = false; + bool brokenReported = false; + for ( const auto& issue : issues ) { + EXPECT_TRUE( issue.widget != tooltipNamed ); + EXPECT_TRUE( issue.widget != hidden ); + unnamedReported |= issue.widget == unnamed; + brokenReported |= issue.widget == broken; + } + EXPECT_TRUE( unnamedReported ); + EXPECT_TRUE( brokenReported ); +} + +UTEST( Accessibility, LeafControlsHideTheirImplementationChildren ) { + UIApplication app( accessibilityTestWindow(), accessibilityTestSettings() ); + ASSERT_NE( app.getUI(), nullptr ); + auto* scene = app.getUI(); + auto* manager = scene->getAccessibilityManager(); + auto* button = UIPushButton::New(); + button->setParent( scene->getRoot() ); + button->setText( "Save" ); + auto buttonRef = manager->getNodeRef( button ); + // The button's text view is its name, not a child label. + EXPECT_EQ( manager->getChildCount( buttonRef ), 0u ); + EXPECT_TRUE( manager->getNodeInfo( buttonRef ).name == String( "Save" ) ); + + auto* menu = UIPopUpMenu::New(); + menu->setParent( scene->getRoot() ); + auto* item = menu->add( "Open", {}, "ctrl+o" ); + auto itemRef = manager->getNodeRef( item ); + EXPECT_EQ( manager->getChildCount( itemRef ), 0u ); + EXPECT_FALSE( manager->getNodeInfo( itemRef ).shortcut.empty() ); + + // Changing the inner text reports a name change on the button itself. + manager->onNativeClientObserved(); + manager->clearPendingEvents(); + button->setText( "Save all" ); + bool buttonRenamed = false; + for ( const auto& event : manager->getPendingEvents() ) { + EXPECT_TRUE( event.ref == buttonRef || event.type != AccessibilityEvent::NameChanged ); + buttonRenamed |= event.ref == buttonRef && event.type == AccessibilityEvent::NameChanged; + } + EXPECT_TRUE( buttonRenamed ); + EXPECT_TRUE( AccessibilityWidgetResolver::getLeafOwner( button->getTextView() ) == button ); + + // Containers still expose their element children. + auto* group = UILinearLayout::NewVertical(); + group->setParent( scene->getRoot() ); + group->setAccessibilityRole( AccessibilityRole::Group ); + auto* label = UITextView::New(); + label->setParent( group ); + label->setText( "Label" ); + EXPECT_EQ( manager->getChildCount( manager->getNodeRef( group ) ), 1u ); +} + +UTEST( Accessibility, RangedTextQueriesMatchTheDocument ) { + UIApplication app( accessibilityTestWindow(), accessibilityTestSettings() ); + ASSERT_NE( app.getUI(), nullptr ); + auto* scene = app.getUI(); + auto* manager = scene->getAccessibilityManager(); + auto* editor = UICodeEditor::New(); + editor->setParent( scene->getRoot() ); + editor->getDocument().textInput( "first\nsecond\nthird" ); + auto ref = manager->getNodeRef( editor ); + const String value = manager->getNodeInfo( ref ).value; + ASSERT_EQ( manager->getTextLength( ref ), static_cast( value.size() ) ); + EXPECT_EQ( manager->getTextLength( ref ), 18 ); + EXPECT_TRUE( manager->getTextRange( ref, 6, 13 ) == String( "second\n" ) ); + EXPECT_TRUE( manager->getTextRange( ref, 13, 99 ) == String( "third" ) ); + EXPECT_TRUE( manager->getTextRange( ref, 4, 4 ).empty() ); + Int32 start = -1; + Int32 end = -1; + ASSERT_TRUE( manager->getTextLineBounds( ref, 8, start, end ) ); + EXPECT_EQ( start, 6 ); + EXPECT_EQ( end, 13 ); + // The last line's newline is not part of the exposed text. + ASSERT_TRUE( manager->getTextLineBounds( ref, 18, start, end ) ); + EXPECT_EQ( start, 13 ); + EXPECT_EQ( end, 18 ); + + // Revisions change with the text, including wholesale resets, and only then. + const auto revision = manager->getTextRevision( ref ); + EXPECT_TRUE( revision.isValid() ); + EXPECT_TRUE( manager->getTextRevision( ref ) == revision ); + editor->getDocument().textInput( "!" ); + const auto edited = manager->getTextRevision( ref ); + EXPECT_TRUE( edited != revision ); + editor->getDocument().reset(); + EXPECT_TRUE( manager->getTextRevision( ref ) != edited ); + EXPECT_EQ( manager->getTextLength( ref ), 0 ); + + // Password inputs never expose their text. + auto* password = UITextInput::New(); + password->setParent( scene->getRoot() ); + password->setText( "secret" ); + password->setMode( UITextInput::TextInputMode::Password ); + auto passwordRef = manager->getNodeRef( password ); + EXPECT_EQ( manager->getTextLength( passwordRef ), 0 ); + EXPECT_TRUE( manager->getTextRange( passwordRef, 0, 6 ).empty() ); + EXPECT_FALSE( manager->getTextRevision( passwordRef ).isValid() ); +} + +UTEST( Accessibility, DocumentRemovalsExposeTheRemovedTextDuringNotification ) { + Doc::TextDocument document; + document.textInput( "hello world" ); + struct Seen { + String inserted; + String removed; + }; + std::vector seen; + class Recorder : public Doc::TextDocument::Client { + public: + Recorder( Doc::TextDocument& document, std::vector& seen ) : + mDocument( document ), mSeen( seen ) {} + + void onDocumentTextChanged( const Doc::DocumentContentChange& change ) override { + // An edit made from a callback is notified while the outer one is still in progress. + if ( nestedEdit ) { + nestedEdit = false; + mDocument.insert( 0, mDocument.endOfDoc(), "!" ); + } + mSeen.push_back( { change.text, mDocument.getNotifiedRemovedText() } ); + } + + void onDocumentUndoRedo( const Doc::TextDocument::UndoRedo& ) override {} + + void onDocumentCursorChange( const Doc::TextPosition& ) override {} + + void onDocumentSelectionChange( const Doc::TextRange& ) override {} + + void onDocumentLineCountChange( const size_t&, const size_t& ) override {} + + void onDocumentLineChanged( const Int64& ) override {} + + void onDocumentSaved( Doc::TextDocument* ) override {} + + void onDocumentClosed( Doc::TextDocument* ) override {} + + void onDocumentDirtyOnFileSystem( Doc::TextDocument* ) override {} + + void onDocumentMoved( Doc::TextDocument* ) override {} + + void onDocumentReset( Doc::TextDocument* ) override {} + + Type getTextDocumentClientType() override { return Type::Auxiliary; } + + bool nestedEdit{ false }; + + private: + Doc::TextDocument& mDocument; + std::vector& mSeen; + } recorder( document, seen ); + document.registerClient( &recorder ); + document.remove( 0, { { 0, 5 }, { 0, 11 } } ); + ASSERT_EQ( seen.size(), 1u ); + EXPECT_TRUE( seen[0].removed == String( " world" ) ); + EXPECT_TRUE( seen[0].inserted.empty() ); + EXPECT_TRUE( document.getText() == String( "hello" ) ); + // Outside a notification there is nothing to report. + EXPECT_TRUE( document.getNotifiedRemovedText().empty() ); + + // A nested insertion sees no removed text, and the outer removal sees its own afterwards. + seen.clear(); + recorder.nestedEdit = true; + document.remove( 0, { { 0, 0 }, { 0, 2 } } ); + document.unregisterClient( &recorder ); + ASSERT_EQ( seen.size(), 2u ); + EXPECT_TRUE( seen[0].inserted == String( "!" ) ); + EXPECT_TRUE( seen[0].removed.empty() ); + EXPECT_TRUE( seen[1].removed == String( "he" ) ); + EXPECT_TRUE( document.getText() == String( "llo!" ) ); +} + +namespace { + +/** Records what the manager delivers. While ignoring, it rejects text changes the way a backend + * does when it has nothing to report them to. */ +class RecordingBackend : public AccessibilityBackend { + public: + bool ignoring{ false }; + std::vector changes; + std::vector announcements; + + std::vector events; + bool active{ true }; + + bool isAvailable() const override { return true; } + + bool hasActiveClients() const override { return active; } + + void onEvent( const AccessibilityPendingEvent& event ) override { events.push_back( event ); } + + bool onTextChanged( AccessibilityNodeRef, const AccessibilityTextChange& change ) override { + if ( ignoring ) + return false; + changes.push_back( change ); + return true; + } + + void announce( const String& message, AccessibilityLive ) override { + announcements.push_back( message ); + } +}; + +bool hasPendingEvent( const AccessibilityManager* manager, AccessibilityNodeRef ref, + AccessibilityEvent event ) { + for ( const auto& pending : manager->getPendingEvents() ) { + if ( pending.ref == ref && pending.type == event ) + return true; + } + return false; +} + +} // namespace + +UTEST( Accessibility, IgnoredTextChangesDoNotConsumeTheFrameBudget ) { + UIApplication app( accessibilityTestWindow(), accessibilityTestSettings() ); + ASSERT_NE( app.getUI(), nullptr ); + auto* scene = app.getUI(); + auto* manager = scene->getAccessibilityManager(); + auto backend = std::make_unique(); + auto* recorder = backend.get(); + manager->setBackend( std::move( backend ) ); + manager->onNativeClientObserved(); + auto* input = UITextInput::New(); + input->setParent( scene->getRoot() ); + input->setText( "eepp" ); + manager->update(); + recorder->changes.clear(); + + // The ignored replacement includes a whole-text reset; it must not silence later edits. + recorder->ignoring = true; + input->setText( "replaced" ); + recorder->ignoring = false; + ASSERT_TRUE( recorder->changes.empty() ); + for ( int i = 0; i < 6; ++i ) + input->getDocument().textInput( "!" ); + // The frame still has its full budget: four exact edits, then one whole-text change that + // covers the rest of the frame. + ASSERT_EQ( recorder->changes.size(), 5u ); + for ( size_t i = 0; i < 4; ++i ) { + EXPECT_EQ( recorder->changes[i].offset, static_cast( 8 + i ) ); + EXPECT_TRUE( recorder->changes[i].inserted == String( "!" ) ); + } + EXPECT_TRUE( recorder->changes[4].isWholeText() ); + + // The budget resets every frame. + manager->update(); + recorder->changes.clear(); + input->getDocument().textInput( "?" ); + ASSERT_EQ( recorder->changes.size(), 1u ); + EXPECT_FALSE( recorder->changes[0].isWholeText() ); +} + +UTEST( Accessibility, QueuedLiveAnnouncementsAreRevalidatedOnDelivery ) { + UIApplication app( accessibilityTestWindow(), accessibilityTestSettings() ); + ASSERT_NE( app.getUI(), nullptr ); + auto* scene = app.getUI(); + auto* manager = scene->getAccessibilityManager(); + auto backend = std::make_unique(); + auto* recorder = backend.get(); + manager->setBackend( std::move( backend ) ); + manager->onNativeClientObserved(); + auto* status = UILinearLayout::NewVertical(); + status->setParent( scene->getRoot() ); + status->applyProperty( CSS::StyleSheetProperty( "aria-live", "polite" ) ); + auto* text = UITextView::New(); + text->setParent( status ); + text->setText( "Saved" ); + manager->update(); + ASSERT_EQ( recorder->announcements.size(), 1u ); + + // Hidden after the change was queued, before delivery. + text->setText( "Saving" ); + status->setAccessibilityHidden( true ); + manager->update(); + EXPECT_EQ( recorder->announcements.size(), 1u ); + + // Gone before delivery. + status->setAccessibilityHidden( false ); + text->setText( "Saved again" ); + eeDelete( text ); + manager->update(); + EXPECT_EQ( recorder->announcements.size(), 1u ); +} + +UTEST( Accessibility, RelationTargetChangesNotifyDependents ) { + UIApplication app( accessibilityTestWindow(), accessibilityTestSettings() ); + ASSERT_NE( app.getUI(), nullptr ); + auto* scene = app.getUI(); + auto* manager = scene->getAccessibilityManager(); + manager->onNativeClientObserved(); + auto* label = UITextView::New(); + label->setParent( scene->getRoot() ); + label->setId( "name_label" ); + label->setText( "Name" ); + auto* hint = UITextView::New(); + hint->setParent( scene->getRoot() ); + hint->setId( "name_hint" ); + hint->setText( "Your full name" ); + auto* input = UITextInput::New(); + input->setParent( scene->getRoot() ); + input->applyProperty( CSS::StyleSheetProperty( "aria-labelledby", "name_label" ) ); + input->applyProperty( CSS::StyleSheetProperty( "aria-describedby", "name_hint" ) ); + const auto ref = manager->getNodeRef( input ); + // A client reads the input, caching its name and description. + EXPECT_TRUE( manager->getNodeInfo( ref ).name == String( "Name" ) ); + manager->clearPendingEvents(); + + label->setText( "Full name" ); + EXPECT_TRUE( hasPendingEvent( manager, ref, AccessibilityEvent::NameChanged ) ); + EXPECT_FALSE( hasPendingEvent( manager, ref, AccessibilityEvent::DescriptionChanged ) ); + EXPECT_TRUE( manager->getNodeInfo( ref ).name == String( "Full name" ) ); + manager->clearPendingEvents(); + + hint->setText( "As on your passport" ); + EXPECT_TRUE( hasPendingEvent( manager, ref, AccessibilityEvent::DescriptionChanged ) ); + EXPECT_FALSE( hasPendingEvent( manager, ref, AccessibilityEvent::NameChanged ) ); + manager->clearPendingEvents(); + + // Renaming the target's id breaks the relation; removing it does too. + label->setId( "other_label" ); + EXPECT_TRUE( hasPendingEvent( manager, ref, AccessibilityEvent::NameChanged ) ); + manager->clearPendingEvents(); + eeDelete( hint ); + EXPECT_TRUE( hasPendingEvent( manager, ref, AccessibilityEvent::DescriptionChanged ) ); + manager->clearPendingEvents(); + + // Unrelated widgets with ids do not touch the input. + auto* other = UITextView::New(); + other->setParent( scene->getRoot() ); + other->setId( "unrelated" ); + other->setText( "Other" ); + EXPECT_FALSE( hasPendingEvent( manager, ref, AccessibilityEvent::NameChanged ) ); +} + +UTEST( Accessibility, SuppressedTextChangesStayWithinTheirWindow ) { + UIApplication app( accessibilityTestWindow(), accessibilityTestSettings() ); + ASSERT_NE( app.getUI(), nullptr ); + auto* secondary = app.createWindow( + WindowSettings( 320, 240, "Accessibility Shared Document", WindowStyle::Default, + WindowBackend::Default, 32, {}, 1, false, true ) ); + ASSERT_NE( secondary, nullptr ); + // One document shown in both windows. + auto document = std::make_shared(); + document->textInput( "eepp" ); + UISceneNode* scenes[2] = { app.getUI(), secondary }; + AccessibilityManager* managers[2]{}; + RecordingBackend* recorders[2]{}; + AccessibilityNodeRef refs[2]; + UICodeEditor* editors[2]{}; + auto addEditor = [&]( size_t i ) { + editors[i] = UICodeEditor::New(); + editors[i]->setParent( scenes[i]->getRoot() ); + editors[i]->setDocument( document ); + refs[i] = managers[i]->getNodeRef( editors[i] ); + }; + for ( size_t i = 0; i < 2; ++i ) { + managers[i] = scenes[i]->getAccessibilityManager(); + auto backend = std::make_unique(); + recorders[i] = backend.get(); + managers[i]->setBackend( std::move( backend ) ); + managers[i]->onNativeClientObserved(); + addEditor( i ); + } + // Node refs are manager-local and each manager allocates its own sequence. Recreate the + // editor of the window that is behind until both editors get the same ref, the case where a + // shared identity would collide. Ids are never reused, so each retry advances that sequence. + for ( int attempt = 0; attempt < 64 && !( refs[0] == refs[1] ); ++attempt ) { + const size_t behind = refs[0].id < refs[1].id ? 0 : 1; + eeDelete( editors[behind] ); + addEditor( behind ); + } + ASSERT_TRUE( refs[0] == refs[1] ); + for ( size_t i = 0; i < 2; ++i ) { + managers[i]->update(); + recorders[i]->changes.clear(); + } + + // A client replaces the text through the first window, which reports the change itself. + managers[0]->setSuppressedTextChanges( refs[0] ); + EXPECT_TRUE( managers[0]->performAction( refs[0], { AccessibilityAction::SetText, "eepp2" } ) ); + managers[0]->setSuppressedTextChanges( {} ); + EXPECT_TRUE( recorders[0]->changes.empty() ); + // The second window's editor changed too and must say so. + ASSERT_EQ( recorders[1]->changes.size(), 2u ); + EXPECT_TRUE( recorders[1]->changes[0].removed == String( "eepp" ) ); + EXPECT_TRUE( recorders[1]->changes[1].inserted == String( "eepp2" ) ); + + // Suppression ended, and it consumed none of the first window's budget. + for ( int i = 0; i < 4; ++i ) + document->textInput( "!" ); + EXPECT_EQ( recorders[0]->changes.size(), 4u ); + for ( const auto& change : recorders[0]->changes ) + EXPECT_FALSE( change.isWholeText() ); +} + +UTEST( Accessibility, ExactRemovalRecordsCarryTheRemovedText ) { + UIApplication app( accessibilityTestWindow(), accessibilityTestSettings() ); + ASSERT_NE( app.getUI(), nullptr ); + auto* manager = app.getUI()->getAccessibilityManager(); + auto backend = std::make_unique(); + auto* recorder = backend.get(); + manager->setBackend( std::move( backend ) ); + manager->onNativeClientObserved(); + auto* editor = UICodeEditor::New(); + editor->setParent( app.getUI()->getRoot() ); + auto& document = editor->getDocument(); + document.textInput( "hello" ); + auto nextFrame = [&] { + manager->update(); + recorder->changes.clear(); + }; + nextFrame(); + + document.deleteToPreviousChar(); + ASSERT_EQ( recorder->changes.size(), 1u ); + EXPECT_EQ( recorder->changes[0].offset, 4 ); + EXPECT_TRUE( recorder->changes[0].removed == String( "o" ) ); + EXPECT_TRUE( recorder->changes[0].inserted.empty() ); + nextFrame(); + + // The quick typing and the deletion form one undo group: undo reinserts "o", then removes + // "hello"; redo replays both in the original order. + document.undo(); + ASSERT_EQ( recorder->changes.size(), 2u ); + EXPECT_TRUE( recorder->changes[0].inserted == String( "o" ) ); + EXPECT_TRUE( recorder->changes[0].removed.empty() ); + EXPECT_EQ( recorder->changes[1].offset, 0 ); + EXPECT_TRUE( recorder->changes[1].removed == String( "hello" ) ); + EXPECT_TRUE( document.isEmpty() ); + nextFrame(); + document.redo(); + ASSERT_EQ( recorder->changes.size(), 2u ); + EXPECT_TRUE( recorder->changes[0].inserted == String( "hello" ) ); + EXPECT_TRUE( recorder->changes[1].removed == String( "o" ) ); + EXPECT_TRUE( document.getText() == String( "hell" ) ); + nextFrame(); + + // A replacement is a removal followed by an insertion. + document.setSelection( { { 0, 0 }, { 0, 4 } } ); + document.textInput( "J" ); + ASSERT_EQ( recorder->changes.size(), 2u ); + EXPECT_TRUE( recorder->changes[0].removed == String( "hell" ) ); + EXPECT_TRUE( recorder->changes[1].inserted == String( "J" ) ); + EXPECT_TRUE( document.getText() == String( "J" ) ); +} + +namespace { + +/** Reads the node back from inside each name and description notification, the way a synchronous + * native observer would. */ +class ObservingBackend : public AccessibilityBackend { + public: + explicit ObservingBackend( AccessibilityManager& manager ) : mManager( manager ) {} + + std::vector> observed; + + bool isAvailable() const override { return true; } + + bool hasActiveClients() const override { return true; } + + void onEvent( const AccessibilityPendingEvent& event ) override { + if ( event.type == AccessibilityEvent::NameChanged ) + observed.emplace_back( event.type, mManager.getNodeInfo( event.ref ).name ); + else if ( event.type == AccessibilityEvent::DescriptionChanged ) + observed.emplace_back( event.type, mManager.getNodeInfo( event.ref ).description ); + } + + private: + AccessibilityManager& mManager; +}; + +} // namespace + +UTEST( Accessibility, IdChangesNotifyOnlyRememberedRelationTargets ) { + UIApplication app( accessibilityTestWindow(), accessibilityTestSettings() ); + ASSERT_NE( app.getUI(), nullptr ); + auto* scene = app.getUI(); + auto* manager = scene->getAccessibilityManager(); + auto backend = std::make_unique( *manager ); + auto* observer = backend.get(); + manager->setBackend( std::move( backend ) ); + scene->update( Time::Zero ); + ASSERT_TRUE( scene->hasActiveAccessibilityClients() ); + // Ids longer than any small-string buffer. + const std::string labelId( "field_label_" + std::string( 64, 'l' ) ); + const std::string hintId( "field_hint_" + std::string( 64, 'h' ) ); + const std::string movedId( "moved_" + std::string( 64, 'm' ) ); + const std::string unrelatedId( "unrelated_" + std::string( 64, 'u' ) ); + auto* label = UITextView::New(); + label->setParent( scene->getRoot() ); + label->setId( labelId ); + label->setText( "Name" ); + auto* hint = UITextView::New(); + hint->setParent( scene->getRoot() ); + hint->setId( hintId ); + hint->setText( "Hint" ); + auto* unrelated = UIWidget::New(); + unrelated->setParent( scene->getRoot() ); + unrelated->setId( unrelatedId ); + auto* input = UITextInput::New(); + input->setParent( scene->getRoot() ); + input->setAccessibilityLabelledBy( labelId ); + input->setAccessibilityDescribedBy( hintId ); + const auto ref = manager->getNodeRef( input ); + const auto info = manager->getNodeInfo( ref ); + EXPECT_TRUE( info.name == String( "Name" ) ); + EXPECT_TRUE( info.description == String( "Hint" ) ); + EXPECT_TRUE( manager->isRelationTarget( String::hash( labelId ) ) ); + EXPECT_TRUE( manager->isRelationTarget( String::hash( hintId ) ) ); + EXPECT_FALSE( manager->isRelationTarget( String::hash( unrelatedId ) ) ); + observer->observed.clear(); + + // An unreferenced long id changes without notifying anyone. + unrelated->setId( unrelatedId + "_renamed" ); + unrelated->setId( unrelatedId ); + EXPECT_TRUE( observer->observed.empty() ); + + // The observer reads the relation after the rename, not before it. + label->setId( movedId ); + ASSERT_EQ( observer->observed.size(), 1u ); + EXPECT_EQ( observer->observed[0].first, AccessibilityEvent::NameChanged ); + EXPECT_FALSE( observer->observed[0].second == String( "Name" ) ); + EXPECT_TRUE( observer->observed[0].second == manager->getNodeInfo( ref ).name ); + observer->observed.clear(); + + // A missing target becoming available again renames the input through the new id. + label->setId( labelId ); + ASSERT_EQ( observer->observed.size(), 1u ); + EXPECT_EQ( observer->observed[0].first, AccessibilityEvent::NameChanged ); + EXPECT_TRUE( observer->observed[0].second == String( "Name" ) ); + observer->observed.clear(); + + hint->setId( movedId ); + ASSERT_EQ( observer->observed.size(), 1u ); + EXPECT_EQ( observer->observed[0].first, AccessibilityEvent::DescriptionChanged ); + EXPECT_FALSE( observer->observed[0].second == String( "Hint" ) ); + observer->observed.clear(); + hint->setId( hintId ); + ASSERT_EQ( observer->observed.size(), 1u ); + EXPECT_TRUE( observer->observed[0].second == String( "Hint" ) ); +} + +UTEST( Accessibility, RootOnlyDeletionStillNotifiesRelationDependents ) { + UIApplication app( accessibilityTestWindow(), accessibilityTestSettings() ); + ASSERT_NE( app.getUI(), nullptr ); + auto* scene = app.getUI(); + auto* root = scene->getRoot(); + auto* label = UITextView::New(); + label->setParent( root ); + label->setId( "root_label" ); + label->setText( "Window name" ); + auto* hint = UITextView::New(); + hint->setParent( root ); + hint->setId( "root_hint" ); + hint->setText( "Window hint" ); + root->setAccessibilityLabelledBy( "root_label" ); + root->setAccessibilityDescribedBy( "root_hint" ); + auto* manager = scene->getAccessibilityManager(); + manager->onNativeClientObserved(); + // The client reads only the root: it is the single widget with an identity. + const auto ref = manager->getNodeRef( root ); + EXPECT_TRUE( manager->getNodeInfo( ref ).name == String( "Window name" ) ); + manager->clearPendingEvents(); + + eeDelete( label ); + EXPECT_TRUE( hasPendingEvent( manager, ref, AccessibilityEvent::NameChanged ) ); + manager->clearPendingEvents(); + eeDelete( hint ); + EXPECT_TRUE( hasPendingEvent( manager, ref, AccessibilityEvent::DescriptionChanged ) ); +} + +UTEST( Accessibility, SceneLifecycleNeverCreatesAManager ) { + auto hasManager = []( const UISceneNode* scene ) { + return scene->getAccessibilityManager() != nullptr; + }; + for ( auto policy : { AccessibilityPolicy::Disabled, AccessibilityPolicy::Auto } ) { + auto settings = accessibilityTestSettings(); + settings.accessibilityPolicy = policy; + UIApplication app( accessibilityTestWindow(), settings ); + ASSERT_NE( app.getUI(), nullptr ); + auto* host = app.getUI(); + auto* secondHost = app.createWindow( accessibilityTestWindow() ); + ASSERT_NE( secondHost, nullptr ); + // Neither host has been updated yet, so neither has a manager. + ASSERT_FALSE( hasManager( host ) ); + ASSERT_FALSE( hasManager( secondHost ) ); + + // Two nesting levels, populated, moved to another host and destroyed. + auto* nested = UISceneNode::New( app.getWindow() ); + nested->setParent( host->getRoot() ); + auto* inner = UISceneNode::New( app.getWindow() ); + inner->setParent( nested->getRoot() ); + for ( auto* scene : { nested, inner } ) { + auto* widget = UIWidget::New(); + widget->setParent( scene->getRoot() ); + eeDelete( widget ); + UIPushButton::New()->setParent( scene->getRoot() ); + } + EXPECT_FALSE( hasManager( host ) ); + nested->setParent( secondHost->getRoot() ); + EXPECT_FALSE( hasManager( host ) ); + EXPECT_FALSE( hasManager( secondHost ) ); + eeDelete( inner ); + eeDelete( nested ); + EXPECT_FALSE( hasManager( host ) ); + EXPECT_FALSE( hasManager( secondHost ) ); + + // A Disabled host never gets one, not even from its own updates. + host->update( Time::Zero ); + EXPECT_EQ( hasManager( host ), policy != AccessibilityPolicy::Disabled ); + } +} + +UTEST( Accessibility, DisconnectedClientsOnlyInvalidateOnRemoval ) { + UIApplication app( accessibilityTestWindow(), accessibilityTestSettings() ); + ASSERT_NE( app.getUI(), nullptr ); + auto* scene = app.getUI(); + auto* manager = scene->getAccessibilityManager(); + auto backend = std::make_unique(); + auto* recorder = backend.get(); + manager->setBackend( std::move( backend ) ); + scene->update( Time::Zero ); + ASSERT_TRUE( scene->hasActiveAccessibilityClients() ); + auto* group = UIWidget::New(); + group->setParent( scene->getRoot() ); + auto* button = UIPushButton::New(); + button->setParent( group ); + auto* edited = UIPushButton::New(); + edited->setParent( group ); + // The client knows the buttons, not their parent. + const auto buttonRef = manager->getNodeRef( button ); + const auto editedRef = manager->getNodeRef( edited ); + + recorder->active = false; + scene->update( Time::Zero ); + ASSERT_FALSE( scene->hasActiveAccessibilityClients() ); + recorder->events.clear(); + eeDelete( button ); + manager->onSubtreeRemoved( edited ); + // Identities are gone and the backend detached their wrappers... + EXPECT_FALSE( manager->isValid( buttonRef ) ); + EXPECT_FALSE( manager->isValid( editedRef ) ); + ASSERT_EQ( recorder->events.size(), 2u ); + for ( const auto& event : recorder->events ) { + EXPECT_EQ( event.type, AccessibilityEvent::Destroyed ); + EXPECT_FALSE( event.related.isValid() ); + } + // ...but nothing was queued for a client, and the parent was not resolved to announce it. + EXPECT_TRUE( manager->getPendingEvents().empty() ); + + // A reconnecting client starts from a consistent tree. + recorder->active = true; + scene->update( Time::Zero ); + const auto groupRef = manager->getNodeRef( group ); + ASSERT_EQ( manager->getChildCount( groupRef ), 1u ); + EXPECT_TRUE( manager->getChild( groupRef, 0 ) == manager->getNodeRef( edited ) ); + EXPECT_FALSE( manager->getNodeRef( edited ) == editedRef ); +} + +UTEST( Accessibility, SpinBoxExposesOnlyItselfWithoutHidingItsInput ) { + UIApplication app( accessibilityTestWindow(), accessibilityTestSettings() ); + ASSERT_NE( app.getUI(), nullptr ); + auto* scene = app.getUI(); + auto* manager = scene->getAccessibilityManager(); + auto* spin = UISpinBox::New(); + spin->setParent( scene->getRoot() ); + spin->setPixelsSize( 120, 32 ); + spin->setMinValue( 0 ); + spin->setMaxValue( 10 ); + spin->setValue( 4 ); + spin->setAccessibilityLabel( "Copies" ); + scene->update( Time::Zero ); + // The leaf role hides the input; it needs no accessibility metadata of its own. + EXPECT_FALSE( spin->getTextInput()->isAccessibilityHidden() ); + const auto ref = manager->getNodeRef( spin ); + const auto info = manager->getNodeInfo( ref ); + EXPECT_EQ( info.role, AccessibilityRole::SpinButton ); + EXPECT_TRUE( info.name == String( "Copies" ) ); + EXPECT_TRUE( info.range.valid ); + EXPECT_EQ( info.range.maximum, 10. ); + EXPECT_EQ( std::stod( info.value.toUtf8() ), 4. ); + EXPECT_EQ( manager->getChildCount( ref ), 0u ); + EXPECT_TRUE( manager->hitTest( spin->getTextInput()->getWorldBounds().getCenter() ) == ref ); + spin->getTextInput()->setFocus(); + EXPECT_TRUE( manager->getKeyboardFocusedNode() == ref ); +} + +UTEST( Accessibility, AriaPropertiesSurviveUntilAClientAttaches ) { + UIApplication app( accessibilityTestWindow(), accessibilityTestSettings() ); + ASSERT_NE( app.getUI(), nullptr ); + auto* scene = app.getUI(); + auto* manager = scene->getAccessibilityManager(); + // Ids longer than the inline string capacity. + const std::string labelId( "accessibility-relation-label-with-a-long-identifier" ); + const std::string hintId( "accessibility-relation-hint-with-a-long-identifier" ); + auto* label = UITextView::New(); + label->setParent( scene->getRoot() ); + label->setId( labelId ); + label->setText( "Destination" ); + auto* hint = UITextView::New(); + hint->setParent( scene->getRoot() ); + hint->setId( hintId ); + hint->setText( "Where files are copied" ); + auto* input = UITextInput::New(); + input->setParent( scene->getRoot() ); + auto* status = UITextView::New(); + status->setParent( scene->getRoot() ); + auto* button = UIPushButton::New(); + button->setParent( scene->getRoot() ); + + // Set while no client exists. + input->applyProperty( CSS::StyleSheetProperty( "aria-labelledby", labelId ) ); + input->applyProperty( CSS::StyleSheetProperty( "aria-describedby", hintId ) ); + status->applyProperty( CSS::StyleSheetProperty( "aria-live", "ASSERTIVE" ) ); + button->applyProperty( CSS::StyleSheetProperty( "aria-label", "@string(copy_now, Copy now)" ) ); + EXPECT_TRUE( input->getAccessibilityLabelledBy() == labelId ); + EXPECT_EQ( status->getAccessibilityLive(), AccessibilityLive::Assertive ); + status->applyProperty( CSS::StyleSheetProperty( "aria-live", "Polite" ) ); + EXPECT_EQ( status->getAccessibilityLive(), AccessibilityLive::Polite ); + + // A client attaching later reads all of it. + manager->onNativeClientObserved(); + auto info = manager->getNodeInfo( manager->getNodeRef( input ) ); + EXPECT_TRUE( info.name == String( "Destination" ) ); + EXPECT_TRUE( info.description == String( "Where files are copied" ) ); + EXPECT_TRUE( manager->getNodeInfo( manager->getNodeRef( button ) ).name == + String( "Copy now" ) ); + + // Reapplying unchanged values reports nothing. + manager->clearPendingEvents(); + input->applyProperty( CSS::StyleSheetProperty( "aria-labelledby", labelId ) ); + input->applyProperty( CSS::StyleSheetProperty( "aria-describedby", hintId ) ); + button->applyProperty( CSS::StyleSheetProperty( "aria-label", "@string(copy_now, Copy now)" ) ); + status->applyProperty( CSS::StyleSheetProperty( "aria-live", "polite" ) ); + EXPECT_TRUE( manager->getPendingEvents().empty() ); + + // Resetting to empty removes the metadata. + input->applyProperty( CSS::StyleSheetProperty( "aria-labelledby", "" ) ); + button->applyProperty( CSS::StyleSheetProperty( "aria-label", "" ) ); + status->applyProperty( CSS::StyleSheetProperty( "aria-live", "off" ) ); + EXPECT_TRUE( input->getAccessibilityLabelledBy().empty() ); + EXPECT_EQ( status->getAccessibilityLive(), AccessibilityLive::Off ); + EXPECT_TRUE( manager->getNodeInfo( manager->getNodeRef( button ) ).name.empty() ); +} + +UTEST( Accessibility, DisconnectReleasesQueriedRowsAndReconnectsFresh ) { + UIApplication app( accessibilityTestWindow(), accessibilityTestSettings() ); + ASSERT_NE( app.getUI(), nullptr ); + auto* scene = app.getUI(); + auto* manager = scene->getAccessibilityManager(); + auto backend = std::make_unique(); + auto* recorder = backend.get(); + manager->setBackend( std::move( backend ) ); + scene->update( Time::Zero ); + auto model = std::make_shared( 6 ); + auto* list = UIListView::New(); + list->setParent( scene->getRoot() ); + list->setModel( model ); + const auto listRef = manager->getNodeRef( list ); + ASSERT_EQ( manager->getChildCount( listRef ), 6u ); + for ( size_t row = 0; row < 6; ++row ) + manager->getChild( listRef, row ); + const auto staleRow = manager->getChild( listRef, 3 ); + ASSERT_TRUE( manager->isValid( staleRow ) ); + // A registration made outside accessibility, sharing row 3's handle. + Models::PersistentModelIndex probe( model->index( 3 ) ); + EXPECT_EQ( model->persistentCount(), 6u ); + + recorder->active = false; + scene->update( Time::Zero ); + // The client's rows are gone and their handles released; the probe's survives. + EXPECT_FALSE( manager->isValid( staleRow ) ); + EXPECT_EQ( model->persistentCount(), 1u ); + model->insertAt( 0, 5 ); + EXPECT_EQ( probe.row(), 4 ); + + recorder->active = true; + scene->update( Time::Zero ); + ASSERT_EQ( manager->getChildCount( listRef ), 7u ); + const auto freshRow = manager->getChild( listRef, 4 ); + EXPECT_TRUE( manager->isValid( freshRow ) ); + EXPECT_FALSE( freshRow == staleRow ); + EXPECT_TRUE( manager->getNodeInfo( freshRow ).name == String( "40" ) ); + EXPECT_FALSE( manager->isValid( staleRow ) ); +} diff --git a/src/tests/unit_tests/modeloperations_tests.cpp b/src/tests/unit_tests/modeloperations_tests.cpp index 685d5a107..76472af20 100644 --- a/src/tests/unit_tests/modeloperations_tests.cpp +++ b/src/tests/unit_tests/modeloperations_tests.cpp @@ -211,6 +211,36 @@ UTEST( ModelInsert, persistentIndexesShiftWithoutIdentityCollisions ) { ASSERT_EQ( third.data().asInt(), 30 ); } +UTEST( ModelInsert, releasedPersistentIndexesKeepSharedHandles ) { + PersistentRowsModel model( false ); + // Two registrations of one row share its handle; a third tracks another row. + PersistentModelIndex first( model.index( 1 ) ); + PersistentModelIndex second( model.index( 1 ) ); + PersistentModelIndex other( model.index( 2 ) ); + PersistentModelIndex copyOfSecond( second ); + + // Releasing one registration keeps the handle for the other. + first.release(); + EXPECT_FALSE( first.hasValidHandle() ); + model.insertAt( 0, 5 ); + EXPECT_EQ( second.row(), 2 ); + EXPECT_EQ( second.data().asInt(), 20 ); + + // Releasing the last one drops it: copies stop tracking, other rows are unaffected. + second.release(); + EXPECT_FALSE( copyOfSecond.hasValidHandle() ); + model.insertAt( 0, 1 ); + EXPECT_EQ( other.row(), 4 ); + EXPECT_EQ( other.data().asInt(), 30 ); + + // Releasing twice, or after the row is gone, does nothing. + second.release(); + PersistentModelIndex deleted( model.index( 0 ) ); + model.deleteAt( 0 ); + deleted.release(); + EXPECT_EQ( other.row(), 3 ); +} + UTEST( ModelInsert, persistentIndexesPreserveInternalIds ) { PersistentRowsModel model( true ); PersistentModelIndex tracked( model.index( 2 ) ); diff --git a/src/tests/unit_tests/uicodeeditor_tests.cpp b/src/tests/unit_tests/uicodeeditor_tests.cpp index 1a9483e8c..c35780582 100644 --- a/src/tests/unit_tests/uicodeeditor_tests.cpp +++ b/src/tests/unit_tests/uicodeeditor_tests.cpp @@ -8,6 +8,7 @@ #include #include #include +#include #include "../../tools/ecode/keybindingshelper.cpp" @@ -140,6 +141,18 @@ UTEST( MainThreadLifetime, DispatcherCanBeAttachedAfterConstruction ) { EXPECT_TRUE( called ); } +UTEST( UICodeEditor, SubclassesReportTheCodeEditorType ) { + UIApplication app( WindowSettings{ 320, 240, "eepp - code editor type test" } ); + // isType() must not dispatch to the subclass's getType(): a text edit is a code editor. + auto* textEdit = UITextEdit::New(); + textEdit->setParent( app.getUI()->getRoot() ); + EXPECT_TRUE( textEdit->isType( UI_TYPE_TEXTEDIT ) ); + EXPECT_TRUE( textEdit->isType( UI_TYPE_CODEEDITOR ) ); + auto* subclass = eeNew( TestableCodeEditor, () ); + subclass->setParent( app.getUI()->getRoot() ); + EXPECT_TRUE( subclass->isType( UI_TYPE_CODEEDITOR ) ); +} + UTEST( UICodeEditor, DefersLongestLineMeasurementForLargeChanges ) { UIApplication app( WindowSettings{ 320, 240, "eepp - deferred longest line test" } ); auto* editor = eeNew( TestableCodeEditor, () ); diff --git a/src/tools/ecode/ecode.cpp b/src/tools/ecode/ecode.cpp index ebfb304f7..79adc00bd 100644 --- a/src/tools/ecode/ecode.cpp +++ b/src/tools/ecode/ecode.cpp @@ -1026,7 +1026,10 @@ void App::onFileDropped( std::string file, bool openBinaryAsDocument ) { bool doesntNeedEmptyEditor = ( Image::isImageExtension( file ) && FileSystem::fileExtension( file ) != "svg" ) || SoundFileFactory::isKnownFileExtension( file ); - if ( node && node->isType( UI_TYPE_CODEEDITOR ) ) { + // Only reuse document editors. Other code editors (a commit message box, a dialog input) must + // not receive the dropped file. + if ( node && node->isType( UI_TYPE_CODEEDITOR ) && + mSplitter->editorExists( node->asType() ) ) { codeEditor = node->asType(); if ( ( codeEditor->getDocument().isLoading() || codeEditor->getDocument().hasFilepath() || !codeEditor->getDocument().isEmpty() ) && diff --git a/src/tools/ecode/plugins/lsp/lspclientserver.cpp b/src/tools/ecode/plugins/lsp/lspclientserver.cpp index 22c7f48ce..b06e518af 100644 --- a/src/tools/ecode/plugins/lsp/lspclientserver.cpp +++ b/src/tools/ecode/plugins/lsp/lspclientserver.cpp @@ -185,7 +185,7 @@ static json toJson( const std::vector& l ) { return result; } -static json toJson( const std::vector& changes ) { +static json toJson( const DocumentContentChanges& changes ) { json result; for ( const auto& change : changes ) { result.push_back( @@ -1689,14 +1689,14 @@ LSPClientServer::LSPRequestHandle LSPClientServer::didSave( TextDocument* doc ) LSPClientServer::LSPRequestHandle LSPClientServer::didChange( const URI& document, int version, const std::string& text, - const std::vector& change ) { + const DocumentContentChanges& change ) { auto params = textDocumentParams( document, version ); params["contentChanges"] = !text.empty() ? json{ json{ MEMBER_TEXT, text } } : toJson( change ); return send( newRequest( "textDocument/didChange", params ) ); } LSPClientServer::LSPRequestHandle -LSPClientServer::didChange( TextDocument* doc, const std::vector& change ) { +LSPClientServer::didChange( TextDocument* doc, const DocumentContentChanges& change ) { Lock l( mClientsMutex ); auto it = mClients.find( doc ); if ( it != mClients.end() ) @@ -1706,11 +1706,11 @@ LSPClientServer::didChange( TextDocument* doc, const std::vector& change ) { + DocumentContentChanges&& change ) { bool shouldStartWorker = false; { Lock l( mDidChangeMutex ); - mDidChangeQueue.push( { document, version, change } ); + mDidChangeQueue.push( { document, version, std::move( change ) } ); if ( !mIsProcessingQueue ) { mIsProcessingQueue = true; shouldStartWorker = true; @@ -1731,7 +1731,7 @@ void LSPClientServer::processDidChangeQueue() { mIsProcessingQueue = false; break; } - change = mDidChangeQueue.front(); + change = std::move( mDidChangeQueue.front() ); mDidChangeQueue.pop(); } // Process outside the lock to avoid blocking diff --git a/src/tools/ecode/plugins/lsp/lspclientserver.hpp b/src/tools/ecode/plugins/lsp/lspclientserver.hpp index 66e50a15d..73368bfda 100644 --- a/src/tools/ecode/plugins/lsp/lspclientserver.hpp +++ b/src/tools/ecode/plugins/lsp/lspclientserver.hpp @@ -6,6 +6,7 @@ #include "lspdocumentclient.hpp" #include "lspprotocol.hpp" #include +#include #include #include #include @@ -24,6 +25,10 @@ using namespace EE::UI::Doc; namespace ecode { +/** didChange content changes. Edits arrive one at a time, so a single change stays inline and + * queueing it allocates no vector storage. */ +using DocumentContentChanges = SmallVector; + class LSPClientServerManager; class LSPClientServer { @@ -125,13 +130,11 @@ class LSPClientServer { LSPRequestHandle didClose( const URI& document ); LSPRequestHandle didChange( const URI& document, int version, const std::string& text, - const std::vector& change = {} ); + const DocumentContentChanges& change = {} ); - LSPRequestHandle didChange( TextDocument* doc, - const std::vector& change = {} ); + LSPRequestHandle didChange( TextDocument* doc, const DocumentContentChanges& change = {} ); - void queueAndProcess( const URI& document, int version, - const std::vector& change = {} ); + void queueAndProcess( const URI& document, int version, DocumentContentChanges&& change ); void processDidChangeQueue(); @@ -306,7 +309,7 @@ class LSPClientServer { struct DidChangeQueue { URI uri; IdType version; - std::vector change; + DocumentContentChanges change; }; std::queue mDidChangeQueue; Mutex mDidChangeMutex; diff --git a/src/tools/ecode/plugins/lsp/lspdocumentclient.cpp b/src/tools/ecode/plugins/lsp/lspdocumentclient.cpp index 15ab8398c..b01261631 100644 --- a/src/tools/ecode/plugins/lsp/lspdocumentclient.cpp +++ b/src/tools/ecode/plugins/lsp/lspdocumentclient.cpp @@ -84,7 +84,9 @@ void LSPDocumentClient::onDocumentTextChanged( const DocumentContentChange& chan ++mVersion; // If several change event are being fired, the thread pool can't guaranteed that it will be // executed in FIFO. Se we accumulate the events in a queue and fire them in correct order. - mServer->queueAndProcess( mDoc->getURI(), mVersion, { change } ); + DocumentContentChanges changes; + changes.push_back( change ); + mServer->queueAndProcess( mDoc->getURI(), mVersion, std::move( changes ) ); requestSymbolsDelayed(); requestSemanticHighlightingDelayed(); }