Add accessible text, relations and live regions; keep dormant a11y free

Text:
- Ranged text access in code points (length, range, line bounds) that
  never copies more than requested, plus a text revision (document UUID
  and modification) so UIA and macOS cache their UTF-16 copies.
- Report document edits as exact change records, up to four per widget
  and frame, then collapse to one whole-text change. Resets, loads and
  document swaps report a whole-text change.
- The removed text is available only while an edit is being notified
  (TextDocument::getNotifiedRemovedText()); DocumentContentChange stays
  { range, text }, so queued records (LSP) copy nothing extra.
- Suppressed text changes are keyed by node ref, so a document shared
  with another window still reports its edits.
- UITextView reports NameChanged; UITextInput overrides onTextChanged()
  to report ValueChanged.

Relations, live regions and shortcuts:
- aria-labelledby, aria-describedby and aria-live (API and CSS). Ids a
  client has read are remembered; renaming, adding or removing their
  widgets notifies the dependents. setId() copies the old id only when
  it is remembered.
- Live regions and UISceneNode::announceForAccessibility(). A newer
  change from the same widget replaces its pending announcement, and
  hidden sources are dropped.
- Menu item shortcuts (std::string text and key) exposed as AT-SPI key
  bindings, the UIA accelerator key and macOS command characters.
- ScrollTo for anything inside a scroll view
  (UIScrollView::scrollIntoView()). Icon-only buttons are named from
  their tooltip.

Dormant path (no client attached):
- Widget notifications are inline: one relaxed atomic load of a
  process-wide count of active root scenes, no call.
- Scene destruction and scene or host changes never create a manager.
  Without a client, notify() queues nothing and widget deletion does
  not register parents to announce it. While only the root has an
  identity, deletion skips the ancestor walk unless a remembered
  relation target may be affected.
- When the last client disconnects, model-view sources release their
  queried rows. Persistent handles count their registrations, so
  handles held elsewhere survive (PersistentModelIndex::release()).
- Spin boxes no longer hide their input explicitly, and
  AccessibilityProperties shrinks from 144 to 136 bytes.

Backends:
- AT-SPI: object:announcement, word and line granularity, ranged
  reads, SetTextContents reported as its minimal diff. Watch
  disconnects per client (NameOwnerChanged arg0 plus a non-blocking
  NameHasOwner check) instead of for every bus name. Without a scene
  thread pool, initialize on a detached thread that shares ownership,
  so teardown never waits on D-Bus timeouts.
- UIA: notification events (resolved at runtime, Windows 10 1709+),
  text ranges served from the revision cache, and a per-frame provider
  drain that skips the lock when nothing is queued.
- macOS: announcements, a UTF-16 text cache per revision, and command
  key attributes.
- EEPP_ACCESSIBILITY_TRACE=1 logs client activity transitions.

Fixes:
- UICodeEditor::isType() no longer dispatches to a subclass's
  getType(), so a UITextEdit is a code editor again.
- ecode: dropped files open only in document editors, not in other
  code editors such as a commit message box.

Tools, tests and docs:
- Widget inspector: role names, shortcut, relation and live rows, and
  an accessibility audit (unnamed controls, broken relations).
- Accessibility example: live status, an icon-only button, and edit and
  reset actions that test_atspi.py checks end to end.
- Regression tests for text records, relations, live regions,
  disconnect and reconnect, scene lifecycle and id changes.
- Add the AccessibilityDormant benchmark (C++ new/new[] counted on the
  measuring thread) and benchmark_accessibility_dormant.py, which fails
  on missing measurements and compares against other builds with
  --compare LABEL=PATH.
- Document the dormant-path rules, what survives a disconnection, and
  the remaining platform limitations.
This commit is contained in:
Martín Lucas Golini
2026-10-10 02:01:52 -03:00
parent 1dd0870187
commit 67660d4c13
55 changed files with 4118 additions and 420 deletions
+4
View File
@@ -31,6 +31,10 @@ class EE_API UUID {
bool isInitialized() const;
uint64_t high() const { return mHigh; }
uint64_t low() const { return mLow; }
bool operator==( const UUID& other ) const {
return mHigh == other.mHigh && mLow == other.mLow;
}
@@ -101,6 +101,10 @@ constexpr AccessibilityActions accessibilityActionMask( AccessibilityAction acti
return 1u << static_cast<Uint32>( action );
}
/** Politeness of an announcement or live region: Polite waits for the client to finish speaking,
* Assertive interrupts it. */
enum class AccessibilityLive : Uint8 { Off, Polite, Assertive };
enum class AccessibilityEvent : Uint8 {
Created,
Destroyed,
@@ -117,6 +121,20 @@ enum class AccessibilityEvent : Uint8 {
BoundsChanged,
};
/** A keyboard shortcut, both as displayed and decomposed for native APIs that want their own
* format (GTK accelerators on AT-SPI, command characters on macOS). */
struct AccessibilityShortcut {
enum Modifier : Uint8 { Control = 1 << 0, Shift = 1 << 1, Alt = 1 << 2, Meta = 1 << 3 };
/** As displayed to the user (UTF-8), for example "Ctrl+S". */
std::string text;
/** The platform key name (UTF-8), for example "S", "F5" or "Return". */
std::string key;
Uint8 modifiers{ 0 };
bool empty() const { return text.empty(); }
};
struct AccessibilityNodeRef {
AccessibilitySourceId source{ 0 };
Uint64 id{ 0 };
@@ -145,11 +163,41 @@ struct AccessibilityTextInfo {
bool valid{ false };
};
/** Identifies one version of an element's text. Equal revisions have equal text, so backends can
* cache converted copies (UTF-16 for UIA and AppKit) instead of rebuilding them per query. */
struct AccessibilityTextRevision {
/** The document's UUID halves: unlike its address, never reused by another document. */
Uint64 documentHigh{ 0 };
Uint64 documentLow{ 0 };
Uint64 modification{ 0 };
bool isValid() const { return documentHigh != 0 || documentLow != 0; }
bool operator==( const AccessibilityTextRevision& other ) const {
return documentHigh == other.documentHigh && documentLow == other.documentLow &&
modification == other.modification;
}
bool operator!=( const AccessibilityTextRevision& other ) const { return !( *this == other ); }
};
/** One edit, in code points: at `offset`, `removed` was replaced by `inserted`. An offset of -1
* means the whole text was replaced (a reset, a reload, or too many edits in one frame). */
struct AccessibilityTextChange {
Int32 offset{ -1 };
String removed;
String inserted;
bool isWholeText() const { return offset < 0; }
};
struct AccessibilityNodeInfo {
AccessibilityRole role{ AccessibilityRole::None };
String name;
String description;
String value;
/** Keyboard shortcut that activates the element. */
AccessibilityShortcut shortcut;
AccessibilityState states{ AccessibilityState::None };
AccessibilityActions actions{ 0 };
AccessibilityRangeInfo range;
@@ -158,6 +206,14 @@ struct AccessibilityNodeInfo {
bool boundsValid{ false };
};
struct AccessibilityAnnouncement {
String message;
AccessibilityLive priority{ AccessibilityLive::Polite };
/** The live-region widget that produced it; invalid for an explicit announcement. A newer
* change from the same widget replaces its pending announcement. */
AccessibilityNodeRef source;
};
struct AccessibilityActionRequest {
AccessibilityAction action{ AccessibilityAction::Focus };
String value;
@@ -9,6 +9,10 @@ namespace EE { namespace Scene {
class Node;
}} // namespace EE::Scene
namespace EE { namespace UI { namespace Doc {
struct DocumentContentChange;
}}} // namespace EE::UI::Doc
namespace EE { namespace UI {
class UISceneNode;
@@ -41,6 +45,29 @@ class EE_API AccessibilityManager {
AccessibilityTextInfo getTextInfo( AccessibilityNodeRef ref ) const;
/** Ranged text access in code points. Prefer these over getNodeInfo()'s value for text
* elements: they never copy more than the requested range. */
Int32 getTextLength( AccessibilityNodeRef ref ) const;
String getTextRange( AccessibilityNodeRef ref, Int32 start, Int32 end ) const;
/** The line containing `offset`: [start, end), end past its newline. */
bool getTextLineBounds( AccessibilityNodeRef ref, Int32 offset, Int32& start,
Int32& end ) const;
AccessibilityTextRevision getTextRevision( AccessibilityNodeRef ref ) const;
/** Reports an edit of a text widget's document, or nullptr when its whole text was replaced.
* The first few edits per widget and frame are forwarded exactly; later ones collapse into a
* single whole-text change, so a replace-all costs one notification. */
void onTextChanged( UIWidget* widget, const Doc::DocumentContentChange* change );
/** While set, text changes of this element are not forwarded: the caller reports them itself
* (AT-SPI SetTextContents sends one minimal diff). Node refs are local to this manager, so
* another window's element sharing the document still reports its changes. Pass an invalid
* ref to clear. */
void setSuppressedTextChanges( AccessibilityNodeRef ref );
size_t getChildCount( AccessibilityNodeRef ref );
AccessibilityNodeRef getChild( AccessibilityNodeRef ref, size_t index );
@@ -64,8 +91,17 @@ class EE_API AccessibilityManager {
/** Updates the scene's cached fast-path flag after an actual native accessibility query. */
void onNativeClientObserved();
/** The scene's last client went away: releases what only clients needed, such as queried
* model rows and their persistent model registrations. Identities are never reused, so a
* reconnecting client cannot reach a stale row. */
void onClientsDisconnected();
void update();
/** Replaces the platform backend, which update() otherwise creates on first use. Intended
* for tests that observe what the manager delivers. */
void setBackend( std::unique_ptr<AccessibilityBackend> backend );
UISceneNode* getSceneNode() const;
void notify( AccessibilityNodeRef ref, AccessibilityEvent event,
@@ -75,6 +111,30 @@ class EE_API AccessibilityManager {
void clearPendingEvents();
/** Asks assistive clients to speak a message without moving focus. Delivered on the next
* update(); a no-op while no client is active. */
void announce( const String& message, AccessibilityLive priority = AccessibilityLive::Polite );
/** Queues the current text of a widget inside a live region, replacing any announcement the
* same widget queued earlier in this frame. */
void onLiveRegionChanged( UIWidget* widget, AccessibilityLive priority );
const std::vector<AccessibilityAnnouncement>& getPendingAnnouncements() const;
/** The widget with this id changed its name, appeared or went away. Widgets that a client
* read and that are named or described by it (aria-labelledby, aria-describedby) report
* NameChanged or DescriptionChanged. Dependents inside `excluded` are skipped. */
void onRelationTargetChanged( const std::string& id, const Scene::Node* excluded = nullptr );
/** Whether an active client may hold a name or description resolved through the id with this
* hash, so its dependents must be told when that id goes away. Allocation-free; it matches
* exactly what onRelationTargetChanged() would act on. */
bool isRelationTarget( String::HashType idHash ) const;
/** A subtree appeared or is going away: notifies the dependents of each relation target in
* it, except those inside the subtree when it is going away. */
void onRelationSubtreeChanged( const Scene::Node* subtree, bool removed );
void onWidgetParentChange( UIWidget* widget );
void onWidgetRemovedFromParent( UIWidget* widget );
@@ -98,6 +158,15 @@ class EE_API AccessibilityManager {
UnorderedMap<AccessibilitySourceId, std::unique_ptr<AccessibilitySource>> mSources;
UnorderedMap<UIWidget*, AccessibilitySourceId> mWidgetSources;
std::vector<AccessibilityPendingEvent> mPendingEvents;
std::vector<AccessibilityAnnouncement> mPendingAnnouncements;
/** Edits forwarded this frame per text element; reset by update(). */
std::vector<std::pair<AccessibilityNodeRef, Uint32>> mTextChangeCounts;
static constexpr Uint32 MaxExactTextChangesPerFrame = 4;
AccessibilityNodeRef mSuppressedTextRef;
/** Ids referenced by aria-labelledby / aria-describedby of widgets a client has read. Only
* changes to these widgets can make a client's cached name or description stale. Ids, not
* widget pointers: entries never dangle, and a stale one costs one scene scan. */
mutable UnorderedSet<String::HashType> mRelationTargets;
struct ChildrenCacheEntry {
std::vector<AccessibilityNodeRef> children;
AccessibilityNodeRef parent;
@@ -117,6 +186,9 @@ class EE_API AccessibilityManager {
/** Returns whether any identity or model source in the subtree was released. */
bool removeSubtreeIdentities( Scene::Node* node );
/** Whether a client can receive events: without one, only invalidation runs. */
bool hasActiveClients() const;
};
}} // namespace EE::UI
@@ -7,21 +7,48 @@ namespace EE { namespace Scene {
class Node;
}} // namespace EE::Scene
namespace EE { namespace UI { namespace Doc {
class TextDocument;
class TextPosition;
}}} // namespace EE::UI::Doc
namespace EE { namespace UI {
class UIWidget;
struct AccessibilityIssue {
const UIWidget* widget{ nullptr };
String message;
};
/** Resolves standard widget semantics directly from the live widget instance. */
class EE_API AccessibilityWidgetResolver {
public:
static AccessibilityRole getRole( const UIWidget* widget );
/** A readable role name for tools and logs, for example "CheckBox". */
static const char* getRoleName( AccessibilityRole role );
/** Controls whose content is fully described by their own name, value and state. Their
* implementation children (a button's text view, a menu item's shortcut label) are not exposed.
*/
static bool isLeafRole( AccessibilityRole role );
/** The outermost leaf control containing the node, or nullptr when no ancestor is a leaf. */
static UIWidget* getLeafOwner( const Scene::Node* node );
/** Whether the widget or any ancestor is hidden from accessibility (aria-hidden). */
static bool isHiddenFromAccessibility( const UIWidget* widget );
static String getName( const UIWidget* widget );
static String getDescription( const UIWidget* widget );
static String getValue( const UIWidget* widget );
/** The keyboard shortcut of a menu item; empty for every other widget. */
static AccessibilityShortcut getShortcut( const UIWidget* widget );
static AccessibilityRangeInfo getRange( const UIWidget* widget );
static AccessibilityTextInfo getText( const UIWidget* widget );
@@ -29,6 +56,24 @@ class EE_API AccessibilityWidgetResolver {
/** Metadata-only queries avoid scanning document lines to calculate offsets. */
static AccessibilityTextInfo getText( const UIWidget* widget, bool includeOffsets );
/** The document behind a text element; nullptr for other widgets and for password inputs,
* whose text is never exposed. */
static const Doc::TextDocument* getTextDocument( const UIWidget* widget );
/** Ranged text access, in code points. None of these copy more than the requested range. */
static Int32 getTextLength( const UIWidget* widget );
static String getTextRange( const UIWidget* widget, Int32 start, Int32 end );
/** The line containing `offset`: [start, end), end past its newline. */
static bool getTextLineBounds( const UIWidget* widget, Int32 offset, Int32& start, Int32& end );
static AccessibilityTextRevision getTextRevision( const UIWidget* widget );
/** The code-point offset of a document position. */
static Int32 getTextOffset( const Doc::TextDocument& document,
const Doc::TextPosition& position );
static AccessibilityState getState( const UIWidget* widget );
static AccessibilityActions getActions( const UIWidget* widget );
@@ -45,6 +90,16 @@ class EE_API AccessibilityWidgetResolver {
static UIWidget* getEventTarget( UIWidget* widget, AccessibilityEvent event );
static bool performAction( UIWidget* widget, const AccessibilityActionRequest& request );
/** Lists common authoring problems in a widget tree: controls without an accessible name and
* aria-labelledby / aria-describedby ids that resolve to nothing. Hidden and invisible
* subtrees are skipped, as are the implementation children of model views. */
static std::vector<AccessibilityIssue> audit( const UIWidget* root );
private:
/** The widget's own name: its label, else its native text, else its tooltip. Relations are
* not followed, so a labelled-by target that is itself labelled by another cannot loop. */
static String getOwnName( const UIWidget* widget );
};
}} // namespace EE::UI
+3
View File
@@ -59,6 +59,9 @@ enum class PropertyId : Uint16 {
AccessibilityLabel,
AccessibilityDescription,
AccessibilityHidden,
AccessibilityLabelledBy,
AccessibilityDescribedBy,
AccessibilityLive,
LayoutWeight,
LayoutGravity,
LayoutWidth,
+9 -1
View File
@@ -775,6 +775,12 @@ class EE_API TextDocument {
std::string toUtf8String();
/** The text removed by the edit being notified: valid only during
* Client::onDocumentTextChanged(), and empty for insertions. Clients that need it later must
* copy it. Change records do not carry it, so consumers that queue them (such as LSP clients)
* never copy text they do not use. */
const String& getNotifiedRemovedText() const;
protected:
friend class TextUndoStack;
friend class FoldRangeService;
@@ -794,6 +800,8 @@ class EE_API TextDocument {
Mutex mClientsMutex;
mutable Mutex mLinesMutex;
mutable std::shared_ptr<Mutex> mDocumentMutex;
/** The removed text of the change being notified; see getNotifiedRemovedText(). */
const String* mNotifiedRemovedText{ nullptr };
TextFormat::Encoding mEncoding{ TextFormat::Encoding::UTF8 };
TextFormat::LineEnding mLineEnding{ TextFormat::LineEnding::LF };
std::atomic<bool> mLoading{ false };
@@ -855,7 +863,7 @@ class EE_API TextDocument {
void notifyDocumentReset();
void notifyTextChanged( const DocumentContentChange& );
void notifyTextChanged( const DocumentContentChange&, const String* removed = nullptr );
void notifyCursorChanged( TextPosition selection = TextPosition() );
+3
View File
@@ -134,6 +134,9 @@ class EE_API Model {
std::weak_ptr<PersistentHandle> registerPersistentIndex( ModelIndex const& );
/** Releases one registration of the handle; the handle is dropped when none remain. */
void releasePersistentIndex( const std::shared_ptr<PersistentHandle>& handle );
void beginInsertRows( ModelIndex const& parent, int first, int last );
void beginInsertColumns( ModelIndex const& parent, int first, int last );
void beginMoveRows( ModelIndex const& sourceParent, int first, int last,
@@ -16,6 +16,9 @@ class PersistentHandle {
PersistentHandle( ModelIndex const& index ) : mIndex( index ) {}
ModelIndex mIndex;
/** PersistentModelIndex constructions that registered this handle and have not released it.
* The model keeps the handle while any registration remains. */
Uint32 mRegistrations{ 0 };
};
class EE_API PersistentModelIndex {
@@ -50,6 +53,12 @@ class EE_API PersistentModelIndex {
bool operator==( ModelIndex const& ) const;
bool operator!=( ModelIndex const& ) const;
/** Gives back the registration made when this index was constructed from a ModelIndex, and
* detaches it. The model drops the handle, and stops updating it on every row operation,
* once all its registrations are released. Indexes that never release keep the handle
* alive: call this at most once per construction, never on a copy. */
void release();
private:
std::weak_ptr<PersistentHandle> mHandle;
};
+14
View File
@@ -526,6 +526,11 @@ class EE_API UISceneNode : public SceneNode {
AccessibilityPolicy getAccessibilityPolicy() const;
/** Asks assistive clients to speak a message (for example "Build finished") without moving
* focus. Costs nothing while no client is active. */
void announceForAccessibility( const String& message,
AccessibilityLive priority = AccessibilityLive::Polite );
bool hasActiveAccessibilityClients() const {
return !( mNodeFlags & NODE_FLAG_CLOSE ) &&
( mHostUISceneNode ? mHostUISceneNode->hasActiveAccessibilityClients()
@@ -1329,6 +1334,15 @@ class EE_API UISceneNode : public SceneNode {
void onWidgetAccessibilitySourceDelete( UIWidget* widget );
/** The manager this scene's widgets report to, or nullptr when none exists yet. Never creates
* one: lifecycle bookkeeping (destruction, scene and host changes) must not initialize
* accessibility. */
AccessibilityManager* getExistingAccessibilityManager() const;
/** Sets this root scene's client-active state and keeps the process-wide count that widget
* notifications test inline in sync with it. */
void setAccessibilityClientActive( bool active );
/**
* @brief Recursively resets tooltips for a node and its children.
*
+5
View File
@@ -57,6 +57,11 @@ class EE_API UIScrollView : public UITouchDraggableWidget {
bool areDefaultKeybindingsEnabled() const { return mDefaultKeybindings; }
/** Scrolls the least distance that makes a descendant of the content fully visible (or its
* top-left corner, when it is larger than the viewport). Returns false when the node is not
* part of this view's content. */
bool scrollIntoView( const Node* node );
protected:
ScrollViewType mViewType;
ScrollBarMode mVScrollMode;
-2
View File
@@ -88,8 +88,6 @@ class EE_API UITextEdit : public UICodeEditor {
const TextPosition& cursor );
virtual void onTextHintsChanged();
virtual void onDocumentSelectionChange( const Doc::TextRange& );
};
}} // namespace EE::UI
+3 -1
View File
@@ -205,6 +205,8 @@ class EE_API UITextInput : public UITextView, public TextDocument::Client {
virtual void onFontColorChanged();
virtual void onTextChanged();
void onThemeLoaded();
virtual void onCursorPosChange();
@@ -245,7 +247,7 @@ class EE_API UITextInput : public UITextView, public TextDocument::Client {
virtual void onDocumentMoved( TextDocument* );
virtual void onDocumentReset( TextDocument* ) {}
virtual void onDocumentReset( TextDocument* ) { notifyAccessibilityTextChanged( nullptr ); }
void onDocumentClosed( TextDocument* ) {};
+55 -2
View File
@@ -2,6 +2,7 @@
#define EE_UIUIWIDGET_HPP
#include <algorithm>
#include <atomic>
#include <eepp/core/small_vector.hpp>
#include <eepp/ui/accessibility/accessibility.hpp>
#include <eepp/ui/css/propertydefinition.hpp>
@@ -26,6 +27,9 @@ class UITooltip;
class UIStyle;
class AccessibilityWidgetResolver;
struct AccessibilityProperties;
namespace Doc {
struct DocumentContentChange;
}
class UIWidget;
class UITextSelectionController;
@@ -108,6 +112,23 @@ class EE_API UIWidget : public UINode {
UIWidget* setAccessibilityHidden( bool hidden );
/** Names this widget after the widget with the given id (aria-labelledby). It takes
* precedence over the accessibility label. The id is resolved from the scene root. */
UIWidget* setAccessibilityLabelledBy( const std::string& id );
const std::string& getAccessibilityLabelledBy() const;
/** Describes this widget with the text of the widget with the given id (aria-describedby). */
UIWidget* setAccessibilityDescribedBy( const std::string& id );
const std::string& getAccessibilityDescribedBy() const;
/** Marks this widget as a live region (aria-live): name and value changes inside it are
* announced to assistive clients without moving focus. */
UIWidget* setAccessibilityLive( AccessibilityLive live );
AccessibilityLive getAccessibilityLive() const;
virtual UITextSelectionController* getTextSelectionController();
virtual const UITextSelectionController* getTextSelectionController() const;
@@ -285,7 +306,7 @@ class EE_API UIWidget : public UINode {
*
* @return The tooltip text string.
*/
String getTooltipText();
String getTooltipText() const;
/**
* @brief Updates the distances to parent borders for anchoring.
@@ -1967,9 +1988,41 @@ class EE_API UIWidget : public UINode {
String resolveAccessibilityDescription() const;
void notifyAccessibilityEvent( AccessibilityEvent event );
/** Whether any scene in the process has an active accessibility client. Notifications test
* it inline, so while no client is attached anywhere they cost one relaxed load and no call.
* Callers that compute an event or payload test it first. */
static bool isAccessibilityActive() {
return sAccessibilityActiveScenes.load( std::memory_order_relaxed ) != 0;
}
void notifyAccessibilityEvent( AccessibilityEvent event ) {
if ( isAccessibilityActive() )
deliverAccessibilityEvent( event );
}
/** Reports an edit of this text widget's document; nullptr when the whole text changed. */
void notifyAccessibilityTextChanged( const Doc::DocumentContentChange* change ) {
if ( isAccessibilityActive() )
deliverAccessibilityTextChanged( change );
}
/** A reset, load or document swap: the whole text changed, and with it the value. */
void notifyAccessibilityWholeTextChanged() {
if ( isAccessibilityActive() )
deliverAccessibilityWholeTextChanged();
}
void detachAccessibilitySource();
private:
/** Root scenes whose accessibility client is active; maintained by UISceneNode. */
static std::atomic<Uint32> sAccessibilityActiveScenes;
void deliverAccessibilityEvent( AccessibilityEvent event );
void deliverAccessibilityTextChanged( const Doc::DocumentContentChange* change );
void deliverAccessibilityWholeTextChanged();
};
}} // namespace EE::UI