Add accessible text, relations and live regions; keep dormant a11y free

Text:
- Ranged text access in code points (length, range, line bounds) that
  never copies more than requested, plus a text revision (document UUID
  and modification) so UIA and macOS cache their UTF-16 copies.
- Report document edits as exact change records, up to four per widget
  and frame, then collapse to one whole-text change. Resets, loads and
  document swaps report a whole-text change.
- The removed text is available only while an edit is being notified
  (TextDocument::getNotifiedRemovedText()); DocumentContentChange stays
  { range, text }, so queued records (LSP) copy nothing extra.
- Suppressed text changes are keyed by node ref, so a document shared
  with another window still reports its edits.
- UITextView reports NameChanged; UITextInput overrides onTextChanged()
  to report ValueChanged.

Relations, live regions and shortcuts:
- aria-labelledby, aria-describedby and aria-live (API and CSS). Ids a
  client has read are remembered; renaming, adding or removing their
  widgets notifies the dependents. setId() copies the old id only when
  it is remembered.
- Live regions and UISceneNode::announceForAccessibility(). A newer
  change from the same widget replaces its pending announcement, and
  hidden sources are dropped.
- Menu item shortcuts (std::string text and key) exposed as AT-SPI key
  bindings, the UIA accelerator key and macOS command characters.
- ScrollTo for anything inside a scroll view
  (UIScrollView::scrollIntoView()). Icon-only buttons are named from
  their tooltip.

Dormant path (no client attached):
- Widget notifications are inline: one relaxed atomic load of a
  process-wide count of active root scenes, no call.
- Scene destruction and scene or host changes never create a manager.
  Without a client, notify() queues nothing and widget deletion does
  not register parents to announce it. While only the root has an
  identity, deletion skips the ancestor walk unless a remembered
  relation target may be affected.
- When the last client disconnects, model-view sources release their
  queried rows. Persistent handles count their registrations, so
  handles held elsewhere survive (PersistentModelIndex::release()).
- Spin boxes no longer hide their input explicitly, and
  AccessibilityProperties shrinks from 144 to 136 bytes.

Backends:
- AT-SPI: object:announcement, word and line granularity, ranged
  reads, SetTextContents reported as its minimal diff. Watch
  disconnects per client (NameOwnerChanged arg0 plus a non-blocking
  NameHasOwner check) instead of for every bus name. Without a scene
  thread pool, initialize on a detached thread that shares ownership,
  so teardown never waits on D-Bus timeouts.
- UIA: notification events (resolved at runtime, Windows 10 1709+),
  text ranges served from the revision cache, and a per-frame provider
  drain that skips the lock when nothing is queued.
- macOS: announcements, a UTF-16 text cache per revision, and command
  key attributes.
- EEPP_ACCESSIBILITY_TRACE=1 logs client activity transitions.

Fixes:
- UICodeEditor::isType() no longer dispatches to a subclass's
  getType(), so a UITextEdit is a code editor again.
- ecode: dropped files open only in document editors, not in other
  code editors such as a commit message box.

Tools, tests and docs:
- Widget inspector: role names, shortcut, relation and live rows, and
  an accessibility audit (unnamed controls, broken relations).
- Accessibility example: live status, an icon-only button, and edit and
  reset actions that test_atspi.py checks end to end.
- Regression tests for text records, relations, live regions,
  disconnect and reconnect, scene lifecycle and id changes.
- Add the AccessibilityDormant benchmark (C++ new/new[] counted on the
  measuring thread) and benchmark_accessibility_dormant.py, which fails
  on missing measurements and compares against other builds with
  --compare LABEL=PATH.
- Document the dormant-path rules, what survives a disconnection, and
  the remaining platform limitations.
This commit is contained in:
Martín Lucas Golini
2026-10-10 02:01:52 -03:00
parent 1dd0870187
commit 67660d4c13
55 changed files with 4118 additions and 420 deletions
@@ -0,0 +1,164 @@
#!/usr/bin/env python3
"""Dormant-path accessibility benchmark: allocations, bytes and time with no client attached.
Runs the AccessibilityDormant benchmarks from eepp-benchmarks (a Release build) several times and
reports the median of each measurement. Each --compare LABEL=PATH runs the same benchmark source
built against another tree as well (for example an exported develop or merge-base revision,
compiled with -DEE_BENCH_NO_ACCESSIBILITY), and prints the current build's differences from it.
Allocation counts and bytes cover ordinary C++ new/new[] on the measuring thread only (not malloc,
other threads or background work) and are deterministic; times are indicative only. A run passes
only when every sample exits cleanly and reports every expected measurement, guarded scenarios
included. A compared build whose run fails still reports its measurements.
"""
import argparse
import json
import re
import statistics
import subprocess
import sys
LINE = re.compile(r"^ACCESSIBILITY_DORMANT (\S+) ?(.*)$")
FIELD = re.compile(r"(\w+)=([-\d.]+)")
# Every build must report these; a run missing any of them did not measure what it claims.
COMMON = {
"typing": ("allocations", "bytes", "ms"),
"deleting_characters": ("allocations", "bytes", "ms"),
"large_deletion": ("allocations", "bytes", "ms"),
"large_deletion_removed_text_bytes": ("value", "allocated_bytes_per_removed_byte"),
"spinbox_construction": ("allocations", "bytes", "ms"),
"spinbox_destruction": ("allocations", "bytes", "ms"),
"deep_hierarchy_lifecycle": ("allocations", "bytes", "ms"),
"nested_scene_lifecycle": ("allocations", "bytes", "ms"),
"list_scroll_updates": ("allocations", "bytes", "ms"),
"list_model_mutations": ("allocations", "bytes", "ms"),
"widget_notifications": ("allocations", "bytes", "ms"),
}
# Only builds with accessibility support (not EE_BENCH_NO_ACCESSIBILITY) report these, including
# the guarded after-disconnect and unreferenced id scenarios.
ACCESSIBILITY = {
"first_update_without_thread_pool": ("allocations", "bytes", "ms"),
"later_update_without_thread_pool": ("allocations", "bytes", "ms"),
"readiness_ms": ("value", "backend_available"),
"model_mutations_never_queried": ("allocations", "bytes", "ms"),
"model_mutations_after_disconnect": ("allocations", "bytes", "ms"),
"set_id_no_client": ("allocations", "bytes", "ms"),
"set_id_active_unreferenced": ("allocations", "bytes", "ms"),
"set_id_active_referenced": ("allocations", "bytes", "ms"),
}
def parse(stdout):
sample = {}
for line in stdout.splitlines():
match = LINE.match(line)
if not match:
continue
name, rest = match.groups()
fields = {}
# Single-value lines ("name=value ...") carry the measurement in the name itself.
if "=" in name:
name, value = name.split("=", 1)
fields["value"] = float(value)
for key, value in FIELD.findall(rest):
fields[key] = float(value)
sample[name] = fields
return sample
def missing(sample, expected):
return sorted(
f"{name}.{key}" if name in sample else name
for name, keys in expected.items()
for key in (keys if name in sample else keys[:1])
if name not in sample or key not in sample[name]
)
def run(executable, samples, timeout, expected):
"""Returns the median of every measurement and the reasons the run did not pass, if any."""
results = {}
failures = []
for index in range(samples):
completed = subprocess.run(
[executable, "--filter=AccessibilityDormant.*"],
capture_output=True,
text=True,
timeout=timeout,
)
if completed.returncode != 0:
failures.append(f"sample {index + 1}: exit code {completed.returncode}")
sample = parse(completed.stdout)
absent = missing(sample, expected)
if absent:
failures.append(f"sample {index + 1}: missing {', '.join(absent)}")
for name, fields in sample.items():
for key, value in fields.items():
results.setdefault(name, {}).setdefault(key, []).append(value)
medians = {
name: {key: statistics.median(values) for key, values in fields.items()}
for name, fields in results.items()
}
return medians, failures
def main():
parser = argparse.ArgumentParser(description=__doc__.splitlines()[0])
parser.add_argument("--executable", default="bin/benchmarks/eepp-benchmarks")
parser.add_argument("--compare", action="append", default=[], metavar="LABEL=PATH",
help="the same benchmark built against another tree")
parser.add_argument("--samples", type=int, default=3)
parser.add_argument("--timeout", type=float, default=300)
parser.add_argument("--json", action="store_true")
args = parser.parse_args()
if args.samples < 1:
parser.error("--samples must be at least 1")
builds = [("current", args.executable)]
for entry in args.compare:
label, _, path = entry.partition("=")
if not label or not path:
parser.error(f"--compare expects LABEL=PATH, got {entry!r}")
if any(label == existing for existing, _ in builds):
# Results are keyed by label: a repeated one would replace another build's results.
reason = "is reserved for --executable" if label == "current" else "is already used"
parser.error(f"--compare label {label!r} {reason}")
builds.append((label, path))
output = {}
failures = {}
for label, path in builds:
# The current build must measure everything; a compared tree may lack accessibility.
expected = {**COMMON, **ACCESSIBILITY} if label == "current" else COMMON
output[label], failures[label] = run(path, args.samples, args.timeout, expected)
passed = {label: not failures[label] for label in failures}
if args.json:
print(json.dumps({"results": output, "guards_passed": passed, "failures": failures},
indent=1, sort_keys=True))
sys.exit(0 if passed["current"] else 1)
current = output["current"]
labels = [label for label, _ in builds]
for key in ("allocations", "bytes", "ms"):
print(f"\n{key} (median of {args.samples})")
print(f"{'measurement':42}" + "".join(f" {label:>16}" for label in labels))
for name in sorted(current):
if key not in current[name]:
continue
row = f"{name:42}"
for label in labels:
value = output[label].get(name, {}).get(key)
if value is None:
row += f" {'-':>16}"
else:
row += f" {value:>16.3f}" if key == "ms" else f" {value:>16.0f}"
print(row)
for label in labels:
for failure in failures[label]:
print(f"\n{label}: {failure} (measurements above are still reported)")
sys.exit(0 if passed["current"] else 1)
if __name__ == "__main__":
main()
+57 -2
View File
@@ -90,7 +90,7 @@ def start_discovery_listener():
text_events = []
def on_children_changed(event):
if event.type.startswith("object:text-"):
if event.type.startswith("object:text-") or event.type == "object:announcement":
text_events.append((event.type, event.detail1, event.detail2, event.any_data))
return
application = event.any_data
@@ -105,7 +105,8 @@ def start_discovery_listener():
listener.register("object:children-changed:add"),
"could not register the AT-SPI application discovery listener",
)
for event in ("object:text-changed", "object:text-caret-moved", "object:text-selection-changed"):
for event in ("object:text-changed", "object:text-caret-moved", "object:text-selection-changed",
"object:announcement"):
check(listener.register(event), f"could not register {event}")
return listener, discovered_process_ids, text_events
@@ -201,6 +202,44 @@ def validate(application, multi_window=False, close_primary=False, process_id=No
line = Atspi.Text.get_string_at_offset(description, 8, Atspi.TextGranularity.LINE)
check((line.content, line.start_offset, line.end_offset) == ("second\n", 6, 13),
f"line granularity returned the wrong range: {line}")
last_line = Atspi.Text.get_string_at_offset(description, 15, Atspi.TextGranularity.LINE)
check((last_line.content, last_line.start_offset, last_line.end_offset) == ("third", 13, 18),
f"last line must exclude the final newline: {last_line}")
word = Atspi.Text.get_string_at_offset(description, 8, Atspi.TextGranularity.WORD)
check((word.content, word.start_offset, word.end_offset) == ("second", 6, 12),
f"word granularity returned the wrong range: {word}")
check(description.get_character_count() == 18, "multiline character count must equal 18")
check(Atspi.Text.get_character_at_offset(description, 6) == ord("s"),
"character at offset returned the wrong character")
if text_events is not None:
# An application edit arrives as one exact insertion, not as a diff of the whole text.
append = find_named(nodes, "Append description line")
del text_events[:]
check(append.do_action(action_index(append, "click")), "append action failed")
def appended():
return [event for event in text_events if event[0] == "object:text-changed:insert"]
deadline = time.monotonic() + 2
while time.monotonic() < deadline and not appended():
pause(0.01)
check(appended() == [("object:text-changed:insert", 18, 9, "\nappended")],
f"application edit was not reported exactly: {text_events}")
check(Atspi.Text.get_text(description, 18, -1) == "\nappended", "appended text query failed")
# Replacing a field's text reports the old contents removed and the new ones inserted.
reset = find_named(nodes, "Reset project name")
del text_events[:]
check(reset.do_action(action_index(reset, "click")), "reset action failed")
def replaced():
return [(event[0], event[1], event[2]) for event in text_events
if event[0].startswith("object:text-changed")]
deadline = time.monotonic() + 2
while time.monotonic() < deadline and len(replaced()) < 2:
pause(0.01)
check(replaced() == [("object:text-changed:delete", 0, 5),
("object:text-changed:insert", 0, 4)],
f"text replacement was not reported as delete and insert: {text_events}")
check(Atspi.Text.get_text(project_name, 0, -1) == "eepp", "reset text query failed")
checkbox = find_named(nodes, "Enable autosave")
check(checkbox.get_role_name() == "check box", "autosave must be a check box")
@@ -239,6 +278,22 @@ def validate(application, multi_window=False, close_primary=False, process_id=No
find_named(nodes, "Colors")
find_named(nodes, "Projects")
# An icon-only button is named by its tooltip; the status line it updates is a live region.
refresh = find_named(nodes, "Refresh status")
check(refresh.get_role() == Atspi.Role.PUSH_BUTTON,
"icon-only button must be named by its tooltip")
check(refresh.do_action(action_index(refresh, "click")), "refresh action failed")
if text_events is not None:
def announced():
return any(event[0] == "object:announcement" and event[3] == "Status refreshed"
for event in text_events)
deadline = time.monotonic() + 2
while time.monotonic() < deadline and not announced():
pause(0.01)
check(announced(), "live region change was not announced: "
f"{[event for event in text_events if event[0] == 'object:announcement']}")
if multi_window:
find_named(nodes, "Secondary accessibility window")
open_window = find_named(nodes, "Open accessibility window")