diff --git a/README.md b/README.md index 54244d9c4..b2567eeb2 100644 --- a/README.md +++ b/README.md @@ -253,6 +253,8 @@ the most basic widgets in a vertical linear layout display. **UI introduction can be found [here](https://cdn.ensoft.dev/eepp-docs/page_uiintroduction.html)**. **The UI data-binding guide can be found [here](docs/articles/uidatabinding.md).** + +**The runtime UI inspector guide can be found [here](docs/articles/ui_inspector.md).** ## UI Widgets with C++ example diff --git a/docs/articles/ui_inspector.md b/docs/articles/ui_inspector.md new file mode 100644 index 000000000..87a1fcdfe --- /dev/null +++ b/docs/articles/ui_inspector.md @@ -0,0 +1,278 @@ +# Runtime UI inspector + +The eepp inspector is a small TCP interface for inspecting and interacting with a running eepp UI. It is useful for debugging layout, CSS, focus, input delivery, multiple windows, and HTML inside `UIWebView`. Its compact JSON replies and server side batches are designed to reduce tool round trips for automated agents. + +## Security and activation + +The capability can be compiled into every eepp application. **It is disabled by default:** a normal run has no inspector listener, token, network thread, or reachable parser. Enabling it starts an explicit developer debugging interface. In control mode, authenticated clients can click and type into the application. + +The listener binds to `127.0.0.1` by default. Every connection must present a bearer token. This is a safeguard against accidental or unintended attachment, including attachment to the wrong process; it is **not a sandbox boundary against malicious software running as the same OS user**. The interface resembles accessibility or OS automation in its ability to inspect and act, but additionally exposes eepp widget classes, CSS values, geometry, nested scenes, and watches. Availability of accessibility support does not enable the inspector. + +Raw TCP is plaintext, including the token, UI text, and actions. Binding to a non-loopback address prints a warning and should be used only on a trusted network. For remote use, leave the inspector on loopback and forward it with `ssh -L 9876:127.0.0.1: host`. Password input text is returned as `[redacted]` in summaries, inspection, trees, and watches. Screenshots capture visible pixels, which can include sensitive information even when semantic text is redacted. Screenshot files remain in the OS temporary directory until the client or OS removes them. A generated token is printed once at startup; anyone who can read that process output can authenticate. A caller supplied token is not printed. + +Set the following environment variables before launching any normal eepp UI application: + +| Variable | Default | Meaning | +| --- | --- | --- | +| `EEPP_INSPECTOR` | unset | Set to `1` to start after the first top-level UI scene is registered. Unset, empty, or `0` disables it. | +| `EEPP_INSPECTOR_ADDRESS` | `127.0.0.1` | TCP bind address. | +| `EEPP_INSPECTOR_PORT` | `0` | TCP port; `0` asks the OS for an available port. | +| `EEPP_INSPECTOR_READ_ONLY` | `0` | Set to `1` to reject every `input.*` command. | +| `EEPP_INSPECTOR_TOKEN` | generated | Optional caller supplied bearer token. | + +```sh +EEPP_INSPECTOR=1 ./bin/ecode +EEPP_INSPECTOR=1 EEPP_INSPECTOR_PORT=9876 EEPP_INSPECTOR_READ_ONLY=1 ./bin/ecode +EEPP_INSPECTOR=1 EEPP_INSPECTOR_TOKEN="$secret" ./bin/ecode +EEPP_INSPECTOR=1 EEPP_INSPECTOR_ADDRESS=0.0.0.0 ./bin/ecode # trusted network only +``` + +Successful startup writes one compact line to stderr: + +```text +EEPP_INSPECTOR={"protocolVersion":1,"host":"127.0.0.1","port":43187,"token":"","pid":19382} +``` + +With `EEPP_INSPECTOR_TOKEN` supplied, the `token` member is omitted. The prefix `EEPP_INSPECTOR=` is stable. The server accepts up to 16 simultaneous TCP clients, and a request line is limited to 1 MiB. Each client's pending output is limited to 4 MiB. A single query returns at most 1000 nodes; a tree returns at most 1000 nodes; a batch accepts at most 100 commands. + +## Python client + +`projects/scripts/eepp-inspect.py` requires Python 3 and only the standard library. It reads `EEPP_INSPECTOR_HOST`, `EEPP_INSPECTOR_PORT`, and `EEPP_INSPECTOR_TOKEN`, or the corresponding `--host`, `--port`, and `--token` options. Use `--pretty` for indented JSON. Default stdout is compact JSON only; diagnostics go to stderr. Ordinary commands connect, authenticate, print one response, and disconnect. + +```sh +export EEPP_INSPECTOR_PORT=43187 EEPP_INSPECTOR_TOKEN='' +python3 projects/scripts/eepp-inspect.py contexts +python3 projects/scripts/eepp-inspect.py query '#preview' +python3 projects/scripts/eepp-inspect.py query --scene scene:2 'table > tr > td.problem' --properties geometry.size css.font-size +python3 projects/scripts/eepp-inspect.py tree --scene scene:2 --depth 4 +python3 projects/scripts/eepp-inspect.py inspect w:42 geometry.size geometry.position css.font-size +python3 projects/scripts/eepp-inspect.py focus --scene scene:2 +python3 projects/scripts/eepp-inspect.py screenshot --scene scene:2 --format png +python3 projects/scripts/eepp-inspect.py screenshot --window win:2 --rect 10 20 400 300 --format webp +python3 projects/scripts/eepp-inspect.py click --scene scene:2 '#submit' +python3 projects/scripts/eepp-inspect.py key --scene scene:2 Enter +python3 projects/scripts/eepp-inspect.py type --scene scene:2 'hello world' +python3 projects/scripts/eepp-inspect.py type --target '#search' 'hello world' +python3 projects/scripts/eepp-inspect.py watch --scene scene:2 '#problem' geometry.size css.font-size --count 3 --timeout 10 +python3 projects/scripts/eepp-inspect.py batch workflow.json +python3 projects/scripts/eepp-inspect.py batch - < workflow.json +python3 projects/scripts/eepp-inspect.py raw '{"method":"ui.query","params":{"selector":"#foo"}}' +python3 projects/scripts/eepp-inspect.py session +``` + +`type --target` sends `input.click` followed by `input.text`, using the clicked widget's scene. `watch` streams one event per line; `--count` counts `watch.changed` events and `--timeout` bounds elapsed seconds. `batch` reads a JSON object containing `commands` from a file or stdin. `session` authenticates once, then sends one JSON request per stdin line and writes every response or event to stdout. It keeps the connection open while stdin is open. `raw` supplies a normal request object, assigning an ID if absent. + +## Transport and envelopes + +Protocol version 1 uses TCP, UTF-8, and newline delimited JSON: one object followed by `\n`. `\r\n` is accepted. TCP read boundaries have no meaning. Messages are compact, and a client may keep a connection open, pipeline requests, and receive unsolicited events between responses. Correlate responses by their non-negative integer `id`; response order is not guaranteed. A connection's first request must be `session.connect`. + +```json +{"id":1,"method":"session.connect","params":{"protocolVersion":1,"token":"","client":{"name":"my-tool","version":"1"}}} +``` + +Success includes `protocolVersion`, `application`, `pid`, `readOnly`, `defaultWindow`, `defaultScene`, and `capabilities`. An invalid token returns `authentication-failed` and closes the connection; a non-v1 version returns `unsupported-protocol` and closes it. Requests use `{"id":17,"method":"ui.query","params":{...}}`; `params` defaults to `{}`. Success is `{"id":17,"result":{...}}`. Failure is `{"id":17,"error":{"code":"...","message":"...","data":{...}}}`; `data` is optional. Events have no ID: `{"method":"watch.changed","params":{...}}`. + +Every command may have a post-command `delay` such as `"100ms"` or `"1s"`. The command runs first; its response or the next batch command waits asynchronously. The normal UI update loop continues. A single delay is limited to 60 seconds. The accepted syntax is a non-negative decimal number followed by `ms` or `s`. + +## Windows, scenes, and handles + +Handles are opaque: `win:*` identifies a native window, `scene:*` a `UISceneNode`, `w:*` a widget, and `sub:*` a watch subscription. Window, scene, and widget IDs are monotonic for the server's lifetime and valid across client reconnections while the objects live. Destroyed objects invalidate their handles. Subscription IDs belong to one connection and are released on disconnect. Handles never contain pointers and do not survive server restarts. + +Each selector executes in **exactly one scene**. An omitted scene means the stable `defaultScene` from the handshake, never the currently active scene. If that scene is destroyed, omission returns `default-scene-unavailable`. A scene determines its native window; a widget handle determines its scene and window. `ui.contexts` reports all discovered top-level and nested scenes. A `UIWebView` document is a separate scene and is classified `webview`, with its owner widget and parent scene. Other embedded scenes are classified `nested`. + +For example, discover the WebView in its parent scene, then query its document scene: + +```json +{"id":10,"method":"ui.query","params":{"selector":"#preview"}} +``` + +The returned WebView summary includes `"documentScene":"scene:2"`. Then: + +```json +{"id":11,"method":"ui.query","params":{"scene":"scene:2","selector":"table > tr > td.problem"}} +{"id":12,"method":"ui.inspect","params":{"handle":"w:95","properties":["geometry.position","geometry.size","css.display","css.font-size"]}} +``` + +`#preview p` in the parent scene never enters the HTML document. `ui.contexts` also reveals secondary native windows and their scenes; pass the secondary `scene:*` to query or act there. + +## Method reference + +All examples below omit `id` where the surrounding explanation is enough; real requests always require it. + +| Method | Parameters and defaults | Result | Notable errors | +| --- | --- | --- | --- | +| `session.connect` | `protocolVersion:1`, `token` required; optional `client` | Protocol version, application, PID, mode, default handles, capabilities | `authentication-failed`, `unsupported-protocol` | +| `session.batch` | `commands` array, at most 100 | `results` object keyed by command `name` or zero-based index string | `batch-command-failed`, `invalid-params` | +| `session.nextFrame` | `count` default 1, maximum 10 | `{ "frames": count }`; mainly used inside a batch | `invalid-params` | +| `ui.contexts` | none | `revision`, default handles, `windows`, `scenes` | `internal-error` | +| `ui.query` | `selector` required; `scene` default; `offset:0`, `limit:50` (maximum 1000); optional `properties`, `maxStringLength:4096` (maximum 65536) | Scene, selector, total, pagination, truncation, compact `nodes` | `invalid-scene`, `selector-invalid` | +| `ui.tree` | `scene` default or `root` handle; `depth:3` (maximum 32), `maxNodes:200` (maximum 1000), `includeText:false` | Flat nodes with `parent`, `children`, and `truncated` | `invalid-widget`, `invalid-scene` | +| `ui.inspect` | Exactly one of `handle` or `selector` plus optional `scene`; optional `properties`, `maxStringLength:4096` (maximum 65536) | Handle, scene, typed `properties`, `unavailable` | `invalid-widget`, `target-not-found`, `target-ambiguous` | +| `ui.focus` | `scene` default | Scene and focused widget summary or `null` | `invalid-scene` | +| `ui.screenshot` | Optional `scene` or `window` (mutually exclusive); optional `rect:[x,y,width,height]`; `format:"png"` | Temporary file path, format, window/scene handles, captured rectangle and size | `invalid-window`, `invalid-scene`, `invalid-params`, `screenshot-failed` | +| `input.click` | Exactly one target; optional `button:"left"`, `count:1` (maximum 2) | Target handle, scene, click point | `target-not-interactable`, `permission-denied` | +| `input.key` | `scene` default, `key` required, `action:"press"`, `modifiers:[]` | Key and action | `invalid-params`, `permission-denied` | +| `input.text` | `scene` default, `text` required (maximum 64 KiB UTF-8) | Character count | `invalid-params`, `permission-denied` | +| `watch.subscribe` | One handle or selector; explicit `properties` array; `initial:true` | Subscription and bound target values | `target-not-found`, `property-unknown` | +| `watch.unsubscribe` | `subscription` required | `{ "unsubscribed": true }` | `invalid-params` | + +`ui.contexts` window entries include handle, native ID, title, pixel size and position, and focus. Scene entries include handle, window, kind, parent scene, owner widget, and root widget; WebView scenes include URI. The `revision` increases on observed topology changes. Create/destroy events are checked periodically while clients are connected; `ui.contexts` is the authoritative current snapshot. + +These compact request/response pairs show each method's envelope. Handle numbers and geometry are illustrative; methods also accept the optional fields in the table above. Errors use the shared error envelope. + +```json +{"id":1,"method":"session.connect","params":{"protocolVersion":1,"token":"secret"}} +{"id":1,"result":{"protocolVersion":1,"application":"ecode","pid":19382,"readOnly":false,"defaultWindow":"win:1","defaultScene":"scene:1","capabilities":["ui.contexts","ui.query","ui.tree","ui.inspect","ui.focus","ui.screenshot","session.batch","session.nextFrame","watch.properties","events.context-lifecycle","input.click","input.key","input.text"]}} +{"id":2,"method":"ui.contexts"} +{"id":2,"result":{"revision":0,"defaultWindow":"win:1","defaultScene":"scene:1","windows":[{"handle":"win:1","id":1,"title":"ecode","sizePx":[800,600],"positionPx":[0,0],"focused":true}],"scenes":[{"handle":"scene:1","window":"win:1","kind":"top-level","parentScene":null,"owner":null,"root":"w:1"}]}} +{"id":3,"method":"ui.query","params":{"selector":"#panel","properties":["geometry.size"],"limit":1}} +{"id":3,"result":{"scene":"scene:1","selector":"#panel","total":1,"offset":0,"returned":1,"truncated":false,"nodes":[{"handle":"w:2","scene":"scene:1","tag":"widget","id":"panel","classes":[],"pseudoClasses":[],"boundsPx":[10,10,100,30],"visible":true,"enabled":true,"focused":false,"properties":{"geometry.size":[100,30]}}]}} +{"id":4,"method":"ui.tree","params":{"depth":1,"maxNodes":50}} +{"id":4,"result":{"scene":"scene:1","root":"w:1","truncated":false,"nodes":[{"handle":"w:1","scene":"scene:1","tag":"root","id":"","classes":[],"pseudoClasses":[],"parent":null,"children":["w:2"]},{"handle":"w:2","scene":"scene:1","tag":"widget","id":"panel","classes":[],"pseudoClasses":[],"parent":"w:1","children":[]}]}} +{"id":5,"method":"ui.inspect","params":{"handle":"w:2","properties":["geometry.size","css.font-size"]}} +{"id":5,"result":{"handle":"w:2","scene":"scene:1","properties":{"geometry.size":[100,30],"css.font-size":"14dp"},"unavailable":[]}} +{"id":6,"method":"ui.focus"} +{"id":6,"result":{"scene":"scene:1","widget":null}} +{"id":14,"method":"ui.screenshot","params":{"scene":"scene:2","format":"png"}} +{"id":14,"result":{"path":"/tmp/eepp-inspector-19382-.png","format":"png","window":"win:1","scene":"scene:2","rectPx":[100,80,480,320],"sizePx":[480,320]}} +{"id":7,"method":"input.click","params":{"handle":"w:2"}} +{"id":7,"result":{"target":"w:2","scene":"scene:1","pointPx":[60,25]}} +{"id":8,"method":"input.key","params":{"key":"Enter","action":"press","modifiers":[]}} +{"id":8,"result":{"key":"Enter","action":"press"}} +{"id":9,"method":"input.text","params":{"text":"hello"}} +{"id":9,"result":{"characters":5}} +{"id":10,"method":"watch.subscribe","params":{"handle":"w:2","properties":["geometry.size"],"initial":true}} +{"id":10,"result":{"subscription":"sub:1","targets":[{"handle":"w:2","values":{"geometry.size":[100,30]}}]}} +{"id":11,"method":"watch.unsubscribe","params":{"subscription":"sub:1"}} +{"id":11,"result":{"unsubscribed":true}} +{"id":12,"method":"session.nextFrame","params":{"count":1}} +{"id":12,"result":{"frames":1}} +{"id":13,"method":"session.batch","params":{"commands":[{"name":"panel","method":"ui.query","params":{"selector":"#panel","limit":1},"return":false},{"name":"size","method":"ui.inspect","params":{"handle":{"$ref":"panel#/nodes/0/handle"},"properties":["geometry.size"]}}]}} +{"id":13,"result":{"results":{"size":{"handle":"w:2","scene":"scene:1","properties":{"geometry.size":[100,30]},"unavailable":[]}}}} +``` + +`ui.query` uses eepp's CSS selector matching and returns an empty `nodes` array for no matches. Summary fields include handle, scene, tag, ID, classes, active pseudo-classes, pixel bounds, visible, enabled, focused, and optional text. Text is exposed for text controls, `UITextNode`, `UITextSpan`, and `UIRichText`; rich-text containers concatenate descendant source text without forcing layout. Summary text is limited to 256 Unicode characters. When `properties` is provided, each returned node also has the requested inspection values, avoiding one `ui.inspect` call per match. `ui.tree` does not cross a nested scene boundary; a WebView entry instead includes `documentScene`. + +`ui.screenshot` captures the native window containing the selected scene. With no target it captures the default scene's full native window. An explicit `window` captures that whole window; an explicit `scene` crops to its visible world bounds, including a WebView document viewport. `rect` overrides that crop and uses top-left native-window pixel coordinates. It must fit entirely inside the window and have positive width and height. Supported formats are `png`, `jpg`, `bmp`, `tga`, `qoi`, and `webp`; eepp's case-insensitive image extension parser also accepts `jpeg` and `jfif` as aliases for `jpg`. The response uses the canonical extension. The server renders the selected window after the current UI update, saves the image in the OS temporary directory, and returns its absolute `path`; it does not send image bytes over NDJSON. The caller is responsible for deleting the file when finished. The path is useful to a local agent; remote clients must separately retrieve the file, for example over SSH. Screenshot observation is available in read-only mode and can be used inside `session.batch`. + +Single-widget methods never pick the first of multiple selector matches: zero matches return `target-not-found`, and multiple matches return `target-ambiguous`. `input.click` uses the window input route and scene hit testing; clipped, covered, invisible, disabled, or off-window targets can return `target-not-interactable`. It does not scroll a target into view. `input.key` accepts names understood by eepp's `Input::getKeyFromName()` plus `Enter` as an alias for `Return`, and modifiers `Ctrl`, `Shift`, `Alt`, `Meta`; action may be `press`, `down`, or `up`. `input.text` sends Unicode text input to the current focus path and does not focus a widget first. Read-only mode omits input capabilities and returns `permission-denied` for these methods. + +## Properties + +`ui.inspect` defaults to `identity.*`, `geometry.*`, and `state.*`. Explicit names and the wildcards `identity.*`, `geometry.*`, `state.*`, `content.*`, `css.*`, and `*` are accepted. `css.*` expands to CSS properties implemented by that widget. Unknown or unavailable properties appear in `unavailable` rather than failing the whole inspection. Long text properties are bounded to 4096 Unicode characters by default. `maxStringLength` can raise that limit to 65536; `truncatedProperties` lists fields cut short. Watches use the 4096 character default. + +| Group | Names and values | +| --- | --- | +| `identity.*` | `tag`, `id`, `classes`, `pseudoClasses` | +| `geometry.*` | `position` and `size` as local pixel `[x,y]`; `worldPosition` as window/world pixel `[x,y]`; `bounds` as `[x,y,width,height]` in the window/world hit-test space | +| `state.*` | `visible`, `enabled`, `focused` booleans | +| `content.*` | `text` from text controls and HTML text widgets (`UITextNode`, `UITextSpan`, `UIRichText`); rich-text source traversal is capped at 2048 nodes and does not enter nested scenes or text controls. Subject to truncation and secret redaction. | +| `css.*` | `css.` as eepp's effective `getPropertyString()` serialization, for example `css.font-size` | + +## Batching and chaining + +`session.batch` executes commands sequentially on the UI thread, without rollback. Each command has `method`, optional `params`, optional unique `name`, optional `return` (default `true`), and optional post-command `delay`. `session.connect` and nested batches are forbidden. A failed step stops later steps and returns `batch-command-failed` with its index, name, and underlying `commandError`; earlier actions remain in effect. + +A later command may recursively replace any parameter value with `{"$ref":"name#/json/pointer"}`. The name must identify an earlier command. The pointer is an RFC 6901 JSON Pointer into that command's **result**. A missing name, forward reference, or bad pointer produces `batch-reference-error` inside the batch failure. A result with `"return":false` is retained for references but omitted from the final wire reply. This makes dependent workflows a single agent/tool interaction. + +The WebView discovery, document query, and inspection flow can be one request: + +```json +{"commands":[ + {"name":"webview","method":"ui.query","return":false,"params":{"selector":"#preview","limit":1}}, + {"name":"problem","method":"ui.query","return":false,"params":{"scene":{"$ref":"webview#/nodes/0/documentScene"},"selector":"td.problem","limit":1}}, + {"name":"inspection","method":"ui.inspect","params":{"handle":{"$ref":"problem#/nodes/0/handle"},"properties":["geometry.size","css.font-size"]}} +]} +``` + +Only `results.inspection` is returned. To request a larger single response instead of many inspect calls, use `ui.query` with `"properties":["geometry.size","css.font-size"]` and an appropriate limit. + +For action and observation in one call: + +```json +{"commands":[ + {"method":"input.click","delay":"100ms","return":false,"params":{"selector":"#toggle"}}, + {"name":"state","method":"ui.inspect","params":{"selector":"#panel","properties":["geometry.size","state.visible","css.display"]}} +]} +``` + +The click happens first; eepp's normal timeout machinery resumes the batch at least 100 ms later while frames keep running. For a frame barrier, insert `{"method":"session.nextFrame","return":false,"params":{"count":1}}` between the click and inspection. This resumes after the next completed normal scene update. Delayed batches do not block other clients or requests. Closing a client cancels its pending batches. Batching reduces agent/tool round trips; inline properties let the client deliberately choose one larger response when that costs less than several small calls. + +## Watches and events + +`watch.subscribe` resolves a selector once and binds to its current matches; it is not a live selector. Watches require explicit property names, without wildcards. The initial values are in the subscribe response by default. The inspector compares only watched targets and properties once per UI frame, coalescing changes to one `watch.changed` event per target per frame. `sequence` increases per subscription. A target's destruction emits `watch.targetRemoved` with reason `target-destroyed`; once none remain, `watch.ended` has reason `no-targets`. Explicit unsubscribe returns directly without an ended event. Disconnect releases the connection's subscriptions. + +```json +{"method":"watch.changed","params":{"subscription":"sub:1","sequence":7,"target":"w:42","changes":{"geometry.size":{"old":[800,600],"new":[760,600]}}}} +``` + +Other events are `ui.windowCreated` and `ui.windowDestroyed` with `revision` and a window descriptor or handle; `ui.sceneCreated` and `ui.sceneDestroyed` with `revision` and a scene descriptor or handle. Events may appear between request responses. Keep a watch client connected while another client sends actions. + +| Event | Example `params` | +| --- | --- | +| `ui.windowCreated` | `{"revision":1,"window":{"handle":"win:2","id":2,"title":"Preview","sizePx":[800,600]}}` | +| `ui.windowDestroyed` | `{"revision":2,"window":"win:2"}` | +| `ui.sceneCreated` | `{"revision":3,"scene":{"handle":"scene:2","window":"win:1","kind":"webview","parentScene":"scene:1","owner":"w:4","root":"w:5"}}` | +| `ui.sceneDestroyed` | `{"revision":4,"scene":"scene:2"}` | +| `watch.changed` | `{"subscription":"sub:1","sequence":1,"target":"w:2","changes":{"geometry.size":{"old":[100,30],"new":[120,30]}}}` | +| `watch.targetRemoved` | `{"subscription":"sub:1","target":"w:2","reason":"target-destroyed"}` | +| `watch.ended` | `{"subscription":"sub:1","reason":"no-targets"}` | + +## Errors + +Stable v1 error codes are: + +| Code | Meaning | +| --- | --- | +| `parse-error` | Malformed JSON. | +| `invalid-request` | Missing/invalid request envelope, ID, or params. | +| `unauthenticated` | Inspector method before connection authentication. | +| `authentication-failed` | Token rejected. | +| `unsupported-protocol` | Client requested a version other than 1. | +| `method-not-found` | Unknown method. | +| `invalid-params` | A method argument is missing, invalid, or beyond a limit. | +| `permission-denied` | An action was requested in read-only mode. | +| `invalid-window`, `invalid-scene`, `invalid-widget` | A handle does not identify a live object of that kind. | +| `default-scene-unavailable` | The original default scene was destroyed. | +| `selector-invalid` | Selector is empty or rejected by selector validation. | +| `target-not-found`, `target-ambiguous` | Single-target selector matched zero or multiple widgets. | +| `target-not-interactable` | Click point cannot reach the target. | +| `property-unknown` | Watch property cannot be read. | +| `input-unavailable` | The scene/window has no usable input context. | +| `screenshot-failed` | No drawable window/scene area, or image capture/save failed. | +| `batch-reference-error` | Invalid command reference or JSON Pointer. | +| `batch-command-failed` | A batch step failed; `data.commandError` contains its cause. | +| `request-too-large` | A request line exceeded 1 MiB; the server closes the connection. | +| `internal-error` | An unexpected command error was caught. | + +## C++ API + +Applications normally need no code. Tests and embedding code may use: + +```cpp +#include + +UIInspectorServerSettings settings; +settings.address = "127.0.0.1"; +settings.port = 0; +settings.readOnly = false; +settings.token = "caller-chosen-secret"; // leave empty to generate one +UIInspectorServer::start( defaultScene, settings ); +auto* server = UIInspectorServer::instance(); +auto port = server->getPort(); +UIInspectorServer::stop(); +``` + +`instance()` is null when disabled. `isRunning()`, `getAddress()`, `getPort()`, and `getToken()` are available while running. Call the API on the UI thread. The framework automatically pumps it during `SceneManager::update()` and stops it during scene-manager teardown. + +The headless `UIInspector` owns the handle registry and scene/widget reads. Protocol dispatch and TCP session code currently share one implementation file; the network thread only frames messages and manages connections, while the scene update pump runs every UI operation. This file layout does not affect the wire protocol. + +## Troubleshooting + +- **Connection refused:** confirm `EEPP_INSPECTOR=1`, read the emitted port, and use the correct host or SSH forward. +- **Authentication failed:** use the token from the same application run; a supplied token is intentionally absent from the descriptor. +- **Query returns nothing:** inspect `ui.contexts`; HTML inside a WebView is in its `documentScene`. +- **Ambiguous selector:** use a more specific selector or a widget handle. +- **Click rejected:** inspect pixel bounds and visibility; the center may be clipped, covered, or outside the window. Scrolling is manual in v1. +- **Key/text has no effect:** call `ui.focus`, then click the intended input before typing. +- **Remote debugging:** keep the listener on loopback and use SSH forwarding; direct raw TCP has no encryption. diff --git a/include/eepp/scene/scenemanager.hpp b/include/eepp/scene/scenemanager.hpp index 83f9be9ff..75884053a 100644 --- a/include/eepp/scene/scenemanager.hpp +++ b/include/eepp/scene/scenemanager.hpp @@ -48,6 +48,9 @@ class EE_API SceneManager { /** Returns the UI scene associated with @p window, or nullptr if none is registered. */ UISceneNode* getUISceneNode( EE::Window::Window* window ); + /** Enumerates registered top-level scenes on the UI thread. */ + void forEachSceneNode( const std::function& callback ) const; + void setCurrentUISceneNode( UISceneNode* uiSceneNode ); /** Removes and destroys all registered scene nodes associated with @p window. */ diff --git a/include/eepp/ui/tools/uiinspector.hpp b/include/eepp/ui/tools/uiinspector.hpp new file mode 100644 index 000000000..18ba56f13 --- /dev/null +++ b/include/eepp/ui/tools/uiinspector.hpp @@ -0,0 +1,71 @@ +#ifndef EE_UI_TOOLS_UIINSPECTOR_HPP +#define EE_UI_TOOLS_UIINSPECTOR_HPP + +#include +#include +#include +#include +#include +#include + +namespace EE { namespace UI { + +/** UI-thread-only inspection domain. Handles are monotonic and invalidated on Node::OnClose. */ +class EE_API UIInspector { + public: + UIInspector(); + + ~UIInspector(); + + void setDefaultScene( UISceneNode* scene ); + + UISceneNode* defaultScene() const; + + std::string windowHandle( EE::Window::Window* window ); + + std::string sceneHandle( UISceneNode* scene ); + + std::string widgetHandle( UIWidget* widget ); + + EE::Window::Window* window( const std::string& handle ) const; + + void invalidateWindow( EE::Window::Window* window ); + + UISceneNode* scene( const std::string& handle ) const; + + UIWidget* widget( const std::string& handle ) const; + + SmallVector topLevelScenes() const; + + WidgetQueryResult query( UISceneNode* scene, const std::string& selector ) const; + + std::string property( UIWidget* widget, const std::string& name ) const; + + bool isSecret( UIWidget* widget ) const; + + private: + struct NodeEntry { + Scene::Node* node{ nullptr }; + Scene::EventConnection close; + }; + using NodeHandles = UnorderedMap; + using ReverseNodeHandles = UnorderedMap; + + std::string registerNode( Scene::Node* node, const char* prefix, NodeHandles& handles, + ReverseNodeHandles& reverse ); + + Uint64 mNextWindow{ 1 }; + Uint64 mNextScene{ 1 }; + Uint64 mNextWidget{ 1 }; + UISceneNode* mDefaultScene{ nullptr }; + UnorderedMap mWindowReverse; + UnorderedMap mWindows; + ReverseNodeHandles mSceneReverse; + NodeHandles mScenes; + ReverseNodeHandles mWidgetReverse; + NodeHandles mWidgets; +}; + +}} // namespace EE::UI + +#endif diff --git a/include/eepp/ui/tools/uiinspectorserver.hpp b/include/eepp/ui/tools/uiinspectorserver.hpp new file mode 100644 index 000000000..bb7a99d49 --- /dev/null +++ b/include/eepp/ui/tools/uiinspectorserver.hpp @@ -0,0 +1,56 @@ +#ifndef EE_UI_TOOLS_UIINSPECTORSERVER_HPP +#define EE_UI_TOOLS_UIINSPECTORSERVER_HPP + +#include +#include +#include + +namespace EE { namespace Window { +class Window; +}} // namespace EE::Window + +namespace EE { namespace UI { +class UISceneNode; + +struct UIInspectorServerSettings { + std::string address{ "127.0.0.1" }; + Uint16 port{ 0 }; + bool readOnly{ false }; + std::string token; +}; + +/** Explicitly enabled TCP inspector. Public methods must be called from the UI thread. */ +class EE_API UIInspectorServer { + public: + static UIInspectorServer* instance(); + + static bool start( UISceneNode* defaultScene, UIInspectorServerSettings settings = {} ); + + static void startFromEnvironment( UISceneNode* defaultScene ); + + static void stop(); + + static void pump(); + + static void notifyWindowDestroyed( EE::Window::Window* window ); + + bool isRunning() const; + + const std::string& getAddress() const; + + Uint16 getPort() const; + + const std::string& getToken() const; + + ~UIInspectorServer(); + + private: + UIInspectorServer(); + struct Impl; + std::unique_ptr mImpl; + static UIInspectorServer* sInstance; +}; + +}} // namespace EE::UI + +#endif diff --git a/projects/linux/ee.files b/projects/linux/ee.files index dded25d1b..410a5c995 100644 --- a/projects/linux/ee.files +++ b/projects/linux/ee.files @@ -376,6 +376,8 @@ ../../include/eepp/ui/tools/uicodeeditorsplitter.hpp ../../include/eepp/ui/tools/uicolorpicker.hpp ../../include/eepp/ui/tools/uidocfindreplace.hpp +../../include/eepp/ui/tools/uiinspector.hpp +../../include/eepp/ui/tools/uiinspectorserver.hpp ../../include/eepp/ui/tools/uiwidgetinspector.hpp ../../include/eepp/ui/uiabstractview.hpp ../../include/eepp/ui/uiapplication.hpp @@ -1174,6 +1176,8 @@ ../../src/eepp/ui/tools/uidocfindreplace.cpp ../../src/eepp/ui/tools/uitextureviewer.cpp ../../src/eepp/ui/tools/uitextureviewer.hpp +../../src/eepp/ui/tools/uiinspector.cpp +../../src/eepp/ui/tools/uiinspectorserver.cpp ../../src/eepp/ui/tools/uiwidgetinspector.cpp ../../src/eepp/ui/uiabstractview.cpp ../../src/eepp/ui/uiapplication.cpp diff --git a/projects/macos/ee.files b/projects/macos/ee.files index fc65f96d2..7faeaa499 100644 --- a/projects/macos/ee.files +++ b/projects/macos/ee.files @@ -373,6 +373,8 @@ ../../include/eepp/ui/tools/uicodeeditorsplitter.hpp ../../include/eepp/ui/tools/uicolorpicker.hpp ../../include/eepp/ui/tools/uidocfindreplace.hpp +../../include/eepp/ui/tools/uiinspector.hpp +../../include/eepp/ui/tools/uiinspectorserver.hpp ../../include/eepp/ui/tools/uiwidgetinspector.hpp ../../include/eepp/ui/uiabstractview.hpp ../../include/eepp/ui/uibackgrounddrawable.hpp @@ -1155,6 +1157,8 @@ ../../src/eepp/ui/tools/uidocfindreplace.cpp ../../src/eepp/ui/tools/uitextureviewer.cpp ../../src/eepp/ui/tools/uitextureviewer.hpp +../../src/eepp/ui/tools/uiinspector.cpp +../../src/eepp/ui/tools/uiinspectorserver.cpp ../../src/eepp/ui/tools/uiwidgetinspector.cpp ../../src/eepp/ui/uiabstractview.cpp ../../src/eepp/ui/uiapplication.cpp diff --git a/projects/scripts/eepp-inspect.py b/projects/scripts/eepp-inspect.py new file mode 100755 index 000000000..64bdf6e70 --- /dev/null +++ b/projects/scripts/eepp-inspect.py @@ -0,0 +1,255 @@ +#!/usr/bin/env python3 +"""Small standard-library client for the eepp inspector NDJSON protocol.""" + +import argparse +import json +import os +import queue +import select +import socket +import sys +import threading +import time + + +def emit(value, pretty=False): + print(json.dumps(value, ensure_ascii=False, indent=2 if pretty else None, + separators=None if pretty else (",", ":")), flush=True) + + +class Client: + def __init__(self, host, port, token): + self.sock = socket.create_connection((host, port), timeout=10) + self.sock.settimeout(None) + self.input = b"" + self.next_id = 2 + reply = self.request({"id": 1, "method": "session.connect", "params": { + "protocolVersion": 1, "token": token, + "client": {"name": "eepp-inspect.py", "version": "1"}}}) + if "error" in reply: + raise RuntimeError(reply["error"].get("message", str(reply["error"]))) + + def close(self): + self.sock.close() + + def send(self, request): + request = dict(request) + if "id" not in request: + request["id"] = self.next_id + self.next_id += 1 + self.sock.sendall((json.dumps(request, ensure_ascii=False, + separators=(",", ":")) + "\n").encode("utf-8")) + return request["id"] + + def receive(self): + while b"\n" not in self.input: + chunk = self.sock.recv(8192) + if not chunk: + raise EOFError("inspector disconnected") + self.input += chunk + if len(self.input) > 4 * 1024 * 1024: + raise ValueError("response exceeds 4 MiB") + line, self.input = self.input.split(b"\n", 1) + return json.loads(line.decode("utf-8").rstrip("\r")) + + def readable(self, timeout=None): + return b"\n" in self.input or bool(select.select([self.sock], [], [], timeout)[0]) + + def request(self, request, on_event=None): + request_id = self.send(request) + while True: + message = self.receive() + if message.get("id") == request_id: + return message + if on_event is not None: + on_event(message) + + +def target(value, scene=None): + return {"handle": value} if value.startswith("w:") else { + **({"scene": scene} if scene else {}), "selector": value} + + +def scene_param(args): + return {"scene": args.scene} if getattr(args, "scene", None) else {} + + +def command(args): + name = args.command + params = scene_param(args) + if name == "contexts": + return "ui.contexts", {} + if name == "query": + params.update(selector=args.selector, offset=args.offset, limit=args.limit) + if args.properties: + params["properties"] = args.properties + if args.max_string_length is not None: + params["maxStringLength"] = args.max_string_length + return "ui.query", params + if name == "tree": + params.update(depth=args.depth, maxNodes=args.max_nodes, + includeText=args.include_text) + if args.root: + params = {key: value for key, value in params.items() if key != "scene"} + params["root"] = args.root + return "ui.tree", params + if name == "inspect": + params.update(target(args.target, args.scene)) + if args.properties: + params["properties"] = args.properties + if args.max_string_length is not None: + params["maxStringLength"] = args.max_string_length + return "ui.inspect", params + if name == "focus": + return "ui.focus", params + if name == "screenshot": + if args.window: + params["window"] = args.window + if args.rect: + params["rect"] = args.rect + params["format"] = args.format + return "ui.screenshot", params + if name == "click": + return "input.click", target(args.target, args.scene) + if name == "key": + params.update(key=args.key, action=args.action, modifiers=args.modifiers) + return "input.key", params + if name == "type": + params["text"] = args.text + return "input.text", params + if name == "watch": + params.update(target(args.target, args.scene)) + params.update(properties=args.properties, initial=not args.no_initial) + return "watch.subscribe", params + if name == "batch": + with (sys.stdin if args.file == "-" else open(args.file, encoding="utf-8")) as stream: + return "session.batch", json.load(stream) + if name == "raw": + return None, json.loads(args.request) + raise ValueError(name) + + +def main(): + parser = argparse.ArgumentParser(description=__doc__) + parser.add_argument("--host", default=os.getenv("EEPP_INSPECTOR_HOST", "127.0.0.1")) + parser.add_argument("--port", type=int, default=int(os.getenv("EEPP_INSPECTOR_PORT", "0"))) + parser.add_argument("--token", default=os.getenv("EEPP_INSPECTOR_TOKEN")) + parser.add_argument("--pretty", action="store_true") + sub = parser.add_subparsers(dest="command", required=True) + sub.add_parser("contexts") + query = sub.add_parser("query") + query.add_argument("selector") + query.add_argument("--scene") + query.add_argument("--offset", type=int, default=0) + query.add_argument("--limit", type=int, default=50) + query.add_argument("--properties", nargs="+") + query.add_argument("--max-string-length", type=int) + tree = sub.add_parser("tree") + tree.add_argument("--scene") + tree.add_argument("--root") + tree.add_argument("--depth", type=int, default=3) + tree.add_argument("--max-nodes", type=int, default=200) + tree.add_argument("--include-text", action="store_true") + inspect = sub.add_parser("inspect") + inspect.add_argument("target", help="widget handle or selector") + inspect.add_argument("properties", nargs="*") + inspect.add_argument("--scene") + inspect.add_argument("--max-string-length", type=int) + focus = sub.add_parser("focus") + focus.add_argument("--scene") + screenshot = sub.add_parser("screenshot", help="save a window or scene image to a temporary path") + screenshot_context = screenshot.add_mutually_exclusive_group() + screenshot_context.add_argument("--scene", help="capture the scene's visible rectangle") + screenshot_context.add_argument("--window", help="capture this native window") + screenshot.add_argument("--rect", nargs=4, type=int, metavar=("X", "Y", "WIDTH", "HEIGHT"), + help="capture a rectangle in native-window pixel coordinates") + screenshot.add_argument("--format", choices=("png", "jpg", "bmp", "tga", "qoi", "webp"), + default="png") + click = sub.add_parser("click") + click.add_argument("target") + click.add_argument("--scene") + key = sub.add_parser("key") + key.add_argument("key") + key.add_argument("--scene") + key.add_argument("--action", choices=("press", "down", "up"), default="press") + key.add_argument("--modifiers", nargs="*", default=[]) + typ = sub.add_parser("type", help="inject text; optional --target clicks first") + typ.add_argument("text") + typ.add_argument("--scene") + typ.add_argument("--target") + watch = sub.add_parser("watch") + watch.add_argument("target") + watch.add_argument("properties", nargs="+") + watch.add_argument("--scene") + watch.add_argument("--no-initial", action="store_true") + watch.add_argument("--timeout", type=float) + watch.add_argument("--count", type=int) + batch = sub.add_parser("batch") + batch.add_argument("file", help="JSON params file, or - for stdin") + sub.add_parser("session", help="authenticated persistent NDJSON stdin/stdout") + raw = sub.add_parser("raw") + raw.add_argument("request", help="JSON request object; id is assigned if omitted") + args = parser.parse_args() + if not args.port or not args.token: + parser.error("--port and --token (or their environment variables) are required") + client = Client(args.host, args.port, args.token) + try: + if args.command == "session": + requests = queue.Queue() + def read_stdin(): + for line in sys.stdin: + requests.put(line) + requests.put(None) + threading.Thread(target=read_stdin, daemon=True).start() + stdin_open = True + outstanding = set() + while stdin_open or outstanding: + try: + line = requests.get_nowait() + if line is None: + stdin_open = False + elif line.strip(): + outstanding.add(client.send(json.loads(line))) + except queue.Empty: + pass + if client.readable(0.05): + message = client.receive() + emit(message) + outstanding.discard(message.get("id")) + return + if args.command == "type" and args.target: + reply = client.request({"method": "input.click", + "params": target(args.target, args.scene)}) + if "error" in reply: + emit(reply, args.pretty) + return 1 + if not args.scene: + args.scene = reply["result"].get("scene") + method, params = command(args) + request = params if method is None else {"method": method, "params": params} + reply = client.request(request) + emit(reply, args.pretty) + if "error" in reply: + return 1 + if args.command == "watch": + deadline = time.monotonic() + args.timeout if args.timeout is not None else None + seen = 0 + while args.count is None or seen < args.count: + remaining = None if deadline is None else max(0, deadline - time.monotonic()) + if remaining == 0 or not client.readable(remaining): + break + event = client.receive() + emit(event, args.pretty) + if event.get("method") == "watch.changed": + seen += 1 + finally: + client.close() + return 0 + + +if __name__ == "__main__": + try: + sys.exit(main()) + except (OSError, ValueError, EOFError, RuntimeError, json.JSONDecodeError) as exc: + print(f"eepp-inspect: {exc}", file=sys.stderr) + sys.exit(2) diff --git a/projects/scripts/tests/test_eepp_inspect.py b/projects/scripts/tests/test_eepp_inspect.py new file mode 100644 index 000000000..eb4f6769f --- /dev/null +++ b/projects/scripts/tests/test_eepp_inspect.py @@ -0,0 +1,121 @@ +"""Transport and CLI smoke tests using a small fake NDJSON inspector.""" + +import json +import pathlib +import socket +import subprocess +import sys +import threading +import unittest + + +CLIENT = pathlib.Path(__file__).resolve().parents[1] / "eepp-inspect.py" + + +class FakeServer: + def __init__(self, handler): + self.listener = socket.socket() + self.listener.bind(("127.0.0.1", 0)) + self.listener.listen(1) + self.port = self.listener.getsockname()[1] + self.handler = handler + self.error = None + self.thread = threading.Thread(target=self.run, daemon=True) + self.thread.start() + + def run(self): + try: + with self.listener: + conn, _ = self.listener.accept() + with conn, conn.makefile("r", encoding="utf-8") as lines: + connect = json.loads(lines.readline()) + assert connect["method"] == "session.connect" + assert connect["params"]["token"] == "test-token" + conn.sendall(b'{"id":1,"result":{"protocolVersion":1}}\n') + self.handler(conn, lines) + except Exception as exc: + self.error = exc + + def join(self): + self.thread.join(timeout=5) + if self.error: + raise self.error + assert not self.thread.is_alive(), "fake server did not finish" + + +def invoke(server, *args, input_text=None): + return subprocess.run( + [sys.executable, str(CLIENT), "--port", str(server.port), "--token", "test-token", *args], + input=input_text, text=True, capture_output=True, timeout=5, + ) + + +class InspectorCliTests(unittest.TestCase): + def test_contexts_emits_compact_json(self): + def handler(conn, lines): + request = json.loads(lines.readline()) + self.assertEqual(request["method"], "ui.contexts") + conn.sendall((json.dumps({"id": request["id"], "result": {"scenes": []}}) + "\n").encode()) + + server = FakeServer(handler) + result = invoke(server, "contexts") + server.join() + self.assertEqual(result.returncode, 0, result.stderr) + self.assertEqual(json.loads(result.stdout)["result"]["scenes"], []) + self.assertEqual(result.stdout.count("\n"), 1) + + def test_batch_reads_stdin_and_preserves_commands(self): + def handler(conn, lines): + request = json.loads(lines.readline()) + self.assertEqual(request["method"], "session.batch") + self.assertFalse(request["params"]["commands"][0]["return"]) + conn.sendall((json.dumps({"id": request["id"], "result": {"results": {}}}) + "\n").encode()) + + server = FakeServer(handler) + definition = '{"commands":[{"method":"ui.query","return":false,"params":{"selector":"*"}}]}' + result = invoke(server, "batch", "-", input_text=definition) + server.join() + self.assertEqual(result.returncode, 0, result.stderr) + self.assertEqual(json.loads(result.stdout)["result"]["results"], {}) + + def test_screenshot_sends_window_rect_and_format(self): + def handler(conn, lines): + request = json.loads(lines.readline()) + self.assertEqual(request["method"], "ui.screenshot") + self.assertEqual(request["params"], { + "window": "win:2", "rect": [5, 6, 30, 20], "format": "qoi"}) + conn.sendall((json.dumps({"id": request["id"], "result": { + "path": "/tmp/example.qoi", "format": "qoi"}}) + "\n").encode()) + + server = FakeServer(handler) + result = invoke(server, "screenshot", "--window", "win:2", "--rect", + "5", "6", "30", "20", "--format", "qoi") + server.join() + self.assertEqual(result.returncode, 0, result.stderr) + self.assertEqual(json.loads(result.stdout)["result"]["path"], "/tmp/example.qoi") + + def test_session_streams_events_between_pipelined_responses(self): + def handler(conn, lines): + first = json.loads(lines.readline()) + second = json.loads(lines.readline()) + self.assertEqual(first["method"], "ui.contexts") + self.assertEqual(second["method"], "ui.focus") + messages = [ + {"method": "watch.changed", "params": {"sequence": 1}}, + {"id": second["id"], "result": {"widget": None}}, + {"id": first["id"], "result": {"scenes": []}}, + ] + conn.sendall("".join(json.dumps(message) + "\n" for message in messages).encode()) + + server = FakeServer(handler) + requests = '{"method":"ui.contexts"}\n{"method":"ui.focus"}\n' + result = invoke(server, "session", input_text=requests) + server.join() + self.assertEqual(result.returncode, 0, result.stderr) + messages = [json.loads(line) for line in result.stdout.splitlines()] + self.assertEqual(messages[0]["method"], "watch.changed") + self.assertEqual({message.get("id") for message in messages[1:]}, {2, 3}) + + +if __name__ == "__main__": + unittest.main() diff --git a/projects/windows/ee.files b/projects/windows/ee.files index 0b7ba0f25..2694b9b2f 100644 --- a/projects/windows/ee.files +++ b/projects/windows/ee.files @@ -366,6 +366,8 @@ ../../include/eepp/ui/tools/uicodeeditorsplitter.hpp ../../include/eepp/ui/tools/uicolorpicker.hpp ../../include/eepp/ui/tools/uidocfindreplace.hpp +../../include/eepp/ui/tools/uiinspector.hpp +../../include/eepp/ui/tools/uiinspectorserver.hpp ../../include/eepp/ui/tools/uiwidgetinspector.hpp ../../include/eepp/ui/uiabstractview.hpp ../../include/eepp/ui/uibackgrounddrawable.hpp @@ -853,6 +855,8 @@ ../../src/eepp/ui/tools/uicodeeditorsplitter.cpp ../../src/eepp/ui/tools/uicolorpicker.cpp ../../src/eepp/ui/tools/uidocfindreplace.cpp +../../src/eepp/ui/tools/uiinspector.cpp +../../src/eepp/ui/tools/uiinspectorserver.cpp ../../src/eepp/ui/tools/uiwidgetinspector.cpp ../../src/eepp/ui/uiabstractview.cpp ../../src/eepp/ui/uibackgrounddrawable.cpp diff --git a/src/eepp/scene/scenemanager.cpp b/src/eepp/scene/scenemanager.cpp index c069ff4f8..1dfd89974 100644 --- a/src/eepp/scene/scenemanager.cpp +++ b/src/eepp/scene/scenemanager.cpp @@ -1,6 +1,7 @@ #include #include #include +#include #include #include @@ -16,6 +17,7 @@ bool SceneManager::isActive() { SceneManager::SceneManager() : mUISceneNode( NULL ), mScopedUISceneNode( NULL ) {} SceneManager::~SceneManager() { + UIInspectorServer::stop(); for ( auto& it : mSceneNodes ) { SceneNode* node = it; eeSAFE_DELETE( node ); @@ -26,6 +28,8 @@ SceneManager::~SceneManager() { SceneNode* SceneManager::add( SceneNode* sceneNode ) { mSceneNodes.push_back( sceneNode ); + if ( sceneNode->isUISceneNode() ) + UIInspectorServer::startFromEnvironment( sceneNode->asType() ); return sceneNode; } @@ -82,6 +86,7 @@ void SceneManager::update( const Time& elapsed ) { sceneNode->update( elapsed ); } } + UIInspectorServer::pump(); } void SceneManager::update() { @@ -119,6 +124,11 @@ UISceneNode* SceneManager::getUISceneNode( EE::Window::Window* window ) { return nullptr; } +void SceneManager::forEachSceneNode( const std::function& callback ) const { + for ( auto* sceneNode : mSceneNodes ) + callback( sceneNode ); +} + void SceneManager::setCurrentUISceneNode( UISceneNode* uiSceneNode ) { mUISceneNode = uiSceneNode; } diff --git a/src/eepp/ui/flexlayouter.cpp b/src/eepp/ui/flexlayouter.cpp index c58dfe595..f4f738e72 100644 --- a/src/eepp/ui/flexlayouter.cpp +++ b/src/eepp/ui/flexlayouter.cpp @@ -473,8 +473,10 @@ void FlexLayouter::measureFlexItems( const Axis& mainAxis, const Axis& crossAxis const auto* minW = item.widget->getUIStyle()->getProperty( PropertyId::MinWidth ); if ( minW ) { Float explicitMin = item.widget->cssWidthPropertyToBorderBoxWidth( *minW ); - if ( explicitMin > item.minMainSize ) - item.minMainSize = explicitMin; + // An explicit minimum replaces the automatic content minimum. In + // particular, min-width: 0 must permit a flex item to shrink below + // the min-content width of an image or other descendant. + item.minMainSize = explicitMin; } const auto* maxW = item.widget->getUIStyle()->getProperty( PropertyId::MaxWidth ); if ( maxW ) @@ -514,8 +516,7 @@ void FlexLayouter::measureFlexItems( const Axis& mainAxis, const Axis& crossAxis const auto* minH = item.widget->getUIStyle()->getProperty( PropertyId::MinHeight ); if ( minH ) { Float explicitMin = item.widget->cssHeightPropertyToBorderBoxHeight( *minH ); - if ( explicitMin > item.minMainSize ) - item.minMainSize = explicitMin; + item.minMainSize = explicitMin; } const auto* maxH = item.widget->getUIStyle()->getProperty( PropertyId::MaxHeight ); if ( maxH ) @@ -1285,15 +1286,11 @@ void FlexLayouter::updateLayout() { Float containerMainSize = mainAxis.horizontal ? containerWidth : containerHeight; Float containerCrossSize = crossAxis.horizontal ? containerWidth : containerHeight; - if ( widthPolicy != SizePolicy::Fixed && mainAxis.horizontal ) - containerMainSize -= totalPaddingMain; - if ( heightPolicy != SizePolicy::Fixed && !mainAxis.horizontal ) - containerMainSize -= totalPaddingMain; - - if ( widthPolicy != SizePolicy::Fixed && crossAxis.horizontal ) - containerCrossSize -= totalPaddingCross; - if ( heightPolicy != SizePolicy::Fixed && !crossAxis.horizontal ) - containerCrossSize -= totalPaddingCross; + // Flex lines use the content box on both axes, including when the + // container has an explicit width or height. The outer dimensions above + // already include padding after CSS box-sizing resolution. + containerMainSize -= totalPaddingMain; + containerCrossSize -= totalPaddingCross; if ( containerMainSize < 0.f ) containerMainSize = 0.f; diff --git a/src/eepp/ui/tools/uiinspector.cpp b/src/eepp/ui/tools/uiinspector.cpp new file mode 100644 index 000000000..e4f0041ec --- /dev/null +++ b/src/eepp/ui/tools/uiinspector.cpp @@ -0,0 +1,137 @@ +#include +#include +#include +#include +#include +#include +#include +#include + +namespace EE { namespace UI { + +UIInspector::UIInspector() = default; + +UIInspector::~UIInspector() = default; + +void UIInspector::setDefaultScene( UISceneNode* scene ) { + mDefaultScene = scene; + if ( scene ) + sceneHandle( scene ); +} + +UISceneNode* UIInspector::defaultScene() const { + return mDefaultScene; +} + +std::string UIInspector::registerNode( Scene::Node* node, const char* prefix, NodeHandles& handles, + ReverseNodeHandles& reverse ) { + if ( !node ) + return {}; + auto found = reverse.find( node ); + if ( found != reverse.end() ) + return found->second; + Uint64& next = prefix[0] == 'w' ? mNextWidget : mNextScene; + std::string handle = std::string( prefix ) + std::to_string( next++ ); + NodeEntry entry; + entry.node = node; + entry.close = node->connect( Scene::Event::OnClose, + [this, node, handle, &handles, &reverse]( const Scene::Event* ) { + auto it = handles.find( handle ); + if ( it != handles.end() ) + it->second.node = nullptr; + reverse.erase( node ); + if ( mDefaultScene == node ) + mDefaultScene = nullptr; + } ); + handles.emplace( handle, std::move( entry ) ); + reverse.emplace( node, handle ); + return handle; +} + +std::string UIInspector::windowHandle( EE::Window::Window* window ) { + if ( !window ) + return {}; + auto found = mWindowReverse.find( window ); + if ( found != mWindowReverse.end() ) + return found->second; + std::string handle = "win:" + std::to_string( mNextWindow++ ); + mWindowReverse.emplace( window, handle ); + mWindows.emplace( handle, window ); + return handle; +} + +std::string UIInspector::sceneHandle( UISceneNode* scene ) { + return registerNode( scene, "scene:", mScenes, mSceneReverse ); +} + +std::string UIInspector::widgetHandle( UIWidget* widget ) { + return registerNode( widget, "w:", mWidgets, mWidgetReverse ); +} + +EE::Window::Window* UIInspector::window( const std::string& handle ) const { + auto it = mWindows.find( handle ); + if ( it == mWindows.end() ) + return nullptr; + bool live = false; + EE::Window::Engine::instance()->forEachWindow( + [&]( EE::Window::Window* candidate ) { live |= candidate == it->second; } ); + return live ? it->second : nullptr; +} + +void UIInspector::invalidateWindow( EE::Window::Window* window ) { + auto it = mWindowReverse.find( window ); + if ( it != mWindowReverse.end() ) { + mWindows.erase( it->second ); + mWindowReverse.erase( it ); + } +} + +UISceneNode* UIInspector::scene( const std::string& handle ) const { + auto it = mScenes.find( handle ); + return it == mScenes.end() ? nullptr : static_cast( it->second.node ); +} + +UIWidget* UIInspector::widget( const std::string& handle ) const { + auto it = mWidgets.find( handle ); + return it == mWidgets.end() ? nullptr : static_cast( it->second.node ); +} + +SmallVector UIInspector::topLevelScenes() const { + SmallVector scenes; + Scene::SceneManager::instance()->forEachSceneNode( [&]( Scene::SceneNode* node ) { + if ( node->isUISceneNode() ) + scenes.push_back( node->asType() ); + } ); + return scenes; +} + +WidgetQueryResult UIInspector::query( UISceneNode* scene, const std::string& selector ) const { + if ( !scene || !scene->getRoot() ) + return {}; + WidgetQueryResult widgets = scene->getRoot()->querySelectorAll( selector ); + widgets.erase( + std::remove_if( widgets.begin(), widgets.end(), + [scene]( UIWidget* widget ) { return widget->getUISceneNode() != scene; } ), + widgets.end() ); + return widgets; +} + +std::string UIInspector::property( UIWidget* widget, const std::string& name ) const { + if ( !widget || !CSS::StyleSheetSpecification::instance()->getProperty( name ) ) + return {}; + return widget->getPropertyString( name ); +} + +bool UIInspector::isSecret( UIWidget* widget ) const { + if ( !widget ) + return false; + if ( widget->isType( UI_TYPE_TEXTINPUT ) && + widget->asType()->getMode() == UITextInput::TextInputMode::Password ) + return true; + if ( widget->isType( UI_TYPE_HTML_INPUT ) && + widget->asType()->getInputType() == "password" ) + return true; + return false; +} + +}} // namespace EE::UI diff --git a/src/eepp/ui/tools/uiinspectorserver.cpp b/src/eepp/ui/tools/uiinspectorserver.cpp new file mode 100644 index 000000000..6c33ec225 --- /dev/null +++ b/src/eepp/ui/tools/uiinspectorserver.cpp @@ -0,0 +1,1527 @@ +#include + +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include + +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include + +using json = nlohmann::json; +using namespace EE::Network; +using namespace EE::Scene; + +namespace EE { namespace UI { + +namespace { + +struct InspectorError { + std::string code; + std::string message; + json data; +}; + +void fail( const char* code, const std::string& message, json data = nullptr ) { + throw InspectorError{ code, message, std::move( data ) }; +} + +json errorObject( const InspectorError& error ) { + json value = { { "code", error.code }, { "message", error.message } }; + if ( !error.data.is_null() ) + value["data"] = error.data; + return value; +} + +std::string tokenFromOS() { + try { + std::random_device random; + if ( random.entropy() <= 0 ) + return {}; + static const char* hex = "0123456789abcdef"; + std::string token( 64, '0' ); + for ( size_t i = 0; i < token.size(); i += 8 ) { + unsigned value = random(); + for ( size_t j = 0; j < 8; ++j ) { + token[i + j] = hex[value & 15]; + value >>= 4; + } + } + return token; + } catch ( const std::exception& ) { + return {}; + } +} + +std::string requireString( const json& params, const char* name ) { + if ( !params.contains( name ) || !params[name].is_string() ) + fail( "invalid-params", std::string( name ) + " must be a string" ); + return params[name].get(); +} + +unsigned boundedUnsigned( const json& params, const char* name, unsigned fallback, + unsigned maximum ) { + if ( !params.contains( name ) ) + return fallback; + if ( !params[name].is_number_unsigned() && + !( params[name].is_number_integer() && params[name].get() >= 0 ) ) + fail( "invalid-params", std::string( name ) + " must be non-negative" ); + auto number = params[name].get(); + if ( number > maximum ) + fail( "invalid-params", std::string( name ) + " exceeds limit" ); + return static_cast( number ); +} + +bool optionalBool( const json& params, const char* name, bool fallback ) { + if ( !params.contains( name ) ) + return fallback; + if ( !params[name].is_boolean() ) + fail( "invalid-params", std::string( name ) + " must be boolean" ); + return params[name].get(); +} + +json vector( const Vector2f& value ) { + return json::array( { value.x, value.y } ); +} + +bool envEnabled( const char* value ) { + return value && *value && std::string( value ) != "0"; +} + +bool validDelay( const std::string& value ) { + size_t suffix = value.size() >= 2 && value.compare( value.size() - 2, 2, "ms" ) == 0 ? 2 + : value.size() >= 1 && value.back() == 's' ? 1 + : 0; + if ( suffix == 0 || value.size() <= suffix || value[value.size() - suffix - 1] == '.' ) + return false; + bool dot = false; + for ( size_t i = 0; i < value.size() - suffix; ++i ) { + if ( value[i] == '.' && !dot && i > 0 ) + dot = true; + else if ( value[i] < '0' || value[i] > '9' ) + return false; + } + return true; +} + +bool validSelector( const std::string& selector ) { + auto first = selector.find_first_not_of( " \t\r\n" ); + if ( first == std::string::npos ) + return false; + auto last = selector.find_last_not_of( " \t\r\n" ); + constexpr std::string_view combinators{ ">+~|" }; + if ( combinators.find( selector[first] ) != std::string_view::npos || + combinators.find( selector[last] ) != std::string_view::npos ) + return false; + int brackets = 0; + int parens = 0; + char quote = 0; + for ( char character : selector ) { + if ( quote ) { + if ( character == quote ) + quote = 0; + continue; + } + if ( character == '\'' || character == '"' ) + quote = character; + else if ( character == '[' ) + ++brackets; + else if ( character == ']' ) { + if ( --brackets < 0 ) + return false; + } else if ( character == '(' ) + ++parens; + else if ( character == ')' ) { + if ( --parens < 0 ) + return false; + } + } + return !quote && brackets == 0 && parens == 0; +} + +} // namespace + +struct UIInspectorServer::Impl { + struct Incoming { + Uint64 client; + std::string line; + bool disconnected{ false }; + }; + struct Outgoing { + Uint64 client; + std::string line; + }; + struct Client { + Uint64 id; + std::unique_ptr socket; + std::string input; + std::string output; + bool authenticated{ false }; + bool closing{ false }; + }; + struct Subscription { + Uint64 client; + Uint64 sequence{ 0 }; + std::map> values; + }; + struct Pending { + Uint64 client; + json id; + json commands; + json results{ json::object() }; + std::map named; + size_t index{ 0 }; + unsigned frames{ 0 }; + bool standalone{ false }; + bool ready{ true }; + bool cancelled{ false }; + }; + + UIInspector inspector; + UIInspectorServerSettings settings; + TcpListener listener; + Uint16 port{ 0 }; + std::atomic running{ false }; + std::atomic authenticatedClients{ 0 }; + std::thread thread; + std::mutex mutex; + std::deque incoming; + std::deque outgoing; + Uint64 nextClient{ 1 }; + Uint64 nextSubscription{ 1 }; + std::map subscriptions; + std::vector> pending; + Uint64 topologyRevision{ 0 }; + unsigned topologyPollFrames{ 0 }; + std::map knownWindows; + std::map knownScenes; + + void enqueue( Uint64 client, const json& message ) { + std::string line = message.dump() + '\n'; + std::lock_guard lock( mutex ); + if ( outgoing.size() < 1024 ) + outgoing.push_back( { client, std::move( line ) } ); + } + + void broadcast( const json& message ) { enqueue( 0, message ); } + + void disconnect( Uint64 client ) { + for ( auto it = subscriptions.begin(); it != subscriptions.end(); ) { + if ( it->second.client == client ) + it = subscriptions.erase( it ); + else + ++it; + } + for ( auto& batch : pending ) { + if ( batch->client == client ) + batch->cancelled = true; + } + } + + void networkLoop() { + SocketSelector selector; + selector.add( listener ); + std::map clients; + while ( running ) { + { + std::lock_guard lock( mutex ); + while ( !outgoing.empty() ) { + auto item = std::move( outgoing.front() ); + outgoing.pop_front(); + for ( auto& pair : clients ) { + if ( ( item.client == 0 && pair.second.authenticated ) || + item.client == pair.first ) { + if ( pair.second.output.size() + item.line.size() > 4 * 1024 * 1024 ) + pair.second.closing = true; + else + pair.second.output += item.line; + } + } + } + } + selector.wait( Milliseconds( 10 ) ); + if ( selector.isReady( listener ) && clients.size() < 16 ) { + auto socket = std::make_unique(); + if ( listener.accept( *socket ) == Socket::Done ) { + socket->setBlocking( false ); + Uint64 id = nextClient++; + selector.add( *socket ); + clients.emplace( id, Client{ id, std::move( socket ) } ); + } + } + for ( auto it = clients.begin(); it != clients.end(); ) { + Client& client = it->second; + if ( selector.isReady( *client.socket ) ) { + char buffer[8192]; + size_t length = 0; + auto status = client.socket->receive( buffer, sizeof( buffer ), length ); + if ( status == Socket::Done ) { + client.input.append( buffer, length ); + if ( client.input.size() > 1024 * 1024 && + client.input.find( '\n' ) == std::string::npos ) { + client.output += + json( { { "id", nullptr }, + { "error", errorObject( { "request-too-large", + "Request line exceeds 1 MiB", + nullptr } ) } } ) + .dump() + + '\n'; + client.closing = true; + } + while ( !client.closing ) { + auto end = client.input.find( '\n' ); + if ( end == std::string::npos ) + break; + if ( end > 1024 * 1024 ) { + client.output += + json( { { "id", nullptr }, + { "error", errorObject( { "request-too-large", + "Request line exceeds 1 MiB", + nullptr } ) } } ) + .dump() + + '\n'; + client.closing = true; + break; + } + std::string line = client.input.substr( 0, end ); + client.input.erase( 0, end + 1 ); + if ( !line.empty() && line.back() == '\r' ) + line.pop_back(); + json request = json::parse( line, nullptr, false ); + if ( !client.authenticated ) { + json id = request.is_object() && request.contains( "id" ) + ? request["id"] + : json( nullptr ); + bool isConnect = request.is_object() && + request.contains( "method" ) && + request["method"].is_string() && + request["method"] == "session.connect"; + bool hasParams = isConnect && request.contains( "params" ) && + request["params"].is_object(); + bool tokenMatches = hasParams && + request["params"].contains( "token" ) && + request["params"]["token"].is_string() && + request["params"]["token"] == settings.token; + if ( !tokenMatches ) { + const char* code = request.is_discarded() ? "parse-error" + : isConnect ? "authentication-failed" + : "unauthenticated"; + const char* message = + request.is_discarded() ? "Malformed JSON" + : isConnect ? "Invalid inspector token" + : "First request must be session.connect"; + client.output += + json( { { "id", id }, + { "error", + errorObject( { code, message, nullptr } ) } } ) + .dump() + + '\n'; + client.closing = true; + break; + } + if ( !request["params"].contains( "protocolVersion" ) || + !request["params"]["protocolVersion"].is_number_integer() || + request["params"]["protocolVersion"] != 1 ) { + client.output += + json( { { "id", id }, + { "error", + errorObject( { "unsupported-protocol", + "Protocol version 1 required", + nullptr } ) } } ) + .dump() + + '\n'; + client.closing = true; + break; + } + client.authenticated = true; + authenticatedClients.fetch_add( 1 ); + } + std::lock_guard lock( mutex ); + if ( incoming.size() < 1024 ) + incoming.push_back( { client.id, std::move( line ) } ); + else + client.closing = true; + } + } else if ( status != Socket::NotReady ) { + client.closing = true; + } + } + if ( !client.output.empty() ) { + size_t sent = 0; + auto status = + client.socket->send( client.output.data(), client.output.size(), sent ); + client.output.erase( 0, sent ); + if ( status == Socket::Error || status == Socket::Disconnected ) { + client.closing = true; + client.output.clear(); + } + } + if ( client.closing && client.output.empty() ) { + if ( client.authenticated ) + authenticatedClients.fetch_sub( 1 ); + selector.remove( *client.socket ); + std::lock_guard lock( mutex ); + incoming.push_back( { client.id, {}, true } ); + it = clients.erase( it ); + } else { + ++it; + } + } + } + } + + UISceneNode* resolveScene( const json& params ) { + if ( !params.contains( "scene" ) ) { + auto* scene = inspector.defaultScene(); + if ( !scene ) + fail( "default-scene-unavailable", "Default scene was destroyed" ); + return scene; + } + auto* scene = inspector.scene( requireString( params, "scene" ) ); + if ( !scene ) + fail( "invalid-scene", "Scene handle is no longer valid" ); + return scene; + } + + UIWidget* resolveWidget( const json& params ) { + if ( params.contains( "handle" ) == params.contains( "selector" ) ) + fail( "invalid-params", "Specify exactly one of handle or selector" ); + if ( params.contains( "handle" ) ) { + if ( params.contains( "scene" ) ) + fail( "invalid-params", "Handle already determines scene" ); + auto* widget = inspector.widget( requireString( params, "handle" ) ); + if ( !widget ) + fail( "invalid-widget", "Widget handle is no longer valid" ); + return widget; + } + auto* scene = resolveScene( params ); + std::string selector = requireString( params, "selector" ); + if ( !validSelector( selector ) ) + fail( "selector-invalid", "Malformed selector" ); + auto matches = inspector.query( scene, selector ); + if ( matches.empty() ) + fail( "target-not-found", "Selector matched no widgets" ); + if ( matches.size() != 1 ) + fail( "target-ambiguous", "Selector matched multiple widgets", + { { "count", matches.size() } } ); + return matches.front(); + } + + json boundedText( const String& content, size_t maxLength, bool* truncated ) { + if ( content.size() > maxLength ) { + if ( truncated ) + *truncated = true; + return content.substr( 0, maxLength ).toUtf8(); + } + return content.toUtf8(); + } + + json textValue( UIWidget* widget, size_t maxLength, bool* truncated = nullptr ) { + if ( inspector.isSecret( widget ) ) + return "[redacted]"; + if ( widget->isType( UI_TYPE_TEXTVIEW ) ) + return boundedText( widget->asType()->getText(), maxLength, truncated ); + if ( widget->isType( UI_TYPE_TEXTNODE ) ) + return boundedText( widget->asType()->getText(), maxLength, truncated ); + if ( !widget->isType( UI_TYPE_RICHTEXT ) ) + return nullptr; + + String content; + auto append = [&]( const String& text ) { + if ( content.size() <= maxLength ) { + const size_t remaining = maxLength + 1 - content.size(); + content.append( text, 0, std::min( text.size(), remaining ) ); + } + }; + if ( widget->isType( UI_TYPE_TEXTSPAN ) ) + append( widget->asType()->getText() ); + + // Read HTML source text without forcing a RichText layout. Stop once the result is full or + // the walk becomes too large for a compact inspection response. + constexpr size_t maxVisitedNodes = 2048; + size_t visited = 0; + Scene::Node* node = widget->getFirstChild(); + while ( node && visited++ < maxVisitedNodes && content.size() <= maxLength ) { + bool descend = true; + if ( node->isWidget() ) { + auto* child = node->asType(); + if ( child->getUISceneNode() != widget->getUISceneNode() || + inspector.isSecret( child ) || child->isType( UI_TYPE_WEBVIEW ) || + child->isType( UI_TYPE_TEXTVIEW ) ) { + descend = false; + } else if ( child->isType( UI_TYPE_TEXTNODE ) ) { + append( child->asType()->getText() ); + descend = false; + } else if ( child->isType( UI_TYPE_TEXTSPAN ) ) { + append( child->asType()->getText() ); + } + } + Scene::Node* next = descend ? node->getFirstChild() : nullptr; + if ( !next ) { + while ( node != widget && !node->getNextNode() ) + node = node->getParent(); + next = node == widget ? nullptr : node->getNextNode(); + } + node = next; + } + if ( node || content.size() > maxLength ) { + if ( truncated ) + *truncated = true; + return content.substr( 0, maxLength ).toUtf8(); + } + return content.toUtf8(); + } + + json summary( UIWidget* widget, bool includeText = true ) { + auto* scene = widget->getUISceneNode(); + auto bounds = widget->getWorldBounds(); + json pseudoClasses = json::array(); + for ( const char* name : widget->getStyleSheetPseudoClassesStrings() ) + pseudoClasses.push_back( name ); + json value = { { "handle", inspector.widgetHandle( widget ) }, + { "scene", inspector.sceneHandle( scene ) }, + { "tag", widget->getElementTag() }, + { "id", widget->getId() }, + { "classes", widget->getClasses() }, + { "pseudoClasses", pseudoClasses }, + { "boundsPx", json::array( { bounds.Left, bounds.Top, bounds.getWidth(), + bounds.getHeight() } ) }, + { "visible", widget->isVisible() }, + { "enabled", widget->isEnabled() }, + { "focused", scene && scene->getEventDispatcher() && + scene->getEventDispatcher()->getFocusNode() == widget } }; + if ( widget->isType( UI_TYPE_WEBVIEW ) ) { + if ( auto* document = widget->asType()->getDocumentSceneNode() ) + value["documentScene"] = inspector.sceneHandle( document ); + } + if ( includeText ) { + bool truncated = false; + json content = textValue( widget, 256, &truncated ); + if ( !content.is_null() ) + value["text"] = std::move( content ); + if ( truncated ) + value["textTruncated"] = true; + } + return value; + } + + json readProperty( UIWidget* widget, const std::string& property, size_t maxStringLength = 4096, + bool* truncated = nullptr ) { + if ( property == "identity.tag" ) + return widget->getElementTag(); + if ( property == "identity.id" ) + return widget->getId(); + if ( property == "identity.classes" ) + return widget->getClasses(); + if ( property == "identity.pseudoClasses" ) + return widget->getStyleSheetPseudoClassesStrings(); + if ( property == "geometry.position" ) + return vector( widget->getPixelsPosition() ); + if ( property == "geometry.size" ) + return vector( widget->getPixelsSize() ); + if ( property == "geometry.worldPosition" ) { + auto bounds = widget->getWorldBounds(); + return json::array( { bounds.Left, bounds.Top } ); + } + if ( property == "geometry.bounds" ) { + auto bounds = widget->getWorldBounds(); + return json::array( + { bounds.Left, bounds.Top, bounds.getWidth(), bounds.getHeight() } ); + } + if ( property == "state.visible" ) + return widget->isVisible(); + if ( property == "state.enabled" ) + return widget->isEnabled(); + if ( property == "state.focused" ) { + auto* scene = widget->getUISceneNode(); + return scene && scene->getEventDispatcher() && + scene->getEventDispatcher()->getFocusNode() == widget; + } + if ( property == "content.text" ) { + return textValue( widget, maxStringLength, truncated ); + } + if ( property.compare( 0, 4, "css." ) == 0 ) { + std::string name = property.substr( 4 ); + if ( CSS::StyleSheetSpecification::instance()->getProperty( name ) ) + return inspector.property( widget, name ); + } + return nullptr; + } + + std::vector properties( UIWidget* widget, const json& params ) { + std::vector request; + if ( !params.contains( "properties" ) ) + return { "identity.*", "geometry.*", "state.*" }; + if ( !params["properties"].is_array() ) + fail( "invalid-params", "properties must be an array" ); + for ( const auto& name : params["properties"] ) { + if ( !name.is_string() ) + fail( "invalid-params", "property names must be strings" ); + request.push_back( name.get() ); + } + return request; + } + + json inspectProperties( UIWidget* widget, const std::vector& requested, + json& unavailable, json& truncated, size_t maxStringLength ) { + json values = json::object(); + std::vector names; + for ( const auto& name : requested ) { + if ( name == "*" || name == "identity.*" ) { + names.insert( names.end(), { "identity.tag", "identity.id", "identity.classes", + "identity.pseudoClasses" } ); + } + if ( name == "*" || name == "geometry.*" ) { + names.insert( names.end(), { "geometry.position", "geometry.worldPosition", + "geometry.size", "geometry.bounds" } ); + } + if ( name == "*" || name == "state.*" ) { + names.insert( names.end(), { "state.visible", "state.enabled", "state.focused" } ); + } + if ( name == "*" || name == "content.*" ) + names.push_back( "content.text" ); + if ( name == "*" || name == "css.*" ) { + for ( auto id : widget->getPropertiesImplemented() ) { + auto* definition = CSS::StyleSheetSpecification::instance()->getProperty( id ); + if ( definition ) + names.push_back( "css." + definition->getName() ); + } + } + if ( name != "*" && name.find( '*' ) == std::string::npos ) + names.push_back( name ); + } + for ( const auto& name : names ) { + bool wasTruncated = false; + json value = readProperty( widget, name, maxStringLength, &wasTruncated ); + if ( value.is_null() ) { + unavailable.push_back( name ); + } else { + if ( value.is_string() && + value.get_ref().size() > maxStringLength ) { + String content = String::fromUtf8( value.get() ); + if ( content.size() > maxStringLength ) { + value = content.substr( 0, maxStringLength ).toUtf8(); + wasTruncated = true; + } + } + if ( wasTruncated ) + truncated.push_back( name ); + values[name] = std::move( value ); + } + } + return values; + } + + json contexts( bool sendEvents = true ) { + json windows = json::array(); + json scenes = json::array(); + std::map newWindows; + std::map newScenes; + Window::Engine::instance()->forEachWindow( [&]( Window::Window* window ) { + auto size = window->getSize(); + auto pos = window->getPosition(); + json entry = { { "handle", inspector.windowHandle( window ) }, + { "id", window->getWindowID() }, + { "title", window->getTitle() }, + { "sizePx", json::array( { size.x, size.y } ) }, + { "positionPx", json::array( { pos.x, pos.y } ) }, + { "focused", window->hasFocus() } }; + newWindows.emplace( entry["handle"].get(), entry ); + windows.push_back( std::move( entry ) ); + } ); + SmallVector, 8> pending; + for ( auto* scene : inspector.topLevelScenes() ) + pending.emplace_back( scene, nullptr, nullptr ); + std::set seen; + while ( !pending.empty() ) { + auto [scene, parent, owner] = pending.back(); + pending.pop_back(); + if ( !scene || !seen.insert( scene ).second ) + continue; + if ( parent && !owner ) { + for ( auto* child = scene->getParent(); child; child = child->getParent() ) { + if ( child->isWidget() && + child->asType()->isType( UI_TYPE_WEBVIEW ) && + child->asType()->getDocumentSceneNode() == scene ) { + owner = child->asType(); + break; + } + } + } + json entry = { + { "handle", inspector.sceneHandle( scene ) }, + { "window", scene->getWindow() + ? json( inspector.windowHandle( scene->getWindow() ) ) + : json( nullptr ) }, + { "kind", !parent ? "top-level" + : owner && owner->isType( UI_TYPE_WEBVIEW ) ? "webview" + : "nested" }, + { "parentScene", + parent ? json( inspector.sceneHandle( parent ) ) : json( nullptr ) }, + { "owner", owner ? json( inspector.widgetHandle( owner ) ) : json( nullptr ) }, + { "root", scene->getRoot() ? json( inspector.widgetHandle( scene->getRoot() ) ) + : json( nullptr ) } }; + if ( owner && owner->isType( UI_TYPE_WEBVIEW ) ) + entry["uri"] = owner->asType()->getCurrentURI().toString(); + newScenes.emplace( entry["handle"].get(), entry ); + scenes.push_back( std::move( entry ) ); + for ( auto* nested : scene->getChildUISceneNodes() ) { + pending.emplace_back( nested, scene, nullptr ); + } + } + if ( sendEvents ) { + for ( const auto& [handle, value] : newWindows ) { + if ( !knownWindows.count( handle ) ) + broadcast( + { { "method", "ui.windowCreated" }, + { "params", + { { "revision", ++topologyRevision }, { "window", value } } } } ); + } + for ( const auto& [handle, value] : newScenes ) { + if ( !knownScenes.count( handle ) ) + broadcast( { { "method", "ui.sceneCreated" }, + { "params", + { { "revision", ++topologyRevision }, { "scene", value } } } } ); + } + for ( const auto& [handle, value] : knownScenes ) { + if ( !newScenes.count( handle ) ) + broadcast( + { { "method", "ui.sceneDestroyed" }, + { "params", + { { "revision", ++topologyRevision }, { "scene", handle } } } } ); + } + for ( const auto& [handle, value] : knownWindows ) { + if ( !newWindows.count( handle ) ) + broadcast( + { { "method", "ui.windowDestroyed" }, + { "params", + { { "revision", ++topologyRevision }, { "window", handle } } } } ); + } + } + knownWindows = std::move( newWindows ); + knownScenes = std::move( newScenes ); + std::string defaultScene = + inspector.defaultScene() ? inspector.sceneHandle( inspector.defaultScene() ) : ""; + json defaultWindow = + inspector.defaultScene() && inspector.defaultScene()->getWindow() + ? json( inspector.windowHandle( inspector.defaultScene()->getWindow() ) ) + : json( nullptr ); + return { { "revision", topologyRevision }, + { "defaultScene", defaultScene.empty() ? json( nullptr ) : json( defaultScene ) }, + { "defaultWindow", defaultWindow }, + { "windows", windows }, + { "scenes", scenes } }; + } + + json execute( Uint64 client, const std::string& method, const json& params ) { + if ( method == "session.connect" ) { + auto context = contexts( false ); + json capabilities = + json::array( { "ui.contexts", "ui.query", "ui.tree", "ui.inspect", "ui.focus", + "ui.screenshot", "session.batch", "session.nextFrame", + "watch.properties", "events.context-lifecycle" } ); + if ( !settings.readOnly ) { + capabilities.push_back( "input.click" ); + capabilities.push_back( "input.key" ); + capabilities.push_back( "input.text" ); + } + return { { "protocolVersion", 1 }, + { "application", + context["windows"].empty() ? "eepp" : context["windows"][0]["title"] }, + { "pid", Sys::getProcessID() }, + { "readOnly", settings.readOnly }, + { "defaultWindow", context["defaultWindow"] }, + { "defaultScene", context["defaultScene"] }, + { "capabilities", capabilities } }; + } + if ( method == "ui.contexts" ) + return contexts(); + if ( method == "ui.screenshot" ) { + if ( params.contains( "window" ) && params.contains( "scene" ) ) + fail( "invalid-params", "Specify window or scene, not both" ); + UISceneNode* scene = nullptr; + EE::Window::Window* window = nullptr; + if ( params.contains( "window" ) ) { + window = inspector.window( requireString( params, "window" ) ); + if ( !window ) + fail( "invalid-window", "Window handle is no longer valid" ); + } else { + scene = resolveScene( params ); + window = scene->getWindow(); + } + if ( !window ) + fail( "screenshot-failed", "Scene has no native window" ); + + std::string format = + params.contains( "format" ) ? requireString( params, "format" ) : "png"; + const auto saveType = Graphics::Image::extensionToSaveType( format ); + if ( saveType == Graphics::Image::SaveType::Unknown || + saveType == Graphics::Image::SaveType::DDS ) + fail( "invalid-params", "Unsupported screenshot format" ); + format = Graphics::Image::saveTypeToExtension( saveType ); + + const int width = static_cast( window->getWidth() ); + const int height = static_cast( window->getHeight() ); + if ( width <= 0 || height <= 0 ) + fail( "screenshot-failed", "Window has no drawable area" ); + int x = 0, y = 0, captureWidth = width, captureHeight = height; + if ( params.contains( "rect" ) ) { + const auto& rect = params["rect"]; + if ( !rect.is_array() || rect.size() != 4 ) + fail( "invalid-params", "rect must be [x,y,width,height]" ); + for ( const auto& value : rect ) { + if ( !value.is_number_integer() ) + fail( "invalid-params", "rect values must be integers" ); + } + const Int64 rx = rect[0].get(); + const Int64 ry = rect[1].get(); + const Int64 rw = rect[2].get(); + const Int64 rh = rect[3].get(); + if ( rx < 0 || ry < 0 || rw <= 0 || rh <= 0 || rx >= width || ry >= height || + rw > width - rx || rh > height - ry ) + fail( "invalid-params", "rect must fit inside the native window" ); + x = static_cast( rx ); + y = static_cast( ry ); + captureWidth = static_cast( rw ); + captureHeight = static_cast( rh ); + } else if ( params.contains( "scene" ) ) { + const auto& bounds = scene->getVisibleWorldBounds(); + const int left = + std::clamp( static_cast( std::floor( bounds.Left ) ), 0, width ); + const int top = + std::clamp( static_cast( std::floor( bounds.Top ) ), 0, height ); + const int right = + std::clamp( static_cast( std::ceil( bounds.Right ) ), 0, width ); + const int bottom = + std::clamp( static_cast( std::ceil( bounds.Bottom ) ), 0, height ); + x = left; + y = top; + captureWidth = right - left; + captureHeight = bottom - top; + if ( captureWidth <= 0 || captureHeight <= 0 ) + fail( "screenshot-failed", "Scene is outside the native window" ); + } + + const std::string suffix = tokenFromOS(); + if ( suffix.empty() ) + fail( "screenshot-failed", "Could not generate a temporary screenshot name" ); + const std::string path = Sys::getTempPath() + "eepp-inspector-" + + std::to_string( Sys::getProcessID() ) + "-" + suffix + "." + + format; + bool saved = false; + { + auto context = Engine::instance()->makeWindowCurrent( window ); + window->clear(); + SceneManager::instance()->draw( window ); + if ( x == 0 && y == 0 && captureWidth == width && captureHeight == height ) { + saved = window->takeScreenshot( path, saveType ); + } else { + auto image = window->getFrontBufferImage(); + auto* cropped = image.crop( Rect( x, y, x + captureWidth, y + captureHeight ) ); + if ( cropped ) { + saved = cropped->saveToFile( path, saveType ); + eeDelete( cropped ); + } + } + } + // The application still needs to present the frame on its normal draw path. + SceneManager::instance()->forEachSceneNode( [&]( SceneNode* node ) { + if ( node->getWindow() == window ) + node->invalidate( nullptr ); + } ); + if ( !saved ) { + FileSystem::fileRemove( path ); + fail( "screenshot-failed", "Could not capture or save the screenshot" ); + } + return { { "path", path }, + { "format", format }, + { "window", inspector.windowHandle( window ) }, + { "scene", scene ? json( inspector.sceneHandle( scene ) ) : json( nullptr ) }, + { "rectPx", json::array( { x, y, captureWidth, captureHeight } ) }, + { "sizePx", json::array( { captureWidth, captureHeight } ) } }; + } + if ( method == "ui.query" ) { + auto* scene = resolveScene( params ); + std::string selector = requireString( params, "selector" ); + if ( !validSelector( selector ) ) + fail( "selector-invalid", "Malformed selector" ); + auto widgets = inspector.query( scene, selector ); + unsigned offset = boundedUnsigned( params, "offset", 0, 1000000 ); + unsigned limit = boundedUnsigned( params, "limit", 50, 1000 ); + unsigned maxStringLength = boundedUnsigned( params, "maxStringLength", 4096, 65536 ); + json nodes = json::array(); + for ( size_t i = offset; i < widgets.size() && nodes.size() < limit; ++i ) { + json node = summary( widgets[i] ); + if ( params.contains( "properties" ) ) { + json unavailable = json::array(); + json truncated = json::array(); + node["properties"] = + inspectProperties( widgets[i], properties( widgets[i], params ), + unavailable, truncated, maxStringLength ); + if ( !unavailable.empty() ) + node["unavailable"] = unavailable; + if ( !truncated.empty() ) + node["truncatedProperties"] = truncated; + } + nodes.push_back( std::move( node ) ); + } + return { { "scene", inspector.sceneHandle( scene ) }, + { "selector", selector }, + { "total", widgets.size() }, + { "offset", offset }, + { "returned", nodes.size() }, + { "truncated", offset + nodes.size() < widgets.size() }, + { "nodes", nodes } }; + } + if ( method == "ui.inspect" ) { + auto* widget = resolveWidget( params ); + json unavailable = json::array(); + json truncated = json::array(); + unsigned maxStringLength = boundedUnsigned( params, "maxStringLength", 4096, 65536 ); + json values = inspectProperties( widget, properties( widget, params ), unavailable, + truncated, maxStringLength ); + json result = { { "handle", inspector.widgetHandle( widget ) }, + { "scene", inspector.sceneHandle( widget->getUISceneNode() ) }, + { "properties", std::move( values ) }, + { "unavailable", unavailable } }; + if ( !truncated.empty() ) + result["truncatedProperties"] = truncated; + return result; + } + if ( method == "ui.focus" ) { + auto* scene = resolveScene( params ); + auto* focus = + scene->getEventDispatcher() ? scene->getEventDispatcher()->getFocusNode() : nullptr; + return { { "scene", inspector.sceneHandle( scene ) }, + { "widget", focus && focus->isWidget() && + focus->asType()->getUISceneNode() == scene + ? json( summary( focus->asType(), false ) ) + : json( nullptr ) } }; + } + if ( method == "ui.tree" ) { + UIWidget* root = nullptr; + UISceneNode* scene = nullptr; + if ( params.contains( "root" ) ) { + if ( params.contains( "scene" ) ) + fail( "invalid-params", "root determines scene" ); + root = inspector.widget( requireString( params, "root" ) ); + if ( !root ) + fail( "invalid-widget", "Tree root no longer exists" ); + scene = root->getUISceneNode(); + } else { + scene = resolveScene( params ); + root = scene->getRoot(); + } + unsigned depth = boundedUnsigned( params, "depth", 3, 32 ); + unsigned maxNodes = boundedUnsigned( params, "maxNodes", 200, 1000 ); + bool includeText = optionalBool( params, "includeText", false ); + json nodes = json::array(); + SmallVector, 8> stack{ + { root, nullptr, 0 } }; + while ( !stack.empty() && nodes.size() < maxNodes ) { + auto [widget, parent, level] = stack.back(); + stack.pop_back(); + if ( !widget || widget->getUISceneNode() != scene ) + continue; + json node = summary( widget, includeText ); + node.erase( "boundsPx" ); + node.erase( "visible" ); + node.erase( "enabled" ); + node.erase( "focused" ); + node["parent"] = + parent ? json( inspector.widgetHandle( parent ) ) : json( nullptr ); + node["children"] = json::array(); + if ( level < depth ) { + SmallVector children; + for ( auto* child = widget->getFirstChild(); child; + child = child->getNextNode() ) { + if ( child->isWidget() && + child->asType()->getUISceneNode() == scene ) { + auto* childWidget = child->asType(); + children.push_back( childWidget ); + node["children"].push_back( inspector.widgetHandle( childWidget ) ); + } + } + for ( auto it = children.rbegin(); it != children.rend(); ++it ) + stack.emplace_back( *it, widget, level + 1 ); + } + nodes.push_back( std::move( node ) ); + } + return { { "scene", inspector.sceneHandle( scene ) }, + { "root", inspector.widgetHandle( root ) }, + { "truncated", !stack.empty() }, + { "nodes", nodes } }; + } + if ( method == "watch.subscribe" ) { + if ( !params.contains( "properties" ) || !params["properties"].is_array() || + params["properties"].empty() ) + fail( "invalid-params", "Explicit properties are required" ); + WidgetQueryResult targets; + if ( params.contains( "handle" ) ) + targets.push_back( resolveWidget( params ) ); + else { + std::string selector = requireString( params, "selector" ); + if ( !validSelector( selector ) ) + fail( "selector-invalid", "Malformed selector" ); + targets = inspector.query( resolveScene( params ), selector ); + } + if ( targets.empty() ) + fail( "target-not-found", "No watch targets" ); + bool initial = optionalBool( params, "initial", true ); + std::string handle = "sub:" + std::to_string( nextSubscription++ ); + Subscription sub{ client }; + json resultTargets = json::array(); + for ( auto* widget : targets ) { + json values = json::object(); + for ( const auto& property : params["properties"] ) { + if ( !property.is_string() || + property.get().find( '*' ) != std::string::npos ) + fail( "invalid-params", "Watches require explicit property names" ); + std::string name = property.get(); + json value = readProperty( widget, name ); + if ( value.is_null() ) + fail( "property-unknown", "Unavailable watch property: " + name ); + values[name] = value; + sub.values[inspector.widgetHandle( widget )][name] = std::move( value ); + } + json target = { { "handle", inspector.widgetHandle( widget ) } }; + if ( initial ) + target["values"] = std::move( values ); + resultTargets.push_back( std::move( target ) ); + } + subscriptions.emplace( handle, std::move( sub ) ); + return { { "subscription", handle }, { "targets", resultTargets } }; + } + if ( method == "watch.unsubscribe" ) { + std::string handle = requireString( params, "subscription" ); + auto it = subscriptions.find( handle ); + if ( it == subscriptions.end() || it->second.client != client ) + fail( "invalid-params", "Unknown subscription" ); + subscriptions.erase( it ); + return { { "unsubscribed", true } }; + } + if ( method.compare( 0, 6, "input." ) == 0 ) { + if ( settings.readOnly ) + fail( "permission-denied", "Inspector is read-only" ); + UISceneNode* scene = nullptr; + UIWidget* target = nullptr; + if ( method == "input.click" ) { + target = resolveWidget( params ); + scene = target->getUISceneNode(); + if ( !target->hasVisibility() || !target->isEnabled() ) + fail( "target-not-interactable", "Target is invisible or disabled" ); + } else { + scene = resolveScene( params ); + } + if ( !scene || !scene->getWindow() || !scene->getEventDispatcher() ) + fail( "input-unavailable", "Scene has no input context" ); + auto context = scene->makeCurrent(); + auto* window = scene->getWindow(); + auto* input = window->getInput(); + if ( !input ) + fail( "input-unavailable", "Window has no input" ); + const std::string sceneHandle = inspector.sceneHandle( scene ); + const std::string windowHandle = inspector.windowHandle( window ); + if ( method == "input.click" ) { + const std::string targetHandle = inspector.widgetHandle( target ); + if ( params.contains( "button" ) && params["button"] != "left" ) + fail( "invalid-params", "Only left click is supported" ); + unsigned count = boundedUnsigned( params, "count", 1, 2 ); + auto bounds = target->getWorldBounds(); + Vector2i point( static_cast( ( bounds.Left + bounds.Right ) * 0.5f ), + static_cast( ( bounds.Top + bounds.Bottom ) * 0.5f ) ); + auto windowSize = window->getSize(); + if ( bounds.getWidth() <= 0 || bounds.getHeight() <= 0 || point.x < 0 || + point.y < 0 || point.x >= windowSize.x || point.y >= windowSize.y ) + fail( "target-not-interactable", "Target is outside the window" ); + auto* hit = + scene->getEventDispatcher()->getSceneNode()->overFind( point.asFloat() ); + bool hitsTarget = false; + for ( auto* node = hit; node; node = node->getParent() ) { + if ( node == target ) { + hitsTarget = true; + break; + } + } + if ( !hitsTarget ) + fail( "target-not-interactable", "Target is clipped or covered" ); + InputEvent motion( InputEvent::MouseMotion ); + motion.WinID = window->getWindowID(); + motion.motion = {}; + motion.motion.x = point.x; + motion.motion.y = point.y; + input->pushEvent( motion ); + for ( unsigned i = 0; i < count; ++i ) { + if ( !inspector.scene( sceneHandle ) || !inspector.window( windowHandle ) || + !inspector.widget( targetHandle ) ) + fail( "target-not-interactable", "Target disappeared during click" ); + InputEvent press( InputEvent::MouseButtonDown ); + press.WinID = window->getWindowID(); + press.button = {}; + press.button.button = EE_BUTTON_LEFT; + press.button.x = point.x; + press.button.y = point.y; + input->pushEvent( press ); + if ( inspector.scene( sceneHandle ) && inspector.window( windowHandle ) ) + scene->getEventDispatcher()->update( Time::Zero ); + else + fail( "input-unavailable", "Input context closed during click" ); + InputEvent release( InputEvent::MouseButtonUp ); + release.WinID = window->getWindowID(); + release.button = press.button; + input->pushEvent( release ); + if ( inspector.scene( sceneHandle ) && inspector.window( windowHandle ) ) + scene->getEventDispatcher()->update( Time::Zero ); + } + return { { "target", targetHandle }, + { "scene", sceneHandle }, + { "pointPx", json::array( { point.x, point.y } ) } }; + } + if ( method == "input.key" ) { + std::string name = requireString( params, "key" ); + Keycode key = input->getKeyFromName( name == "Enter" ? "Return" : name ); + if ( key == 0 ) + fail( "invalid-params", "Unknown key name" ); + if ( params.contains( "action" ) && !params["action"].is_string() ) + fail( "invalid-params", "action must be a string" ); + std::string action = params.value( "action", "press" ); + if ( action != "press" && action != "down" && action != "up" ) + fail( "invalid-params", "Unknown key action" ); + Uint32 modifiers = 0; + if ( params.contains( "modifiers" ) ) { + if ( !params["modifiers"].is_array() ) + fail( "invalid-params", "modifiers must be array" ); + for ( const auto& mod : params["modifiers"] ) { + if ( mod == "Ctrl" ) + modifiers |= KEYMOD_LCTRL; + else if ( mod == "Shift" ) + modifiers |= KEYMOD_LSHIFT; + else if ( mod == "Alt" ) + modifiers |= KEYMOD_LALT; + else if ( mod == "Meta" ) + modifiers |= KEYMOD_LMETA; + else + fail( "invalid-params", "Unknown modifier" ); + } + } + auto sendKey = [&]( InputEvent::EventType type ) { + InputEvent event( type ); + event.WinID = window->getWindowID(); + event.key = {}; + event.key.keysym.sym = key; + event.key.keysym.scancode = input->getScancodeFromKey( key ); + event.key.keysym.mod = modifiers; + input->pushEvent( event ); + }; + if ( action != "up" ) + sendKey( InputEvent::KeyDown ); + if ( action != "down" && inspector.window( windowHandle ) ) + sendKey( InputEvent::KeyUp ); + return { { "key", name }, { "action", action } }; + } + if ( method == "input.text" ) { + std::string content = requireString( params, "text" ); + if ( content.size() > 65536 ) + fail( "invalid-params", "Text exceeds 64 KiB" ); + for ( auto character : String::fromUtf8( content ) ) { + if ( !inspector.window( windowHandle ) ) + fail( "input-unavailable", "Window closed during text input" ); + InputEvent event( InputEvent::TextInput ); + event.WinID = window->getWindowID(); + event.text = {}; + event.text.text = character; + input->pushEvent( event ); + } + return { { "characters", String::fromUtf8( content ).size() } }; + } + } + fail( "method-not-found", "Unknown inspector method: " + method ); + return nullptr; + } + + bool resolveReferences( json& value, const std::map& named ) { + if ( value.is_object() && value.size() == 1 && value.contains( "$ref" ) ) { + if ( !value["$ref"].is_string() ) + fail( "batch-reference-error", "Reference must be a string" ); + std::string reference = value["$ref"].get(); + auto hash = reference.find( '#' ); + if ( hash == std::string::npos ) + fail( "batch-reference-error", "Reference needs # pointer" ); + auto it = named.find( reference.substr( 0, hash ) ); + if ( it == named.end() ) + fail( "batch-reference-error", "Unknown or forward command reference" ); + try { + value = it->second.at( json::json_pointer( reference.substr( hash + 1 ) ) ); + } catch ( const json::exception& ) { + fail( "batch-reference-error", "Invalid result pointer" ); + } + return true; + } else if ( value.is_array() || value.is_object() ) { + bool referenced = false; + for ( auto& item : value ) + referenced |= resolveReferences( item, named ); + return referenced; + } + return false; + } + + void advanceBatch( const std::shared_ptr& batch ) { + while ( !batch->cancelled && batch->ready && batch->index < batch->commands.size() ) { + size_t index = batch->index; + const json& original = batch->commands[index]; + std::string name; + try { + if ( !original.is_object() || !original.contains( "method" ) || + !original["method"].is_string() ) + fail( "invalid-params", "Batch command needs method" ); + if ( original.contains( "name" ) && !original["name"].is_string() ) + fail( "invalid-params", "Batch command name must be a string" ); + name = original.value( "name", "" ); + if ( !name.empty() && batch->named.count( name ) ) + fail( "invalid-params", "Duplicate command name" ); + std::string method = original["method"].get(); + if ( method == "session.batch" || method == "session.connect" ) + fail( "invalid-params", "Nested batch/connect forbidden" ); + json params = original.value( "params", json::object() ); + if ( !params.is_object() ) + fail( "invalid-params", "Command params must be object" ); + bool referenced = resolveReferences( params, batch->named ); + Time duration = Time::Zero; + if ( original.contains( "delay" ) ) { + std::string delay = requireString( original, "delay" ); + if ( !validDelay( delay ) ) + fail( "invalid-params", "Invalid delay" ); + duration = Time::fromString( delay ); + if ( duration.asMilliseconds() < 0 || duration.asMilliseconds() > 60000 ) + fail( "invalid-params", "Delay exceeds 60 seconds" ); + } + json result; + if ( method == "session.nextFrame" ) { + unsigned count = boundedUnsigned( params, "count", 1, 10 ); + if ( count == 0 ) + fail( "invalid-params", "Frame count must be positive" ); + batch->frames = count + 1; + result = { { "frames", count } }; + } else { + try { + result = execute( batch->client, method, params ); + } catch ( const InspectorError& error ) { + if ( referenced && error.code == "invalid-params" ) + fail( "batch-reference-error", "Referenced value is incompatible" ); + throw; + } + } + if ( !name.empty() ) + batch->named.emplace( name, result ); + if ( optionalBool( original, "return", true ) ) + batch->results[name.empty() ? std::to_string( index ) : name] = result; + ++batch->index; + if ( original.contains( "delay" ) ) { + batch->ready = false; + UISceneNode* scene = inspector.defaultScene(); + if ( !scene ) { + auto scenes = inspector.topLevelScenes(); + if ( !scenes.empty() ) + scene = scenes.front(); + } + if ( scene ) { + scene->getRoot()->setTimeout( [batch]() { batch->ready = true; }, + duration ); + } else { + fail( "default-scene-unavailable", "Cannot schedule delay" ); + } + break; + } + if ( batch->frames ) { + batch->ready = false; + break; + } + } catch ( const InspectorError& error ) { + enqueue( batch->client, + { { "id", batch->id }, + { "error", + errorObject( { "batch-command-failed", + "Batch command failed", + { { "index", index }, + { "name", name }, + { "commandError", errorObject( error ) } } } ) } } ); + batch->cancelled = true; + } + } + if ( !batch->cancelled && batch->index == batch->commands.size() && batch->ready && + batch->frames == 0 ) { + enqueue( + batch->client, + { { "id", batch->id }, + { "result", batch->standalone ? batch->results.value( "0", json::object() ) + : json( { { "results", batch->results } } ) } } ); + batch->cancelled = true; + } + } + + void handleRequest( Uint64 client, const std::string& line ) { + json request = json::parse( line, nullptr, false ); + json id = request.is_object() && request.contains( "id" ) ? request["id"] : json( nullptr ); + try { + if ( request.is_discarded() ) + fail( "parse-error", "Malformed JSON" ); + if ( !request.is_object() || ( !id.is_number_unsigned() && + !( id.is_number_integer() && id.get() >= 0 ) ) ) + fail( "invalid-request", "Request needs a non-negative integer id" ); + if ( !request.contains( "method" ) || !request["method"].is_string() ) + fail( "invalid-request", "Request needs method" ); + json params = request.value( "params", json::object() ); + if ( !params.is_object() ) + fail( "invalid-request", "params must be an object" ); + std::string method = request["method"].get(); + if ( request.contains( "delay" ) && method != "session.batch" ) { + json command = request; + command.erase( "id" ); + auto batch = std::make_shared(); + batch->client = client; + batch->id = id; + batch->commands = json::array( { command } ); + batch->standalone = true; + pending.push_back( batch ); + advanceBatch( batch ); + return; + } + if ( method == "session.batch" ) { + if ( !params.contains( "commands" ) || !params["commands"].is_array() || + params["commands"].size() > 100 ) + fail( "invalid-params", "Batch requires at most 100 commands" ); + auto batch = std::make_shared(); + batch->client = client; + batch->id = id; + batch->commands = params["commands"]; + pending.push_back( batch ); + advanceBatch( batch ); + return; + } + if ( method == "session.nextFrame" ) { + auto batch = std::make_shared(); + batch->client = client; + batch->id = id; + batch->commands = json::array( { { { "method", method }, { "params", params } } } ); + batch->standalone = true; + pending.push_back( batch ); + advanceBatch( batch ); + return; + } + json result = execute( client, method, params ); + enqueue( client, { { "id", id }, { "result", result } } ); + } catch ( const InspectorError& error ) { + enqueue( client, { { "id", id }, { "error", errorObject( error ) } } ); + } catch ( const std::exception& ) { + enqueue( client, + { { "id", id }, + { "error", errorObject( { "internal-error", "Inspector command failed", + nullptr } ) } } ); + } + } + + void updateWatches() { + for ( auto it = subscriptions.begin(); it != subscriptions.end(); ) { + auto& sub = it->second; + for ( auto target = sub.values.begin(); target != sub.values.end(); ) { + auto* widget = inspector.widget( target->first ); + if ( !widget ) { + enqueue( sub.client, { { "method", "watch.targetRemoved" }, + { "params", + { { "subscription", it->first }, + { "target", target->first }, + { "reason", "target-destroyed" } } } } ); + target = sub.values.erase( target ); + continue; + } + json changes = json::object(); + for ( auto& [name, old] : target->second ) { + json current = readProperty( widget, name ); + if ( current != old ) { + changes[name] = { { "old", old }, { "new", current } }; + old = std::move( current ); + } + } + if ( !changes.empty() ) + enqueue( sub.client, { { "method", "watch.changed" }, + { "params", + { { "subscription", it->first }, + { "sequence", ++sub.sequence }, + { "target", target->first }, + { "changes", changes } } } } ); + ++target; + } + if ( sub.values.empty() ) { + enqueue( sub.client, + { { "method", "watch.ended" }, + { "params", + { { "subscription", it->first }, { "reason", "no-targets" } } } } ); + it = subscriptions.erase( it ); + } else + ++it; + } + } + + void pump() { + std::deque work; + { + std::lock_guard lock( mutex ); + work.swap( incoming ); + } + for ( const auto& item : work ) { + if ( item.disconnected ) + disconnect( item.client ); + else + handleRequest( item.client, item.line ); + } + for ( auto& batch : pending ) { + if ( batch->cancelled ) + continue; + if ( batch->frames ) { + if ( --batch->frames == 0 ) { + batch->ready = true; + advanceBatch( batch ); + } + } else if ( batch->ready ) { + advanceBatch( batch ); + } + } + pending.erase( std::remove_if( pending.begin(), pending.end(), + []( const auto& item ) { return item->cancelled; } ), + pending.end() ); + if ( !subscriptions.empty() ) + updateWatches(); + if ( authenticatedClients.load() && ++topologyPollFrames >= 30 ) { + topologyPollFrames = 0; + contexts(); + } + } +}; + +UIInspectorServer* UIInspectorServer::sInstance = nullptr; + +UIInspectorServer::UIInspectorServer() : mImpl( std::make_unique() ) {} + +UIInspectorServer::~UIInspectorServer() = default; + +UIInspectorServer* UIInspectorServer::instance() { + return sInstance; +} + +bool UIInspectorServer::start( UISceneNode* defaultScene, UIInspectorServerSettings settings ) { + if ( sInstance ) + return false; + auto server = std::unique_ptr( new UIInspectorServer() ); + bool generated = settings.token.empty(); + if ( generated ) + settings.token = tokenFromOS(); + if ( settings.token.empty() ) + return false; + server->mImpl->settings = std::move( settings ); + server->mImpl->inspector.setDefaultScene( defaultScene ); + if ( server->mImpl->listener.listen( server->mImpl->settings.port, + IpAddress( server->mImpl->settings.address ) ) != + Socket::Done ) + return false; + server->mImpl->port = server->mImpl->listener.getLocalPort(); + server->mImpl->listener.setBlocking( false ); + server->mImpl->running = true; + server->mImpl->thread = std::thread( [&impl = *server->mImpl]() { impl.networkLoop(); } ); + json descriptor = { { "protocolVersion", 1 }, + { "host", server->mImpl->settings.address }, + { "port", server->mImpl->port }, + { "pid", Sys::getProcessID() } }; + if ( generated ) + descriptor["token"] = server->mImpl->settings.token; + std::fprintf( stderr, "EEPP_INSPECTOR=%s\n", descriptor.dump().c_str() ); + if ( server->mImpl->settings.address != "127.0.0.1" && + server->mImpl->settings.address != "::1" ) + std::fprintf( stderr, "EEPP_INSPECTOR_WARNING=non-loopback raw TCP is plaintext\n" ); + sInstance = server.release(); + return true; +} + +void UIInspectorServer::startFromEnvironment( UISceneNode* defaultScene ) { + if ( sInstance || !envEnabled( std::getenv( "EEPP_INSPECTOR" ) ) ) + return; + UIInspectorServerSettings settings; + if ( const char* address = std::getenv( "EEPP_INSPECTOR_ADDRESS" ) ) + settings.address = address; + if ( const char* port = std::getenv( "EEPP_INSPECTOR_PORT" ) ) { + char* end = nullptr; + unsigned long parsed = std::strtoul( port, &end, 10 ); + if ( !*port || *end || parsed > 65535 ) { + std::fprintf( stderr, "EEPP_INSPECTOR_ERROR=invalid port\n" ); + return; + } + settings.port = static_cast( parsed ); + } + settings.readOnly = envEnabled( std::getenv( "EEPP_INSPECTOR_READ_ONLY" ) ); + if ( const char* token = std::getenv( "EEPP_INSPECTOR_TOKEN" ) ) + settings.token = token; + start( defaultScene, std::move( settings ) ); +} + +void UIInspectorServer::stop() { + if ( !sInstance ) + return; + auto* server = sInstance; + sInstance = nullptr; + server->mImpl->running = false; + if ( server->mImpl->thread.joinable() ) + server->mImpl->thread.join(); + server->mImpl->listener.close(); + delete server; +} + +void UIInspectorServer::pump() { + if ( sInstance ) + sInstance->mImpl->pump(); +} + +void UIInspectorServer::notifyWindowDestroyed( EE::Window::Window* window ) { + if ( sInstance ) + sInstance->mImpl->inspector.invalidateWindow( window ); +} + +bool UIInspectorServer::isRunning() const { + return mImpl->running; +} + +const std::string& UIInspectorServer::getAddress() const { + return mImpl->settings.address; +} + +Uint16 UIInspectorServer::getPort() const { + return mImpl->port; +} + +const std::string& UIInspectorServer::getToken() const { + return mImpl->settings.token; +} + +}} // namespace EE::UI diff --git a/src/eepp/ui/uihtmlwidget.cpp b/src/eepp/ui/uihtmlwidget.cpp index 03d7d4203..1be3fd929 100644 --- a/src/eepp/ui/uihtmlwidget.cpp +++ b/src/eepp/ui/uihtmlwidget.cpp @@ -731,6 +731,11 @@ void UIHTMLWidget::collectStackingScopeItems( UIHTMLWidget* container, SmallVect void UIHTMLWidget::collectStackingScopeChild( Node* child, SmallVector& out, bool directChildren ) const { + // A promoted positioned descendant must not escape a hidden ancestor's + // subtree. The normal node traversal stops at that ancestor, while this + // flattened paint list would otherwise visit and paint its descendants. + if ( !child->isVisible() ) + return; if ( directChildren ) out.push_back( child ); if ( !child->isType( UI_TYPE_HTML_WIDGET ) ) diff --git a/src/eepp/ui/uiscenenode.cpp b/src/eepp/ui/uiscenenode.cpp index b76656ca8..8d5eb878d 100644 --- a/src/eepp/ui/uiscenenode.cpp +++ b/src/eepp/ui/uiscenenode.cpp @@ -206,6 +206,7 @@ UISceneNode::UISceneNode( EE::Window::Window* window, bool importDefaultResource } UISceneNode::~UISceneNode() { + onClose(); if ( mAsyncResourceLoadState ) { mAsyncResourceLoadState->owner.store( nullptr, std::memory_order_release ); mAsyncResourceLoadState->alive.store( false, std::memory_order_release ); @@ -422,8 +423,8 @@ UISceneNode* UISceneNode::setSmoothScrollEnabled( bool enabled, bool applyNow ) rootScene->mSmoothScrollEnabled = enabled; if ( applyNow ) { const auto applyToScene = [enabled]( auto&& self, UISceneNode* scene ) -> void { - for ( auto* widget : scene->findAllByType( - UI_TYPE_TOUCH_DRAGGABLE_WIDGET ) ) + for ( auto* widget : + scene->findAllByType( UI_TYPE_TOUCH_DRAGGABLE_WIDGET ) ) widget->setSmoothScrollEnabled( enabled ); for ( auto* childScene : scene->mChildUISceneNodes ) self( self, childScene ); diff --git a/src/eepp/window/engine.cpp b/src/eepp/window/engine.cpp index d2a8d482d..a4730673d 100644 --- a/src/eepp/window/engine.cpp +++ b/src/eepp/window/engine.cpp @@ -20,6 +20,7 @@ #include #include #include +#include #include #include #include @@ -348,6 +349,7 @@ EE::Window::Window* Engine::createWindow( WindowSettings Settings, ContextSettin } void Engine::destroyWindow( EE::Window::Window* window ) { + UIInspectorServer::notifyWindowDestroyed( window ); mWindows.erase( window->getWindowID() ); if ( window == mWindow ) { diff --git a/src/tests/unit_tests/uihtml_flex_test.cpp b/src/tests/unit_tests/uihtml_flex_test.cpp index d8d2e7599..383dc3651 100644 --- a/src/tests/unit_tests/uihtml_flex_test.cpp +++ b/src/tests/unit_tests/uihtml_flex_test.cpp @@ -1533,6 +1533,36 @@ UTEST( FlexContainer, iterativeFlexResolutionWithMinWidths ) { Engine::destroySingleton(); } +UTEST( FlexContainer, explicitZeroMinWidthOverridesIntrinsicMinimum ) { + Engine::instance()->createWindow( WindowSettings( 1024, 650, "Flex explicit minimum test", + WindowStyle::Default, WindowBackend::Default, + 32, {}, 1, false, true ), + ContextSettings( false, 0, 0, GLv_default, true, false ) ); + init_flex_test(); + auto* scene = SceneManager::instance()->getUISceneNode(); + scene->loadLayoutFromString( Tools::HTMLFormatter::HTMLtoXML( R"html( + +
Text
+
+
+ + )html" ) ); + scene->updateDirtyLayouts(); + auto* row = scene->getRoot()->findByClass( "row" )->asType(); + auto* screenshot = scene->getRoot()->findByClass( "screenshot" )->asType(); + ASSERT_TRUE( row != nullptr && screenshot != nullptr ); + EXPECT_LE( screenshot->getPixelsPosition().x + screenshot->getPixelsSize().getWidth(), + row->getPixelsSize().getWidth() - 48.f + 2.f ); + EXPECT_NEAR( screenshot->getPixelsSize().getWidth(), 520.f, 5.f ); + Engine::destroySingleton(); +} + UTEST( FlexContainer, crossAxisAutoMargins ) { // G4: margin: auto on the cross axis should center/position the item // within the line before align-self applies (ยง8.1). diff --git a/src/tests/unit_tests/uihtml_float_tests.cpp b/src/tests/unit_tests/uihtml_float_tests.cpp index e402df515..e4c3f82a9 100644 --- a/src/tests/unit_tests/uihtml_float_tests.cpp +++ b/src/tests/unit_tests/uihtml_float_tests.cpp @@ -246,6 +246,31 @@ UTEST( UIHTMLFloat, positionedDescendantParticipatesInRootStackingScope ) { Engine::destroySingleton(); } +UTEST( UIHTMLFloat, hiddenAncestorExcludesPromotedDescendant ) { + init_float_test(); + auto* scene = SceneManager::instance()->getUISceneNode(); + auto* root = UIHTMLWidget::New(); + root->setParent( scene->getRoot() ); + root->setPixelsSize( 300, 200 ); + auto* hidden = UIHTMLWidget::New(); + hidden->setParent( root ); + hidden->setPixelsSize( 300, 200 ); + auto* promoted = UIHTMLWidget::New(); + promoted->setParent( hidden ); + promoted->setPixelsSize( 200, 100 ); + promoted->setCSSPosition( CSSPosition::Absolute ); + promoted->setZIndex( 10 ); + ASSERT_EQ( root->debugGetHTMLPaintOrder().size(), 2u ); + hidden->setDisplay( CSSDisplay::None ); + const auto hiddenOrder = root->debugGetHTMLPaintOrder(); + EXPECT_TRUE( hiddenOrder.empty() ); + EXPECT_FALSE( promoted->isHTMLPaintPromoted() ); + EXPECT_NE( root->overFind( { 10, 10 } ), promoted ); + hidden->setDisplay( CSSDisplay::Block ); + EXPECT_EQ( root->debugGetHTMLPaintOrder().size(), 2u ); + Engine::destroySingleton(); +} + UTEST( UIHTMLFloat, directStackingGroupIsAtomic ) { init_float_test(); UISceneNode* sceneNode = SceneManager::instance()->getUISceneNode(); diff --git a/src/tests/unit_tests/uiinspector_tests.cpp b/src/tests/unit_tests/uiinspector_tests.cpp new file mode 100644 index 000000000..56777f41d --- /dev/null +++ b/src/tests/unit_tests/uiinspector_tests.cpp @@ -0,0 +1,782 @@ +#include "utest.hpp" + +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include + +using namespace EE; +using namespace EE::Network; +using namespace EE::Scene; +using namespace EE::UI; +using namespace EE::Window; +using json = nlohmann::json; + +namespace { + +class InspectorClient { + public: + InspectorClient( Uint16 port ) { + connected = socket.connect( IpAddress( "127.0.0.1" ), port, Seconds( 2 ) ) == Socket::Done; + if ( connected ) + socket.setBlocking( false ); + } + + void send( const json& request ) { sendBytes( request.dump() + "\n" ); } + + void sendBytes( const std::string& bytes ) { + socket.setBlocking( true ); + socket.send( bytes.data(), bytes.size() ); + socket.setBlocking( false ); + } + + json receive() { + using namespace std::chrono; + auto deadline = steady_clock::now() + seconds( 3 ); + while ( steady_clock::now() < deadline ) { + auto end = input.find( '\n' ); + if ( end != std::string::npos ) { + std::string line = input.substr( 0, end ); + input.erase( 0, end + 1 ); + return json::parse( line ); + } + Engine::instance()->updateInput(); + SceneManager::instance()->update(); + char bytes[4096]; + size_t length = 0; + if ( socket.receive( bytes, sizeof( bytes ), length ) == Socket::Done ) + input.append( bytes, length ); + std::this_thread::sleep_for( milliseconds( 1 ) ); + } + return nullptr; + } + + bool connected{ false }; + TcpSocket socket; + std::string input; +}; + +bool connectInspector( InspectorClient& client, const char* token ) { + client.send( { { "id", 1 }, + { "method", "session.connect" }, + { "params", { { "protocolVersion", 1 }, { "token", token } } } } ); + json reply = client.receive(); + return reply.is_object() && reply.contains( "result" ); +} + +} // namespace + +UTEST( UIInspector, HandlesInvalidateAndScenesStaySeparate ) { + UIApplication app( WindowSettings( 320, 240, "Inspector Domain", WindowStyle::Default, + WindowBackend::Default, 32, {}, 1, false, true ), + UIApplication::Settings( + Sys::getProcessPath() + ".." + FileSystem::getOSSlash(), 1.f, true ), + ContextSettings( false, 0, 0, GLv_default, true, false ) ); + auto* scene = app.getUI(); + EXPECT_EQ( UIInspectorServer::instance(), nullptr ); + auto* webview = UIWebView::New(); + webview->setId( "preview" ); + webview->setParent( scene->getRoot() ); + auto* document = webview->getDocumentSceneNode(); + auto* inside = UIWidget::New(); + inside->setId( "inside" ); + inside->setParent( document->getRoot() ); + UIInspector inspector; + inspector.setDefaultScene( scene ); + std::string handle = inspector.widgetHandle( inside ); + std::string handleAgain = inspector.widgetHandle( inside ); + EXPECT_STREQ( handle.c_str(), handleAgain.c_str() ); + EXPECT_EQ( inspector.query( scene, "#inside" ).size(), 0u ); + EXPECT_EQ( inspector.query( document, "#inside" ).size(), 1u ); + SmallVector registered; + SmallVector registeredHandles; + for ( size_t i = 0; i < 64; ++i ) { + auto* widget = UIWidget::New(); + widget->setParent( scene->getRoot() ); + registered.push_back( widget ); + registeredHandles.push_back( inspector.widgetHandle( widget ) ); + } + EXPECT_EQ( inspector.widget( registeredHandles.back() ), registered.back() ); + registered.front()->close(); + SceneManager::instance()->update(); + EXPECT_EQ( inspector.widget( registeredHandles.front() ), nullptr ); + EXPECT_EQ( inspector.widget( registeredHandles.back() ), registered.back() ); + inside->close(); + SceneManager::instance()->update(); + EXPECT_EQ( inspector.widget( handle ), nullptr ); + ASSERT_TRUE( UIInspectorServer::start( scene ) ); + EXPECT_TRUE( UIInspectorServer::instance()->getToken().size() >= 32 ); + UIInspectorServer::stop(); + Engine::destroySingleton(); +} + +UTEST( UIInspector, TcpBatchWatchAndMultipleClients ) { + UIApplication app( WindowSettings( 320, 240, "Inspector TCP", WindowStyle::Default, + WindowBackend::Default, 32, {}, 1, false, true ), + UIApplication::Settings( + Sys::getProcessPath() + ".." + FileSystem::getOSSlash(), 1.f, true ), + ContextSettings( false, 0, 0, GLv_default, true, false ) ); + auto* scene = app.getUI(); + auto* widget = UIWidget::New(); + widget->setId( "observed" ); + widget->setPixelsSize( 100, 40 ); + widget->setParent( scene->getRoot() ); + auto* password = UITextInput::NewPassword(); + password->setId( "password" ); + password->setPixelsPosition( 0, 60 ); + password->setText( String::fromUtf8( std::string_view( "secret-value" ) ) ); + password->setParent( scene->getRoot() ); + auto* longText = UITextView::New(); + longText->setId( "long-text" ); + longText->setPixelsPosition( 0, 110 ); + longText->setText( String::fromUtf8( std::string( 5000, 'x' ) ) ); + longText->setParent( scene->getRoot() ); + ASSERT_TRUE( UIInspectorServer::start( scene, { "127.0.0.1", 0, false, "test-token" } ) ); + auto* server = UIInspectorServer::instance(); + ASSERT_TRUE( server->getPort() != 0 ); + InspectorClient first( server->getPort() ); + InspectorClient second( server->getPort() ); + ASSERT_TRUE( first.connected ); + ASSERT_TRUE( second.connected ); + ASSERT_TRUE( connectInspector( first, "test-token" ) ); + ASSERT_TRUE( connectInspector( second, "test-token" ) ); + first.send( + { { "id", 2 }, + { "method", "ui.query" }, + { "params", { { "selector", "#observed" }, { "properties", { "geometry.size" } } } } } ); + json query = first.receive(); + ASSERT_TRUE( query.contains( "result" ) ); + ASSERT_EQ( query["result"]["total"].get(), 1 ); + std::string handle = query["result"]["nodes"][0]["handle"]; + second.send( + { { "id", 6 }, { "method", "ui.query" }, { "params", { { "selector", "#password" } } } } ); + json secretQuery = second.receive(); + ASSERT_TRUE( secretQuery.contains( "result" ) ); + EXPECT_TRUE( secretQuery["result"]["nodes"][0]["text"] == "[redacted]" ); + second.send( { { "id", 7 }, + { "method", "ui.inspect" }, + { "params", { { "selector", "#password" }, { "properties", { "*" } } } } } ); + json secretInspect = second.receive(); + ASSERT_TRUE( secretInspect.contains( "result" ) ); + EXPECT_TRUE( secretInspect["result"]["properties"]["content.text"] == "[redacted]" ); + second.send( { { "id", 8 }, + { "method", "ui.inspect" }, + { "params", + { { "selector", "#long-text" }, + { "properties", { "content.text" } }, + { "maxStringLength", 10 } } } } ); + json truncated = second.receive(); + ASSERT_TRUE( truncated.contains( "result" ) ); + EXPECT_EQ( truncated["result"]["properties"]["content.text"].get().size(), 10u ); + EXPECT_TRUE( truncated["result"].contains( "truncatedProperties" ) ); + first.send( { { "id", 3 }, + { "method", "watch.subscribe" }, + { "params", { { "handle", handle }, { "properties", { "geometry.size" } } } } } ); + json watch = first.receive(); + ASSERT_TRUE( watch.contains( "result" ) ); + widget->setPixelsSize( 120, 40 ); + json changed = first.receive(); + std::string method = changed["method"].get(); + ASSERT_STREQ( method.c_str(), "watch.changed" ); + ASSERT_EQ( changed["params"]["changes"]["geometry.size"]["new"][0].get(), 120 ); + int clicks = 0; + widget->onClick( [&]( const MouseEvent* ) { + ++clicks; + widget->setPixelsSize( 140, 40 ); + } ); + second.send( { { "id", 5 }, + { "method", "input.click" }, + { "params", { { "selector", "#observed" } } } } ); + json click = second.receive(); + ASSERT_TRUE( click.contains( "result" ) ); + json clickChange = first.receive(); + std::string clickMethod = clickChange["method"].get(); + ASSERT_STREQ( clickMethod.c_str(), "watch.changed" ); + EXPECT_EQ( clickChange["params"]["changes"]["geometry.size"]["new"][0].get(), 140 ); + EXPECT_EQ( clicks, 1 ); + second.send( + { { "id", 4 }, + { "method", "session.batch" }, + { "params", + { { "commands", + json::array( { { { "name", "found" }, + { "method", "ui.query" }, + { "return", false }, + { "params", { { "selector", "#observed" } } } }, + { { "name", "state" }, + { "method", "ui.inspect" }, + { "params", + { { "handle", { { "$ref", "found#/nodes/0/handle" } } }, + { "properties", { "geometry.size" } } } } } } ) } } } } ); + json batch = second.receive(); + ASSERT_TRUE( batch.contains( "result" ) ); + EXPECT_TRUE( batch["result"]["results"].contains( "state" ) ); + EXPECT_FALSE( batch["result"]["results"].contains( "found" ) ); + second.send( + { { "id", 10 }, + { "method", "session.batch" }, + { "params", + { { "commands", + json::array( + { { { "method", "ui.inspect" }, + { "params", + { { "handle", { { "$ref", "unknown#/handle" } } } } } } } ) } } } } ); + json badReference = second.receive(); + EXPECT_TRUE( badReference.contains( "error" ) && + badReference["error"]["code"] == "batch-command-failed" ); + EXPECT_TRUE( badReference["error"]["data"]["commandError"]["code"] == "batch-reference-error" ); + second.send( + { { "id", 11 }, + { "method", "session.batch" }, + { "params", + { { "commands", + json::array( { { { "method", "input.click" }, + { "delay", "bad" }, + { "params", { { "selector", "#observed" } } } } } ) } } } } ); + json badDelay = second.receive(); + EXPECT_TRUE( badDelay.contains( "error" ) ); + EXPECT_EQ( clicks, 1 ); + widget->close(); + Engine::instance()->updateInput(); + SceneManager::instance()->update(); + json removed = first.receive(); + json ended = first.receive(); + EXPECT_TRUE( removed["method"] == "watch.targetRemoved" ); + EXPECT_TRUE( ended["method"] == "watch.ended" ); + second.send( + { { "id", 9 }, { "method", "ui.inspect" }, { "params", { { "handle", handle } } } } ); + json invalid = second.receive(); + EXPECT_TRUE( invalid.contains( "error" ) && invalid["error"]["code"] == "invalid-widget" ); + UIInspectorServer::stop(); + Engine::destroySingleton(); +} + +UTEST( UIInspector, InputActionErrorsAndKeyEvents ) { + UIApplication app( WindowSettings( 320, 240, "Inspector Input", WindowStyle::Default, + WindowBackend::Default, 32, {}, 1, false, true ), + UIApplication::Settings( + Sys::getProcessPath() + ".." + FileSystem::getOSSlash(), 1.f, true ), + ContextSettings( false, 0, 0, GLv_default, true, false ) ); + auto* scene = app.getUI(); + auto* first = UIWidget::New(); + first->setId( "first" ); + first->setClass( "ambiguous" ); + first->setPixelsPosition( 10, 10 ); + first->setPixelsSize( 30, 30 ); + first->setParent( scene->getRoot() ); + auto* second = UIWidget::New(); + second->setClass( "ambiguous" ); + second->setPixelsPosition( 50, 10 ); + second->setPixelsSize( 30, 30 ); + second->setParent( scene->getRoot() ); + auto* hidden = UIWidget::New(); + hidden->setId( "hidden" ); + hidden->setPixelsPosition( 10, 50 ); + hidden->setPixelsSize( 30, 30 ); + hidden->setParent( scene->getRoot() ); + hidden->setVisible( false ); + auto* outside = UIWidget::New(); + outside->setId( "outside" ); + outside->setPixelsPosition( 400, 400 ); + outside->setPixelsSize( 30, 30 ); + outside->setParent( scene->getRoot() ); + auto* covered = UIWidget::New(); + covered->setId( "covered" ); + covered->setPixelsPosition( 100, 100 ); + covered->setPixelsSize( 40, 40 ); + covered->setParent( scene->getRoot() ); + auto* blocker = UIWidget::New(); + blocker->setPixelsPosition( 100, 100 ); + blocker->setPixelsSize( 40, 40 ); + blocker->setParent( scene->getRoot() ); + int clicks = 0; + first->onClick( [&]( const MouseEvent* ) { ++clicks; } ); + covered->onClick( [&]( const MouseEvent* ) { ++clicks; } ); + scene->flushDirtyStyleAndLayout(); + ASSERT_TRUE( UIInspectorServer::start( scene, { "127.0.0.1", 0, false, "input-token" } ) ); + InspectorClient client( UIInspectorServer::instance()->getPort() ); + ASSERT_TRUE( client.connected ); + ASSERT_TRUE( connectInspector( client, "input-token" ) ); + const std::array, 4> rejectedTargets{ + { { ".ambiguous", "target-ambiguous" }, + { "#hidden", "target-not-interactable" }, + { "#outside", "target-not-interactable" }, + { "#covered", "target-not-interactable" } } }; + int id = 2; + for ( const auto& [selector, expectedCode] : rejectedTargets ) { + client.send( { { "id", id++ }, + { "method", "input.click" }, + { "params", { { "selector", selector } } } } ); + json response = client.receive(); + ASSERT_TRUE( response.contains( "error" ) ); + EXPECT_TRUE( response["error"]["code"] == expectedCode ); + } + EXPECT_EQ( clicks, 0 ); + client.send( { { "id", id++ }, + { "method", "input.click" }, + { "params", { { "selector", "#first" } } } } ); + ASSERT_TRUE( client.receive().contains( "result" ) ); + EXPECT_EQ( clicks, 1 ); + + scene->getEventDispatcher()->setFocusNode( first ); + int keyDown = 0; + int keyUp = 0; + Uint32 lastModifiers = 0; + first->addEventListener( Event::KeyDown, [&]( const Event* event ) { + ++keyDown; + lastModifiers = event->asKeyEvent()->getMod(); + } ); + first->addEventListener( Event::KeyUp, [&]( const Event* event ) { + ++keyUp; + lastModifiers = event->asKeyEvent()->getMod(); + } ); + client.send( + { { "id", id++ }, + { "method", "input.key" }, + { "params", + { { "key", "F" }, { "action", "down" }, { "modifiers", { "Ctrl", "Shift" } } } } } ); + ASSERT_TRUE( client.receive().contains( "result" ) ); + EXPECT_EQ( keyDown, 1 ); + EXPECT_EQ( keyUp, 0 ); + EXPECT_EQ( lastModifiers & ( KEYMOD_LCTRL | KEYMOD_LSHIFT ), + static_cast( KEYMOD_LCTRL | KEYMOD_LSHIFT ) ); + client.send( { { "id", id++ }, + { "method", "input.key" }, + { "params", { { "key", "F" }, { "action", "up" } } } } ); + ASSERT_TRUE( client.receive().contains( "result" ) ); + EXPECT_EQ( keyDown, 1 ); + EXPECT_EQ( keyUp, 1 ); + client.send( { { "id", id++ }, + { "method", "input.key" }, + { "params", { { "key", "F" }, { "action", "press" } } } } ); + ASSERT_TRUE( client.receive().contains( "result" ) ); + EXPECT_EQ( keyDown, 2 ); + EXPECT_EQ( keyUp, 2 ); + client.send( + { { "id", id++ }, { "method", "input.key" }, { "params", { { "key", "not-a-key" } } } } ); + json invalidKey = client.receive(); + ASSERT_TRUE( invalidKey.contains( "error" ) ); + EXPECT_TRUE( invalidKey["error"]["code"] == "invalid-params" ); + EXPECT_EQ( keyDown, 2 ); + EXPECT_EQ( keyUp, 2 ); + UIInspectorServer::stop(); + Engine::destroySingleton(); +} + +UTEST( UIInspector, HTMLTextSummariesAndProperties ) { + UIApplication app( WindowSettings( 320, 240, "Inspector HTML Text", WindowStyle::Default, + WindowBackend::Default, 32, {}, 1, false, true ), + UIApplication::Settings( + Sys::getProcessPath() + ".." + FileSystem::getOSSlash(), 1.f, true ), + ContextSettings( false, 0, 0, GLv_default, true, false ) ); + auto* scene = app.getUI(); + auto* webview = UIWebView::New(); + webview->setId( "preview" ); + webview->setPixelsSize( 220, 160 ); + webview->setParent( scene->getRoot() ); + auto* document = webview->getDocumentSceneNode(); + auto* paragraph = UIRichText::NewParagraph(); + paragraph->setId( "article" ); + paragraph->setParent( document->getRoot() ); + auto* source = UITextNode::New(); + source->setId( "source" ); + source->setText( "Hello " ); + source->setParent( paragraph ); + auto* span = UITextSpan::NewStrong(); + span->setId( "emphasis" ); + span->setText( "world" ); + span->setParent( paragraph ); + auto* punctuation = UITextNode::New(); + punctuation->setText( "!" ); + punctuation->setParent( span ); + paragraph->setPixelsSize( 200, 30 ); + ASSERT_TRUE( UIInspectorServer::start( scene, { "127.0.0.1", 0, false, "html-token" } ) ); + InspectorClient client( UIInspectorServer::instance()->getPort() ); + ASSERT_TRUE( client.connected ); + ASSERT_TRUE( connectInspector( client, "html-token" ) ); + client.send( + { { "id", 2 }, { "method", "ui.query" }, { "params", { { "selector", "#preview" } } } } ); + json preview = client.receive(); + ASSERT_TRUE( preview.contains( "result" ) ); + std::string documentScene = preview["result"]["nodes"][0]["documentScene"]; + client.send( { { "id", 3 }, + { "method", "ui.query" }, + { "params", { { "scene", documentScene }, { "selector", "#article" } } } } ); + json article = client.receive(); + ASSERT_TRUE( article.contains( "result" ) ); + ASSERT_EQ( article["result"]["nodes"].size(), 1u ); + EXPECT_TRUE( article["result"]["nodes"][0]["text"] == "Hello world!" ); + std::string articleHandle = article["result"]["nodes"][0]["handle"]; + client.send( { { "id", 4 }, + { "method", "ui.query" }, + { "params", { { "scene", documentScene }, { "selector", "#emphasis" } } } } ); + json emphasis = client.receive(); + ASSERT_TRUE( emphasis.contains( "result" ) ); + EXPECT_TRUE( emphasis["result"]["nodes"][0]["text"] == "world!" ); + client.send( { { "id", 5 }, + { "method", "ui.query" }, + { "params", { { "scene", documentScene }, { "selector", "#source" } } } } ); + json sourceResult = client.receive(); + ASSERT_TRUE( sourceResult.contains( "result" ) ); + EXPECT_TRUE( sourceResult["result"]["nodes"][0]["text"] == "Hello " ); + client.send( + { { "id", 6 }, + { "method", "ui.inspect" }, + { "params", { { "handle", articleHandle }, { "properties", { "content.text" } } } } } ); + json inspection = client.receive(); + ASSERT_TRUE( inspection.contains( "result" ) ); + EXPECT_TRUE( inspection["result"]["properties"]["content.text"] == "Hello world!" ); + client.send( { { "id", 11 }, + { "method", "ui.query" }, + { "params", + { { "scene", documentScene }, + { "selector", "#article" }, + { "properties", { "content.text" } } } } } ); + json inlineProperties = client.receive(); + ASSERT_TRUE( inlineProperties.contains( "result" ) ); + EXPECT_TRUE( inlineProperties["result"]["nodes"][0]["properties"]["content.text"] == + "Hello world!" ); + client.send( + { { "id", 12 }, + { "method", "ui.tree" }, + { "params", { { "root", articleHandle }, { "depth", 0 }, { "includeText", true } } } } ); + json tree = client.receive(); + ASSERT_TRUE( tree.contains( "result" ) ); + EXPECT_TRUE( tree["result"]["nodes"][0]["text"] == "Hello world!" ); + client.send( + { { "id", 7 }, + { "method", "watch.subscribe" }, + { "params", { { "handle", articleHandle }, { "properties", { "content.text" } } } } } ); + json watch = client.receive(); + ASSERT_TRUE( watch.contains( "result" ) ); + EXPECT_TRUE( watch["result"]["targets"][0]["values"]["content.text"] == "Hello world!" ); + span->setText( "eepp" ); + json changed = client.receive(); + ASSERT_TRUE( changed.contains( "method" ) ); + EXPECT_TRUE( changed["method"] == "watch.changed" ); + EXPECT_TRUE( changed["params"]["changes"]["content.text"]["old"] == "Hello world!" ); + EXPECT_TRUE( changed["params"]["changes"]["content.text"]["new"] == "Hello eepp!" ); + client.send( { { "id", 8 }, + { "method", "watch.unsubscribe" }, + { "params", { { "subscription", watch["result"]["subscription"] } } } } ); + ASSERT_TRUE( client.receive().contains( "result" ) ); + auto* password = UITextInput::NewPassword(); + password->setText( String::fromUtf8( std::string_view( "do-not-expose" ) ) ); + password->setParent( paragraph ); + client.send( + { { "id", 13 }, + { "method", "ui.inspect" }, + { "params", { { "handle", articleHandle }, { "properties", { "content.text" } } } } } ); + json parentText = client.receive(); + ASSERT_TRUE( parentText.contains( "result" ) ); + EXPECT_TRUE( parentText["result"]["properties"]["content.text"] == "Hello eepp!" ); + auto* longSource = UITextNode::New(); + longSource->setId( "long-source" ); + longSource->setText( String::fromUtf8( std::string( 300, 'x' ) ) ); + longSource->setParent( paragraph ); + client.send( { { "id", 9 }, + { "method", "ui.query" }, + { "params", { { "scene", documentScene }, { "selector", "#long-source" } } } } ); + json longSummary = client.receive(); + ASSERT_TRUE( longSummary.contains( "result" ) ); + EXPECT_EQ( longSummary["result"]["nodes"][0]["text"].get().size(), 256u ); + EXPECT_TRUE( longSummary["result"]["nodes"][0]["textTruncated"] == true ); + client.send( { { "id", 10 }, + { "method", "ui.inspect" }, + { "params", + { { "scene", documentScene }, + { "selector", "#article" }, + { "properties", { "content.text" } }, + { "maxStringLength", 10 } } } } ); + json shortInspection = client.receive(); + ASSERT_TRUE( shortInspection.contains( "result" ) ); + EXPECT_TRUE( shortInspection["result"]["properties"]["content.text"] == "Hello eepp" ); + EXPECT_TRUE( shortInspection["result"]["truncatedProperties"][0] == "content.text" ); + UIInspectorServer::stop(); + Engine::destroySingleton(); +} + +UTEST( UIInspector, WebViewSecondaryWindowAndDeferredBatch ) { + UIApplication app( WindowSettings( 320, 240, "Inspector Primary", WindowStyle::Default, + WindowBackend::Default, 32, {}, 1, false, true ), + UIApplication::Settings( + Sys::getProcessPath() + ".." + FileSystem::getOSSlash(), 1.f, true ), + ContextSettings( false, 0, 0, GLv_default, true, false ) ); + auto* mainScene = app.getUI(); + auto* webview = UIWebView::New(); + webview->setId( "preview" ); + webview->setPixelsSize( 200, 100 ); + webview->setParent( mainScene->getRoot() ); + auto* document = webview->getDocumentSceneNode(); + auto* htmlWidget = UIWidget::New(); + htmlWidget->setId( "inside" ); + htmlWidget->setPixelsSize( 60, 30 ); + htmlWidget->setParent( document->getRoot() ); + int innerClicks = 0; + htmlWidget->onClick( [&]( const MouseEvent* ) { ++innerClicks; } ); + auto* textInput = UITextInput::New(); + textInput->setId( "edit" ); + textInput->setPixelsPosition( 0, 40 ); + textInput->setPixelsSize( 100, 30 ); + textInput->setParent( document->getRoot() ); + auto* secondary = + app.createWindow( WindowSettings( 280, 180, "Inspector Secondary", WindowStyle::Default, + WindowBackend::Default, 32, {}, 1, false, true ), + ContextSettings( false, 0, 0, GLv_default, true, false ) ); + ASSERT_TRUE( secondary != nullptr ); + auto* button = UIWidget::New(); + button->setId( "secondary-button" ); + button->setPixelsPosition( 30, 30 ); + button->setPixelsSize( 100, 50 ); + button->setParent( secondary->getRoot() ); + int clicks = 0; + button->onClick( [&]( const MouseEvent* ) { ++clicks; } ); + mainScene->flushDirtyStyleAndLayout(); + secondary->flushDirtyStyleAndLayout(); + ASSERT_TRUE( + UIInspectorServer::start( mainScene, { "127.0.0.1", 0, false, "contexts-token" } ) ); + InspectorClient client( UIInspectorServer::instance()->getPort() ); + ASSERT_TRUE( client.connected ); + ASSERT_TRUE( connectInspector( client, "contexts-token" ) ); + client.send( { { "id", 2 }, { "method", "ui.contexts" } } ); + json contexts = client.receive(); + ASSERT_TRUE( contexts.contains( "result" ) ); + EXPECT_EQ( contexts["result"]["windows"].size(), 2u ); + std::string secondaryScene; + std::string secondaryWindow; + for ( const auto& item : contexts["result"]["scenes"] ) { + if ( item["kind"] == "top-level" && + item["window"] != contexts["result"]["defaultWindow"] ) { + secondaryScene = item["handle"].get(); + secondaryWindow = item["window"].get(); + } + } + ASSERT_TRUE( !secondaryScene.empty() ); + client.send( + { { "id", 3 }, { "method", "ui.query" }, { "params", { { "selector", "#preview" } } } } ); + json preview = client.receive(); + ASSERT_TRUE( preview.contains( "result" ) ); + std::string documentScene = preview["result"]["nodes"][0]["documentScene"]; + std::string previewHandle = preview["result"]["nodes"][0]["handle"]; + bool foundDocumentScene = false; + for ( const auto& item : contexts["result"]["scenes"] ) { + if ( item["handle"] == documentScene ) { + foundDocumentScene = true; + EXPECT_TRUE( item["kind"] == "webview" ); + EXPECT_TRUE( item["parentScene"] == contexts["result"]["defaultScene"] ); + EXPECT_TRUE( item["owner"] == previewHandle ); + EXPECT_TRUE( item["window"] == contexts["result"]["defaultWindow"] ); + } + } + EXPECT_TRUE( foundDocumentScene ); + client.send( { { "id", 24 }, + { "method", "ui.screenshot" }, + { "params", { { "scene", documentScene } } } } ); + json documentImage = client.receive(); + ASSERT_TRUE( documentImage.contains( "result" ) ); + EXPECT_TRUE( documentImage["result"]["scene"] == documentScene ); + EXPECT_TRUE( documentImage["result"]["sizePx"] == json::array( { 200, 100 } ) ); + FileSystem::fileRemove( documentImage["result"]["path"].get() ); + client.send( { { "id", 25 }, + { "method", "ui.screenshot" }, + { "params", { { "window", secondaryWindow } } } } ); + json secondaryImage = client.receive(); + ASSERT_TRUE( secondaryImage.contains( "result" ) ); + EXPECT_TRUE( secondaryImage["result"]["sizePx"] == json::array( { 280, 180 } ) ); + FileSystem::fileRemove( secondaryImage["result"]["path"].get() ); + int secondaryKeyDown = 0; + button->addEventListener( Event::KeyDown, [&]( const Event* event ) { + if ( event->asKeyEvent()->getKeyCode() == KEY_F ) + ++secondaryKeyDown; + } ); + secondary->getEventDispatcher()->setFocusNode( button ); + client.send( { { "id", 11 }, + { "method", "input.key" }, + { "params", { { "scene", secondaryScene }, { "key", "F" } } } } ); + ASSERT_TRUE( client.receive().contains( "result" ) ); + EXPECT_EQ( secondaryKeyDown, 1 ); + client.send( + { { "id", 4 }, { "method", "ui.query" }, { "params", { { "selector", "#inside" } } } } ); + json outer = client.receive(); + ASSERT_TRUE( outer.contains( "result" ) ); + EXPECT_EQ( outer["result"]["total"].get(), 0 ); + client.send( { { "id", 5 }, + { "method", "ui.query" }, + { "params", { { "scene", documentScene }, { "selector", "#inside" } } } } ); + json inner = client.receive(); + ASSERT_TRUE( inner.contains( "result" ) ); + EXPECT_EQ( inner["result"]["total"].get(), 1 ); + client.send( { { "id", 7 }, + { "method", "input.click" }, + { "params", { { "scene", documentScene }, { "selector", "#inside" } } } } ); + json innerClick = client.receive(); + ASSERT_TRUE( innerClick.contains( "result" ) ); + EXPECT_EQ( innerClicks, 1 ); + client.send( { { "id", 8 }, + { "method", "input.click" }, + { "params", { { "scene", documentScene }, { "selector", "#edit" } } } } ); + json editClick = client.receive(); + ASSERT_TRUE( editClick.contains( "result" ) ); + client.send( { { "id", 9 }, + { "method", "input.text" }, + { "params", { { "scene", documentScene }, { "text", "h\u00e9llo" } } } } ); + json typed = client.receive(); + ASSERT_TRUE( typed.contains( "result" ) ); + std::string entered = textInput->getText().toUtf8(); + EXPECT_STREQ( entered.c_str(), "h\u00e9llo" ); + client.send( + { { "id", 10 }, + { "method", "input.key" }, + { "params", + { { "scene", documentScene }, { "key", "Backspace" }, { "action", "press" } } } } ); + json pressed = client.receive(); + ASSERT_TRUE( pressed.contains( "result" ) ); + std::string afterBackspace = textInput->getText().toUtf8(); + EXPECT_STREQ( afterBackspace.c_str(), "h\u00e9ll" ); + client.send( + { { "id", 6 }, + { "method", "session.batch" }, + { "params", + { { "commands", + json::array( + { { { "method", "input.click" }, + { "return", false }, + { "delay", "20ms" }, + { "params", + { { "scene", secondaryScene }, { "selector", "#secondary-button" } } } }, + { { "method", "session.nextFrame" }, { "return", false } }, + { { "name", "state" }, + { "method", "ui.inspect" }, + { "params", + { { "scene", secondaryScene }, + { "selector", "#secondary-button" }, + { "properties", { "state.visible" } } } } } } ) } } } } ); + json batch = client.receive(); + ASSERT_TRUE( batch.contains( "result" ) ); + EXPECT_EQ( clicks, 1 ); + EXPECT_TRUE( batch["result"]["results"].contains( "state" ) ); + UIInspectorServer::stop(); + Engine::destroySingleton(); +} + +UTEST( UIInspector, AuthenticationFramingAndReadOnly ) { + UIApplication app( WindowSettings( 320, 240, "Inspector Security", WindowStyle::Default, + WindowBackend::Default, 32, {}, 1, false, true ), + UIApplication::Settings( + Sys::getProcessPath() + ".." + FileSystem::getOSSlash(), 1.f, true ), + ContextSettings( false, 0, 0, GLv_default, true, false ) ); + ASSERT_TRUE( UIInspectorServer::start( app.getUI(), { "127.0.0.1", 0, true, "read-token" } ) ); + Uint16 port = UIInspectorServer::instance()->getPort(); + InspectorClient wrong( port ); + ASSERT_TRUE( wrong.connected ); + wrong.send( { { "id", 1 }, + { "method", "session.connect" }, + { "params", { { "protocolVersion", 1 }, { "token", "wrong" } } } } ); + json rejected = wrong.receive(); + ASSERT_TRUE( rejected.contains( "error" ) ); + std::string code = rejected["error"]["code"]; + EXPECT_STREQ( code.c_str(), "authentication-failed" ); + InspectorClient client( port ); + ASSERT_TRUE( client.connected ); + ASSERT_TRUE( connectInspector( client, "read-token" ) ); + client.sendBytes( "{\"id\":2,\"method\":\"ui.con" ); + client.sendBytes( "texts\"}\r\n{\"id\":3,\"method\":\"ui.focus\"}\n" ); + json contexts = client.receive(); + json focus = client.receive(); + ASSERT_TRUE( contexts.contains( "result" ) ); + ASSERT_TRUE( focus.contains( "result" ) ); + EXPECT_EQ( contexts["id"].get(), 2 ); + EXPECT_EQ( focus["id"].get(), 3 ); + const std::array actions{ + { { { "method", "input.click" }, { "params", { { "selector", "*" } } } }, + { { "method", "input.key" }, { "params", { { "key", "Enter" } } } }, + { { "method", "input.text" }, { "params", { { "text", "blocked" } } } } } }; + int actionId = 4; + for ( auto action : actions ) { + action["id"] = actionId++; + client.send( action ); + json denied = client.receive(); + ASSERT_TRUE( denied.contains( "error" ) ); + EXPECT_TRUE( denied["error"]["code"] == "permission-denied" ); + } + client.sendBytes( "not-json\n" ); + json parse = client.receive(); + ASSERT_TRUE( parse.contains( "error" ) ); + std::string parseCode = parse["error"]["code"]; + EXPECT_STREQ( parseCode.c_str(), "parse-error" ); + UIInspectorServer::stop(); + Engine::destroySingleton(); +} + +UTEST( UIInspector, ScreenshotCapturesRenderedPixelsAndCrop ) { + UIApplication app( WindowSettings( 320, 240, "Inspector Screenshot", WindowStyle::Default, + WindowBackend::Default, 32, {}, 1, false, true ), + UIApplication::Settings( + Sys::getProcessPath() + ".." + FileSystem::getOSSlash(), 1.f, true ), + ContextSettings( false, 0, 0, GLv_default, true, false ) ); + auto* scene = app.getUI(); + auto* panel = UIWidget::New(); + panel->setPixelsSize( 320, 240 ); + panel->setBackgroundColor( Color::Red ); + panel->setParent( scene->getRoot() ); + ASSERT_TRUE( UIInspectorServer::start( scene, { "127.0.0.1", 0, true, "image-token" } ) ); + InspectorClient client( UIInspectorServer::instance()->getPort() ); + ASSERT_TRUE( client.connected ); + ASSERT_TRUE( connectInspector( client, "image-token" ) ); + client.send( { { "id", 2 }, { "method", "ui.screenshot" } } ); + json full = client.receive(); + ASSERT_TRUE( full.contains( "result" ) ); + std::string fullPath = full["result"]["path"]; + EXPECT_TRUE( FileSystem::fileExists( fullPath ) ); + EXPECT_TRUE( full["result"]["sizePx"] == json::array( { 320, 240 } ) ); + Graphics::Image fullImage( fullPath ); + EXPECT_EQ( fullImage.getWidth(), 320u ); + EXPECT_EQ( fullImage.getHeight(), 240u ); + EXPECT_GT( fullImage.getPixel( 10, 10 ).r, 200 ); + EXPECT_LT( fullImage.getPixel( 10, 10 ).b, 50 ); + FileSystem::fileRemove( fullPath ); + + panel->setBackgroundColor( Color::Blue ); + client.send( { { "id", 3 }, + { "method", "ui.screenshot" }, + { "params", { { "rect", { 8, 12, 40, 30 } }, { "format", "bmp" } } } } ); + json cropped = client.receive(); + ASSERT_TRUE( cropped.contains( "result" ) ); + std::string croppedPath = cropped["result"]["path"]; + Graphics::Image croppedImage( croppedPath ); + EXPECT_EQ( croppedImage.getWidth(), 40u ); + EXPECT_EQ( croppedImage.getHeight(), 30u ); + EXPECT_GT( croppedImage.getPixel( 2, 2 ).b, 200 ); + EXPECT_LT( croppedImage.getPixel( 2, 2 ).r, 50 ); + FileSystem::fileRemove( croppedPath ); + + client.send( { { "id", 4 }, + { "method", "ui.screenshot" }, + { "params", { { "rect", { 300, 0, 40, 30 } } } } } ); + json invalidRect = client.receive(); + ASSERT_TRUE( invalidRect.contains( "error" ) ); + EXPECT_TRUE( invalidRect["error"]["code"] == "invalid-params" ); + client.send( + { { "id", 5 }, { "method", "ui.screenshot" }, { "params", { { "format", "gif" } } } } ); + json invalidFormat = client.receive(); + ASSERT_TRUE( invalidFormat.contains( "error" ) ); + EXPECT_TRUE( invalidFormat["error"]["code"] == "invalid-params" ); + UIInspectorServer::stop(); + Engine::destroySingleton(); +}