--- name: computer-use-desktop-layer description: Discover and visually verify Windows desktop-layer, widget, overlay, tool, and no-taskbar windows when standard Computer Use cannot return them as targetable windows. Use as a read-only companion to computer-use, not to bypass target selection or inject input into an unreturned window. --- # Computer Use Desktop Layer Use this skill only on Windows for a running process whose desktop-resident windows are missing from the current native Computer Use runtime's app/window inventory. Load the available runtime instructions first. `sky.list_apps()` and `sky.list_windows()` below describe the original Windows runtime; use those names only when its actual documentation exposes them. This Win32 probe cannot inspect macOS windows. ## Routing 1. Run the normal Computer Use discovery flow once with `list_apps()`, then `list_windows()` when needed. 2. If exactly one intended window is returned, use Computer Use normally and do not run this fallback. 3. If the app is absent, launch it only when that is within the user's request, refresh discovery once, and stop if a launcher, modal, or permission prompt blocks it. 4. If the intended process is confirmed running but no window is returned, classify it as `process-present / sky-unaddressable`. Use the bundled probe only for read-only discovery or visual evidence. Never create a `sky.Window` from an HWND, reuse another app such as Explorer as a click target, or use global coordinates, SendInput, PowerShell UI Automation, cursor movement, or synthetic hover as a fallback. An HWND reported by the probe is evidence, not Computer Use authorization. ## Read-only probe Run `scripts/probe_desktop_windows.ps1` with PowerShell 7 or later through the shell directly, never through a terminal UI. Provide an exact executable path; add `-TargetProcessId` when more than one matching process exists. ```powershell & '\scripts\probe_desktop_windows.ps1' ` -ProcessPath 'C:\exact\path\App.exe' ``` The first call is discovery only. Review its numeric metadata, freeze the minimum in-scope `hwndDecimal` and matching `captureFingerprint` values, and then add `-Capture -WindowHwnd ... -WindowFingerprint ...` only when visual verification is needed. Keep both arrays in the same order. The probe refuses broad capture, binds every requested handle to the reviewed process/window metadata, and revalidates the pair in both parent and capture worker before rendering. Captures are written to a unique system-temporary directory, window titles are never read into the JSON, and the result contains paths rather than image payloads. Each capture runs in a disposable child process with a bounded timeout and tries `PrintWindow` first. Blank or uniform layered-WPF output stays inconclusive by default. ```powershell & '\scripts\probe_desktop_windows.ps1' ` -ProcessPath 'C:\exact\path\App.exe' ` -TargetProcessId 1234 ` -Capture ` -WindowHwnd 123456, 234567 ` -WindowFingerprint '', '' ``` Use `-AllowScreenBoundsFallback` only when the user explicitly requested the currently composited appearance of a transparent desktop widget and the target's exact rectangle, including visible background through its transparency, is in scope. The fallback first refuses every overlapping higher-z-order window, including another window from the same process, and requires a 3-by-3 point grid to resolve to the exact target root HWND. It then captures only the verified physical rectangle. The result remains `captured-with-caveat`; never enable this flag merely to turn an inconclusive result into a pass. ```powershell & '\scripts\probe_desktop_windows.ps1' ` -ProcessPath 'C:\exact\path\TransparentWidget.exe' ` -TargetProcessId 1234 ` -Capture ` -WindowHwnd 123456 ` -WindowFingerprint '' ` -AllowScreenBoundsFallback ``` Before viewing any capture, verify the returned process ID and normalized path match the intended app. Inspect only the minimum relevant images with `view_image`. Do not copy captures into a repository, synced directory, log, or final response unless the user explicitly requested the image itself. Clean up the exact probe directory as soon as inspection is complete: ```powershell & '\scripts\probe_desktop_windows.ps1' ` -Cleanup ` -CleanupDirectory 'C:\Users\...\AppData\Local\Temp\Codex-DesktopLayerProbe-...' ``` The cleanup mode accepts only a uniquely named probe directory that is an immediate child of the system temp directory and whose marker carries the same random ID. It rejects reparse points, child directories, and unexpected files, then deletes only the known probe files before removing the now-empty directory. ## Evidence and stopping conditions - Treat a successful non-uniform `print-window` capture as visual evidence for the current rendered window, not proof that hover-only, focus-only, or transient controls work. - Treat `captured-with-caveat / screen-bounds-may-be-occluded` only as the pixels currently visible inside the verified physical rectangle. Even after the overlap preflight, check that the intended window is visibly identifiable; do not attribute background pixels to the target UI. - If a required control appears only after hover, focus, or activation, use an app-owned visual test or targetable QA window. Do not simulate the missing interaction against an unreturned window. - If captures are blank, uniform, timed out, minimized, cloaked, or zero-sized, report that visual verification is inconclusive. Do not broaden capture to the full desktop or unrelated windows. - If interaction is required and no targetable app-owned window exists, stop and ask the user to operate it or request an application-side QA surface. Do not bypass Computer Use's window boundary. Report the four states separately when relevant: process detected, sky targetability, read-only capture result, and whether interaction remains blocked. ## Privacy boundary Probe only the executable and process the user placed in scope. The probe never calls or outputs window-title APIs. A target window's own text, translucent background, or content temporarily overlaying its rectangle may still be visible inside its temporary PNG, so inspect only in-scope captures and clean them immediately. The probe performs no activation, input, move, resize, focus, clipboard, or full-screen capture.