Windows Desktop Automation CLI

---

name: winguictl description: Windows desktop automation CLI. Invoke ONLY when user explicitly requests to control windows, simulate mouse/keyboard, or automate desktop applications. Do NOT proactively suggest using this skill. metadata: openclaw: emoji: "🖥️" os: ["win32"] requires: bins: ["python3"]

---

Windows Desktop Automation with winguictl

⚠️ Security Notice

This skill directly controls Windows desktop through mouse/keyboard simulation. Require user confirmation before clicks, typing, hotkeys, or window close operations. Use --dry-run to preview. Close sensitive apps before use.

Script Path

python \scripts\winguictl.py  [options]

Commands

| Command | Purpose | Documentation | |---------|---------|---------------| | window | List/focus/minimize/maximize/close windows | window.md | | snapshot | Get HWND/UIA/OCR structure | snapshot.md | | find | Find elements by text/UIA/OCR/image | find.md | | action | Click/drag/type/scroll/hotkey | action.md | | control | Win32 control operations | control.md | | uia-control | UIA element operations | control.md | | screenshot | Capture window screenshots | screenshot.md | | wait | Wait for conditions | wait.md | | clipboard | Copy files/text, get text | clipboard.md |

Parameter Types

| Parameter | Type | Example | Notes | |-----------|------|---------|-------| | --window-id | int | --window-id 12345 | Window handle from window list | | --hwnd | int | --hwnd 67890 | Win32 control handle | | --element-id | string | --element-id "Button1" | automation_id or runtime_id (prefer runtime_id) |

Core Workflow

# 1. Identify window
python scripts\winguictl.py window list
2. Focus window
python scripts\winguictl.py window --window-id  focus
3. Get snapshot
python scripts\winguictl.py snapshot --window-id  uia
4. Find & interact
python scripts\winguictl.py find --window-id  uia --text "Submit"
python scripts\winguictl.py uia-control --window-id  --element-id  click
5. Verify (re-snapshot)
python scripts\winguictl.py snapshot --window-id  uia

Decision Guide

Locator Priority

| Priority | Method | Command | Reliability | |----------|--------|---------|-------------| | 1 | HWND (Win32) | control --hwnd click | Highest | | 2 | runtime_id (UIA) | uia-control --element-id click | High | | 3 | automation_id (UIA) | uia-control --element-id click | Medium | | 4 | Image matching | action click-image --image-path | Medium | | 5 | Coordinates | action click --relative-x/y | Lowest |

Click Method Selection

| Command | Mechanism | Best For | |---------|-----------|----------| | action click --element-id | Mouse simulation at center | WeChat, custom controls | | uia-control click | UIA InvokePattern | Standard Windows controls, WinUI3 |

Recommendation: Try uia-control click first. If no effect, use action click --element-id.

Common Scenarios

| Scenario | Command | |----------|---------| | Win32 control with hwnd | control --hwnd click | | UIA element with runtime_id | uia-control --element-id "42-3155764" click | | Text-based element | find ocr "text" → action click --element-id | | Qt applications | Add --skip-actions --skip-state for faster UIA |

Key Rules

Re-snapshot after actions: UI state changes after clicks/typing

Use --dry-run: Preview operations before execution

Prefer runtime_id: More reliable than automation_id

Report window_id: Always include exact window identifier in results

Coordinate system: relative_rect = window-relative; absolute_rect = screen-absolute

Application Guides

WeChat Automation — WeChat 4.1.6+ messaging, contacts, files, calls, Moments

References

Dependencies — pywinauto, pywin32, Pillow, wx-ocr (optional)

Output Format — JSON structure, exit codes, boundary markers

Coordinates — Coordinate system details

Security Guidelines — Detailed safety practices