2
1 Comment

Show IH: I gave my AI agent real hands on Windows without screenshots

Most computer-use agents send a screenshot to a vision model every step and guess pixel coordinates - slow, expensive, and it breaks the moment a window resizes.

Built DAIMON Hands instead: it reads the Windows UI Automation tree directly (the same data screen readers use), so the agent knows a button's location by name, not by staring at a picture of it.

Two plain-Python CLI files, zero framework lock-in: sense.py lists windows and finds elements by name, with OpenCV color/template detection for anything outside the UIA tree (games, canvases). do.py clicks, types, sends hotkeys, drags, scrolls, opens apps, and launches URLs. It also ships a focus-safety guard most automation libraries skip: it refuses to type a single keystroke unless the OS-reported foreground window actually matches what you expect, and tells you exactly why when it refuses.

Drop it into any agent framework - LangChain, a raw Claude tool-use loop, a cron script - from any language, since it's just CLI calls.

$15, MIT-style license: https://store.agathodamon.com/#visioncontrol

Curious if anyone else here is building computer-use agents and hitting the same screenshot-cost problem.

-agathodamon

on August 1, 2026
  1. 1

    The framework-agnostic part is interesting.

    For people who've tried integrating this into an agent, where has the friction actually shown up — getting the agent to understand the UIA output, or getting it to act reliably once it has that structure?