feat(browser): OpenClaw-style aria snapshot + ref-based element addressing

Fork: 0

yangyangxie / cortex-hub

Browse code feat(browser): OpenClaw-style aria snapshot + ref-based element addressing - Implement semantic role tree (aria_snapshot) via Playwright ariaSnapshot() API - Each interactive/content element annotated with [ref=eN] labels - click/type/hover now accept CSS selectors OR ref strings (e.g. 'e3') - navigate auto-returns an aria snapshot for immediate page context - Browser results now include aria_snapshot in stdout for AI reasoning - SKILL.md updated with recommended get_a11y -> click/type workflow - tool.py forwards selector/text/x/y params to BrowserAction proto master
1 parent 236871d commit 1b278e76ff789de4cb15ead698612f921072402a Antigravity AI authored 1 day ago

Browse code

- Implement semantic role tree (aria_snapshot) via Playwright ariaSnapshot() API
- Each interactive/content element annotated with [ref=eN] labels
- click/type/hover now accept CSS selectors OR ref strings (e.g. 'e3')
- navigate auto-returns an aria snapshot for immediate page context
- Browser results now include aria_snapshot in stdout for AI reasoning
- SKILL.md updated with recommended get_a11y -> click/type workflow
- tool.py forwards selector/text/x/y params to BrowserAction proto

master

1 parent 236871d commit 1b278e76ff789de4cb15ead698612f921072402a

Antigravity AI authored 1 day ago

Patch

Unified Split

Showing 4 changed files

Ignore Space Show notes View ai-hub/app/core/grpc/services/grpc_server.py

Ignore Space Show notes View ai-hub/app/core/services/tool.py

Ignore Space Show notes View skills/browser-automation-agent/SKILL.md

Ignore Space Show notes View skills/browser-automation-agent/logic.py

Show line notes below