0.83.0
takahirom/arbigent0.83.0Sep 16, 2026by takahirom
Upgrading?
AI Summary
This release extends project variables to initialization methods, enforces error handling for unresolved placeholders, and fixes a regression where the AI decision request did not include the annotated screenshot.
Key Highlights
- Project variables (`{{name}}`) now resolve in `LaunchApp`, `CleanupData`, and `OpenLink` initialization methods.
- Unresolved placeholders now cause a scenario failure before execution, providing clear error messages.
- The AI decision request now includes the Set-of-Mark annotated screenshot instead of just the raw screenshot.
Breaking Changes
- Unresolved `{{name}}` placeholders now fail the scenario before it runs.
New Features
- Project-level variables in initialization methods
- Unified error reporting for undefined variables before execution
- Sending annotated screenshots in AI decision requests
Full Release Notes
This release extends `{{name}}` variables to initialization methods, makes an unresolved placeholder an error, and fixes a regression that sent the model the screenshot without the numbered boxes.
## Project Variables in Initialization Methods (#443)
`{{name}}` now resolves in `LaunchApp.packageName`, `CleanupData.packageName` and `OpenLink.link`, with defaults in the project YAML:
```yaml
settings:
variables:
appId: "com.example.app"
scenarios:
- id: "launch-app"
goal: "Confirm the home screen is shown"
initializationMethods:
- type: "LaunchApp"
packageName: "{{appId}}"
```
```bash
arbigent run --variables="appId=com.example.app.debug"
```
`--variables` (and `.arbigent/settings.yml`, and the UI) overrides `settings.variables` per key, so a job can swap out just the application id of a debug build. `{{inputs.*}}` resolves in those fields too, and its load-time check was extended to match.
## An Unresolved `{{name}}` Fails the Scenario Before It Runs (#444)
**Breaking change.** A goal with a `{{name}}` that nothing defines used to run with the placeholder passed straight to the AI. Now every unresolved reference is reported at once, before any device, AI or MCP work:
```
Scenario "launch-app" references variables that are not defined:
- {{appId}} in LaunchApp packageName of scenario "launch-app"
- {{user}} in the goal of scenario "launch-app"
Give it a default in settings.variables in the project YAML, pass it in with
--variables="appId=com.example.app", or write \{{appId}} if you meant the text itself.
```
Write `\{{name}}` for literal text. Placeholders outside the variable-name grammar, such as `{{user:id}}`, are unaffected. One rejected scenario does not stop the others, and loading a project never fails for this.
## The Model Sees the Annotated Screenshot Again (#445)
The Set-of-Mark annotated screenshot — the numbered boxes that map what the model sees onto the `<ELEMENTS>` indices — was drawn, saved and shown in the UI, but the **raw** screenshot was what went into the decision request. It had been that way for eight months. Both providers now send the annotated file; image assertions keep the raw screenshot on purpose.
The annotated image is drawn in the element coordinate system, so the boxes land on the elements. On Android that is pixels and the dimensions do not change; the drawn boxes make the PNG about 17% larger. On iOS it is logical points, so what is sent is 393x852 instead of the raw 1178x2556 — a change of coordinate system, not a quality setting. Input tokens stay about the same either way.
## What's Changed
* Support project-level variables in initialization methods by @takahirom in https://github.com/takahirom/arbigent/pull/443
* Fail a scenario with unresolved variables before it runs by @takahirom in https://github.com/takahirom/arbigent/pull/444
* Send the Set-of-Mark annotated screenshot in the decision request by @takahirom in https://github.com/takahirom/arbigent/pull/445
**Full Changelog**: https://github.com/takahirom/arbigent/compare/0.82.0...0.83.0
Related Repositories
BreezeAPP 是一款為 Android 和 iOS 平台開發的純手機 AI 應用程式。從 App Store下載,即可在不連網的狀態下享受多項 AI 功能。源碼由聯發創新基地(MediaTek Research)提供。我們旨在推廣兩個概念: 人人都可以在自己的手機上自由選擇並運行不同的LLM - one is free to choose one's own LLM to run on a phone,以及任何app開發者都可以輕鬆寫作創意的純手機AI應用 - any dev can create purely phone-based AI apps easily。
A collective list of free APIs
The AI that really does things. Any OS. Any Platform. The lobster way. 🦞
Interactive roadmaps, guides and other educational content to help developers grow in their careers.
Reverse engineer anything with agents, from app behavior down to native binaries.