0.31.0

takahirom/arbigent0.31.0Mar 1, 2025by takahirom

AI Summary

This release addresses inconsistency in AI image recognition by optimizing prompts and reducing context size. It introduces a breaking change by renaming a template variable to improve project consistency.

Key Highlights

  • Reduced context size and optimized prompts to improve AI image visibility.
  • Added explicit image referencing in the UI_STATE prompt.
  • Renamed `{{COMMAND_TEMPLATES}}` to `{{ACTION_TEMPLATES}}` (Breaking Change).

Breaking Changes

  • Renamed `{{COMMAND_TEMPLATES}}` to `{{ACTION_TEMPLATES}}` in userPromptTemplate

New Features

  • Improved user prompts with default messages for empty states
  • Refactored agent command types to agent action types

Full Release Notes

AIs tend not to see the images, which makes the test inconsistent. We addressed this with several approaches:
* Reduced context size by optimizing prompts
* Added image referencing in the UI_STATE prompt

## Breaking Change for Users Using userPromptTemplate
To make the project and prompts consistent, we have renamed `{{COMMAND_TEMPLATES}}` to `{{ACTION_TEMPLATES}}`. This will generate an error so you can identify if you are using the incorrect syntax.

This is the latest userPromptTemplate template.

```
<GOAL>{{USER_INPUT_GOAL}}</GOAL>

<STEP>
Current step: {{CURRENT_STEP}}
Max step: {{MAX_STEP}}

<PREVIOUS_STEPS>
{{STEPS}}
</PREVIOUS_STEPS>
</STEP>

<UI_STATE>
Please refer to the image above.
<ELEMENTS>
{{UI_ELEMENTS}}
</ELEMENTS>
<FOCUSED_TREE>
{{FOCUSED_TREE}}
</FOCUSED_TREE>
</UI_STATE>

<INSTRUCTIONS>
Based on the above, decide on the next action to achieve the goal. Please ensure not to repeat the same action. The action must be one of the following:
</INSTRUCTIONS>

<ACTION_OPTIONS>
{{ACTION_TEMPLATES}}
</ACTION_OPTIONS>
```

## What's Changed
* Improve user prompts by adding default messages for empty focused tree and UI elements by @takahirom in https://github.com/takahirom/arbigent/pull/185
* Refactor agent command types to agent action types for consistency in naming by @takahirom in https://github.com/takahirom/arbigent/pull/186


**Full Changelog**: https://github.com/takahirom/arbigent/compare/0.30.0...0.31.0