0.32.0

takahirom/arbigent0.32.0Mar 3, 2025by takahirom

AI Summary

This release focuses on enhancing AI accuracy through configurable options and refactoring prompt handling. It introduces settings for temperature, image detail, and WebP support while improving screenshot processing.

Key Highlights

Added AiOptions for controlling AI behavior, including Temperature, Image Detail, and Image Format settings.
Implemented support for lossy WebP images to potentially improve AI image understanding.
Refactored UserPromptTemplate to include an index element for better UI state representation.
Improved screenshot handling and cleanup in the refactored codebase.

New Features

Temperature control for AI operations
High and low fidelity image detail options
Lossy WebP image format support
UserPromptTemplate index element addition
Refactored screenshot processing

Full Release Notes

## Add AiOptions for Exploring and Improving AI Accuracy

<img width="388" alt="image" src="https://github.com/user-attachments/assets/034d0bcc-f6d1-470b-bf85-3037b0e6c3b2" />

We introduced several options to enhance AI accuracy:
* Temperature: This controls how randomly the AI operates.
* Image Detail: Options like "high" and "low" determine how the AI interprets images.
  Please refer to https://platform.openai.com/docs/guides/vision#low-or-high-fidelity-image-understanding for more details. I don’t think you need to adjust this, but you can experiment to see if it makes a difference.
* Image Format: 
  We can now pass lossy WebP images to the AI. I’m not entirely sure why, but I believe it may improve the AI’s image understanding. Feel free to try it out and see the results.

## What's Changed
* Add tests for UserPromptTemplate to validate command template handling by @takahirom in https://github.com/takahirom/arbigent/pull/187
* Add index element to UserPromptTemplate for enhanced UI state representation by @takahirom in https://github.com/takahirom/arbigent/pull/188
* Add ai options support by @takahirom in https://github.com/takahirom/arbigent/pull/189
* Update UserPromptTemplate to simplify UI instructions by @takahirom in https://github.com/takahirom/arbigent/pull/190
* Add support for custom image formats in AI options by @takahirom in https://github.com/takahirom/arbigent/pull/192
* Add image detail options to AI settings and enhance UI components by @takahirom in https://github.com/takahirom/arbigent/pull/191
* Add options to e2e by @takahirom in https://github.com/takahirom/arbigent/pull/193
* Refactor screenshot handling to improve image processing and cleanup by @takahirom in https://github.com/takahirom/arbigent/pull/194


**Full Changelog**: https://github.com/takahirom/arbigent/compare/0.31.0...0.32.0