Starstruk Logo MarkStarstruk

v0.2.81

mlc-ai/web-llmv0.2.81Feb 18, 2026by akaashrp

AI Summary

Major release featuring Engine and API overhaul with ChatModule refactored into Engine/MLCEngine, consolidated constructor/reload behavior, multi-model loading, and improved worker lifecycle. Includes OpenAI API mirroring, expanded prebuilt model support across Llama, Mistral, Gemma, Qwen, and Phi families, WebGPU performance improvements, and XGrammar integration for JSON-schema constrained generation.

Key Highlights

  • Engine and API overhaul: ChatModule refactored into Engine/MLCEngine with multi-model loading
  • OpenAI API support: mirrored chat/completions APIs with function calling and embeddings
  • Expanded prebuilt model support: Llama 2/3/3.1/3.2, Mistral, Gemma 2, Qwen2/2.5/3, Phi family including vision models
  • WebGPU improvements: more GPU-side kernels, better OOM/deviceLost handling, IndexedDB caching
  • XGrammar integration: JSON-schema and grammar-constrained generation with structural tags

Breaking Changes

  • ChatModule API refactored into Engine/MLCEngine architecture

New Features

  • Engine/MLCEngine architecture with consolidated constructor/reload behavior
  • Multi-model loading support
  • Improved worker lifecycle and concurrency handling
  • OpenAI-compatible chat/completions API mirroring
  • Function calling support
  • Embeddings support
  • Unified conversation template schema with custom templates
  • WebGPU performance and reliability improvements
  • IndexedDB caching support
  • XGrammar JSON-schema/grammar-constrained generation
  • XGrammar structural tag for tool calls
  • TVM-FFI integration updates
  • ServiceWorkerEngine and Chrome extension demos
  • New RAG and doc-chat examples

Full Release Notes

* Engine and API overhaul: ChatModule refactored into Engine/MLCEngine, consolidated constructor/reload behavior, multi-model loading, better worker lifecycle, concurrency handling
* OpenAI API: mirror chat/completions APIs, stateful options, function calling and embeddings support
* Conversation templates: unified conversation template schema with custom templates
* Expanded prebuilt model support: added support for more models (Llama 2/3/3.1/3.2, Mistral variants, Gemma 2, Qwen2/2.5/3, Phi family, including vision)
* Runtime and caching: WebGPU performance/reliability improvements (more GPU-side kernels, better OOM/deviceLost handling), wasm/prebuilt versioning updates, support IndexedDB caching
* XGrammar integration: JSON-schema/grammar-constrained generation, XGrammar structural tag
* TVM-FFI integration: refactor for compatibility with more recent TVM commits and TVM FFI 
* Examples: ServiceWorkerEngine + updated Chrome extension demos, new RAG/doc-chat examples, tool calls via structural tag
* CI: GitHub actions for linting and pre-commit hooks 

**Full Changelog**: https://github.com/mlc-ai/web-llm/compare/v0.2.0...v0.2.81