Starstruk Logo MarkStarstruk

v0.11.0

tracel-ai/burnv0.11.0Dec 1, 2023by nathanielsimard

AI Summary

The main feature is the introduction of automatic kernel fusion, alongside a reorganization of the organization name and backend names.

Key Highlights

  • Introduction of automatic kernel fusion (burn-fusion).
  • Organization renamed from burn-rs to tracel-ai.
  • Refactoring of backend names and feature flags.

Breaking Changes

  • Refactored backend names.
  • Updated the feature flags of burn.

New Features

  • Burn Fusion for automatic kernel optimization.
  • GroupNorm module.
  • New tensor operators: unsqueeze_dim, narrow, stack, chunk, tril, triu.
  • ONNX support for Gather, Cos, Exp, Gelu, Log, Neg, ConvTranspose2D, Sqrt.
  • Matmul and Reduce kernels with autotune capabilities.

Full Release Notes

The main feature of Burn v0.11.0 is automatic kernel fusion, which is still in active development but already usable. Many enhancement and new features have been added throughout the framework, for better efficiency and reliability.

Warnings:

- There are some breaking changes, see below.
- The organization has been renamed from burn-rs to tracel-ai.

# Changes

## Overall changes

- [Breaking] Refactor backend names @nathanielsimard

- [Breaking] Updated the feature flags of burn to improve usability @nathanielsimard

- Update of Burn's Readme @nathanielsimard @louisfd

## Burn Fusion

- Innovative automatic kernel fusion algorithm @nathanielsimard

- Relative computation graph cache @nathanielsimard

## Burn Core

- GroupNorm module @dcvz

- Allow for int and bool constant tensors in modules @nathanielsimard

- Quiet softmax in transformers @wbrickner

## Burn Tensor

- New operators in tensor API: unsqueeze_dim, narrow, stack, chunk, tril, triu @dcvz

- Recip operation support on all backends @gzsombor

- Implement DoubleEndedIterator for DimIter @wcshds

## Burn Compute

- Major Autotune refactor @louisfd

## Burn Import

- ONNX Support for Gather @CohenAriel

- ONNX Support for Cos, Exp, Gelu, Log, Neg @antimora 

- ONNX Support ConvTranspose2D @npatsakula, @antimora, 

- ONNX Support for Sqrt @edmondop

- Support count_include_pad attr in avg_pool2d ONNX @antimora 

## Burn Train

- Add warmup consideration for estimated training time @nathanielsimard

## Burn WGPU

- New Matmul kernels @louisfd

- New Reduce kernel @louisfd

- Add Autotune capabilities to Matmul and Reduce @louisfd

- Support of kernel fusion for element-wise operations @nathanielsimard @louisfd

## Burn Candle

- Support conv_transpose_1d @louisfd

- Enable accelerate for MacOS CPU @dcvz

## Backend Comparison

- Custom Gelu benchmarks @nathanielsimard

- Persistence of results in json @louisfd

## Bugfixes

- Allow arbitrary precision threshold for float equality assertion @meteor-lsw

- Update serde_rusqlite to the new version with MIT/Apache2 license @antimora

- Fix SQLite database tests on Windows @syl20bnr

- Fix max_dim and min_dim tensor operations @gzsombor

- Fix inplace double binary broadcasting in the LibTorch backend @nathanielsimard

## Documentation

- Add Python details in the Book's getting started @antimora

- Miscellaneous Book fixes @syl20bnr @mks-h

## Continuous Integration

- Add test coverage @Luni-4

- Speedup typos check @Luni-4

- Dependency checks @Luni-4

- Vulnerability checks @Luni-4

# Thanks

Thanks to all aforemetioned contributors.