2.10.1

juliangarnier/anime2.10.1Sep 7, 2022by vladmandic

AI Summary

A major update that switches the default backend to WebGL, adds InsightFace and MobileFaceNet models, and introduces model hosting support. It also includes significant improvements to type safety, testing coverage, and error handling.

Key Highlights

  • Switched default backend from custom `humangl` to `webgl`.
  • Added InsightFace and MobileFaceNet models for face embedding.
  • Added support for hosting models on GitHub and Google Cloud Bucket.
  • Enhanced type safety and increased test coverage for NodeJS and Browser.

New Features

  • InsightFace model support
  • MobileFaceNet model support
  • Model hosting support (GitHub/Cloud)
  • WebGL backend
  • Type safety improvements

Full Release Notes

**Human v2.10** is flagged as major release purely due to number of changes and fixes, but there are no breaking changes

- Update **TFJS** to **3.20.0**  
- Update **TypeScript** to **4.8**  
- Switch default backend from custom `humangl` to `webgl`  
  Stability and availability of features in standard **TFJS** allows to switch back  
- Add **InsightFace** model as alternative for face embedding/descriptor detection  
  Compatible with multiple variations of **InsightFace** models  
  Configurable using `config.face.insightface` config section  
  See `demo/faceid/index.ts` for usage  
  Models can be downloaded from <https://github.com/vladmandic/insightface>  
- Add `human.check()` which validates all kernel ops for currently loaded models with currently selected backend  
  Example: `console.error(human.check());`  
- Add `config.softwareKernels` config option which uses **CPU** implementation for missing ops  
  Disabled by default  
  If enabled, it is used by face and hand rotation correction (`config.face.rotation` and `config.hand.rotation`)  
- Add underlying **tensorflow** library version detection when running in NodeJS to  
  `human.env` and check if **GPU** is used for acceleration  
  Example: `console.log(human.env.tensorflow)`  
- Treat models that cannot be found & loaded as non-critical error  
  Instead of creating runtime exception, `human` will now report that model could not be loaded  
- Improve `human.reset()` method to reset all config values to defaults
- Host models in <https://github.com/vladmandic/human-models>  
  Models can be directly used without downloading to local storage  
  Example: `modelBasePath: 'https://vladmandic.github.io/human-models/models/'`  
- Allow hosting models in **Google Cloud Bucket**  
  Hosted models can be directly used without downloading to local storage  
  Example: `modelBasePath: 'https://storage.googleapis.com/human-models/'`  
- Stricter linting rules for both **TypeScript** and **JavaScript**  
  See `./eslintrc.json` for details  
- Enhanced type safety across entire library  
- Fix **MobileFaceNet** model as alternative for face embedding/descriptor detection  
  Configurable using `config.face.mobilefacenet` config section  
- Fix **EfficientPose** module as alternative body detection  
- Fix **NanoDet** module as alternative object detection  
- Fix `demo/multithread/node-multiprocess.js` demo  
- Fix `human.match` when using mixed descriptor lengths  
- Fix **WASM** feature detection issue in TFJS with Edge/Chromium  
  Example: `console.log(human.env.wasm)`  
- Reorganized init & load order for faster library startup
- Increased **NodeJS** test coverage  
  Run using: `npm run test`  
  Runs tests for `tfjs-node`, `tfjs-node-gpu` and `wasm`  
- Increased **Browser** test coverage  
  Run using: `demo/browser.html`  
  Runs tests for `webgl`, `humangl`, `webgpu` and `wasm`  
  Runs tests for ESM and IIFE versions of library  
- Add `config.flags` settings to allow setting of custom **TFJS** flags during backend configuration
- Increase availability of alternative models  
  See `models/model.json` for full list  
- Update profiling methods in `human.profile()`  
- Update project dependencies to latest versions