v2.5.0

Project-HAMi/HAMiv2.5.0Feb 6, 2025by github-actions[bot]

AI Summary

Major update introducing Dynamic MIG support and centralizing all configurations into a single configMap, ensuring safety during reinstallations.

Key Highlights

  • Support dynamic MIG feature
  • All configurations in a central configMap
  • Reinstall HAMi will not crash GPU tasks

New Features

  • Dynamic MIG feature
  • Centralized configMap
  • Fix for cuMallocAsync issues
  • Fix for high glibc images

Full Release Notes

<!-- Release notes generated using configuration in .github/release.yml at v2.5.0 -->

Major features:
1. Support dynamic mig feature, please refer to this [document](https://github.com/Project-HAMi/HAMi/blob/master/docs/dynamic-mig-support.md)
2. Reinstall Hami will NOT crash GPU tasks
3. Put all configurations into a configMap, you can customize hami installation by modify its content: see [details](https://github.com/Project-HAMi/HAMi/blob/master/docs/config.md) 

Major bug fixes:
1. Fix an issue where hami-core will stuck on tasks using 'cuMallocAsync'
2. Fix hami-core stuck on high glib images, like 'tf-serving:latest'

## What's Changed
### ⬆️ Dependencies
* Bump aquasecurity/trivy-action from 0.28.0 to 0.29.0 by @dependabot in https://github.com/Project-HAMi/HAMi/pull/631
* Bump nvidia/cuda from 12.4.1-base-ubuntu22.04 to 12.6.3-base-ubuntu22.04 in /docker by @dependabot in https://github.com/Project-HAMi/HAMi/pull/676
* Bump actions/upload-artifact from 4.4.3 to 4.5.0 by @dependabot in https://github.com/Project-HAMi/HAMi/pull/717
* Bump docker/build-push-action from 6.9.0 to 6.10.0 by @dependabot in https://github.com/Project-HAMi/HAMi/pull/644
* Bump docker/build-push-action from 6.10.0 to 6.11.0 by @dependabot in https://github.com/Project-HAMi/HAMi/pull/792
### 🔨 Other Changes
* Fix Kubernetes version string handling by stripping metadata by @Nimbus318 in https://github.com/Project-HAMi/HAMi/pull/623
* Update vGPUmonitor to add dynamic adjustment on core and memory limit by @archlitchi in https://github.com/Project-HAMi/HAMi/pull/624
* feat: support device plugin daemonset update strategy by @devenami in https://github.com/Project-HAMi/HAMi/pull/628
* add ut about schedule policy by @yt-huang in https://github.com/Project-HAMi/HAMi/pull/638
* Fix: Refactor the license based on the approaches used in OpenSearch and ElasticSearch. by @haitwang-cloud in https://github.com/Project-HAMi/HAMi/pull/626
* add ut for the scheduler by @shijinye in https://github.com/Project-HAMi/HAMi/pull/645
* docs(issue-tmpl): add FAQ link to issue templates by @Nimbus318 in https://github.com/Project-HAMi/HAMi/pull/647
* fix: filter device registry to node by @lengrongfu in https://github.com/Project-HAMi/HAMi/pull/639
* Add self-hosted runner by @archlitchi in https://github.com/Project-HAMi/HAMi/pull/659
* fix-example-yaml  by @WQL782795 in https://github.com/Project-HAMi/HAMi/pull/667
* update docs by @yangshiqi in https://github.com/Project-HAMi/HAMi/pull/668
* add ut for ascend by @shijinye in https://github.com/Project-HAMi/HAMi/pull/664
* optimization map init in test by @lengrongfu in https://github.com/Project-HAMi/HAMi/pull/678
* Optimize monitor by @for800000 in https://github.com/Project-HAMi/HAMi/pull/683
* fix code lint faild by @lengrongfu in https://github.com/Project-HAMi/HAMi/pull/685
* fix(helm): Add NODE_NAME env var to the vgpu-monitor container from spec.nodeName by @Nimbus318 in https://github.com/Project-HAMi/HAMi/pull/687
* fix vGPUmonitor deviceidx is always 0 by @lengrongfu in https://github.com/Project-HAMi/HAMi/pull/684
* add ut for pkg/scheduler/event.go by @Penguin-zlh in https://github.com/Project-HAMi/HAMi/pull/688
* add ut for nodes by @shijinye in https://github.com/Project-HAMi/HAMi/pull/695
* add license for pkg/scheduler/event_test.go by @Penguin-zlh in https://github.com/Project-HAMi/HAMi/pull/706
* fix: exception happen when creating multiple ascend-gpu pods concurrently by @lijm87 in https://github.com/Project-HAMi/HAMi/pull/575
* add ut for device/nvidia by @shijinye in https://github.com/Project-HAMi/HAMi/pull/657
* add ut for pkg/monitor/nvidia/v0/spec.go by @yt-huang in https://github.com/Project-HAMi/HAMi/pull/670
* Enable Dynamic-mig feature for HAMi by @archlitchi in https://github.com/Project-HAMi/HAMi/pull/708
* Fix chart can not be deployed properly by @archlitchi in https://github.com/Project-HAMi/HAMi/pull/711
* Fix NodeLock issue by @archlitchi in https://github.com/Project-HAMi/HAMi/pull/714
* fix example yaml by @lixd in https://github.com/Project-HAMi/HAMi/pull/709
* add ut for device/cambricon by @shijinye in https://github.com/Project-HAMi/HAMi/pull/712
* Update dynamic mig documents and examples by @archlitchi in https://github.com/Project-HAMi/HAMi/pull/718
* random time may be zero by @shijinye in https://github.com/Project-HAMi/HAMi/pull/697
* fix grafana dashboard and clarify dashboard usage more clearly. by @jiangsanyin in https://github.com/Project-HAMi/HAMi/pull/543
* doc(README): add examples for GPU sharing and update-examples by @xiaoyao in https://github.com/Project-HAMi/HAMi/pull/665
* add ut for github.com/Project-HAMi/HAMi/pkg/scheduler/pod.go by @yt-huang in https://github.com/Project-HAMi/HAMi/pull/673
* Add design document to 'dynamic-mig' feature by @archlitchi in https://github.com/Project-HAMi/HAMi/pull/725
* fix(doc): fix a typo and resolve markdown warnings in the tasklist by @elrondwong in https://github.com/Project-HAMi/HAMi/pull/724
* add ut for  pkg/util/nodelock/nodelock.go by @learner0810 in https://github.com/Project-HAMi/HAMi/pull/719
* test: add ut for pkg/version/version.go by @Penguin-zlh in https://github.com/Project-HAMi/HAMi/pull/677
* Update on mig mode by @archlitchi in https://github.com/Project-HAMi/HAMi/pull/726
* Update documents for config & config_cn by @archlitchi in https://github.com/Project-HAMi/HAMi/pull/729
* set PASS_DEVICE_SPECS ENV to device-plugin by @jingzhe6414 in https://github.com/Project-HAMi/HAMi/pull/690
* fix device-plugin-version by @learner0810 in https://github.com/Project-HAMi/HAMi/pull/743
* feat: Return the nodes that failed to be scheduled back to the scheduler by @chaunceyjiang in https://github.com/Project-HAMi/HAMi/pull/746
* fix(log): fix missing log output in nvidiadeviceplugin server by @elrondwong in https://github.com/Project-HAMi/HAMi/pull/735
* support configuration resources limits and requests by @flpanbin in https://github.com/Project-HAMi/HAMi/pull/739
* feat(test): add TestMarshalNodeDevices scenarios by @elrondwong in https://github.com/Project-HAMi/HAMi/pull/747
* print flags for device-plugin and scheduler by @flpanbin in https://github.com/Project-HAMi/HAMi/pull/756
* Fix typos, add more contributors and maintainers. by @yangshiqi in https://github.com/Project-HAMi/HAMi/pull/765
* Add a mind map(Chinese and English) to help understand this project by @oceanweave in https://github.com/Project-HAMi/HAMi/pull/764
* [Docs] update config pages by @windsonsea in https://github.com/Project-HAMi/HAMi/pull/760
* add ut for device-map by @KubeKyrie in https://github.com/Project-HAMi/HAMi/pull/762
* refactor(ci): use go.mod file for Go version in workflows by @yxxhero in https://github.com/Project-HAMi/HAMi/pull/766
* support set log level for device plugin by @flpanbin in https://github.com/Project-HAMi/HAMi/pull/771
* feat: Restart/Upgrade device-plugin will not affect services. by @chaunceyjiang in https://github.com/Project-HAMi/HAMi/pull/767
* add ut nvml devices by @KubeKyrie in https://github.com/Project-HAMi/HAMi/pull/773
* add ut for device-map by @KubeKyrie in https://github.com/Project-HAMi/HAMi/pull/772
* Optimize the time format layout by @learner0810 in https://github.com/Project-HAMi/HAMi/pull/741
* fix: nvidia-device-plugin no version info by @chaunceyjiang in https://github.com/Project-HAMi/HAMi/pull/779
* HAMi supports e2e by @Rei1010 in https://github.com/Project-HAMi/HAMi/pull/775
* Proposal:  enable E2E test by @Rei1010 in https://github.com/Project-HAMi/HAMi/pull/633
* add ut for device/iluvatar by @shijinye in https://github.com/Project-HAMi/HAMi/pull/795
* add ut for device/hygon by @shijinye in https://github.com/Project-HAMi/HAMi/pull/787
* add ut for pkg/monitor/nvidia/v1 by @shijinye in https://github.com/Project-HAMi/HAMi/pull/780
* refactor(logging): enhance log messages for device resource counting  by @haitwang-cloud in https://github.com/Project-HAMi/HAMi/pull/778
* Enrich pod health check by @Rei1010 in https://github.com/Project-HAMi/HAMi/pull/801
* docs: fix broken link by @lixd in https://github.com/Project-HAMi/HAMi/pull/802
* Optimize the E2E execution logic by @Rei1010 in https://github.com/Project-HAMi/HAMi/pull/803
* optimize MetricsBindAddress to MetricsBindPort by @phoenixwu0229 in https://github.com/Project-HAMi/HAMi/pull/796
* fix: handle the node nil issue & E2E test failure by @haitwang-cloud in https://github.com/Project-HAMi/HAMi/pull/804
* add ut for device/mthreads by @shijinye in https://github.com/Project-HAMi/HAMi/pull/808
* fix: Resolve formatting issue in ConfigMap causing display anomalies by @lixd in https://github.com/Project-HAMi/HAMi/pull/814
* [docs] Update ascend910b-support.md by @windsonsea in https://github.com/Project-HAMi/HAMi/pull/816
* Refine metrics logs by @haitwang-cloud in https://github.com/Project-HAMi/HAMi/pull/817
* Update mig-related logics and refine logs  by @archlitchi in https://github.com/Project-HAMi/HAMi/pull/833
* Add 910B4 config to device-configmap for ascend by @lijm87 in https://github.com/Project-HAMi/HAMi/pull/828
* [docs] fix: glibc version requirement in README by @chinaran in https://github.com/Project-HAMi/HAMi/pull/826
* Update HAMi-core for v2.5.0 by @archlitchi in https://github.com/Project-HAMi/HAMi/pull/834
* FIx multi-process device memory count issue by @archlitchi in https://github.com/Project-HAMi/HAMi/pull/835
* bump version to v2.5.0 by @wawa0210 in https://github.com/Project-HAMi/HAMi/pull/836
* Fix CI by @archlitchi in https://github.com/Project-HAMi/HAMi/pull/838
* Fix CI release by @archlitchi in https://github.com/Project-HAMi/HAMi/pull/840
* Fix release ci by @archlitchi in https://github.com/Project-HAMi/HAMi/pull/841
* Fix Dockerfile to make CI pass by @archlitchi in https://github.com/Project-HAMi/HAMi/pull/846
* Fix E2E failure with pod status check by @Rei1010 in https://github.com/Project-HAMi/HAMi/pull/847
* Fix scheduler crash if a 'mig' task running accidentally on a 'hami-core' GPU by @archlitchi in https://github.com/Project-HAMi/HAMi/pull/848

## New Contributors
* @yt-huang made their first contribution in https://github.com/Project-HAMi/HAMi/pull/638
* @shijinye made their first contribution in https://github.com/Project-HAMi/HAMi/pull/645
* @WQL782795 made their first contribution in https://github.com/Project-HAMi/HAMi/pull/667
* @yangshiqi made their first contribution in https://github.com/Project-HAMi/HAMi/pull/668
* @for800000 made their first contribution in https://github.com/Project-HAMi/HAMi/pull/683
* @Penguin-zlh made their first contribution in https://github.com/Project-HAMi/HAMi/pull/688
* @lixd made their first contribution in https://github.com/Project-HAMi/HAMi/pull/709
* @jiangsanyin made their first contribution in https://github.com/Project-HAMi/HAMi/pull/543
* @xiaoyao made their first contribution in https://github.com/Project-HAMi/HAMi/pull/665
* @elrondwong made their first contribution in https://github.com/Project-HAMi/HAMi/pull/724
* @learner0810 made their first contribution in https://github.com/Project-HAMi/HAMi/pull/719
* @jingzhe6414 made their first contribution in https://github.com/Project-HAMi/HAMi/pull/690
* @flpanbin made their first contribution in https://github.com/Project-HAMi/HAMi/pull/739
* @oceanweave made their first contribution in https://github.com/Project-HAMi/HAMi/pull/764
* @windsonsea made their first contribution in https://github.com/Project-HAMi/HAMi/pull/760
* @KubeKyrie made their first contribution in https://github.com/Project-HAMi/HAMi/pull/762
* @yxxhero made their first contribution in https://github.com/Project-HAMi/HAMi/pull/766
* @Rei1010 made their first contribution in https://github.com/Project-HAMi/HAMi/pull/775
* @phoenixwu0229 made their first contribution in https://github.com/Project-HAMi/HAMi/pull/796
* @chinaran made their first contribution in https://github.com/Project-HAMi/HAMi/pull/826

**Full Changelog**: https://github.com/Project-HAMi/HAMi/compare/v2.4.1...v2.5.0