v0.2.2

## Major changes

* Bump up to PyTorch v2.1 + CUDA 12.1 ([vLLM+CUDA 11.8 is also provided](https://vllm.readthedocs.io/en/latest/getting_started/installation.html#install-with-pip))
* Extensive refactoring for better tensor parallelism & quantization support
* New models: Yi, ChatGLM, Phi
* Changes in scheduler: from 1D flattened input tensor to 2D tensor
* AWQ support for all models
* Added LogitsProcessor API
* Preliminary support for SqueezeLLM

## What's Changed
* Change scheduler & input tensor shape by @WoosukKwon in https://github.com/vllm-project/vllm/pull/1381
* Add Mistral 7B to `test_models` by @WoosukKwon in https://github.com/vllm-project/vllm/pull/1366
* fix typo by @WrRan in https://github.com/vllm-project/vllm/pull/1383
* Fix TP bug by @WoosukKwon in https://github.com/vllm-project/vllm/pull/1389
* Fix type hints by @lxrite in https://github.com/vllm-project/vllm/pull/1427
* remove useless statements by @WrRan in https://github.com/vllm-project/vllm/pull/1408
* Pin dependency versions by @thiagosalvatore in https://github.com/vllm-project/vllm/pull/1429
* SqueezeLLM Support by @chooper1 in https://github.com/vllm-project/vllm/pull/1326
* aquila model add rope_scaling by @Sanster in https://github.com/vllm-project/vllm/pull/1457
* fix: don't skip first special token. by @gesanqiu in https://github.com/vllm-project/vllm/pull/1497
* Support repetition_penalty by @beginlner in https://github.com/vllm-project/vllm/pull/1424
* Fix bias in InternLM by @WoosukKwon in https://github.com/vllm-project/vllm/pull/1501
* Delay GPU->CPU sync in sampling by @Yard1 in https://github.com/vllm-project/vllm/pull/1337
* Refactor LLMEngine demo script for clarity and modularity by @iongpt in https://github.com/vllm-project/vllm/pull/1413
* Fix logging issues by @Tostino in https://github.com/vllm-project/vllm/pull/1494
* Add py.typed so consumers of vLLM can get type checking by @jroesch in https://github.com/vllm-project/vllm/pull/1509
* vLLM always places spaces between special tokens by @blahblahasdf in https://github.com/vllm-project/vllm/pull/1373
* [Fix] Fix duplicated logging messages by @zhuohan123 in https://github.com/vllm-project/vllm/pull/1524
* Add dockerfile by @skrider in https://github.com/vllm-project/vllm/pull/1350
* Fix integer overflows in attention & cache ops by @WoosukKwon in https://github.com/vllm-project/vllm/pull/1514
* [Small] Formatter only checks lints in changed files by @cadedaniel in https://github.com/vllm-project/vllm/pull/1528
* Add `MptForCausalLM` key in model_loader by @wenfeiy-db in https://github.com/vllm-project/vllm/pull/1526
* [BugFix] Fix a bug when engine_use_ray=True and worker_use_ray=False and TP>1 by @beginlner in https://github.com/vllm-project/vllm/pull/1531
* Adding a health endpoint by @Fluder-Paradyne in https://github.com/vllm-project/vllm/pull/1540
* Remove `MPTConfig` by @WoosukKwon in https://github.com/vllm-project/vllm/pull/1529
* Force paged attention v2 for long contexts by @Yard1 in https://github.com/vllm-project/vllm/pull/1510
* docs: add description by @lots-o in https://github.com/vllm-project/vllm/pull/1553
* Added logits processor API to sampling params by @noamgat in https://github.com/vllm-project/vllm/pull/1469
* YaRN support implementation by @Yard1 in https://github.com/vllm-project/vllm/pull/1264
* Add Quantization and AutoAWQ to docs by @casper-hansen in https://github.com/vllm-project/vllm/pull/1235
* Support Yi model by @esmeetu in https://github.com/vllm-project/vllm/pull/1567
* ChatGLM2 Support by @GoHomeToMacDonal in https://github.com/vllm-project/vllm/pull/1261
* Upgrade to CUDA 12 by @zhuohan123 in https://github.com/vllm-project/vllm/pull/1527
* [Worker] Fix input_metadata.selected_token_indices in worker by @ymwangg in https://github.com/vllm-project/vllm/pull/1546
* Build CUDA11.8 wheels for release by @WoosukKwon in https://github.com/vllm-project/vllm/pull/1596
* Add Yi model to quantization support by @forpanyang in https://github.com/vllm-project/vllm/pull/1600
* Dockerfile: Upgrade Cuda to 12.1 by @GhaziSyed in https://github.com/vllm-project/vllm/pull/1609
* config parser: add ChatGLM2 seq_length to `_get_and_verify_max_len` by @irasin in https://github.com/vllm-project/vllm/pull/1617
* Fix cpu heavy code in async function _AsyncLLMEngine._run_workers_async by @dominik-schwabe in https://github.com/vllm-project/vllm/pull/1628
* Fix #1474 - gptj AssertionError : assert param_slice.shape == loaded_weight.shape by @lihuahua123 in https://github.com/vllm-project/vllm/pull/1631
* [Minor] Move RoPE selection logic to `get_rope` by @WoosukKwon in https://github.com/vllm-project/vllm/pull/1633
* Add DeepSpeed MII backend to benchmark script by @WoosukKwon in https://github.com/vllm-project/vllm/pull/1649
* TP/quantization/weight loading refactor part 2 - Refactor quantized linear logic and extend quantization support to all models by @zhuohan123 in https://github.com/vllm-project/vllm/pull/1622
* Remove `MptConfig`   by @megha95 in https://github.com/vllm-project/vllm/pull/1668
* feat(config): support parsing torch.dtype by @aarnphm in https://github.com/vllm-project/vllm/pull/1641
* Fix loading error when safetensors contains empty tensor by @twaka in https://github.com/vllm-project/vllm/pull/1687
* [Minor] Fix duplication of ignored seq group in engine step by @simon-mo in https://github.com/vllm-project/vllm/pull/1666
* [models] Microsoft Phi 1.5 by @maximzubkov in https://github.com/vllm-project/vllm/pull/1664
* [Fix] Update Supported Models List by @zhuohan123 in https://github.com/vllm-project/vllm/pull/1690
* Return usage for openai requests by @ichernev in https://github.com/vllm-project/vllm/pull/1663
* [Fix] Fix comm test by @zhuohan123 in https://github.com/vllm-project/vllm/pull/1691
* Update the adding-model doc according to the new refactor by @zhuohan123 in https://github.com/vllm-project/vllm/pull/1692
* Add 'not' to this annotation: "#FIXME(woosuk): Do not use internal method" by @linotfan in https://github.com/vllm-project/vllm/pull/1704
* Support Min P Sampler by @esmeetu in https://github.com/vllm-project/vllm/pull/1642
* Read quantization_config in hf config by @WoosukKwon in https://github.com/vllm-project/vllm/pull/1695
* Support download models from www.modelscope.cn by @liuyhwangyh in https://github.com/vllm-project/vllm/pull/1588
* follow up of #1687 when safetensors model contains 0-rank tensors by @twaka in https://github.com/vllm-project/vllm/pull/1696
* Add AWQ support for all models by @WoosukKwon in https://github.com/vllm-project/vllm/pull/1714
* Support fused add rmsnorm for LLaMA by @beginlner in https://github.com/vllm-project/vllm/pull/1667
* [Fix] Fix warning msg on quantization by @WoosukKwon in https://github.com/vllm-project/vllm/pull/1715
* Bump up the version to v0.2.2 by @WoosukKwon in https://github.com/vllm-project/vllm/pull/1689

## New Contributors
* @lxrite made their first contribution in https://github.com/vllm-project/vllm/pull/1427
* @thiagosalvatore made their first contribution in https://github.com/vllm-project/vllm/pull/1429
* @chooper1 made their first contribution in https://github.com/vllm-project/vllm/pull/1326
* @beginlner made their first contribution in https://github.com/vllm-project/vllm/pull/1424
* @iongpt made their first contribution in https://github.com/vllm-project/vllm/pull/1413
* @Tostino made their first contribution in https://github.com/vllm-project/vllm/pull/1494
* @jroesch made their first contribution in https://github.com/vllm-project/vllm/pull/1509
* @skrider made their first contribution in https://github.com/vllm-project/vllm/pull/1350
* @cadedaniel made their first contribution in https://github.com/vllm-project/vllm/pull/1528
* @wenfeiy-db made their first contribution in https://github.com/vllm-project/vllm/pull/1526
* @Fluder-Paradyne made their first contribution in https://github.com/vllm-project/vllm/pull/1540
* @lots-o made their first contribution in https://github.com/vllm-project/vllm/pull/1553
* @noamgat made their first contribution in https://github.com/vllm-project/vllm/pull/1469
* @casper-hansen made their first contribution in https://github.com/vllm-project/vllm/pull/1235
* @GoHomeToMacDonal made their first contribution in https://github.com/vllm-project/vllm/pull/1261
* @ymwangg made their first contribution in https://github.com/vllm-project/vllm/pull/1546
* @forpanyang made their first contribution in https://github.com/vllm-project/vllm/pull/1600
* @GhaziSyed made their first contribution in https://github.com/vllm-project/vllm/pull/1609
* @irasin made their first contribution in https://github.com/vllm-project/vllm/pull/1617
* @dominik-schwabe made their first contribution in https://github.com/vllm-project/vllm/pull/1628
* @lihuahua123 made their first contribution in https://github.com/vllm-project/vllm/pull/1631
* @megha95 made their first contribution in https://github.com/vllm-project/vllm/pull/1668
* @aarnphm made their first contribution in https://github.com/vllm-project/vllm/pull/1641
* @simon-mo made their first contribution in https://github.com/vllm-project/vllm/pull/1666
* @maximzubkov made their first contribution in https://github.com/vllm-project/vllm/pull/1664
* @ichernev made their first contribution in https://github.com/vllm-project/vllm/pull/1663
* @linotfan made their first contribution in https://github.com/vllm-project/vllm/pull/1704
* @liuyhwangyh made their first contribution in https://github.com/vllm-project/vllm/pull/1588

**Full Changelog**: https://github.com/vllm-project/vllm/compare/v0.2.1...v0.2.2