v0.3.3

## Major changes

* StarCoder2 support
* Performance optimization and LoRA support for Gemma
* 2/3/8-bit GPTQ support
* Integrate Marlin Kernels for Int4 GPTQ inference
* Performance optimization for MoE kernel
* [Experimental] AWS Inferentia2 support
* [Experimental] Structured output (JSON, Regex) in OpenAI Server

## What's Changed
* Update a comment in `benchmark_serving.py` by @ronensc in https://github.com/vllm-project/vllm/pull/2934
* Added early stopping to completion APIs by @Maxusmusti in https://github.com/vllm-project/vllm/pull/2939
* Migrate MistralForCausalLM to LlamaForCausalLM by @esmeetu in https://github.com/vllm-project/vllm/pull/2868
* Use Llama RMSNorm for Gemma by @WoosukKwon in https://github.com/vllm-project/vllm/pull/2974
* chore(vllm): codespell for spell checking  by @mspronesti in https://github.com/vllm-project/vllm/pull/2820
* Optimize GeGLU layer in Gemma by @WoosukKwon in https://github.com/vllm-project/vllm/pull/2975
* [FIX] Fix issue #2904 by @44670 in https://github.com/vllm-project/vllm/pull/2983
* Remove Flash Attention in test env by @WoosukKwon in https://github.com/vllm-project/vllm/pull/2982
* Include tokens from prompt phase in `counter_generation_tokens` by @ronensc in https://github.com/vllm-project/vllm/pull/2802
* Fix nvcc not found in vllm-openai image by @zhaoyang-star in https://github.com/vllm-project/vllm/pull/2781
* [Fix] Fix assertion on Mistral YaRN model len by @WoosukKwon in https://github.com/vllm-project/vllm/pull/2984
* Port metrics from `aioprometheus` to `prometheus_client` by @hmellor in https://github.com/vllm-project/vllm/pull/2730
* Add LogProbs for Chat Completions in OpenAI by @jlcmoore in https://github.com/vllm-project/vllm/pull/2918
* Optimized fused MoE Kernel, take 2 by @pcmoritz in https://github.com/vllm-project/vllm/pull/2979
* [Minor] Remove gather_cached_kv kernel by @WoosukKwon in https://github.com/vllm-project/vllm/pull/3043
* [Minor] Remove unused config file by @esmeetu in https://github.com/vllm-project/vllm/pull/3039
* Fix using CuPy for eager mode by @esmeetu in https://github.com/vllm-project/vllm/pull/3037
* Fix stablelm by @esmeetu in https://github.com/vllm-project/vllm/pull/3038
* Support Orion model by @dachengai in https://github.com/vllm-project/vllm/pull/2539
* fix `get_ip` error in pure ipv6 environment by @Jingru in https://github.com/vllm-project/vllm/pull/2931
* [Minor] Fix type annotation in fused moe by @WoosukKwon in https://github.com/vllm-project/vllm/pull/3045
* Support logit bias for OpenAI API by @dylanwhawk in https://github.com/vllm-project/vllm/pull/3027
* [Minor] Fix StableLMEpochForCausalLM -> StableLmForCausalLM by @WoosukKwon in https://github.com/vllm-project/vllm/pull/3046
* Enables GQA support in the prefix prefill kernels by @sighingnow in https://github.com/vllm-project/vllm/pull/3007
* multi-lora documentation fix by @ElefHead in https://github.com/vllm-project/vllm/pull/3064
* Restrict prometheus_client >= 0.18.0 to prevent errors when importing pkgs by @AllenDou in https://github.com/vllm-project/vllm/pull/3070
* Support inference with transformers-neuronx by @liangfu in https://github.com/vllm-project/vllm/pull/2569
* Add LoRA support for Gemma by @WoosukKwon in https://github.com/vllm-project/vllm/pull/3050
* Add Support for 2/3/8-bit GPTQ Quantization Models by @chu-tianxiang in https://github.com/vllm-project/vllm/pull/2330
* Fix: `AttributeError` in OpenAI-compatible server by @jaywonchung in https://github.com/vllm-project/vllm/pull/3018
* add cache_config's info to prometheus metrics. by @AllenDou in https://github.com/vllm-project/vllm/pull/3100
* Support starcoder2 architecture by @sh0416 in https://github.com/vllm-project/vllm/pull/3089
* Fix building from source on WSL by @aliencaocao in https://github.com/vllm-project/vllm/pull/3112
* [Fix] Don't deep-copy LogitsProcessors when copying SamplingParams by @njhill in https://github.com/vllm-project/vllm/pull/3099
* Add guided decoding for OpenAI API server by @felixzhu555 in https://github.com/vllm-project/vllm/pull/2819
* Fix: Output text is always truncated in some models by @HyperdriveHustle in https://github.com/vllm-project/vllm/pull/3016
* Remove exclude_unset in streaming response by @sh0416 in https://github.com/vllm-project/vllm/pull/3143
* docs: Add tutorial on deploying vLLM model with KServe by @terrytangyuan in https://github.com/vllm-project/vllm/pull/2586
* fix relative import path of protocol.py by @Huarong in https://github.com/vllm-project/vllm/pull/3134
* Integrate Marlin Kernels for Int4 GPTQ inference by @robertgshaw2-neuralmagic in https://github.com/vllm-project/vllm/pull/2497
* Bump up to v0.3.3 by @WoosukKwon in https://github.com/vllm-project/vllm/pull/3129

## New Contributors
* @Maxusmusti made their first contribution in https://github.com/vllm-project/vllm/pull/2939
* @44670 made their first contribution in https://github.com/vllm-project/vllm/pull/2983
* @jlcmoore made their first contribution in https://github.com/vllm-project/vllm/pull/2918
* @dachengai made their first contribution in https://github.com/vllm-project/vllm/pull/2539
* @dylanwhawk made their first contribution in https://github.com/vllm-project/vllm/pull/3027
* @ElefHead made their first contribution in https://github.com/vllm-project/vllm/pull/3064
* @AllenDou made their first contribution in https://github.com/vllm-project/vllm/pull/3070
* @jaywonchung made their first contribution in https://github.com/vllm-project/vllm/pull/3018
* @sh0416 made their first contribution in https://github.com/vllm-project/vllm/pull/3089
* @aliencaocao made their first contribution in https://github.com/vllm-project/vllm/pull/3112
* @felixzhu555 made their first contribution in https://github.com/vllm-project/vllm/pull/2819
* @HyperdriveHustle made their first contribution in https://github.com/vllm-project/vllm/pull/3016
* @terrytangyuan made their first contribution in https://github.com/vllm-project/vllm/pull/2586
* @Huarong made their first contribution in https://github.com/vllm-project/vllm/pull/3134

**Full Changelog**: https://github.com/vllm-project/vllm/compare/v0.3.2...v0.3.3