v0.34.0
2 tiếng nói · chưa gắn nhãn trưởng lab · 2 nền tảng · chưa có số tương tác · Nguồn crawl
Chi tiết nguồn
<h1>v0.34.0</h1> <p><strong>1,061 commits from 412 contributors (120 new).</strong> Thank you!</p> <h2>TL;DR</h2> <ul> <li><strong>10 new models</strong> (Mistral Large 4, MiniMax-M3.1-Flash, Nemotron-3.5-Transcribe, and more!)</li> <li><strong>Extensible KV cache is on by default:</strong> vLLM sizes the KV cache from memory actually free after warmup, so most models get a bigger cache and more throughput - in most cases you should now <strong>remove <code>gpu_memory_utilization</code></strong> from your configs.</li> <li><strong>Model Runner V1 is gone.</strong> Model Runner V2 is now the only GPU runner.</li> <li><strong>Big speedups</strong> on Blackwell and MI355X: up to 1.95× decode (Step5 on GB200), 1.6× throughput (GLM-5.3-Flash), 1.24–1.40× faster long-prompt TTFT (GLM-5.3).</li> <li><strong>Upgrade check:</strong> default block size is now 64, ROCm 7.2 images are gone, and <code>huggingface_hub >= 2.0.0</code> is required. See <a href="#before-you-upgrade">Before you upgrade</a>.</li> </ul> <h2>Before you upgrade</h2> <div class="markdown-alert markdown-alert-important"><p class="markdown-alert-title"><svg data-component="Octicon" class="octicon octicon-report mr-2" viewBox="0 0 16 16" version="1.1" width="16" height="16" aria-hidden="true"><path d="M0 1.75C0 .784.784 0 1.75 0h12.5C15.216 0 16 .784 16 1.75v9.5A1.75 1.75 0 0 1 14.25 13H8.06l-2.573 2.573A1.458 1.458 0 0 1 3 14.543V13H1.75A1.75 1.75 0 0 1 0 11.25Zm1.75-.25a.25.25 0 0 0-.25.25v9.5c0 .138.112.25.25.25h2a.75.75 0 0 1 .75.75v2.19l2.72-2.72a.749.749 0 0 1 .53-.22h6.5a.25.25 0 0 0 .25-.25v-9.5a.25.25 0 0 0-.25-.25Zm7 2.25v2.5a.75.75 0 0 1-1.5 0v-2.5a.75.75 0 0 1 1.5 0ZM9 9a1 1 0 1 1-2 0 1 1 0 0 1 2 0Z"></path></svg>Important</p><p>These changes can break existing setups.</p> </div> <table> <thead> <tr> <th>Change</th> <th>What to do</th> <th>PR</th> </tr> </thead> <tbody> <tr> <td>Model Runner V1 removed</td> <td>Remove <code>VLLM_USE_V2_MODEL_RUNNER=0</code>; it now fails at startup. Port V1 logits processors to <code>vllm.v1.worker.gpu.sample.logits_processor</code>.</td> <td><a class="issue-link js-issue-link" data-error-text="Failed to load title" data-id="6001962406" data-permission-text="Title is private" data-url="https://github.com/vllm-project/vllm/issues/63373" data-hovercard-type="pull_request" data-hovercard-url="/vllm-project/vllm/pull/63373/hovercard" href="https://github.com/vllm-project/vllm/pull/63373">#63373</a></td> </tr> <tr> <td>Default <code>--block-size</code> is 64 (was 16)</td> <td>Pass <code>--block-size 16</code> to keep the old behavior. ROCm paged attention still uses 16.</td> <td><a class="issue-link js-issue-link" data-error-text="Failed to load title" data-id="5989713643" data-permission-text="Title is private" data-url="https://github.com/vllm-project/vllm/issues/63231" data-hovercard-type="pull_request" data-hovercard-url="/vllm-project/vllm/pull/63231/hovercard" href="https://github.com/vllm-project/vllm/pull/63231">#63231</a></td> </tr> <tr> <td>Extensible KV cache on by default; <code>gpu_memory_utilization</code> defaults to 1.0</td> <td><strong>In most cases it's now best to not set <code>gpu_memory_utilization</code></strong>. This should provide maximum kv cache capacity while avoiding runtime OOMs. Use it to share a GPU with other processes or when extensible kv cache is disabled.</td> <td><a class="issue-link js-issue-link" data-error-text="Failed to load title" data-id="5427044802" data-permission-text="Title is private" data-url="https://github.com/vllm-project/vllm/issues/56492" data-hovercard-type="pull_request" data-hovercard-url="/vllm-project/vllm/pull/56492/hovercard" href="https://github.com/vllm-project/vllm/pull/56492">#56492</a></td> </tr> <tr> <td>ROCm 7.2 artifacts dropped</td> <td>Use the ROCm 10.0 wheels and images. <code>-rocm72</code> is no longer published.</td> <td><a class="issue-link js-issue-link" data-error-text="Failed to load title" data-id="5877058226" data-permission-text="Title is private" data-url="https://github.com/vllm-project/vllm/issues/61939" data-hovercard-type="pull_request" data-hovercard-url="/vllm-project/vllm/pull/61939/hovercard" href="https://github.com/vllm-project/vllm/pull/61939">#61939</a></td> </tr> <tr> <td><code>huggingface_hub >= 2.0.0</code> required</td> <td>Also requires <code>tokenizers >= 0.23.3</code> and <code>timm >= 1.0.30</code>.</td> <td><a class="issue-link js-issue-link" data-error-text="Failed to load title" data-id="6123285409" data-permission-text="Title is private" data-url="https://github.com/vllm-project/vllm/issues/64779" data-hovercard-type="pull_request" data-hovercard-url="/vllm-project/vllm/pull/64779/hovercard" href="https://github.com/vllm-project/vllm/pull/64779">#64779</a></td> </tr> <tr> <td>QuTLASS / <code>fp_quant</code> removed</td> <td><code>fp_quant</code> checkpoints no longer load.</td> <td><a class="issue-link js-issue-link" data-error-text="Failed to load title" data-id="6170612057" data-permission-text="Title is private" data-url="https://github.com/vllm-project/vllm/issues/65230" data-hovercard-type="pull_request" data-hovercard-url="/vllm-project/vllm/pull/65230/hovercard" href="https://github.com/vllm-project/vllm/pull/65230">#65230</a></td> </tr> <tr> <td>Marlin W4A8 removed</td> <td>Use <code>--quantization humming</code> with <code>VLLM_HUMMING_INPUT_QUANT_CONFIG</code> instead of <code>VLLM_MARLIN_INPUT_DTYPE</code>.</td> <td><a class="issue-link js-issue-link" data-error-text="Failed to load title" data-id="6194757596" data-permission-text="Title is private" data-url="https://github.com/vllm-project/vllm/issues/65542" data-hovercard-type="pull_request" data-hovercard-url="/vllm-project/vllm/pull/65542/hovercard" href="https://github.com/vllm-project/vllm/pull/65542">#65542</a></td> </tr> <tr> <td>Keye-VL and Keye-VL-1.5 removed</td> <td>—</td> <td><a class="issue-link js-issue-link" data-error-text="Failed to load title" data-id="6080460497" data-permission-text="Title
- v0.34.0 vllm-project/vllm releases · 19 thg 11, 2026
- vLLM v0.34.0 is live! 🚀 X · 19 thg 11, 2026