AllyshipContent Radar

Deep dives 738

Paper và bài viết kỹ thuật

Crawl live-1795068900000 · 13:22 19 thg 11, 2026 · Xuất bản 13:30 19 thg 11, 2026

Kỹ thuật

langgraph==1.2.23

<p>Changes since 1.2.22</p> <ul> <li>release(langgraph): 1.2.23 (<a class="issue-link js-issue-link" data-error-text="Failed to load title" data-id="6482730562" data-permission-text="Title is private" data-url="https://github.com/langchain-ai/langgraph/issues/9742" data-hovercard-type="pull_request" data-hovercard-url="/langchain-ai/langgraph/pull/9742/hovercard" href="https://github.com/langchain-ai/langgraph/pull/9742">#9742</a>)</li> <li>release(prebuilt): 1.1.3 (<a class="issue-link js-issue-link" data-error-text="Failed to load title" data-id="6482729770" data-permission-text="Title is private" data-url="https://github.com/langchain-ai/langgraph/issues/9741" data-hovercard-type="pull_request" data-hovercard-url="/langchain-ai/langgraph/pull/9741/hovercard" href="https://github.com/langchain-ai/langgraph/pull/9741">#9741</a>)</li> <li>fix(langgraph,prebuilt): give Send-dispatched tool calls the same state as an edge (<a class="issue-link js-issue-link" data-error-text="Failed to load title" data-id="6472223558" data-permission-text="Title is private" data-url="https://github.com/langchain-ai/langgraph/issues/9728" data-hovercard-type="pull_request" data-hovercard-url="/langchain-ai/langgraph/pull/9728/hovercard" href="https://github.com/langchain-ai/langgraph/pull/9728">#9728</a>)</li> <li>chore(deps): bump opentelemetry-api from 1.45.1 to 1.45.2 in /libs/langgraph (<a class="issue-link js-issue-link" data-error-text="Failed to load title" data-id="6377648315" data-permission-text="Title is private" data-url="https://github.com/langchain-ai/langgraph/issues/9696" data-hovercard-type="pull_request" data-hovercard-url="/langchain-ai/langgraph/pull/9696/hovercard" href="https://github.com/langchain-ai/langgraph/pull/9696">#9696</a>)</li> <li>fix(langgraph): avoid warning races in dynamic schemas (<a class="issue-link js-issue-link" data-error-text="Failed to load title" data-id="6408559559" data-permission-text="Title is private" data-url="https://github.com/langchain-ai/langgraph/issues/9716" data-hovercard-type="pull_request" data-hovercard-url="/langchain-ai/langgraph/pull/9716/hovercard" href="https://github.com/langchain-ai/langgraph/pull/9716">#9716</a>)</li> </ul>

Nguyên văn nguồn
Chi tiết nguồn
Kỹ thuật

langgraph-prebuilt==1.1.3

<p>Changes since prebuilt==1.1.2</p> <ul> <li>release(prebuilt): 1.1.3 (<a class="issue-link js-issue-link" data-error-text="Failed to load title" data-id="6482729770" data-permission-text="Title is private" data-url="https://github.com/langchain-ai/langgraph/issues/9741" data-hovercard-type="pull_request" data-hovercard-url="/langchain-ai/langgraph/pull/9741/hovercard" href="https://github.com/langchain-ai/langgraph/pull/9741">#9741</a>)</li> <li>fix(langgraph,prebuilt): give Send-dispatched tool calls the same state as an edge (<a class="issue-link js-issue-link" data-error-text="Failed to load title" data-id="6472223558" data-permission-text="Title is private" data-url="https://github.com/langchain-ai/langgraph/issues/9728" data-hovercard-type="pull_request" data-hovercard-url="/langchain-ai/langgraph/pull/9728/hovercard" href="https://github.com/langchain-ai/langgraph/pull/9728">#9728</a>)</li> <li>fix(prebuilt): make wrap_tool_call state overrides reach the tool (<a class="issue-link js-issue-link" data-error-text="Failed to load title" data-id="6473031915" data-permission-text="Title is private" data-url="https://github.com/langchain-ai/langgraph/issues/9730" data-hovercard-type="pull_request" data-hovercard-url="/langchain-ai/langgraph/pull/9730/hovercard" href="https://github.com/langchain-ai/langgraph/pull/9730">#9730</a>)</li> </ul>

Nguyên văn nguồn
Chi tiết nguồn
Kỹ thuật

b12532

<details open=""> <p>opencl: allow adjusting KV padding in decomposed FA (<a class="issue-link js-issue-link" data-error-text="Failed to load title" data-id="6462356977" data-permission-text="Title is private" data-url="https://github.com/ggml-org/llama.cpp/issues/32522" data-hovercard-type="pull_request" data-hovercard-url="/ggml-org/llama.cpp/pull/32522/hovercard" href="https://github.com/ggml-org/llama.cpp/pull/32522">#32522</a>)</p> <p>Assisted-by: Codex</p> </details> <p><strong>Website:</strong></p> <ul> <li><a href="https://llama.app" rel="nofollow">https://llama.app</a></li> </ul> <p><strong>Attestations:</strong></p> <ul> <li><a href="https://github.com/ggml-org/llama.cpp/attestations/69491345">https://github.com/ggml-org/llama.cpp/attestations/69491345</a></li> </ul> <p><strong>macOS/iOS:</strong></p> <ul> <li><a href="https://github.com/ggml-org/llama.cpp/releases/download/b12532/llama-b12532-bin-macos-arm64.tar.gz">macOS Apple Silicon (arm64)</a></li> <li>macOS Apple Silicon (arm64, KleidiAI enabled) <a href="https://github.com/ggml-org/llama.cpp/pull/23780" data-hovercard-type="pull_request" data-hovercard-url="/ggml-org/llama.cpp/pull/23780/hovercard">DISABLED</a></li> <li><a href="https://github.com/ggml-org/llama.cpp/releases/download/b12532/llama-b12532-bin-macos-x64.tar.gz">macOS Intel (x64)</a></li> <li><a href="https://github.com/ggml-org/llama.cpp/releases/download/b12532/llama-b12532-xcframework.zip">iOS XCFramework</a></li> </ul> <p><strong>Linux:</strong></p> <ul> <li><a href="https://github.com/ggml-org/llama.cpp/releases/download/b12532/llama-b12532-bin-ubuntu-x64.tar.gz">Ubuntu x64 (CPU)</a></li> <li><a href="https://github.com/ggml-org/llama.cpp/releases/download/b12532/llama-b12532-bin-ubuntu-arm64.tar.gz">Ubuntu arm64 (CPU)</a></li> <li><a href="https://github.com/ggml-org/llama.cpp/releases/download/b12532/llama-b12532-bin-ubuntu-s390x.tar.gz">Ubuntu 24.04 s390x (CPU)</a></li> <li><a href="https://github.com/ggml-org/llama.cpp/releases/download/b12532/llama-b12532-bin-ubuntu-26.04-s390x.tar.gz">Ubuntu 26.04 s390x (CPU)</a></li> <li><a href="https://github.com/ggml-org/llama.cpp/releases/download/b12532/llama-b12532-bin-ubuntu-zdnn-s390x.tar.gz">Ubuntu 26.04 s390x (zDNN)</a></li> <li><a href="https://github.com/ggml-org/llama.cpp/releases/download/b12532/llama-b12532-bin-ubuntu-vulkan-x64.tar.gz">Ubuntu x64 (Vulkan)</a></li> <li><a href="https://github.com/ggml-org/llama.cpp/releases/download/b12532/llama-b12532-bin-ubuntu-vulkan-arm64.tar.gz">Ubuntu arm64 (Vulkan)</a></li> <li><a href="https://github.com/ggml-org/llama.cpp/releases/download/b12532/llama-b12532-bin-ubuntu-cuda-12.8-x64.tar.gz">Ubuntu x64 (CUDA 12)</a> - <a href="https://github.com/ggml-org/llama.cpp/releases/download/b12532/cudart-llama-b12532-bin-ubuntu-cuda-12.8-x64.tar.gz">CUDA 12.8 libraries</a></li> <li><a href="https://github.com/ggml-org/llama.cpp/releases/download/b12532/llama-b12532-bin-ubuntu-cuda-13.5-x64.tar.gz">Ubuntu x64 (CUDA 13)</a> - <a href="https://github.com/ggml-org/llama.cpp/releases/download/b12532/cudart-llama-b12532-bin-ubuntu-cuda-13.5-x64.tar.gz">CUDA 13.5 libraries</a></li> <li><a href="https://github.com/ggml-org/llama.cpp/releases/download/b12532/llama-b12532-bin-ubuntu-cuda-13.5-arm64.tar.gz">Ubuntu arm64 (CUDA 13)</a> - <a href="https://github.com/ggml-org/llama.cpp/releases/download/b12532/cudart-llama-b12532-bin-ubuntu-cuda-13.5-arm64.tar.gz">CUDA 13.5 libraries</a></li> <li><a href="https://github.com/ggml-org/llama.cpp/releases/download/b12532/llama-b12532-bin-ubuntu-rocm-10.0-x64.tar.gz">Ubuntu x64 (ROCm 10.0)</a></li> <li><a href="https://github.com/ggml-org/llama.cpp/releases/download/b12532/llama-b12532-bin-ubuntu-openvino-2026.4.1-x64.tar.gz">Ubuntu x64 (OpenVINO)</a></li> <li><a href="https://github.com/ggml-org/llama.cpp/releases/download/b12532/llama-b12532-bin-ubuntu-sycl-fp32-x64.tar.gz">Ubuntu x64 (SYCL FP32)</a></li> <li><a href="https://github.com/ggml-org/llama.cpp/releases/download/b12532/llama-b12532-bin-ubuntu-sycl-fp16-x64.tar.gz">Ubuntu x64 (SYCL FP16)</a></li> <li><a href="https://github.com/ggml-org/llama.cpp/releases/download/b12532/llama-b12532-bin-linux-arm64-snapdragon.tar.gz">Linux arm64 (Snapdragon: CPU, Adreno GPU, Hexagon NPU)</a> - <a href="https://github.com/ggml-org/llama.cpp/blob/master/docs/backend/snapdragon/linux.md">setup guide</a></li> </ul> <p><strong>Android:</strong></p> <ul> <li><a href="https://github.com/ggml-org/llama.cpp/releases/download/b12532/llama-b12532-bin-android-arm64.tar.gz">Android arm64 (CPU)</a></li> <li><a href="https://github.com/ggml-org/llama.cpp/releases/download/b12532/llama-b12532-bin-android-arm64-snapdragon.tar.gz">Android arm64 (Snapdragon: CPU, Adreno GPU, Hexagon NPU)</a> - <a href="https://github.com/ggml-org/llama.cpp/blob/master/docs/backend/snapdragon/README.md">setup guide</a></li> </ul> <p><strong>Windows:</strong></p> <ul> <li><a href="https://github.com/ggml-org/llama.cpp/releases/download/b12532/llama-b12532-bin-win-cpu-x64.zip">Windows x64 (CPU)</a></li> <li><a href="https://github.com/ggml-org/llama.cpp/releases/download/b12532/llama-b12532-bin-win-cpu-arm64.zip">Windows arm64 (CPU)</a></li> <li><a href="https://github.com/ggml-org/llama.cpp/releases/download/b12532/llama-b12532-bin-win-opencl-adreno-arm64.zip">Windows arm64 (OpenCL Adreno)</a></li> <li><a href="https://github.com/ggml-org/llama.cpp/releases/download/b12532/llama-b12532-bin-win-cuda-12.4-x64.zip">Windows x64 (CUDA 12)</a> - <a href="https://github.com/ggml-org/llama.cpp/releases/download/b12532/cudart-llama-bin-win-cuda-12.4-x64.zip">CUDA 12.4 DLLs</a></li> <li><a href="https://github.com/ggml-org/llama.cpp/releases/download/b12532/llama-b12532-bin-win-cuda-13.5-x64.zip">Windows x64 (CUDA 13)</a> - <a href="https://github.com/ggml-org/llama.cpp/releases/download/b12532/cudart-llama-bin-win-cuda-13.5-x64.zip">CUDA 13.5 DLLs</a></li> <li><a href="https://github.com/ggml-org/llama.cpp/releases/download/b12532/lla

Nguyên văn nguồn
Chi tiết nguồn
  • b12532 ggml-org/llama.cpp releases · 19 thg 11, 2026
Đánh giá

Building Reasoning Models from the Ground Up: A Full Video Course

I have written a lot about reasoning models here over the last two years on Ahead of AI, including a recent article on mechanisms to control reasoning effort levels in contemporary LLMs . Reasoning models are still as relevant as ever. In fact, almost all current-generation LLMs today are reasoning models or offer a reasoning mode. For those who are interested in how reasoning models work, including inference scaling, reinforcement learning, and distillation, I wrote my recent Build Reasoning Model (From Scratch), which implements these from the ground up in PyTorch, which, in my experience, is still the best way to understand and learn how things really work. Now, in the last few months, I also sat down and recorded accompanying lectures I wanted to share with you. So, below, you will find eight reasoning model-from-scratch coding videos (11 hours of content). We start with a pre-trained LLM and implement methods to evaluate and improve it in Python and PyTorch. So, if you enjoyed my Coding LLMs from the Ground Up course last year, this is essentially the sequel (but you can also watch it without the prior course as a prerequisite). I’m currently taking a short break and will be back soon with more exciting LLM research roundups next month. In the meantime, I hope you enjoy coding along! How LLMs, reasoning models, and agents relate and fit together As a small side note, and since I often get asked how reasoning models relate to today’s AI agents, here’s a short section on this before getting started with the videos. A reasoning model is essentially an upgrade from a conventional LLM, that is, adding more post-training via reinforcement learning with verifiable rewards (and optionally inference scaling techniques) to make it more capable of solving more challenging problems. Classically, reasoning models are developed for math and code, but the capabilities can generalize to other problem domains as well. A reasoning model then functions as the engine inside an agent harness, where the agent harness is a software or tooling framework around the LLM/reasoning model to tackle even more challenging problems (bigger coding tasks, travel booking, and all other sorts of personal assistant tasks). The task of the harness is sophisticated context management and giving the LLM access to tools. Figure 1: An engine analogy for conventional LLMs, reasoning models, and agent harnesses. Figure from my Components of a Coding Agent article. In this analogy depicted in the figure above, the model is the engine, and the agent harness adds the machinery that lets us use it for various things (as an upgrade from a simple chatbot interface). An agent can use either a conventional LLM or a reasoning model, but reasoning models are the more typical choice as they are more capable. (As mentioned above, most current-gen LLMs are reasoning models with variable reasoning effort settings.) So, the bottom line is that reasoning models, as the “engine,” remain an important part of understanding modern LLM agents. The resources accompanying these topics shown above are: The Build a Large Language Model (From Scratch) book and videos The Build A Reasoning Model (From Scratch) book (and videos below) The Components of a Coding Agent article on Substack The videos below focus on developing the reasoning model that can power such a system. Scope and prerequisites The reasoning-from-scratch videos focus on a small pre-trained Qwen3 model, where we implement the reasoning methods ourselves in Python and PyTorch. Starting with a pre-trained model lets us focus on the interesting part: evaluation, inference-time scaling, reinforcement learning, and distillation. Regarding the prerequisites, I recommend being comfortable with Python and PyTorch. I designed the reasoning materials as standalone content, so the previous Build A Large Language Model (From Scratch) content is recommended but not strictly required. Compute-resource-wise, you can work through the early text-generation and evaluation examples on a CPU. For the later experiments, especially training, a GPU makes the work much more practical. The first video covers the setup and some more recommendations; the repository also includes hardware and environment recommendations . Figure 2: An overview of the topics being covered. The companion book These videos accompany my book, Build a Reasoning Model (From Scratch) . If you like to read through the explanation in more detail and at your own pace, work through exercises, and so on, I recommend the book as a companion here. For the most in-depth learning experience, my suggestion is to read a chapter, work through its code alongside the corresponding video, and then try the exercises. The book is available from Manning and Amazon , and the print edition is in color! Figure 3: A random excerpt from the Build a Reasoning Model (From Scratch) print book. Thank you to everyone who has picked up a copy or recommended it to a friend. Your support helps me keep creating these materials. The videos As a last, quick side note before getting to the main videos below: For a shorter, more conceptual overview, I also recommend my AI Reasoning Models course on LinkedIn Learning (it should be free for 24h when accessing it via the link there, which shouldn’t then require a LinkedIn Premium subscription). The course is about 90 minutes and explains how reasoning models work, covering the main ideas behind inference-time scaling, reinforcement learning, and distillation. It’s a good optional preview if you’d like the main ideas without working through the full implementations below. To get to the main part of this article, here are the eight main lecture videos. 1. Motivation and code setup The first video starts with an explanation of the relationship between conventional LLMs, reasoning models, and agents. Next, the main topic of the video is setting up the coding environment. This includes installing the Python and PyTorch dependencie

  • We start with a pre-trained LLM and implement methods to evaluate and improve it in Python and PyTorch.
Nguyên văn nguồn
Chi tiết nguồn
Kỹ thuật

b12531

<details open=""> <p>mtmd: support voxcpm2 (<a class="issue-link js-issue-link" data-error-text="Failed to load title" data-id="6434264762" data-permission-text="Title is private" data-url="https://github.com/ggml-org/llama.cpp/issues/32443" data-hovercard-type="pull_request" data-hovercard-url="/ggml-org/llama.cpp/pull/32443/hovercard" href="https://github.com/ggml-org/llama.cpp/pull/32443">#32443</a>)</p> <ul> <li> <p>mtmd: support voxcpm</p> </li> <li> <p>speed up a bit</p> </li> <li> <p>various fixes</p> </li> </ul> </details> <p><strong>Website:</strong></p> <ul> <li><a href="https://llama.app" rel="nofollow">https://llama.app</a></li> </ul> <p><strong>Attestations:</strong></p> <ul> <li><a href="https://github.com/ggml-org/llama.cpp/attestations/69455423">https://github.com/ggml-org/llama.cpp/attestations/69455423</a></li> </ul> <p><strong>macOS/iOS:</strong></p> <ul> <li><a href="https://github.com/ggml-org/llama.cpp/releases/download/b12531/llama-b12531-bin-macos-arm64.tar.gz">macOS Apple Silicon (arm64)</a></li> <li>macOS Apple Silicon (arm64, KleidiAI enabled) <a href="https://github.com/ggml-org/llama.cpp/pull/23780" data-hovercard-type="pull_request" data-hovercard-url="/ggml-org/llama.cpp/pull/23780/hovercard">DISABLED</a></li> <li><a href="https://github.com/ggml-org/llama.cpp/releases/download/b12531/llama-b12531-bin-macos-x64.tar.gz">macOS Intel (x64)</a></li> <li><a href="https://github.com/ggml-org/llama.cpp/releases/download/b12531/llama-b12531-xcframework.zip">iOS XCFramework</a></li> </ul> <p><strong>Linux:</strong></p> <ul> <li><a href="https://github.com/ggml-org/llama.cpp/releases/download/b12531/llama-b12531-bin-ubuntu-x64.tar.gz">Ubuntu x64 (CPU)</a></li> <li><a href="https://github.com/ggml-org/llama.cpp/releases/download/b12531/llama-b12531-bin-ubuntu-arm64.tar.gz">Ubuntu arm64 (CPU)</a></li> <li><a href="https://github.com/ggml-org/llama.cpp/releases/download/b12531/llama-b12531-bin-ubuntu-s390x.tar.gz">Ubuntu 24.04 s390x (CPU)</a></li> <li><a href="https://github.com/ggml-org/llama.cpp/releases/download/b12531/llama-b12531-bin-ubuntu-26.04-s390x.tar.gz">Ubuntu 26.04 s390x (CPU)</a></li> <li><a href="https://github.com/ggml-org/llama.cpp/releases/download/b12531/llama-b12531-bin-ubuntu-zdnn-s390x.tar.gz">Ubuntu 26.04 s390x (zDNN)</a></li> <li><a href="https://github.com/ggml-org/llama.cpp/releases/download/b12531/llama-b12531-bin-ubuntu-vulkan-x64.tar.gz">Ubuntu x64 (Vulkan)</a></li> <li><a href="https://github.com/ggml-org/llama.cpp/releases/download/b12531/llama-b12531-bin-ubuntu-vulkan-arm64.tar.gz">Ubuntu arm64 (Vulkan)</a></li> <li><a href="https://github.com/ggml-org/llama.cpp/releases/download/b12531/llama-b12531-bin-ubuntu-cuda-12.8-x64.tar.gz">Ubuntu x64 (CUDA 12)</a> - <a href="https://github.com/ggml-org/llama.cpp/releases/download/b12531/cudart-llama-b12531-bin-ubuntu-cuda-12.8-x64.tar.gz">CUDA 12.8 libraries</a></li> <li><a href="https://github.com/ggml-org/llama.cpp/releases/download/b12531/llama-b12531-bin-ubuntu-cuda-13.5-x64.tar.gz">Ubuntu x64 (CUDA 13)</a> - <a href="https://github.com/ggml-org/llama.cpp/releases/download/b12531/cudart-llama-b12531-bin-ubuntu-cuda-13.5-x64.tar.gz">CUDA 13.5 libraries</a></li> <li><a href="https://github.com/ggml-org/llama.cpp/releases/download/b12531/llama-b12531-bin-ubuntu-cuda-13.5-arm64.tar.gz">Ubuntu arm64 (CUDA 13)</a> - <a href="https://github.com/ggml-org/llama.cpp/releases/download/b12531/cudart-llama-b12531-bin-ubuntu-cuda-13.5-arm64.tar.gz">CUDA 13.5 libraries</a></li> <li><a href="https://github.com/ggml-org/llama.cpp/releases/download/b12531/llama-b12531-bin-ubuntu-rocm-10.0-x64.tar.gz">Ubuntu x64 (ROCm 10.0)</a></li> <li><a href="https://github.com/ggml-org/llama.cpp/releases/download/b12531/llama-b12531-bin-ubuntu-openvino-2026.4.1-x64.tar.gz">Ubuntu x64 (OpenVINO)</a></li> <li><a href="https://github.com/ggml-org/llama.cpp/releases/download/b12531/llama-b12531-bin-ubuntu-sycl-fp32-x64.tar.gz">Ubuntu x64 (SYCL FP32)</a></li> <li><a href="https://github.com/ggml-org/llama.cpp/releases/download/b12531/llama-b12531-bin-ubuntu-sycl-fp16-x64.tar.gz">Ubuntu x64 (SYCL FP16)</a></li> <li><a href="https://github.com/ggml-org/llama.cpp/releases/download/b12531/llama-b12531-bin-linux-arm64-snapdragon.tar.gz">Linux arm64 (Snapdragon: CPU, Adreno GPU, Hexagon NPU)</a> - <a href="https://github.com/ggml-org/llama.cpp/blob/master/docs/backend/snapdragon/linux.md">setup guide</a></li> </ul> <p><strong>Android:</strong></p> <ul> <li><a href="https://github.com/ggml-org/llama.cpp/releases/download/b12531/llama-b12531-bin-android-arm64.tar.gz">Android arm64 (CPU)</a></li> <li><a href="https://github.com/ggml-org/llama.cpp/releases/download/b12531/llama-b12531-bin-android-arm64-snapdragon.tar.gz">Android arm64 (Snapdragon: CPU, Adreno GPU, Hexagon NPU)</a> - <a href="https://github.com/ggml-org/llama.cpp/blob/master/docs/backend/snapdragon/README.md">setup guide</a></li> </ul> <p><strong>Windows:</strong></p> <ul> <li><a href="https://github.com/ggml-org/llama.cpp/releases/download/b12531/llama-b12531-bin-win-cpu-x64.zip">Windows x64 (CPU)</a></li> <li><a href="https://github.com/ggml-org/llama.cpp/releases/download/b12531/llama-b12531-bin-win-cpu-arm64.zip">Windows arm64 (CPU)</a></li> <li><a href="https://github.com/ggml-org/llama.cpp/releases/download/b12531/llama-b12531-bin-win-opencl-adreno-arm64.zip">Windows arm64 (OpenCL Adreno)</a></li> <li><a href="https://github.com/ggml-org/llama.cpp/releases/download/b12531/llama-b12531-bin-win-cuda-12.4-x64.zip">Windows x64 (CUDA 12)</a> - <a href="https://github.com/ggml-org/llama.cpp/releases/download/b12531/cudart-llama-bin-win-cuda-12.4-x64.zip">CUDA 12.4 DLLs</a></li> <li><a href="https://github.com/ggml-org/llama.cpp/releases/download/b12531/llama-b12531-bin-win-cuda-13.5-x64.zip">Windows x64 (CUDA 13)</a> - <a href="https://github.com/ggml-org/llama.cpp/releases/download/b12531/cudart-llama-bin-win-cuda-13.5-x64.zip">CUDA 13.5 DLLs</a></li> <li><a href="https:/

Nguyên văn nguồn
Chi tiết nguồn
  • b12531 ggml-org/llama.cpp releases · 19 thg 11, 2026
Kỹ thuật

b12530

<details open=""> <p>ggml : WASM Relaxed SIMD Enhancement (<a class="issue-link js-issue-link" data-error-text="Failed to load title" data-id="3937119279" data-permission-text="Title is private" data-url="https://github.com/ggml-org/llama.cpp/issues/19590" data-hovercard-type="pull_request" data-hovercard-url="/ggml-org/llama.cpp/pull/19590/hovercard" href="https://github.com/ggml-org/llama.cpp/pull/19590">#19590</a>)</p> <ul> <li> <p>add relaxed simd</p> </li> <li> <p>fix indentation</p> </li> <li> <p>remove comments and whitespace</p> </li> <li> <p>wasm: fix relaxed SIMD kernels and add q3_K/q6_K support</p> </li> <li> <p>rm trailing space</p> </li> </ul> <hr> <p>Co-authored-by: Xuan Son Nguyen <a href="mailto:son@huggingface.co">son@huggingface.co</a></p> </details> <p><strong>Website:</strong></p> <ul> <li><a href="https://llama.app" rel="nofollow">https://llama.app</a></li> </ul> <p><strong>Attestations:</strong></p> <ul> <li><a href="https://github.com/ggml-org/llama.cpp/attestations/69435594">https://github.com/ggml-org/llama.cpp/attestations/69435594</a></li> </ul> <p><strong>macOS/iOS:</strong></p> <ul> <li><a href="https://github.com/ggml-org/llama.cpp/releases/download/b12530/llama-b12530-bin-macos-arm64.tar.gz">macOS Apple Silicon (arm64)</a></li> <li>macOS Apple Silicon (arm64, KleidiAI enabled) <a href="https://github.com/ggml-org/llama.cpp/pull/23780" data-hovercard-type="pull_request" data-hovercard-url="/ggml-org/llama.cpp/pull/23780/hovercard">DISABLED</a></li> <li><a href="https://github.com/ggml-org/llama.cpp/releases/download/b12530/llama-b12530-bin-macos-x64.tar.gz">macOS Intel (x64)</a></li> <li><a href="https://github.com/ggml-org/llama.cpp/releases/download/b12530/llama-b12530-xcframework.zip">iOS XCFramework</a></li> </ul> <p><strong>Linux:</strong></p> <ul> <li><a href="https://github.com/ggml-org/llama.cpp/releases/download/b12530/llama-b12530-bin-ubuntu-x64.tar.gz">Ubuntu x64 (CPU)</a></li> <li><a href="https://github.com/ggml-org/llama.cpp/releases/download/b12530/llama-b12530-bin-ubuntu-arm64.tar.gz">Ubuntu arm64 (CPU)</a></li> <li><a href="https://github.com/ggml-org/llama.cpp/releases/download/b12530/llama-b12530-bin-ubuntu-s390x.tar.gz">Ubuntu 24.04 s390x (CPU)</a></li> <li><a href="https://github.com/ggml-org/llama.cpp/releases/download/b12530/llama-b12530-bin-ubuntu-26.04-s390x.tar.gz">Ubuntu 26.04 s390x (CPU)</a></li> <li><a href="https://github.com/ggml-org/llama.cpp/releases/download/b12530/llama-b12530-bin-ubuntu-zdnn-s390x.tar.gz">Ubuntu 26.04 s390x (zDNN)</a></li> <li><a href="https://github.com/ggml-org/llama.cpp/releases/download/b12530/llama-b12530-bin-ubuntu-vulkan-x64.tar.gz">Ubuntu x64 (Vulkan)</a></li> <li><a href="https://github.com/ggml-org/llama.cpp/releases/download/b12530/llama-b12530-bin-ubuntu-vulkan-arm64.tar.gz">Ubuntu arm64 (Vulkan)</a></li> <li><a href="https://github.com/ggml-org/llama.cpp/releases/download/b12530/llama-b12530-bin-ubuntu-cuda-12.8-x64.tar.gz">Ubuntu x64 (CUDA 12)</a> - <a href="https://github.com/ggml-org/llama.cpp/releases/download/b12530/cudart-llama-b12530-bin-ubuntu-cuda-12.8-x64.tar.gz">CUDA 12.8 libraries</a></li> <li><a href="https://github.com/ggml-org/llama.cpp/releases/download/b12530/llama-b12530-bin-ubuntu-cuda-13.5-x64.tar.gz">Ubuntu x64 (CUDA 13)</a> - <a href="https://github.com/ggml-org/llama.cpp/releases/download/b12530/cudart-llama-b12530-bin-ubuntu-cuda-13.5-x64.tar.gz">CUDA 13.5 libraries</a></li> <li><a href="https://github.com/ggml-org/llama.cpp/releases/download/b12530/llama-b12530-bin-ubuntu-cuda-13.5-arm64.tar.gz">Ubuntu arm64 (CUDA 13)</a> - <a href="https://github.com/ggml-org/llama.cpp/releases/download/b12530/cudart-llama-b12530-bin-ubuntu-cuda-13.5-arm64.tar.gz">CUDA 13.5 libraries</a></li> <li><a href="https://github.com/ggml-org/llama.cpp/releases/download/b12530/llama-b12530-bin-ubuntu-rocm-10.0-x64.tar.gz">Ubuntu x64 (ROCm 10.0)</a></li> <li><a href="https://github.com/ggml-org/llama.cpp/releases/download/b12530/llama-b12530-bin-ubuntu-openvino-2026.4.1-x64.tar.gz">Ubuntu x64 (OpenVINO)</a></li> <li><a href="https://github.com/ggml-org/llama.cpp/releases/download/b12530/llama-b12530-bin-ubuntu-sycl-fp32-x64.tar.gz">Ubuntu x64 (SYCL FP32)</a></li> <li><a href="https://github.com/ggml-org/llama.cpp/releases/download/b12530/llama-b12530-bin-ubuntu-sycl-fp16-x64.tar.gz">Ubuntu x64 (SYCL FP16)</a></li> <li><a href="https://github.com/ggml-org/llama.cpp/releases/download/b12530/llama-b12530-bin-linux-arm64-snapdragon.tar.gz">Linux arm64 (Snapdragon: CPU, Adreno GPU, Hexagon NPU)</a> - <a href="https://github.com/ggml-org/llama.cpp/blob/master/docs/backend/snapdragon/linux.md">setup guide</a></li> </ul> <p><strong>Android:</strong></p> <ul> <li><a href="https://github.com/ggml-org/llama.cpp/releases/download/b12530/llama-b12530-bin-android-arm64.tar.gz">Android arm64 (CPU)</a></li> <li><a href="https://github.com/ggml-org/llama.cpp/releases/download/b12530/llama-b12530-bin-android-arm64-snapdragon.tar.gz">Android arm64 (Snapdragon: CPU, Adreno GPU, Hexagon NPU)</a> - <a href="https://github.com/ggml-org/llama.cpp/blob/master/docs/backend/snapdragon/README.md">setup guide</a></li> </ul> <p><strong>Windows:</strong></p> <ul> <li><a href="https://github.com/ggml-org/llama.cpp/releases/download/b12530/llama-b12530-bin-win-cpu-x64.zip">Windows x64 (CPU)</a></li> <li><a href="https://github.com/ggml-org/llama.cpp/releases/download/b12530/llama-b12530-bin-win-cpu-arm64.zip">Windows arm64 (CPU)</a></li> <li><a href="https://github.com/ggml-org/llama.cpp/releases/download/b12530/llama-b12530-bin-win-opencl-adreno-arm64.zip">Windows arm64 (OpenCL Adreno)</a></li> <li><a href="https://github.com/ggml-org/llama.cpp/releases/download/b12530/llama-b12530-bin-win-cuda-12.4-x64.zip">Windows x64 (CUDA 12)</a> - <a href="https://github.com/ggml-org/llama.cpp/releases/download/b12530/cudart-llama-bin-win-cuda-12.4-x64.zip">CUDA 12.4 DLLs</a></li> <li><a href="https://github.com/ggml-org/llama.cpp/releas

Nguyên văn nguồn
Chi tiết nguồn
  • b12530 ggml-org/llama.cpp releases · 19 thg 11, 2026
Kỹ thuật

langchain-openai==1.8.3

<p>Changes since langchain-openai==1.8.2</p> <p>release(openai): 1.8.3 (<a class="issue-link js-issue-link" data-error-text="Failed to load title" data-id="6479479320" data-permission-text="Title is private" data-url="https://github.com/langchain-ai/langchain/issues/41950" data-hovercard-type="pull_request" data-hovercard-url="/langchain-ai/langchain/pull/41950/hovercard" href="https://github.com/langchain-ai/langchain/pull/41950">#41950</a>)<br> fix(openai): enable Codex prompt caching on GPT-6.2 (<a class="issue-link js-issue-link" data-error-text="Failed to load title" data-id="6478925585" data-permission-text="Title is private" data-url="https://github.com/langchain-ai/langchain/issues/41949" data-hovercard-type="pull_request" data-hovercard-url="/langchain-ai/langchain/pull/41949/hovercard" href="https://github.com/langchain-ai/langchain/pull/41949">#41949</a>)</p>

Nguyên văn nguồn
Chi tiết nguồn
Kỹ thuật

b12529

<details open=""> <p>llama : fix sticky NaN in recurrent state (<a class="issue-link js-issue-link" data-error-text="Failed to load title" data-id="6276362295" data-permission-text="Title is private" data-url="https://github.com/ggml-org/llama.cpp/issues/31913" data-hovercard-type="pull_request" data-hovercard-url="/ggml-org/llama.cpp/pull/31913/hovercard" href="https://github.com/ggml-org/llama.cpp/pull/31913">#31913</a>)</p> <p>Co-authored-by: Alexey Kopytko <a href="mailto:alexey@kopytko.com">alexey@kopytko.com</a></p> </details> <p><strong>Website:</strong></p> <ul> <li><a href="https://llama.app" rel="nofollow">https://llama.app</a></li> </ul> <p><strong>Attestations:</strong></p> <ul> <li><a href="https://github.com/ggml-org/llama.cpp/attestations/69418514">https://github.com/ggml-org/llama.cpp/attestations/69418514</a></li> </ul> <p><strong>macOS/iOS:</strong></p> <ul> <li><a href="https://github.com/ggml-org/llama.cpp/releases/download/b12529/llama-b12529-bin-macos-arm64.tar.gz">macOS Apple Silicon (arm64)</a></li> <li>macOS Apple Silicon (arm64, KleidiAI enabled) <a href="https://github.com/ggml-org/llama.cpp/pull/23780" data-hovercard-type="pull_request" data-hovercard-url="/ggml-org/llama.cpp/pull/23780/hovercard">DISABLED</a></li> <li><a href="https://github.com/ggml-org/llama.cpp/releases/download/b12529/llama-b12529-bin-macos-x64.tar.gz">macOS Intel (x64)</a></li> <li><a href="https://github.com/ggml-org/llama.cpp/releases/download/b12529/llama-b12529-xcframework.zip">iOS XCFramework</a></li> </ul> <p><strong>Linux:</strong></p> <ul> <li><a href="https://github.com/ggml-org/llama.cpp/releases/download/b12529/llama-b12529-bin-ubuntu-x64.tar.gz">Ubuntu x64 (CPU)</a></li> <li><a href="https://github.com/ggml-org/llama.cpp/releases/download/b12529/llama-b12529-bin-ubuntu-arm64.tar.gz">Ubuntu arm64 (CPU)</a></li> <li><a href="https://github.com/ggml-org/llama.cpp/releases/download/b12529/llama-b12529-bin-ubuntu-s390x.tar.gz">Ubuntu 24.04 s390x (CPU)</a></li> <li><a href="https://github.com/ggml-org/llama.cpp/releases/download/b12529/llama-b12529-bin-ubuntu-26.04-s390x.tar.gz">Ubuntu 26.04 s390x (CPU)</a></li> <li><a href="https://github.com/ggml-org/llama.cpp/releases/download/b12529/llama-b12529-bin-ubuntu-zdnn-s390x.tar.gz">Ubuntu 26.04 s390x (zDNN)</a></li> <li><a href="https://github.com/ggml-org/llama.cpp/releases/download/b12529/llama-b12529-bin-ubuntu-vulkan-x64.tar.gz">Ubuntu x64 (Vulkan)</a></li> <li><a href="https://github.com/ggml-org/llama.cpp/releases/download/b12529/llama-b12529-bin-ubuntu-vulkan-arm64.tar.gz">Ubuntu arm64 (Vulkan)</a></li> <li><a href="https://github.com/ggml-org/llama.cpp/releases/download/b12529/llama-b12529-bin-ubuntu-cuda-12.8-x64.tar.gz">Ubuntu x64 (CUDA 12)</a> - <a href="https://github.com/ggml-org/llama.cpp/releases/download/b12529/cudart-llama-b12529-bin-ubuntu-cuda-12.8-x64.tar.gz">CUDA 12.8 libraries</a></li> <li><a href="https://github.com/ggml-org/llama.cpp/releases/download/b12529/llama-b12529-bin-ubuntu-cuda-13.5-x64.tar.gz">Ubuntu x64 (CUDA 13)</a> - <a href="https://github.com/ggml-org/llama.cpp/releases/download/b12529/cudart-llama-b12529-bin-ubuntu-cuda-13.5-x64.tar.gz">CUDA 13.5 libraries</a></li> <li><a href="https://github.com/ggml-org/llama.cpp/releases/download/b12529/llama-b12529-bin-ubuntu-cuda-13.5-arm64.tar.gz">Ubuntu arm64 (CUDA 13)</a> - <a href="https://github.com/ggml-org/llama.cpp/releases/download/b12529/cudart-llama-b12529-bin-ubuntu-cuda-13.5-arm64.tar.gz">CUDA 13.5 libraries</a></li> <li><a href="https://github.com/ggml-org/llama.cpp/releases/download/b12529/llama-b12529-bin-ubuntu-rocm-10.0-x64.tar.gz">Ubuntu x64 (ROCm 10.0)</a></li> <li><a href="https://github.com/ggml-org/llama.cpp/releases/download/b12529/llama-b12529-bin-ubuntu-openvino-2026.4.1-x64.tar.gz">Ubuntu x64 (OpenVINO)</a></li> <li><a href="https://github.com/ggml-org/llama.cpp/releases/download/b12529/llama-b12529-bin-ubuntu-sycl-fp32-x64.tar.gz">Ubuntu x64 (SYCL FP32)</a></li> <li><a href="https://github.com/ggml-org/llama.cpp/releases/download/b12529/llama-b12529-bin-ubuntu-sycl-fp16-x64.tar.gz">Ubuntu x64 (SYCL FP16)</a></li> <li><a href="https://github.com/ggml-org/llama.cpp/releases/download/b12529/llama-b12529-bin-linux-arm64-snapdragon.tar.gz">Linux arm64 (Snapdragon: CPU, Adreno GPU, Hexagon NPU)</a> - <a href="https://github.com/ggml-org/llama.cpp/blob/master/docs/backend/snapdragon/linux.md">setup guide</a></li> </ul> <p><strong>Android:</strong></p> <ul> <li><a href="https://github.com/ggml-org/llama.cpp/releases/download/b12529/llama-b12529-bin-android-arm64.tar.gz">Android arm64 (CPU)</a></li> <li><a href="https://github.com/ggml-org/llama.cpp/releases/download/b12529/llama-b12529-bin-android-arm64-snapdragon.tar.gz">Android arm64 (Snapdragon: CPU, Adreno GPU, Hexagon NPU)</a> - <a href="https://github.com/ggml-org/llama.cpp/blob/master/docs/backend/snapdragon/README.md">setup guide</a></li> </ul> <p><strong>Windows:</strong></p> <ul> <li><a href="https://github.com/ggml-org/llama.cpp/releases/download/b12529/llama-b12529-bin-win-cpu-x64.zip">Windows x64 (CPU)</a></li> <li><a href="https://github.com/ggml-org/llama.cpp/releases/download/b12529/llama-b12529-bin-win-cpu-arm64.zip">Windows arm64 (CPU)</a></li> <li><a href="https://github.com/ggml-org/llama.cpp/releases/download/b12529/llama-b12529-bin-win-opencl-adreno-arm64.zip">Windows arm64 (OpenCL Adreno)</a></li> <li><a href="https://github.com/ggml-org/llama.cpp/releases/download/b12529/llama-b12529-bin-win-cuda-12.4-x64.zip">Windows x64 (CUDA 12)</a> - <a href="https://github.com/ggml-org/llama.cpp/releases/download/b12529/cudart-llama-bin-win-cuda-12.4-x64.zip">CUDA 12.4 DLLs</a></li> <li><a href="https://github.com/ggml-org/llama.cpp/releases/download/b12529/llama-b12529-bin-win-cuda-13.5-x64.zip">Windows x64 (CUDA 13)</a> - <a href="https://github.com/ggml-org/llama.cpp/releases/download/b12529/cudart-llama-bin-win-cuda-13.5-x64.zip">CUDA 13.5 DLLs</a></li> <li><a href="https

Nguyên văn nguồn
Chi tiết nguồn
  • b12529 ggml-org/llama.cpp releases · 19 thg 11, 2026
Tác tử

v19.0.0

Câu hỏi </li> <li>Added btw mode: drafts beginning with ?

<h2>@oh-my-pi/pi-agent-core</h2> <h3>Added</h3> <ul> <li>Added support for excluding custom messages from model context and compaction or branch summaries, while allowing queued steering and follow-up messages to continue correctly after excluded messages.</li> </ul> <h2>@oh-my-pi/pi-ai</h2> <h3>Added</h3> <ul> <li>Added a <code>prewarm</code> stream option for Anthropic Messages requests to refresh prompt caches without generating billable output; it is rejected for budget thinking, forced tool choice, and structured output.</li> <li>Claude usage reports now include the account subscription plan in <code>metadata.subscriptionPlan</code>, supporting Max 5x, Max 20x, Pro, Team, and Enterprise plans.</li> <li>Added <code>retryUnreachableBroker</code> and <code>onBrokerRetry</code> options to <code>discoverAuthStorage</code>, allowing hosts to control startup retries for temporarily unavailable authentication brokers and display retry status.</li> </ul> <h3>Fixed</h3> <ul> <li>Fixed Claude account failover after a shared weekly usage limit recovers, ensuring accounts become available again when usage reporting no longer shows that window.</li> <li>Authentication broker startup now tolerates temporary broker restarts and connection interruptions by retrying for up to 30 seconds, while still failing immediately for rejected tokens or malformed snapshots.</li> <li>Improved authentication broker compatibility across client and broker versions by tolerating unknown credential, usage, and observed-usage fields and values.</li> <li>Fixed Codex Spark usage meters so reports that identify the meter only by <code>metered_feature</code> remain correctly scoped to the Spark model.</li> <li>Fixed Antigravity CLI requests for models whose available IDs include effort suffixes, such as resolving <code>gemini-4-hydrogen</code> when only <code>gemini-4-hydrogen-high</code> is offered.</li> </ul> <h2>@oh-my-pi/pi-catalog</h2> <h3>Added</h3> <ul> <li>Added support for prompt-cache prewarming on the first-party Anthropic endpoint via the <code>isPromptCachePrewarmSupported</code> compatibility capability.</li> </ul> <h3>Fixed</h3> <ul> <li>Fixed hosted judgment requests for TypeSafe, OpenRouter, Command Code, and OpenCode Zen when evaluating more than 32 questions.</li> <li>Improved Devin image handling to support more conversation images within provider limits, and updated image compaction for Devin Gemini models to reflect their billed token usage.</li> <li>Fixed Devin Gemini model pricing so models switch from promotional to list pricing when promotions end.</li> <li>Fixed Kiro API key authentication (<code>KIRO_API_KEY</code> and <code>ksk_…</code>) so model listing, chat, and <code>/usage</code> requests work correctly.</li> </ul> <h2>@oh-my-pi/pi-coding-agent</h2> <h3>Breaking Changes</h3> <ul> <li>Changed SDK cache warming controls: createAgentSession.cacheWarming now accepts the mode cap "off", "streaming", or "idle"; the lower of this value and providers.cacheWarming applies. Use "off" instead of false.</li> <li>Replaced artifact://N links with readable, listable, and completable local://N.log files at the session's local:// root. artifact:// links in existing sessions are not migrated.</li> <li>Replaced session.handoff(focus) with session.compact(focus, { mode: "handoff" }); use abortCompaction() to cancel it.</li> <li>Consolidated shared background processes under one per-user daemon rather than project-specific brokers. Daemons and services from older versions are not adopted; omp ps now reports global, project, and session units together, removes --global, changes --json output to {daemon, reports}, and no longer adds ?project= to dsm:// handles.</li> <li>Renamed SDK deferSkillDescriptionCompression to deferHintCompression and startSkillDescriptionCompression() to startHintCompression(); these settings and methods now also cover MCP namespace hints.</li> </ul> <h3>Added</h3> <ul> <li>Added built-in MCP transforms, including the Codex Desktop node_repl replacement. Built-ins can be disabled with mcp.builtinTransforms; /mcp list identifies replaced servers and /mcp enable node_repl can explicitly connect one.</li> <li>Added optional MCP eval namespaces through mcp.evalNamespaces, exposing servers as JavaScript and Python namespaces with documentation at xd://eval/. Results preserve text and structured data, and binary resources are saved as local:// files.</li> <li>Added model-generated names for /tan agents and their tray cards, with a readable random fallback when naming fails or times out.</li> <li>Added configurable cache-warming strategies (auto, keepalive, and compact), including long-tool-call keepalive and pre-expiry prompt compaction. Cache-warming events now report the selected method and costs.</li> <li>Added optional cross-instance task discovery and messaging with task.crossInstance, exposing same-folder instances through agent://peers.</li> <li>Added btw mode: drafts beginning with ? submit as /btw questions and receive distinct UI indication.</li> <li>Added Tern live cards for running subagents and background jobs, including progress, tool counts, status, controls, transcripts, and restored access to finished agents.</li> <li>Added daemon update survival and optional TCP hosting through OMP_DAEMON_TCP, preserving running services, PTYs, and language servers across daemon upgrades.</li> <li>Loaded speech-to-text and text-to-speech models once per machine, unloading them after 15 minutes of inactivity.</li> <li>Added ovnfs worktree support to /wt, the project resume picker, and omp worktree list/clear, while preserving session resume from the source repository after checkout removal.</li> <li>Added Claude subscription plans to omp usage and /usage output.</li> <li>Added progress reporting for background skill-description and MCP namespace hint generation.</li> <li>Added excludeFromContext to pi.sendMessage payloads so extensions can display and save per-turn messages without sending the

Nguyên văn nguồn
Chi tiết nguồn
  • v19.0.0 Can Bölük — Oh My Pi releases · 19 thg 11, 2026
Kỹ thuật

b12528

<details open=""> <p>server: fix sleep state for v1/realtime (<a class="issue-link js-issue-link" data-error-text="Failed to load title" data-id="6477722568" data-permission-text="Title is private" data-url="https://github.com/ggml-org/llama.cpp/issues/32567" data-hovercard-type="pull_request" data-hovercard-url="/ggml-org/llama.cpp/pull/32567/hovercard" href="https://github.com/ggml-org/llama.cpp/pull/32567">#32567</a>)</p> </details> <p><strong>Website:</strong></p> <ul> <li><a href="https://llama.app" rel="nofollow">https://llama.app</a></li> </ul> <p><strong>Attestations:</strong></p> <ul> <li><a href="https://github.com/ggml-org/llama.cpp/attestations/69408923">https://github.com/ggml-org/llama.cpp/attestations/69408923</a></li> </ul> <p><strong>macOS/iOS:</strong></p> <ul> <li><a href="https://github.com/ggml-org/llama.cpp/releases/download/b12528/llama-b12528-bin-macos-arm64.tar.gz">macOS Apple Silicon (arm64)</a></li> <li>macOS Apple Silicon (arm64, KleidiAI enabled) <a href="https://github.com/ggml-org/llama.cpp/pull/23780" data-hovercard-type="pull_request" data-hovercard-url="/ggml-org/llama.cpp/pull/23780/hovercard">DISABLED</a></li> <li><a href="https://github.com/ggml-org/llama.cpp/releases/download/b12528/llama-b12528-bin-macos-x64.tar.gz">macOS Intel (x64)</a></li> <li><a href="https://github.com/ggml-org/llama.cpp/releases/download/b12528/llama-b12528-xcframework.zip">iOS XCFramework</a></li> </ul> <p><strong>Linux:</strong></p> <ul> <li><a href="https://github.com/ggml-org/llama.cpp/releases/download/b12528/llama-b12528-bin-ubuntu-x64.tar.gz">Ubuntu x64 (CPU)</a></li> <li><a href="https://github.com/ggml-org/llama.cpp/releases/download/b12528/llama-b12528-bin-ubuntu-arm64.tar.gz">Ubuntu arm64 (CPU)</a></li> <li><a href="https://github.com/ggml-org/llama.cpp/releases/download/b12528/llama-b12528-bin-ubuntu-s390x.tar.gz">Ubuntu 24.04 s390x (CPU)</a></li> <li><a href="https://github.com/ggml-org/llama.cpp/releases/download/b12528/llama-b12528-bin-ubuntu-26.04-s390x.tar.gz">Ubuntu 26.04 s390x (CPU)</a></li> <li><a href="https://github.com/ggml-org/llama.cpp/releases/download/b12528/llama-b12528-bin-ubuntu-zdnn-s390x.tar.gz">Ubuntu 26.04 s390x (zDNN)</a></li> <li><a href="https://github.com/ggml-org/llama.cpp/releases/download/b12528/llama-b12528-bin-ubuntu-vulkan-x64.tar.gz">Ubuntu x64 (Vulkan)</a></li> <li><a href="https://github.com/ggml-org/llama.cpp/releases/download/b12528/llama-b12528-bin-ubuntu-vulkan-arm64.tar.gz">Ubuntu arm64 (Vulkan)</a></li> <li><a href="https://github.com/ggml-org/llama.cpp/releases/download/b12528/llama-b12528-bin-ubuntu-cuda-12.8-x64.tar.gz">Ubuntu x64 (CUDA 12)</a> - <a href="https://github.com/ggml-org/llama.cpp/releases/download/b12528/cudart-llama-b12528-bin-ubuntu-cuda-12.8-x64.tar.gz">CUDA 12.8 libraries</a></li> <li><a href="https://github.com/ggml-org/llama.cpp/releases/download/b12528/llama-b12528-bin-ubuntu-cuda-13.5-x64.tar.gz">Ubuntu x64 (CUDA 13)</a> - <a href="https://github.com/ggml-org/llama.cpp/releases/download/b12528/cudart-llama-b12528-bin-ubuntu-cuda-13.5-x64.tar.gz">CUDA 13.5 libraries</a></li> <li><a href="https://github.com/ggml-org/llama.cpp/releases/download/b12528/llama-b12528-bin-ubuntu-cuda-13.5-arm64.tar.gz">Ubuntu arm64 (CUDA 13)</a> - <a href="https://github.com/ggml-org/llama.cpp/releases/download/b12528/cudart-llama-b12528-bin-ubuntu-cuda-13.5-arm64.tar.gz">CUDA 13.5 libraries</a></li> <li><a href="https://github.com/ggml-org/llama.cpp/releases/download/b12528/llama-b12528-bin-ubuntu-rocm-10.0-x64.tar.gz">Ubuntu x64 (ROCm 10.0)</a></li> <li><a href="https://github.com/ggml-org/llama.cpp/releases/download/b12528/llama-b12528-bin-ubuntu-openvino-2026.4.1-x64.tar.gz">Ubuntu x64 (OpenVINO)</a></li> <li><a href="https://github.com/ggml-org/llama.cpp/releases/download/b12528/llama-b12528-bin-ubuntu-sycl-fp32-x64.tar.gz">Ubuntu x64 (SYCL FP32)</a></li> <li><a href="https://github.com/ggml-org/llama.cpp/releases/download/b12528/llama-b12528-bin-ubuntu-sycl-fp16-x64.tar.gz">Ubuntu x64 (SYCL FP16)</a></li> <li><a href="https://github.com/ggml-org/llama.cpp/releases/download/b12528/llama-b12528-bin-linux-arm64-snapdragon.tar.gz">Linux arm64 (Snapdragon: CPU, Adreno GPU, Hexagon NPU)</a> - <a href="https://github.com/ggml-org/llama.cpp/blob/master/docs/backend/snapdragon/linux.md">setup guide</a></li> </ul> <p><strong>Android:</strong></p> <ul> <li><a href="https://github.com/ggml-org/llama.cpp/releases/download/b12528/llama-b12528-bin-android-arm64.tar.gz">Android arm64 (CPU)</a></li> <li><a href="https://github.com/ggml-org/llama.cpp/releases/download/b12528/llama-b12528-bin-android-arm64-snapdragon.tar.gz">Android arm64 (Snapdragon: CPU, Adreno GPU, Hexagon NPU)</a> - <a href="https://github.com/ggml-org/llama.cpp/blob/master/docs/backend/snapdragon/README.md">setup guide</a></li> </ul> <p><strong>Windows:</strong></p> <ul> <li><a href="https://github.com/ggml-org/llama.cpp/releases/download/b12528/llama-b12528-bin-win-cpu-x64.zip">Windows x64 (CPU)</a></li> <li><a href="https://github.com/ggml-org/llama.cpp/releases/download/b12528/llama-b12528-bin-win-cpu-arm64.zip">Windows arm64 (CPU)</a></li> <li><a href="https://github.com/ggml-org/llama.cpp/releases/download/b12528/llama-b12528-bin-win-opencl-adreno-arm64.zip">Windows arm64 (OpenCL Adreno)</a></li> <li><a href="https://github.com/ggml-org/llama.cpp/releases/download/b12528/llama-b12528-bin-win-cuda-12.4-x64.zip">Windows x64 (CUDA 12)</a> - <a href="https://github.com/ggml-org/llama.cpp/releases/download/b12528/cudart-llama-bin-win-cuda-12.4-x64.zip">CUDA 12.4 DLLs</a></li> <li><a href="https://github.com/ggml-org/llama.cpp/releases/download/b12528/llama-b12528-bin-win-cuda-13.5-x64.zip">Windows x64 (CUDA 13)</a> - <a href="https://github.com/ggml-org/llama.cpp/releases/download/b12528/cudart-llama-bin-win-cuda-13.5-x64.zip">CUDA 13.5 DLLs</a></li> <li><a href="https://github.com/ggml-org/llama.cpp/releases/download/b12528/llama-b12528-bin-win-cuda-13.5-arm64.zip"

Nguyên văn nguồn
Chi tiết nguồn
  • b12528 ggml-org/llama.cpp releases · 19 thg 11, 2026
Kỹ thuật

b12527

<details open=""> <p>model : limite: reject mudd configs that index past mudd_tap_idx on load (<a class="issue-link js-issue-link" data-error-text="Failed to load title" data-id="6474460663" data-permission-text="Title is private" data-url="https://github.com/ggml-org/llama.cpp/issues/32562" data-hovercard-type="pull_request" data-hovercard-url="/ggml-org/llama.cpp/pull/32562/hovercard" href="https://github.com/ggml-org/llama.cpp/pull/32562">#32562</a>)</p> <p>n_mudd_taps comes from limite.mudd.tap_count, a scalar read separately<br> from the length-checked tap_indices array. mudd_tap_idx is a fixed<br> std::array&lt;int32_t, 512&gt; indexed at [i*n_taps] in both the tensor load<br> and the graph build, so a crafted GGUF with n_layer * n_mudd_taps &gt; 512<br> reads past the array.</p> <p>Reject that at hparams load; the one check covers both index sites.</p> </details> <p><strong>Website:</strong></p> <ul> <li><a href="https://llama.app" rel="nofollow">https://llama.app</a></li> </ul> <p><strong>Attestations:</strong></p> <ul> <li><a href="https://github.com/ggml-org/llama.cpp/attestations/69393968">https://github.com/ggml-org/llama.cpp/attestations/69393968</a></li> </ul> <p><strong>macOS/iOS:</strong></p> <ul> <li><a href="https://github.com/ggml-org/llama.cpp/releases/download/b12527/llama-b12527-bin-macos-arm64.tar.gz">macOS Apple Silicon (arm64)</a></li> <li>macOS Apple Silicon (arm64, KleidiAI enabled) <a href="https://github.com/ggml-org/llama.cpp/pull/23780" data-hovercard-type="pull_request" data-hovercard-url="/ggml-org/llama.cpp/pull/23780/hovercard">DISABLED</a></li> <li><a href="https://github.com/ggml-org/llama.cpp/releases/download/b12527/llama-b12527-bin-macos-x64.tar.gz">macOS Intel (x64)</a></li> <li><a href="https://github.com/ggml-org/llama.cpp/releases/download/b12527/llama-b12527-xcframework.zip">iOS XCFramework</a></li> </ul> <p><strong>Linux:</strong></p> <ul> <li><a href="https://github.com/ggml-org/llama.cpp/releases/download/b12527/llama-b12527-bin-ubuntu-x64.tar.gz">Ubuntu x64 (CPU)</a></li> <li><a href="https://github.com/ggml-org/llama.cpp/releases/download/b12527/llama-b12527-bin-ubuntu-arm64.tar.gz">Ubuntu arm64 (CPU)</a></li> <li><a href="https://github.com/ggml-org/llama.cpp/releases/download/b12527/llama-b12527-bin-ubuntu-s390x.tar.gz">Ubuntu 24.04 s390x (CPU)</a></li> <li><a href="https://github.com/ggml-org/llama.cpp/releases/download/b12527/llama-b12527-bin-ubuntu-26.04-s390x.tar.gz">Ubuntu 26.04 s390x (CPU)</a></li> <li><a href="https://github.com/ggml-org/llama.cpp/releases/download/b12527/llama-b12527-bin-ubuntu-zdnn-s390x.tar.gz">Ubuntu 26.04 s390x (zDNN)</a></li> <li><a href="https://github.com/ggml-org/llama.cpp/releases/download/b12527/llama-b12527-bin-ubuntu-vulkan-x64.tar.gz">Ubuntu x64 (Vulkan)</a></li> <li><a href="https://github.com/ggml-org/llama.cpp/releases/download/b12527/llama-b12527-bin-ubuntu-vulkan-arm64.tar.gz">Ubuntu arm64 (Vulkan)</a></li> <li><a href="https://github.com/ggml-org/llama.cpp/releases/download/b12527/llama-b12527-bin-ubuntu-cuda-12.8-x64.tar.gz">Ubuntu x64 (CUDA 12)</a> - <a href="https://github.com/ggml-org/llama.cpp/releases/download/b12527/cudart-llama-b12527-bin-ubuntu-cuda-12.8-x64.tar.gz">CUDA 12.8 libraries</a></li> <li><a href="https://github.com/ggml-org/llama.cpp/releases/download/b12527/llama-b12527-bin-ubuntu-cuda-13.5-x64.tar.gz">Ubuntu x64 (CUDA 13)</a> - <a href="https://github.com/ggml-org/llama.cpp/releases/download/b12527/cudart-llama-b12527-bin-ubuntu-cuda-13.5-x64.tar.gz">CUDA 13.5 libraries</a></li> <li><a href="https://github.com/ggml-org/llama.cpp/releases/download/b12527/llama-b12527-bin-ubuntu-cuda-13.5-arm64.tar.gz">Ubuntu arm64 (CUDA 13)</a> - <a href="https://github.com/ggml-org/llama.cpp/releases/download/b12527/cudart-llama-b12527-bin-ubuntu-cuda-13.5-arm64.tar.gz">CUDA 13.5 libraries</a></li> <li><a href="https://github.com/ggml-org/llama.cpp/releases/download/b12527/llama-b12527-bin-ubuntu-rocm-10.0-x64.tar.gz">Ubuntu x64 (ROCm 10.0)</a></li> <li><a href="https://github.com/ggml-org/llama.cpp/releases/download/b12527/llama-b12527-bin-ubuntu-openvino-2026.4.1-x64.tar.gz">Ubuntu x64 (OpenVINO)</a></li> <li><a href="https://github.com/ggml-org/llama.cpp/releases/download/b12527/llama-b12527-bin-ubuntu-sycl-fp32-x64.tar.gz">Ubuntu x64 (SYCL FP32)</a></li> <li><a href="https://github.com/ggml-org/llama.cpp/releases/download/b12527/llama-b12527-bin-ubuntu-sycl-fp16-x64.tar.gz">Ubuntu x64 (SYCL FP16)</a></li> <li><a href="https://github.com/ggml-org/llama.cpp/releases/download/b12527/llama-b12527-bin-linux-arm64-snapdragon.tar.gz">Linux arm64 (Snapdragon: CPU, Adreno GPU, Hexagon NPU)</a> - <a href="https://github.com/ggml-org/llama.cpp/blob/master/docs/backend/snapdragon/linux.md">setup guide</a></li> </ul> <p><strong>Android:</strong></p> <ul> <li><a href="https://github.com/ggml-org/llama.cpp/releases/download/b12527/llama-b12527-bin-android-arm64.tar.gz">Android arm64 (CPU)</a></li> <li><a href="https://github.com/ggml-org/llama.cpp/releases/download/b12527/llama-b12527-bin-android-arm64-snapdragon.tar.gz">Android arm64 (Snapdragon: CPU, Adreno GPU, Hexagon NPU)</a> - <a href="https://github.com/ggml-org/llama.cpp/blob/master/docs/backend/snapdragon/README.md">setup guide</a></li> </ul> <p><strong>Windows:</strong></p> <ul> <li><a href="https://github.com/ggml-org/llama.cpp/releases/download/b12527/llama-b12527-bin-win-cpu-x64.zip">Windows x64 (CPU)</a></li> <li><a href="https://github.com/ggml-org/llama.cpp/releases/download/b12527/llama-b12527-bin-win-cpu-arm64.zip">Windows arm64 (CPU)</a></li> <li><a href="https://github.com/ggml-org/llama.cpp/releases/download/b12527/llama-b12527-bin-win-opencl-adreno-arm64.zip">Windows arm64 (OpenCL Adreno)</a></li> <li><a href="https://github.com/ggml-org/llama.cpp/releases/download/b12527/llama-b12527-bin-win-cuda-12.4-x64.zip">Windows x64 (CUDA 12)</a> - <a href="https://github.com/ggml-org/llama.cpp/releases/download/b12527/cudart-llama-bin-w

Nguyên văn nguồn
Chi tiết nguồn
  • b12527 ggml-org/llama.cpp releases · 19 thg 11, 2026
Đánh giá

Introducing the Amazon SageMaker JumpStart Model Leaderboard

Câu hỏi That choice is powerful, but it also creates a real question: Which model should you start with?

<p>Amazon SageMaker JumpStart gives you hundreds of open-weight foundation models you can deploy quickly. That choice is powerful, but it also creates a real question: Which model should you start with? Comparing models usually means reading model cards one at a time and hunting for benchmark scores that different sources published under different conditions. Then you still must guess how each model will perform, and what it will cost, for your use case.</p> <p>Today we are launching the <strong>Amazon SageMaker JumpStart Model Leaderboard</strong>, a new way to compare <a href="https://console.aws.amazon.com/sagemaker/home#/launch?source=jumpstart&amp;page=jumpstart" target="_blank" rel="noopener">JumpStart models</a> side by side on quality, speed, and cost, directly in Amazon SageMaker Studio. In this post, we explain what the leaderboard shows, define the key terms in plain language, and walk through how to go from a shortlist to a deployed model.</p> <h2 id="what-is-the-jumpstart-model-leaderboard">What is the JumpStart Model Leaderboard?</h2> <p>The leaderboard is a new view of the JumpStart base models page in SageMaker Studio, right next to the Categories and All models views. It uses the same search bar and filters you already know, and it opens on a leaderboard of large language models (LLMs). You see rankings immediately without answering setup questions first. For each model, it shows two kinds of information:</p> <ol type="1"> <li><strong>Quality scores</strong> on three public benchmarks (BIG-Bench Hard, GPQA Diamond, and MMLU-Pro) that AWS runs itself, using the same evaluation setup for every model so the numbers are comparable.</li> <li><strong>Performance and cost metrics</strong> that AWS measures by sending a standardized test workload to an endpoint using each JumpStart deployment configuration, so you see how fast a model responds and what it costs to run.</li> </ol> <p>Because both kinds of numbers come from the same place and the same method, you can make a like-for-like comparison instead of stitching together data from different websites. At launch, the leaderboard covers <strong>70+ models</strong> from providers including Meta, Google, Qwen, IBM, and Arcee, with more added over time.</p> <p>The following screenshot shows the leaderboard in SageMaker Studio, with benchmark scores, performance metrics, and estimated cost for each model.</p> <div style="width: 810px" class="wp-caption alignnone"> <a href="https://d2908q01vomqb2.cloudfront.net/f1f836cb4ea6efb2a0b1b99f41ad8b103eff4b59/2026/11/18/ML-22496-1-1.png" target="_blank" rel="noopener"><img src="https://d2908q01vomqb2.cloudfront.net/f1f836cb4ea6efb2a0b1b99f41ad8b103eff4b59/2026/11/18/ML-22496-1-1.png" alt="SageMaker JumpStart Model Leaderboard in SageMaker Studio showing benchmark scores, performance metrics, and estimated cost per model" width="800"></a> <p class="wp-caption-text">Figure 1: Leaderboard overview in SageMaker Studio</p> </div> <h2 id="key-concepts">Key concepts</h2> <p>This section defines the terms the leaderboard uses: the quality benchmarks, the use case profiles and optimization goals that shape the performance numbers, and the performance metrics themselves.</p> <h3 id="quality-benchmarks">Quality benchmarks</h3> <p>A benchmark is a standardized test: a fixed set of questions with known answers. Running every model through the same test gives you a score you can compare. At launch, the leaderboard reports three reasoning benchmarks for LLMs:</p> <table border="1px" width="100%" cellpadding="10px"> <tbody> <tr> <td><strong>Benchmark</strong></td> <td><strong>What it measures, in plain terms</strong></td> </tr> <tr> <td><strong>BIG-Bench Hard (BBH)</strong></td> <td>A set of challenging tasks that require multi-step reasoning, such as logical deduction, tracking objects, and multi-step arithmetic.</td> </tr> <tr> <td><strong>GPQA Diamond</strong></td> <td>Graduate-level science questions that are difficult to answer by searching the web. A good signal of deep reasoning.</td> </tr> <tr> <td><strong>MMLU-Pro</strong></td> <td>Broad knowledge and reasoning across many subjects, such as science, law, and engineering, using harder multiple-choice questions.</td> </tr> </tbody> </table> <p>Higher is better for all three. The leaderboard shows each benchmark separately rather than blending them into one number. That keeps the results transparent: You can weigh the benchmarks that matter for your application and check each score against the public definition of the benchmark.</p> <h3 id="use-case-profiles-and-optimization-goals">Use case profiles and optimization goals</h3> <p>Speed and cost depend on your workload and on how the model is deployed. Two menus above the leaderboard control which performance and cost numbers you see:</p> <ol type="1"> <li><strong>Use case profile.</strong> Select the profile closest to your workload from the View benchmarks for your use case menu. The default is Max Context. The profile changes only the performance and cost metrics. Quality benchmark scores are the same for every profile.</li> <li><strong>Optimization goal.</strong> For each profile, a model can have several deployment configurations, and the leaderboard shows metrics for one at a time. Choose which one with the Optimized for menu. The default is Balanced.</li> </ol> <p>Each optimization goal picks a different configuration:</p> <table border="1px" width="100%" cellpadding="10px"> <tbody> <tr> <td><strong>Optimization goal</strong></td> <td><strong>Configuration shown</strong></td> </tr> <tr> <td><strong>Balanced</strong></td> <td>The configuration with the lowest estimated cost per token. Because cost per token depends on both instance price and throughput, this balances what you pay per hour against how fast the instance generates tokens.</td> </tr> <tr> <td><strong>Cost optimized</strong></td> <td>The configuration with the lowest hourly instance price. Latency can be higher and throughput lower

Nguyên văn nguồn
Chi tiết nguồn
Kỹ thuật

langchain==1.5.2

<p>Changes since langchain==1.5.1</p> <p>release(langchain): 1.5.2 (<a class="issue-link js-issue-link" data-error-text="Failed to load title" data-id="6477918167" data-permission-text="Title is private" data-url="https://github.com/langchain-ai/langchain/issues/41948" data-hovercard-type="pull_request" data-hovercard-url="/langchain-ai/langchain/pull/41948/hovercard" href="https://github.com/langchain-ai/langchain/pull/41948">#41948</a>)<br> feat(langchain): tag synthetic HITL tool messages with source metadata (<a class="issue-link js-issue-link" data-error-text="Failed to load title" data-id="6461091843" data-permission-text="Title is private" data-url="https://github.com/langchain-ai/langchain/issues/41928" data-hovercard-type="pull_request" data-hovercard-url="/langchain-ai/langchain/pull/41928/hovercard" href="https://github.com/langchain-ai/langchain/pull/41928">#41928</a>)</p>

Nguyên văn nguồn
Chi tiết nguồn
Đánh giá

Design responsible AI controls for Amazon Aurora PostgreSQL

<p>A customer support chatbot that queries historical tickets and knowledge base articles stored in <a href="https://aws.amazon.com/rds/aurora/" target="_blank" rel="noopener">Amazon Aurora PostgreSQL-Compatible Edition</a> can deliver fast, context-aware answers. However, it also introduces challenges around privacy, accuracy, safety, and governance, especially when the underlying data contains sensitive customer information. In this post, we map the eight AWS Responsible AI (RAI) dimensions to Aurora PostgreSQL capabilities. We then walk through a design pattern for the Explainability dimension, using a Retrieval Augmented Generation (RAG) based customer support chatbot as the reference use case. This post implements one dimension, Explainability, in depth. Treat it as Part 1 of a series rather than a full build-out of all eight.</p> <p><a href="https://aws.amazon.com/rds/aurora/postgresql-features/" target="_blank" rel="noopener">Amazon Aurora PostgreSQL-Compatible Edition</a> makes it more straightforward to operationalize responsible AI through its native <a href="https://docs.aws.amazon.com/AmazonRDS/latest/AuroraUserGuide/aurora-ml.html" target="_blank" rel="noopener">Aurora Machine Learning (Aurora ML)</a> capabilities. With Aurora ML, you can invoke <a href="https://aws.amazon.com/sagemaker/" target="_blank" rel="noopener">Amazon SageMaker</a> endpoints and <a href="https://aws.amazon.com/bedrock/" target="_blank" rel="noopener">Amazon Bedrock</a> foundation models directly from SQL, without data movement, custom extract, transform, and load (ETL) pipelines, or complex integrations. This post uses Aurora ML to generate vector embeddings from SQL, keeping the embedding pipeline inside the database layer.</p> <p>AWS defines eight core Responsible AI dimensions, published through the <a href="https://docs.aws.amazon.com/wellarchitected/latest/responsible-ai-lens/responsible-ai-lens.html" target="_blank" rel="noopener">AWS Well-Architected Responsible AI Lens</a>, <a href="https://aws.amazon.com/bedrock/guardrails/" target="_blank" rel="noopener">Amazon Bedrock Guardrails</a>, and the <a href="https://aws.amazon.com/ai/responsible-ai/" target="_blank" rel="noopener">Responsible Use of AI Guide</a>. These dimensions (Fairness, Explainability, Privacy and Security, Safety, Controllability, Veracity and Robustness, Governance, and Transparency) provide a structured framework for evaluating and implementing trustworthy AI systems.</p> <h2 id="use-case-overview">Use case overview</h2> <p>Customer support systems often struggle to provide accurate, context-aware responses while maintaining privacy and compliance. Consider a customer support chatbot built on Amazon Aurora PostgreSQL that uses RAG with <a href="https://github.com/pgvector/pgvector" target="_blank" rel="noopener">pgvector</a> embeddings, Amazon Bedrock model inference, and AWS Responsible AI guardrails to deliver grounded, explainable responses.</p> <p>A customer can ask questions such as “Why was my last bill high?” or “Where is my last order?” and receive accurate answers with citations, while sensitive data remains obscured through personally identifiable information (PII) redaction and access controls.</p> <h2 id="mapping-the-aws-responsible-ai-dimensions-to-aurora-postgresql">Mapping the AWS Responsible AI dimensions to Aurora PostgreSQL</h2> <p>The following table maps the eight AWS Responsible AI dimensions to practical implementation strategies for a customer support chatbot built on Amazon Aurora PostgreSQL-Compatible Edition. It highlights how database features, AWS AI services, and governance controls work together to deliver a trustworthy, secure, and explainable RAG-based chatbot.</p> <table border="1px" width="100%" cellpadding="10px"> <tbody> <tr> <td><strong>RAI Dimension</strong></td> <td><strong>What It Means</strong></td> <td><strong>Implementation Approaches</strong></td> <td><strong>Key PostgreSQL / Aurora Features</strong></td> <td><strong>Primary AWS Services / Tools</strong></td> </tr> <tr> <td>1. Fairness</td> <td>Chatbot should provide equal quality responses regardless of language, region, or customer profile</td> <td>Bias testing using diverse support queries. Decouple demographics from embeddings. Metadata filtering and multilingual testing.</td> <td>pgvector, JSONB metadata, SQL filtering, views</td> <td>Amazon Bedrock Model Evaluation, Amazon SageMaker Clarify</td> </tr> <tr> <td>2. Explainability</td> <td>Users and support teams should know why the chatbot gave a specific answer</td> <td>Log retrieved chunks, similarity scores, and citations. RAG tracing with source attribution. Full interaction logging.</td> <td>chat_interactions table, array columns, JSONB logs, pgvector similarity</td> <td>Amazon Bedrock Knowledge Bases, Amazon Bedrock InvokeModel</td> </tr> <tr> <td>3. Privacy and Security</td> <td>Protect customer PII such as name, phone, and order details in tickets</td> <td>PII redaction before embedding. Row-Level Security (RLS), encryption, IAM auth, RBAC, and anonymization.</td> <td>Row-Level Security, pgcrypto, roles, policies</td> <td>Amazon Bedrock Guardrails, AWS KMS, Amazon Comprehend, AWS IAM</td> </tr> <tr> <td>4. Safety</td> <td>Help prevent harmful, toxic, or inappropriate responses</td> <td>Content filters, denied topics, prompt injection protection, rate limiting, and moderation.</td> <td>SQL functions, interaction logs</td> <td>Amazon Bedrock Guardrails</td> </tr> <tr> <td>5.&nbsp;Controllability</td> <td>Ability to steer chatbot behavior and escalate to humans</td> <td>Human-in-the-loop escalation, feedback table, feature flags, prompt control, and kill switch.</td> <td>chat_feedback table, escalation tables, triggers</td> <td>Amazon Bedrock Guardrails, Amazon CloudWatch, AWS CloudTrail</td> </tr> <tr> <td>6. Veracity and Robustness</td> <td>Verify accurate responses grounded in support data and avoid hallucinations</td> <td>RAG grounding with similarity threshold. Versi

Nguyên văn nguồn
Chi tiết nguồn
Kiến trúc

v0.34.0

<h1>v0.34.0</h1> <p><strong>1,061 commits from 412 contributors (120 new).</strong> Thank you!</p> <h2>TL;DR</h2> <ul> <li><strong>10 new models</strong> (Mistral Large 4, MiniMax-M3.1-Flash, Nemotron-3.5-Transcribe, and more!)</li> <li><strong>Extensible KV cache is on by default:</strong> vLLM sizes the KV cache from memory actually free after warmup, so most models get a bigger cache and more throughput - in most cases you should now <strong>remove <code>gpu_memory_utilization</code></strong> from your configs.</li> <li><strong>Model Runner V1 is gone.</strong> Model Runner V2 is now the only GPU runner.</li> <li><strong>Big speedups</strong> on Blackwell and MI355X: up to 1.95× decode (Step5 on GB200), 1.6× throughput (GLM-5.3-Flash), 1.24–1.40× faster long-prompt TTFT (GLM-5.3).</li> <li><strong>Upgrade check:</strong> default block size is now 64, ROCm 7.2 images are gone, and <code>huggingface_hub &gt;= 2.0.0</code> is required. See <a href="#before-you-upgrade">Before you upgrade</a>.</li> </ul> <h2>Before you upgrade</h2> <div class="markdown-alert markdown-alert-important"><p class="markdown-alert-title"><svg data-component="Octicon" class="octicon octicon-report mr-2" viewBox="0 0 16 16" version="1.1" width="16" height="16" aria-hidden="true"><path d="M0 1.75C0 .784.784 0 1.75 0h12.5C15.216 0 16 .784 16 1.75v9.5A1.75 1.75 0 0 1 14.25 13H8.06l-2.573 2.573A1.458 1.458 0 0 1 3 14.543V13H1.75A1.75 1.75 0 0 1 0 11.25Zm1.75-.25a.25.25 0 0 0-.25.25v9.5c0 .138.112.25.25.25h2a.75.75 0 0 1 .75.75v2.19l2.72-2.72a.749.749 0 0 1 .53-.22h6.5a.25.25 0 0 0 .25-.25v-9.5a.25.25 0 0 0-.25-.25Zm7 2.25v2.5a.75.75 0 0 1-1.5 0v-2.5a.75.75 0 0 1 1.5 0ZM9 9a1 1 0 1 1-2 0 1 1 0 0 1 2 0Z"></path></svg>Important</p><p>These changes can break existing setups.</p> </div> <table> <thead> <tr> <th>Change</th> <th>What to do</th> <th>PR</th> </tr> </thead> <tbody> <tr> <td>Model Runner V1 removed</td> <td>Remove <code>VLLM_USE_V2_MODEL_RUNNER=0</code>; it now fails at startup. Port V1 logits processors to <code>vllm.v1.worker.gpu.sample.logits_processor</code>.</td> <td><a class="issue-link js-issue-link" data-error-text="Failed to load title" data-id="6001962406" data-permission-text="Title is private" data-url="https://github.com/vllm-project/vllm/issues/63373" data-hovercard-type="pull_request" data-hovercard-url="/vllm-project/vllm/pull/63373/hovercard" href="https://github.com/vllm-project/vllm/pull/63373">#63373</a></td> </tr> <tr> <td>Default <code>--block-size</code> is 64 (was 16)</td> <td>Pass <code>--block-size 16</code> to keep the old behavior. ROCm paged attention still uses 16.</td> <td><a class="issue-link js-issue-link" data-error-text="Failed to load title" data-id="5989713643" data-permission-text="Title is private" data-url="https://github.com/vllm-project/vllm/issues/63231" data-hovercard-type="pull_request" data-hovercard-url="/vllm-project/vllm/pull/63231/hovercard" href="https://github.com/vllm-project/vllm/pull/63231">#63231</a></td> </tr> <tr> <td>Extensible KV cache on by default; <code>gpu_memory_utilization</code> defaults to 1.0</td> <td><strong>In most cases it's now best to not set <code>gpu_memory_utilization</code></strong>. This should provide maximum kv cache capacity while avoiding runtime OOMs. Use it to share a GPU with other processes or when extensible kv cache is disabled.</td> <td><a class="issue-link js-issue-link" data-error-text="Failed to load title" data-id="5427044802" data-permission-text="Title is private" data-url="https://github.com/vllm-project/vllm/issues/56492" data-hovercard-type="pull_request" data-hovercard-url="/vllm-project/vllm/pull/56492/hovercard" href="https://github.com/vllm-project/vllm/pull/56492">#56492</a></td> </tr> <tr> <td>ROCm 7.2 artifacts dropped</td> <td>Use the ROCm 10.0 wheels and images. <code>-rocm72</code> is no longer published.</td> <td><a class="issue-link js-issue-link" data-error-text="Failed to load title" data-id="5877058226" data-permission-text="Title is private" data-url="https://github.com/vllm-project/vllm/issues/61939" data-hovercard-type="pull_request" data-hovercard-url="/vllm-project/vllm/pull/61939/hovercard" href="https://github.com/vllm-project/vllm/pull/61939">#61939</a></td> </tr> <tr> <td><code>huggingface_hub &gt;= 2.0.0</code> required</td> <td>Also requires <code>tokenizers &gt;= 0.23.3</code> and <code>timm &gt;= 1.0.30</code>.</td> <td><a class="issue-link js-issue-link" data-error-text="Failed to load title" data-id="6123285409" data-permission-text="Title is private" data-url="https://github.com/vllm-project/vllm/issues/64779" data-hovercard-type="pull_request" data-hovercard-url="/vllm-project/vllm/pull/64779/hovercard" href="https://github.com/vllm-project/vllm/pull/64779">#64779</a></td> </tr> <tr> <td>QuTLASS / <code>fp_quant</code> removed</td> <td><code>fp_quant</code> checkpoints no longer load.</td> <td><a class="issue-link js-issue-link" data-error-text="Failed to load title" data-id="6170612057" data-permission-text="Title is private" data-url="https://github.com/vllm-project/vllm/issues/65230" data-hovercard-type="pull_request" data-hovercard-url="/vllm-project/vllm/pull/65230/hovercard" href="https://github.com/vllm-project/vllm/pull/65230">#65230</a></td> </tr> <tr> <td>Marlin W4A8 removed</td> <td>Use <code>--quantization humming</code> with <code>VLLM_HUMMING_INPUT_QUANT_CONFIG</code> instead of <code>VLLM_MARLIN_INPUT_DTYPE</code>.</td> <td><a class="issue-link js-issue-link" data-error-text="Failed to load title" data-id="6194757596" data-permission-text="Title is private" data-url="https://github.com/vllm-project/vllm/issues/65542" data-hovercard-type="pull_request" data-hovercard-url="/vllm-project/vllm/pull/65542/hovercard" href="https://github.com/vllm-project/vllm/pull/65542">#65542</a></td> </tr> <tr> <td>Keye-VL and Keye-VL-1.5 removed</td> <td>—</td> <td><a class="issue-link js-issue-link" data-error-text="Failed to load title" data-id="6080460497" data-permission-text="Title

Nguyên văn nguồn
Chi tiết nguồn
  • v0.34.0 vllm-project/vllm releases · 19 thg 11, 2026
Kỹ thuật

How Salesforce is advancing global-scale model serving with cross-Region inference routing in Amazon SageMaker AI

<p><em>Thanks to Amit Gupta, Senior Director- AI/ML and Mark Wang, Software Engineering PMTS from Salesforce for your contributions in the vision and partnership of cross-Region inference routing on SageMaker AI.</em></p> <p>Salesforce’s Agentforce Foundation team serves a wide variety of models for products across Salesforce, including Agentforce and the Salesforce Trust Layer, in 30+ global Regions. These include both models trained in house and open source models. Hosting models separately in each Region splits capacity and makes traffic spikes harder to absorb, which turns a capacity shortfall into an availability problem.</p> <p>This post shows how Salesforce set up the cross-Region inference routing in SageMaker AI and how it lets the team add capacity securely while keeping its data residency commitments. It also walks through how Salesforce sees where each request was served and why. The cross-Region inference routing gives an application one invocation path across SageMaker AI endpoints in several Regions and accounts, and sends each request to an endpoint that can serve it. Those endpoints are Salesforce’s own, already deployed. The post starts with the capacity problem Salesforce faced, explains how the router works and how Salesforce integrated it into its Internal Gateway, walks through an example setup, and then describes what the team observed.</p> <h2 id="why-serving-these-models-makes-capacity-a-reliability-problem">Why serving these models makes capacity a reliability problem</h2> <p>Salesforce’s Agentforce Foundation team serves customers from the Region their requests come from, across 30+ global Regions and hundreds of AWS accounts, and they provision capacity based on each customer’s traffic. Every customer has its own peak window, and those windows often overlap. Provisioning for the overlapping peak would be the safe choice, but it costs too much. Customers also onboard new business units at short notice, so they must grow the fleet quickly. Adding capacity quickly, Region by Region, leaves Salesforce with capacity fragmentation: some Regions have idle instances while others run out.</p> <p>Before the cross-Region inference routing in SageMaker AI, Salesforce used its Internal Gateway to send traffic that the local endpoints couldn’t handle to other Regions. The team built and operated this path, supported by basic health checks, retries, and Region-switching logic. That approach had two gaps. First, the gateway had no capacity signals from either the local or the remote endpoint, so it couldn’t tell when to shift traffic or whether the destination had room. Second, cross-Region traffic had to leave the virtual private cloud (VPC) through a NAT gateway and an internet gateway and travel over public AWS networking. In design reviews, the team also raised the risk of cascading failures: moving traffic during an incident can overload the next Region, so Salesforce needed a cap on cross-Region traffic.</p> <p>Building this logic separately in every application doesn’t scale across Salesforce. The team needed a managed, capacity-aware routing path.</p> <h2 id="what-the-inference-router-does">What the inference router does</h2> <p>The inference router is a Regional SageMaker AI resource in your account. Your application invokes the router instead of an endpoint, passing a routing target name with the same request body and content type it sends to <code>InvokeEndpoint</code> today. A routing target is a name you choose for one logical model, such as a toxicity classifier, and it lists the endpoints and inference components that may serve that model by their full Amazon Resource Names (ARNs). A router holds one routing target per model, and a problem with one model’s endpoints does not move traffic for the others. A routing target can mix single-model endpoints and inference components.</p> <p>Because each destination is named by its ARN, it can be in another Region or another AWS account. The account that owns an endpoint grants access through a resource-based endpoint policy that allows one named router to invoke it, which is how Salesforce runs it.</p> <p>The router prefers its own Region, and while the local destination serves successfully, requests stay there. The router judges each destination by the outcomes of recent requests. Platform faults, 5xx errors from the model container, throttles, and timeouts lower a destination’s health. As they persist, the router moves a growing share of new requests to healthy destinations, typically starting within about a minute. It can take several minutes for a destination with little traffic. Signals such as scaling progress and failed instance launches affect how quickly the router responds once errors appear, but they do not move traffic by themselves. After recovery, traffic returns over several minutes, more slowly than it left, so it does not swing back and forth. When a destination fails a request without running the model, for example by throttling it, the router retries that request once on another destination.</p> <p>Client errors such as invalid input do not move traffic, because they would fail the same way in any Region. Model latency does not move traffic either, because a slow response often reflects a long prompt or output and does not by itself mean the endpoint is unhealthy. The router also does not deploy models, copy artifacts, or add instances. It uses the capacity you have deployed, and each endpoint’s own autoscaling keeps adding instances as its traffic grows.</p> <h2 id="how-salesforce-integrated-the-router-into-its-internal-gateway">How Salesforce integrated the router into its Internal Gateway</h2> <p>In Salesforce’s Internal Gateway, the router call sits exactly where the direct <code>InvokeEndpoint</code> call used to be: in the per-request egress path, after the data residency check and before response handling. That check is the one routing decision the gateway keeps. A request tagged with a reside

Nguyên văn nguồn
Chi tiết nguồn
Kỹ thuật

b12525: ggml, ci : use llvm 22.1.8 (#32548)

<ul> <li>ggml, ci : use llvm 22.1.8</li> </ul> <p>Signed-off-by: Adrien Gallouët <a href="mailto:angt@huggingface.co">angt@huggingface.co</a></p> <ul> <li>Update build-cpu.yml too</li> </ul> <p>Signed-off-by: Adrien Gallouët <a href="mailto:angt@huggingface.co">angt@huggingface.co</a></p> <ul> <li>Fix clang 22</li> </ul> <p>Signed-off-by: Adrien Gallouët <a href="mailto:angt@huggingface.co">angt@huggingface.co</a></p> <ul> <li>Fix release.yml (arm64 with vulkan)</li> </ul> <p>Signed-off-by: Adrien Gallouët <a href="mailto:angt@huggingface.co">angt@huggingface.co</a></p> <ul> <li>Fix server.yml</li> </ul> <p>Signed-off-by: Adrien Gallouët <a href="mailto:angt@huggingface.co">angt@huggingface.co</a></p> <hr> <p>Signed-off-by: Adrien Gallouët <a href="mailto:angt@huggingface.co">angt@huggingface.co</a></p>

Nguyên văn nguồn
Chi tiết nguồn
Kiến trúc

⚡️Anthropic: Know Before You IPO — Tanay Jaipuria, Wing VC

Câu hỏi What did it cost to get there?

Anthropic reported $17B in quarterly revenue. What did it cost to get there? Tanay Jaipuria, a partner at Wing Venture Capital, opens the S-1 with swyx and explains how the picture changes when you count the compute spent on free users and model training. The opening update walks through the preliminary IPO price range, Anthropic’s roadshow, Cowork adoption, the AMD inference slide, and where Claude Fable 5.2 and GPT-6.2 Astra landed on the leaderboard, with Fable demos in motion. Then Tanay gets into how Anthropic financed its growth, why it matters whether customers use Claude Code or buy tokens through a cloud partner, and how model labs might put spare compute to work. A Latent Space Lightning Pod. The interview was recorded November 6, 2026. The opening update was recorded November 18, ahead of final pricing. Chapters 0:00 Anthropic’s IPO update 0:49 The price range, valuation and revenue 3:05 Cowork and Claude Code adoption 4:01 Anthropic’s AMD inference slide 4:40 Compute costs and the margin target 5:31 Claude Fable 5.2 and GPT-6.2 Astra 7:38 Inside the S-1 with Tanay Jaipuria 9:44 What drove revenue growth 11:56 Capital efficiency 13:59 Meta’s VR bet 15:38 Why Anthropic used so little capital early on 17:18 Where free and paid inference costs go 21:27 The next big workloads after coding 24:50 Training, inference and the cost of data 28:30 Training data and copyright 31:50 Claude Code, cloud partners and distribution 36:16 Why go public now? 40:06 Enterprise AI spending 42:51 Who captures the value of inference 48:20 AI for science and wet labs 51:17 What to watch next Show notes Tanay’s S-1 breakdown — https://www.tanayj.com/p/anthropic-s-1-breakdown Anthropic’s roadshow, financial highlights and prospectus — https://ipo.anthropic.com/us/ Tanay at Wing Venture Capital — https://www.wing.vc/people/tanay-jaipuria Our earlier IPO conversation with Jamin Ball — https://youtu.be/P735f1xlpL0 More from Latent Space — https://latent.space/

Nguyên văn nguồn
Chi tiết nguồn
Kiến trúc

DaisyRose-27B: Pushing the Frontier of Post-Training Efficiency

Today we are releasing DaisyRose-27B , RadixArk’s first model, built on Qwen3.8-27B and trained with reinforcement learning using Miles . We trained it on long-horizon agent tasks with sessions up to 1M tokens, with just 1,200 curated tasks in 18 days on 144 H200 GPUs. DaisyRose-27B improves Terminal-Bench 4.1 pass@1 from 5.6% to 12.0% . Beyond Terminal-Bench 4.1, it also delivers gains on a variety of other reasoning, instruction following, and coding benchmarks. The model retains Qwen3.8-27B’s architecture, tokenizer, chat template, image understanding, and 256K-token context window . The longer training sessions use harness summarization to span multiple trajectories. See the model card for model properties and inference settings. We recommend serving DaisyRose-27B with SGLang and DSpark speculative decoding. The SGLang cookbook provides launch configurations for different GPUs and checkpoint formats.

Nguyên văn nguồn
Chi tiết nguồn
Đánh giá

How Social Department built culture-aware video discovery

<p><em>This post is co-written by Mariah Miller and Sirajus Salekin from AWS and Nicholas Chase from Social Department.</em></p> <p>Entertainment marketing teams manage tens of thousands of video assets across studios, networks, and streaming applications. Most campaigns begin with the same video discovery problem: finding the one moment in the footage that will resonate with an audience. When a brief lands, the clock starts, and what they need is almost never something a database can describe.</p> <p>Semantic search, on its own, has a ceiling: It answers the question you can articulate. Ask it for a character holding a coffee mug and it will find every coffee mug in a thousand episodes. The queries that actually drive entertainment campaigns look nothing like that. They are questions of judgment: which moment in a show will travel, which line is quotable, and which single frame will work as a meme.</p> <p>Those questions can’t be typed into a search box. Answering them requires two things the search index doesn’t have: an editorial definition of what “quotable” or “memeable” means for a given library, and a continuously refreshed view of what is moving outside the library. Historically, answering them meant a person with deep knowledge of the show watching footage for a week, and that person doesn’t scale across a catalog.</p> <p>In this post, we describe how Social Department, the AI operating system for entertainment marketing, set out to encode those editorial questions as executable retrieval pipelines. The result is QuickSearch, a set of single-action queries that return the clips a marketer is looking for without requiring them to describe it: Shareable Scenes, Quotable Quotes, Memeable Moments, and Trend Align.</p> <p>The models in this solution run on Amazon Bedrock, so customer data stays within the AWS data boundary. It isn’t shared with model providers and isn’t used to train the underlying models.</p> <p>Social Department’s approach to responsible AI is that AI proposes and people decide. In production, code checks generated output against the source material (for example, a quote is discarded if it isn’t in the episode transcript). Amazon Bedrock Guardrails helps screen customer search text for prompt attacks, and a person on the customer’s team approves every post before it publishes.</p> <p>Social Department owns the pipeline architecture, the retrieval system, and the editorial criteria behind each action. TrackIt, an AWS Premier Tier Services Partner, implemented and delivered against that design. TwelveLabs’ Marengo 3.0 and Pegasus 1.2 models on <a href="https://aws.amazon.com/bedrock/" target="_blank" rel="noopener">Amazon Bedrock</a> provide the scene-level video reasoning the pipeline builds on. AWS supplies the infrastructure, the model access including Anthropic’s Claude models, and <a href="https://aws.amazon.com/opensearch-service/" target="_blank" rel="noopener">Amazon OpenSearch Service</a>, using its serverless option, Amazon OpenSearch Serverless, for retrieval.</p> <p><strong>A note on scope:</strong>&nbsp;This post describes the AWS portions of the system: the ingest and indexing pipeline, the retrieval path, and the services that run them. The components Social Department owns, including its media and data backbone, content analysis and discovery engine, editorial criteria, and evaluation program, are described only to the depth needed to follow the architecture. The named services that follow are the ones this post is about, not an inventory of Social Department’s full system.</p> <h2 id="about-social-department">About Social Department</h2> <p>Social Department is the AI operating system for entertainment marketing, serving major studios, networks, streamers, and FAST applications. It automates campaign strategy, asset creation, publishing, and performance optimization across a client’s full content library. Most of those functions depend on locating the right moment inside the footage first, and semantic clip discovery has been core to the product from the beginning.</p> <h2 id="why-semantic-search-alone-was-insufficient">Why semantic search alone was insufficient</h2> <p>Entertainment marketing operates under two constraints that a general-purpose search layer can’t satisfy.</p> <h3 id="the-valuable-queries-are-the-ones-nobody-can-phrase">The valuable queries are the ones nobody can phrase</h3> <p>A marketer can describe an object, a location, or a line of dialogue, but not what will travel. That judgment, this moment lands and that one doesn’t, is the actual skill in the job. It was never automatable because it depends on knowing how an audience behaves rather than what a video contains. A search layer that only understands content requires a human to supply the intent, and supplying that intent well requires having already watched the show.</p> <h3 id="trend-signal-lives-outside-the-library-and-moves-on-a-different-clock">Trend signal lives outside the library and moves on a different clock</h3> <p>No amount of video understanding tells you what audiences are talking about this morning. That information is created outside the library and changes hourly, while a library changes only when new content is added. A system that answers these queries has to acquire trend signal continuously and reconcile it against the library at request time.</p> <p>That distinction drove the core architectural decision: content understanding and trend acquisition run as independent pipelines on their own schedules and are joined only when a user selects a QuickSearch action.</p> <h2 id="solution-overview">Solution overview</h2> <p>With QuickSearch, teams can interact with a video library in a different way, one where the system proposes rather than waits. It provides four actions, each with its own retrieval strategy, editorial criteria, and ranking model. The walkthrough that follows describes how each is implemented. The list here introduces what they do.</p> <ol type=

Nguyên văn nguồn
Chi tiết nguồn
201–220 / 562
Trang 11 / 29