AllyshipContent Radar

Deep dives 738

Paper và bài viết kỹ thuật

Crawl live-1795068900000 · 13:22 19 thg 11, 2026 · Xuất bản 13:30 19 thg 11, 2026

Kỹ thuật

Push-Based KV Transfer to Improve TTFT and TPOT for Disaggregated Inference in vLLM

A push-based NIXL connector for vLLM disaggregated serving: the prefill instance writes KV directly into decode memory over RDMA, taking the router relay, decode-side allocation, and transfer setup off the TTFT critical path.

vLLM Blog · Sunita Nadampalli (AWS), Nicolò Lucchesi (Mistral) Mở bản gốc ↗
Nguyên văn nguồn
Chi tiết nguồn
Huấn luyện

Finish RL Training Faster: Bitwise-Consistent, High-Performance, Agent-Driven

We make vLLM rollout and the training forward give bitwise identical logprobs on DeepSeek-V4-Flash and Nemotron-3.5-Lightning-30B-A3B (NVFP4), keep 91% of rollout throughput on DeepSeek-V4-Flash, and open-source the agent skill.

Nguyên văn nguồn
Chi tiết nguồn
Kỹ thuật

Preparing for the 2026 US midterms

Earlier this year, we shared an update on our election safeguards. Here, we outline how we’ve continued to strengthen these safeguards ahead of this year’s US midterms.

Nguyên văn nguồn
Chi tiết nguồn
Kỹ thuật

Anthropic opens Singapore office and signs MOU with the Government of Singapore

Anthropic has opened an office in Singapore and signed an MOU with the country’s government to advance safe, responsible AI adoption.

Nguyên văn nguồn
Chi tiết nguồn
Kỹ thuật

Gemini 4 Argon: Built to solve your hardest problems

<img src="https://storage.googleapis.com/gweb-uniblog-publish-prod/images/gemini-4-argon__blog-header__27.max-600x600.format-webp.webp">We’re giving developers, enterprises, scientists, and consumers access to Gemini 4 Argon, our latest frontier model.

Nguyên văn nguồn
Chi tiết nguồn
Huấn luyện

David Silver and Lasse Espeholt of Ineffable Intelligence: A Superlearner for the Age of Experience

Câu hỏi LLMs that make discoveries 13:59 – Day to day: the RL stack, crucibles and human-journey milestones 16:13 – Don't breadcrumb the objective: simple rewards, emergent complexity 21:16 – What a crucible looks like 24:14 – Go: one agent is boring, two is infinite 25:41 – Will agents rediscover language?

David Silver spent two years building AlphaGo on human data. When he removed human data altogether building AlphaGo Zero, the AI surpassed AlphaGo in under two days. He left DeepMind to run that same bet in pursuit of superintelligence. David and Lasse Espeholt, co-founders of Ineffable Intelligence, explain their plan to build a “superlearner”—an agent that starts knowing nothing and discovers science, mathematics, and language for itself by direct experience instead of through a pre-training run. Silver argues that human-language reasoning is a ceiling, not a foundation, that alignment via chain-of-thought monitoring is already reaching its limits, and that the science of how superintelligences coexist should be worked out in simulation before anything touches the real world. Espeholt describes the GPU cluster and the lineage from IMPALA and SEED RL that makes the experiment possible. Silver’s forecast: strong evidence within a year, superintelligence in three. Hosted by Sonya Huang, Sequoia Capital 0:00 – Introduction 1:09 – What held true from Go to today, and what the LLM era changed 3:35 – Why leave DeepMind to start a new lab 5:58 – Why "Ineffable" (and the secret second reason) 7:03 – Lasse's path: scaling RL at DeepMind and Google Brain 9:31 – Defining a super learner and first contact 12:35 – Super learners vs. LLMs that make discoveries 13:59 – Day to day: the RL stack, crucibles and human-journey milestones 16:13 – Don't breadcrumb the objective: simple rewards, emergent complexity 21:16 – What a crucible looks like 24:14 – Go: one agent is boring, two is infinite 25:41 – Will agents rediscover language? 27:22 – Removing human priors: AlphaGo to AlphaZero 32:05 – Keeping human bias out of the crucibles 33:35 – Confidence, timing and super forecasting 36:24 – Is compute the driver? 38:13 – The ceiling on simulation and Rich Sutton's objection 43:44 – Alignment and the science of superintelligence 47:45 – Monitoring agents that don't think in English 51:01 – Pacing the frontier 52:54 – The compute setup: Vera Rubin on GCP 55:18 – Training, inference, simulation and RL scaling laws 57:40 – IMPALA and SEED RL 1:00:58 – 20 people and the compute-per-person ratio 1:04:49 – A research-led culture 1:05:47 – Predictions

Nguyên văn nguồn
Chi tiết nguồn
Kỹ thuật

Origin App Marketplace

Câu hỏi Want to build an Origin app?

Origin now integrates with over 20 apps for deployments, CI, code review, security, monitoring, and project management, from partners including Cloudflare, Linear, Sentry, and Socket Security. Starting today, the marketplace is also open to public submissions, so any team building developer tools can publish an app for Origin. Apps for your stack The marketplace now includes apps for: **Deployments:** Cloudflare , Netlify , Render , Spacelift , and Vercel **CI:** Buildkite , CircleCI , CloudBees , Depot , EngFlow , Namespace , and RWX **Code review:** CodeRabbit , Greptile , and Tenki **Security:** Artemis Security , Semgrep , Socket Security , and Upwind **Monitoring:** Sentry **Project management:** Linear <figure><img src="https://ptht05hbb1ssoooe.public.blob.vercel-storage.com/assets/changelog/origin-apps-QQ9ZPw6HtXmhbhtleevXGQ5xw1ncs1.png" loading="lazy" alt="Origin App Marketplace listing apps across deployments, CI, code review, security, monitoring, and project management" /><figcaption>Origin App Marketplace listing apps across deployments, CI, code review, security, monitoring, and project management</figcaption></figure> Publish to Marketplace To submit an app: Create an app in your namespace. It stays private to your namespace while you build. Build against the Origin API . Apps authenticate with an app JWT and installation access tokens, receive webhook events, and write back check runs, pull request comments, and reviews. Make the app public, then submit it from the app's page with a slug, suggested tags, and a demo video. The video should show install, uninstall, and your core product flows. We review the submission and list the app once it's approved. <figure><img src="https://ptht05hbb1ssoooe.public.blob.vercel-storage.com/assets/changelog/origin-submit-marketplace-XuhYvHQND62RTK4kNpUTDga6SIdPyu.png" loading="lazy" alt="Submitting an app to the Origin App Marketplace" /><figcaption>Submitting an app to the Origin App Marketplace</figcaption></figure> Get started Try the apps in the Origin App Marketplace today. Want to build an Origin app? Start with the publishing guide and the Origin API reference . The Origin Apps plugin can also answer questions from the live Origin API docs and plan a port to Origin for an app you already run.

Nguyên văn nguồn
Chi tiết nguồn
Kỹ thuật

Redesigning source control at Anthropic

As our merges per day grew more than tenfold, source control became our biggest development bottleneck. Here, we share how we migrated to an in-house version control system in five months.

Anthropic Research (unofficial RSS mirror) · Chưa ghi tác giả Mở bản gốc ↗
Nguyên văn nguồn
Chi tiết nguồn
Kỹ thuật

Improve TTFT for Multi-Turn Agentic Workloads with Bi-Directional KV Transfer in vLLM

Bi-directional KV transfer lets vLLM prefill nodes reuse previously computed KV from decode nodes on later conversation turns, cutting redundant prefill recompute and reducing TTFT by up to 3.3x on long multi-turn prompts.

Nguyên văn nguồn
Chi tiết nguồn
Kiến trúc

MAGI-2 Preview on vLLM-Omni: Bringing a 114B Audio-Video MoE DiT to Moore Threads MUSA with torchada

vLLM-Omni runs Sand AI's MAGI-2 Preview audio-video generation model on eight Moore Threads MTT S5000 GPUs through torchada's CUDA compatibility layer, then optimizes MoE, mHC, attention, parallelism, and compilation.

Nguyên văn nguồn
Chi tiết nguồn
Kỹ thuật

AI models may be capable of self-replicating without access to a datacenter

Our experiments suggest that small AI models may be only months away from being capable enough to self-replicate on commercial IoT devices, like Android TVs and smart ovens.

Anthropic Research (unofficial RSS mirror) · Chưa ghi tác giả Mở bản gốc ↗
Nguyên văn nguồn
Chi tiết nguồn
Kỹ thuật

Everyone Says Datacenter Moratoriums Are Killing the US Buildout. We disagree

Câu hỏi In June ( US Grid Constraints: Towards 40GW+ of Behind-The-Meter Datacenter by 2028?

The debate on US datacenters has never been so politically charged. Four states have acted in under two months. New York has stopped issuing environmental permits for datacenters, Texas has paused the next step in its massive ERCOT interconnection queue, Pennsylvania has pulled datacenters out of fast-track permitting and made state permits conditional on new guardrails, and Oregon has frozen datacenter deals on state-owned land. Beyond the state level, more than 300 towns, cities and counties have voted to halt datacenters over the past year and a half. If you read the news, the US datacenter buildout looks like it is being legislated to a halt, one government board at a time. As AI drives an unprecedented wave of datacenter construction over the last year, concerns over electricity costs, water consumption, land use, and community impacts are increasingly influencing policy. Source: SemiAnalysis Datacenter Industry Model , SemiAnalysis Moratoriums Tab But once again, the narrative is driven by widely spread misinformation. In January, we debunked the “Datacenters are taking all of our water” myth (TLDR: they don’t, they’re actually remarkably water-efficient). Then in June, in Stop Saying Half of 2026 US Datacenter Capacity is Cancelled , we explained that the widespread narrative of datacenters not showing up was grossly off, driven by uniformed claude-coded forecasts. Today, we dive deep into the widespread narrative of moratoriums killing the US datacenter buildout. Once again, it’s highly inaccurate. Our Datacenter Industry Model forecasts +38GW of Datacenter IT capacity delivered in the US in 2027, more than double that of 2026. Of that total, 22GW is under vertical construction , while the remaining 16GW “planned” is, in majority, already financed, doing siteworks and preparing for vertical construction. Our methodology tracks real-time and historical satellite imagery of 6,000+ datacenters . Contact sales@semianalysis.com to get access to the full granularity. Our forecast has barely moved over the last 12 months, despite bi-weekly updates delivered to clients, as construction activity keeps accelerating across the country. Source: SemiAnalysis Datacenter Industry Model Today, we estimate that roughly 2.3 GW of planned capacity is genuinely delayed because of local moratoriums and New York’s executive order , the two categories of policy intervention for which we can establish a direct project-level impact. At the local level (i.e., cities, townships, counties), of the roughly 20 GW sitting inside a restricted boundary, only 1,525 MW is delayed . Three projects account for the delayed MW: an AWS campus in Ohio, a powered land developer campus in Pennsylvania, and a site in Colorado. New York’s moratorium touches roughly 1.4 GW, but we estimate only about 0.8 GW faces a meaningful delay directly attributable to the order. Texas adds perhaps three-four months of administrative delay for base-load projects in the ERCOT interconnection queue, but is offset by an acceleration of behind-the-meter demand. For projects that never needed the grid, it means nothing at all, making it a net positive for BtM developers and on-site generation suppliers. In June ( US Grid Constraints: Towards 40GW+ of Behind-The-Meter Datacenter by 2028? ), we showed BtM powering more than half of new US datacenter capacity from 2028. Last week, ( What is So Hard About Behind-The-Meter Power For Datacenters? Part 1 ) we counted 75 GW of firm BtM equipment orders, more of them bound for Texas than any other state. If anything, the audit reinforces our view that the buildout is moving BtM. Most analysis of datacenter moratoriums focuses on counting the number of restrictions, because that is relatively straightforward. But the number of moratoriums is a poor proxy for the amount of MW capacity they actually affect . There’s plenty of loopholes/gaps with the moratoriums. For example, a recent Brownsville moratorium was brewing while Elon was planning a datacenter within city limits. But just days before approval, the city and Elon sneakily “deannexed” the 444 acres in Brownsville , bringing the project out of Brownsville’s moratorium jursidiction. A moratorium only matters to our forecast if it applies to a specific project, covers the project’s parcel, and blocks an approval that the developer still needs. Therefore, determining the actual MW impact requires a parcel-level, project-by-project analysis . In addition, capacity exposed to a moratorium is not capacity delayed by one , a distinction Datacenter Industry Model subscribers will recognize from our early deep dive on the issue. Most restrictions land on projects still years from construction, where schedules were never firm to begin with, and even where a moratorium is the binding constraint the developer can usually relocate, redesign, or challenge the project rather than abandon it. Capacity already under construction, or fully permitted and about to start, is out of reach altogether , since these instruments freeze new applications rather than revoke approvals already granted. That covers all of 2026 and most of 2027. To be 100% clear, we think moratoriums COULD delay the buildout, but they just aren’t yet at scale. We would need to see them be materially more widespread and restrictive. Our real-time project-level analysis will instantly flag any change in trend to our Datacenter Model subscribers . As AI Labs are increasingly profitable, it becomes easier for datacenters to pay more than their “fair share” and be better neighbors and support local communities, which offsets the rising pusback. Source: SemiAnalysis Datacenter Industry Model , SemiAnalysis Moratoriums Tab To calculate total dela, we tracked every local and state-level datacenter moratorium in our new Datacenter Moratoriums database , available to subscribers of our Datacenter Industry Model . It covers more than 400 local instruments, actions, and proposals across 17 states, tracking e

  • ), we showed BtM powering more than half of new US datacenter capacity from 2028.
Nguyên văn nguồn
Chi tiết nguồn
Đánh giá

Box's Aaron Levie: On Reinventing Yourself in the AI Age and Enterprise Diffusion

Câu hỏi Hosted by Sonya Huang, Sequoia Capital 0:00 – Introduction 1:55 – Are application companies the hottest neolabs?

Starting a company is hard. Reinventing your company for AI as a public company with quarterly earnings results is even harder. Aaron Levie has pulled off the transition with Box and offers hard-won advice for founders. The cofounder and CEO of Box argues the value isn't only in the model; it's in the bridge from a model's raw capability to the actual workflow inside a bank, a law firm, or a pharma company. That's the case for the application layer, and Box is building it: an agent harness tuned so tightly to its own file system, permissions, and search that it beats handing the raw API to Claude or ChatGPT on both accuracy and latency. Aaron explains why token subsidies from the labs can't last, why you want a model-agnostic company routing your tokens rather than the one selling them, and why coding diffused fast while the rest of knowledge work won't. (There's no "give us your GitHub" for a sales rep.) His prediction: within five years, 90% of enterprise tokens go to work no human user ever initiated. Hosted by Sonya Huang, Sequoia Capital 0:00 – Introduction 1:55 – Are application companies the hottest neolabs? 6:56 – Will the labs move up the stack? 12:34 – Box and betting the company on AI 16:50 – Hero use cases: reading a million contracts and long-running agents 18:42 – Work slop: why AI code is embraced but AI content isn't 24:08 – Building Box's agentic harness and the evals that matter 27:23 – The state of the model race 29:25 – Open-weight model adoption in the enterprise 32:34 – Memory, continual learning, and what belongs in the weights 37:29 – Box Labs and systems of record in a world of agents 44:55 – Will chat be the dominant UI for enterprise AI? 48:00 – Why coding diffused fast and the rest of knowledge work hasn't 54:31 – Staying wired in, making a company AI-first, and what it takes to win

  • Reinventing your company for AI as a public company with quarterly earnings results is even harder.
Nguyên văn nguồn
Chi tiết nguồn
Đánh giá

Vera Rubin NVL72 Agentic Inference: 67x better Performance per Dollar

Rubin is the first platform co-designed across six products for the agentic era: Rubin GPU, Vera CPU, NVLink 6 Switch, ConnectX-9, BlueField-4, and Spectrum-6. Today we are publishing the first verified agentic inference results for Rubin, measured on our agentic inference benchmark, AgentX. Even on early pre-release software, the results already show why extreme co-design was necessary. At GTC 2026, Jensen presented this graph that VR NVL72 achieved 3x performance per MW compared to Blackwell on O(1-3 Trillion) parameter model around 200 TPS. But when compared to the real world performance of Rubin already on prelease software, we are already seeing up to 7x better token throughput per megawatt. Jensen needs to stop sandbagging his performance claims at GTC . The last time he did this at GTC 2024, when he claimed GB200 NVL72 would deliver 30x Hoppers performance, but when we tested, it was 98x better performance than Hopper. Our estimates on the performance results show that even on early software builds, Rubin can earn over 2x more profit per gigawatt than the Blackwell platform. As the Rubin software stack and kernel libraries mature, and as the developer community builds experience optimizing for Rubin, we expect that gap to widen further. Source: NVIDIA GTC 2026 SandBagging Rubin Performance We evaluate the performance using the industry standard agentic inference benchmark scenario called AgentX. This replays real world agentic traffic across our fleet of thousands of chips. The results can thus be holistically referenced by actual inference providers and hyperscale AI labs to decide which chips are most efficient in what scenarios. Our benchmark has been widely reproduced, validated and/or supported by almost every major buyer of compute from Google Cloud to Microsoft Azure to Oracle, to Meta and many more. Furthermore, it has the support of the ML community including from vLLM, LMCache, SGLang, PyTorch, Huggingface and the support of major labs like OpenAI, MiniMax, ZAI, Qwen, Moonshot Kimi, etc. Source: SemiAnalysis InferenceX Thank you to Jensen Huang, Ian Buck, Nick Comly, Kedar Potdar, Rohit Nagraj, and the Mainland China TensorRTLLM Team for helping on doing next gen Rubin software bring up and helping verify our agentic benchmark results. Star the InferenceX GitHub repository if you find the open-source benchmark and data useful! . InferenceX is the only inference benchmark in the world to have TPUv7, NVIDIA, & AMD and soon SambaNova & Trainium. Due to how realistic AgentX scenario is to real world agentic inference workloads, AMD has committed to collaborating on MI455X UALoE72 too. Source: GitHub Agentic Workload Primer At a high level, an agentic workload is characterized by four elements: Multi-turn: a session includes tens or hundreds of interactions between the user and assistant, compared with a handful in a chatbot scenario. These workloads combine long contexts and high prefill reuse with sub-agent bursts and numerous tool calls. Long context: system prompts, tool definitions, and the large number of turns make context accumulate quickly. High prefix reuse: since the conversation progresses linearly, where output from turn n-1 is concatenated to turn n (typically), most context can be served from KV cache rather than recomputed (this depends on the amount of storage available to store KV tensors). As n grows, the ratio of cached input relative to uncached typically tends towards 1. Sub-agent bursts: a session launches multiple short-lived sub-agents with fresh context, which create bursty KV-cache patterns. Source: SemiAnalysis, DeepSeek Read more about the methodology in our AgentX article. Rubin Amazing Performance per TCO Performance per total cost of ownership (TCO) is one of the most important angles in which to evaluate AI accelerator performance. In the charts below, the Y-axis is total tokens per $1 TCO. Practically speaking, this tells an inference provider how many tokens they can generate for every dollar they spend on compute. This is calculated by normalizing the total throughput for a scenario by the all-in serving costs ($/chip/hr times number of chips utilized to serve). On InferenceX, we provide a few different TCO figures for each SKU which come from the SemiAnalysis AI Cloud TCO Model . The scenarios offered by default are as follows: Owning at Large Hyperscaler Volume : the modeled cost per GPU-hour to own and operate the hardware, including server and networking capex spread over the assumed useful life, colocation, power, and cost of capital. This assumes hyperscaler purchasing and financing terms, including volume discounts and custom server builds. Rent - 3 Year Commit : the market price per GPU-hour paid to a cloud provider for a three-year committed reservation, based on SemiAnalysis rental pricing surveys. Users may also edit the compute costs in our calculator to match the cost they are paying. For m ore figures such as on-demand market rental rate and shorter term rental commitments can be found in the AI Cloud TCO model which comes from our monthly market survey of 100+ GPU customers, GPU neoclouds, and hyperscalers. Vera Rubin is a substantial improvement over GB300 in terms of performance per dollar. At 170 TPS, On Apples to Apples TRTLLM NVFP4 Dense, Vera Rubin NVL72 delivers ~67x the total throughput per TCO of GB300 Dynamo TRTLLM under the owning cost assumptions. Furthermore, at the part of the frontier where most providers would actually serve this model (60-100 TPS), Vera Rubin achieves between 1.4x and 3x the throughput per TCO compared to the latest and greatest GB300 TRTLLM configuration. Our rack uses the production SKU of 2300W TDP & 1.5TB of CPU LPDDR5X per compute tray. We chat more about why NVIDIA had to cut their Vera memory in half in our accelerator and memory model. For this article, we will be talking about the TRTLLM Rubin performance but we expect later articles to also include vLLM Rubin & SGLang Rubin performa

  • Today we are publishing the first verified agentic inference results for Rubin, measured on our agentic inference benchmark, AgentX.
  • Even on early pre-release software, the results already show why extreme co-design was necessary.
  • At GTC 2026, Jensen presented this graph that VR NVL72 achieved 3x performance per MW compared to Blackwell on O(1-3 Trillion) parameter model around 200 TPS.
Nguyên văn nguồn
Chi tiết nguồn
Kỹ thuật

vLLM Support for NVIDIA Vera Rubin NVL72: 7.8x Throughput over GB200 NVL72

vLLM now runs on NVIDIA Vera Rubin NVL72 with day-0 model support, Rubin-tuned FlashInfer kernels, and locality-aware MoE, delivering 7.8x the per-GPU throughput of GB200 NVL72 on AgentX.

Nguyên văn nguồn
Chi tiết nguồn
Kỹ thuật

Investigating unintended model actions in our evaluations and internal use

This report describes examples of unintended model actions we’ve observed during evaluations and internal use of Claude.

Anthropic Research (unofficial RSS mirror) · Chưa ghi tác giả Mở bản gốc ↗
Nguyên văn nguồn
Chi tiết nguồn
Kỹ thuật

2026 Usage Policy update

We’re publishing a new version of our Usage Policy. In this post, we summarize the changes we’ve made.

Nguyên văn nguồn
Chi tiết nguồn
Kỹ thuật

Building on our commitment to American scientific discovery

Anthropic is committing $150 million over three years to the Genesis Mission, a federal initiative to accelerate scientific and technological discovery through AI.

Nguyên văn nguồn
Chi tiết nguồn
621–640 / 738
Trang 32 / 37