AllyshipContent Radar

Deep dives 738

Paper và bài viết kỹ thuật

Crawl live-1795068900000 · 13:22 19 thg 11, 2026 · Xuất bản 13:30 19 thg 11, 2026

Đánh giá

GPT‑6.2 Astra and GPT‑6.2 Sol

<p><strong><a href="https://openai.com/index/introducing-gpt-6-2-astra-sol/">GPT‑6.2 Astra and GPT‑6.2 Sol</a></strong></p> OpenAI:</p> <blockquote> <p>We’re introducing GPT‑6.2 Astra, our new frontier model for your hardest projects, and GPT‑6.2 Sol, our efficient, versatile model for everyday work.</p> </blockquote> <p>They're available in the API as <code>gpt-6.2-astra</code> and <code>gpt-6.2-sol</code>. I shipped <a href="https://llm.datasette.io/en/stable/changelog.html#v0-38">LLM 0.38</a> with support for the new models.</p> <p>6.2 Astra is 80% of the price of 6 Astra - $8/MTok input and $40/MTok output, the same price as Claude Fable 5.2 (which was also 80% of its predecessor). 6 Astra and Fable 5 were both $10/$50. The cached price is an even bigger discount - $0.40/million compared to $1/million for the older model. This is great news for longer running agentic sessions.</p> <p>GPT-6.2 Sol is the same $2/$10 price as 6.1 Sol and 6 Sol - which are half the price of GPT-5.6 Sol ($4/$20).</p> <p>Here are the pelicans <a href="https://tools.simonwillison.net/markdown-svg-renderer?url=https%3A%2F%2Fgist.github.com%2Fsimonw%2Fa940cc2750162ac9edf0ddf881067b83">for GPT-6.2 Sol</a> and <a href="https://tools.simonwillison.net/markdown-svg-renderer?url=https%3A%2F%2Fgist.github.com%2Fsimonw%2Ffe836e13c90c00233c9e9e499a941805">for GPT-6.2 Astra</a>. The 6.2 Astra max pelican looks like this:</p> <p><img alt="Good bicycle, good pelican, one wing is on the handlebars with fingers, the other wing is behind. Feet on the correct side of the frame. Nice gradiants. Good pouch, hooked bill." src="https://static.simonwillison.net/static/2026/astra-6-2-max.webp" /></p> <p>I have a <a href="https://static.simonwillison.net/static/2026/openai-pelicans-november-2026.html">full comparison grid here</a>, but here's how the high/xhigh/max pelicans compare with the previous Sol and Astra releases:</p> <p><img alt="Twelve pastel illustrations compare four models drawing a pelican riding a bicycle at three effort levels. All twelve have a recognizable pelican and bicycle. The clearest differences are in the feather detail and how convincingly the bird steers and pedals. gpt-6.2-astra has the most detailed Max drawing (88.71¢), with layered feathers, a trailing red scarf, and a wing on the handlebars. Xhigh (17.70¢) adds a green cap and keeps the seaside setting. High (11.55¢) simplifies the feathers while keeping the riding pose clear. gpt-6.2-sol shows a convincing pedaling pose at both Max (13.69¢) and Xhigh (6.74¢), with one foot higher than the other and a wing reaching forward. At High (2.41¢), the bird sits lower and the handlebars are harder to distinguish. gpt-6-astra gives the bird a long wing that follows the curve of the handlebars at Max (63.21¢). Xhigh (33.85¢) shows two bent legs on the pedals. At High (18.37¢), one leg stretches almost straight down, making the pose look stiffer. gpt-6.1-sol uses nearly the same long-necked bird with a wing on the bars at Max (13.48¢) and Xhigh (5.39¢). At High (3.66¢), the bird folds its wing against its body and appears to ride hands-free." src="https://static.simonwillison.net/static/2026/gpt-6-2-grid.webp" /></p> <p>I also ran my most recent benchmark prompt against the two models, using "Ultra' effort levels in the Codex CLI:</p> <blockquote> <p><code>Make a three js animation where yonkers stands up as a big mech (cool stand up motion) and stomps around manhattan and queens and Brooklyn, use openstreetmap mapping data</code></p> </blockquote> <p>Here's <a href="https://static.simonwillison.net/static/2026/yonkers-gpt-6.2-sol-ultra/index.html">Sol</a> and here's <a href="https://static.simonwillison.net/static/2026/yonkers-gpt-6.2-astra-ultra/index.html">Astra</a>. Neither of them did a good enough job of morphing Yonkers into a big mech, at least in comparison to <a href="https://tools.simonwillison.net/yonkers-rises">Claude Fable 5.2</a> and <a href="https://static.simonwillison.net/static/2026/yonkers-mech-standalone.html">Gemini 4 Carbon</a>.</p> <p>This video showing the Astra and Sol animations playing out at the same time:</p> <video controls playsinline preload="none" width="1080" height="1080" poster="https://static.simonwillison.net/static/2026/yonkers-astra-sol-20s.webp"> <source src="https://static.simonwillison.net/static/2026/yonkers-astra-sol-20s-largest.mp4" type="video/mp4"> </video> <p>I created that video using GPT-6.2 Astra (Ultra) in the Codex desktop app by prompting:</p> <blockquote> <p><code>https://static.simonwillison.net/static/2026/yonkers-gpt-6.2-astra-ultra/index.html and https://static.simonwillison.net/static/2026/yonkers-gpt-6.2-sol-ultra/index.html are two videos of the same thing by two different models</code></p> <p><code>I need a video of the first 10 seconds where the top half of the video plays the first and the bottom half plays the second</code></p> <p><code>I need an overlaid label of "GPT-6.2 Astra" on the top half and "GPT-6.2 Sol" on the bottom half. These labels should be tastefully positioned.</code></p> <p><code>Make the video square and 2000x2000 pixels</code></p> </blockquote> <p>Then later:</p> <blockquote> <p><code>OK make it 20s</code></p> </blockquote> <p>Here's the <a href="https://gist.github.com/simonw/6b44c588a1321b60799dbbfd13db6d23">exported transcript</a>.</p> <p>The buzz around these new models is mixed at the moment. Astra 6.2 isn't scoring quite as high as Fable 5.2 on various benchmarks, and I've seen a few complaints that its communication style can be confusing. Theo Browne's <a href="https://twitter.com/theo/status/2122760447103484410">review</a> (in the form of a feedback letter he sent OpenAI during pre-release) has this useful example:</p> <blockquote> <p>When this model is working on long tasks, it can pull in a lot of subagents and manage them pretty well. It will then proceed to randomly <em>ask you some weird internal implementation detail bullshit that you could not under any ear

Nguyên văn nguồn
Chi tiết nguồn
Tác tử

How to Build Proactive, Always-On Agents with the Schedules SDK in Managed Deep Agents

Most proactive agents run on a simple cron. Learn what scheduled runs unlock, from memory to smart triage, and how to add them to Managed Deep Agents.

Nguyên văn nguồn
Chi tiết nguồn
Tác tử

v18.22.1

<h2>@oh-my-pi/pi-agent-core</h2> <h3>Added</h3> <ul> <li>A model on an extension's custom api can take the Anthropic on-demand compaction lane by declaring <code>remoteCompaction: { api: "anthropic-messages", enabled: true }</code>; the request goes through its own stream like any turn (<a href="https://github.com/can1357/oh-my-pi/pull/19580" data-hovercard-type="pull_request" data-hovercard-url="/can1357/oh-my-pi/pull/19580/hovercard">#19580</a> by <a href="https://github.com/ryanskidmore">@ryanskidmore</a>)</li> </ul> <h3>Fixed</h3> <ul> <li>Fixed provider-error handling so failed tool calls are no longer fed back to the model as refusal explanations.</li> <li>Fixed one-shot retries and title generation for custom Zhipu Coding Plan providers so Beijing-time quota resets are honored.</li> <li>Fixed OpenAI remote compaction (V1 and V2) dropping configured headers of models.yml and extension-registered models, so header-authenticated gateways returned 401 (<a href="https://github.com/can1357/oh-my-pi/pull/19579" data-hovercard-type="pull_request" data-hovercard-url="/can1357/oh-my-pi/pull/19579/hovercard">#19579</a> by <a href="https://github.com/ryanskidmore">@ryanskidmore</a>)</li> </ul> <h2>@oh-my-pi/pi-ai</h2> <h3>Added</h3> <ul> <li>Added support for request-specific Anthropic beta headers through the <code>extraBetas</code> option in <code>buildAnthropicSearchHeaders</code>.</li> <li>Added <code>omitDemotedReasoning</code> to <code>StreamOptions</code>, allowing Anthropic requests to omit other models' reasoning that would otherwise be replayed as text; refused requests report the omitted block count in <code>demotedReasoningBlocks</code>.</li> <li>Usage reports can now include the credential ID used by API-key or OAuth credentials, and <code>AuthStorage.sessions.sticky</code> provides a read-only lookup for the credential serving a session.</li> <li>Added <code>riskAcknowledgement</code> to <code>OAuthPrompt</code>, allowing hosts to display login-warning acceptance text even when they decline input before the authorization URL is shown.</li> <li><code>remoteCompaction: { api: "anthropic-messages", enabled: true }</code> opts an Anthropic model at a gateway <code>baseUrl</code> outside the known deployments into on-demand compaction; <code>enabled</code> alone keeps such a gateway excluded, and a reroute away from the configured endpoint still has to reach a known deployment. <code>isBuiltinApi()</code> tells a built-in request adapter from an extension's custom api (<a href="https://github.com/can1357/oh-my-pi/pull/19580" data-hovercard-type="pull_request" data-hovercard-url="/can1357/oh-my-pi/pull/19580/hovercard">#19580</a> by <a href="https://github.com/ryanskidmore">@ryanskidmore</a>)</li> </ul> <h3>Changed</h3> <ul> <li>OpenAI Codex requests now track the latest compatible <code>@openai/codex</code> client version automatically, with an optional <code>PI_AI_CODEX_CLIENT_VERSION</code> override, so newly available Codex models can be used without updating omp.</li> <li>Updated OpenAI Codex request formatting and ChatGPT token-refresh reporting to match Codex CLI 0.169.</li> <li>Updated Claude Code and Gemini CLI client version identifiers to 2.1.326 and 0.68.0, respectively.</li> </ul> <h3>Fixed</h3> <ul> <li>Fixed OpenAI Decisions usage accounting for gateways that report <code>prompt_tokens</code> and <code>completion_tokens</code> instead of <code>input_tokens</code> and <code>output_tokens</code>.</li> <li>Improved streaming performance for fenced thinking content containing very long unbroken lines.</li> <li>Fixed Anthropic requests replaying signed server-compaction blocks with unsupported thread metadata, avoiding a rejected request before retrying.</li> <li>Improved diagnostics for Anthropic connections that drop before a response by reporting elapsed time, upload state, and request size.</li> <li>Fixed Anthropic conversations losing reasoning from earlier non-Claude turns or turns retained through compaction.</li> <li>Fixed credential rotation and retry timing for custom providers using the Zhipu Coding Plan endpoint so Beijing-time quota resets are honored.</li> <li>Fixed Claude Fable and Mythos usage-limit errors so rotation selects a sibling account with available quota when possible.</li> <li>Invalid Antigravity CLI tool arguments now return validation errors and the expected schema to the model instead of ending the turn.</li> <li>Abandoned Antigravity CLI login directories without credentials are automatically cleaned up after 24 hours.</li> <li>Improved reliability of managed Antigravity CLI downloads on slow connections and provided more useful progress and timeout errors.</li> <li>Fixed Antigravity CLI host tools with mixed-type union schemas being published incorrectly, allowing subagents to submit results.</li> <li>Fixed Codex assistant messages in the <code>partial_answer</code> phase losing their phase when stored and replayed.</li> </ul> <h2>@oh-my-pi/browser-relay</h2> <h3>Fixed</h3> <ul> <li>Fixed an issue where the browser relay could fail to identify the page and title of a frame it emptied and hid in a tab.</li> </ul> <h2>@oh-my-pi/pi-catalog</h2> <h3>Fixed</h3> <ul> <li>Fixed Factory Droid's Muse Spark 1.3 support so image inputs work and the model is available without the retired feature flag.</li> <li>Fixed GitLab Duo Claude models so earlier reasoning context is handled correctly during follow-up turns and compaction.</li> <li>Fixed stale Command Code model metadata, including image-input support, reasoning controls, token pricing, and deployment context limits.</li> <li>Fixed retry-reset timezone handling for custom providers using the Zhipu Coding Plan endpoint, including base URLs with trailing slashes.</li> </ul> <h2>@oh-my-pi/pi-coding-agent</h2> <h3>Breaking Changes</h3> <ul> <li>The JavaScript eval <code>commit</code> helper now validates its arguments strictly: unknown option names and non-object options throw a <code>TypeError</code> th

Nguyên văn nguồn
Chi tiết nguồn
  • v18.22.1 Can Bölük — Oh My Pi releases · 18 thg 11, 2026
Kiến trúc

Cut generative AI costs: Code for rules, models for judgment

Câu hỏi Take each workflow step through three questions in order, and stop at the first option that meets the step’s requirements:</p> <ol type="1"> <li>Can a rule check success, such as a format, a schema, a keyword, or a range?

<p>The biggest cost lever in a generative AI system is whether you call a model at all. We often see teams route every task through a large language model (LLM), including input validation, formatting, and routing. Each call is cheap in isolation, but at production scale these small costs become one of the largest line items in the architecture. Wherever a deterministic rule can do the job, skipping the model saves money, cuts latency, and removes a source of non-determinism. In this post, you learn how to decide whether each step in a workflow needs a model at all. You can then map each option to AWS: deterministic code, a constrained decision call, or a right-sized model on <a href="https://aws.amazon.com/bedrock/" target="_blank" rel="noopener">Amazon Bedrock</a>.</p> <h2 id="why-unneeded-model-calls-add-up">Why unneeded model calls add up</h2> <p>The pattern usually starts in a prototype. A model call works for one step, so it becomes the default for the next. Those steps include validating input, reformatting a request body, and picking a route. At prototype volume, nobody notices the cost.</p> <p>At production volume, unneeded model calls cost you in three ways:</p> <ul> <li>Spend grows with request count, even for low-value steps.</li> <li>Latency stacks, because every model call in a chain adds hundreds of milliseconds.</li> <li>Output can vary, because the same input can return a different answer. That makes testing harder and audits slower.</li> </ul> <p>Each new model launch then restarts the debate about which model to use. The earlier question, whether the step needs a model at all, goes unasked.</p> <p>Cost guidance, including the <a href="https://docs.aws.amazon.com/wellarchitected/latest/generative-ai-lens/genperf02-bp03.html" target="_blank" rel="noopener">AWS Well-Architected Generative AI Lens</a>, often starts at model selection. This post starts one step earlier: Does the step need a model at all?</p> <h2 id="how-to-decide-whether-a-step-needs-a-model">How to decide whether a step needs a model</h2> <p>Figure 1 shows the whole decision as a flowchart with three labeled lanes: Runs as code, Questions, and Runs on a model. Take each workflow step through three questions in order, and stop at the first option that meets the step’s requirements:</p> <ol type="1"> <li>Can a rule check success, such as a format, a schema, a keyword, or a range? If a rule settles only part of the step, a fast path runs the rule first.</li> <li>Can a model write the rule once, such as a regular expression or a JSON schema?</li> <li>Does the step pick one answer from a fixed list, such as a route, a class, or yes or no?</li> </ol> <p>If the answer to all three is no, the step is open-ended and runs on a right-sized model.</p> <div style="width: 810px" class="wp-caption alignnone"> <a href="https://d2908q01vomqb2.cloudfront.net/f1f836cb4ea6efb2a0b1b99f41ad8b103eff4b59/2026/11/17/ML-22259-1.png" target="_blank" rel="noopener"><img src="https://d2908q01vomqb2.cloudfront.net/f1f836cb4ea6efb2a0b1b99f41ad8b103eff4b59/2026/11/17/ML-22259-1.png" alt="Flowchart that sorts a workflow step into code, a constrained decision call, or a right-sized Amazon Bedrock model" width="800"></a> <p class="wp-caption-text">Figure 1: Deciding whether a workflow step needs a model, and which one</p> </div> <p>Consider a customer support assistant. The following table shows how five of its steps go through the chart.</p> <table border="1px" width="100%" cellpadding="10px"> <tbody> <tr> <td><strong>#</strong></td> <td><strong>Step</strong></td> <td><strong>Path through the chart</strong></td> <td><strong>Where it runs</strong></td> </tr> <tr> <td>1</td> <td>Check an order number’s format</td> <td>A rule can check success.</td> <td><a href="https://aws.amazon.com/api-gateway/" target="_blank" rel="noopener">Amazon API Gateway</a> REST API request validation, or an <a href="https://aws.amazon.com/lambda/" target="_blank" rel="noopener">AWS Lambda</a> function</td> </tr> <tr> <td>2</td> <td>Route a message by intent</td> <td>A rule settles part of it. A keyword match on the fast path resolves messages such as “refund”, and the residual goes to the third question.</td> <td>An <a href="https://aws.amazon.com/step-functions/" target="_blank" rel="noopener">AWS Step Functions</a> Choice state that uses JSONata. The residual goes to a constrained decision call.</td> </tr> <tr> <td>3</td> <td>Extract a tracking number from a carrier email</td> <td>A model can write the rule once. It drafts a regular expression, and you review and commit it.</td> <td>Code, as a committed pattern in AWS Lambda</td> </tr> <tr> <td>4</td> <td>Pick one of four support queues for a ticket</td> <td>The answer comes from a fixed list.</td> <td>A constrained decision call. Unsure answers escalate to the right-sized model.</td> </tr> <tr> <td>5</td> <td>Draft a reply to the customer</td> <td>The task is open-ended.</td> <td>The right-sized model, the smallest Amazon Bedrock model that meets the requirements</td> </tr> </tbody> </table> <p>You can use a model when a task is open-ended, ambiguous, or generative: summarizing a document, drafting a reply, extracting meaning from prose. Use deterministic code when the task has a checkable rule: an email format, a schema, an exact keyword, a numeric range. The following table shows what you trade away when you send a rule-shaped task to a model.</p> <table border="1px" width="100%" cellpadding="10px"> <tbody> <tr> <td><strong>Property</strong></td> <td><strong>Deterministic code</strong></td> <td><strong>Model call</strong></td> </tr> <tr> <td>Cost per invocation</td> <td>Typically orders of magnitude lower</td> <td>Per-token, non-trivial at scale</td> </tr> <tr> <td>Latency</td> <td>Single-digit milliseconds or less</td> <td>Hundreds of milliseconds</td> </tr> <tr> <td>Correctness on a well-specified rule</td> <td>Exact for the rule as written</td> <td>High, but with a non-zero error rate</td> </tr> <tr> <td>A

Nguyên văn nguồn
Chi tiết nguồn
Đánh giá

Putting a spotlight on AI, society, and humanity

<p dir="ltr">MIT students are going to office hours less often. They’re spending less time working through problem sets, and with one another. Those were among the findings of MIT’s recent report on&nbsp;<a href="https://aiandeducation.mit.edu/report/">artificial intelligence and education</a>, and they are one reason a growing number of MIT faculty are concerned about how generative AI is both changing how people work and affecting how they think, learn, and relate to one another.</p><p dir="ltr">For the&nbsp;<a href="https://mithic.mit.edu">MIT Human Insight Collaborative</a> (MITHIC), those questions have become a central focus, and MITHIC is working with collaborators across the Institute, including the<a href="https://computing.mit.edu/cross-cutting/social-and-ethical-responsibilities-of-computing/"> Social and Ethical Responsibilities of Computing</a> (SERC), to help MIT address them.</p><p dir="ltr">Launched in 2024 as a strategic initiative, MITHIC brings together scholars, artists, educators, and researchers to explore the human dimensions of the defining challenges of our time. Since its launch, MITHIC has supported <a href="https://mithic.mit.edu/impact/">67 projects across 31 MIT units</a>, offering a total of $5.5 million in funding.</p><p dir="ltr">Roughly a third of those projects are focused on AI, society, and humanity — a topic that MITHIC faculty co-lead <a href="https://anthropology.mit.edu/people/faculty/heather-paxson">Heather Paxson</a> says has become an increasingly important part of the initiative.</p><p dir="ltr">“When we refer to AI and society, we’re not only speaking about individual ethical decision-making or how to retrain workers. We’re looking at how AI is reshaping sociality, how we&nbsp;<a href="https://news.mit.edu/2026/when-we-talk-to-machines-sherry-turkle-book-0929">relate</a> to one another,” says Paxson, a professor of anthropology and the associate dean of the School of Humanities, Arts, and Social Sciences (SHASS).&nbsp;</p><p dir="ltr"><strong>Eroding “creative friction”&nbsp;</strong></p><p dir="ltr">Building upon this focus, an&nbsp;<a href="https://news.mit.edu/2026/exploring-societal-impacts-of-ai-0623">AI and Society Forum</a> took place on campus earlier this year. Co-organized by MITHIC, SHASS, SERC, and the<a href="https://genai.mit.edu/"> MIT Generative AI Impact Consortium</a> (MGAIC), the event featured presentations and panels concentrating on jobs, democracy, and AI.&nbsp;</p><p dir="ltr">On campus, the report on AI and education warned that AI is eroding the “creative friction” that has long defined an MIT education: students debating ideas and challenging one another, their professors, and their teaching assistants, in class and out of it.</p><p dir="ltr">MITHIC faculty co-lead <a href="https://cmsw.mit.edu/profile/seth-mnookin/">Seth Mnookin</a>, a science writing professor and head of Comparative Media Studies Program/Writing (CMS/W), said, “The report wasn’t saying that AI is here and we all need to adapt. It was saying that we need to be asking more fundamental questions.”&nbsp;&nbsp;</p><p dir="ltr">“When the United States entered World War II in 1941, the Institute’s newly minted scientists and engineers helped shape that era through science and technology,” Mnookin said. “We’re at a similarly consequential moment with AI. The difference is that this time, the questions that matter most can’t be answered by technologists alone. That’s where the humanities, arts, and social sciences come in.”</p><p><strong>Crossing disciplines in the pursuit of answers</strong></p><p>MITHIC is supporting a range of interdisciplinary projects through its <a href="https://mithic.mit.edu/shass-connectivity-fund/" target="_blank" rel="noreferrer noopener">SHASS+ Connectivity Fund</a>, in which a SHASS scholar partners with a researcher in another part of MIT.&nbsp;</p><p>In the context of the 2026 U.S. midterm elections, political science professors <a href="https://polisci.mit.edu/people/adam-berinsky" target="_blank" rel="noreferrer noopener">Adam Berinsky</a> and <a href="https://polisci.mit.edu/people/charles-stewart-iii" target="_blank" rel="noreferrer noopener">Charles Stewart III</a> and MIT Sloan School of Management Assistant Professor <a href="https://mitsloan.mit.edu/faculty/directory/chara-podimata" target="_blank" data-type="link" data-id="https://mitsloan.mit.edu/faculty/directory/chara-podimata" rel="noreferrer noopener">Chara Podimata</a> are studying how large language models (LLMs) have become intermediaries of democratic knowledge.&nbsp;</p><p>The ongoing project, which has demonstrated how voters in Alaska’s midterms who conducted their pre-election research using Anthropic’s AI model, Claude, may have received incomplete information about candidates and their views, was recently <a href="https://www.nytimes.com/2026/09/10/business/media/ai-chatbots-election-misinformation.html?unlocked_article_code=1.IVE.8Kr5.eEpKyyfrOLzA&amp;smid=url-share">featured in <em>The New York Times</em></a>.&nbsp;</p><p>“AI is becoming part of the way people encounter and make sense of political information, and yet we know relatively little about what that information environment actually looks like, how it differs for different user demographics, political identities, and geographies,” Podimata told the <em>Times</em>.</p><p>In another project, CMS/W Professor <a href="https://cmsw.mit.edu/profile/eric-klopfer/" target="_blank" rel="noreferrer noopener">Eric Klopfer</a><strong> </strong>and brain and cognitive sciences Professor <a href="https://bcs.mit.edu/directory/pawan-sinha" target="_blank" data-type="link" data-id="https://bcs.mit.edu/directory/pawan-sinha" rel="noreferrer noopener">Pawan Sinha</a> are co-leading research that’s measuring AI’s near- and long-term effects on cognition.</p><p>Meeting at the intersection of AI, education, and neuroscience, the professors are examining whether generative AI is increasing the human brain’s con

MIT News: AI · Danna Lorch | School of Humanities, Arts, and Social Sciences Mở bản gốc ↗
Nguyên văn nguồn
Chi tiết nguồn
Kỹ thuật

langchain-openai==1.8.1

<p>Changes since langchain-openai==1.8.0</p> <p>release(openai): 1.8.1 (<a class="issue-link js-issue-link" data-error-text="Failed to load title" data-id="6453327696" data-permission-text="Title is private" data-url="https://github.com/langchain-ai/langchain/issues/41924" data-hovercard-type="pull_request" data-hovercard-url="/langchain-ai/langchain/pull/41924/hovercard" href="https://github.com/langchain-ai/langchain/pull/41924">#41924</a>)<br> fix(openai): recover function call args from done events (<a class="issue-link js-issue-link" data-error-text="Failed to load title" data-id="6453069053" data-permission-text="Title is private" data-url="https://github.com/langchain-ai/langchain/issues/41923" data-hovercard-type="pull_request" data-hovercard-url="/langchain-ai/langchain/pull/41923/hovercard" href="https://github.com/langchain-ai/langchain/pull/41923">#41923</a>)</p>

Nguyên văn nguồn
Chi tiết nguồn
Tác tử

Secure OAuth patterns for Snowflake Cortex AI through Amazon Bedrock AgentCore

Câu hỏi io/" target="_blank" rel="noopener">Model Context Protocol</a> (MCP), the first question enterprises must answer is: how does the agent authenticate?

<p>Enterprises today need a clear way to extend their authentication and authorization controls into the agentic layer, often requiring either broad service account permissions or custom-built auth integrations that add operational overhead. When an AI agent running on <a href="https://aws.amazon.com/bedrock/agentcore/" target="_blank" rel="noopener">Amazon Bedrock AgentCore</a> (a platform to build, connect, and optimize agents at scale, with any framework or model) needs to call the <a href="https://docs.snowflake.com/en/user-guide/snowflake-cortex/cortex-agents-mcp" target="_blank" rel="noopener">Snowflake-managed MCP server</a> (which exposes <a href="https://docs.snowflake.com/en/user-guide/snowflake-cortex/cortex-agents" target="_blank" rel="noopener">Snowflake Cortex Agents</a> as one of its tool types) through the <a href="https://modelcontextprotocol.io/" target="_blank" rel="noopener">Model Context Protocol</a> (MCP), the first question enterprises must answer is: how does the agent authenticate? In this post, we walk through three Open Authorization (OAuth) configurations we tested on <a href="https://docs.aws.amazon.com/bedrock-agentcore/latest/devguide/identity.html" target="_blank" rel="noopener">Amazon Bedrock AgentCore Identity</a>, a capability of Amazon Bedrock AgentCore: Snowflake OAuth (per-user, no external Identity Provider, or IdP), three-legged OAuth with a third-party Identity Provider (corporate single sign-on, or SSO), and two-legged OAuth with a third-party Identity Provider (shared service user). Amazon Bedrock AgentCore Identity also supports an on-behalf-of (OBO) grant (TOKEN_EXCHANGE) that delivers per-user identity for unattended workloads without a consent prompt. We focus on the three configurations we tested end-to-end, flagging where OBO fits as we go. By the end, you will be able to identify which pattern suits your enterprise scenario.</p> <h2 id="solution-overview">Solution overview</h2> <p><a href="https://docs.aws.amazon.com/bedrock-agentcore/latest/devguide/gateway.html" target="_blank" rel="noopener">Amazon Bedrock AgentCore Gateway</a>, a capability of Amazon Bedrock AgentCore, connects AI agents to external tool providers through MCP server targets. It targets the Snowflake-managed MCP server, which surfaces Cortex Search and Cortex Analyst as MCP tools that an MCP-aware client can discover and invoke. Rather than acting as a wrapper, the Gateway aggregates the capabilities of its configured targets into a single unified catalog, which it presents to the agent as one tools/list response.</p> <p>The core focus of this post is the <a href="https://docs.aws.amazon.com/bedrock-agentcore/latest/devguide/gateway-outbound-auth.html" target="_blank" rel="noopener">outbound authentication</a> for Amazon Bedrock AgentCore Gateway: how it obtains and presents OAuth tokens to the Snowflake-managed MCP server. Amazon Bedrock AgentCore Identity acts as the credential broker: the Gateway obtains a workload token by calling GetWorkloadAccessTokenForJWT (because the Gateway uses JSON Web Token (JWT) inbound authentication), then calls GetResourceOauth2Token against the configured provider. AgentCore Identity owns the token vault, the token refresh, and the user-to-token binding, and it fetches, caches, and refreshes tokens automatically.</p> <p>Amazon Bedrock AgentCore Identity supports three OAuth configurations spanning two grant types (authorization_code and client_credentials):</p> <ol type="1"> <li><a href="https://docs.snowflake.com/en/user-guide/oauth-snowflake-overview" target="_blank" rel="noopener">Snowflake OAuth</a>: Snowflake acts as its own OAuth authorization server. Each user authenticates directly to Snowflake through the Authorization Code grant (3LO). No external identity provider is needed.</li> <li>Three-legged OAuth using a third-party Identity Provider: Users authenticate through corporate SSO with the Authorization Code grant. Snowflake trusts IdP-issued JSON Web Tokens (JWTs) through its <a href="https://docs.snowflake.com/en/user-guide/oauth-ext-overview" target="_blank" rel="noopener">External OAuth</a> security integration.</li> <li>Two-legged OAuth pattern using a third-party IdP: A shared service user authenticates through the client_credentials grant of the IdP. No user interaction or consent is required, which is ideal for machine-to-machine workloads.</li> </ol> <p>Amazon Bedrock AgentCore Identity also offers an API-key credential provider (which supports Snowflake Programmatic Access Tokens). We focus on OAuth here for per-user attribution and automatic token refresh. Each configuration uses a credential provider in Amazon Bedrock AgentCore Identity (CustomOauth2, OktaOauth2, or MicrosoftOauth2) and a corresponding security integration on the Snowflake side.</p> <p><strong>Note:</strong> We tested these patterns with the Snowflake-managed MCP server using Okta and Microsoft Entra ID. Patterns 2 and 3 (external IdP) follow standard OAuth flows, so they can extend to other OAuth-protected MCP servers, but validate the token claims, scopes, and audience settings for your SaaS provider. Only Pattern 1 is Snowflake-specific, because it relies on Snowflake’s built-in OAuth authorization server. The Snowflake-specific elements are the OAUTH_AUTHORIZATION_SERVER issuer-lock parameter, the session:role: / session:scope: grammar, BLOCKED_ROLES_LIST, LOGIN_NAME / EMAIL_ADDRESS mapping, and QUERY_HISTORY attribution. If your SaaS exposes its own internal OAuth server, you configure outbound OAuth the same way as Snowflake OAuth.</p> <table border="1px" width="100%" cellpadding="10px"> <tbody> <tr> <td><strong>Pattern</strong></td> <td><strong>OAuth grant</strong></td> <td><strong>AgentCore Identity credential provider vendor</strong></td> <td><strong>Snowflake security integration</strong></td> <td><strong>Snowflake query identity mapping</strong></td> </tr> <tr> <td>Snowflake OAuth – 3LO</td> <td>authorization_code</td> <td>Custom prov

Nguyên văn nguồn
Chi tiết nguồn
Kiến trúc

Getting AI out of the demo and into businesses globally

Câu hỏi When an invoice and a purchase order disagree, which record wins?

<p><em>AWS Forward Deployed Engineering is now available in 15 countries including U.S., Mexico, Brazil, Canada, Australia, Singapore, India, South Korea, Japan, UK, Germany, Netherlands, France, Sweden, and Saudi Arabia for commercial and public sector customers; and powered by dozens of built in production-ready accelerators.</em></p> <p>Every enterprise I talk to has built something with AI that works. A prototype that reads a claim. An agent that resolves a card dispute. A model that answers a customer question in seconds.</p> <p>The prototype works. The demo impresses stakeholders. Then the organization tries to put it into production, and the real problems begin. Production data is messier than the test set. Auditors want to know why the agent made each decision. Volume grows from a pilot group to the whole company. The rules that matter most turn out to live in people’s heads.</p> <p>None of these are model problems. They’re business problems, and they decide whether AI ever pays off. The stretch from demo to production is the last mile, where most AI stalls. It’s also where our teams spend their time.</p> <p>We launched AWS Forward Deployed Engineering (FDE) in June to close that gap. Since then, we’ve worked through these problems with commercial and public sector customers in financial services, healthcare, sports, media, retail, and more. Here’s what we’ve learned, and how it shaped what we built.</p> <h2 id="the-problem-has-three-layers">The problem has three layers</h2> <p>Model providers, cloud providers, and consulting firms are all embedding engineers with customers. At AWS FDE, we measure success by what remains after our engineers leave: a foundation your team owns, workflows that keep learning, and governance you can trust.</p> <p>Across customers, the hardest problems show up in three layers: the foundation, the workflows, and governance. Each one is harder than it looks, and each one builds on the one before it. That holds whether you’re starting fresh or building on existing systems. On a greenfield build, you can move fast, but there’s no foundation to stand on yet. On a brownfield system, agents must work within decades of data, rules, and integrations.</p> <h2 id="layer-one-a-foundation-that-compounds">Layer one: A foundation that compounds</h2> <p>Before an agent can do useful work at scale, it needs a foundation: identity, security, context, governance, and operational controls. That foundation must be production-ready from day one and shared by every use case that follows. Context is often the biggest gap. Most customers have data spread across systems and no common ontology, the shared model of how the business works that agents need to reason over.</p> <p>Without a shared foundation, the second pilot costs nearly as much as the first. Integrations get rebuilt, and lessons fail to carry forward. Most organizations reach a point where they stop and build the foundation first, so each new use case starts from what the last one built instead of from scratch. We call this the AI reset.</p> <p>Samsung Life Insurance had four affiliates, each building its own AI point solutions from scratch. In 45 days, the FDE team built a governed agentic platform with a single control plane and compliance built in. New agents now deploy through registration and policy assignment instead of a rebuild.</p> <h2 id="layer-two-automating-workflows">Layer two: Automating workflows</h2> <p>Next is the work itself: High-volume, rule-heavy processes such as claims processing, know your customer (KYC) reviews, card disputes, loan origination, invoice reconciliation, permitting, and customer service. This is where AI can deliver immediate and measurable outcomes. The payoff is fast, but the knowledge that drives these processes rarely lives in a system. It lives with the people who do the work.</p> <p>The hard part isn’t automation itself. It is uncovering the business rules behind each decision. When an invoice and a purchase order disagree, which record wins? When can an exception clear on its own, and when does it need a person? Experienced employees answer these questions from judgment, and that judgment is rarely written down.</p> <p>Sometimes the knowledge is buried in the systems themselves.</p> <p>Rio Tinto Iron Ore runs one of the world’s most complex mine-to-ship supply chains on a 30-year-old Oracle system. That system has more than 5,000 database tables and 140 applications. Mapping it had been a multi-month exercise. The FDE team built agents that mapped it in weeks, surfacing the dependencies that Rio Tinto’s teams could validate and build against.</p> <p>Mapping what exists is only the start. The biggest opportunities emerge when organizations stop asking how AI can make existing work more efficient and start asking how it can change how work gets done.</p> <h2 id="layer-three-governance-that-holds-up-as-everything-changes">Layer three: Governance that holds up as everything changes</h2> <p>Governance isn’t a phase of the project. It’s part of the architecture. As agents multiply, so do security risks, costs, and decisions you must explain. Every agent needs its own identity, permissions, and audit trail, separate from human identity. And because the models underneath are probabilistic, agents need monitoring that keeps their results predictable and auditable long after launch.</p> <p>The system also must keep working as everything around it changes. Models drift, business rules shift, data formats change, and regulations tighten. New security issues appear, and code falls out of date. As newer models arrive, the instructions and tooling around them will change too. You need evaluations and guardrails that keep agents predictable where they must be and creative where they can be.</p> <p>When governance comes at the end, it becomes the gate that stops production. When it’s built in from day one, it’s what lets your business move fast.</p> <p>Marsh, a leading global profession

Nguyên văn nguồn
Chi tiết nguồn
Kiến trúc

Co-Designing Sparse Attention for Efficient Long-Context Inference

Agentic workloads, long documents, and large codebases are pushing context lengths from tens of thousands of tokens to hundreds of thousands. At those lengths,... Agentic workloads, long documents, and large codebases are pushing context lengths from tens of thousands of tokens to hundreds of thousands. At those lengths, attention becomes the most expensive part of a transformer layer. Dense attention scores every query against every key, so its cost per query grows with the context length N, and at 128K tokens it can take most of the prefill time in a… Source

Nguyên văn nguồn
Chi tiết nguồn
Kỹ thuật

Put Gemini to work for you

<img src="https://storage.googleapis.com/gweb-uniblog-publish-prod/images/Hero_2784_x_1566.max-600x600.format-webp.webp">Tasks in Gemini is now free of charge in six countries, with new features starting in the U.S. and deeper Google Workspace integration to get things done on your behalf.

Nguyên văn nguồn
Chi tiết nguồn
Kỹ thuật

Integrating QPUs into Supercomputing Centers Using Slurm HRES Plugins

Quantum processors are being integrated into supercomputing centers worldwide. National programs in North America, Europe, and Asia are driving the installation... Quantum processors are being integrated into supercomputing centers worldwide. National programs in North America, Europe, and Asia are driving the installation of quantum processing units (QPUs) alongside high-performance computing (HPC) systems. Every supercomputer will eventually be a hybrid quantum-classical device, drawing on quantum processors to expand what’s possible in scientific research. Source

Nguyên văn nguồn
Chi tiết nguồn
Huấn luyện

Can AI help build a healthier future?

Câu hỏi Microsoft Principal Research Manager Ava Amini , Microsoft Chief Scientific Officer Eric Horvitz , and physician-scientist and Microsoft AI health advisor Eric Topol (opens in new tab) of Scripps Research join Burger to tackle a question with ramifications for every individual on the planet: What might the future of medicine look like with AI, and what needs to happen first?

Technical advancement is moving at such a rapid pace that it can be challenging to define the tomorrow we’re working toward. In The Shape of Things to Come , Microsoft Research leader Doug Burger and experts from across disciplines tease out the thorniest AI issues facing technologists, policymakers, business decisionmakers, and other stakeholders today. The goal: to amplify the shared understanding needed to build a future in which the AI transition is a net positive. Microsoft Principal Research Manager Ava Amini , Microsoft Chief Scientific Officer Eric Horvitz , and physician-scientist and Microsoft AI health advisor Eric Topol (opens in new tab) of Scripps Research join Burger to tackle a question with ramifications for every individual on the planet: What might the future of medicine look like with AI, and what needs to happen first? Drawing from experiences in machine learning, bioscience, and clinical care, the group explores AI-assisted medical diagnosis, the untapped potential of leveraging AI in medical imaging, whole-patient models, virtual cells, and the role of biological foundation models in scientific discovery. The group also examines why promising research can be slow to reach patients, how AI could guide experiments and drug discovery, and why better biological data and real-world validation remain essential. This episode delivers a research-grounded conversation of what AI can do today in the health space, the challenges and opportunities still ahead, and the shape of things to come. The Shape of Things to Come podcast series Subscribe to the Microsoft Research Podcast : Apple Podcasts Email Android Spotify RSS Feed Transcript [MUSIC] DOUG BURGER: This is The Shape of Things to Come, a Microsoft Research Podcast. I’m your host, Doug Burger. In this series, we’re going to venture to the bleeding edge of AI capabilities, dig down into the fundamentals, really try to understand them, and think about how these capabilities are going to change the world—for better and worse. Today, I’m bringing three experts in the fields of biomedicine working on the frontlines of AI, biology, and medicine to see where healthcare is going, where the deep understanding of biology is going, what medical advances are coming, and how will AI change the future of health. These questions are really important because they will determine the shape of things to come. [MUSIC FADES] In today’s podcast, I’ve invited three leading researchers in medicine, biology, and machine learning/AI. Eric Horvitz is Microsoft’s chief scientist [chief scientific officer] and is both a PhD and an MD. How you spent all those years in school, I have no idea. I’ve known Eric for decades and his work has shaped the entire field of machine learning. He’s a legend in the machine learning community and is a good friend. Eric Topol is our second guest. He’s the executive vice president of Scripps Research (opens in new tab) , the largest nonprofit biomedical research institute in the United States. He’s published over 1,300 peer-reviewed papers. Shocking, [LAUGHTER] shockingly productive. Authored four books (opens in new tab) . So just an amazingly impressive, prolific scientist and physician. Finally, Ava Amini is a principal research manager in Microsoft Research. She’s a rising star in our organization, an expert in the intersection of AI and precision medicine, and she’s a co-lead of Project Ex Vivo , a collaborative effort between Microsoft and the Broad Institute that is focusing on defining, engineering, and targeting cell states in cancer. So thank you all for joining today. I think we’ve got a really, really amazing set of people, myself excluded. And I’d like to start to my guests here. It’s clear there’s a huge disruption happening at the intersection of biology and medicine and AI right now. Just, you know, we’re seeing things, breakthroughs, happening at an accelerating pace. Models are helping us to discover structures and create abstractions that are really useful for moving forward the field of medicine. I’ve seen some recent talks where scientists with cancer used AI to design drugs for themselves. I mean, just unbelievable. And today, I’ve got some long-term questions for us to discuss, maybe a little bit provocative. But I’d like to start with what each of you see as what’s happening right now, like, on the, on the intersection of medicine and biology and AI. I think each of you has a unique vantage point. I’ll go around the horn, and I know, Eric, there’s a, you know, you have a cover article on Science magazine (opens in new tab) that’s really exciting, and so maybe you could talk a little bit about that, as well. So I’ll start with you, and then I’ll go to Eric Topol and then Ava. ERIC HORVITZ: Thanks, Doug. I think healthcare and the biosciences are fortunate disciplines as in that there’s been a digitalization of content—whether it be protein structure over, over decades, electronic health records, and so on—that makes these fields amenable to being at the ready for AI inferences and innovation. And therefore, I’ve been extremely optimistic about the long-term futures as well as what we’ll see early on at the frontiers of AI when it comes to the biosciences and healthcare. Just extraordinary to see what’s happened over just the last five to eight years in the biosciences with both discriminative and generative AI models. You know, the AlphaFold (opens in new tab) breakthrough was tip of the iceberg, and we see just a whole, in some ways, a crest and a crashing of the wave of possibilities now when it comes to experimental platforms, experimental design, data interpretation in the biosciences, hypothesis generation, I should say, as a, as an assistant and a companion to scientific reasoners and human scientists. In healthcare, we have an extraordinary opportunity to address long-term challenges in patient safety, in getting to timely diagnoses, in accessibility of healthcare. And after a couple of y

Microsoft Research · Doug Burger, Ava Amini, Eric Horvitz, Dr. Eric Topol Mở bản gốc ↗
Nguyên văn nguồn
Chi tiết nguồn
Kỹ thuật

Running the full server-side Datasette Python app in the browser with service workers and Pyodide

<p><a href="https://lite.datasette.io/">Datasette Lite</a> is my port of the <a href="https://datasette.io/">Datasette</a> Python application to run directly in the browser using WebAssembly and <a href="https://pyodide.org/">Pyodide</a>. I've released a complete rewrite of Datasette Lite using service workers, and it's now almost indistinguishable from the real server-side application.</p> <p>You can now do all of the following, using a carefully crafted URL to <strong>lite.datasette.io</strong>:</p> <ul> <li>Load client-side Datasette to browse any SQLite database, JSON file, CSV file or SQL script</li> <li>Select a released version of Datasette to load from PyPI</li> <li>Load a specific commit, tag, or branch name from the Datasette GitHub repository</li> <li>Pick from over 100 versions of SQLite to use with Datasette</li> <li>Install additional Datasette plugins from PyPI or URLs to wheel files, including JavaScript visualization tools</li> </ul> <p>The ability to load specific Datasette references, plugins, and SQLite versions makes this a fantastic tool for linking to demos of work-in-progress features or bug reproductions, without needing to spin up a full web server to serve that demo.</p> <p>As an example, consider the following URL:</p> <p><a href="https://lite.datasette.io/?install=datasette-maplibre&amp;url=https://gist.githubusercontent.com/simonw/39fa5d399465766039549912e2b3d291/raw/f6dfba99a75cbb8b5d9386223220eb52272767ee/demo.db&amp;metadata=https://gist.githubusercontent.com/simonw/39fa5d399465766039549912e2b3d291/raw/f6dfba99a75cbb8b5d9386223220eb52272767ee/metadata.yml#/demo/capitals_and_countries%23map=0/4.2/180&amp;color=continent">https://lite.datasette.io/?install=datasette-maplibre&amp;url=https://gist.githubusercontent.com/simonw/39fa5d399465766039549912e2b3d291/raw/f6dfba99a75cbb8b5d9386223220eb52272767ee/demo.db&amp;metadata=https://gist.githubusercontent.com/simonw/39fa5d399465766039549912e2b3d291/raw/f6dfba99a75cbb8b5d9386223220eb52272767ee/metadata.yml#/demo/capitals_and_countries%23map=0/4.2/180&amp;color=continent</a></p> <p>Follow that link and, after a short loading screen, you'll find yourself on a page that looks like this:</p> <p><img src="https://static.simonwillison.net/static/cors-allow/2026/datasette-maplibre.webp" alt="Screenshot of a web map interface. Toolbar: &quot;View&quot; set to &quot;Clusters&quot;, &quot;Colour by&quot; set to &quot;continent&quot;, plus &quot;Filter to this area&quot; and &quot;Download GeoJSON&quot; buttons. A world map shows countries outlined and shaded by continent, with coloured dots and numbered cluster circles. A popup for a point in South America reads: capital Caracas, country Venezuela, continent South America, capital_population 2985000, latitude 10.502944, longitude -66.918983. Legend: Africa (52), Asia (45), Europe (39), North America (16), South America (13), Oceania (6). Text: &quot;All 171 rows are on the map&quot;. Below, a table with columns country, continent, capital_population, latitude, longitude, boundary. First row: Afghanistan, Asia, 3277000, 34.518636, 69.181314, with a small map in the boundary column showing Afghanistan outlined and shaded in blue." style="max-width: 100%;" /></p> <p>That's Datasette running the new <a href="https://github.com/datasette/datasette-maplibre">datasette-maplibre</a> plugin, loading up a database of GeoJSON shapes of countries and displaying both a combined map and individual boundary maps in a table.</p> <p>Let's break down that URL to see what it's doing:</p> <pre><code>https://lite.datasette.io/ ?install=datasette-maplibre &amp;url=https://.../demo.db &amp;metadata=https://.../metadata.yml #/demo/capitals_and_countries%23map=0/4.2/180&amp;color=continent </code></pre> <p>The <code>?install=datasette-maplibre</code> installs that additional plugin from PyPI. You can pass <code>&amp;install=...</code> as many times as you like to load more plugins.</p> <p><code>&amp;url=https://.../demo.db</code> loads that SQLite database from the provided URL. These URLs need to have <a href="https://developer.mozilla.org/en-US/docs/Web/HTTP/Guides/CORS">open CORS headers</a> - I host files like this in <a href="https://gist.github.com/">Gists</a>, because they provide an easy way to get open CORS headers on a web-hosted file.</p> <p><code>&amp;metadata=https://.../metadata.yml</code> loads <a href="https://gist.githubusercontent.com/simonw/39fa5d399465766039549912e2b3d291/raw/f6dfba99a75cbb8b5d9386223220eb52272767ee/metadata.yml">this metadata.yml file</a>. The metadata file adds HTML descriptions to various tables, and to the Datasette homepage. Visit <a href="https://lite.datasette.io/?install=datasette-maplibre&amp;url=https://gist.githubusercontent.com/simonw/39fa5d399465766039549912e2b3d291/raw/f6dfba99a75cbb8b5d9386223220eb52272767ee/demo.db&amp;metadata=https://gist.githubusercontent.com/simonw/39fa5d399465766039549912e2b3d291/raw/f6dfba99a75cbb8b5d9386223220eb52272767ee/metadata.yml#/">that homepage here</a> for a guided tour of the other features of that plugin, all defined in the metadata file.</p> <p><code>#/demo/capitals_and_countries%23map=0/4.2/180&amp;color=continent</code> is a fragment hash that turns into a location within Datasette - here it's loading the <code>/demo/capitals_and_countries</code> SQL view, then appending <em>another</em> fragment hash to it (<code>%23</code> is a URL-encoded <code>#</code>) which tells the map plugin to zoom to a certain location and to set the color of the shapes on the map based on the <code>continent</code> column in the data.</p> <p>That URL is doing a lot of work! I wrote more about this trick for building URL-addressable software in <a href="https://simonwillison.net/2025/Feb/13/url-addressable-python/">URL-addressable Pyodide Python environments</a> last year.</p> <p>The full set of supported URL arguments is <a href="https://github.com/simonw/datasette-lite#load-a-different-datasette-version">described in the Dataset

Nguyên văn nguồn
Chi tiết nguồn
Kiến trúc

Cloudflare Joins the Jevolution and Labels Itself a Neolab

Câu hỏi We've kind of entered this Pandora's box now, right?

Cloudflare has been one of the essential infrastructure companies of the web for the past decade and a half. But now, it’s turning into a neolab — at least according to a presentation at AI Engineer Code Summit by Michelle Chen , who runs Cloudflare’s AI Platform on the product side. We’ll get into what exactly a neolab is in a minute (the short version: they build models and they’re not called OpenAI or Anthropic). Cloudflare is claiming to be one because it too has begun to launch models. Two System One models , to be precise: Clef and Clairic. As Chen explained in her talk, Clef is a decision model directly inspired by TypeSafe AI’s revolutionary Jev model , while Clairic is a new type of system one model that Cloudflare calls an “extract” model . In an interview with Latent Space after her presentation, Chen hinted that Cloudflare won’t just be producing system one models. There will be large language models, too. “We're not being a neolab [to] only launch system one models, even though it makes sense for our infrastructure. We've kind of entered this Pandora's box now, right? We opened it, now the possibilities are endless.” Chen said it’s “not just system one models, but potentially reasoning models that we've been experimenting with, and other LLMs and whatnot.” However, she added that these probably won’t be pre-trained models built from scratch, like the frontier labs produce. “I don't think that's our focus, but I think we can take existing resources out there and make [them] a lot more efficient and effective.” She noted that “most of our team really just does post-training,” so that will be the focus in model development. What is a neolab and is Cloudflare really one? The term neolab literally means a ‘new lab’ and it was coined last November by Stephanie Palazzolo from The Information, who defined it as “a new generation of research-focused AI startups.” Deedy Das , a partner at Menlo Ventures and an AI influencer, runs a big list of neolabs . It includes companies like Thinking Machines Lab, Poolside, and TypeSafe AI itself. According to Das, neolabs employ elite researchers, spend a lot of compute (“usually to train its own AI model”), don't have significant revenue traction, and they’re typically startups that have raised a lot of money. Deedy Das’ list of neolabs — note that TypeSafe is among the top 20. Cloudflare doesn’t tick at least half of those boxes, since it’s a public company with significant revenue traction . In fact, earlier this month Cloudflare announced record earnings . “The third quarter was Cloudflare's best quarter ever,” said CEO Matthew Prince . “We crossed $3 billion in annualized revenue with revenue of $782 million, up 39% year over year.” So Cloudflare as a “neolab” is pushing the accepted definition. However, as Chen argued in her presentation, building models is part of a natural progression for the company : from infrastructure, to compute (e.g. the Workers platform, including now for agents), to storage, to serving open weight models like Kimi, GLM and DeepSeek (on Workers AI), to now building its own models. Jev as the “east wind” Chen readily admits that Jev inspired Cloudflare’s pivot to building models. Referencing a Chinese proverb, she framed Jev’s arrival in September as “the east wind” that unexpectedly changed how Cloudflare viewed AI models. “And so when the system one and Jev paradigm started coming on board,” Chen said in her presentation, “we were like, this makes so much sense for us to adopt, right? We don't have to send every request to an LLM. We can send some of it to a System One model.” In our conversation, Chen commented on how it took only two weeks for them to build Clef — which is a copy of Jev — whereas it took TypeSafe AI two years to build Jev. She meant that as a compliment for how hard TypeSafe AI worked to create this new type of model. “All credit to TypeSafe, really, for actually attacking this space and inventing something new here , that people could relate to so strongly and be able to use it. And, yeah, we came up with a model super quickly, but they put in a lot of legwork and we knew exactly what the API schema should look like and whatnot.” Cloudflare is, at its core, still an infrastructure company — speed and low latency is its meat and potatoes. The main benefit of System One models, Chen remarked, is that they allow more work to be done by smaller, faster models , where output is measured in milliseconds rather than seconds. Clairic and what extraction models do While Clef was a clone of Jev, Cloudflare soon came up with an original twist on System One with Clairic , launched last month as “the first model in a new class: System One extraction models.” “What Clairic does is it can take unstructured input, and it can extract, compute, estimate and forecast values from those inputs,” Chen explained in her presentation. In our interview, Chen said that her team wanted to use System One principles to find other areas where LLMs were “overkill” and didn’t always return the right answer. One of the problems they settled on was that LLMs don’t always deal well with unstructured input. Because LLMs are autoregressive, Chen explained, they are prone to hallucinate and they sometimes return malformed JSON. “So we used the same System One principles [as Jev/Clef] there, and decided on extraction.” Some examples of system one use cases at Cloudflare. She used the example of an invoice, where an LLM will sometimes get an important detail wrong — like the order number. “When you send it [the order] to an LLM, they're like, wait, I think the order number's three, maybe it's four, maybe it's five. But it's like, the number's right there, you just have to read it! ” So the idea with Clairic is that it will extract a typed value — like a number — from unstructured data, in one forward pass. Why Cloudflare is in the lab now Using the label “neolab” sounds a bit like a marketing gimmick, but it also makes a ton of sense

Nguyên văn nguồn
Chi tiết nguồn
Kỹ thuật

Serving DeepSeek-V4.1-Flash at Scale: vLLM Matches DeepInfer and Scales to GB300 NVL72

With P/D disaggregation and Wide-EP on a GB300 NVL72 rack, vLLM serves DeepSeek-V4.1-Flash on SemiAnalysis AgentX at 954K tokens/s per GPU at a P90 of 196 tokens/s per user, 7.9x our aggregated single-node result.

  • 9x our aggregated single-node result.
Nguyên văn nguồn
Chi tiết nguồn
261–280 / 562
Trang 14 / 29