AllyshipContent Radar

Deep dives 738

Paper và bài viết kỹ thuật

Crawl live-1795068900000 · 13:22 19 thg 11, 2026 · Xuất bản 13:30 19 thg 11, 2026

Kỹ thuật

How employment is changing at Anthropic

Câu hỏi How has AI restructured work and employment at Anthropic?

How has AI restructured work and employment at Anthropic? As a company at the frontier of AI adoption, Anthropic could be a leading indicator of how AI will shift work organization, hiring, and labor demand across the economy.

Anthropic Research (unofficial RSS mirror) · Chưa ghi tác giả Mở bản gốc ↗
Nguyên văn nguồn
Chi tiết nguồn
Kỹ thuật

Working at the frontier shapes expectations for the future

We’re sharing two reports: the first covers how AI is changing employment inside Anthropic, and the second looks at the gap between how the public and Anthropic employees expect AI to affect work and the economy.

Anthropic Research (unofficial RSS mirror) · Chưa ghi tác giả Mở bản gốc ↗
Nguyên văn nguồn
Chi tiết nguồn
Đánh giá

ASML Is Building a Hybrid Bonding Tool!

ASML has not announced anything yet, but its patent filings, a news report and its own roadmap all suggest so. We believe it will start with a die-to-wafer (D2W) hybrid bonding (HB) tool. This is harder to build than wafer-to-wafer but is where ASML could gain an edge. We see the move as a natural extension of ASML’s vertical integration, and one that should strengthen its customer ties. In this article we unpack the tool’s design end to end, starting with patent filings. What makes it unique is that it runs on a track . That changes how the tool and the dies move: each die rides its own holder past the process stations, and the tools get closer at each station to process the dies. The track’s independent shuttles also lift station utilization, save on duplicate tooling, and enable heterogeneous integration. We believe ASML could gain an edge in alignment accuracy and particle control for Hybrid Bonding. These are the two factors that set bonding yield. And ASML has expertise in solving the same problems in lithography. Despite the advantages, ASML still needs to solve challenges around feed-forward metrology, the die-to-wafer contact, the modification of the non-bonding tools so they can process dies on the track, and keeping the whole loop clean. We expect advanced logic, where D2W already runs in volume, to adopt ASML’s D2W HB tool first. This is bad news for Besi . Logic accounts for most of Besi’s HB business today concentrated at TSMC and Intel. ASML has been working with both for more than two decades and just announced a joint photomask initiative in September. We won’t be surprised to see a similar move in the back end process. Besi’s HBM prospects are also weakening. Korean challengers are catching up, in addition to an HBM adoption that we expect to slip to 2029 or beyond. ASML is Building a Hybrid Bonder We did a deep dive on hybrid bonding in 2024, and we still see it as the most transformative innovation in semiconductor manufacturing since EUV. Wafer-to-wafer (W2W) hybrid bonding (HB) is already in volume in image sensors and 3D NAND, and DRAM makers will be next. Die-to-wafer (D2W) HB is moving slower , because the industry still lacks mature tools that combine satisfying alignment accuracy and throughput (our check indicates upper bound in current applications is only one wafer per hour), which keeps cost per die high. Today D2W runs in volume only in logic: AMD has shipped hybrid-bonded V-Cache since 2022, and TSMC’s SoIC bonds at about 6um pitch today and targets 4.5um around 2029. The market expected HBM to become the big D2W volume driver from 2028. However, our view is different: we believe HBM will not adopt D2W HB before 2029 , if at all. Besi has led D2W and is the only supplier with tools in volume production. Everyone else is still trying to get a first tool qualified. But we believe the situation will soon change, because ASML is building a D2W hybrid bonding tool. Management hasn’t announced anything yet. But the company filed 27 bonding patent families between December 2022 and November 2024, most of them D2W, and the true count is likely higher because filings made after November 2024 are not yet public. The Elec also reported in March that ASML had started the architecture design of a hybrid bonder with Prodrive Technologies and VDL ETG. In the anchor filing ( WO2025201788A1 ), ASML drew how its bonding flow works. Interestingly, the system looks nothing like a traditional D2W bonder (see Advanced Packaging Part 5 for how those work). Instead of a bond head picking each die off a diced wafer and placing it, ASML moves single dies on independent holders around a track (Fig 1). Each die passes inspection, cleaning, plasma activation and metrology (topography, leveling and alignment) stations before it reaches the bonding station where the chuck lifts the bottom wafer up to meet the die. We unpack the design in the sections below. Fig 1: ASML’s track-based D2W bonder, SA annotated and simplified Source: ASML; SemiAnalysis Wafer Fabrication Equipment Model ASML could be building a W2W hybrid bonding tool as well. According to a patent analysis by Inha University , ASML is likely building W2W HB equipment on its TWINSCAN platform, leaning on its wafer overlay accuracy. We focus on the D2W machine in this note, because it is the harder tool to build and the one where ASML could gain a real edge. …as Part of its Vertical Integration Initiative We believe this is a continuation of ASML’s effort to integrate vertically and tighten customer relationships. ASML bought Cymer for its light source in 2013, HMI for e-beam inspection in 2016 and Berliner Glas for optics modules in 2020. It shipped its first back-end product, the XT:260 i-line packaging scanner, in 3Q25. And most recently, ASML announced joint initiatives with TSMC , Intel Foundry and Samsung to move the industry from 6-inch to 12-inch photomasks for High NA EUV. Building a bonder would be a natural next step. In fact, ASML’s own roadmap already puts bonding inside the front-end flow. Its 2024 Investor Day listed W2W and D2W hybrid bonding for NAND, DRAM and logic (Fig 2). And a July 2024 filing shows how the bonder and the scanner would work together by sharing die position data. Today a scanner printing on a bonded wafer must pre-align and coarse-align every die before it can expose. In ASML’s design the bonder already knows where it put each die and hands that map (a “coordinate system” in the filing) to the scanner, which checks a few marks and exposes. Customers get faster lithography on bonded wafers, and ASML gets to sell the bonder and the scanner as a pair. Fig 2: ASML’s map of front-end 3D integration Source: ASML Investor Day, 14-Nov-2024 We expect advanced logic to be the first adopter. Logic is where D2W HB already runs in volume, and where the alignment requirement keeps tightening. TSMC calls overlay “always the biggest challenge” for SoIC. As its HB pitch moves from 6um to 4.5um by 2029, th

Nguyên văn nguồn
Chi tiết nguồn
Kiến trúc

Release v5.20.0

<h1>Release v5.20.0</h1> <h2>New Model additions</h2> <h3>Muse Spark Open</h3> <a target="_blank" rel="noopener noreferrer" href="https://private-user-images.githubusercontent.com/73884904/674958908-bf34ea27-0edf-4ded-ba6f-905b04096126.png?jwt=eyJ0eXAiOiJKV1QiLCJhbGciOiJIUzI1NiJ9.eyJpc3MiOiJnaXRodWIuY29tIiwiYXVkIjoicmF3LmdpdGh1YnVzZXJjb250ZW50LmNvbSIsImtleSI6ImtleTUiLCJleHAiOjE3OTUwNjk2MTgsIm5iZiI6MTc5NTA2OTMxOCwicGF0aCI6Ii83Mzg4NDkwNC82NzQ5NTg5MDgtYmYzNGVhMjctMGVkZi00ZGVkLWJhNmYtOTA1YjA0MDk2MTI2LnBuZz9YLUFtei1BbGdvcml0aG09QVdTNC1ITUFDLVNIQTI1NiZYLUFtei1DcmVkZW50aWFsPUFLSUFWQ09EWUxTQTUzUFFLNFpBJTJGMjAyNjExMTklMkZ1cy1lYXN0LTElMkZzMyUyRmF3czRfcmVxdWVzdCZYLUFtei1EYXRlPTIwMjYxMTE5VDA2MjE1OFomWC1BbXotRXhwaXJlcz0zMDAmWC1BbXotU2lnbmF0dXJlPWI5NWZiMDI5ODdjODMyMmMyMDQ3ZWRmNGRjYzE4NTc0ZjA4YjJkZDFkMzU2OWFjZjk3MGRhOTlmMWUyMjc2NDcmWC1BbXotU2lnbmVkSGVhZGVycz1ob3N0JnJlc3BvbnNlLWNvbnRlbnQtdHlwZT1pbWFnZSUyRnBuZyJ9.XI1jwnzn0bkkZ_RSiJVriCGnYGooy_YXxcp1oeL05Wk"><img width="2000" height="1080" alt="image" src="https://private-user-images.githubusercontent.com/73884904/674958908-bf34ea27-0edf-4ded-ba6f-905b04096126.png?jwt=eyJ0eXAiOiJKV1QiLCJhbGciOiJIUzI1NiJ9.eyJpc3MiOiJnaXRodWIuY29tIiwiYXVkIjoicmF3LmdpdGh1YnVzZXJjb250ZW50LmNvbSIsImtleSI6ImtleTUiLCJleHAiOjE3OTUwNjk2MTgsIm5iZiI6MTc5NTA2OTMxOCwicGF0aCI6Ii83Mzg4NDkwNC82NzQ5NTg5MDgtYmYzNGVhMjctMGVkZi00ZGVkLWJhNmYtOTA1YjA0MDk2MTI2LnBuZz9YLUFtei1BbGdvcml0aG09QVdTNC1ITUFDLVNIQTI1NiZYLUFtei1DcmVkZW50aWFsPUFLSUFWQ09EWUxTQTUzUFFLNFpBJTJGMjAyNjExMTklMkZ1cy1lYXN0LTElMkZzMyUyRmF3czRfcmVxdWVzdCZYLUFtei1EYXRlPTIwMjYxMTE5VDA2MjE1OFomWC1BbXotRXhwaXJlcz0zMDAmWC1BbXotU2lnbmF0dXJlPWI5NWZiMDI5ODdjODMyMmMyMDQ3ZWRmNGRjYzE4NTc0ZjA4YjJkZDFkMzU2OWFjZjk3MGRhOTlmMWUyMjc2NDcmWC1BbXotU2lnbmVkSGVhZGVycz1ob3N0JnJlc3BvbnNlLWNvbnRlbnQtdHlwZT1pbWFnZSUyRnBuZyJ9.XI1jwnzn0bkkZ_RSiJVriCGnYGooy_YXxcp1oeL05Wk" content-type-secured-asset="image/png" style="max-width: 100%; height: auto; max-height: 1080px;"></a> <p>Muse Spark Open is a multimodal mixture-of-experts model from Meta that takes text, images, audio, and video as input and generates text. It has 816B total parameters (42B active per token) and a context window of up to 1,048,576 tokens shared across modalities. It is trained for agentic coding tasks such as multi-file editing, debugging, and tool use. Checkpoints are available in BF16 and NVFP4 formats.</p> <p><strong>Links:</strong> <a href="https://huggingface.co/docs/transformers/main/en/model_doc/muse_spark" rel="nofollow">Documentation</a> | <a href="http://hf.co/blog/meta-muse-spark-open" rel="nofollow">Blog Post</a></p> <ul> <li>Add Muse Spark Open (<a class="issue-link js-issue-link" data-error-text="Failed to load title" data-id="5870734437" data-permission-text="Title is private" data-url="https://github.com/huggingface/transformers/issues/49644" data-hovercard-type="pull_request" data-hovercard-url="/huggingface/transformers/pull/49644/hovercard" href="https://github.com/huggingface/transformers/pull/49644">#49644</a>) by <a class="user-mention notranslate" data-hovercard-type="user" data-hovercard-url="/users/guarin/hovercard" data-octo-click="hovercard-link-click" data-octo-dimensions="link_type:self" href="https://github.com/guarin">@guarin</a> in <a href="https://github.com/huggingface/transformers/pull/49644" data-hovercard-type="pull_request" data-hovercard-url="/huggingface/transformers/pull/49644/hovercard">#49644</a></li> </ul> <h3>Ling 3.0</h3> <p>Ling 3.0 Flash is a hybrid Mixture-of-Experts language model from InclusionAI. Its decoder groups five Kimi Delta Attention (KDA) layers with one Multi-head Latent Attention (MLA) layer. The MLA blocks use interleaved rotary embeddings and a head-wise output gate, and the feed-forward blocks combine routed and shared experts. It is natively supported in Transformers as <code>BailingHybridForCausalLM</code>, so it loads without <code>trust_remote_code=True</code>.</p> <p><strong>Links:</strong> <a href="https://huggingface.co/docs/transformers/main/en/model_doc/bailing_hybrid" rel="nofollow">Documentation</a></p> <ul> <li>[Model] Add native Ling 3.0 Flash support (<a class="issue-link js-issue-link" data-error-text="Failed to load title" data-id="5772220508" data-permission-text="Title is private" data-url="https://github.com/huggingface/transformers/issues/49446" data-hovercard-type="pull_request" data-hovercard-url="/huggingface/transformers/pull/49446/hovercard" href="https://github.com/huggingface/transformers/pull/49446">#49446</a>) by <a class="user-mention notranslate" data-hovercard-type="user" data-hovercard-url="/users/icavan/hovercard" data-octo-click="hovercard-link-click" data-octo-dimensions="link_type:self" href="https://github.com/icavan">@icavan</a> in <a href="https://github.com/huggingface/transformers/pull/49446" data-hovercard-type="pull_request" data-hovercard-url="/huggingface/transformers/pull/49446/hovercard">#49446</a></li> </ul> <h3>Kolibri1</h3> <p>Kolibri 1 is Aleph Alpha's mixture-of-experts reasoning model, focused on German and English. Each of its 50 layers routes every token to 6 of 384 experts using sigmoid routing, and adds one shared expert. Four sliding-window attention layers alternate with one full-attention layer, and RoPE is applied only in the sliding-window layers.</p> <p><strong>Links:</strong> <a href="https://huggingface.co/docs/transformers/main/en/model_doc/kolibri1" rel="nofollow">Documentation</a></p> <ul> <li>Add Kolibri 1 (<a class="issue-link js-issue-link" data-error-text="Failed to load title" data-id="5778636610" data-permission-text="Title is private" data-url="https://github.com/huggingface/transformers/issues/49463" data-hovercard-type="pull_request" data-hovercard-url="/huggingface/transformers/pull/49463/hovercard" href="https://github.com/huggingface/transformers/pull/49463">#49463</a>) by <a class="user-mention notranslate" data-hovercard-type="user" data-hovercard-url="/users/lucamotz/hovercard" data-octo-click="hovercard-link-click" data-octo-dimensions="link_ty

Nguyên văn nguồn
Chi tiết nguồn
Kỹ thuật

Introducing Timestral

State-of-the-art probabilistic forecasting in four sizes (5M–209M params), CPU-efficient, zero-shot ready.

Nguyên văn nguồn
Chi tiết nguồn
Kiến trúc

Why I still haven’t bought into true RSI

We’re in an era where a few organizations are using thousands of concurrent agents to improve their processes and output. These organizations happen to be just the frontier AI labs, in particular OpenAI and Anthropic. In the last few weeks, I’ve been pondering what it means for so many employees across these organizations to rapidly update their expectations for the pace of AI progress and associated risks. A core perspective I have is that the frontier labs and broader frenetic, competitive culture in the San Francisco AI scene set up an environment that amplifies any AI concern. This has some benefits in causing more general audience awareness of AI, as fear sells, but exaggerating risk timelines or severity will have negative second-order effects. I remember many loud AI safety debates, and their associated clouds over the viability of open-source AI, in 2023 and 2024 — the primary risks then did not arrive in the forecasted timelines. The general populace of these two key labs was very anxious about AI risks and the rate of progress even a year ago, and especially as agents got stronger product-market fit at the start of 2026. This cultural precondition, when exposed to the reality that thousands of agents will constantly be working fairly productively in your business, will only increase this anxiety. The step from this anxiety, and incidents like OpenAI-HuggingFace, to extinction risks feels very religious. Share Richard Ngo had an apt summary of the situation: Now a large proportion of the AI safety community is implicitly or explicitly orienting to futures where an intelligence explosion occurs within a few years. My default expectation (absent an extensive pause) is that a similar thing will happen: they’ll turn out to be directionally correct (relative to the expectations of almost anyone not linked to the community) but factually wrong. Specifically, we won’t have superintelligence within the next 8 years, but things will still be moving so fast that it’ll *feel* like the people who argued for short timelines were right. … I wanted to say something now because it feels like the level of bandwagoning towards “singularity soon” is getting pretty wild. Personally, I think this view aligns closely to what I outlined in my alternate scenario to true recursive self-improvement (RSI), which I called lossy self-improvement . A summary of this view is that: Automatable research is too narrow to achieve a massive net acceleration in progress, in the face of scaling laws’ exponential costs, Diminishing returns of more AI agents in parallel are real, & Resource bottlenecks and politics are a major factor in building strong LLMs (and AI can do much less to accelerate this). So, I’m left balancing the above, latent increase in the cultural temperature with the potential that the labs have seen genuinely scary, specific breakthroughs that are not public yet. My expectation is that more of the current AI safety concern is on the former – scaled agents working – but I hold high levels of uncertainty here. Foundational, imagination-based AI breakthroughs are the sort of thing that would make me update my RSI timelines from closer to a tool to sustain progress in the face of exponential costs (scaling laws), to something more unpredictable and/or unstable. Interconnects AI is a reader-supported publication. Consider becoming a subscriber. Some of the best recent resources on RSI have been Dwarkesh’s podcasts with Noam Brown and the trio of John Schulman , Beren Millidge and Charlie O’Neill . I have a few important reflections from both of them. First, the podcast with Noam Brown made me internalize how big of a short-term acceleration mass inference capacity is. These labs will throw thousands of agents at important, measurable problems. At the same time, compute capacity available to them is going to continue to scale. I have my doubts that the labs can afford to spend a constant portion of this compute on internal R&D as the total volume goes up, especially with plans to IPO, as they face increased scrutiny on basic economics. It is important to not confuse massive steps in inference-time scaling, a dynamic which should be fairly predictable, with being the outputs of RSI, which is highly uncertain. Second, the trio podcast debating the state of the art in technical capacities induced more of a surprising reaction that I haven’t fully settled. Through the first hour or so of this podcast, where they debate the role of RL, distillation, scaling, inference-time compute, etc., I found myself strongly agreeing with the distribution of claims. A TLDR would be that our current techniques work and let us solve problems we know how to state, but they don’t result in a magical level of generalization to unknown, harder problems in most partially verifiable domains (i.e. progress in math is an exception, rather than a rule). The surprise of this podcast was the end, where they were predicting timelines for various thresholds of AI. I had GPT-6-Astra summarize the answers provided to three questions from Dwarkesh, of the form “when will AI reach X ability”: All timelines are relative to the interview date. Drop-in remote worker for broad white-collar work over a month Charlie O’Neill: ~1 year with programmatic access to workplace tools; ~2 years if it must operate through a browser. Means ordinary white-collar work, not highly creative research. Beren Millidge: ~3 years for full generality; 80–90% coverage sooner. Main uncertainties: online learning and the long tail of tasks. John Schulman: ~1 year for an “okay” version, with uneven capabilities that improve over time. 10× productivity uplift for AI researchers Charlie O’Neill: 5–10 years. Bottleneck: absorbing information and deciding which experiment to run next. Beren Millidge: Finds John’s ~2-year estimate plausible, but gives no independent timeline. Assumes AI can run successive experiments and learn from feedback; other bottlenecks would remain.

  • We’re in an era where a few organizations are using thousands of concurrent agents to improve their processes and output.
  • Personally, I think this view aligns closely to what I outlined in my alternate scenario to true recursive self-improvement (RSI), which I called lossy self-improvement .
  • A summary of this view is that: Automatable research is too narrow to achieve a massive net acceleration in progress, in the face of scaling laws’ exponential costs, Diminishing returns of more AI agents in parallel are real, & Resource bottlenecks and politics are a major factor in building strong LLMs (and AI can do much less to accelerate this).
Nguyên văn nguồn
Chi tiết nguồn
Đánh giá

An update on independent evaluators

To evaluate the safety of our systems, Anthropic is giving employee-like access to a team from METR, a nonprofit that evaluates frontier AI models, and we're in early discussions with several other organizations, including Hugging Face.

Nguyên văn nguồn
Chi tiết nguồn
Kỹ thuật

How Hugging Face’s Investor Spots Winners

Check out Gemini Audio: https://goo.gle/Every-Gemini-Audio Bet on technical founders, because their first idea probably won’t be the one that sticks. Their technical skills let them build for a new market when their original business changes direction. That’s the view of Matt Hartman, founder and managing partner of Factorial Capital. Hartman was part of the Betaworks team that wrote the first check into Hugging Face in 2016, when it was a chatbot app for teenagers. Today, it’s the largest community for open-source AI and is set to be acquired by Nvidia for about $13 billion. On this week’s episode of The Every Podcast, Hartman tells Natalia Quintero how his investment thesis has evolved over time, how his portfolio founders became one of his best sources of new deals, and why vibe coding his own app gave him a new respect for SaaS. If you found this episode interesting, please like, subscribe, comment, and share. Start your free trial today: https://every.to/subscribe?utm_source=youtube 0:00 Introduction 5:06 How Hugging Face went from a teen chatbot to the home of open-source AI 9:52 The three decisions that set Hugging Face on its path 12:49 Hartman's approach to finding the newly technically possible 18:10 Why Hartman backs technical founders 21:10 How Factorial’s portfolio founders source its next deals 27:09 Why faster iteration makes product insight the most valuable skill 34:28 What Hartman learned building Ghost Pepper, an open-source voice-to-text app 40:02 Hartman's current theses, from non-language models to full-stack AI apps 44:14 Why the best founders right now are experimenting Links to resources mentioned in the episode: Factorial Capital: https://factorialcap.com Ghost Pepper, Matt's open-source voice-to-text app: https://github.com/matthartman/ghost-pepper Superpowers, Jesse Vincent's skill for Claude Code: https://github.com/obra/superpowers Hugging Face: https://huggingface.co Follow Every: https://x.com/every Follow Natalia Quintero: https://x.com/NataliaZarina Follow Matt Hartman: https://x.com/MattHartman

Nguyên văn nguồn
Chi tiết nguồn
Kỹ thuật

The Overhang

We are still on an exponential curve of AI development. I try to put out a Substack post every couple weeks or so, yet, as the pace speeds up, that sometimes feels too slow. In the weeks since my last post, we had the apparent cracking of one of the most famous problems in math by an AI (accompanied by controversy) and widespread discussions about the risks posed by AI and what to do about it (also accompanied by controversy). I think these concerns, along with a mounting set of other worries, come down to the same problem I have with my posts: how slowly our very human systems and processes work to keep up with the pace of AI development. I don't think the people worried about this are wrong, but I also think a sole focus on future AIs, as important as that is, ignores the fact that AI, right now, is already incredibly capable. In fact, the new GPT-6 Astra and Fable 5.1 are already enough for transformative impact in large sections of the economy and they can reliably do weeks worth of human work when properly guided and harnessed. A few fun examples of that: I had GPT-6 Astra turn a 1977 text adventure game called Zork into a full 3D action-adventure game you can play . Zork has no graphics and each location is a paragraph of prose, so the AI had to decide what the white house looks like, what a grue looks like (the original only tells you that you are likely to be eaten by one in the dark), and how to turn “fight the troll” into an action sequence. I also had Fable 5.1 try to reconstruct Italian author Umberto Eco’s library in 3D . Eco kept tens of thousands of books in his Milan apartment and the AI could not find a floor plan, so it instead decided to work from a dozen videos, the foundation's photographs of each bookcase, and two library catalogues. It read spines frame by frame, inferred the rooms, and placed the 5,000 or so books it could identify among 27,000 shelf slots. It marked every book certain, guess, or unknown, and drew the bookcases the cameras never reached in fog. This task, like the Zork game and a lot of real-world work I have had the AI do recently, would have taken weeks of human work involving researchers, coders, and designers. But here we are. My point is that, while there is a lot of debate over what future models will do, the current capabilities of existing models are barely being used, and are often not even well understood. For example, I did not know GPT-6 Astra could operate Blender (a sophisticated piece of 3D modelling software) until it did. I gave it a copy of my upcoming book, Co-Existence , and asked it to create a trailer for the book from the perspective of an AI. Without clear instructions from me, it proceeded to use Blender and build out an entire animated 3D scene (not an easy task), along with a script with some jokes and reveals (I did reject the first joke it added, but the second was quite good). It then figured out how to generate voices and music and sound effects and gave me this film 45 minutes later. The final product feels a little more ominous than I would like, but that was the AI’s decision, not mine. To see how much further it could go, I prompted: “That’s good, but I actually want you to make an action movie trailer based on Co-Existence. Have fun with it. No more than 30 seconds.” Again, it wrote a script and made a 3D prototype in Blender. After I asked for a more cinematic version, it used the Blender animation as a storyboard, operated a video generator through my browser, and edited the generated shots into the final trailer. I gave some minor creative feedback, but never touched any production decision or even knew exactly how it was accomplishing its tasks. You can see the results here. There are plenty of flaws in these efforts that you can spot. But they are also examples of the AI exercising a kind of judgement and creativity, things that not long ago were considered uniquely human traits. And they were all done with just a fraction of the token budget of the ChatGPT account I pay for. I think these are fun demonstrations, but they are also a bit scary because AI is getting better at things that were once purely human. Still, none of these projects happened on their own. I chose them, I knew enough about Zork and Eco and my own book to see where the AI went wrong, and to ask for a second version when the first wasn't right. The capability overhang, the gap between what these models can do and what almost anyone is doing with them, is an opportunity because most people don't bring their own advantages to AI, and those who do get much more out of it. That is why I think we will need to focus on the individual traits we have that remain useful even as AI abilities improve. You are not trying to compete with AI in producing outputs, that is a losing game. Instead, you want to use your human advantages as basis of working with AI to do things that neither of you could do alone. In my book, I outline four particular personal advantages that matter a lot if you want to use AI in unique and enhancing ways: deep knowledge, wide knowledge, taste, and agency. The Four Advantages The first two advantages come from what you know. Deep knowledge is the expertise that comes from understanding a field or subject so well that you build intuition around it to quickly and accurately make decisions. It is how an experienced accountant can glance at a spreadsheet and know something is wrong, or how a golf pro can watch a swing and instantly understand the mistake the golfer is making. It is also why I could tell within seconds that the first trailer was more ominous than the book actually is. Deep knowledge is the realm of the specialist, and it is the only way to truly understand the shape of the Jagged Frontier, because only experts can understand the patterns of where AI succeeds or fails, at least in their area of expertise. It also helps you adapt to change because deep knowledge makes it easier to switch from being someone who does

  • You can see the results here.
  • That is why I think we will need to focus on the individual traits we have that remain useful even as AI abilities improve.
Nguyên văn nguồn
Chi tiết nguồn
Đánh giá

Engrams Embedding Entendre: Codesign for Efficient DRAM/SSD Offloading

Câu hỏi Source: SemiAnalysis What is Being Memorized?

Engram extends standard token embeddings with learned multi-token lookups. Recurring local patterns retrieve vectors directly, reducing the need to reconstruct them through attention and feed-forward layers. With Engram model architecture optimization, it allows for lower HBM capacity to be needed for models at the same quality. This does not mean there won’t be an insane demand for HBM but it just means that model architecture will continue to innovate around constraints. This model architecture design is naturally codesigned for parameter offloading: each token accesses a few embedding rows whose addresses depend on token IDs, not hidden states. The runtime can prefetch those rows from host DRAM while earlier layers compute, keeping the table outside HBM without transferring entire weight matrices. Our Memory model contains our latest estimates of quarter by quarter HBM, DRAM, & NAND supply and demand. Source: SemiAnalysis Offloading frees HBM for model weights and KV cache, potentially supporting larger batches or more concurrent sessions. When DRAM becomes the next constraint, NVMe offers another tier. Recommendation systems already cache frequently or recently accessed embedding rows in faster memory while backing colder rows with SSDs. After NVIDIA roadmap had to change due to massively despec’ing Rubin Ultra from 1024GB to now ~200GB of HBM per chip , model architecture optimizations like emgram maybe helpful. Our DeepSeek-V4.1-Flash configuration uses roughly 189 GiB of memory for Engram. We replace it with a memory-mapped (mmap) file and measure serving performance with offload to SSD. Later on in our report, we will show our Engram offloading experiments along with the official InferenceX agentic inference serving results on Engram models like DeepSeekv4.1 Flash across all 6 NVIDIA GPU SKUs along with MI355X. Unsurprisingly, the CUDA Moat is still mogging MI355X on the ultra popular DeepSeekV4.1 Flash model. We also show how even on high capacity HBM SKUs, offloading emgrams to DRAM could result in even better performance for most of the pareto than keeping the emgram in HBM. Our benchmark has been widely reproduced, validated and/or supported by almost every major buyer of compute from Google Cloud to Microsoft Azure to Oracle, to Meta and many more. Furthermore, it has the support of the ML community including from vLLM, LMCache, SGLang, PyTorch, Huggingface and the support of major labs like OpenAI, MiniMax, ZAI, Qwen, Moonshot Kimi, etc. Source: InferenceX Star the InferenceX GitHub repository if you find the open-source benchmark and data useful! . InferenceX is the only inference benchmark in the world to have TPUv7, Jalapeño, Nvidia Rubin NVL72, AMD, and soon, SambaNova and Trainium. Due to how realistic AgentX scenario is to real world agentic inference workloads, AMD has committed to collaborating on MI455X UALoE72 too. Source: SemiAnalysis Engram Benefits DeepSeek did not release the original paper’s two trained Engram models. We replicated its setup on fineweb-edu using the released code and training hyperparameters, at an estimated 6E18 FLOPs per run. We observed the same U-shape scaling: Source: SemiAnalysis Engram improved performance over pure MoE baselines. We also reproduced DeepSeek’s results, where earlier-layer representations with Engram resembled those of later layers. Source: SemiAnalysis What is Being Memorized? Like the original Engram paper, it is possible to probe Engram’s gate scores to see what n-grams DeepSeek-V4.1-Flash makes the most use of. Our gate scan finds names, code fragments, relational phrasing, and boilerplate. These examples prioritize interesting-ness over gate strength. One unexpected result was Wright : Ace Attorney . Source: SemiAnalysis These examples suggest learned memory optimizes the training objective, not a judgment of which facts deserve storage. Licenses, bibliography fragments, API scaffolding, and website furniture can provide prediction shortcuts, so the value of additional Engram capacity may depend on what survives data preparation. This does not show that table capacity is “wasted”: the evaluation-corpus scan establishes neither training exposure nor the capacity occupied by each category. For offloading, strong gates do not identify cache-hot rows. Low gates do not automatically save reads either: computing the gate requires the retrieved key and undoes the performance gain of a fused kernel. Skipping reads would require a separate usefulness predictor before retrieval. Removing Engram In the original paper’s inference-time ablation , factual-knowledge benchmarks retained just 29–44% of their original performance, while reading comprehension retained 81–93%. This is due to the training–inference mismatch. The resulting degradation therefore measures this trained model’s dependence on Engram, not the performance difference between models trained with and without it. Source: DeepSeek In our ablations, suppressing Engram worsens token likelihood across all evaluated domains, especially encyclopedia text and several code corpora. Surprisingly, GSM8K accuracy stays within measured run-to-run variation and removing Engram has no effect. Engram is not a detachable dictionary beside an unchanged MoE. Removing it changes downstream features and expert selection. We tested whether rerouting hurts or compensates by holding tokens fixed in a teacher-forced experiment on CRUXEval, a code-reasoning benchmark of small Python functions where the model predicts a function’s output from its code and an input, and scoring the reference answer. Source: CRUXEval Removing Engram raised answer loss from 0.2848 to 0.3093 bits/token. Forcing the ablated model to use the original Engram-on expert choices made it worse still, at 0.3375 bits/token. Source: SemiAnalysis Rerouting partially compensates for the missing memory. Memory features and expert selection work together, rather than following a clean “memory stores facts; experts reason” d

  • Recurring local patterns retrieve vectors directly, reducing the need to reconstruct them through attention and feed-forward layers.
  • We also show how even on high capacity HBM SKUs, offloading emgrams to DRAM could result in even better performance for most of the pareto than keeping the emgram in HBM.
  • We observed the same U-shape scaling: Source: SemiAnalysis Engram improved performance over pure MoE baselines.
Nguyên văn nguồn
Chi tiết nguồn
Kỹ thuật

Google DeepMind and Singapore Government collaborate to build fine-tuned AI model for scam detection

Learn how Google DeepMind and GovTech Singapore uses lightweight on-device Gemma models to intercept conversational scams in private chats with zero cloud data egress.

Nguyên văn nguồn
Chi tiết nguồn
Kỹ thuật

Adaptive Verification Without a Confidence Head

An online-fitted acceptance estimator that lets every draft-model speculator use Adaptive Verification, by predicting the target's acceptance probability from the drafter's own logits and refitting continuously during serving.

Nguyên văn nguồn
Chi tiết nguồn
601–620 / 738
Trang 31 / 37