AllyshipContent Radar

Hands-on 193

Công cụ, thử nghiệm và hướng dẫn thực hành

Crawl live-1795068900000 · 13:22 19 thg 11, 2026 · Xuất bản 13:29 19 thg 11, 2026

43

My son loves the VR game Gorilla Tag, and this weekend he and I learned how to make and import new maps (with Codex doing the heavy lifting in Unity), and now he's playing maps that he has made

54.5

Hướng dẫn johnennis

My son loves the VR game Gorilla Tag, and this weekend he and I learned how to make and import new maps (with Codex doing the heavy lifting in Unity), and now he's playing maps that he has made Here's a launch video for one of his maps Crazy times! Media

X · 16 thg 11, 2026

Nguyên văn nguồn
Chi tiết nguồn
44

How the Media Went From Rumor to Rumor to Sell Jensen Huang’s “Return” to Nvidia

53.8

Hướng dẫn dream1984tube

How the Media Went From Rumor to Rumor to Sell Jensen Huang’s “Return” to Nvidia Today at 2:03 p.m., the exact same @FT reporter who first told the world Jensen Huang was having an affair, @AsiaLens, dropped another "exclusive": "Jensen Huang is considering a comeback as Nvidia's chief executive." Notice the victory lap, right there in the second line: "tech executive resigned over reports he had affairs with a suspected Chinese spy and a subordinate." The only fact on the record: Huang admitted to an affair. No name. No spy. This time, the reporter buried his own byline, last of three. Maybe hoping we'd forget what he already admitted on X: he never saw any U.S. intelligence. He just believed that U.S. intelligence believed a Chinese spy story. Belief built on belief built on belief. Ten days later, over a spy case the DOJ, the DNI and the CIA still can't even confirm, he somehow found brand-new sources to believe in: "Huang has discussed with confidants the possibility of returning, according to multiple people familiar with the matter." Say it twice: According to multiple people familiar with the matter. Then comes the thriller script: "One Huang ally told the FT they were putting together a 'war room' of Silicon Valley executives to lobby for the former Nvidia chief's return and would try to garner support from the White House." Read those words twice: Huang ally. Huang ally. "War room." "War room." Garner support from the White House. Wait a second, I thought this wasn't about Jensen at all. t was Napoleon on Elba island, plotting his Hundred Days. Watch the clockwork: 2:03 p.m. FT publishes. 2:10 p.m. @TheInsiderPaper reposts. 2:12 p.m. @zerohedge reposts. 2:14 p.m. @business runs its story. 2:15 p.m. @FT's own X account finally tweets it. Bloomberg had a full article up a minute before the FT tweeted its own scoop. Guess the Bloomberg Terminal really is worth the money. Just another coincidence. The @WSJ was way more patient. They let the "comeback" story run for seven hours. Watched Nasdaq 100 futures jump 200+ points. Then, right before Monday's pre-market open, they dropped their "definitive" piece and slammed the door, starring double-standard @SecScottBessent and Yale's @JeffSonnenfeld (Senior Associate Dean, founder of the Chief Executive Leadership Institute). Turns out the professor gives up his weekends, just like I do, to binge "House of Cards: Silicon Valley Edition." He even comes pre-loaded with headline-ready lines like "tear up the rulebook." I think he's a Joy Division fan, just like me. I'm not falling for the media trap of debating "corporate governance." Look at the calendar. Ever since this broke on November 11, Sonnenfeld has been doing non-stop interviews: The Guardian, the WSJ, everywhere. What a "phenomenal presence" in Wall Street media. Every appearance: zero room for a defense, one-sided against Huang, cranking up the heat every time. Just another coincidence. And right there at the end of that very same FT piece, they quietly slipped in the exit line: "Even allies were unsure how to rehabilitate Huang's reputation, acknowledged one person involved in efforts to reinstate Huang." "One person involved in efforts." Everything is a coincidence.

X · 16 thg 11, 2026

Nguyên văn nguồn
Chi tiết nguồn
46

Understanding Positional Embeddings in Modern LLMs

52.5

Hướng dẫn Sebastian Raschka, PhD

This article explains how LLMs encode and process token position information by covering classic and contemporary methods like RoPE. It’s essentially a comprehensive journey where we will work through the mechanisms with visual examples, a little math, and a small PyTorch implementation. And, of course, we will then also look at how modern architectures adapt or omit explicit positional encodings. Below, you can find a preview of which positional embedding choices are used in 2026 open-weight models. We will discuss these choices later after covering the underlying mechanisms. Figure 1: A preview of the gallery’s 2026 architectures, grouped by positional encoding. Architecture figures taken from my LLM gallery . But why spending so much time writing/reading about this rather minor building block? LLMs keep evolving, and as I was preparing a new article on the latest LLM architecture trends this year, I realized that I never covered the positional information methods used in LLMs in any article so far. So, instead of letting the other article get out of hand, I thought it might make sense to publish this as a standalone building block for the upcoming LLM architecture and trends analysis. Also, over the years, many readers have asked me for a tutorial on relative positional embeddings (aka RoPE) to follow up on the learned absolute embeddings used in GPT. It’s been on my TODO list for a long time, so I finally revisited my old draft and put this walkthrough together, which will cover different positional methods in LLMs; (finally) how RoPE works; what methods are used by recent open LLMs. Note that this article assumes some familiarity with basic LLM architectures and works well as a follow-up to Build a Large Language Model (From Scratch) . If the concepts and computation of the attention internals, particularly query, key, and value vectors, are a little fuzzy, I recommend checking my self-attention walkthrough as a refresher: The article is also very technical and likely too detailed for some. However, I really wanted to have this foundation out there for future articles to reference. And no worries, the next one will return to a more comfortable style. PS: I started experimenting with Substack’s new “perks” system. As a thank-you to supporters, I’ve added image exports and metadata from the architecture gallery for you to use in your own analyses. PPS: If you are already familiar with how position embeddings work, feel free to fast-forward to Section 6, Positional choices in modern architectures . Note that this article may be best read in a web browser than an email client. I recommend checking out the web view here , which also has a navigation bar on the left side. 1. Where position information enters an LLM Let’s start with the overall architecture. A tokenizer encodes the input text into token IDs. Next, the input token embedding layer of an LLM maps each of these IDs to a vector of numbers. We usually refer to these entries in that vector as “features” (similar to classic ML). The transformer blocks inside the LLM then modify these vectors using attention and feedforward layers. Lastly, the model’s output layer produces logit scores (converted into probabilities) for the next token. The following shows this path in GPT-2 (a relatively classic LLM) and MiniMax-M3 (a more recent LLM released this year; a placeholder for most other recent LLMs). As we can see, the architectures are not too different from each other, but before we get to the positional information parts, let’s focus on the orange blocks for now (the token embedding layers) and the transformer blocks above them; the other architecture differences aren’t needed for this explanation. Figure 2: Token embedding layers in GPT-2 and MiniMax-M3, highlighted in orange. (Side note: M3’s first three out of 60 feedforward layers are dense not MoEs.) A token embedding layer, shown in the figure above, maps each token ID to a vector. Position information is then added afterward. Concretely, and as we will discuss in more detail later, GPT-2 adds a position vector to each token embedding before the modified embedding enters the first transformer block. On the other hand, MiniMax-M3 adds the positional information in each transformer block using a method called RoPE inside attention. We will talk about these methods in more detail later, but first, let’s start with the big picture information explaining why explicit positional information is added or needed. 1.1 Why add positional information? Consider the following two input sentences as a simple example: “The red fox jumps over the gray fox.” “The gray fox jumps over the red fox.” Changing the word order changes which fox does the jumping, as illustrated in the following figure. Figure 3: Changing the word order changes which fox jumps. For the examples in this article, we will treat each word as one token for simplicity. (As you know, real tokenizers can split words into smaller subword tokens.) The first sentence then has eight token positions, from 0 through 7. Figure 4: Token IDs and positions in the toy sentence. # Assume the following small vocabulary. # Real vocabularies typically have 50k to 200k entries vocab = { "The": 0, "red": 1, "fox": 2, "jumps": 3, "gray": 4, "over": 5, "the": 6, } (If you are interested in how vocabularies are built, check out my tokenizer materials on GitHub .) Note that “fox”, which appears twice in the input sentence, gets the same token ID 2 in both cases. And since they both have the same token ID, irrespective of the position in the input sentence, they would get the same embedding vector from the embedding layer. (For those who are interested in more details on how this selection from the embedding table looks like, check Appendix A .) Now, let’s consider a regular self-attention mechanism with no positional encoding (and no causal mask), where every input token can attend to every other token in the input. In this scenario, swapping any tw

Ahead of AI · 15 thg 11, 2026

Nguyên văn nguồn
Chi tiết nguồn
47

AI models are inevitably going to keep getting better and better at all coding, tool and computer use, math, and other coding-related tasks. As a result, agents that can use software to do their work,

51.1

Hướng dẫn levie

AI models are inevitably going to keep getting better and better at all coding, tool and computer use, math, and other coding-related tasks. As a result, agents that can use software to do their work, or generate code from scratch, will improve at a faster rate than all other domains of knowledge work. On one hand that could mean that only a small subset of work can get automated. But probably what happens is that more and more work just becomes coding-shaped to adapt to what agents are good at. You can see it in the most recent examples of model releases in the past couple of months, where agents can generate code that becomes animated videos, games, 3D models for products, and so on. We also know that tool use gives agents the ability to work with much of the same software that people already work with. This helps in cyber, life sciences, finance, and a number of other domains. Our mental model traditionally has been that the models have to be able to know everything about anything to generate value, but in reality they *really* have to know how to use or generate software effectively combined with deep domain expertise to make the right judgment calls inside of workflows. When you have that powerful combination, almost everything else gets unlocked. This is why the acceleration of agents in knowledge work is only going to increase from here.

X · 15 thg 11, 2026

Nguyên văn nguồn
Chi tiết nguồn
48

Here's the full guide: how to check any benchmark in 5 minutes, and how to run an eval on your own documents - https://www.datalab.to/blog/how-to-evaluate-ocr .

49.2

Hướng dẫn vikparuchuri

Here's the full guide: how to check any benchmark in 5 minutes, and how to run an eval on your own documents - https://www.datalab.to/blog/how-to-evaluate-ocr .

X · 14 thg 11, 2026

Nguyên văn nguồn
Chi tiết nguồn
43–48 / 106
Trang 8 / 18