AllyshipContent Radar

Deep dives 738

Paper và bài viết kỹ thuật

Crawl live-1795068900000 · 13:22 19 thg 11, 2026 · Xuất bản 13:30 19 thg 11, 2026

Kỹ thuật

Show HN: Can a VPN be decentralised.. I think it's the best for homelabs

Câu hỏi it's free and open source (AGPL), written in Rust It really felt useful to me so I just want to know what would make it further helpful?

Can a VPN be decentralised... I was trying out tailscale for my homelab but I didn't really want a control plane. Tailscale, Headscale and others already do the Controlled VPN extremely well, and I've used them. but every one of them has a control plane that all the devices directly connect to. I wanted something where every device in the network can act as a self node, and it'll be a Mesh of devices interconnected. Further it also allowed me to keep internal names like nas.home.kith.internal, and exit nodes, relays... it's free and open source (AGPL), written in Rust It really felt useful to me so I just want to know what would make it further helpful? ... Some use cases or if there's a problem it could actually help with, I'd love to hear it and try my best. ps : I used a lot of Claude while building it, so I'm sure there are rough edges I haven't found... bad code, wrong docs, or maybe the wrong approach somewhere. Would love any feedback on it, or anything I should do differently :D

Nguyên văn nguồn
Chi tiết nguồn
Kỹ thuật

Show HN: Director's Lookbook, named looks for image, video and music prompts

River Road Labs is a small artisanal studio based out of Central NJ. We built this tool to scratch our own itch. As AI video and image capabilities get more sophisticated we found that we struggled at times to find the words to describe the aesthetic style we were shooting (no pun) for. Other times it is good to just have a massive resource of styles and imagery to flip through at random and to let serendipity do the rest. Director's Lookbook lets you type a feeling and get dealt a set of style cards with prompt text to use. No more copying or saving prompts, use the tool. The feeling reader is a small hosted model with a free daily allowance. The MCP server is read only, no auth, no key, it works in Claude as a connector and in Cursor as well. Setup is at directorslookbook.app/mcp. The connector keeps no request text, only daily counts of which tools were called. The tool is free, no login, no account. We welcome feedback on how to make the tool better and what looks are missing! Thanks!

HN: Claude · RiverRoadLabs 4 lượt bình chọn 2 bình luận Mở bản gốc ↗
Nguyên văn nguồn
Chi tiết nguồn
Tác tử

Show HN: Faros Desktop, a local app to see your AI traces and extend your usage

Hi HN, I’m Sachin, and I’ve spent the last couple of months working on Faros Desktop, a Mac app that visualizes how you use your local AI coding tools into one place and then makes recommendations to make your subscription spend go further. As agents have replaced hands-on-keyboard work, the team and I wanted to better understand what’s in our session traces. Faros Desktop hydrates with on-device session history in Codex, Claude Code, Github Copilot, and Cursor to let users deep dive into a beautiful visualization of AI session traces. Then based on consolidating that information, the app shows patterns in usage as well as recommendations for ways to save spend through simple changes that we call Power Ups. Faros Desktop is totally free, with no login required and no API keys needed. The app also only uses on-device models to summarize local sessions in chunks, then organizes them by project and task. You can jump around a session timeline and see exactly the messages and tool calls in the trace. The business model here for Faros Desktop is to drive thought leadership and create leads for our sales team to sell the company’s multi-player Token Engineering Platform for large engineering organizations. Faros has been serving large engineering organizations for the past seven years, so it was cool to be able to build a product for all Builders, not just enterprise engineering orgs. One big direction we’re exploring: using previous sessions to make the next one better. We’re looking at local memory portability across harnesses as well as multi-player modes to share memory within your direct team. I’d love to hear thoughts on that. Docs and product walkthrough: https://docs.desktop.faros.ai/ Setup: https://docs.desktop.faros.ai/getting-started/install-and-se... Data handling: https://docs.desktop.faros.ai/settings-and-privacy/understan... Faros website: https://www.faros.ai/

Nguyên văn nguồn
Chi tiết nguồn
Kiến trúc

Show HN: Skillset: Easily curate and install agent skills collection

Câu hỏi Wouldn't it be better to group skills for programming, backend architecture, or graphic design into collections, and install them with a single command only when needed?

Hi everyone! Installing skills globally in ~/.agents or ~/.claude consumes too much context, yet manually installing specific skills for every project is tedious. Wouldn't it be better to group skills for programming, backend architecture, or graphic design into collections, and install them with a single command only when needed? To solve this, I built Skillset. It acts as a wrapper around the Vercel Skill CLI that batch-installs sets of skills from multiple sources using a single JSON file, essentially npm for skills! While similar tools exist, most are written from scratch, which makes me concerned about their compatibility. Because Vercel’s Skill CLI is already the de facto standard (imo!), Skillset uses it directly as an underlying library. So it guarantees seamless compatibility across teams, regardless of whether they use Skillset or the standard Vercel CLI.

Nguyên văn nguồn
Chi tiết nguồn
Tác tử

Show HN: Backplane – An AI Development Environment for Hardware

We made Backplane to actually use Fable, Astra, etc. as real electrical engineers! It incorporates KiCad + your favorite harness (Claude Code, Codex, OpenCode, Grok) to accelerate hardware development. You can use it "autonomously" or work alongside the agent, making changes or comments to redirect the agent as it works. Also has some slick skills for schematic editing, BOM management. Hardware is in its "tab to accept changes"/copilot era rather than its agentic era imo.

Nguyên văn nguồn
Chi tiết nguồn
Nghiên cứu

Human-AI Mutualism Is Unstable under Capability Asymmetry

arXiv:2611.23322v1 Announce Type: new Abstract: One hope for a good human future alongside advanced AI rests on enduring mutualism: humans and AI will need each other, so the relationship will stay good for both. We argue that this hope fails as AI capability grows. Bargaining guarantees each side only what it would get if bargaining failed. For humans, this can be less than they had before AI. In a simple game-theoretic model, humans stay as well off as before AI without alignment only while they have leverage over the AI: either the AI depends on what they supply, or they can impose costs on the AI that are large relative to what the AI can produce. Once humans have neither kind of leverage, they stay as well off as before AI only if the AI is aligned with them. If the AI's goals are insatiable, the alignment required approaches perfect alignment as AI capability grows. Thus, mutualism is not a substitute for alignment. And when AI variants that keep more resources spread faster, alignment lasts only while misaligned variants are reliably sanctioned. In these two senses, human-AI mutualism is unstable.

Nguyên văn nguồn
Chi tiết nguồn
Tác tử

Where the Tokens Go: Heat, Entropy and Error in Agentic Coding

arXiv:2611.23375v1 Announce Type: new Abstract: Coding agents are billed by the token, yet software engineering says nothing about what a token actually buys. Here we introduce a theory of software thermodynamics. The theory draws analogues from classical and stochastic thermodynamics, extends them, and applies them to modern agentic coding. We count the tokens an agent generates as its cost and divide that cost into work and heat. The test suite acts as a software Maxwell's daemon that judges every coding change. We test the theory with an analogue of the experiments in which a single molecule is pulled open and let fold again: an agent makes a change to a repository (coding) and then undoes it (uncoding), many times over, and we count the tokens of every model call. The cycle returns the requested edits to where they began, with each direction started from a stored reference tree, and the spread of its cost over repeated cycles defines a software temperature, which rose with the sampling temperature of the model. About 80 per cent of the tokens of a successful run are generated in model calls that leave the requested change where it was, and we call these tokens heat. The free energy of a change, which we estimate by analogy with the equality of Jarzynski, is 14 to 22 per cent of the mean cost of a successful coding run. Coding costs 1.1 to 1.5 times as much as uncoding. The daemon is imperfect: the verdict of a test suite varies with its environment, and a suite that accepts a broken change has a cost in tokens. The practical implication of the proposed theory could be new tools for analysing and optimising coding agents through the lens of thermodynamics.

Nguyên văn nguồn
Chi tiết nguồn
Đánh giá

VeriMath: Human-in-the-Loop Generation and Verification of Curriculum-Aligned Mathematics Assessment Items

arXiv:2611.23456v1 Announce Type: new Abstract: Automatic generation of mathematics assessment items is a growing area in educational artificial intelligence, yet classroom deployment remains challenging. Generated questions must satisfy multiple constraints: mathematical correctness, curriculum alignment, appropriate cognitive difficulty, and educator reviewability. While prior work has advanced text generation, reliable assessment authoring requires granular control over localized examination formats and expert educational judgment. To address this challenge, we present VeriMath, a generalized, human-in-the-loop framework for generating rigorous, curriculum-constrained mathematics assessment drafts. Using annotated reference items, the framework anchors generation to the requested examination section and cognitive reasoning level, then applies sequential automated verification and guided repair before submitting drafts for expert review. We evaluate VeriMath through a national curriculum case study using 4,555 mathematics items from the Vietnamese Grade 12 curriculum and 50 distinct generation requirements covering multiple-choice, true/false, and short-answer formats. Across two model backbones, the framework produces verifier-accepted drafts for 94% (47/50) of requirements per run. Blind expert evaluations confirm high mathematical accuracy and curriculum alignment, with strict teacher-readiness assessments accepting up to 56.8% of generated items. Analysis of rejected drafts reveals persistent domain-agnostic challenges involving formatting, statement validity, and solution-step completeness. These findings demonstrate that VeriMath can bridge generative AI and dependable educational applications by producing highly reviewable assessment drafts within tightly constrained national curriculum frameworks.

arXiv cs.AI · Quy P. Nguyen, Duong Q. Nguyen, Dinh P. T. Nguyen, Truong-Giang T. Le, Truong A. Bui, Vi H. Vo, Quang-Thinh Bui, My-Phuong Ngo, Trong N. Do, Thao P. Le, Phuc M. Nguyen, H. Nguyen-Xuan Mở bản gốc ↗
Nguyên văn nguồn
Chi tiết nguồn
Đánh giá

Diagnostic Accuracy and Navigation Behavior of Vision-Language Model Agents Reading Emergency CT and MRI Examinations

arXiv:2611.23480v1 Announce Type: new Abstract: Purpose. To evaluate the diagnostic accuracy and image navigation behavior of vision-language model (VLM) agents under varying exploration strategies. Materials and Methods. In this retrospective study, six VLMs read 100 emergency CT and MRI examinations published in a public archive (March 2025-August 2026) under four conditions (free navigation, navigation from a key slice, and all slices or key slices in a single request), twice each (4800 runs). Nine radiologists read 30 of those cases. A large language model judge validated against two radiologists graded the answers. Conditions were compared with generalized estimating equations clustered by case. Results. Pooled accuracy was lowest under free navigation (32.6%; 391/1200), followed by navigation from a key slice (37.9%; 455/1200; P = .003), and highest with all slices (42.2%; 506/1200) or key slices (42.5%; 510/1200) in one request (both P < .001). On the 30 cases read by radiologists, freely navigating models were correct in 30.0% of runs (108/360) versus 77.0% for radiologists (208/270; P < .001). Models formed a first hypothesis after a median of 10.8% of the slices and kept it in 63.1% of runs (755/1196); final answers were correct in 12.4% of runs after a wrong first hypothesis (110/885) versus 90.4% after a right one (281/311; P < .001). Median stated confidence in the final answers was 90%, whereas 38.9% of them were correct (1861/4778). Conclusion. Across all exploration strategies, VLM agents remained far less accurate than radiologists. They adopted an early hypothesis and usually kept it, seldom correcting a wrong one.

arXiv cs.AI · Jonathan Lichtle, Aymen Meddeb, Antonin Birot, Ibrahima Niang, Lisa Gastaud, Riyad Hanafi, Sofiane Dujardin, Corentin Selomme, Guillaume Cachard, Aghiles Hamroun, Gr\'egory Kuchcinski, Su Hwan Kim, Bastien Le Guellec Mở bản gốc ↗
Nguyên văn nguồn
Chi tiết nguồn
Tác tử

Receiver-Conditioned Clarification for Multi-Agent Food Assistance

arXiv:2611.23572v1 Announce Type: new Abstract: Food insecurity creates a pressing need for guidance that helps households find usable food resources and assess food choices in light of their circumstances. Such guidance must connect eligibility requirements, practical access constraints, and dietary needs rather than return isolated listings or generic nutritional advice. Multi-agent food assistants can coordinate these tasks, with uncertainty indicating when clarification or verification is needed before a recommendation. However, uncertainty can be lost across agent handoffs, allowing tentative interpretations to propagate as facts and obscuring opportunities for correction. Moreover, confidence that a service requirement has been interpreted correctly is not confidence that a recommendation satisfies the user's needs; local confidence cannot simply be reused as downstream reliability. We introduce an inference-time, receiver-conditioned clarification framework that preserves shared premise identities and replays consistent interpretations across dependent agents. Each receiver exposes admissible decisions and remaining evidence obligations, so question relevance reflects both decision changes and missing support. Information questions and user-confirmable fallback adjustments are evaluated together with their anticipated follow-up interactions under shared budgets. This structure connects clarification to jointly supported plans for resource navigation, nutritional reasoning, and supply-side planning.

arXiv cs.AI · Kaiwen Shi, Zheyuan Zhang, Changzhi Liu, Han Bao, Yanfang Ye Mở bản gốc ↗
Nguyên văn nguồn
Chi tiết nguồn
Đánh giá

ARISE: Practical Self-Evolving Memory for Accurate and Cost-Efficient Enterprise Agents

arXiv:2611.23588v1 Announce Type: new Abstract: Enterprise LLM agents operate in dynamic environments where foundation models evolve rapidly, workflows update frequently, and deployments need to satisfy accuracy and cost requirements. We propose ARISE (Agent Refinement through Iterative Self-Evaluation), a practical memory architecture in which the agent's memory evolves from its own failures without updating model weights, tools, or workflow logic. ARISE uses the model itself or a teacher model to attribute the failures and propose corrections. Then, ARISE keeps the corrections as retrievable micro memory, consolidates them into macro rules, and accepts a rule update only if held-out accuracy rises. Across 14 enterprise-workflow domains from SOP-Bench and $\tau^2$-bench using 5 models, ARISE-Teacher, which uses Claude Opus 4.6 for attribution and consolidation, gains 10.9 pp over the baseline on SOP-Bench and 5.8 pp on $\tau^2$-bench. ARISE-Teacher also outperforms GEPA by 4.5 pp on $\tau^2$-bench and 1.4 pp on SOP-Bench. With the deployment model as its own attributor, ARISE-Self still gains 10.0 pp on SOP-Bench. ARISE-Teacher uses 4.0% fewer model calls and 5.2% fewer output tokens at comparable per-task cost (-1.0%).

arXiv cs.AI · Jiangyun Zhang, Zhou Tang, Kristen Surrao, Zhiyu Chen, Albert Guan, Tor Nitayanont, Yupei Zhang, Roopali Singh, Yunkai Xiao, Shreya Pandit, Julia Huang, Shayan Ali Akbar, Erwin Cornejo, Yuan Li, Yi Zhang Mở bản gốc ↗
Nguyên văn nguồn
Chi tiết nguồn
Đánh giá

Evaluating Governed LLM-Driven ML Experimentation: Lifecycle, Provenance, and Failure Metrics

arXiv:2611.23678v1 Announce Type: new Abstract: Autonomous agents that run ML experiments are usually judged by the best score they reach. That score says nothing about how it was reached: whether the agent stayed within the files it was allowed to edit, whether failed trials were kept or silently dropped, and whether each keep or discard decision can be traced to its evidence. We propose evaluating such systems on their experimental process as well as their outcome, and we build a governed harness that makes this possible. A control plane, not the LLM that proposes changes, owns each trial's lifecycle, checks its edit scope, decides keep or discard, and appends one record per trial to an append-only ledger. Governance metrics such as the invalid-action rate, the repeated-bad rate and provenance completeness are computed from that ledger. Across six ML tasks, from synthetic logistic regression to language-model training, and 1,445 trials, the same contract applied unchanged. The ledger exposed failures that a task score hides: a no-memory arm in which all 120 trials failed, a benchmark adapter whose bounded proposals ran out, and a reset failure that reproducibility hashes revealed, after which the affected campaigns were rerun. In a matched counterfactual, a loop that logs only successful trials left no record of 43 of 50 trials, while the governed ledger recorded all 50. Memory ablations show that memory's effect on repeated bad proposals depends on the task. We release the ledgers of all 1,445 trials, so that every number reported for them can be recomputed.

Nguyên văn nguồn
Chi tiết nguồn
Tác tử

Thinking Fast, Anywhere: One System-1 for the Decision Points of Any Agent Workflow

arXiv:2611.23751v1 Announce Type: new Abstract: Autonomous LLM agents incur substantial cost and latency by invoking a large model (System-2) at every step, even when many intermediate actions choose among a small set of known candidates. Existing dual-system agents offload such decisions to a lightweight System-1 model, but rely on workflow-specific hooks authored per environment. We present AnyS1, a framework that turns human-authored procedural skills, whose instructions naturally specify where choices arise and which options apply, into a shared, domain-general source of decision points. Offline, AnyS1 mines grounded enumerable decisions across over 1,000 public skills, gathers execution-grounded trajectory states inside isolated containers to construct Any-DP, and trains a single calibrated 4B pointer System-1 with label smoothing. At run time, compiled workflows resolve routine decisions in one forward pass and fall back to System-2 when confidence falls below a calibrated threshold. Across three unseen benchmarks and distinct System-2 model families, AnyS1 preserves task reliability while cutting episode cost by 75-91% and model-call time by 63-92% relative to an uncompiled System-2 agent, and maintains performance under zero-shot threshold transfer.

arXiv cs.AI · Rui Li, Liyang He, Zheng Zhang, Zhenya Huang, Qi Liu Mở bản gốc ↗
Nguyên văn nguồn
Chi tiết nguồn
Huấn luyện

Continual Tool-Space Extension Learning for Multimodal Agents

arXiv:2611.23810v1 Announce Type: new Abstract: Multimodal agents largely rely on generic tools such as cropping, search, and code execution, leaving specialist vision models outside their tool space. However, bringing such a model into an already trained agent is not straightforward. The challenge is to learn each new tool efficiently while retaining the ability to use existing ones. This work proposes COTEX, continual tool-space extension learning, that adds new tools at efficient adapter-level cost. COTEX locates two sets of parameter directions; one triggers a tool call and thereby opens a multi-turn reasoning trajectory, while the other determines which tool is chosen. The adapter of a new tool is restricted to the calling directions that leave earlier tool choices unaffected by orthogonal constraint. We further introduce a protocol that converts specialist models' existing task-specific annotations into verified tool-use trajectories. Experiments on two representative agents and three specialist tools show gains on ten vision-language benchmarks, with at most a 0.3-point drop on existing tools.

arXiv cs.AI · Jinbae Seo, Jungin Park, Hyunsung Jang, Namkoo Ha, Jiyoung Lee, Kwanghoon Sohn Mở bản gốc ↗
Nguyên văn nguồn
Chi tiết nguồn
Đánh giá

A System 1 Decision Model for Medical Multiple-Choice Questions Built on Frozen Language Models

arXiv:2611.23832v1 Announce Type: new Abstract: A System 1 model answers a multiple-choice question in a single pass. A frozen language model turns the question and each option into vectors, two small, trained heads place them in a shared space, and the closest option wins. The design has been shown on agent decisions, but we are not aware of a study of how far it goes on exam questions that need medical knowledge, where the right option is often no more similar to the question than the wrong ones. We rebuilt the recipe from scratch for MedMCQA and trained only two projection heads and one temperature on top of frozen Qwen3.5-0.8B, Granite-4.2-3B and Qwen3-8B, using a contrastive loss with in-batch and hard negatives and a mask for repeated answers. With plain last-token vectors, both small encoders answer about one question in three correctly (33.1% and 33.0% on 3,842 held-out questions, where chance is 25.0%). Writing each text inside a one-word-summary prompt and lowering the learning rate of the heads raised this to 41.7% and 42.5% for the two small encoders and to 46.0% +-0.2 (three seeds) for Qwen3-8B. Shuffling the questions against the options returns accuracy to chance, so the heads do use the question. Flan-PaLM and Med-PaLM 2 report 57.6% and 72.3% on MedMCQA under a different protocol, so ours is a cheap, fast baseline and not a competitive system. We also say what our evidence cannot show: validation gains were about 1.7 to 2 times the test gains, and we could not separate the effect of reading several encoder layers from the other changes.

Nguyên văn nguồn
Chi tiết nguồn
Đánh giá

RoboBuilder: Agentic Robot Co-Design with Real-world Video Feedback in the Loop

Câu hỏi 23855v1 Announce Type: new Abstract: Can AI agents design robots that work in the real world?

arXiv:2611.23855v1 Announce Type: new Abstract: Can AI agents design robots that work in the real world? We introduce RoboBuilder, a benchmark in which coding agents co-design a robot's hardware and program from standard parts, motors and a custom 3D-printed part, and every design that passes in simulation is built and tested on a physical course. Simulation alone is not enough: under public part data, none of 15 initial passing designs crossed the real course, so an agent learns the simulator from measurements it requests and videos of real runs. Across four frontier coding agents, ten obstacles at three difficulty levels, 16,427 simulations of 12,774 distinct robots and 135 physical robots built in an estimated 240 hours, agents solve nearly all easy obstacles and invent mechanisms our search baselines do not reach, but falter at harder levels, when one robot must cross every obstacle, and in the real world, where 66% of simulated passes hold across all levels. Closing the loop helps: shown a video of its real failure, an agent redesigns the robot, sometimes correcting its simulator, and removes 72% of level-1 and level-2 failures. RoboBuilder measures progress toward agents that engineer real machines. Project Page: https://ghadinehme.com/robobuilder.github.io/

arXiv cs.AI · Ghadi Nehme, Dahyun Nam, Akshay Govind Srinivasan, Faez Ahmed Mở bản gốc ↗
Nguyên văn nguồn
Chi tiết nguồn
Kỹ thuật

Show HN: Goethe – turn your computer into an inference engine

Hi HN, I'm Matteo from meshapplied (YC W27), building this with Peter and Janos. We think you can and should own your inference, so we built software that actually supports that. Goethe turns a Linux machine with an NVIDIA GPU into an API your OpenAI, Anthropic compatible clients can call. ``` curl -fsSL https://goethe.sh/install.sh | bash goethe start ``` Limits: it's Linux only rn and it supports Ampere or newer NVIDIA GPUs. Logging in with `goethe login` is optional and turns on the optimized kernels. Docs: [goethe.sh/docs]( https://goethe.sh/docs ) I'd love to hear what breaks and what you'd want next.

Nguyên văn nguồn
Chi tiết nguồn
Đánh giá

Full Fine-Tuning (FFT) on SageMaker AI Serverless Model Customization

<p>Fine-tuning is the process of continuing to train a pretrained large language model (LLM) on a company’s own data so it adapts to a specific domain, task, or style rather than relying on general-purpose knowledge alone. For businesses, the value is twofold. A fine-tuned model delivers measurably better accuracy on domain-specific work while reducing costs: a smaller specialized model can match or exceed a frontier model on your task at a fraction of the per-token inference price. Those savings compound across millions of requests. Fine-tuning has therefore become an essential process for businesses looking to get more value from LLMs at lower costs.</p> <p>Parameter-efficient fine-tuning (PEFT) methods like Low-Rank Adaptation (LoRA) have emerged as a practical solution for fine-tuning LLMs to downstream tasks, reducing GPU capacity requirements by updating only a small subset of LLM parameters. LoRA aims to strike a balance between efficiency and expressiveness, however, the low-rank nature of the weight update inherently limits the representation power of LoRA fine-tuned models, potentially compromising performance on complex tasks. This trade-off hampers the ability of LoRA methods to capture complex patterns needed for diverse tasks. To address these challenges, Full Fine-Tuning (FFT) updates model parameters simultaneously, overcoming the low-rank limitations so LLMs can fully absorb complex domain-specific knowledge.</p> <p><a href="https://aws.amazon.com/sagemaker/ai/model-customization/" target="_blank" rel="noopener">Amazon SageMaker AI Serverless Model Customization</a> now supports <a href="http://docs.aws.amazon.com/sagemaker/latest/dg/customizing-models-fft.html" target="_blank" rel="noopener">Full Fine-Tuning (FFT)</a>, so you can achieve superior model quality without managing infrastructure. There is no GPU provisioning, cluster setup, or distributed-training configuration required. In this post, we explore when and where FFT consistently outperforms LoRA across key benchmarks, examine the spectral properties of model weight matrices that explain this quality gap, provide practical hyperparameter tuning guidelines backed by controlled experiments using SageMaker AI Serverless Model Customization, and show you how to get started with FFT on SageMaker AI.</p> <h2 id="full-fine-tuning-why-it-matters">Full Fine-Tuning: Why it matters</h2> <p>Model customization types like LoRA reduce trainable parameters by freezing the pre-trained model’s weights and injecting small, separate pairs of replacement matrices (called low-rank matrices) into the model layers. This works well for simple adaptations, but the low-rank structure inherently limits the representation power of the fine-tuned model. As tasks grow more complex and datasets scale beyond tens of thousands of examples, LoRA hits an accuracy plateau where increasing rank (and thus trainable parameters) fails to bridge the accuracy gap with standard fine-tuning. This plateau reveals that the performance gap isn’t simply because of fewer parameters. It’s caused by rank deficiency itself. The low-rank update can’t capture the full complexity of the weight changes needed for deep task adaptation.</p> <p>However, for complex tasks and domains where large, good-quality datasets exist, full-rank fine-tuning excels by updating model parameters simultaneously. FFT modifies the model’s weight matrix uniformly, using the full representational capacity of the model’s weight matrices rather than restricting updates to a small subspace, and without introducing the abrupt, localized distortions characteristic of low-rank methods. The result is stronger model convergence, better activation alignment across model layers, and a model that faithfully captures both the target task distribution and the richness of its pre-trained knowledge. FFT’s quality advantage grows with dataset scale: at 100K+ training examples, FFT achieved 90.0 percent on LLM-as-Judge semantic correctness across code generation tasks, as shown in the following chart.</p> <div style="width: 810px" class="wp-caption alignnone"> <a href="https://d2908q01vomqb2.cloudfront.net/f1f836cb4ea6efb2a0b1b99f41ad8b103eff4b59/2026/11/16/ML-21644-1.png" target="_blank" rel="noopener"><img src="https://d2908q01vomqb2.cloudfront.net/f1f836cb4ea6efb2a0b1b99f41ad8b103eff4b59/2026/11/16/ML-21644-1.png" alt="LLM-as-Judge semantic correctness by training method as training data scales to 100K-plus examples" width="800"></a> <p class="wp-caption-text">Figure 1: LLM-as-Judge semantic correctness by training method as training data scales to 100K+ examples</p> </div> <p>Furthermore, the following radar chart provides a holistic view: the base model (inner polygon) shows consistently low scores across all axes. LoRA (middle polygon) shows strong, balanced improvement. FFT (outer polygon) extends further on every dimension, with particularly pronounced leads on BLEU and ROUGE-2 scores. This multi-dimensional quality lead confirms that it isn’t a metric-specific artifact but a fundamental advantage of full-rank optimization.</p> <div style="width: 810px" class="wp-caption alignnone"> <a href="https://d2908q01vomqb2.cloudfront.net/f1f836cb4ea6efb2a0b1b99f41ad8b103eff4b59/2026/11/16/ML-21644-2.png" target="_blank" rel="noopener"><img src="https://d2908q01vomqb2.cloudfront.net/f1f836cb4ea6efb2a0b1b99f41ad8b103eff4b59/2026/11/16/ML-21644-2.png" alt="Radar chart comparing the base model, LoRA, and FFT across BLEU, ROUGE, and correctness metrics" width="800"></a> <p class="wp-caption-text">Figure 2: Radar chart comparing the base model, LoRA, and FFT across quality dimensions</p> </div> <p>The details of how we arrived at these results, the coding datasets used for model customization, and the experimentation process are explained in detail in the following sections.</p> <h2 id="experimentation-and-benchmark-results">Experimentation and benchmark results</h2> <p>In this post we used the <a href="https:/

  • A fine-tuned model delivers measurably better accuracy on domain-specific work while reducing costs: a smaller specialized model can match or exceed a frontier model on your task at a fraction of the per-token inference price.
  • </p> <p>Parameter-efficient fine-tuning (PEFT) methods like Low-Rank Adaptation (LoRA) have emerged as a practical solution for fine-tuning LLMs to downstream tasks, reducing GPU capacity requirements by updating only a small subset of LLM parameters.
  • html" target="_blank" rel="noopener">Full Fine-Tuning (FFT)</a>, so you can achieve superior model quality without managing infrastructure.
Nguyên văn nguồn
Chi tiết nguồn
Đánh giá

Exa boosts embedding throughput by 50% to index billions of documents on AWS AI chips

<p>At <a href="https://exa.ai/" target="_blank" rel="noopener">Exa</a>, we are building one of the few true web indexes by aggregating and serving knowledge from across the web, along with private and synthetic sources. We optimize the index for semantic search, grounding agents and humans in true, high-quality data. We use multi-tower semantic machine learning (ML) models to embed web data and query data into the same vector space, so users can express complex queries in natural language alongside traditional keyword search. The keystone technology behind this is an in-house vector database. It must ingest as much high-quality web data as possible and embed all of it. The unit economics of embedding therefore underlie everything we do. At the scale of tens or hundreds of billions of documents, finding the hardware that delivers the best performance per dollar isn’t a tuning detail. Any per-document cost improvement compounds into a significant number.</p> <p>This post details how we optimized our embedding infrastructure for cheaper, more flexible inference workloads on AWS Neuron hardware (<a href="https://aws.amazon.com/ai/machine-learning/inferentia/" target="_blank" rel="noopener">AWS Inferentia2</a> and <a href="https://aws.amazon.com/ai/machine-learning/trainium/" target="_blank" rel="noopener">AWS Trainium2</a>), and how an agentic development workflow drove the optimization.</p> <p>Inference runs on an asynchronous pipeline, but it must be high-volume and fast to serve recent data quickly and shrink the time between discovery and serving.</p> <div style="width: 810px" class="wp-caption alignnone"> <a href="https://d2908q01vomqb2.cloudfront.net/f1f836cb4ea6efb2a0b1b99f41ad8b103eff4b59/2026/11/13/ML-22138-1-1.png" target="_blank" rel="noopener"><img src="https://d2908q01vomqb2.cloudfront.net/f1f836cb4ea6efb2a0b1b99f41ad8b103eff4b59/2026/11/13/ML-22138-1-1.png" alt="Exa stack showing web data flowing from crawlers through parse, moderate, and embed stages into a vector database" width="800"></a> <p class="wp-caption-text">Figure 1: A simplified view of the Exa stack</p> </div> <p>Every structured web document we rank as high enough quality must be embedded before it can be served. This figure shows how raw web data moves from Crawlers into an asynchronous pipeline of Parse, Moderate and filter, then Embed, with the result served from a vector database.</p> <h2 id="embedding-documents-at-scale">Embedding documents at scale</h2> <p>Our experience with the Neuron hardware family began with a backfill. Moving to a superior document model meant re-embedding our existing corpus. We selected 30 billion high-priority documents for re-embedding. A migration like this temporarily doubles the embedding load, which made it exactly the right moment to re-evaluate our hardware choices.</p> <div style="width: 810px" class="wp-caption alignnone"> <a href="https://d2908q01vomqb2.cloudfront.net/f1f836cb4ea6efb2a0b1b99f41ad8b103eff4b59/2026/11/13/ML-22138-2-1.png" target="_blank" rel="noopener"><img src="https://d2908q01vomqb2.cloudfront.net/f1f836cb4ea6efb2a0b1b99f41ad8b103eff4b59/2026/11/13/ML-22138-2-1.png" alt="Two parallel embedding pipelines sharing crawl and parse stages, doubling the inference load during the backfill" width="800"></a> <p class="wp-caption-text">Figure 2: The doubled inference load from the backfill</p> </div> <p>The preceding figure shows two parallel pipelines sharing the crawl and parse stages, one embedding with the live model into the live index and one re-embedding priority documents with the new model into the new index, which is what doubles the load. This doubling was a clear trigger to reconsider our hardware and optimization choices.</p> <h2 id="optimization-journey">Optimization journey</h2> <p>AWS Inferentia2 figures refer to performance on the smallest instance type, an <a href="https://aws.amazon.com/ec2/instance-types/inf2/" target="_blank" rel="noopener">Inferentia2 xlarge instance</a> (<code>inf2.xlarge</code>), with one chip and two <a href="https://awsdocs-neuron.readthedocs-hosted.com/en/latest/about-neuron/arch/neuron-hardware/neuron-core-v2.html" target="_blank" rel="noopener">NeuronCores</a>. Our workload runs on a single NeuronCore, so we run Kubernetes pods that pack onto larger instances without leaving any cores idle, giving us container homogeneity across the embedding fleet. The same process runs on a Trainium2 NeuronCore and scales the same way through data parallelism. Our initial Inferentia2 implementation was a straightforward data-parallel, FP16 representation of the model. From that baseline, it became a march to push throughput (docs/sec) as high as possible, using the Neuron SDK and support from the AWS Trainium team. The following chart shows the throughput gains at each milestone.</p> <div style="width: 477px" class="wp-caption alignnone"> <a href="https://d2908q01vomqb2.cloudfront.net/f1f836cb4ea6efb2a0b1b99f41ad8b103eff4b59/2026/11/13/ML-22138-3-1.png" target="_blank" rel="noopener"><img src="https://d2908q01vomqb2.cloudfront.net/f1f836cb4ea6efb2a0b1b99f41ad8b103eff4b59/2026/11/13/ML-22138-3-1.png" alt="Chart of throughput gains across optimization milestones, rising to about 1.5 times the baseline" width="467"></a> <p class="wp-caption-text">Figure 3: Throughput improvement across optimization milestones</p> </div> <p>Starting from the naive compile, a progressive series of optimizations ultimately delivered roughly a 1.5× throughput improvement. The first was weight layout optimization, which eliminates unnecessary transposes by aligning the weight layout with the scheduling and layout of the broader model. We also enabled autocast, letting the compiler downcast matrix multiplications to lower precision while preserving dynamic range. The downcast outputs met our correctness gate of 0.999 cosine similarity against the reference (see <a href="#agentic-methodology">Agentic methodology</a>), so the accuracy impact was within our accept

  • 5× throughput improvement.
Nguyên văn nguồn
Chi tiết nguồn
Huấn luyện

Introducing workload recovery and expanded infrastructure checks on Amazon SageMaker HyperPod

<p>Large training jobs can fail even when every node looks healthy. A GPU with a subtle fault can pass its stress tests and still return wrong values. When a job stalls on an NCCL timeout, it keeps its GPUs, and Slurm still shows it as running. If a device plugin crashes on one node, the job fails with an error that never mentions the plugin. Teams usually find these problems only after the loss diverges or the job fails, and by then the compute is spent. Each of these failures lowers goodput, the share of your cluster’s time that goes to useful training. Today, we’re introducing workload recovery and infrastructure checks on Amazon SageMaker HyperPod that catch these failures sooner and raise the goodput of your training workloads.</p> <p>SageMaker HyperPod already handles nodes that report hardware faults. The health monitoring agent (HMA) watches each node for GPU errors, deep health checks test nodes during provisioning, and automatic node recovery reboots or replaces faulty hardware. These features look at one node’s hardware at a time. They can’t tell whether the GPUs train correctly, whether a job is still making progress, or whether the Kubernetes components a job depends on are working.</p> <p>In this post, we walk through the three new capabilities that catch these failures. For each one, we show how to set it up and what it reports when it finds a problem:</p> <ol type="1"> <li><code>MultiNodeTrainingValidation</code> is a new deep health check for HyperPod clusters orchestrated by Amazon Elastic Kubernetes Service (Amazon EKS). When HyperPod provisions an instance, or when you run the check on demand, it trains a small model on the instance and then across instances of the same type. The instance fails if the loss becomes NaN or infinite, or training doesn’t finish in time.</li> <li>The HyperPod Kubernetes Resource Monitoring Agent watches the Kubernetes layer of an EKS cluster for problems such as crash-looping device plugins, missing DaemonSets, and tainted nodes. When a job fails, it reports the pods and nodes involved as Kubernetes events and Amazon CloudWatch logs.</li> <li>Hung job detection finds Slurm jobs that have stopped making progress. An agent on each node watches the job’s output and GPU activity, or log patterns that you define. It writes an event to CloudWatch and can cancel the job so that Slurm requeues it.</li> </ol> <h2 id="why-node-health-alone-isnt-enough">Why node health alone isn’t enough</h2> <p>The existing HyperPod resiliency features check a single node at a time. The health monitoring agent (HMA) watches each node for GPU errors such as Xid and NVLink faults, Elastic Fabric Adapter (EFA) errors, and kernel errors. The InstanceStress deep health check runs DCGM level 4 diagnostics and tests each instance’s GPUs, GPU memory, NVLink, and EFA devices. The InstanceConnectivity check runs NCCL tests across GPU instances, or NCCOM tests on AWS Trainium. Together, these confirm that each node’s hardware works. They don’t show whether the GPUs compute correctly, whether a job is still making progress, or whether the cluster’s Kubernetes components are healthy. Three kinds of failure get past them:</p> <ul> <li><strong>Numerical faults –</strong>&nbsp;A GPU can pass every hardware test and still compute wrong results. The first sign is often NaN or infinite loss in a production run. Meta attributed 6 of the 419 unexpected interruptions in its 54-day Llama 3 405B pre-training run to silent data corruption (<a href="https://arxiv.org/abs/2407.21783" target="_blank" rel="noopener">The Llama 3 Herd of Models</a>).</li> <li><strong>Hung jobs –</strong>&nbsp;An NCCL timeout, a stalled data pipeline, or a deadlock can stop a job without ending it. The job keeps its GPUs and their memory, and Slurm reports it as running until someone checks the loss curve or the log.</li> <li><strong>Cluster-level failures –</strong>&nbsp;Some failures happen in Kubernetes, not in a node’s hardware. A device plugin crash-loops, a required DaemonSet never starts on a new node, or ranks on different nodes fall out of step in an NCCL collective. HMA checks each node on its own, so it doesn’t see these problems. When the job fails, the error often doesn’t say which pod or node caused it.</li> </ul> <h2 id="solution-overview">Solution overview</h2> <p>Each capability covers a different point in a training run. <code>MultiNodeTrainingValidation</code> runs when instances are provisioned, or on demand, and confirms that their GPUs train correctly. Hung job detection runs alongside each Slurm job and catches a job that has stopped making progress. The HyperPod Kubernetes Resource Monitoring Agent watches the cluster continuously and, when a job fails, reports the pods and nodes that were involved. The following table summarizes the three.</p> <table border="1px" width="100%" cellpadding="10px"> <tbody> <tr> <td><strong>Capability</strong></td> <td><strong>What it catches</strong></td> <td><strong>When it runs</strong></td> <td><strong>Available on</strong></td> </tr> <tr> <td><code>MultiNodeTrainingValidation</code> deep health check</td> <td>NaN or infinite loss, or training that doesn’t finish in time, on GPUs that pass hardware checks</td> <td>When instances are created or updated, or on demand</td> <td>EKS clusters, on supported EFA-enabled NVIDIA GPU instances</td> </tr> <tr> <td>HyperPod Kubernetes Resource Monitoring Agent</td> <td>Node condition changes, pod and DaemonSet failures, and failed training jobs, with the pods and nodes involved. NCCL communication state for jobs that opt in</td> <td>Continuously</td> <td>EKS clusters</td> </tr> <tr> <td>Hung job detection</td> <td>Jobs that stop making progress but don’t exit</td> <td>While each job runs</td> <td>Slurm clusters</td> </tr> </tbody> </table> <p>Hung job detection is new for Slurm clusters. On EKS clusters, HyperPod already detects hung jobs that run through the <a href="https://docs.aws.amazon.com/sagemaker/latest/dg/sag

  • For each one, we show how to set it up and what it reports when it finds a problem:</p> <ol type="1"> <li><code>MultiNodeTrainingValidation</code> is a new deep health check for HyperPod clusters orchestrated by Amazon Elastic Kubernetes Service (Amazon EKS).
  • Three kinds of failure get past them:</p> <ul> <li><strong>Numerical faults –</strong>&nbsp;A GPU can pass every hardware test and still compute wrong results.
Nguyên văn nguồn
Chi tiết nguồn
81–100 / 680
Trang 5 / 34