AllyshipContent Radar

News 2934

Chủ đề cộng đồng đang thảo luận

Crawl live-1795137300000 · 08:15 20 thg 11, 2026 · Xuất bản 08:31 20 thg 11, 2026

213 / 2934 chủ đề đạt ngưỡng 55 trong khoảng này.

181

WorldRuler: Interactive World Modeling under Configurable Rules

70

9 tiếng nói · chưa gắn nhãn trưởng lab · 1 nền tảng · 9 tương tác đã đo · Nguồn crawl

Mingliang Zhai, Chuanhao Li và 7 nguồn khác 1 bài viết · Hugging Face Papers
Nguyên văn nguồn
Chi tiết nguồn

Rules govern how player actions change interactive worlds. Different rules create distinct gameplay experiences within similar scenes and action spaces. For customizable gameplay, generative world models must follow user-defined rules beyond real-world behaviors. However, realizing user-defined rules presents two challenges: Rule-State Persistence and Rule-to-Visual Realization. The former requires preserving rule definitions, dependencies, and world states across saving and loading, so that restored worlds follow the established rules. The latter requires accurately realizing rule-induced changes on designated objects and attributes, particularly when affected objects are unseen. In this paper, we propose WorldRuler, which organizes rule definitions, dependencies, and object states in an executable 3D rule graph and connects rule execution, video generation, and perceptual feedback in a closed loop. The graph supports joint saving and restoration of rules and world states, while dynamic object binding grounds rule-specified entities in visual instances and uses stored states to constrain generation from their first appearance. We further introduce object attribute state replacement and counterfactual reconstruction training, enabling accurate attribute control and compatible appearance and motion changes while limiting interference with unrelated content. We develop an automated data collection and annotation pipeline using GTA-V and Habitat to construct RuleVerse, a dataset of rule-driven interactions with detailed state and event annotations, including cases where rules take effect before affected objects become visible. Extensive experiments demonstrate the effectiveness of WorldRuler in preserving rule-state consistency across saving and loading and accurately realizing rule outcomes on designated objects and attributes.

182

Mechanistic Critics for Sample-Efficient NPU Design-Space Exploration

70

5 tiếng nói · chưa gắn nhãn trưởng lab · 1 nền tảng · 7 tương tác đã đo · Nguồn crawl

Yicheng He, Yuqi Xue và 3 nguồn khác 1 bài viết · Hugging Face Papers
Nguyên văn nguồn
Chi tiết nguồn

NPU design-space exploration for LLM inference mainly relies on an expensive simulator that often returns only scalar feedback such as latency and power. Such feedback can rank evaluated configurations, but gives little information for deciding why a configuration is slow or which architecture parameter should be changed next. This limitation is especially severe because prefill and decode workloads can be limited by different resources, including compute throughput and HBM bandwidth. We propose CriticNPU, a sample-efficient DSE framework built around an online-calibrated mechanistic critic for NPU architecture design. The critic uses explicit hardware limits to evaluate NPU configurations: before simulation, it estimates whether a proposed configuration is likely to improve over the current best and filters weak candidates; after simulation, it aggregates operator-level runtimes into prefill/decode bottleneck summaries and predicts the effect of candidate parameter changes. We instantiate the critic with a Roofline-style model that derives compute and memory limits from architecture parameters and calibrates per-operator-type effective limits using simulator observations. And CriticNPU combines this critic with a workload extractor and an architecture planner to propose configurations. Across six LLM workloads and 16 deployment configurations per model, CriticNPU achieves best-or-tied-best latency in 74% of prefill and 87.5% of decode settings while using roughly half the simulator calls of competitive DSE baselines. We conduct extensive ablation studies and detailed analyses to validate the effectiveness of our method from multiple perspectives. Our code is available at https://github.com/bruno686/CriticNPU.

183

PReMind: When Personal Memory Meets Visual Hazards

68

5 tiếng nói · chưa gắn nhãn trưởng lab · 1 nền tảng · 4 tương tác đã đo · Nguồn crawl

Yeongtak Oh, Donghyeok Lee và 3 nguồn khác 1 bài viết · Hugging Face Papers
Biên tập tiếng Việt
Chi tiết nguồn

Bài báo nói rằng trợ lý thị giác-ngôn ngữ được cá nhân hóa sử dụng quan sát hình ảnh và bộ nhớ cá nhân để đưa ra lời khuyên phù hợp. Khi cảnh có nguy hiểm vật lý, bối cảnh cá nhân như dị ứng, bệnh lý hoặc ràng buộc vận động có thể làm thay đổi mức độ trợ giúp phù hợp; do đó cần nhận biết nguy hiểm và áp dụng thông tin đúng của người dùng. Họ giới thiệu PReMind-Bench để đánh giá đồng thời kết hợp bộ nhớ cá nhân và các nguy hiểm, và đề xuất PReMind—khung reinforcement learning hai pha—để huấn luyện đồng thời grounded memory use và hazard response.

Personalized vision-language assistants use visual observations and personal memories to provide advice tailored to the user. When the observed scene contains a physical hazard, personal context, such as allergies, medical conditions, or mobility constraints, can change what help is appropriate. Responding appropriately therefore requires recognizing the visible hazard and applying the relevant person's information. We introduce PReMind-Bench, a diagnostic benchmark that evaluates this combination across diverse hazards and personal-memory conditions. It separately measures memory access, grounding, abstention, and hazard response, and assesses joint success on interdependency cases. Existing models often satisfy one requirement while failing the other, and training only for safety or personalization leaves a gap in joint success. We therefore propose PReMind, a two-phase reinforcement learning framework that combines on-demand and forced-call interactions (memory always retrieved) to jointly train grounded memory use and hazard response. At 4B, PReMind raises joint success from 12.9% to 28.6% over its base on the interdependency set and improves hazard adequacy and correct abstention on the main test. Joint-success gains persist under forced-call evaluation and under a second LLM judge across three Phase II seeds; averaged over these seeds, joint success also improves on unseen hazard groups. These findings highlight the need to evaluate and train personal assistants to connect visual evidence with the relevant person's context.

  • At 4B, PReMind raises joint success from 12.9% to 28.6% over its base on the interdependency set and improves hazard adequacy and correct abstention on the main test.
  • At 4B, PReMind raises joint success from 12.9% to 28.6% over its base on the interdependency set and improves hazard adequacy and correct abstention on the main test.
  • Joint-success gains persist under forced-call evaluation and under a second LLM judge across three Phase II seeds; averaged over these seeds, joint success also improves on unseen hazard groups.
  • These findings highlight the need to evaluate and train personal assistants to connect visual evidence with the relevant person's context.
184

RoughCut-Bench: A Benchmark for Automatic Video Rough-Cut with Large Vision-Language Models

68

10 tiếng nói · chưa gắn nhãn trưởng lab · 1 nền tảng · 4 tương tác đã đo · Nguồn crawl

Ruisi Zhao, You Li và 8 nguồn khác 1 bài viết · Hugging Face Papers
Nguyên văn nguồn
Chi tiết nguồn

Video rough-cut is a core stage in media production workflows, organizing raw media materials into an initial video cut. Despite recent advances in large vision-language models (LVLMs) for video-related tasks, automatic rough-cut remains underexplored, without a comprehensive benchmark for evaluation or mature workflows for producing executable cut plans. To address the evaluation gap, we introduce RoughCut-Bench, which covers two practical settings: (1) a visually driven setting over video cases, covering highlight detection and selection, source-video ordering, and adherence to user requirements when provided; and (2) a speech-driven setting over automatic speech recognition (ASR) transcripts, covering filler removal, expression deduplication, and low-value content trimming. Across these settings, the benchmark contains 101 visually driven cases, including 831 raw videos with 1,476 highlight annotations. It also includes 84 speech-driven cases, each consisting of an ASR transcript with annotated cleanup spans. The evaluation compares cut plans with these annotations in both settings and additionally uses a human-aligned agentic evaluator to assess rendered-cut quality in the visual setting. To apply LVLMs to automatic rough-cut, we further propose agentic workflows that guide them to produce structured cut plans from input materials. Experiments show that current LVLMs remain limited in producing expert-aligned cut plans. Our benchmark, code, and prompts are released at https://whalesong-zrs.github.io/RoughCut-Bench-Project-Page.

185

Late Entry in Looped Language Models

68

4 tiếng nói · chưa gắn nhãn trưởng lab · 1 nền tảng · 4 tương tác đã đo · Nguồn crawl

Ziqi Jin, Jiaxi Li và 2 nguồn khác 1 bài viết · Hugging Face Papers
Nguyên văn nguồn
Chi tiết nguồn

Looped language models deepen reasoning at a compact parameter count by iterating a shared layer stack T times per token, turning recurrence depth directly into sequential decoding latency. Conventional early-exit decoding halts after initial passes without cache completion, truncating cached key-value states of past tokens to shallow depth. We study the opposite strategy, late entry, where new tokens skip initial passes and enter directly into the final K T alongside subsequent steps at zero extra serial calls. Projecting these finished T-pass wake states inside the existing output GEMM (Wake verification, governed by logit-margin tolerance tau in [0, inf]) unifies greedy-exact T-pass decoding (tau = 0, matching K = T block decoding within bf16 reduction order), margin-gated verification (tau > 0), and non-rollback lossy late entry (tau = inf) inside a single static CUDA graph. On Ouro-2.6B (T = 4), K = 2 Wake reaches 5.2x greedy-exact speedup at tau = 0 (+26% over K = 4 block decoding on GSM8K) and 6.8x at tau = inf (+16.9 points over trained early exit at matched speed). Scaling test-time recurrence across T in {8, 16, 32, 64} on Huginn-3.5B cuts K = T/4 rollbacks 21.5x with depth, driving greedy-exact (tau = 0) speedups to 6.1x (T = 32) and 8.4x (T = 64, +153% over K = 64 block) and tau = inf speedups to 10.6 to 12.5x.

186

Recurrent Domain Incremental Adaptation of Time-Series Foundation Models

68

5 tiếng nói · chưa gắn nhãn trưởng lab · 1 nền tảng · 3 tương tác đã đo · Nguồn crawl

Sachith Abeywickrama, Peiliang Gong và 3 nguồn khác 1 bài viết · Hugging Face Papers
Nguyên văn nguồn
Chi tiết nguồn

Pretrained Time-Series Foundation Models (TSFMs) provide strong representations for downstream classification tasks. Yet adapting a TSFM under changing operating conditions remains challenging. A model must improve on each newly observed condition without sacrificing conditions it has already learned, often from only a small labeled calibration set and without knowing the operating condition at inference time. We find that this difficulty has an important geometric structure. Under an operating-condition shift, the embedding space of a TSFM does not move uniformly: some local regions change substantially, while others remain nearly unchanged. Consequently, a global adaptation can unnecessarily move embeddings that were already correctly classified. To address this, we propose the novel PACT (Piecewise Adaptation via Chart Transport). PACT partitions the frozen source embedding space into local charts and learns lightweight residual transports for the regions affected by each new condition. An explicit identity path is always retained, and a learned gate softly selects among the identity and accumulated transports for each input. The pretrained TSFM encoder, classifier head, and previously learned transports remain fixed. With only 5% labeled calibration data per arriving condition and no condition identity at test time, PACT maintains an average macro-F1 above 0.95 across a four-condition bearing-fault stream. Under homogeneous tasks and label spaces, similar retention-adaptation behavior is observed across public benchmarks. Code is available at https://github.com/Sachithx/PACT.

187

IDEA: Intent-Dominance Evidence Audit for Inference-Time Backdoor Detection in GUI Agents

68

8 tiếng nói · chưa gắn nhãn trưởng lab · 1 nền tảng · 3 tương tác đã đo · Nguồn crawl

Zhizhuo Kou, Sizai Hou và 6 nguồn khác 1 bài viết · Hugging Face Papers
Nguyên văn nguồn
Chi tiết nguồn

Organizations increasingly deploy downloaded open-weight GUI agents. A backdoored agent passes acceptance tests on clean data and misbehaves only when a visual trigger appears on screen, turning an unvetted checkpoint into a supply-chain risk. However, existing sample-level detectors target image classifiers and CLIP-style encoders, leaving step-level backdoor detection for GUI agents unaddressed. On clean steps, the instruction drives the action; when an unreferenced screen region hijacks the step, the instruction loses control over the decision state or that region dictates it instead. We term this signature intent--evidence dominance inversion and propose IDEA (Intent-Dominance Evidence Audit) to detect backdoor activation at inference time for each step without trigger knowledge or attack data. Specifically, Intent Counterfactual Probing (ICP) measures how far redirected instructions move the decision state, and Region Causal Audit (RCA) quantifies how far ablating a candidate region moves it. A gated conformal score compares both displacements against clean steps at the same distance from the clean manifold, providing a finite-sample false-positive guarantee. Extensive experiments on Qwen2.5-VL-3B across five attack families show that IDEA consistently detects backdoor activation under a controlled false-positive rate and outperforms state-of-the-art detection methods, while localizing the trigger region for every alarm and adding only modest inference overhead.

188

DrawScore: Evaluating Generated CAD Models against Engineering Drawings

68

9 tiếng nói · chưa gắn nhãn trưởng lab · 1 nền tảng · 3 tương tác đã đo · Nguồn crawl

Haorui Chen, Shiteng Lu và 7 nguồn khác 1 bài viết · Hugging Face Papers
Nguyên văn nguồn
Chi tiết nguồn

Automatic CAD generation is commonly evaluated with volumetric IoU, Chamfer Distance, and surface F-score, which barely penalize local errors that occupy only a small fraction of a part's volume or surface area, such as a misplaced hole or a slot on the wrong face, even though such errors are critical to manufacturing. We formalize these metrics in a unified integral form and show that the weight of a local error decays as the (k+q)-th power of its size, where k is the dimension of the integration domain and q is determined by the penalty function. Guided by this analysis, we propose DrawScore, which projects both models into three orthographic views with visible and hidden lines, as in engineering drawings, and applies a saturating penalty along the projected curves, so that the weight of a local error decays only linearly. Controlled experiments confirm the predicted decay orders, and on outputs of existing CAD generation agents, DrawScore exposes errors in outputs that IoU rates as nearly perfect. DrawScore further improves generation as a reinforcement learning reward and, for CAD generation from engineering drawings, serves as a test-time verifier without 3D ground truth or an additional reward model. Our code is available at https://github.com/UESTC-Simon/DrawScore.

189

PIPETTE: Prompt Injection Purple-Teaming for Robust LLM Agents

67

4 tiếng nói · chưa gắn nhãn trưởng lab · 1 nền tảng · 2 tương tác đã đo · Nguồn crawl

Lipeng He, Yihan Wang và 2 nguồn khác 1 bài viết · Hugging Face Papers
Nguyên văn nguồn
Chi tiết nguồn

Large language model (LLM) agents routinely interact with untrusted external data, exposing them to indirect prompt injection (IPI) attacks that hijack agents into executing attacker-chosen tasks. To uncover these vulnerabilities at scale, automated red-teaming methods often leverage reinforcement learning (RL) to generate attacks that achieve high attack success rates (ASR). Model developers use the attacks discovered during internal red-teaming to harden agents through adversarial training before deployment, a collaborative offensive-defensive process known in cybersecurity as purple-teaming. However, we show that achieving high ASR during red-teaming does not guarantee robustness gains from subsequent adversarial training. Specifically, we find that two state-of-the-art RL-based red-teaming methods, RL-Hammer and PISmith, suffer from mode collapse, repeatedly discovering similar attack strategies whose additional successes add no additional robustness improvements to adversarial training. To address this gap, we introduce PIPETTE, a unified prompt injection purple-teaming framework that effectively converts red-teaming discoveries into defender robustness improvements. PIPETTE employs a dictionary learning-based reward that encourages broader coverage of attack strategies during red-teaming. It also integrates an on-policy adversarial training pipeline tailored to IPI defense, enabling agents to resist attacks while completing legitimate user tasks. Extensive evaluations across agentic benchmarks (AgentDojo and ASB) demonstrate that PIPETTE produces defenders that achieve near-zero attack success rates on static attacks and can withstand strong adaptive attacks, while maintaining benign task utility close to that of the undefended baseline.

190

Movie2Game-Bench: Holistic Evaluation of Film-to-Game Adaptation Beyond Playability

67

4 tiếng nói · chưa gắn nhãn trưởng lab · 1 nền tảng · 2 tương tác đã đo · Nguồn crawl

Jialing Xie, Yifan Zhou và 2 nguồn khác 1 bài viết · Hugging Face Papers
Nguyên văn nguồn
Chi tiết nguồn

Adapting a film into a game requires selecting source elements and translating them into player actions, rules, and feedback. We introduce Movie2Game-Bench (M2G-bench), a benchmark for evaluating film-to-game adaptation beyond playability. M2G-bench combines general evaluation, movie-specific evaluation, and playtesting by three GUI model players. The protocol connects source content to gameplay steps and implementation evidence, capturing both content preservation and interactive realization. The benchmark comprises 22 films and 658 source-specific reference items. Evaluating 14 models on the same films with their selected primary harnesses reveals complementary strengths in game quality, adaptation, and player experience. A shared weakness concerns playable sequences: this category has the lowest content-presence mean in all 14 models, averaging 37.9 out of 100 compared with 54.8 for iconic elements. Among judgments of retained emotionaldevelopment content, narrative choices account for 60.0% and scenario operation for 7.2%. These findings identify two linked priorities for film-to-game adaptation: preserving source situations that can support gameplay and connecting retained content to player actions and consequences.

191

A Foundation Model to Measure and Predict Human Behaviour

67

5 tiếng nói · chưa gắn nhãn trưởng lab · 1 nền tảng · 2 tương tác đã đo · Nguồn crawl

Haoran Ye, Tianrui Huo và 3 nguồn khác 1 bài viết · Hugging Face Papers
Nguyên văn nguồn
Chi tiết nguồn

Large language models (LLMs) are increasingly used to infer psychological attributes, predict individual behaviour, and simulate human populations. However, despite generating plausible responses, they often misrepresent individuals and distort population statistics. Here we introduce Chiron, a foundation model to measure and predict human behaviour. We jointly fine-tune an LLM to represent psychological constructs, predict individual behaviour, and align population response distributions with human statistics. The training and evaluation data span 36 behavioural domains and 78 social science experiments, comprising 288,307 person records, 1,411,750 atomic behavioural decisions, and 99,953 psychometric assessments over 109 psychological constructs. In our evaluations, Chiron's psychological measurements agree more closely with human self-reports and exhibit fewer biases that undermine psychometric validity. It predicts individual behaviour more accurately than frontier LLMs (up to 28.4% relative improvement). In simulations of social science experiments, it more accurately captures the direction and magnitude of human experimental effects (82.2--94.3% lower effect-size estimation error). Together, these results establish Chiron as a unified computational instrument for measuring and predicting human behaviour.

192

Our goal in making Solveit was not to automate our team out of the process, but to help ourselves grow. It has been so helpful to us internally, that my co-founder @ericries said we have a moral imper

64

3 tiếng nói · chưa gắn nhãn trưởng lab · 1 nền tảng · chưa có số tương tác · Nguồn crawl

jeremyphoward, dgwyer và 1 nguồn khác 0 bài viết · X
Nguyên văn nguồn
Chi tiết nguồn

Our goal in making Solveit was not to automate our team out of the process, but to help ourselves grow. It has been so helpful to us internally, that my co-founder @ericries said we have a moral imperative to share it more widely. Here it is: https://solve.it.com

193

Ask HN: Aren't the current cyber attacks suspicious?

57

1 tiếng nói · chưa gắn nhãn trưởng lab · 1 nền tảng · 34 tương tác đã đo · Nguồn crawl

arisAlexis 1 bài viết · HN: AI agent
Nguyên văn nguồn
Chi tiết nguồn

Just looking at history and having some bayesian mindset the fact that: FAA,NSA,MSFT,APPLE,EQUINOR all got hacked in a week. Isn't it obvious that some nation state or activist/terrorist group has gotten hold of an AI hacking tool/Agent that can wreak havoc? It's not business as usual anymore, it's an agentic cyber war that has started isn't it? * Edit: about bayesian, I wrote two comments below explaining the logic. I didn't even think about the next bayesian update meaning that there is a trilateral superpower meeting for the first time.

195

Launch HN: StarSling (YC P25) – Self-improving CI for agent-volume repos

60

1 tiếng nói · chưa gắn nhãn trưởng lab · 1 nền tảng · 137 tương tác đã đo · Nguồn crawl

yonasb 1 bài viết · HN: AI agent
Nguyên văn nguồn
Chi tiết nguồn

We’re Yonas and Daniel and we’re building StarSling ( https://starsling.dev ), fast GitHub Actions runners with AI agents that ship PRs to continuously improve your tests and workflows. Every CI job gets its own ephemeral sandbox on the fastest Zen 5 machines and NVMe Disk with unlimited concurrency so a burst of agent PRs all run at once. We’re different from other runners because ours come with agents that cut down your CI bill over time. PRs are generated through our CI harness, which runs an agent loop that’s tuned to improve your CI speed and reduce cost. Mastra now merges over 1,800 PRs a month and we got their slowest workflow from 29m 56s on GitHub-hosted runners down to 5m 06s (5.87x) on StarSling. We realized the true value of having an agent loop running on top of fast runners when our agents found an 11-month-old performance bottleneck in better-auth. The fix made one of their test jobs 5.4x faster: https://github.com/better-auth/better-auth/pull/10762 . I previously founded StackShare, which grew to over 1.5M developers. Daniel helped build Netflix's internal developer portal. Last year we started building AI agents for DevOps because shipping code to prod fell way behind agentic coding. We hit the CI bottleneck ourselves as we were building agents. Our own agents shipped so many PRs that at peak our job queues were backed up for hours and our CI bill got close to our inference bill. We now support per-second billing, and just shipped StarSling Disks - persistent disks that give you a 4.9x speedup in dependencies ready over actions/cache. You can see a quick demo of disks in action here : https://www.youtube.com/watch?v=2JrrpRm9Mww . Our CI harness runs a loop that has agents: - Analyze your workflows, run logs, and machine telemetry - Propose an optimization - Test your unchanged workflow side by side the proposed change - Deterministically judge that the difference is statistically significant - Ship the winner to you as a PR with benchmarks evidence We benchmarked open source repos on our runners and average 10x speedup in wall clock time with 70% less runner time: - https://github.com/starsling-demo-org/benchmark-gogs (12.8x) - https://github.com/starsling-demo-org/benchmark-shellcheck (5.5x) - https://github.com/starsling-demo-org/benchmark-chatwoot (3.9x) - https://github.com/starsling-demo-org/benchmark-transfer.sh (11.7x) - https://github.com/starsling-demo-org/benchmark-mirotalksfu (36.4x) Yes, you can do these optimizations yourself with prompts so we’ve released skills for this: https://github.com/starslingdev/skills . The benefit of using our agents is that you don’t need to spend tokens and CI compute finding optimizations yourself. We started building our runners on AWS at the beginning of the year and realized that was too expensive. But at that time there weren’t any cloud providers that could give us the same machines we were getting on AWS, billed per minute (microVMs). So we convinced a handful of sandbox providers to get Zen 5s and started benchmarking them publicly ( https://github.com/starslingdev/hpc-sandbox-benchmarks ). We’ve learned that fast infrastructure is table stakes for CI, but it’s not sufficient on its own. To get fast and cost effective CI you need a process that continues making improvements. We turned CI optimizations from OpenAI, Anthropic, Linear and others into playbooks for your agent: https://starsling.dev/ci-playbooks . To try it, give your agent this URL: https://docs.starsling.dev/start . You get 2,000 free minutes. Our agents start opening PRs once you've hit about 10,000 minutes. Today we announced our CI Challenge: if StarSling can't save you at least 10% on your next CI bill, we'll literally give you $1,000. Eligibility and rules here: https://starsling.dev/blog/ci-challenge . We’d love your feedback on the headless experience and what you’d like to see improved in your own CI setup! https://starsling.dev

196

MAI Code 2 and Code 2 Flash

65

1 tiếng nói · chưa gắn nhãn trưởng lab · 1 nền tảng · 593 tương tác đã đo · Nguồn crawl

lemonish97 1 bài viết · HN: LLM
Nguyên văn nguồn
Chi tiết nguồn
197

Introducing GPT-6.2 Astra and Sol

70

3 tiếng nói · chưa gắn nhãn trưởng lab · 2 nền tảng · chưa có số tương tác · Nguồn crawl

OpenAI, chatgpt và 1 nguồn khác 1 bài viết · OpenAI News, X
Nguyên văn nguồn
Chi tiết nguồn

Introducing GPT‑6.2 Astra and Sol, with better professional outputs, coding, computer use, and efficiency at lower cost across ChatGPT Work, Codex, and the API.

198

Not Everything Needs an LLM

62

1 tiếng nói · chưa gắn nhãn trưởng lab · 1 nền tảng · 261 tương tác đã đo · Nguồn crawl

speckx 1 bài viết · HN: LLM
Nguyên văn nguồn
Chi tiết nguồn
199

Put Gemini to work for you

62

2 tiếng nói · chưa gắn nhãn trưởng lab · 2 nền tảng · chưa có số tương tác · Nguồn crawl

Josh Woodward, joshwoodward 1 bài viết · Google AI, X
Nguyên văn nguồn
Chi tiết nguồn

<img src="https://storage.googleapis.com/gweb-uniblog-publish-prod/images/Hero_2784_x_1566.max-600x600.format-webp.webp">Tasks in Gemini is now free of charge in six countries, with new features starting in the U.S. and deeper Google Workspace integration to get things done on your behalf.

200

Show HN: TrnRT – An open-source LLM inference engine for AWS Trainium

58

1 tiếng nói · chưa gắn nhãn trưởng lab · 1 nền tảng · 62 tương tác đã đo · Nguồn crawl

sanchitmonga22 1 bài viết · HN: open weights
Nguyên văn nguồn
Chi tiết nguồn

Hi HN, Sanchit here from RunAnywhere (YC W26). We just open-sourced TrnRT: Trainium RunTime, our inference engine for AWS Trainium. We started 3 weeks back, and currently we’re able to serve and run GLM 5.3 flash (FP8) on AWS trn2.48xlarge, and will bring more models support soon. It’s open to contributions :) What it does today: - Serves 2 instances of GLM-5.3 flash on a single box, 8 trn2 with a cache aware router. - Can process 3.57M TPM (tokens per minute) on SemiAnalysis's AgentX benchmarks. - Runs decode as one NKI kernel per layer half, or for the 11 sparse-attention layers fuses both blocks into a single kernel, for faster performance. We started working on Trn2 because we wanted to explore it and see if we can come close to beating it’s performance in terms of cost efficiency compared to H100/H200. A big MoE with 1M context window such as GLM-5.3 flash requires a lot of memory, and memory/tb on a trn2.48xlarge costs less than half compared to 8x h200 on AWS, and hence we wanted to experiment with it and now we’re sharing some early results. But AWS's own stack didn't support the model: vLLM Neuron's latest release did not register any GLM model, and NxDI, AWS's own distributed inference framework, is no longer under active development. We also saw the drop of SGLang Neuron implementation yesterday, but there is no good support for Trn2, which is the only chip available for public use right now. We ran semiAnalysis's AgentX benchmarks, which replays real coding agent sessions: - 1 box served 28.9 RPM(requests per minute) with 32 agents at batch 12 (also known as concurrency) -The P50 TTFT with a cached prompt at 16 agents is ~1.24 seconds with a decode of 35.9 tps. Initially, the recurrence in one kda (linear attention) layer was ~2% of the layer's attention time at batch 1, and the rest was projections, transposes and XLA glue between small kernels. So we wrote the megakernel so that it covers the whole layer in 2 kernel dispatch, which made decode 25% faster at batch 8. The sparse attention kernel keep a layer’s working weights in SBUF, distributing weighted streaming across the cores and handling the interchip communication within There’s an approximation in the sparse-attention prefill, affecting about 1.2% of block selections. You can refer to the write-up for the profiling results: https://runanywhere.ai/blog/glm-5-3-flash-trainium2-nki-mega... Please give it a try yourself on a trn2.48xlarge and reproduce the AgentX results by following the steps in docs/RESULTS.md. Happy to collaborate and hear any feedback. Deploying high performant dedicated inference at lab like performance and half the price for open models such as the GLM-5.3 family, Kimi K3, Deepseek flash etc. is how we make money. You can talk to us regarding the same at founders@runanywhere.ai. We hope you find this useful. If you are working with Trainium or have experience with NKI kernels, we'd love to get your honest feedback. Feel free to share the models you want support for next, and we welcome any contributions to the repository.

181–200 / 213
Trang 10 / 11