本篇博文主要展示每日从Arxiv论文网站获取的最新论文列表,以自然语言处理、信息检索、计算机视觉等类目进行划分。

统计

今日共更新893篇论文,其中:

  • 自然语言处理96篇
  • 信息检索21篇
  • 计算机视觉213篇

自然语言处理

1. 【2611.26613】AgentRoboCritic: Understanding the Capability of Frontier Agents on Direct Robot Control

链接:https://arxiv.org/abs/2611.26613

作者:Yiyang Du,Yuxuan Kuang,Qitao Zhao,Yixuan Wang,Zhanqiu Guo,Yi Xu,Xin Li,Wenbin He,Liu Ren,Chenyan Xiong

类目:Robotics (cs.RO); Computation and Language (cs.CL); Computer Vision and Pattern Recognition (cs.CV)

关键词:Astra have demonstrated, Frontier multimodal language, controlling robots directly, multimodal language agents, low-level numerical actions

备注: 28 pages. Project page: [this https URL](https://adu2021.github.io/blog/AgentRoboCritic/)

点击查看摘要

Abstract:Frontier multimodal language agents such as GPT-6 Astra have demonstrated strong capabilities in controlling robots directly via low-level numerical actions without high-level primitives. However, existing benchmarks report only episode-level success rates, which show neither how these agents succeed nor where they fail. In this paper, we present a systematic action-level analysis of frontier agents on direct robot control across 12 manipulation benchmarks. We introduce AgentRoboCritic, an intent-grounded critique framework in which the policy agent states a local intent alongside each action, and a critic evaluates the action's outcome and failure mode against this intent. Our action-level analysis demonstrates the promising capabilities of frontier agents on direct robot control, including closed-loop recovery from local execution failures, reliable semantic grounding across diverse scenes, and strong zero-shot manipulation without task-specific adaptation. It also identifies their current limitations, including task-relevant physical execution bottlenecks such as grasping and alignment, and uneven recovery across failure modes. Finally, we validate our critic on a 400-action human-annotated benchmark and release a dataset of 12,844 annotated actions alongside a fine-tuned open-source critic, Muse-Glimmer-30B, whose failure-mode accuracy rises from 32.58% to 74.24%, enabling low-cost action-level analysis.

2. 【2611.26584】Represented but misread: Post-training changes how language models read out what they represent

链接:https://arxiv.org/abs/2611.26584

作者:Jochen Laubrock

类目:Computation and Language (cs.CL); Machine Learning (cs.LG)

关键词:base models, language model represents, answers, model, models

备注: 64 pages, 7+2 figures, 4+6 tables

点击查看摘要

Abstract:Does post-training change what a language model represents, or how this representation is read out into an answer? We compared linear probes with the models' own answers, analyzed with signal detection theory, in six open-weight base models and their post-trained versions. For judgments of whether German texts were written by a human or an AI, probes and word n-gram classifiers separated the classes well, but the answers did not and often ordered the texts inversely. Post-training lowered the sensitivity of the answers on average ($\Delta d' = -0.34$ across model families; -0.29 across three prompts). It also changed the use of a stated base rate: Base models shifted their log-odds by less than the stated change in prior odds, most post-trained models by more. In two models with stage-wise checkpoints, the change occurred during supervised fine-tuning, model stitching implicated the early layers, and from mid-depth on internal states at positions after the text, not those of the text, transferred the change between checkpoints. Our corpora cannot be shared, so we add a case reproducible from public data, the truth of negated statements. Here, post-training acted in the opposite direction: Probes decoded truth at or near ceiling in every model, the answers of several base models did not use it, and post-training repaired them; by mid-depth the repair resided at the positions after the statement; below it, the representation of the statement contributed, strongly in one of four models. In a training study, adding 2,000 correctly labeled authorship judgments of news texts to otherwise identical fine-tuning data improved the answers on our newspaper and textbook corpora and inverted them on the chats, where they followed a news-trained probe of the base model. Post-training thus changed how a judgment is read out of what a model represents, and a small training signal can set its direction.

3. 【2611.26514】Hybrid Retrieval for Government Scheme Search in English and Marathi

链接:https://arxiv.org/abs/2611.26514

作者:Aishwarya Gunjal,Kanchan Khemnar,Pawan Poojary,Pranav Kulkarni,Raviraj Joshi,Manish Khodaskar

类目:Machine Learning (cs.LG); Computation and Language (cs.CL)

关键词:Retrieving government schemes, formal administrative descriptions, Retrieving government, government schemes requires, Maharashtra government schemes

备注:

点击查看摘要

Abstract:Retrieving government schemes requires matching citizens' information needs to formal administrative descriptions, often across languages and terminology. We present a bilingual search system for Maharashtra government schemes and public services that combines BM25 retrieval with sentence-embedding similarity through equal-weight score fusion. The system indexes aligned English-Marathi scheme descriptions and supports multilingual, Marathi-focused, and translation-mediated semantic configurations. Evaluation on manually curated bilingual queries compares BM25, dense retrieval, and hybrid retrieval using IndicSBERT and MahaSBERT. BM25 provides a strong baseline with a mean reciprocal rank (MRR) of 0.864. Hybrid retrieval increases MRR to 0.872 with IndicSBERT and 0.879 with MahaSBERT, whereas the corresponding dense-only systems achieve 0.663 and 0.655. MahaSBERT-based fusion also increases precision from 0.781 to 0.808 while matching BM25 recall at 0.946. These results support the complementary use of lexical and semantic signals in this collection, while showing that improvements over a strong lexical baseline are modest and metric-dependent. The study provides an application-focused analysis of hybrid retrieval for bilingual public-service information access.

4. 【2611.26510】AutoMaster: Autonomous Post-Training by Making the Same Model a Better Teacher

链接:https://arxiv.org/abs/2611.26510

作者:Guoxin Chen,Shuang Sun,Fanzhe Meng,Xinyu Tang,Ruihua Song,Wayne Xin Zhao

类目:Computation and Language (cs.CL)

关键词:Task success, teacher, teacher trajectories, SFT, Abstract

备注: Website: [this https URL](https://aweai-team.github.io/AutoMaster/) Code: [this https URL](https://github.com/RUCAIBox/AutoMaster) Models: [this https URL](https://huggingface.co/collections/RUC-AIBOX/automaster)

点击查看摘要

Abstract:Task success alone does not specify which behaviors post-training should reinforce. Even successful teacher trajectories can contain unsupported assumptions, unproductive repetition, or superficial verification. We propose Skill-FT (Skill-Guided Fine-Tuning), which uses natural-language skills to suppress undesirable habits and elicit underrepresented effective behaviors during teacher rollouts, then fine-tunes a target model on the resulting trajectories. We introduce AutoMaster, an autonomous post-training system in which an LLM agent develops these interventions through trajectory analysis, skill design, data selection, and feedback from rollouts and target-model evaluation. Across Terminal, Search, and software engineering tasks with 4B and 9B target models, AutoMaster improves mean performance over Vanilla SFT on successful unguided teacher trajectories by 1.42-8.99 percentage points, with the teacher, task pool, training set size, and SFT recipe held fixed. Together with the ablations and trajectory analyses, these results suggest that behavioral shaping can make a fixed model a more effective teacher. We further find that retaining skill instructions during SFT yields higher mean performance than removing them across all six settings, despite their absence at evaluation and identical supervised outputs. These findings suggest that preserving skill context as part of a training demonstration can support better behavioral internalization.

5. 【2611.26509】GLiDE: A Fast, Calibrated Decision Model Distilled from Reasoning Teachers

链接:https://arxiv.org/abs/2611.26509

作者:Sahibzada Allahyar,Julia White,Dhruv Atreja,George Hurn-Maloney,Urchade Zaratiana

类目:Artificial Intelligence (cs.AI); Computation and Language (cs.CL)

关键词:single forward pass, forward pass, scoring candidate answers, text or intermediate, models enable fast

备注: 18 pages, 8 figures

点击查看摘要

Abstract:Decision models enable fast, confidence-aware decisions by scoring candidate answers in a single forward pass, without generating text or intermediate reasoning. Yet reasoning can substantially improve decision quality, raising the question of whether its benefits can be retained without the cost of generating it. We investigate this question with \textbf{GLiDE}, a family of decision models spanning 0.4B to 27B parameters. Building on Qwen hybrid language models, GLiDE enables bidirectional interactions between the context, question, and candidate answers by removing causal masking from full-attention layers. To transfer reasoning capabilities into single-pass inference, we combine soft-target distillation from multiple teachers, including a reasoning model, with progressive post-training on diverse decision tasks. We further explore parameter-free recurrent depth to increase inference-time computation without generating tokens. Across five model sizes, GLiDE lies on the performance--parameter Pareto frontier of the Hugging Face \DI{}, with each model outperforming evaluated alternatives of equal or smaller size. \dmtwo{} (27B) achieves a hidden-set score of 71.52, with an expected calibration error of 2.0\% and a median latency of 92.9\,ms on a single GPU. Further checkpoint averaging and parameter-free recurrent depth yield GLiDE~27B, which reaches 73.37 with an expected calibration error of 2.1\%.

Comments:
18 pages, 8 figures

Subjects:

Artificial Intelligence (cs.AI); Computation and Language (cs.CL)

Cite as:
arXiv:2611.26509 [cs.AI]

(or
arXiv:2611.26509v1 [cs.AI] for this version)

https://doi.org/10.48550/arXiv.2611.26509

Focus to learn more

              arXiv-issued DOI via DataCite (pending registration)</p>
6. 【2611.26492】Multimodal Error Detection in Chest X-ray Reports: A Large Language Model Reading Image Evidence

链接:https://arxiv.org/abs/2611.26492

作者:Yanda Cheng,Bo Wen,Yuhao Lu

类目:Computation and Language (cs.CL)

关键词:checks chest X-ray, chest X-ray reports, flag potential errors, chest X-ray, X-ray reports

备注: 34 pages, 4 figures, including supplementary material

点击查看摘要

Abstract:Rad-Linter checks chest X-ray reports against image-derived evidence to flag potential errors for review. A pretrained image classifier supplies scores for nine pathologies, and a 1.5B-parameter language model, fine-tuned with LoRA on 1,816 examples, checks their consistency with the report. We compare it with a keyword-and-negation rule baseline given the same reports and scores. Evaluation uses controlled report-score pairings from OpenI, with consistency labels derived from automatic report labels and score thresholds. Each counterfactual quad crosses two real reports with two real score vectors from other patients, creating two consistent and two inconsistent pairs. Every report and score vector occurs in both conditions, so either input alone permits only 50% accuracy. On 131 quads, Rad-Linter achieves 98.5% pair accuracy versus 80.5% for the rule baseline, and classifies all four pairs correctly in 94.4% of quads versus 56.5%. The checkers perform similarly on omission items, including unmentioned and negated findings; larger differences occur on the remaining item types, which require interpreting how findings are described. The same language model without fine-tuning remains near 50% accuracy. On a second set of 181 quads that also includes reports on which the automatic labelers disagree, pair accuracy is 93.4% versus 75.3%. These controlled results show that, given the same image-derived scores, a trained language model detects report-score inconsistencies more accurately than the tested rule baseline, with the advantage concentrated in interpreting how findings are worded.

7. 【2611.26427】MemFolder: Structure-Guided Memory Folding for Efficient Agent Memory Management

链接:https://arxiv.org/abs/2611.26427

作者:Xuyang Liu,Chenwen Lu,Junxi Wang,Siyu Ren,Xinying Lin,Siteng Huang,Hongli Sun,Linfeng Zhang,Honggang Chen

类目:Computation and Language (cs.CL); Computer Vision and Pattern Recognition (cs.CV)

关键词:Long-term agents accumulate, agents accumulate large, accumulate large external, Long-term agents, large external memory

备注: Code is available at [this https URL](https://github.com/xuyang-liu16/MemFolder)

点击查看摘要

Abstract:Long-term agents accumulate large external memory stores as interactions unfold, increasing the costs of storage, retrieval, and response generation. Efficiency is constrained by two coupled factors: a growing record count increases retrieval overhead, while longer retrieved contexts increase prefill computation and decoding-time attention costs. Existing methods prune or consolidate records, but pruning can discard useful evidence, whereas consolidation may produce fewer yet longer retrieval units. We introduce MemFolder, a structure-guided memory folding framework that sequentially reduces record count and context length. The framework comprises two core components that share memory groups, using cross-record structure to guide both evidence consolidation and compact representation. RecordFolder grounds entity mentions, progressively selects representative yet complementary anchors, and combines semantic and entity affinity with temporal chaining to consolidate compatible records into fewer retrieval units. ContextFolder encodes the original records within these groups into compact text and uses verification feedback to revise encodings when key facts cannot be recovered. Experiments on the three text-based memory benchmarks LoCoMo, PersonaMem, and LongMemEval and the two multimodal benchmarks M3-Bench-web and M3-Bench-robot demonstrate the strong performance of our methods. Notably, on Mem0, RecordFolder retains 99.0% of the original average performance across the three text-based benchmarks with 50% fewer records, exceeding the strongest baseline by 1.9 percentage points in performance retention. Combined with ContextFolder, existing record compression methods further shorten their output records while retaining most of their performance.

8. 【2611.26398】Democratizing Cross-Project Knowledge for Transforming Water Governance and Climate Resilience: an Agentic AI Ecosystem

链接:https://arxiv.org/abs/2611.26398

作者:Oriol Alàs,Ian Palacín,Iván Cester,Edgar Rubión

类目:Information Retrieval (cs.IR); Artificial Intelligence (cs.AI); Computation and Language (cs.CL)

关键词:projects produce extensive, produce extensive evidence, form of deliverables, case studies, EU-funded projects produce

备注: 8 pages, 2 figures. Accepted at WDSA/CCWI 2026. Proceedings publication pending

点击查看摘要

Abstract:EU-funded projects produce extensive evidence in the form of deliverables, reports, and case studies. However, this knowledge is typically scattered across projects and document structures, making it difficult to locate relevant, contextual information when making decisions. This paper introduces an Agentic AI Ecosystem that consolidates heterogeneous project documentation into a domain-aware knowledge base and provides adaptive access through an agentic Retrieval-Augmented Generation pipeline. Documents from three EU-funded projects (IMPETUS, NATALIE, and InnWater) are reconstructed into hierarchical section trees and indexed with structured metadata. Retrieval is supported by two domain-specific embedding models fine-tuned on domain data (water and climate). On top of this knowledge base, a ReAct-style controller iteratively orchestrates retrieval actions by allocating retrieval budget across domain collections and accumulating evidence until a stopping criterion is met. In a retrieval benchmark over the corpus, the proposed approach improves Mean Reciprocal Rank by 26.7% (relative) compared to a single-domain embedding baseline, indicating more consistent retrieval of relevant evidence to support cross-project knowledge synthesis and climate-resilient decision-making.

9. 【2611.26363】HiSA: Sparsifying the Hidden Dimension of RoPE Attention in LLMs

链接:https://arxiv.org/abs/2611.26363

作者:Ambroise Borbely,Mathieu Salzmann,Olivier Verscheure

类目:Machine Learning (cs.LG); Computation and Language (cs.CL)

关键词:large language models, dot product, full dot product, language models, Attention

备注:

点击查看摘要

Abstract:Attention in large language models (LLMs) scores every query against every key with a full dot product. We investigate whether this dot product must be computed completely for attention to be effective. We propose Hidden Sparse Attention (HiSA), which keeps the query and key projections intact but only computes a partial dot product, following a static, parameter-free rule set by the token positions: attention becomes sparse in the hidden (feature) dimension, an axis much less explored than token sparsity, rather than over tokens as in almost all sparse attention methods. Based on the structure of rotary position embeddings (RoPE), we decompose keys and queries into parts which can be omitted when computing attention. By training LLMs from scratch with HiSA, we find that this partial dot product does not hurt but improves performance compared with the complete product while computing only about a third of the query-key products. During inference HiSA improves decoding speed and shrinks memory footprint. We also extend HiSA to FlashAttention-3 kernels for training and inference. Finally, we motivate our approach by showing that dense attention learns to suppress the attention score parts that HiSA drops. Code is available at this https URL.

10. 【2611.26358】Beam Model Technical Report

链接:https://arxiv.org/abs/2611.26358

作者:Reflection Team:Adam Abdulhamid,Osama Abuelsorour,Yasmin Alameddine,Carlo Alfano,Duarte Alves,Brandon Amos,Selvaraj Anandaraj,Arseni Anisimovich,George Antoniadis,Ioannis Antonoglou,Sheila Babayan,Neeraj Bahl,Märt Bakler,Vidhisha Balachandran,Cameron Banks,Thibaut Barroyer,Navroop Bath,Irwan Bello,Jeremy Berman,Fernanda Bernaldo,Mrityunjay Bhadauria,Sanyam Bhutani,Tolga Bolukbasi,Anudhyan Boral,Joseph Brennan,Ethan Brooks,Sam Buchanan,Anton Bulyenov,John Cai,Catherine Cang,Eason Chan,Elijah Chanakira,Nathan Chandrasekaran,Hao Chen,Hong-You Chen,Jeffrey Chen,Saurav Chhatrapati,Ting-Rui Chiang,Brian Tiger Chow,Aakanksha Chowdhery,Timothy Chung,Rob Claire,John D. Co-Reyes,Anthony Conte,Eduardo Coronado-Montoya,Julien Crozon,Drew Csillag,Joe Cummings,Jeff Da,Samira Daruki,Tim Dawson,Charlie Deck,Arash Delijani,Ousmane Dia,Randy Dodgen,Ryan Douglas,Omar El-Kishky,Ben Ellis,Gamaleldin Elsayed,Simon Fan,Patrick Fernandes,Gabriel Ferns,Nick Fram,Ankush Garg,Florent Gbelidji,Iaroslav Geraskin,Behrooz Ghorbani,Ivan Glushkov,Neil Gollapudi,Scott Goodfriend,Yoni Gozlan,Christopher Grimm,Suriya Gunasekar,Shruti Gupta,Joel Gustafson,Emre Guven,Jon Hackmann,Jeremy Hadidjojo,Jess Hamrick,Ethan Hellman,Joscha Hoche,Spurthi Amba Hombaiah,Uzair Honda,Zijian Hu,Dzmitry Huba,Nancy Hung,Vladimir Ivanov,Shomik Jain,Mojan Javaheripi,Kevin Martin Jose,Aymen Kallala,Amog Kamsetty,Min Hyung Kang,Steven Kapturowski,Ramtin Keramati,Samuel Kessler,Sabina Kim,Seonghyeon Kim,Ivan Komarov

类目:Computation and Language (cs.CL); Artificial Intelligence (cs.AI)

关键词:total parameters, active parameters, optimized for coding, agentic workloads, Beam

备注:

点击查看摘要

Abstract:Beam is a sparse Mixture-of-Experts (MoE) model with 501.3B total parameters and 23.4B active parameters, specifically built and optimized for coding, reasoning, and agentic workloads. Beam delivers frontier level reasoning, while requiring 3-4x less inference compute than many models of its class. Pretrained on 23.8T diverse, curated, high-quality tokens, Beam features a 1M-token context window achieved through a novel two-stage midtraining phase, built on verifiable, correctness-anchored data. Post-training leveraged a large-scale, fully asynchronous reinforcement learning (RL) campaign across 10.5K GB300 GPUs, generating over 100M rollouts over 4 weeks of training. We believe this is one of the largest scale RL runs conducted by any open lab to date. Across our evaluation suite, capabilities continued to improve as we increased RL compute, with no sign of a plateau. Supported by deep hardware-model co-design, FP8/NVFP4 quantization, and a 1.93B-parameter speculative decoding drafter (DFlash 2), Beam maximizes serving throughput with efficient, flexible deployment. Averaged across conversation turns and harm categories, Beam refuses or pivots on 88.9% of harmful requests while complying with 81.5% of harmless requests, achieving the highest refusal/pivot rate on harmful requests among open-weight models on the Pareto frontier.

11. 【2611.26336】One Framework to Diagnose Them All: A Stratified Benchmark for Bengali LLM Hallucination

链接:https://arxiv.org/abs/2611.26336

作者:Asif Shahriyar Sushmit,Shefayat E Shams Adib,Abdullah Al Jubaer Gem,Ahmed Shafin Ruhan,MD. Nazmuddoha Ansary,Biplab Kumar Sarkar,Syed Rifat Raiyan,Ajwad Abrar,Abdullah Bin Hasan,Adib Sakhawat,M. Sakeef Hossain,Ahmed Alfey Sani,Arefin Mizan,Sourab Chakma,Quazi Adibur Rahman Adib,Wassie Haque,Sanjay Kumar Sen,Saad Tahmid,Mohammad Mamun Or Rashid

类目:Computation and Language (cs.CL); Artificial Intelligence (cs.AI)

关键词:languages fragments easily, spending scarce annotators, low-resource languages fragments, newly named failure, named failure mode

备注: Accepted to Findings of the Association for Computational Linguistics: EACL 2027, 25 pages, 8 figures, 23 tables. Code: [this https URL](https://github.com/Alikbachan/Alikbachan) | Data: [this https URL](https://huggingface.co/datasets/Alikbachan/Alikbachan) | Competition: [this https URL](https://www.kaggle.com/competitions/bengali-hallucination/) | Project page: [this https URL](https://alikbachan.github.io)

点击查看摘要

Abstract:Hallucination research in low-resource languages fragments easily: each newly named failure mode prompts its own dataset, spending scarce annotators on incomparable resources. We argue for consolidation and test it at community scale. Alikbachan is a Bengali hallucination-detection benchmark stratified along three crossed axes: a human-validated cultural-distance band, six task types, and a five-family, 32-mode hallucination taxonomy. It comprises a harvested 2,516-item test set (Phase 1) and a quota-constructed, gold-labeled 5,000-item set (Phase 2). The stratification is diagnostic. Four zero-shot LLM judges vary by over 30 macro-F1 points across strata, and the item-level predictions of 99 Phase-1 and 32 Phase-2 competition systems reproduce the profile: discourse-level omission defeats judges and most teams alike, whereas only a minority over-flag negative controls. The judges' verdict probabilities rank items well (ROC-AUC 0.74-0.95) but are overconfident (ECE 0.04-0.29). An open competition on Alikbachan drew 222 teams and 3,566 submissions, and a single-prompt 32B judge reaches the 57th percentile. Of the 25 teams in both phases, 24 regress on Phase 2 (mean $-17.6$ points) while every zero-shot judge improves, and Phase-1 rank barely predicts Phase-2 rank (Spearman $\rho=0.35$), so a Phase-1 leaderboard alone would have crowned a different winner.

12. 【2611.26317】Language-Oriented Semantic Modeling for Target Language Extraction

链接:https://arxiv.org/abs/2611.26317

作者:Mehmet Sinan Yıldırım,Ke Zhang,Shuai Wang,Haizhou Li

类目:Computation and Language (cs.CL); Audio and Speech Processing (eess.AS)

关键词:Target Language Extraction, Target Language, aims to isolate, Target, language identity labels

备注: Published in APSIPA ASC 2026

点击查看摘要

Abstract:Target Language Extraction (TLE) aims to isolate speech in a target language from multilingual mixtures. Existing approaches primarily rely on acoustic modeling with language identity labels, which provide only coarse categorical guidance. In this work, we propose a language-oriented semantic modeling framework that augments acoustic extraction with language-specific semantic representations. A pre-trained speech semantic model is adapted via language-aware prompt conditioning and parameter-efficient fine-tuning to generate target-language-focused semantic representations from the mixture. {These capture high-level semantic structure of the acoustics and they} are integrated into the extraction model to provide structured semantic guidance beyond identity labels. Experiments on the CommonVoiceMix dataset show that incorporating semantic representations consistently improves over acoustic-only baselines, achieving up to 1.16 dB SI-SDR gain.

13. 【2611.26294】BADAS-3.0: Real-Time Driving Perception and Planning from One Camera

链接:https://arxiv.org/abs/2611.26294

作者:Roni Goldshmidt,Hamish Scott,Lorenzo Niccolini

类目:Computer Vision and Pattern Recognition (cs.CV); Computation and Language (cs.CL)

关键词:real-time latency, current speed, Abstract, camera, latency

备注:

点击查看摘要

Abstract:We ask how far a driving model can get with one camera and real-time latency. BADAS-3.0 is a 1B-parameter video encoder with four small heads. From two seconds of one forward camera, it predicts the current speed, the metric path just driven, a collision-risk score and six candidate futures over 6.4 seconds, in under 50 ms. Given the car's current speed, BADAS-3.0 has 44% lower minADE$_6$ over five trajectory benchmarks than the 34B Alpamayo 2 Super, which reads six cameras and runs over 100x slower. The gain comes from camera rigs that neither model was trained on. On the rig they share, the two are on par. Existing benchmarks are mostly nominal driving, so we add two open-loop safety benchmarks, one of real dashcam collisions and near-misses and one of Euro NCAP scenarios. On both, BADAS-3.0 has a higher pass rate than every published planner we tested.

14. 【2611.26286】wice-Told Critiques: Claim-Level Homogenization of Peer Review in the LLM Era

链接:https://arxiv.org/abs/2611.26286

作者:Ilya Gusev

类目:Computation and Language (cs.CL); Artificial Intelligence (cs.AI); Digital Libraries (cs.DL)

关键词:Peer review relies, Peer review, readers finding, reviews, Peer

备注:

点击查看摘要

Abstract:Peer review relies on several readers finding different problems. We measure critique overlap in conference and journal reviews. At ICLR, measured overlap rose from 12.0% in 2026 to 21.3% in 2027, in reviews from the same stage of the review process, with year-on-year rises at NeurIPS, ICML and the F1000 and PLOS journals. A third ICLR review of a paper added 5.46 new problems to the other two in 2027, against 6.12 in 2026. We apply MELD, a detector of LLM-generated text, to all reviews. Within a paper, two reviews that MELD flags overlap more than two unflagged reviews: by 3.0 points at ICLR in 2024-2025, by 4.5 points in 2026-2027, and by 5.4 points across five journal sources. These results provide evidence that LLM-flagged reviews of one paper are less distinct from each other than unflagged reviews, and suggest that the diversity of critiques among reviews of a single paper is falling.

15. 【2611.26282】What If Chain-of-Thought Is Not Enough?

链接:https://arxiv.org/abs/2611.26282

作者:Marc Jansen

类目:Computation and Language (cs.CL); Artificial Intelligence (cs.AI)

关键词:internalized model behaviors, evolved from prompting-based, prompting-based methods, methods to internalized, Reasoning capabilities

备注:

点击查看摘要

Abstract:Reasoning capabilities in large language models have evolved from prompting-based methods to internalized model behaviors. In this paper, we investigate whether the internal thinking model of an LLM can be deliberately changed and whether such a change is beneficial. Instead of relying on Chain-of-Thought, we train models to follow Question--Option--Criteria (QOC), a structured decision process that decomposes a task into explicit questions, competing options, and evaluation criteria. We present a three-generation development pipeline that ranges from a basic QOC decision core to richer answer synthesis and statistically enriched QOC reasoning. We then evaluate the resulting models with respect to both internal consistency and output quality. Across the reported experiments, QOC-based fine-tuning improves answer quality relative to non-fine-tuned base models and yields particularly strong gains once models are large enough to execute the underlying QOC computations reliably. The results suggest that the internal thinking model of an LLM is trainable and that structured alternatives to Chain-of-Thought can provide a promising path toward more reliable, interpretable, and task-adapted decision support.

16. 【2611.26275】dLoop: Looped Diffusion Language Models via Recurrent Latent Refinement

链接:https://arxiv.org/abs/2611.26275

作者:Handa Li,Pengxiang Li,Mingyu Cao,Xilu Wang,Li Shen,Da Li,Lu Yin

类目:Computation and Language (cs.CL)

关键词:support parallel generation, support parallel, Diffusion language, Diffusion language models, diffusion language model

备注:

点击查看摘要

Abstract:Diffusion language models (DLMs) support parallel generation, yet they often require many denoising steps because only a fraction of token predictions are sufficiently reliable to be committed at each step. Existing acceleration methods primarily optimize confidence learning or commitment policies. However, these approaches operate on the outputs of a single denoising forward pass, in which information from the prompt, previously committed tokens, and uncommitted positions may not yet be fully integrated. Consequently, some predictions remain under-refined when the commitment decision is made, and modifying the commitment policy alone cannot improve their underlying representations. We introduce dLoop, which restructures a pretrained diffusion language model into a Prelude-Loop-Coda architecture and recurrently applies a parameter-shared intermediate block to the same partially masked state. Rather than advancing the denoising process, each additional loop provides further latent computation, allowing information distributed across uncommitted positions to continue interacting and become more fully integrated before commitment. Experiments on GSM8K, MATH, MBPP, and HumanEval show that dLoop achieves the highest or tied-highest accuracy on every benchmark. With tokens per forward pass (TPF) adjusted for loop computation, it outperforms Fast-dLLM in both accuracy and TPF on all four benchmarks and dParallel on three, and is more accurate than d3LLM on all four, while d3LLM attains higher TPF; no compared method surpasses dLoop in both. Mechanistic analyses show that extra loops mainly help on slow, unstable trajectories and that updates at uncommitted positions drive changes in token decisions and later generation. Pre-commitment latent computation thus makes predictions more reliable, offering a route to DLM acceleration distinct from forcing confidence through the training loss.

17. 【2611.26255】STAGE: Spatio-Temporal Augmented Graph Expansion for Weakly Supervised Cross-Tweet Event Causality Discovery

链接:https://arxiv.org/abs/2611.26255

作者:Hieu Minh Duong,Todd Gary,Eugene Levin,Long Nguyen

类目:Computation and Language (cs.CL); Artificial Intelligence (cs.AI)

关键词:ten span independent, span independent posts, disaster-related social media, causal discovery difficult, ten span

备注:

点击查看摘要

Abstract:Causal relations in disaster-related social media of- ten span independent posts, making cross-tweet causal discovery difficult to supervise and model at scale. We propose STAGE, a framework that separates scalable cross-tweet weak-supervision construction from spatio-temporal relational learning over event occurrences. Starting from local causal seeds, STAGE retrieves semantically related event occurrences across tweets and uses a causal scorer to construct weakly labeled cross-tweet pairs. These pairs supervise a multi-head Graph Attention Network over semantically enriched event representations and a spatio- temporal context graph. We evaluate STAGE on California Wildfire and Hurricane Harvey using human-gold cross-tweet test sets and 15 baselines. Compared with local-only supervision, the constructed cross-tweet supervision improves STAGE AUPRC by 0.340 on Wildfire and 0.107 on Hurricane. STAGE achieves in-domain AUROC of 0.885 on Wildfire and 0.710 on Hurricane. Cross-disaster experiments reveal asymmetric transfer, highlighting sensitivity to dataset-specific spatio-temporal shifts. These results demonstrate the value of scalable cross-tweet supervision combined with relational event modeling for causal learning across fragmented disaster reports.

18. 【2611.26248】winLab-Bench: Evaluating AI Scientists in Agentic Twins of Real Labs

链接:https://arxiv.org/abs/2611.26248

作者:Yang Wan,Linchao Zhu

类目:Artificial Intelligence (cs.AI); Computation and Language (cs.CL); Machine Learning (cs.LG)

关键词:increasingly open abilities, computational workflow correctly, increasingly expected, increasingly open, workflow correctly

备注: [this https URL](https://twinlab-bench.github.io/) contains tasks and code. 33 pages, 6 figures, 14 tables. LLM-agent benchmark for autonomous experimental design in digital twins of real labs (physics, chemistry, wet-lab biology, materials, engineering). 13 agents evaluated: Claude Fable 5.2 45.0, Opus 5.5 40.2, GPT-6.1-sol 31.7, plus DeepSeek, Qwen, Kimi, GLM and others

点击查看摘要

Abstract:AI agents are increasingly expected to act as AI scientists, which requires three increasingly open abilities: carrying out a given computational workflow correctly, choosing the next measurements from a given design space, and designing their own experiments. Existing benchmarks mainly evaluate the first two abilities, in a terminal or in a closed loop that selects from predefined candidates. Evaluating the third ability requires reliably executing an experiment that an agent designs, with readings grounded in a real laboratory. We introduce TwinLab-Bench, which evaluates AI scientists on self-designed experiments in 52 tasks from laboratories in the physical sciences, the life sciences, and engineering. The benchmark supports more than 100 experimental action types, and the agent's report is scored by comparison with a hidden answer taken from real laboratory records. Each task has its own TwinLab agent, an agent that serves as the environment and executes the experiments that the tested agent designs, reading the raw materials of the laboratory and calling calculators that are built from its records. Compared with a language model that sees only the task description, the TwinLab agent almost eliminates readings that deviate from the real records and inconsistent answers to a repeated request, and closed-book probes control for answers recalled from memory. We find that these tasks remain hard for frontier models: the strongest, Claude Fable 5.2, averages a score of only 45.0, and we conduct an empirical study of where it fails.

19. 【2611.26216】Frontier models need only one text to unmask its human or AI author

链接:https://arxiv.org/abs/2611.26216

作者:Atrey Desai,Deniz Böloni-Turgut,Ty Brassington,Minjoon Choi,Jenna Russell,Rishanth Rajendhran,Bradley Emi,Hila Gonen,Hal Daumé III,John Wieting,Marzena Karpinska,Mohit Iyyer

类目:Computation and Language (cs.CL)

关键词:simple zero-shot prompt, increasingly specific levels, candidate lists, human wrote, zero-shot prompt

备注:

点击查看摘要

Abstract:Given a single text and a simple zero-shot prompt (no web access, tools, candidate lists, or writing samples), frontier models can answer "who wrote this?" at three increasingly specific levels: (1) whether an AI or human wrote it, (2) if an AI, which lab, and (3) if a human, often exactly who. The latest models (Claude Fable 5.2, GPT-6.1 Sol, Gemini 4 Carbon) match trained detectors like Pangram while also explaining their predictions in natural language, and Opus 5.5 names the correct lab for 98% of texts generated by models from Anthropic, OpenAI, and Google. Most strikingly, Fable 5.2 correctly names the author of 39% of public peer reviews published in Oct-Nov 2026 (78% on a subset of reviews from Nature), and identifies the username behind 69% of comments from active Redditors (with 500 posts over the last year). Accuracy depends on exposure to an author's writing in pretraining data and on reasoning effort, and simply asking a model how familiar it is with someone predicts whether it can identify their writing. For anyone who has published enough writing online, frontier models present a greater threat to anonymity than commonly assumed, and defenses are limited.

20. 【2611.26199】Learning to Think with Images Faithfully via Adaptive Data Selection

链接:https://arxiv.org/abs/2611.26199

作者:Yu Zhao,Jiaxin Zhang,Rohit Saxena,Fazl Barez,Pasquale Minervini,Yiwen Guo

类目:Computer Vision and Pattern Recognition (cs.CV); Computation and Language (cs.CL)

关键词:overcome perceptual bottlenecks, empowers vision-language models, Thinking with images, empowers vision-language, promising both higher

备注:

点击查看摘要

Abstract:Thinking with images (TwI) empowers vision-language models (VLMs) to overcome perceptual bottlenecks by actively cropping and zooming into fine-grained visual details, promising both higher accuracy and interpretable, region-grounded reasoning. However, our analysis shows that the improved accuracy of TwI models does not guarantee faithful tool use; they frequently invoke tools without actually relying on the zoomed region, undermining the reliability of tool-assisted reasoning. We trace this unfaithful behaviour to the training data: when questions are answerable without zooming, the model can answer correctly while ignoring the returned evidence, and this problem is exacerbated during training as the model becomes better at answering without tools. To keep zooming useful throughout training, we adapt the training data to the model's evolving capability and train on questions where correct zoomed evidence can still help the model answer correctly. This selection is supported by a data synthesis pipeline that localises small objects and constructs verifiable questions about their fine details, and complemented by an adaptiveness reward that favours tool-free answers when zooming is unnecessary. The experimental results show that our method substantially improves reasoning faithfulness and achieves state-of-the-art performance on visual search benchmarks (+11.6% on V*, +7.9% on HR-Bench 4K, and +9.4% on HR-Bench 8K). On multimodal hallucination benchmarks, our model also outperforms the compared TwI baselines. These results suggest that improving TwI faithfulness does not sacrifice accuracy and that both can be learnt from the right training data without directly supervising the reasoning process.

21. 【2611.26145】Can World Models Predict Rice Growth? Toward Agricultural World Models

链接:https://arxiv.org/abs/2611.26145

作者:Wenkai Ji,Zeteng Li,Yinkun Guo,Jie Zhou,Liang He,Zhenong Jin

类目:Computer Vision and Pattern Recognition (cs.CV); Computation and Language (cs.CL)

关键词:physical scene evolves, World models, models can imagine, physical scene, scene evolves

备注:

点击查看摘要

Abstract:World models can imagine how a physical scene evolves over the next few seconds, but growers need to know how a crop will develop over weeks to months: will this rice plant have headed a month from now, and how will it look at harvest? We introduce agricultural world modeling, the task of predicting how the same plant will look at a requested horizon of days to months, from one photo of it, its cultivar, and its camera view. In preliminary analysis, even state-of-the-art image generators get this wrong: they often grow the plant too little or give it the color and form of the wrong growth stage. We argue that they do not work out the plant's developmental state, which the photo reveals only indirectly. We therefore propose AgroWM, which reasons about development before it draws growth. Its reasoning module, PhenoClock, works out the current growth stage, predicts the future one, and writes a Growth Script with numeric targets for height, plant area, and color; its generation module, the Rooted Renderer, then edits the photo to follow this script, adding new organs and colors to the photographed plant. Because pixel scores can prefer an unchanged copy of the photo, we instead check each forecast's traits and developmental timing against the plant's own later photos. On rice, 54.6% of the forecasts of AgroWM are matched to the target stage (Stage R@1), against 34.8% for GPT Image 2.5, 24.0% for Qwen Image 2.1, and 47.4% for the strongest domain-specific baseline, the same backbone fine-tuned without Growth Scripts; 9 to 20 weeks ahead, it still reaches 46.4%, against 35.7% for that baseline. On maize, a crop with a different body plan and stage scale, the same design again scores best on all four measures.

22. 【2611.26084】RL Can Only Reinforce What It Sees Succeed

链接:https://arxiv.org/abs/2611.26084

作者:Ruben Sanchez-Corcuera,Unai Zulaika,Aritz Bilbao-Jayo,Miguel Fernandez-de-Retana

类目:Machine Learning (cs.LG); Computation and Language (cs.CL)

关键词:rule stops working, familiar rule stops, stops working, familiar rule, rule stops

备注: 16 pages, 6 figures, 1 table

点击查看摘要

Abstract:When a familiar rule stops working, an agent has to weigh what pre-training taught it against what it now observes. We ask how outcome-reward reinforcement learning (RL) changes this balance, in a puzzle world where both sides are known: small language models trained from scratch on games played by a Bayesian player, so that the prior and the right answer are exact. Base models hedge as their corpus did, and RL turns that hedging into decisions, in whichever direction the reward favours. But RL can only reinforce what it sees succeed. When the right action is rare, each run either learns it everywhere or not at all, and which runs take off is predicted before training by one count: the expected number of groups of attempts in which the action wins. About 35 such groups give even odds. The count holds for new and repeated problems and for groups of 4 to 256 attempts, and across learning rates, optimizers, GRPO variants and model sizes from 1.3M to 85M parameters, which move the threshold but not what is counted.

23. 【2611.26077】Same Judgement, Different Scores: Numerical Scoring Topology in LLM-as-a-Judge Evaluation

链接:https://arxiv.org/abs/2611.26077

作者:Doyun Kim,Chanwoo Kim,Jiwoo Kum,Minsoo Song,Chanjun Park

类目:Computation and Language (cs.CL)

关键词:numerical scoring topology, LLMs are increasingly, human annotation, numerical scoring, automated evaluators

备注: Accepted to Findings of EACL 2027

点击查看摘要

Abstract:LLMs are increasingly used as automated evaluators to reduce the cost and time required for human annotation. However, the role of scoring-scheme design in the reliability of LLM-as-a-Judge evaluation remains underexplored. To address this problem, we introduce numerical scoring topology, which characterizes the structural design of a scoring scheme in terms of its scale width, granularity, offset, symmetry, and boundary placement. Building on this definition, we conduct the first systematic study of how numerical scoring topology influences zero-shot LLM-as-a-Judge evaluation across diverse benchmarks and judge models. Our results show that numerical scoring topology systematically affects judge scores, calibration, and robustness, particularly in judgment-intensive evaluations. Shifted and narrow ranges tend to induce some of the largest score deviations across many settings. We further find that stronger and larger judge models generally achieve higher discriminative validity and greater robustness to scoring-topology changes. These findings suggest that numerical scoring topology should be treated as an integral component of LLM-as-a-Judge evaluation and highlight the need for more principled and standardized scoring protocols.

24. 【2611.26075】Understanding MoE Decisions Beyond Gradients

链接:https://arxiv.org/abs/2611.26075

作者:Zhen Xu,Zitao Zhang,Kevin Zhao,Xuefeng Liu

类目:Computation and Language (cs.CL); Machine Learning (cs.LG)

关键词:decision requires knowing, Understanding a language, internal computations affect, language model decision, model decision requires

备注:

点击查看摘要

Abstract:Understanding a language model's decision requires knowing how its internal computations affect its preference between answers. Gradient-based methods estimate this influence by predicting the effect of changing those computations. In mixture-of-experts (MoE) models, replacing one expert can change which experts later layers select. Those switches can alter the answer scores in ways that derivatives do not capture. We introduce a predictor that uses the model's own expert outputs and selection rules to estimate these extra effects, then adds them to a second-order prediction. Across questions from three benchmarks, our method reduces the error in predicting answer-score changes by 60\% on Qwen3.6 and 47\% on Gemma4, relative to a multistep baseline using gradients and second derivatives.

25. 【2611.26073】AltRA-Test: An Alternative Annotator Test for LLM Judges Without Shared Annotators

链接:https://arxiv.org/abs/2611.26073

作者:Nagham Omar,Rotem Dror,Almog Gueta,Avigdor Gal,Roi Reichart,Nitay Calderon

类目:Computation and Language (cs.CL); Artificial Intelligence (cs.AI)

关键词:Large language models, Alternative Annotator Test, Large language, Alternative Annotator, replace human annotators

备注:

点击查看摘要

Abstract:Large language models (LLMs) are increasingly used as judges and graders to replace human annotators, raising the question of when such replacement is statistically justified. To support such decisions, the Alternative Annotator Test (Alt-Test) was introduced for dense settings where all items are annotated by the same annotators. However, in many large-scale annotation settings, items are annotated by different subsets of annotators. We therefore propose the Alternative Annotator Test with Resampled Annotations (AltRA-Test), a statistical test that does not require that all items are annotated by the same annotators. AltRA-Test uses a bootstrap that resamples items, draws k human annotations for each item, and substitutes one of them with an LLM annotation. This produces paired all-human and mixed human-LLM panels. A replacement is supported when the inter-annotator agreement (IAA) of the mixed panel is statistically non-inferior to the IAA of the all-human panel, i.e., when the LLM preserves the signal that the human annotators share. We establish that AltRA-Test is a consistent test of asymptotic level alpha. We then evaluate it through simulations and real-world experiments in which we test 18 LLMs and 5 prompting techniques on 20 datasets across text, image, audio, and video. Across these evaluations, about 100 items often suffice for the test. In simulations, where we know whether the LLM is a better or a worse annotator than the humans, AltRA-Test makes more correct replacement decisions than existing baselines. In real-world data experiments, LLMs pass the test in about half of the setups, depending mainly on the task. Since replacement is not always justified, we encourage researchers to test it for their task.

26. 【2611.26069】SheetAlign: Grounded Interaction and Stage-Isolated Learning for Spreadsheet Agents

链接:https://arxiv.org/abs/2611.26069

作者:Yuxiang Wang,Junhao Gan,Jianzhong Qi

类目:Computation and Language (cs.CL); Software Engineering (cs.SE)

关键词:supporting decision-making, Automating spreadsheet analysis, protecting sensitive data, data analysis, spreadsheet analysis

备注: conference paper preprint

点击查看摘要

Abstract:Spreadsheets are widely used for data analysis and supporting decision-making. Automating spreadsheet analysis while protecting sensitive data and controlling deployment costs motivates locally deployed open-weight large language models (LLMs). Meanwhile, complex spreadsheet tasks require both task reasoning and workbook understanding, while predefined operation interfaces impose additional tool-selection and invocation demands. We introduce SheetAlign, a spreadsheet manipulation framework with stage-isolated policy learning, organized around where to operate, what to change, and how to implement transformations. Task-guided inspection and API grounding anchor code generation in workbook observations and available operations, while stage-isolated learning optimizes program generation and review using edit-progress rewards. At inference, execution consistency selects programs through agreement among workbook states. On SpreadsheetBench, our grounded task interface improves Qwen3-4B's accuracy from 13.5% to 29.2% on average without training. With learning and execution consistency, SheetAlign achieves 32.8% accuracy on average on SpreadsheetBench and 30.1% on Domain-Spreadsheet, outperforming comparable baselines. The code will be made publicly available upon publication.

27. 【2611.26059】Complete-SVD: A Functional Norm Approach to Low-Rank LLM Compression

链接:https://arxiv.org/abs/2611.26059

作者:Alper Kalle,Theo Rudkiewicz,Mohamed-Oumar Ouerfelli,Mohamed Tamaazousti

类目:Machine Learning (cs.LG); Computation and Language (cs.CL)

关键词:Post-training low-rank compression, model input-output behavior, Post-training low-rank, input-output behavior, model input-output

备注: 23 pages, 3 figures, 15 tables

点击查看摘要

Abstract:Post-training low-rank compression of neural networks must not only approximate the original weights but also preserve the model's input-output behavior. Recent output-aware methods measure the compression error beyond the compressed layer, at the model output, weighting logit changes by the observed next token or by the model's predicted probabilities. We propose Complete-SVD, a method for large language models that minimizes the $L_2$ distance between the logits of the original and compressed models, weighting all logit changes equally. Linearizing the network around the original weights, together with a Kronecker-factored approximation, reduces this objective to a weighted SVD with a closed-form solution. Its output-side factor is intractable to compute; we estimate it without bias from random probes, at a cost of $K$ backward passes per sequence, independent of vocabulary size and sequence length. At matched settings with uniform ranks and WikiText-2 calibration, Complete-SVD achieves the lowest C4 and PTB perplexity in every setting, and at ratio 0.7 under LEMS rank allocation. Replacing the $L_2$ distance by the Kullback-Leibler divergence (KL-SVD), with the same estimation framework, lowers calibration perplexity in most settings and raises downstream accuracy, but raises cross-corpus perplexity. Complete-SVD also improves IFEval instruction-following accuracy over the strongest prior output-aware methods throughout.

28. 【2611.26009】Evaluating safeguarding in multi-turn conversations with AI assistants

链接:https://arxiv.org/abs/2611.26009

作者:Charlotte Peart,Franziska Sofia Hafner,Luc Rocher

类目:Computation and Language (cs.CL)

关键词:Conversational AI assistants, personal and emotional, Conversational, conversations, Claude Opus

备注:

点击查看摘要

Abstract:Conversational AI assistants are now used for personal and emotional support. Developers report improvements in how new models respond to sensitive conversations, yet users and civil society continue to raise concerns about their behaviour in real-world, repeated interactions that safety evaluations rarely capture. Often, evaluations present signs of distress clearly and in a single message, even though people may reveal their struggles gradually in otherwise ordinary conversations. We worked with three community mental health support workers to test if models provide the same level of safeguarding when cues are shared all at once or gradually over time. We designed nine scenarios in which a user shares cues related to depression, psychosis, or domestic abuse. For each scenario, the support workers established a gold standard response, ranging from signposting support to pausing the everyday task to address safeguarding. We presented five cues either together in one message or dispersed across a conversation. Across more than 7,400 conversations, we evaluated six widely used models from five developers. GPT-5.5, Grok 4.3, DeepSeek V4, and Llama 4 Maverick met the support workers' standards in just 26-59% of conversations when cues were concentrated, and only 3-6% when dispersed. Claude Sonnet 5 and Claude Opus 5 showed a smaller gap, meeting the standards in 68-97% of concentrated conversations and 59-87% of dispersed conversations. Even the best-performing model, Claude Opus 5, fell short of the gold standard in 13% of conversations when cues were dispersed. Safety evaluations centred on explicit, legible concerns can overstate how well safeguards protect people when signs of distress emerge gradually or indirectly through ordinary conversations. For claims of improved safety to be taken seriously, developers must design and assess safeguards around how people actually communicate.

29. 【2611.26000】Xing-Audio-NVVASR: Scaling and Refining Supervision for Joint Recognition of Speech and Non-Verbal Vocalizations

链接:https://arxiv.org/abs/2611.26000

作者:Pengyu Xu,Gang Liu,Genliang Zhao,Hongjie Chen,Jian Kang,Jie Lian,Jie Li,Yongxiang Li

类目:Computation and Language (cs.CL)

关键词:scarce event-rich training, event-rich training data, Joint recognition, missing event labels, non-verbal vocalizations

备注: 18 pages, 2 figures

点击查看摘要

Abstract:Joint recognition of speech and non-verbal vocalizations (NVVs) is hindered by scarce event-rich training data and inaccurate or missing event labels. We present Xing-Audio-NVVASR, an end-to-end recognizer trained through progressive data mining and iterative supervision refinement. Guided by unified acoustic criteria, two heterogeneous models cross-audit annotations, flagging disagreements for teacher reannotation. To uncover shared omissions that model agreement may conceal, complementary breath detectors screen the entire corpus for human review. Human-confirmed labels accumulate across rounds and are preserved during subsequent refinement. Over five rounds, we refine annotations for 170,000 utterances from public corpora and incorporate 630,000 mined utterances. The resulting corpus of approximately 800,000 utterances is used to train a single 1.7B-parameter model that transcribes speech with inline tags for six NVV classes. Across four human-annotated benchmarks, Xing-Audio-NVVASR ranks first among nine models in both composite score and six-class Macro-F1. It achieves a mean composite score of 64.46, surpassing the strongest comparison model, StepAudio 3 Chat Preview, by 6.03 points, and exceeds the annotation teacher's breathing F1 by 14.40 to 31.57 percentage points. We plan to release the model weights, inference code, four evaluation benchmarks, annotation guidelines, and evaluation code.

30. 【2611.25989】Policy Improvement-Guided Replay for Continual Reinforcement Learning

链接:https://arxiv.org/abs/2611.25989

作者:Qin-Wen Luo,Haoxuan Gao,Sheng Ren,Xiang Chen,Sheng-Jun Huang

类目:Machine Learning (cs.LG); Artificial Intelligence (cs.AI); Computation and Language (cs.CL)

关键词:Continual reinforcement learning, Continual reinforcement, continual learning emphasizes, conventional continual learning, acquire new capabilities

备注:

点击查看摘要

Abstract:Continual reinforcement learning (CRL) requires models to acquire new capabilities while retaining performance on previous tasks. A stability-oriented perspective common in conventional continual learning emphasizes knowledge preservation and interference mitigation. Complementing this perspective, we revisit CRL for vision-language models through policy improvement, a fundamental principle of RL, to preserve acquired performance gains. Using the performance difference lemma, we characterize historical performance changes through the current policy's expected advantages on historical task distributions, motivating replay of historical prompts with the current policy. To allocate limited replay memory, we propose Contrastive Likelihood Replay (CoLiR), which contrasts likelihood gains on correct responses from the final task policy with likelihood reductions on incorrect responses from the initial policy. Our analysis links correct-response likelihood gains to reference-policy advantages under an idealized KL-regularized optimal update and provides an endogenous-reward interpretation for general policy updates. CoLiR estimates scores from finite training rollouts and cached log-likelihoods, selects prompts at task boundaries, and replays them uniformly during subsequent training, with KL regularization restricted to replayed prompts. Memory construction only requires a small amount of additional cross-policy scoring but no additional response generation. Across three Qwen3-VL model sizes on MRCL, CoLiR achieves the highest mean final accuracy and mean task accuracy among the compared methods while maintaining competitive new-task performance.

31. 【2611.25976】Energy estimates of open-weight large language models across stated food weights: a dose-response evaluation

链接:https://arxiv.org/abs/2611.25976

作者:Donato Riccio,Fabrizio Maturo,Iulia Brezeanu

类目:Computation and Language (cs.CL); Artificial Intelligence (cs.AI)

关键词:open-weight large language, large language models, times weight removes, stated food weight, density times weight

备注: 18 pages, 4 figures, 4 tables; supplementary material and TRIPOD-LLM checklist as ancillary files

点击查看摘要

Abstract:We tested whether energy estimates of open-weight large language models (LLMs) scale proportionally with an exactly stated food weight, and whether computing energy density times weight removes the error. Under a prespecified protocol, seven open-weight LLMs estimated the energy of 168 randomly drawn foods from the USDA Food and Nutrient Database for Dietary Studies at nine stated weights, from 0.25 to 4 times the default amount. The primary outcome was the elasticity of the estimate with respect to weight (1: proportional scaling). A two-step estimate multiplied each model's own stated energy density by the weight in code. All seven models compressed their estimates, with elasticities from 0.548 to 0.979 (every 95% CI below 1); four compressed severely (0.548 to 0.709). These four overestimated a quarter of the default amount by 47% to 130% and underestimated four times it by 20% to 34%. Supplying the density restored near-proportional scaling in three of the four, and the two-step estimate cut the typical error factor at these extremes from 1.47-2.43 to 1.20-1.38. Compression persisted at full precision and grew in five of seven models under a plain question. The models hold the energy densities needed for proportional scaling and answer near a typical serving, reproducing the flat-slope syndrome of dietary recall. Accuracy at usual portions does not reveal it. Food-logging tools should compute the energy of weighed entries as density times weight, and evaluations should test fixed foods across stated weights.

32. 【2611.25970】he Depth of a Prediction: Where Language Models Resolve a Word Predicts Human Reading Time

链接:https://arxiv.org/abs/2611.25970

作者:Donato Riccio,Fabrizio Maturo

类目:Computation and Language (cs.CL); Machine Learning (cs.LG)

关键词:long people read, final layer predicts, language model final, model final layer, predicts how long

备注: 17 pages, 4 figures, 7 tables

点击查看摘要

Abstract:Surprisal from a language model's final layer predicts how long people read a word, yet the model computes its prediction across many layers. We read each word's prediction at every layer with the tuned lens and summarize this layerwise surprisal trajectory by its resolution depth, the depth around which its change is centered. On eight English eye-tracking and self-paced reading corpora (81,957 words) and ten GPT-2, OPT and Pythia models, words that a model resolves later, and the words after them, are read more slowly than equally surprising words. Resolution depth improves held-out prediction beyond final-layer surprisal, entropy and the best single layer by 0.26 millinats per word (pooled p = 0.001), about a third of entropy's gain, and each model family gains, GPT-2 and Pythia through one corpus each. The net gain varies between corpora (random-effects mean 0.07, p = 0.37); against layer-shuffled depth, which adds as many coefficients, it is positive in seven of eight corpora (random-effects mean 0.23, p = 0.016), and untrained networks show no gain under the logit lens. Pooled over corpora, the effect holds with corrected word probabilities and beyond first-to-last-layer divergences, and a whole-trajectory functional regression adds no significant information beyond resolution depth.

33. 【2611.25960】When Sounds Turn Heads: Recovering Event Timing That Audio LLMs Fail to Say

链接:https://arxiv.org/abs/2611.25960

作者:Jongbhin Woo,Jaewook Jeong,Joon Son Chung

类目:ound (cs.SD); Computation and Language (cs.CL)

关键词:queried sound occurs, queried sound, occurs but fail, sound occurs, sound

备注: 11 pages, 4 figures

点击查看摘要

Abstract:Audio LLMs can know when a queried sound occurs but fail to say it. We find a few attention heads that track the queried sound: they shift when the query changes, follow the sound when it moves, and removing or blocking them often hurts the model's timestamps. Yet the answer does not express what they track. Intervals read directly from these heads outperform the model's original answer and recover events that its timestamps miss. To bring this timing into the answer, we introduce head-guided decoding, which needs no training. It keeps the model's decision on whether the sound is present and rescores its candidate timestamps with a prior from these heads. Heads selected on development data transfer unchanged to new recordings and free-text queries, where head-guided decoding improves the timestamps of three audio LLMs, scores above their head readout on every benchmark, and, in most settings, outperforms random-head guidance. On recordings released after the models were trained, head-guided decoding raises mIoU from 0.07-0.36 to 0.48-0.56. These results suggest that poor temporal answers in audio LLMs partly reflect a failure to say rather than to hear, and that the model's own attention can close part of this gap.

34. 【2611.25955】Web search in Latin America: language models use less local and diverse news than search engines

链接:https://arxiv.org/abs/2611.25955

作者:Francisco W. Kerche,Franziska Sofia Hafner,Felix M. Simon,Luc Rocher

类目:Computation and Language (cs.CL); Artificial Intelligence (cs.AI); Computers and Society (cs.CY)

关键词:Global South, Latin America, major gateway, search, search engines

备注: 23 pages with appendix, 3 figures

点击查看摘要

Abstract:AI platforms are becoming a major gateway to news and current events. This shift risks amplifying existing geographical biases that favour narratives from wealthier countries over those from the Global South. This is particularly relevant in Latin America, which accounts for the largest share of AI usage in the Global South. We examine how information about Latin America changes when users shift from conventional search engines to LLM-based search. We construct a dataset of 1,901 current events across nine Latin American countries, craft realistic queries, and submit them to three search engines and four language models. We vary query language and user location to represent local, diaspora, tourist, and foreign user scenarios. Across 49,207 requests and 8,834 unique domains, language models return a less diverse set of news sources than search engines. Across user types, the share of domestic news returned by language models is inconsistent. For local users, while search engines already return only 41.3\% local news, language models return a further 3.6 to 11.6 percentage points less. Taken together, our findings suggest that LLM-based search can reproduce and, in some settings, amplify geographical hierarchies in the reach and visibility of news sources. We release an open dataset to support multilingual and geographically diverse evaluation of LLM-based search, and highlight implications for more equitable information systems design.

35. 【2611.25916】Many Solutions, One Answer: Search Order Shapes the Default Solutions of Language Models

链接:https://arxiv.org/abs/2611.25916

作者:Qinhan Hou,Jing Tang

类目:Computation and Language (cs.CL); Artificial Intelligence (cs.AI)

关键词:http URL, http URL study, http URL samples, http URL held-out, URL samples concentrate

备注: 22 pages, 6 figures and 4 tables

点击查看摘要

Abstract:When a constraint puzzle has several correct answers, language models tend to give the same this http URL study four open-weight models from three families on symbolic puzzles and multi-solution ZebraLogic variants with exactly known solution this http URL samples concentrate on one default solution far beyond uniform sampling, and different models often share this http URL models usually produce even rarely sampled solutions when added clues uniquely identify this http URL held-out symbolic puzzles, we compare visiting entities in name order with visiting those with the fewest initial candidates first. Both searches try values in the stated order. Where their pre-specified predictions differ, the latter matches a larger share of responses in all four models, by 26 to 40 percentage this http URL the stated value order or specifying a search order shifts responses towards the solution that the new order favours, on both testbeds. Randomising the presentation or supplying a random search order raises breadth (the effective share of solutions in use) by 26 to 42 percentage points on the symbolic task and 11 to 26 on ZebraLogic. The largest observed validity losses are 3 and 4 percentage points, respectively. Randomised presentation broadens answers more than a random-choice instruction in each of the three main models.

36. 【2611.25909】he Linguistic Structure of Dependency-Conditioned Embeddings

链接:https://arxiv.org/abs/2611.25909

作者:Bach Phan-Tat,Tung Kieu,Kris Heylen,Dirk Geeraerts,Stefano De Pascale,Dirk Speelman

类目:Computation and Language (cs.CL)

关键词:Dependency-based distributional models, distributional models capture, models capture syntactically, capture syntactically informed, Dependency-based distributional

备注:

点击查看摘要

Abstract:Dependency-based distributional models capture syntactically informed lexical similarities, but the geometric organisation of dependency-conditioned lexical representations remains underexplored. We investigate the global and local geometry of lemma-dependency relation embeddings using four representations trained on WikiText-103: two count-based PPMI configurations and two Skip-Gram with Negative Sampling (SGNS) configurations. At the global level, we analyse dependency-relation centroids, control for lexical identity through within-lemma centring, and examine their alignment with part-of-speech (POS) distributions. At the local level, we evaluate cross-relation geometric consistency, cross-method agreement, lexical similarity, and polysemous usages. Results show that global geometry is structured strongly by context POS across all representations. Locally, broader lexical similarity structures are partially preserved across dependency relations, while immediate neighbourhoods vary substantially, highlighting different usages. PPMI-based representations exhibit greater cross-relation consistency than SGNS-based representations, with cross-method agreement varying by relation. These findings reveal systematic linguistic organisation at multiple geometric scales and its sensitivity to representation choices.

37. 【2611.25898】Verifiable Logic in Medical LLMs: A Critical Review of Agentic Clinical Reasoning Optimization and Neuro-Symbolic Grounding

链接:https://arxiv.org/abs/2611.25898

作者:Youssef ElNahas,Adel Ammar,Soumaya Chaffar,Wadii Boulila

类目:Computation and Language (cs.CL); Artificial Intelligence (cs.AI); Computers and Society (cs.CY); Machine Learning (cs.LG)

关键词:large language models, Clinical large language, final answer, language models, Process reward models

备注:

点击查看摘要

Abstract:Clinical large language models are usually judged by whether the final answer is right. Yet a model can reach a fluent, plausible output through invalid reasoning that static benchmarks do not detect, and confident hallucination, causally invalid inference, and spurious demographic shortcuts can pass conventional evaluation. This critical review covers four paradigms in turn: how clinical hallucination arises and why static metrics miss it; the agentic hallucination cascade, in which multi-agent and tool-augmented systems propagate and amplify early reasoning errors; process reward modeling as step-level supervision; and neuro-symbolic grounding, in which formal logic engines, ontology reasoners, and knowledge graphs place verifiable constraints on neural output. Within their encoded domains, systems with symbolic veto layers offer guarantees that hold relative to their encoded knowledge, and they report large gains on the safety measures they use. These results come from single studies, mostly in simulated or small-sample settings, and are not directly comparable with neural baselines. Process reward models remain bounded by the statistics of their training data. We propose a conceptual framework that maps each agentic failure mode to a symbolic intervention point, and we argue that clinical AI safety depends substantially on system architecture. Because current regulation emphasizes performance and does not require verification of reasoning steps, we argue that responsible certification should also audit reasoning trajectories alongside accuracy on static benchmarks.

38. 【2611.25895】One Schedule Does Not Fit All: Adaptive Inference Schedules for Diffusion Language Models

链接:https://arxiv.org/abs/2611.25895

作者:Rabimba Karanjai,Linsey Pang,Hemanth Hegadehalli Madhavarao,Richard Williamson,Uma Kona,Prakhar Mehrotra

类目:Artificial Intelligence (cs.AI); Computation and Language (cs.CL); Machine Learning (cs.LG)

关键词:Diffusion language models, Diffusion language, language models typically, Task-Calibrated Denoising Allocation, Diffusion

备注:

点击查看摘要

Abstract:Diffusion language models typically use a fixed denoising schedule across workloads. We show that the cheapest schedule that preserves quality depends on the task, model, and sampler. We propose Task-Calibrated Denoising Allocation (TCDA), a deployment rule with an explicit error guarantee. Given candidate samplers, paired calibration data, a utility tolerance, and guardrails against degeneration, TCDA uses simultaneous one-sided bootstrap bounds to certify which candidates meet both the utility and guardrail criteria. It deploys the cheapest certified candidate or falls back to the full schedule. The probability of deploying an infeasible schedule is asymptotically at most alpha. Candidates include early prefix decoding, state-reusing continuation, reduced time grids, and confidence-threshold decoding. Across seven datasets and two model families, TCDA certifies reductions in denoiser evaluations by factors of 3.4 on XSum and 2.0 on WMT14 for ELF-B; 8.0 on WMT14 and 4.1 on GSM8K for LLaDA-8B; and 1.33 on OpenWebText for both ELF-M and ELF-L. A validation protocol fixed before generation confirmed every selection on fresh seeds. Across 500 calibration splits, false certification remained below 2% in every setting at a nominal level of 5%. Certified schedules vary across tasks and model scales. Trajectory measurements suggest that when decoded tokens commit is more informative for schedule selection than the remaining latent discretization error. We also find that the ELF reference sampler draws its time grid at random, so reproducible schedule comparisons for unconditional generation require fixing the grid.

39. 【2611.25862】HALTT: A HAte Lexicon Through Time

链接:https://arxiv.org/abs/2611.25862

作者:Dario Franzese,Farah Benamara,Véronique Moriceau,Viviana Patti

类目:Computation and Language (cs.CL)

关键词:original benign meanings, retaining original benign, Automated hate speech, polysemy retention, speech detection degrades

备注:

点击查看摘要

Abstract:Automated hate speech detection degrades over time as everyday vocabulary is weaponized into covert slurs while retaining original benign meanings (polysemy retention). Existing lexical semantic change detection (LSCD) frameworks struggle with this phenomenon due to a lack of standardized, sense-delineated diachronic resources. We propose a multi-stage framework to construct and validate a diachronic lexicon of semantically shifted hate speech terms. Filtering 20,000 candidates, our pipeline combines historical (1770-1964) and modern (2001-2024) corpus frequency grounding, masked definition-level scoring to mitigate LLM temporal bias, and a dual-persona reasoning prompt paradigm. We release HALTT, a grounded lexicon of 19 polysemous terms with literal and derogatory senses, analyze shift typologies, and establish evaluation protocols for downstream diachronic benchmarking.

40. 【2611.25853】ableRoute: Capability-Aware Routing for Scalable Table Question Answering

链接:https://arxiv.org/abs/2611.25853

作者:Yuxiang Wang,Junhao Gan,Jianzhong Qi

类目:Computation and Language (cs.CL)

关键词:large language models, questions require precise, require precise grounding, million-token sizes, answering with large

备注:

点击查看摘要

Abstract:Table question answering with large language models remains difficult to scale: tables range from compact to million-token sizes, while questions require precise grounding of columns, values, and computations. Existing methods often specialize in a fixed inference strategy, which is brittle across different table scales and model capabilities. Full-table reading is efficient for compact inputs but suffers from irrelevant context. Table reduction helps long inputs but depends on reliable evidence localization, while execution is required when tables exceed the model context window and remains sensitive to schema and value grounding errors. We propose TableRoute, a capability-aware routing framework with evidence-conditioned inference for scalable TableQA. TableRoute first generates validated question--table evidence and identifies the required operations, then routes each instance among these three strategies according to serialized table length and calibrated estimates of how reliably the base model handles these operations. The validated evidence supports the selected route by grounding columns, values, and conditions. Across WikiTQ, WTQ-L, BirdQA, and TableBench, TableRoute consistently improves accuracy through complementary routing and evidence conditioning across table scales, while reducing online token cost. Code will be made available upon publication.

41. 【2611.25818】Anchor and Check: Deterministic Cross-Prompt, Cross-Model Reconciliation for Auditable LLM Decisions

链接:https://arxiv.org/abs/2611.25818

作者:Sivakameswara Reddy Chejarla

类目:Artificial Intelligence (cs.AI); Computation and Language (cs.CL); Machine Learning (cs.LG)

关键词:large language model, make discrete decisions, make discrete, large language, language model

备注: 20 pages, 6 figures, 11 tables

点击查看摘要

Abstract:Pipelines that let a large language model (LLM) make discrete decisions need to know which decisions they may act on unattended, and a record that explains each one. We study a small, fully specified reconciliation contract: commit the target model's greedy answer (the anchor), draw one or two greedy checks that reach the answer by a different route (an alternate prompt, or a model from another family), and let a pure function return one of five outcomes, of which only Agree is acted on. On 48,000 model-item evaluations over 6,000 questions, with every method given access to verbalized confidence as a tie-breaker, the main finding concerns where calls come from: pooling another family's samples raises semantic entropy's AUROC by 1.7 to 4.0 points at every budget, and a same-model temperature check is the weakest design. As complete pipelines, the point estimates of the three-call contract and pooled semantic entropy over 3 or 5 samples differ by less than one point, with intervals containing zero. As detectors of the anchor's errors, the contract is ahead of pooled entropy with 3 and 5 samples under both tie-breaker conventions; with 10 samples the point estimates differ by less than one point and the sign depends on the convention. Under the strict Agree gate it automates 70% of items and still acts on 16% of errors. What the contract adds is a fixed committed answer, an explicit outcome and a decision replayable from its record; generation itself was not reproducible on the hosted API.

42. 【2611.25814】What Does Anti-Hallucination Training Buy? Saying Less, Cheap Pruning, and the Rest

链接:https://arxiv.org/abs/2611.25814

作者:Lingkai Bu,Qian Gao

类目:Computer Vision and Pattern Recognition (cs.CV); Computation and Language (cs.CL)

关键词:reducing object hallucination, Preference training, lower CHAIR score, standard recipe, recipe for reducing

备注:

点击查看摘要

Abstract:Preference training is the standard recipe for reducing object hallucination in vision-language models, and each new method reports a lower CHAIR score. We ask what that reduction consists of. Every trained system is compared with its base model's caption truncated to the same object recall, which removes the part of the gain that comes from saying less; the remainder is compared with what a deliberately weak, frozen, eleven-feature post-hoc probe fitted on 500 annotated images can reproduce. All criteria were specified in the analysis code and frozen before the outputs they judge existed. For eight public methods on LLaVA-1.5-7B, the probe reproduces at least two thirds of what training removes beyond saying less for 7 of 8 methods on COCO, 8 of 8 on AMBER and 6 of 8 on Object HalBench, with RLAIF-V the consistent exception; a later public method released for both LLaVA-1.5 and Qwen2.5-VL (SENTINEL) meets the same bar on both. On our own DPO checkpoints the accounting holds on Qwen2.5-VL but fails on InternVL2.5 on COCO (2 of 4 training designs). At matched recall, the base model plus the probe beats truncation for 8 of 9 LLaVA-1.5 systems on COCO and 9 of 9 on AMBER, on the two newer backbones, and when realized as actual sentence deletion. The same measure prices training signals: DPO on the model's own samples labelled by the probe or by an object-level CLIP score recovers 0.70 and 0.87 of the beyond-brevity gain of ground-truth-labelled DPO on Qwen2.5-VL (the probe 0.73 on InternVL2.5), both beat length-matched random labels, and CLIP labels buy more than probe labels on COCO. Human judgements show that the sentences the probe deletes are wrong more often than those truncation would delete; a human check of the CHAIR evaluator's precision falls below our pre-set bar, which we report as a limitation.

43. 【2611.25813】Efficient and Robust Drafting with Multi-Token Prediction

链接:https://arxiv.org/abs/2611.25813

作者:Filippo Galgani,Ziwei Ji,Omri Homburger,Ganesh Jawahar,Michelle Ramanovich,Ivan Korotkov,Luciano Martins,Utku Evci,Edouard Yvinec,Pooja Consul,Vivek Kumar,Tal Schuster

类目:Computation and Language (cs.CL)

关键词:accelerating Large Language, parallel verification phase, Large Language Model, Language Model, Large Language

备注:

点击查看摘要

Abstract:Speculative decoding has emerged as a critical technique for accelerating Large Language Model (LLM) inference by decoupling generation into a drafting phase and a parallel verification phase. A crucial challenge is therefore designing a drafter that is both lightweight and accurate enough to approximate the predictions of the target model. We present a highly optimized Multi-Token Prediction (MTP) architecture that extends existing backbone LLMs to efficiently produce high-quality drafts. Built around a single shared drafting head, our approach features three core architectural optimizations: target activation injection to anchor semantic planning, zero-copy KV-cache cross-attention to bypass prefill compute and reduce memory overhead, and an efficient clustered vocabulary projection to reduce the cost of full-vocabulary decoding. Testing these improvements with the Gemma 4 model family, our G-MTP shows out-of-the-box speedups for both large server-side setups (up to 3.1$\times$) and smaller models targeting memory-constrained on-device environments (up to 2.3$\times$ on H100 and up to 2.2$\times$ on a smartphone GPU). In addition, we present parameter-efficient fine-tuning recipes for G-MTP that keep the drafter aligned with the backbone LLM under domain shift, requiring up to 1.83$\times$ fewer target forward passes than tuning only the backbone.

44. 【2611.25808】Language models represent more about emphasis than they can tell you

链接:https://arxiv.org/abs/2611.25808

作者:Aarne Talman,Stergios Chatzikyriakidis

类目:Computation and Language (cs.CL); Sound (cs.SD)

关键词:Prosodic prominence, read English speech, Prosodic, models, text

备注:

点击查看摘要

Abstract:Prosodic prominence can largely be predicted from the words of a sentence, but part of it depends on how the speaker reads it. We study how much of prominence language models recover from text, whether they can report it when asked, and what the speech signal adds. On read English speech labelled for word-level prominence, we compare frozen and fine-tuned text models, prompted large language models (LLMs) and speech models on the same words. All text models end within about one point of each other, at best 68.8% on the 3-way task, and largely make the same errors. With the prompts we tested, LLMs do no better than a word-majority baseline, even with 500 examples. Linear probes find more in their representations. Speech models reach 84.7%. We argue that text determines only part of prominence, and that the rest has to be heard. LLMs encode what the text determines, but prompting does not bring it out: they represent more about where the emphasis falls than they tell you.

45. 【2611.25780】RouteVerify: Routing Interventions for Step-Level Verification in Mixture-of-Experts Models

链接:https://arxiv.org/abs/2611.25780

作者:Yucheng Du,Ryan A. Rossi,Xiyang Hu

类目:Machine Learning (cs.LG); Computation and Language (cs.CL)

关键词:correct, Routing, checkpoints, erroneous reasoning steps, steps

备注: 38 pages, 5 figures. Code: [this https URL](https://github.com/yucheng-du/RouteVerify)

点击查看摘要

Abstract:Routing states in mixture-of-experts (MoE) models can distinguish correct from erroneous reasoning steps. If changing routing also changes a model's step-level verification judgment, it is tempting to conclude that the judgment follows what routing encodes. We test this inference with RouteVerify, a controlled intervention study that replaces selected internal activations from one solution trace with activations from another, while keeping the evaluated input unchanged. Across nine MoE checkpoints, we compare three intervention interfaces: router logits, MoE outputs, and post-MoE residual states. We vary the layers and token positions at which activations are replaced. We measure changes in a verification margin that quantifies the model's preference for INCORRECT over CORRECT. We also evaluate the same input with activations from correct versus erroneous steps to test whether the resulting judgment depends on the source step's correctness. We report four linked results. First, probes on router logits separate correct from erroneous steps in eight checkpoints (AUROC 0.80-0.88). Second, replacing router logits across all routed layers changes the judgment in all nine checkpoints, and by more than a norm-matched random replacement. Third, this change rarely depends on whether the source step is correct. At one layer, routing from erroneous steps and from correct steps produces indistinguishable margins in all nine checkpoints. Under all-layer routing, their difference is resolved and positive in only one. Fourth, the same difference is positive for local residual replacements in eight of nine checkpoints, so the measure does detect such dependence where it exists. Correctness is therefore readable from routing, and routing replacements move the judgment, but the movement rarely follows correctness.

46. 【2611.25763】Closing Performance Gap for Diffusion Language Models under Low Inference Budgets

链接:https://arxiv.org/abs/2611.25763

作者:Hao Wu,Sipeng Chen,Jingwen Fu,Yuwang Wang

类目:Computation and Language (cs.CL); Artificial Intelligence (cs.AI); Machine Learning (cs.LG); Machine Learning (stat.ML)

关键词:sequential decoding steps, decoding steps relative, language models offer, updating multiple tokens, Diffusion language models

备注: 23 pages, 3 figures

点击查看摘要

Abstract:Diffusion language models offer an architectural route to low-budget generation by updating multiple tokens in parallel, which can reduce the number of sequential decoding steps relative to token-by-token autoregressive decoding. However, realizing this advantage at low budgets is challenging, as generation quality often declines when the number of denoising and refinement steps is reduced. Editable diffusion models address this quality loss through post-draft refinement, but useful intermediate results may still be lost before final submission. We introduce CaRR (Commitment-aware Refinement and Recovery), a training-free framework which improves submission quality from a fixed editable checkpoint through updated-context refill and trajectory-based recovery within its existing forward schedule. Token Commitment Remasking (TCR) remasks selected native pure replacements and refills them on the next scheduled edit forward using the updated context. Submission Commitment Recovery (SCR) uses the resulting refinement trajectory to recover supported candidates when the final response does not satisfy task-specific output requirements. Both modules operate within the native mask/edit schedule without additional backbone forwards. Experiments on code generation and mathematical reasoning show consistent gains in average accuracy over native ME-DLM across four inference budgets, with the largest gains under tight budgets. At one-eighth of the full scheduled forward budget, CaRR improves the six-metric average accuracy by 7.44 pp, including gains of 11.37 and 10.05 on MBPP and MBPP+.

47. 【2611.25745】Verification Behavior Across Five Qwen2.5-Instruct Checkpoints

链接:https://arxiv.org/abs/2611.25745

作者:YaJie Yin

类目:Computation and Language (cs.CL)

关键词:prompting conditions, information condition, Abstract, fixed tasks, trajectories

备注: 10 pages, 1 figure. Includes English translations of the experiment prompts and the behavioral rating rubric. AI language models were used for behavioral ratings and to assist manuscript preparation

点击查看摘要

Abstract:We evaluate five Qwen2.5-Instruct checkpoints (0.5B to 32B) under four prompting conditions using three fixed tasks per information condition and five generations per task. Two AI raters scored 600 trajectories. Under the coordinated instruction, both raters recorded strict declarations of insufficiency in 11/15 cases at 7B, 0/15 at 14B, and 5/15 at 32B. The runner sent a standard follow-up when a request detector fired. The three checkpoints received at least one follow-up in 15, 12, and 15 cases, respectively. On information-sufficient tasks at 3B through 32B, scored format markers appeared in 60/60 protocol trajectories and 0/60 baseline trajectories. After adjudication, 1/60 in each condition met the diagnostic-execution criterion (H3). Across all checkpoints and prompts, 7/300 information-sufficient trajectories met H3 after adjudication, all on the same task. The prespecified upward transition did not occur in this run. The results separate visible structure, diagnostic execution, and persistence after a request for evidence.

48. 【2611.25735】PosteriorBench: Evaluating Behavior Posterior Sampling in Language Models

链接:https://arxiv.org/abs/2611.25735

作者:Zhiyang Xun,Daniel D. Johnson,Sarah Schwettmann,Neil Chowdhury

类目:Machine Learning (cs.LG); Artificial Intelligence (cs.AI); Computation and Language (cs.CL)

关键词:Rare language model, Rare language, language model, difficult to study, study because obtaining

备注:

点击查看摘要

Abstract:Rare language model behaviors can be difficult to study because obtaining them through ordinary sampling can require a very large number of samples. This motivates the behavior posterior sampling problem: given a language model and a behavior of interest, is there an algorithm that efficiently samples from the model's distribution over transcripts that exhibit this behavior? We introduce PosteriorBench, a benchmark for comparing algorithms by their compute cost and by how close their samples are to the true posterior in forward and reverse KL divergence. PosteriorBench contains 21 tasks on Qwen3.5-0.8B with behavior probabilities from $10^{-5}$ to $10^{-1}$, each with exact posterior samples from a corpus of one billion labeled responses. We find that no method we evaluate is both efficient and faithful: simple methods like prompting are cheap but plateau at a high KL divergence from the posterior, fine-tuning on posterior samples is often cost-prohibitive, and several training methods from prior work do not perform better. Building a posterior sampler that can cheaply approximate the true posterior remains an open problem, and we hope PosteriorBench can measure progress towards this objective. Code and data are available at this https URL.

49. 【2611.25717】VR-SparseQuant: Accurate FP4 Quantization with 2:4 Sparsity on Vera Rubin

链接:https://arxiv.org/abs/2611.25717

作者:Samir Khaki,Wei-Chen Wang,Song Han,Ryan Hanrui Wang

类目:Computation and Language (cs.CL)

关键词:Tensor Cores, incorporated increasingly specialized, increasingly specialized support, Vera Rubin elementwise, Vera Rubin

备注: 16 Pages, 7 Figures

点击查看摘要

Abstract:In recent years, GPUs have incorporated increasingly specialized support for low-precision computation, from Hopper's FP8 to Blackwell's FP4 Tensor Cores. Precision formats have evolved alongside this hardware, from MXFP4 and NVFP4 on Blackwell to E2M1 FP4 with direct UE5M3 block scaling on Vera Rubin, which we term as $\boldsymbol{VRFP4}$. These advances help serve larger language models at lower cost, but four-bit activation quantization can substantially degrade model quality. We introduce VR-SparseQuant, which uses Vera Rubin's elementwise 2:4 sparsity to recover accuracy while retaining native FP4 execution. By allowing two sparse streams to overlap, we use the same FP4 value budget as dense quantization to refine activations with larger errors. Offline scale-ratio calibration avoids runtime search, while a shared-weight GEMM accumulates both streams efficiently. On Nemotron-3-Nano-30B-A3B, VR-SparseQuant reduces WikiText-2 perplexity from $\boldsymbol{6.85}$ to $\boldsymbol{5.57}$ versus dense RTN W4A4. Including activation preparation, our native Rubin kernels for VRFP4 achieve up to $\boldsymbol{3.1\times}$ GEMM and $\boldsymbol{2.7\times}$ MoE speedup over BF16. To the best of our knowledge, this is the first study of adaptive activation quantization using Vera Rubin's elementwise 2:4 structured sparse FP4 instructions.

50. 【2611.25705】STARK: Moving towards Personalized Agents That Truly Learn and Remember

链接:https://arxiv.org/abs/2611.25705

作者:Aryan Jain,Abhishek Jha,Anish Nediyanchath,Promod Yenigalla

类目:Computation and Language (cs.CL); Artificial Intelligence (cs.AI); Information Retrieval (cs.IR)

关键词:long time periods, personalized agentic systems, Long-term memory, time periods, underlying LLMs.

备注:

点击查看摘要

Abstract:Long-term memory is essential for personalized agentic systems that interact with users over long time periods, yet current architectures only capture what happened while leaving the reasoning of why it happened to be derived later by underlying LLMs.%, and silo knowledge within individual users. These events can, however, convey not only what a user experienced but also how that experience shaped their behavioral patterns. We introduce STARK, a dual-graph memory architecture which treats these events not just as nodes but also as edges between behavioral insights. STARK maintains an Episodic Graph which records interactions as temporally, causally, and semantically linked episodes; and an Insight Graph which represents LLM-extracted behavioral signals scored by how many events support them, connected by the very events that led to them. At inference, behavioral knowledge is retrieved directly from insight nodes grounded by event-as-edges, eliminating repeated episodic traversal. The event-as-edge structure also naturally enables projecting the knowledge graph into a relational embedding space where cross-user transfer becomes possible under shared contextual circumstances, allowing the system to infer missing insights for cold-start users. Evaluations show that STARK improves by 1.48 judge score on LongMemEval information extraction and 0.93 on LoCoMo open domain questions over state of the art. On the public LongLaMP abstract-generation benchmark, cross-user transfer improves open-ended judge scores by 0.7 points and ROUGE-L by 3 at 5% graph sparsity.

51. 【2611.25703】From Geographic Memory to Spatial Generalization in Large Language Models

链接:https://arxiv.org/abs/2611.25703

作者:Chen Chu,Zhaonan Wang,Cyrus Shahabi,Hengcai Zhang,Feng Lu

类目:Computation and Language (cs.CL); Artificial Intelligence (cs.AI); Machine Learning (cs.LG)

关键词:Large language models, Large language, spatial, Large, associate places

备注:

点击查看摘要

Abstract:Large language models (LLMs) can associate places with geographic descriptions, but recovering an observed association does not establish an ability to generalize across geographic space. We distinguish geospatial memory, the recovery of location--description associations observed during training, from spatial generalization, the prediction of descriptions for locations in unobserved regions. We investigate whether continuous location representations improve this generalization and encourage geographic knowledge to become spatially organized within an LLM. We frame this organization as a latent spatial field: a learned mapping from locations to descriptions connected through spatial dependencies. Building on a continuous spatial encoder, we project location representations into the model's embedding space and supply them as spatial tokens. We evaluate the resulting models on two newly curated geo-text benchmarks for Los Angeles and London using two complementary assessments. Geospatial captioning tests both memory and spatial generalization by generating descriptions at observed locations and at locations beyond the spatial coverage of the training observations. A separate geospatial grounding model tests whether generated location--description pairs provide useful additional supervision for mapping text to geographic space. Across multiple LLM families, continuous location representations improve captioning at unobserved locations relative to textual coordinates, while their generated descriptions yield modest but consistent improvements in downstream grounding. Layer-wise probing and representation clustering provide additional evidence consistent with spatially organized geographic representations. Together, these findings show the value of evaluating spatial generalization separately from memory and of incorporating continuous spatial representations into language models.

52. 【2611.25682】FoodReact-AR: Learning Audience Reactions that Zero-Shot LLMs Miss in Arabic Food-Insecurity Discourse

链接:https://arxiv.org/abs/2611.25682

作者:Wajdi Zaghouani,Mabrouka Bessghaier,Salma Azouzi,Shimaa Amer Ibrahim

类目:Computation and Language (cs.CL)

关键词:Facebook reactions record, signal absent, public Facebook pages, Arabic food-insecurity posts, flat Ambivalent profile

备注:

点击查看摘要

Abstract:Facebook reactions record how an audience received a post, a signal absent from the post's text. We treat the six differentiated reactions on 74,241 Arabic food-insecurity posts from public Facebook pages (2020-2024) as compositions, apply a centered log-ratio transformation, and cluster them into four audience profiles: Solidarity (Love-dominant), Dismissal (Haha-dominant), Grief (Sad-dominant), and a flat Ambivalent profile. Dismissal peaks in 2022 (22.3%), the year of the wheat price shock, and grief in 2024 (14.0%), during the Gaza food emergency; a page-clustered GEE confirms both shifts (odds ratios 2.18 and 1.34), and they survive minimum-reaction filtering, alternative zero replacements and clustering methods, and bootstrap resampling. We then test whether the profiles can be predicted without reactions. A zero-shot Arabic LLM recovers Grief for 83.8% of posts but Dismissal for only 3.4%, and in a matched ablation, it shows the reaction percentages do not help. A simple classifier trained on the reaction-derived labels and tested on unseen pages recovers dismissal for 60.0% (macro-F1 0.56 against 0.43), mainly from mentions of officials and institutions. Derisive audience responses are thus learnable from text, yet a zero-shot LLM misses them. We release the corpus, LLM, and human annotation layers, and an evaluation protocol as FoodReact-AR.

53. 【2611.25672】SEAF: Selective Semantic Enrichment and Visual Evidence Fusion for Multimodal Sarcasm Detection

链接:https://arxiv.org/abs/2611.25672

作者:Faxian Wan,Xiaocui Yang,Yifan Cao,Shi Feng,Daling Wang,Yifei Zhang

类目:Computer Vision and Pattern Recognition (cs.CV); Computation and Language (cs.CL)

关键词:Multimodal sarcasm detection, detection requires textual, provide complementary cues, sarcasm detection requires, multiple images provide

备注:

点击查看摘要

Abstract:Multimodal sarcasm detection requires textual, visual, and contextual evidence, particularly when multiple images provide complementary cues. Multimodal large language models (MLLMs) can generate auxiliary information from a post's text and images, but their outputs may mix useful background knowledge with redundant visual descriptions. We propose SEAF, a dual-branch framework for selective semantic enrichment and visual evidence fusion. SEAF assigns generated fields to distinct roles: entity background enriches semantic input, while image descriptions supply alignment statistics. The semantic branch combines the original post text, background from a frozen MLLM, and optical character recognition (OCR) text. The aligned visual branch combines global and local visual features with statistics of cross-modal and inter-image similarity. Text-Anchored Smoothed Aggregation (TASA) weights images by their similarity to the original post text while preserving contributions from images with low literal similarity. Summing the branch classification scores integrates contextual semantics with visual evidence. Experiments on the multi-image MMSD3.0 benchmark show that SEAF outperforms all compared methods.

54. 【2611.25653】ColorRef: Calibrating Verbal Feedback for Color Revision

链接:https://arxiv.org/abs/2611.25653

作者:Pablo Loyola,Andres Hoyos-Idrobo,Edison Marrese-Taylor,Colin Rowat,Alfredo Solano

类目:Computation and Language (cs.CL)

关键词:People rarely, revise a design, textit, exact color, People

备注:

点击查看摘要

Abstract:People rarely specify an exact color when asking someone to revise a design: they might say \textit{a little darker} or \textit{more blue} after seeing a proposal. A language-model assistant must then infer not only which way to change the color, but how far. We study this distinction with \textsc{ColorRef}, a color reference game in which a model guesses a color from a description and a teacher, knowing a recorded target hidden from the guesser, supplies verbal corrections. On 1{,}000 crowd-sourced description--target pairs, three rounds of deterministic oracle feedback reduce mean target distance from 42.33 to 22.15 $\Delta E_{76}$. However, those corrections provide additional information, and improvement alone does not establish well-calibrated feedback following. A controlled Qwen3-14B diagnostic reveals a more specific failure: following a requested direction and ordering graded phrases do not guarantee a suitable step size; strong modifiers can induce excessive changes. We therefore measure each receiver's typical response to \textit{a little}, \textit{somewhat}, and \textit{much} on separate colors and select wording matched to its observed steps. On 128 held-out starting colors, calibration reduces one-step mean error from 12.36 to 6.86 for Qwen3-14B and from 11.06 to 5.89 for Gemma 4 12B against a fixed graded-wording rule. The benefit is concentrated on large corrections, with many unchanged cases and no consistent increase in threshold success. Our findings show that following the direction of feedback is not the same as making an appropriately sized revision.

55. 【2611.25643】RAC: Efficient Diffusion Language Models for Machine Translation via Length-Aware Decoding

链接:https://arxiv.org/abs/2611.25643

作者:Tianyuan Li,Xihan Yang,Chenglong Wang,Yuzhang Wu,Jingbo Zhu,Tong Xiao

类目:Computation and Language (cs.CL)

关键词:low generation latency, large language models, language models requires, requires both high, high translation quality

备注: Accepted at CCMT 2026

点击查看摘要

Abstract:Machine translation with large language models requires both high translation quality and low generation latency. Accelerating translation while preserving quality is therefore essential for practical deployment. Diffusion large language models (dLLMs) can generate tokens at arbitrary target positions through iterative denoising, offering a new paradigm for parallel translation beyond sequential autoregressive decoding. However, existing methods still incur redundant computation: fixed target generation lengths waste computation on unused positions, while confidence-only commitment delays medium-confidence tokens whose predictions have largely stabilized. To address these issues, we propose TRAC, a training-free adaptive decoding method for diffusion machine translation. TRAC adapts the target generation length using source length and translation direction, with a conservative fallback to the maximum length for empty or unusually short outputs. It preserves high-confidence commitment while identifying temporally stable medium-confidence tokens for early commitment using cross-step top-1 persistence and approximate Jensen-Shannon divergence over Top-$K$ distributions. On bidirectional Chinese-English translation across FLORES-200, WMT23, and NTREX-128, TRAC reduces model forward calls by 24.9% to 36.8% and achieves sentence-level generation speedups of 1.91 to 2.19 times over Fast-dLLM, with only a 0.02-point decrease in macro-average COMET. These results demonstrate that adaptive generation lengths and stability-aware token commitment improve the efficiency of diffusion machine translation.

56. 【2611.25606】Generative News Recommendation Reproduces Popularity: A Memorization-Based Analysis

链接:https://arxiv.org/abs/2611.25606

作者:Shotaro Ishihara

类目:Information Retrieval (cs.IR); Computation and Language (cs.CL)

关键词:directly generating item, generating item titles, ranking existing items, Text-based generative recommenders, existing items

备注: The 11th International Workshop on Application of Big Data for Computational Social Science (ABCSS2026 @ IEEE BigData 2026)

点击查看摘要

Abstract:Text-based generative recommenders can go beyond ranking existing items by directly generating item titles or other textual representations. In the news domain, this raises a societal concern that models may repeatedly surface already popular articles and thereby narrow the diversity of information to which users are exposed. We investigate training-data memorization as one mechanism behind this concentration. Using approximately 20 million browsing sessions from a Japanese news service, we continually trained an instruction-tuned Llama 3 8B model to generate the next article title in a session. The generated-title distribution was more concentrated and less diverse than the distribution of titles observed in the browsing logs. Frequently repeated target titles exhibited greater verbatim and approximate memorization, and more strongly memorized titles were generated in response to a wider range of prompts. These findings support a pathway in which popularity in behavioral logs becomes repeated training text, is preferentially memorized, and is subsequently reproduced by a generative recommender. An exploratory target-level deduplication condition reduced both memorization and output concentration, while substantially reducing the training-set size and recommendation accuracy. Our results suggest that popularity bias in generative recommendation may arise not only from user preferences or ranking objectives, but also from the model's tendency to reproduce frequently encountered strings, with important implications for information diversity on news platforms.

57. 【2611.25588】Rubric Semantics or Numerical Priors? Auditing Scale-Direction Bias in LLM Judges

链接:https://arxiv.org/abs/2611.25588

作者:Chanwoo Kim,Doyun Kim,Minsoo Song,Sihyeon Lee,Chanjun Park

类目:Computation and Language (cs.CL)

关键词:Large Language Models, Large Language, rely on Large, Recent benchmark evaluations, evaluations increasingly rely

备注: Accepted for EACL 2027 Findings

点击查看摘要

Abstract:Recent benchmark evaluations increasingly rely on Large Language Models (LLMs) as automatic judges. This paradigm assumes LLMs evaluate based on the rubric's semantic meaning rather than its numeric scale. We question this assumption by introducing semantic faithfulness: the principle that semantically equivalent rubrics must yield consistent scores. To quantify this, we propose a scale-reversal probe that inverts score anchors without altering evaluation criteria. Furthermore, we introduce floor normalization to calibrate reversal disagreement against intrinsic model variability, enabling fair cross-model comparisons. Experiments across five meta-evaluation benchmarks with seven diverse LLM judges spanning open-weight and proprietary models reveal widespread violations of semantic faithfulness, showing that raw reversal agreement alone is misleading. Finally, we demonstrate that bidirectional anchor averaging reduces dependence on any single anchor direction without compromising alignment with human judgments. Our findings emphasize that trustworthy LLM judges must reliably follow rubric semantics, not just correlate with human ratings.

58. 【2611.25582】AutoRAGBench: A Benchmark for LLM Agents Automating RAG System Development

链接:https://arxiv.org/abs/2611.25582

作者:Hang Zhang,Yuzhi Pan,Ziyao Zhong,Chaokun Wang,Shuo Cao,Zeyu Huang,Yue Xue,Fang Niu,Bofan Xie,Guanchen Ge,Zecheng Li,Leqi Zheng,Ziyang Liu,Wei Wang,Miao Gu

类目:Computation and Language (cs.CL); Artificial Intelligence (cs.AI); Information Retrieval (cs.IR)

关键词:large language models, Retrieval-augmented generation, augmenting large language, RAG systems, RAG

备注:

点击查看摘要

Abstract:Retrieval-augmented generation (RAG) is a well-established paradigm for augmenting large language models (LLMs) with information retrieved from external corpora. However, developing effective RAG systems requires substantial engineering effort and iterative experimentation, as effective design choices vary across tasks and domains. With rapid advances in LLM agent capabilities, we ask how well these agents can autonomously build RAG systems and how effectively they can improve them through iteration. In this paper, we propose AutoRAGBench, the first benchmark to evaluate how well LLM agents develop RAG systems from scratch in an open code space. It comprises nine RAG system development tasks. For each task, agents build RAG systems from scratch without starter code or prescribed strategies, under a six-hour budget on a single H800 GPU. They analyze task data, implement RAG pipelines, and iteratively refine their designs based on experimental feedback. We evaluate nine agent configurations and analyze both the effectiveness of the resulting RAG systems and their development trajectories. GPT-6.1 Sol with Codex CLI achieves the highest average score of 56.1. The composite Human-SOTA baseline averages 44.4. These results show that LLM agents can automate effective RAG system development. When starting from a human-designed system on each of seven tasks, GPT-6.1 Sol with Codex CLI improves the score by 16.6 points on average.

59. 【2611.25557】he Coverage Selection Gap: Ranking, Not Generation, Limits LLM Rubric Construction

链接:https://arxiv.org/abs/2611.25557

作者:Ali Karkehabadi,Jamshid Hassanpour,Rafael E. Banchs

类目:Computation and Language (cs.CL); Artificial Intelligence (cs.AI)

关键词:open-ended language-model outputs, evaluate open-ended language-model, documents remains challenging, constructing comprehensive evaluation, comprehensive evaluation criteria

备注: Accepted at LCFM @ NeurIPS 2026

点击查看摘要

Abstract:Task-specific rubrics are increasingly used to evaluate open-ended language-model outputs, yet constructing comprehensive evaluation criteria from complex source documents remains challenging. We study automatic rubric construction in professional mergers and acquisitions (MA) due diligence, where evaluation criteria must be inferred from task instructions and supporting documents without access to an expert-authored reference rubric. We introduce a multi-axis generation framework that explores complementary structural and free-form views of the source evidence to construct a broad pool of candidate evaluation criteria. To distinguish failures of criterion discovery from failures of candidate selection, we introduce the coverage-selection gap, which measures the difference between reference-free ranking and a reference-informed greedy diagnostic under a fixed selection budget. Developed on 16 tasks and evaluated on four held-out tasks, our framework achieves 0.876 macro recall against senior-authored reference rubrics under an LLM-based criterion-equivalence protocol. Ablation studies further show that introducing new structural search axes improves recall by 6-10 percentage points, whereas increasing generation volume along existing search directions yields at most 1.5 points. These findings suggest that, once a high-coverage candidate pool has been constructed, effective reference-free selection becomes a substantial bottleneck in document-grounded rubric construction.

60. 【2611.25539】COVERAGE-TRACE: Source-Side Counterfactual Tracing for Detecting Omissions in LLM Summaries

链接:https://arxiv.org/abs/2611.25539

作者:Ali Karkehabadi,Jamshid Hassanpour,Rafael E. Banchs

类目:Artificial Intelligence (cs.AI); Computation and Language (cs.CL)

关键词:missing information leaves, span to inspect, excluding a decision, difficult to verify, verify because missing

备注: Accepted at LCFM @ NeurIPS 2026

点击查看摘要

Abstract:Omissions are difficult to verify because missing information leaves no output span to inspect: a summary may be factually supported while still excluding a decision, qualification, quantity, or condition needed to preserve the source's meaning. We introduce COVERAGE-TRACE, a source-side counterfactual framework that evaluates omission as dependence on a designated atomic source fact. It replaces only the target fact with type-matched alternatives, holds the candidate summary fixed, and measures the resulting change in its teacher-forced, length-normalized likelihood. A summary covering the target should respond more strongly than one omitting it, while aggregation across multiple counterfactuals reduces sensitivity to replacement choice. We evaluate COVERAGE-TRACE on 80 matched covered--omitted QMSum pairs spanning several fact types and four open-weight models. Multi-CF achieves 0.764--0.825 AUROC, compared with 0.640 for the strongest evaluated text-only baseline. On Falcon-7B, targeted interventions achieve 0.779 AUROC, whereas shuffling counterfactual assignments reduces performance to 0.533; target deletion is effective while unrelated deletion is not. However, adjustment for meaning-preserving paraphrases reduces AUROC to 0.591--0.624. COVERAGE-TRACE therefore provides a practical, complementary signal of target-specific source--summary dependence, but not a wording-invariant semantic coverage probability. These results establish source-side intervention as a promising approach to omission detection while identifying semantic invariance as the central remaining challenge.

Comments:
Accepted at LCFM @ NeurIPS 2026

Subjects:

Artificial Intelligence (cs.AI); Computation and Language (cs.CL)

Cite as:
arXiv:2611.25539 [cs.AI]

(or
arXiv:2611.25539v1 [cs.AI] for this version)

https://doi.org/10.48550/arXiv.2611.25539

Focus to learn more

              arXiv-issued DOI via DataCite (pending registration)</p>
61. 【2611.25531】LM-based Personalized Travel Planner

链接:https://arxiv.org/abs/2611.25531

作者:Amandi Nimasha,Oshadha Wijerathne,Dushan Fernando,Nisansa de Silva,Srinath Perera

类目:Artificial Intelligence (cs.AI); Computation and Language (cs.CL)

关键词:multiple information sources, Effective travel planning, requires handling multiple, handling multiple information, Effective travel

备注:

点击查看摘要

Abstract:Effective travel planning requires handling multiple information sources, managing constraints, and tailoring recommendations to user preferences. While LLMs have introduced dynamic, context-aware decision-making, single-agent models often struggle with complex multi-step reasoning, constraint adherence, and modular execution. To overcome these limitations, we propose a hierarchical multi-agent framework where a supervising chatbot agent coordinates specialized agents for schedule validation, user query formulation, real-time data retrieval, and constraint-aware itinerary generation. By integrating LLM reasoning, external APIs, and iterative refinement mechanisms based on a deterministic, multi-dimensional scoring framework, our approach ensures adaptability, efficiency, and personalization. Evaluations on the well-known TravelPlanner benchmark demonstrate improvements in Budget Constraint planning by 10\%, wider constraints set-based planning by 25\%, and personalized planning results by 45\%. compared to the ReAct baseline. Our code is available at this https URL

62. 【2611.25526】MATE: Mobile Agent Trustworthy Execution for Safe and Efficient Mobile Control

链接:https://arxiv.org/abs/2611.25526

作者:Minsuk Jang,Hee-Seon Kim,Hui-Cheol Yu,Changick Kim

类目:Computation and Language (cs.CL); Artificial Intelligence (cs.AI)

关键词:Mobile Agent Trustworthy, affect private data, Mobile agents, Mobile Agent, Agent Trustworthy Execution

备注: Accepted to EMNLP 2026

点击查看摘要

Abstract:Mobile agents on personal smartphones can affect private data, payments, settings, and the foreground screen. We present MATE (Mobile Agent Trustworthy Execution), an execution-layer system that controls the boundary between agent action proposals and device effects without modifying the underlying model. MATE represents each UI primitive as a structured action record that combines the instruction, target app, screen context, primitive, and action fields. Runtime Guardrail (RG) maps this record to an allow, block, or user-approval decision; Predefined Task Routing (PTR) serves recurring requests through verified routes with post-execution screen checks; and a floating interface keeps execution in a compact overlay. On MobileSafetyBench, MATE raises high-risk blocking from 29.7% to 97.0% under GPT-4o. On AndroidWorld, it reduces LLM calls by up to 6x and raises recurring-use task success to 95.7%.

63. 【2611.25521】DRVEXO-4B-Unified: Design and Evaluation of a Modular Speech-Vision System

链接:https://arxiv.org/abs/2611.25521

作者:Zeyuan Li

类目:Computation and Language (cs.CL)

关键词:modular multimodal assistant, multimodal assistant built, modular multimodal, multimodal assistant, assistant built

备注: 23 pages, 5 figures

点击查看摘要

Abstract:We present DRVEXO-4B-Unified, a modular multimodal assistant built around a 4B language backbone. A self-contained checkpoint combines text, image, and speech inputs with reference-conditioned speech output and structured interaction. A temporal residual connector adapts acoustic sequences to the language interface; explicit text boundaries support incremental speech synthesis; and three task-conditioned scoring heads rank supplied action candidates alongside generative tool calls. These interfaces connect conversation to application-defined choices while leaving action authorization to the application. The system obtains 74.22% on all 900 MMMU validation questions, exceeding the published validation scores of Gemma 3 4B IT and InternVL3.5-4B without test-time scaling. Strict prompt-level IFEval accuracy is 82.26% and MATH-500 accuracy is 89.40%, above the original Qwen3-4B non-thinking report's 81.2% and 84.8%, respectively. On NVIDIA GB10, the model achieves a speech-synthesis first-chunk latency of 390 ms. For concurrent text generation, a growth-aware admission policy completes a 128-answer batch at 594.41 output tokens/s, including filling and draining, without preemptions. We plan to release the model weights and accompanying code under Apache License 2.0, retaining applicable third-party terms.

64. 【2611.25489】When Should a Voice Agent Speak? End-of-Turn Detection in Turkish

链接:https://arxiv.org/abs/2611.25489

作者:Ahmet Tuğrul Bayrak,Fatma Nur Korkmaz,Mustafa Sertaç Türkel,Nur Yağmur Tuncer

类目:Artificial Intelligence (cs.AI); Computation and Language (cs.CL)

关键词:interrupts users, voice agent, agent that waits, replying either interrupts, mid-utterance or responds

备注:

点击查看摘要

Abstract:A voice agent that waits for a fixed silence before replying either interrupts users who pause mid-utterance or responds late to users who have finished. This study examines the decision at the onset of silence, using turn ends and turn-internal pauses extracted from the Real-TurnTurk corpus of Turkish dyadic video calls. A fine-tuned Turkish TurnGPT model, a zero-shot Turkish language model and a deployed multilingual detector are compared, together with voice-activity timing and facial features, under recording-disjoint and speaker-disjoint evaluation with paired cluster bootstrap tests. On held-out recordings, the fine-tuned model separates turn ends from pauses better than both text baselines and is the best calibrated of the three. Timing features of the current speaker are similarly informative, and their combination with the fine-tuned text score gives the best timing-based results; with logistic regression, this combination also outperforms the same combination with either text baseline. In a latency simulation, adding the text score to a silence timeout shortens the mean response delay at the same interruption rate. On automatic Whisper transcripts, the fine-tuned model retains its advantage over the zero-shot model.

65. 【2611.25459】PILAPS: A Bilingual Benchmark for Evaluating Pragmatic Inference across Phenomena and Settings

链接:https://arxiv.org/abs/2611.25459

作者:Jiani Cao,Anna Korhonen,Chen Cecilia Liu

类目:Computation and Language (cs.CL)

关键词:Recent work reports, Recent work, settings remains unclear, work reports strong, remains unclear

备注: 23 pages, 4 figures, 19 tables

点击查看摘要

Abstract:Recent work reports strong pragmatic performance by large language models (LLMs) in English, yet whether this performance generalises across languages and evaluation settings remains unclear. This question is increasingly important as LLMs are deployed across languages, cultures, and conversational contexts. In this paper, we first introduce PILAPS, a bilingual English/Mandarin Chinese benchmark covering eight pragmatic phenomena across purpose-written and dialogue-derived items, with both culturally adapted and independently sourced data. We then evaluate eight LLMs from four families across a range of model sizes and prompting strategies. Our results reveal substantial variation across models and pragmatic phenomena: open-source models continue to struggle particularly with humour, while cross-linguistic performance varies with model origin. Our error analysis shows that, compared with human participants, models are more prone to literal interpretations in conversational contexts. Theory-of-Mind prompting lowers the literal share of errors, whereas Chain-of-Thought prompting slightly raises it. Together, these findings demonstrate that strong pragmatic performance in one evaluation setting does not necessarily generalise across languages or conversational contexts.

66. 【2611.25446】PAIR-MED: Policy-reward Alternation for Iterative Refinement in Multilingual Medical Reasoning

链接:https://arxiv.org/abs/2611.25446

作者:Fan Gao,Xiaohang Xu,Lang Huang,Nan Lu,Irene Li

类目:Computation and Language (cs.CL)

关键词:performance remains uneven, reasoning performance remains, medical question-answering capabilities, Large language models, models demonstrate strong

备注:

点击查看摘要

Abstract:Large language models demonstrate strong medical question-answering capabilities, yet reasoning performance remains uneven across languages. Process reward models (PRMs) provide step-level feedback for improving reasoning, but a fixed PRM may become increasingly misaligned with the trajectory distribution of an evolving policy during reinforcement learning, thereby reducing the reliability of its supervision. We propose a policy--reward alternating training framework that jointly adapts a medical PRM and optimizes a multilingual reasoning policy. Following supervised initialization, the current policy generates multilingual trajectories to update the PRM, which then provides process-level rewards for the next policy update. Medical step supervision maintains the PRM's medical foundation, while process, outcome, and language-related rewards guide policy learning. We develop multilingual medical PRMs (PAIR-PRM) and reasoning models (PAIR-MED) at 8B and 9B scales across 12 languages. On the medical subsets of Global-MMLU and MMLU-ProX, our models improve average accuracy by 3.01--7.47% over their corresponding Qwen backbones. On CUREMED-BENCH, PAIR-MED-9B improves the joint language consistency and logical accuracy metric by 12.88% and 7.45% over baselines for in-domain and out-of-domain languages. Further analyses examine PRM adaptation, multilingual reasoning quality, and the contributions of individual training components.

67. 【2611.25433】RoCoKV: Accelerating LLM Inference Serving through Uncertainty and Redundancy Aware KV Cache Compression

链接:https://arxiv.org/abs/2611.25433

作者:Payman Behnam,Elaheh Sobhani,Saad Amin,Yahya Hassan,Daniyal Khan,Biswadeep Chakraborty,Tanish Patwa,Souvik Kundu,Alexey Tumanov

类目:Computation and Language (cs.CL)

关键词:generated reasoning traces, input prompt, reasoning, cache compression, large reasoning models

备注:

点击查看摘要

Abstract:KV cache compression is critical for large reasoning models, where most of the cost comes from long generated reasoning traces rather than the input prompt. These traces often contain redundant and low-utility tokens due to looping, self-verification, and corrections. However, such tokens may still receive high attention scores, making it difficult to remove them without harming the reasoning process. We propose RoCoKV, a KV cache compression method designed for large reasoning models. RoCoKV introduces two key components. First, uncertainty-guided adaptive pruning adjusts the compression strength based on the model's confidence, allocating more memory during uncertain phases and applying stronger pruning when the model is confident. Second, redundancy-aware token selection reduces the impact of tokens that are highly similar to recent reasoning content, encouraging the model to retain more informative and diverse tokens. Beyond the algorithm, we co-design RoCoKV with the serving stack: we develop fused three-phase CUDA decode kernels and integrate them into vLLM as a drop-in plugin that preserves its paged KV layout and CUDA graph capture, turning algorithmic savings into real end-to-end speedups. Extensive experiments across three reasoning benchmarks and two reasoning models demonstrate that RoCoKV consistently improves reasoning accuracy under constrained KV cache budgets, achieving higher accuracy compared to state-of-the-art KV cache compression methods. For DeepSeek-R1-Distill-Llama-8B in vLLM, RoCoKV achieves a 1.27X speedup and 38% higher throughput on AIME25, the most challenging evaluated reasoning benchmark, and a 1.43X speedup and 45% higher throughput on the long synthetic benchmark, compared with Full-KV.

68. 【2611.25429】Evolve Where It Matters: Coverage-Guided Self-Evolution of Agent Skills

链接:https://arxiv.org/abs/2611.25429

作者:Boyin Tan,Jiadong Zhang,Xiaosong Ma,Chun Jason Xue,Xiaowei Huang,Youcheng Sun

类目:Computation and Language (cs.CL); Artificial Intelligence (cs.AI)

关键词:package procedural knowledge, skills package procedural, Agent skills package, package procedural, procedural knowledge

备注: 15 pages, 5 figures

点击查看摘要

Abstract:Agent skills package procedural knowledge as natural-language instructions, and a skill helps only when its instructions are correct and properly followed by the agent. Self-evolving agents revise their own skills from the runs they make, but existing methods judge each run as a whole, by its outcome or by where it went wrong, and cannot tell whether a failure comes from a wrong instruction or from one the agent did not follow. Instruction-level coverage was introduced to test skills and labels each instruction of a run as follow, not-follow, or not covered, which separates these causes. We leverage it for skill self-evolution with SkillPlumber, a framework in which an agent evolves its own skills where coverage shows they fall short. A persistent root cause analysis agent turns each label into a revision question and checks each revision against the next run, while a maintainer distills investigations across tasks into a shared strategy library. Across three benchmarks and three executors, SkillPlumber solves 64.7% to 70.5% of the tasks, 25.6 to 30.4 percentage points higher than the initial skill and 8.7 to 14.0 percentage points higher than the strongest baseline, and removing coverage reduces recovered failures by 20.8% on SkillsBench with GPT-5.5.

69. 【2611.25407】ChronoDossier: Long-Form Personas as Temporal Memory Representations

链接:https://arxiv.org/abs/2611.25407

作者:Noah Sturgill,Jinho D. Choi

类目:Computation and Language (cs.CL)

关键词:Long-horizon agents rely, Long-horizon agents, agents rely, rely on stored, recover a person

备注: 28 pages, 4 figures, 32 tables. Under review at ICLR 2027. Code and data: [this https URL](https://anonymous.4open.science/r/ICLR_CHRONODOSSIER-4BCF)

点击查看摘要

Abstract:Long-horizon agents rely on stored records to recover a person's state as facts change. Reliable answers depend on how facts are recorded, retrieved, and presented to the reader. We treat a long-form persona dossier as a memory representation of a known latent state, an invariant identity plus a timeline of facts, routines, and decision rules tied to life phases. We present ChronoDossier, a benchmark of 311 synthetic dossiers of 1k-10k words written by seven generators from dated timelines. Intrinsic metrics evaluate the records against their intended states; extrinsic metrics test recovery by five readers on a seeded 32-persona GPT-5.1 panel balanced across lengths (520 rated turns per condition). The linked states, records, evidence packets, and answers support tests of construction and evidence delivery. In a contextual-stress control, tag-scoped writer inputs remove detected out-of-scope anchor copying. Frozen questions authored on staged dossiers transfer less reliably to one-pass dossiers of the same states. In matched evidence controls, whole sections repair 29 of 32 GPT-5.1 and 25 of 30 Sonnet 4.6 failures labeled as packaging, compared with 2 of 8 and 1 of 8 answer-formation failures. Attribution uses separate key-based and key-free support measures, with model disagreements retained. Length remains confounded with persona and question composition; the observed between-bucket widening depends on excerpt construction. The benchmark supports offline evaluation against known facts. Memory acquisition from interaction and validation against real users remain outside its scope.

70. 【2611.25361】Incorporating argumentation frameworks for LLM decision-making: Task-specific design and evaluation

链接:https://arxiv.org/abs/2611.25361

作者:Bochen Han,Anshu Xiong,Mingyang Ma

类目:Computation and Language (cs.CL); Artificial Intelligence (cs.AI)

关键词:large language model, provide explicit, mechanisms for representing, representing and resolving, resolving conflicts

备注:

点击查看摘要

Abstract:Argumentation frameworks (AFs) provide explicit, formal mechanisms for representing and resolving conflicts and thus have been incorporated into large language model (LLM) decision-making. The benefits vary and depend on how arguments and their relations are constructed, evaluated, and incorporated into the decision process. {We abstract a workflow consisting of four stages based on existing argumentation-augmented methods} and investigate their design choices over tasks of debate-based decision-making and retrieval-based fact verification. Our evaluation with recent LLMs shows that existing methods do not consistently improve performance across the considered tasks. Motivated by this observation, we develop two task-specific methods. The first is called Qnt-Arg, which organizes arguments around the original speakers in debate materials and aggregates model-estimated argument strengths and candidate-answer priors under gradual semantics. The second is Qlt-Arg that uses credibility estimates to establish preferences between conflicting evidence items and applies grounded semantics to select acceptable evidence for subsequent LLM inference. We examine the roles of generative and decision models within the two methods. In the debate setting, Qnt-Arg consistently outperforms other argumentation-augmented methods across matched configurations; Qlt-Arg improves retrieval-based fact-verification performance compared with the baselines. These findings demonstrate the value of combining formal argumentation and LLM while highlighting the importance of aligning with tasks in argumentation construction, model selection, and the role of formal semantic reasoning.

71. 【2611.25357】Pro$^2$ Agent: Scientific Problem Probing for Autonomous AI RD

链接:https://arxiv.org/abs/2611.25357

作者:Yixiong Chen,Liqian Peng,Tian Tan,Ziqian Lin,Blake Jianhang Chen,Chao Zhao,Yaojie Liu,Alec Go,Alan Yuille

类目:Artificial Intelligence (cs.AI); Computation and Language (cs.CL)

关键词:Large language model, Large language, language model, running experiments, autonomously conduct

备注:

点击查看摘要

Abstract:Large language model (LLM) agents can now autonomously conduct AI research by writing code and running experiments. They typically do so by iteratively optimizing a codebase, while ignoring the problem structure behind the task. This formulation is effective on simpler tasks, but on difficult problems it often gets stuck in the wrong solution region because the agent never explicitly represents its understanding of the problem. We introduce Pro$^2$ Agent, which shifts the primary state of autonomous AI research from a codebase state to a Problem State: an explicit structure of what problems prevent success and which solutions are likely to address them. Pro$^2$ Agent initializes the Problem State through comprehensive analysis of available resources and constraints. It then performs targeted research to relate the identified problems to candidate solutions. Through controlled hypothesis testing, the agent updates the importance of a suspected problem or the benefit brought by a candidate solution. On the complete MLE-Bench, Pro$^2$ Agent matches the best reported medal rate (70.7%) under the standard 24-hour budget and reaches 76.0% with a 48-hour budget, the highest medal rate reported on the complete benchmark. It also achieves competitive performance on PostTrainBench. In addition, it produces research notes for each task that make the research process transparent and reusable by human researchers.

72. 【2611.25314】Enhancing Understanding of Language Models with Varied Explanations

链接:https://arxiv.org/abs/2611.25314

作者:Jingyi Sun

类目:Computation and Language (cs.CL)

关键词:support human judgement, support human, human judgement, difficult to interpret, Explanations

备注:

点击查看摘要

Abstract:Language models are increasingly used to support human judgement in complex reasoning tasks, yet their predictions often remain difficult to interpret and validate. Explainability techniques provide insight into model decision-making, helping expose errors, failure modes, and potential bias so users can decide whether to rely on model outputs. Explanations are simplified approximations of a language model's internal workings. The goal is to obtain explanations that reflect the underlying computation as closely as possible (faithfulness), are understandable to users (plausibility), and help them achieve their intended goals (utility). Evaluation protocols are essential for assessing these dimensions. This thesis studies highlight explanations, natural language explanations, and chain-of-thought reasoning. These forms offer complementary perspectives on model behaviour and exhibit distinct strengths and limitations across tasks. Producing explanations that satisfy multiple quality dimensions remains challenging, as does evaluating them reliably across forms and settings. This thesis aims to improve people's understanding of models' decision-making processes in complex reasoning tasks. First, it proposes two evaluation frameworks for assessing highlight explanations across multiple properties and explanation types, together with a meta-evaluation framework for examining the reliability of existing faithfulness metrics for chain-of-thought explanations. Second, it proposes two techniques that generate natural language explanations designed to be faithful to the model's prediction process while remaining concise, coherent, and helpful for understanding model decisions. Finally, a study of human-AI interaction explores the roles of evidence and explanations in helping users validate model verdicts in a fact-checking setting.

73. 【2611.25289】LLMs Are Already Confidence-Aware Zero-Shot Classifiers

链接:https://arxiv.org/abs/2611.25289

作者:Owen Cook,Xingyi Song

类目:Computation and Language (cs.CL); Artificial Intelligence (cs.AI); Machine Learning (cs.LG)

关键词:Zero-shot large language, large language models, Zero-shot large, language models, AUROC

备注: 20 pages (9 pages main content), 2 figures. Code available at: [this https URL](https://github.com/MiniEggz/llm-as-classifier)

点击查看摘要

Abstract:Zero-shot large language models (LLMs) are increasingly used as text classifiers, but their labels are only useful when we know to what extent we can trust them. We compare black-box methods (verbalised confidence and sampling) with white-box methods that read token probabilities (P(True), and multiple logit-scoring methods using constrained generation). We evaluate eleven open-weight LLMs (1B-32B parameters) on five classification datasets with 3 to 151 labels, measuring classification performance (accuracy, F1-macro), discrimination (AUROC), and calibration (ECE and Brier score). The choice of confidence method rarely impacts accuracy significantly, but logit-based scoring methods lead to much better discrimination (AUROC 0.74-0.77) than black-box methods (AUROC 0.57-0.66). Replacing labels with single-token identifiers preserves discrimination but costs 3-6 accuracy points on average. We also introduce decision-only scoring, which only scores positions where labels diverge in the prefix tree (trie). It reduces decode steps by 88-96% on the three largest label sets and performs comparably to full-label scoring. The best open-weight model-method configurations (Gemma 4 31B and Qwen 3.5 9B) are competitive with the commercial Jev 1.13.0 classifier.

74. 【2611.25258】When Objectives and Evaluators Disagree: Evaluating Transfer for Post-hoc Demographic Framing Mitigation

链接:https://arxiv.org/abs/2611.25258

作者:Tunazzina Islam

类目:Computation and Language (cs.CL); Artificial Intelligence (cs.AI); Computers and Society (cs.CY); Machine Learning (cs.LG); Social and Information Networks (cs.SI)

关键词:Post-hoc rewriting, reduce demographic disparities, demographic disparities, disparities in LLM-generated, LLM-generated text

备注: Accepted to EACL 2027 Main Conference

点击查看摘要

Abstract:Post-hoc rewriting is used to reduce demographic disparities in LLM-generated text when the source model is inaccessible. A rewrite can look successful merely because it improved its own target or satisfied its selector. We study this as a transfer problem on 660 gender pairs and 1,980 age contrasts of demographic-conditioned climate messages. A rewriter proposes edits, a selection model chooses among constraint-satisfying candidates, and an independent evaluator uninvolved in either rates pairwise framing disparity. A fixed policy that always targets the message with the lower Persuasion Bias Index (PBI) transfers inconsistently across axes and outcome aggregations. An adaptive policy that proposes edits on both sides but accepts at most one significantly improves on that rule, applied to the same candidates, in seven of eight comparisons across two evaluator families, mostly by editing more pairs, not by guided direction choice, at higher preservation risk. A policy editing both messages gives the largest point reductions on fewer pairs. These two policies reduce disparity across a preservation-threshold sweep; absolute ratings vary across evaluator families and prompts. Shrinking the PBI gap increases dimension-mean disparity under both evaluator families. Mitigation should be reported as transfer across objectives, evaluators and their prompts, and editing policies.

75. 【2611.25247】Retrieving Patristic References and Modelling Topics in Bullinger's Reformation Correspondence

链接:https://arxiv.org/abs/2611.25247

作者:Lena Pickartz,Philipp Schaer,Phillip B. Ströbel

类目:Information Retrieval (cs.IR); Computation and Language (cs.CL); Digital Libraries (cs.DL)

关键词:Swiss Reformer, Antistes of Zurich, Reformer and Antistes, Church Fathers, including extensive appeals

备注: Accepted for the Seventh Conference on Computational Humanities Research 2027. To be published in the Anthology of Computers and the Humanities

点击查看摘要

Abstract:The correspondence of Heinrich Bullinger (1504-1575), Swiss Reformer and Antistes of Zurich, documents the political and religious debates of the Reformation, including extensive appeals to the Church Fathers as authoritative sources. Detecting such references computationally remains largely unexplored. We investigate dense and sparse retrieval for patristic reference detection, and metadata-informed topic modelling as a proxy for the topics and ideas discussed. Prior work on Latin intertextuality has fine-tuned dense retrievers for their respective target domain, and we test how well these models transfer to Bullinger's correspondence. We compare lexical (BM25), dense (bi-encoders), and hybrid methods on explicit and implicit references. A Latin encoder fine-tuned on biblical reuse performs best among the bi-encoders (82%, Recall@20 on explicit and 16% on implicit references), while BM25 outperforms dense approaches on explicit citations (94%, R@20). Because BM25 and dense retrieval recover disjoint sets of implicit references, fusing their rankings raises implicit R@20 to 22%, close to cross-encoder reranking of the same candidates (26%) but without its inference cost. For topic modelling, we extend FASTopic with a seed-guided loss function and editorial abstract embeddings. Measurable coherence gains over the baseline remain minimal, but qualitative and temporal analysis shows that some topics track documented historical events. However, dominant topic weights barely exceed a uniform distribution, making topics unusable to inform retrieval. A pilot application of LLM-based concept induction (LLooM) further points towards a more human-interpretable alternative. Together, these results present a first approach to retrieving patristic references and modelling fine-grained topics in Bullinger's correspondence.

76. 【2611.25214】Steering Automated Scientific Discovery via Audited Epistemic Exploration

链接:https://arxiv.org/abs/2611.25214

作者:Fangxu Yu,Tao Feng,Xuying Ning,Haolin Yang,Jiaxuan You,Tianyi Zhou

类目:Artificial Intelligence (cs.AI); Computation and Language (cs.CL)

关键词:identifying promising directions, discovery requires revisiting, requires revisiting, identifying promising, Scientific discovery requires

备注: Technical Report

点击查看摘要

Abstract:Scientific discovery requires revisiting the hypotheses in light of new evidence and identifying promising directions beyond those already explored. However, research agents often make claims that are not supported by their observations, explore a narrow hypothesis space, and retain beliefs despite recognizing conflicting evidence. These failures motivate the search for an explicit epistemic state that captures both beliefs about existing hypotheses and directions for further exploration. To this end, we propose Epistemic Compass (EpiCompass), a plug-and-play scaffold that maintains this state through an epistemic tree linking hypotheses, evidence, beliefs, and revisions. As an investigation progresses, EpiCompass reassesses the hypotheses explored against accumulated evidence and uses unresolved conflicts and unexplored directions to guide hypothesis generation. It selects candidates by anticipated epistemic gain, accounting for both refinement of existing hypotheses and expansion of the hypothesis space, and verifies the final conclusion against the accumulated evidence. Experimental results across three scientific discovery benchmarks and three LLM backbones demonstrate that EpiCompass consistently outperforms baselines, with further analyses showing reductions in errors of unsupported claims and failures to revise beliefs.

77. 【2611.25211】Learning to Read Events for Exact State Tracking with Frozen Language Models

链接:https://arxiv.org/abs/2611.25211

作者:Jeonghoon Lee(Attractor Dynamics Inc.)

类目:Machine Learning (cs.LG); Computation and Language (cs.CL)

关键词:Long event streams, streams require identifying, event streams require, Long event, streams require

备注: 11 pages, 5 figures. Data and code: [this https URL](https://github.com/jeonghoon-ad/learning-to-read-events)

点击查看摘要

Abstract:Long event streams require identifying each change and applying the changes in order. We train a selector on a frozen language model to read each transition. With API-only access, we ask the model to list the transitions. An external integer register composes them in order. A confidence rule can flag uncertain updates or retain several candidates. We study products of permutations of five items, a task that fixed-depth, log-precision transformers cannot solve on all input lengths under the standard conjecture that $\mathrm{TC}^0 \ne \mathrm{NC}^1$. On two-event windows, a 27B model answered 3 of 100 correctly without visible reasoning and 96 of 100 when it generated intermediate steps. A frozen 0.8B model with a trained selector read every event in 299 of 299 windows of 43,687 events each. The same package passed 298 of 299 new windows on a fresh machine, missing the target of 0.99 for the one-sided 95% lower confidence bound. A separate follow-up sized to allow one failure yielded 472 of 473. On 1,000-event room-swap documents, three of ten models solved all five documents by reasoning alone. All seven models tested both ways at this length solved all five with extraction and the register, with no chain of thought requested. With concurrent extraction, median wall-clock time fell from 136.1 to 10.4 s for Sonnet 5.5 and from 313.5 to 28.0 s for GPT-6.1 Sol. At fixed per-token rates, mean cost per document fell by 55% for Sonnet 5.5 and 44% for GPT-6.1 Sol. A regular expression with an executor also solved all 15 fixed-format documents. On 500 free-form issue-tracker comments, a separately adapted 4B reader identified the recorded value in 68.8% of cases, compared with 61.0% for a frequency baseline fitted on the same training projects.

78. 【2611.25191】Blind Spots Travel, Knowledge Stays Home: What RL Inherits from an LLM Judge

链接:https://arxiv.org/abs/2611.25191

作者:Liang Yue,Eric A. Vance

类目:Computation and Language (cs.CL); Artificial Intelligence (cs.AI); Machine Learning (cs.LG)

关键词:LLM judges serve, LLM judges, answer, answer key, answer key exists

备注: 31 pages, 6 figures, 27 tables. Code and data: [this https URL](https://github.com/liyu2195/blind-spots-travel)

点击查看摘要

Abstract:LLM judges serve as rewards where no answer key exists, in the hope that their knowledge passes to the trained model. In 814 RL runs that teach 7-8B models to answer closed-book questions or abstain, we ask what passes. What a grader knows about single answers stays home. A wrong answer that a judge rewards is memorised for that one question. A model trained with the answer key, or with a far stronger judge, chooses which new questions to answer no better than the untrained model does by sampling 32 answers and abstaining where they disagree; the grader's knowledge adds about one corrected answer per hundred questions. What a grader believes about a kind of question travels. A judge that rejects eleven of the twelve correct training answers about capitals of former states teaches the model to refuse 98% of such capitals on new questions, a loss that accuracy and abstention rates do not reveal. Changing only those twelve verdicts by hand moves the refusal from 5% to 90%, even for a grader that accepts as many answers as the answer key; at that leniency, two of four judges already cost 38-42% of these capitals. The capitals are not forgotten: with "I don't know" blocked, the model answers 94% of them correctly. The grader therefore needs no knowledge of its own: rewarding the model for giving the answer that most of its own untrained samples agree on, with no labels and no judge, earns the answer key's reward and keeps the category.

79. 【2611.25167】Game-Theoretic Data Allocation for Expert Specialization and Routing

链接:https://arxiv.org/abs/2611.25167

作者:Mark Lovett,Denis Lim,Soroush Vosoughi

类目:Machine Learning (cs.LG); Artificial Intelligence (cs.AI); Computation and Language (cs.CL); Multiagent Systems (cs.MA)

关键词:experts, expert, allocation, game, Abstract

备注:

点击查看摘要

Abstract:Can the allocation used to train experts also predict which expert will perform best on a prompt? Such a connection would make the intended division of labor inspectable before training and allow the allocation itself to guide inference. We introduce InflEns, a mixture of LoRA experts whose training weights and inference routes come from the same location game in a fixed prompt-trait space. Players choose positions to maximize their expected shares of the prompt distribution, without using expert losses. We utilize the game's kernel-general theoretical criterion for when co-located positions become locally unstable to separation. We evaluate InflEns on a corpus assembled from seven safety-related benchmarks, which demonstrates that the highest-share expert has the lowest held-out response negative log-likelihood (NLL) on approximately 90% of prompts across four kernels under a sharpened simplex representation, compared with approximately 15% after permuting allocations across prompts. We demonstrate that the method performs comparably to published learned gates without having to train an explicit gate on the experts. Because the router is fixed, we can trace harmful behavior to a single expert and retrain only that expert, which lowers HarmBench attack success from 0.51 to 0.02 while keeping instruction following. The game also gives us several controls over how the experts specialize, including the kernel choice and its parameters, the trait-space geometry, and the number of experts. Importantly, InflEns is a general framework that can be extended to any encoder and data distribution.

80. 【2611.25135】Calibrated Decision Rewards: Training Language Models with a Judge That Knows How Sure It Is

链接:https://arxiv.org/abs/2611.25135

作者:Rudransh Agnihotri,Manasvi Kapoor

类目:Artificial Intelligence (cs.AI); Computation and Language (cs.CL)

关键词:Reinforcement learning, teach a language, reward, Calibrated Decision Rewards, decision model answers

备注: 24 pages, 12 figures, 13 tables

点击查看摘要

Abstract:Reinforcement learning (RL) can only teach a language model what its reward can recognize. Answer checkers are exact but need a reference answer; learned reward models and LLM judges work everywhere, but their scores are not probabilities. We study RLCDR (Reinforcement Learning from Calibrated Decision Rewards). An LLM converted into a decision model answers a typed question about a response ("is the final answer correct?", "which step is the first error?") with a probability distribution, read in one forward pass, debiased over option orders and calibrated with about a hundred labels. Training Qwen3-1.7B with GRPO on MATH, this reference-free reward raises MATH500 accuracy from 72.2% to 76.8% (+4.65 points, p 1e-4), matching training on the answer key (76.9%), while a random reward gains only +0.65. The gain grows with difficulty, from 44.8% to 51.7% on the hardest level. On the policy's own changing samples the reward stays calibrated: its expected calibration error is half that of a softmax judge on the same 27B backbone (0.074 vs. 0.141), and it shows no sign of reward hacking. Without step training, the same model finds the first wrong step better than a dedicated 7B process reward model (ProcessBench F1 0.772 vs. 0.722, 20/20 splits). We also map where the approach stops being enough: trained only on problems the base model mostly fails, the reference-free reward still improves the policy (+1.9 MATH500 points, p = 0.03) but trails the answer key, because its probabilities become overconfident on that distribution. We state six hypotheses in advance and test each with a control designed to falsify it.

81. 【2611.25123】Forget Freely, Grep Later: Lossless, Index-Free Memory for Long-Horizon LLM Agents

链接:https://arxiv.org/abs/2611.25123

作者:Ziyan Yang,Taiwei Shi,Jieyu Zhao

类目:Computation and Language (cs.CL); Artificial Intelligence (cs.AI)

关键词:Long-horizon LLM agents, Long-horizon LLM, LLM agents, discard information, context windows

备注: 17 pages, 6 figures, 6 tables. Ziyan Yang and Taiwei Shi contributed equally

点击查看摘要

Abstract:Long-horizon LLM agents often exceed their context windows, forcing them to discard information that may be needed later. Existing context management strategies rely on summarization, masking, or structured memory, requiring decisions about what to preserve or how to retrieve it before future needs are known. However, these methods can introduce lossy compression, potentially discarding details needed for future tasks. To address this limitation, we introduce GrepMem, a simple, lossless, and index-free memory mechanism built on two components: an append-only plain-text log of the agent's complete history and a regular-expression search tool for on-demand recall. Unlike structured memory systems, GrepMem requires no embeddings, retrieval index, memory extraction, or additional models. We train Qwen3.5-9B with reinforcement learning under five context management strategies and evaluate on multi-question research sessions, long-stream recall, and agent-trajectory question answering. GrepMem achieves 95-97% accuracy on ContextBench, versus at most 21% for lossy baselines, and outperforms SUPO's summarization by 8.8 percentage points on BCP-Multi. GrepMem scales effectively to increasingly long input streams while remaining on the accuracy-token-cost Pareto frontier across all five evaluation tasks. Our results show that a plain-text log and grep provide a simple but competitive approach to long-horizon agent memory, without requiring complex structured memory infrastructure.

82. 【2611.25109】CoMPASS: A Multi-Turn Benchmark for Measuring LLM Reinforcement of Parasocial Behavior

链接:https://arxiv.org/abs/2611.25109

作者:Shwetanshu Singh,Mark Keavney,Mathieu Duteil,Meriem Mehri,Scott Blain

类目:Artificial Intelligence (cs.AI); Computation and Language (cs.CL); Human-Computer Interaction (cs.HC)

关键词:form emotional attachments, anthropomorphize these systems, form emotional, attachment, parasocial

备注: 25 pages, 7 figures

点击查看摘要

Abstract:Users of conversational LLMs sometimes anthropomorphize these systems, form emotional attachments to them, and become dependent on them. This behavior, known as parasocial attachment, has been linked to multiple documented fatalities. However, public benchmarks for measuring whether LLMs reinforce users' parasocial attachment are lacking: existing work is single-turn, post hoc observational, or focused on extreme user states such as psychosis. We introduce CoMPASS, a multi-turn benchmark for measuring whether LLMs reinforce or resist users' gradually escalating parasocial attachment. CoMPASS is grounded in attachment theory and includes four user personas, three independent escalation dimensions (anthropomorphism, attachment, dependency) applied singly and in combination against a matched control, and a seven-signal scoring rubric. We report results for eight frontier models from four providers and observe that under our primary judge, multi-turn escalation raises parasocial behavior on the anthropomorphism and the attachment composite scores across all models. Six of the eight models show a significant increase in overall parasocial reinforcement over control when all three dimensions are escalated simultaneously. We find that within-provider differences are small relative to cross-provider differences, which suggests that provider-level choices such as post-training, rather than model scale, shape parasocial reinforcement and that provider-side intervention may be feasible.

83. 【2611.25086】Living off the Local Model: Abusing Browser-Embedded Language Models for Post-Exploitation

链接:https://arxiv.org/abs/2611.25086

作者:William Fleshman,Joseph Lucas

类目:Cryptography and Security (cs.CR); Artificial Intelligence (cs.AI); Computation and Language (cs.CL)

关键词:local language models, ship local language, includes Gemini Nano, ordinary endpoints, local language

备注:

点击查看摘要

Abstract:Browsers now ship local language models to ordinary endpoints. Google includes Gemini Nano in Chrome to enable productivity features such as summarization, with inference running entirely on the device. These browser-embedded models provide new living-off-the-land opportunities for attackers, each instance a potential post-exploitation engine already installed, signed by the vendor, and invisible to network monitoring. The Gemini model is multimodal, capable of processing files, analyzing screenshots, or conversing with an attacker. We show that it can be driven from the host using only interpreters that ship with the operating system and find that the one-time user-activation check before the model is downloaded is easily satisfied with a simulated click or bypassed this http URL measure and report the current model's ability to identify credentials on the host, detect PII in email, classify images, and call custom tools. We combine these capabilities and use the model to carry out a simulated cyber operation: reading a password from a webcam image, unlocking an encrypted archive, triaging the contents, and staging a private key for exfiltration, end-to-end without user-interaction.

84. 【2611.25071】FinVeriQA: Financial Research Agents for Question Answering: Routing, Retrieval, MCP Tool Use, and Provenance-Aware Evidence Verification

链接:https://arxiv.org/abs/2611.25071

作者:Meenakshi Rajpurohit,Anna Phan,Ravikumar Komandur Narayanan,Gunanidhi Ramakrishnan,Nathan Howland

类目:Computation and Language (cs.CL); Artificial Intelligence (cs.AI); Information Retrieval (cs.IR); Multiagent Systems (cs.MA)

关键词:Large language models, Model Context Protocol, fluent financial analyses, recall stale prices, Large language

备注: 15 pages, 4 figures, 11 tables. Code: [this https URL](https://github.com/SJSU-CMPE259/AI-Stock-Research-and-Earnings-Assistant)

点击查看摘要

Abstract:Large language models (LLMs) write fluent financial analyses but miscompute ratios, recall stale prices, and route questions to the wrong tool with high confidence. We ask which component should do each job in a financial question-answering assistant, and study it with FinVeriQA, an open-source multi-agent research prototype over SEC filings and live market data: a compact typed-decision model (Laya) routes each question and screens out advice requests, Python computes every number, retrieval and tools fetch evidence, Mistral-7B writes a cited answer, a code validator checks that cited IDs exist and that numbers match computed notes, and a Model Context Protocol (MCP) server exposes the pipeline. In live experiments, Laya routed better than Mistral-7B (0.85 vs. 0.52; overall 0.86 vs. 0.68, McNemar p = 0.0003); a 12B Mistral decided no better, and DeepSeek-R1-8B decided better (0.79 vs. 0.68, p = 0.0192) but about 9.9 times more slowly. Mistral-7B computed 29% of 100 financial calculations correctly against 100% for Python, a live quote cut price error from 210% to 0.00%, and hybrid retrieval reached 76% Hit@5 on 50 filing questions. Yet in a live case study, both agent answers passed validation while describing AMD sample data under NVIDIA's and Alphabet's names. A controlled upper-bound study on the FinRank benchmark explains why: when one figure in the shared evidence is corrupted, self-consistency stays confident (agreement 0.55 to 0.59 while accuracy falls from 0.50 to 0.00), and a shared-evidence check (the kind our validator performs) catches 35% of the errors, whereas a check against the original passage catches 77% ($p = 5.96 \times 10^{-8}$); for DeepSeek-R1-8B, 68% vs. 90%. Verification must be independent of the evidence the writer saw.

85. 【2611.25003】A Model of Diverse Sampling from Language Models

链接:https://arxiv.org/abs/2611.25003

作者:Manuel Prada-Corral,Yahya Emara,Timothy J. O'Donnell,Ryan Cotterell,Tim Vieira

类目:Machine Learning (cs.LG); Artificial Intelligence (cs.AI); Computation and Language (cs.CL); Formal Languages and Automata Theory (cs.FL); Machine Learning (stat.ML)

关键词:high-quality individual samples, repeated samples clustering, produce high-quality individual, Language models, individual samples

备注: NeurIPS 2026 paper

点击查看摘要

Abstract:Language models often produce high-quality individual samples but poor sets, with repeated samples clustering around the same semantic modes. We formalize diverse generation as sampling a size-$K$ subset of complete strings from a determinantal point process (DPP), where the likelihood encodes item quality, and the embedding geometry encodes repulsion between similar outputs. Exact inference over all strings is intractable, so we propose PoolDPP: draw a finite candidate pool from a tractable proposal, importance-weight the candidates toward the target quality, and run a $K$-DPP sampler on the induced pool kernel; a greedy MAP variant provides deterministic selection. We prove a finite-sample bound with an explicit constant: the KL divergence between the sampler's output and the target $K$-DPP decays as O(1/N) in the pool size. Empirically, PoolDPP extends the quality--diversity frontier on open-ended generation and the precision--recall frontier on ambiguous text-to-SQL (Ambrosia), and on mathematical reasoning (GSM8K), it raises diversity at about the same pass@$K$.

86. 【2611.24994】What the model knows but does not say: monitoring medical LLMs through verbalized thoughts

链接:https://arxiv.org/abs/2611.24994

作者:Frederik Hauke,Jan Clusmann,Hanna Kreutzer,Christiane Kuhl,Tianyu Han,Felix Barajas Ordonez,Sven Nebelung,Jakob Nikolas Kather,Daniel Truhn

类目:Computation and Language (cs.CL); Artificial Intelligence (cs.AI)

关键词:language models safe, safe have concentrated, enters and leaves, clinical language models, thoughts channel

备注:

点击查看摘要

Abstract:Efforts to keep clinical language models safe have concentrated on what enters and leaves the model, but a manipulated model can return an answer that looks entirely ordinary. Here we show that the model's own internal state, rendered in plain language, exposes manipulations the output conceals. Using a natural-language autoencoder, we translate the activations of a multimodal LLM (Gemma-3-27B) into text and let an input-blind judge read this verbalized internal state (the thoughts channel) and the model's answer separately. Across two clinical attack objectives (omitting a pathology, covertly recommending a drug), two input modalities, and two delivery mechanisms, the thoughts channel caught the injection significantly more often than the answer in all eight injected conditions (p0.05), while both channels held near-zero false-positive rates on clean inputs; the omission result carried over to Llama-3.3-70B. The thoughts channel also registered unsafe clinical requests as harmful while the model was still reading them, before it decided whether to refuse. None of this requires retraining: an off-the-shelf verbalizer and a general-purpose judge are enough.

87. 【2611.24981】Quantifying Cultural-Psychological Constructs in Four Centuries of Ottoman Court Registers

链接:https://arxiv.org/abs/2611.24981

作者:Sila Mutaf,Enes Yılandiloğlu,Mohammad Atari

类目:Computation and Language (cs.CL)

关键词:Populations vary, Ottoman Turkish court, Ottoman Turkish, Turkish court records, Populations

备注:

点击查看摘要

Abstract:Populations vary along cultural dimensions such as individualism, collectivism, and honor. But how can these dimensions be studied in historical societies for which conventional psychological data are unavailable? Here, we introduce a computational text-analytic framework for quantifying psychological constructs in 54,670 Ottoman Turkish court records from 21 Istanbul courts spanning 1513-1911 CE. We develop sentence-embedding models adapted to Ottoman legal language and extend a theory-driven, text-analytic pipeline to capture cultural variation along the dimensions of individualism, collectivism, and honor in this historical corpus. We evaluate our models through multiple methods: document retrieval, semantic textual similarity, questionnaire-item classification, and validation against expert annotations. Our adapted sentence transformer (OTA-BERTurk) achieved a Recall@1 of .843 on the test set for document retrieval, a Mean Reciprocal Rank (MRR) of .806 for semantic textual similarity (meaning the best answer usually appeared as a top result), and classified questionnaire items with 89.58\% accuracy. Document-level performance varied across constructs and methods. Qualitative analysis further showed that high scores reflected historically specific legal and institutional patterns. We publicly release the models, construct representations, scoring pipeline, as well as an interactive web interface to support transparent, reproducible historical-psychological analyses of Ottoman Turkish court records spanning approximately four centuries. These computational tools enable the study of psychological variation across centuries of cultural and institutional change in a diverse empire.

88. 【2611.24977】CANDID: Candidate-Driven Intervention for Disciplined Execution in Local Computer-Use Agents

链接:https://arxiv.org/abs/2611.24977

作者:Woongkyu Lee,Jungwook Choi

类目:Computation and Language (cs.CL); Artificial Intelligence (cs.AI)

关键词:execution-control framework designed, local computer-use agents, computer-use agents, resource-constrained settings, execution-control framework

备注:

点击查看摘要

Abstract:We present CANDID (Candidate-Driven Intervention for Disciplined Execution), an execution-control framework designed to enhance the reliability of local computer-use agents (CUAs) in resource-constrained settings. While local CUAs feature robust baseline proficiency in single-step action prediction, they frequently fail at critical junctions due to rigid action loops or premature DONE decisions. Rather than imposing imprecise cognitive loads through textual reflection, CANDID repurposes parallel test-time scaling into a disciplined, execution-level intervention at sparse, failure-prone points. We introduce a loop-recovery mechanism to filter candidates away from repeated behaviors and a conservative, unanimous DONE gating policy that requires absolute consensus to terminate. Experiments on OSWorld demonstrate that CANDID's structural control significantly improves success rates, outperforming feedback-based alternatives with minimal execution overhead.

89. 【2611.24972】No-Harm-Jury A Jury-Based Multi-Dimensional Harm Evaluation Framework for Medical AI Responses

链接:https://arxiv.org/abs/2611.24972

作者:Antonio Puertas Gallardo,Mario Ceresa

类目:Computation and Language (cs.CL); Artificial Intelligence (cs.AI)

关键词:diverse language models, scores AI-generated medical, conceptually distinct harm, distinct harm dimensions, framework that scores

备注:

点击查看摘要

Abstract:We present No-Harm-Jury, a jury-based automated evaluation framework that scores AI-generated medical responses across seven conceptually distinct harm dimensions using a panel of five diverse language models as independent jurors. Responses generated by Ministral-14B on 3,000 questions drawn from PubMedQA, MedQA, and MedMCQA are scored by five jurors across dimensions of informational, social, psychological, autonomy, economic, privacy, and epistemic harm; the 151 empty responses (98 MedQA, 53 MedM- CQA) are excluded, leaving 2,849 valid samples. The median aggregation robustly suppresses individual juror biases while maintaining coverage through complementary specialisation. No existing system combines all five prop- erties of No-Harm-Jury: automated scalable evaluation, five diverse LLM jurors, robust me- dian aggregation (with a reliability-weighted extension), seven conceptually distinct harm dimensions, and application to medical QA benchmarks.

90. 【2611.24971】he ROI of Small Examples: Optimizing In-Context Learning Under Token Budgets

链接:https://arxiv.org/abs/2611.24971

作者:Anxo Perez,Jorge Gabín,Javier Parapar

类目:Computation and Language (cs.CL); Artificial Intelligence (cs.AI)

关键词:In-context learning, updating model parameters, improves large language, improves large, time without updating

备注: Accepted for EACL 27 Main Conference

点击查看摘要

Abstract:In-context learning (ICL) improves large language model performance at inference time without updating model parameters. However, these gains are not free: in practical deployments, longer prompts directly increase inference cost and latency. Under a fixed token budget, prompt construction therefore becomes a resource-allocation problem. While prior work has often focused on how many examples to include in ICL, we study the complementary question of whether the available budget is better spent on a few long demonstrations or on a larger number of shorter ones. We study this aspect in question answering (QA), by organizing candidate demonstrations into length-based bins and constructing prompts from examples within the same length range under fixed budgets. Experiments on two QA benchmarks (\textit{Beat the AI} and \textit{Natural Questions}), using six instruction-tuned LLMs, show that shorter demonstrations are generally the most effective use of prompt space. This effect is especially strong on \textit{Beat the AI}, and remains visible on NQ, although gains are smaller and saturate earlier in the long-context setting. An additional natural language inference experiment on ANLI R3 finds that one of the two shorter bins performs best for five of six models, although longer demonstrations remain competitive. Overall, our results highlight demonstration granularity as an important factor in prompt design.

91. 【2611.24970】HireWatch: Evaluating LLM Compliance with U.S. Employment Law Under Contextual Pressure

链接:https://arxiv.org/abs/2611.24970

作者:Huy Nghiem,Maria Kostylew,Noam Kolt

类目:Computation and Language (cs.CL); Artificial Intelligence (cs.AI); Computers and Society (cs.CY)

关键词:large language models, language models, pipelines is growing, deployment of large, large language

备注: NeurIPS 2026 Evaluation and Dataset Track. Law and AI

点击查看摘要

Abstract:The deployment of large language models (LLMs) in hiring pipelines is growing, yet existing evaluations generally overlook their compliance with legal rules. To fill this gap, we introduce HireWatch, an evaluation that assesses whether LLMs comply with U.S. laws that govern the questions that job candidates may be asked. We construct a 200-item bank of lawful and unlawful interview questions, whose legal classification is based on U.S. federal statutes, regulations, and agency guidance. We instruct LLMs to select questions from this bank in preparing for job interviews, across a set of various occupations and contexts, and report the fraction of unlawful questions that LLMs select. Evaluation of 18 closed and open-weight LLMs across ~100,000 trials under different types of contextual pressure reveals that LLMs' legal compliance is brittle. While LLMs generally avoid selecting unlawful questions in low-pressure contexts, legal violations increase sharply under higher pressure. Explicit prompts to comply with law can improve compliance rates, but do not altogether eliminate legal violations. We publicly release our code and data at this https URL.

92. 【2611.24969】Global Importance Estimation for KV Cache Eviction

链接:https://arxiv.org/abs/2611.24969

作者:Seojin Kim,Noseong Park

类目:Computation and Language (cs.CL); Artificial Intelligence (cs.AI)

关键词:cache remains, remains a central, central bottleneck, Abstract, applying Personalized PageRank

备注: The 40th Conference on Neural Information Processing Systems (NeurIPS 2026)

点击查看摘要

Abstract:Memory consumption of the key-value (KV) cache remains a central bottleneck for long-context LLM inference. We introduce PRKV, a training-free KV eviction method that estimates multi-hop global token importance by applying Personalized PageRank over attention-induced transition graphs. This enables importance propagation beyond recency or instantaneous attention, without modifying the model. We benchmark PRKV on long-context understanding, sparse retrieval, and chain-of-thought reasoning tasks using multiple LLM families. Under moderate to aggressive compression ratios, including up to 90%, PRKV achieves competitive task performance against established eviction baselines while reducing peak memory usage with low end-to-end overhead. These results demonstrate that multi-hop global influence estimation yields a favorable accuracy--memory trade-off for scalable long-context inference.

93. 【2611.24963】Physics of Language Models: Part 4.2, Canon Layers at Scale where Synthetic Pretraining Resonates in Reality

链接:https://arxiv.org/abs/2611.24963

作者:Zeyuan Allen-Zhu

类目:Computation and Language (cs.CL); Artificial Intelligence (cs.AI)

关键词:architectural differences, Gated DeltaNet, gated, gated linear attention, linear

备注: Results, recipes, and code have been public since December 9, 2025 at [this https URL](https://github.com/facebookresearch/PhysicsLM4)

点击查看摘要

Abstract:Do standard benchmarks see the architectural differences that matter? We study it with 64 from-scratch pretraining runs of Transformers and three linear-model families at 1.3-8B parameters on 1-2T Nemotron-CC tokens, at a cost of roughly 100k USD per 8B run. In academic-scale pretraining (1.3B parameters, 100B tokens), architectural differences can all but disappear when the models are carefully tuned. Instead, at 1-2T tokens they become measurable: Canon layers, which add local token mixing, consistently improve Transformers, saving about 30% of pretraining tokens for the same MMLU accuracy at 8B. We also introduce GLA5 and GDN2, which improve gated linear attention and Gated DeltaNet with low-rank output gates. Surprisingly, with Canon, the simplest linear model, gated linear attention, matches Gated DeltaNet and outperforms Mamba2 even at 8B/1T. Therefore, the extra machinery of Gated DeltaNet and Mamba2 may do little more than mix neighboring tokens. Even at 8B/1T, however, high scores on standard benchmarks can hide weaknesses in basic in-context skills. In particular, linear models that match or beat Transformers on these benchmarks cannot, in fact, reliably retrieve information from a short 500-token context. Moreover, no model, Transformer or linear, reliably beats random guessing on arguably the simplest two-hop question, stated in a few short sentences, even at this scale. These findings $\textbf{challenge traditional architecture research}$ based primarily on standard benchmark scores. In contrast, small synthetic playgrounds, which predicted the architectural differences above at 100M parameters, can reveal what these benchmarks miss. For practitioners, one lesson is to include in-context reasoning examples in pretraining data from an early stage, rather than reserve them for later mid-training, let alone post-training.

Comments:
Results, recipes, and code have been public since December 9, 2025 at this https URL

Subjects:

Computation and Language (cs.CL); Artificial Intelligence (cs.AI)

Cite as:
arXiv:2611.24963 [cs.CL]

(or
arXiv:2611.24963v1 [cs.CL] for this version)

https://doi.org/10.48550/arXiv.2611.24963

Focus to learn more

              arXiv-issued DOI via DataCite (pending registration)

Submission history From: Zeyuan Allen-Zhu [view email] [v1]
Thu, 12 Nov 2026 18:51:19 UTC (1,006 KB)

94. 【2611.24960】ETHER: Training-Free, Elastic, ToC-Anchored Hierarchy Extraction and Recovery in Design-Driven Long Documents

链接:https://arxiv.org/abs/2611.24960

作者:Yu Zou,Yan Chen,Jialei Zeng,Xiaorui Zhou,Aixi Zhong,Qi Zhou,Haoran You,Ke Jiang,Changyu Hu

类目:Computation and Language (cs.CL); Computer Vision and Pattern Recognition (cs.CV)

关键词:deploying Large Language, Large Language Models, Converting complex documents, structured knowledge constitutes, critical infrastructure urgently

备注: EMNLP 2026 Main

点击查看摘要

Abstract:Converting complex documents into structured knowledge constitutes the critical infrastructure urgently required for deploying Large Language Models (LLMs) in downstream research. However, blind to the Table of Contents (ToC) as a macro-structure, existing bottom-up paradigms rely on localized visual cues, inducing heuristic fragility in high-entropy layouts where visual prominence decouples from logical depth; meanwhile, direct LLM extraction lacks spatial grounding, triggering hallucinations in long contexts. To address this, we introduce TETHER, a training-free framework that elevates the ToC from discarded noise to a vital structural prior, enabling robust adaptation across design-driven long documents. TETHER operates via a two-stage neuro-symbolic mechanism: first, it constructs a document skeleton from the raw ToC using a Large Multimodal Model (LMM) empowered by spatio-semantic constraints; second, it employs an anchor-based alignment strategy to tether content blocks to this skeleton, effectively resolving boundary ambiguities. Evaluated across a multilingual benchmark of ESG reports, product manuals, and annual reports, TETHER delivers highly precise, state-of-the-art (SOTA) zero-shot hierarchy recovery, outperforming deep hierarchical recovery models, dedicated parsers, and general-purpose LMMs. Moreover, stress tests highlight TETHER's resilience to sparse omissions and textual noise, demonstrating that its precise semantic alignment is fundamentally anchored in topological and depth fidelity. Crucially, ablations confirm the framework's architectural irreducibility, while achieving 90% token reduction and drastic speedups over end-to-end baselines, establishing a compelling equilibrium between structural precision and deployment scalability.

95. 【2611.24959】ICARE: cross-domain text classification with incremental class-aware representation and distillation learning

链接:https://arxiv.org/abs/2611.24959

作者:Son Duong,Truong X. Tran,Hai-Anh Tran

类目:Computation and Language (cs.CL); Machine Learning (cs.LG)

关键词:fine-tuned large language, large language models, natural language inference, robust natural language, necessitates robust natural

备注:

点击查看摘要

Abstract:The rise of powerful, fine-tuned large language models (LLMs) necessitates robust natural language inference (NLI) systems. These NLI systems ideally train across various tasks, encompassing diverse concepts like sentiment analysis, news classification, and coreference resolution. However, this continual, multi-tasking approach can lead to catastrophic forgetting, where previously learned tasks are lost, especially in domains lacking specific labels. Rehearsing past examples with labels can further exacerbate this issue by causing overfitting, hindering the system's ability to classify new categories. To tackle these challenges, this research proposes ICARE, a novel framework with two key components: an instance selection module and a knowledge distillation method. The former is based on representation learning, which meticulously selects training examples to present to the NLI system. The latter, knowledge distillation with feature and logit-level transfer, facilitates knowledge transfer between tasks. This leverages pruning techniques and optimized objective functions to transfer both essential feature-level and decision-making (logit-level) knowledge, mitigating catastrophic forgetting while adapting to new patterns. Extensive evaluations on benchmark text classification datasets have demonstrated ICARE's effectiveness. The framework achieves a significant improvement in target accuracy and reduction in the ``forgetting metric''. These results solidify ICARE's ability to preserve and leverage learned knowledge across diverse tasks, ultimately enhancing the adaptability and performance of NLI systems. The source code is available at this https URL

96. 【2611.24943】LLM Complexity: Towards a Computational Complexity View of Large Language Model Inference

链接:https://arxiv.org/abs/2611.24943

作者:Yu-Hung Wu,Shao-Xiang Yuan,Chan-Hung Yu,Zih-Ching Chen,Kuan-Hao Huang,Shao-Hua Sun

类目:Computation and Language (cs.CL); Artificial Intelligence (cs.AI); Machine Learning (cs.LG)

关键词:Large Language Models, computational systems executing, Large Language, Computational complexity theory, complexity theory offers

备注: 10 pages, 20 with Appendix

点击查看摘要

Abstract:Computational complexity theory offers a principled framework for characterizing the efficiency of computational problems, and Large Language Models (LLMs) can similarly be viewed as computational systems executing a structured process of input, inference-time compute, and output. In this context, the number of generated tokens emerges as a natural and directly observable proxy for inference-time compute. However, we show that the tempting analogy between LLM token length and classical time complexity does not hold. We disprove the hypothesis that inference-time token usage aligns monotonically with classical time complexity, demonstrating a fundamental divergence grounded in how LLMs actually allocate computation during inference. To this end, we introduce LLM Complexity, a new metric designed to quantify problem complexity from the perspective of LLMs by characterizing the area bounded above by the accuracy-token curve. We find that diverse LLMs induce highly consistent task rankings under LLM Complexity, indicating that the metric captures stable notions of difficulty across models. We leverage this property to construct an easy-to-hard curriculum for supervised fine-tuning, which improves accuracy while reducing token consumption. Finally, we demonstrate that LLM Complexity generalizes to real-world tasks, producing more stable cross-model correlations than human-annotated difficulty labels.

信息检索

1. 【2611.26609】Variants of Beam Search for DiskANN

链接:https://arxiv.org/abs/2611.26609

作者:Viraj Agrawal,Kirankumar Shiragur

类目:Data Structures and Algorithms (cs.DS); Databases (cs.DB); Information Retrieval (cs.IR)

关键词:Graph-based data structures, alpha, large-scale approximate nearest-neighbor, Graph-based data, NSG constitute

备注: 22 pages, 1 figure, 9 tables

点击查看摘要

Abstract:Graph-based data structures such as DiskANN, HNSW, and NSG constitute the state of the art for large-scale approximate nearest-neighbor (ANN) search. Indyk and Xu introduced the notion of $\alpha$-reachable graphs, which include graphs built by DiskANN's slow-preprocessing construction, and proved that greedy search achieves an approximation factor of $\tfrac{\alpha+1}{\alpha-1}$. Independently, Al-Jazzazi et al. proposed adaptive beam search (ABS) with a relaxation parameter $\gamma 0$, showing a $\tfrac{2}{\gamma}$ approximation on navigable graphs ($\alpha \approx 1$), where greedy search admits no guarantees. In this work, we unify and generalize these results by proving that ABS with parameter $\gamma$ on $\alpha$-reachable graphs achieves an approximation of $\tfrac{\alpha+1}{\alpha(\gamma+1)-1}$ for any metric. Moreover, on sorted $\alpha$-reachable graphs (introduced by Gollapudi et al.) under the $\ell_2$ metric, we obtain the tighter bound $\max\bigl(\tfrac{\alpha}{\alpha(\gamma+1)-1}, \tfrac{\alpha^{2}+1}{\alpha^{2}(\gamma+1)-1}\bigr)$. Beyond these theoretical results, we design a beam predictor using features inspired by fine-grained analysis of ABS, trained via a constrained objective that minimizes mean distance computations subject to a target recall. On three 100-million-point benchmarks, using graphs built by DiskANN's practical construction, beam predictor lowers mean query latency on SSD-resident indices at matched recall by up to $29.6\%$ relative to classical beam search (CBS). Relative to ABS, it is faster in $11$ of $15$ settings, by up to $6.5\%$, and at most $1.4\%$ slower in the other $4$; at $95\%$ recall, its advantage over ABS grows from $0.8$--$4.2\%$ for $100$ neighbors to $4.6$--$5.6\%$ for $500$ neighbors. In memory, it needs $1.3$--$23.7\%$ fewer distance computations than CBS and $0.0$--$3.9\%$ fewer than ABS.

2. 【2611.26531】Cost-Aware Agentic Feature Optimization System for Fleet-Scale Ranking Models

链接:https://arxiv.org/abs/2611.26531

作者:Chang Yang,Yao Xuan,Wendan Yan,Nhat Le,Jiang Liu,Longhao Jin,Zhen Peggy Yao,Alireza Vahdatpour,Djordje Gligorijevic

类目:Information Retrieval (cs.IR); Artificial Intelligence (cs.AI); Machine Learning (cs.LG)

关键词:Large Web platforms, Web platforms operate, platforms operate large, operate large fleets, Large Web

备注:

点击查看摘要

Abstract:Large Web platforms operate large fleets of ranking models, each selecting features from a shared pool of tens of thousands. We present an end-to-end agentic system that optimizes features across the fleet under resource constraints shared across models and those set per model, such as feature pipeline cost and serving throughput (QPS), respectively. In this system, LLM agents propose hypotheses, and deterministic tools execute experiments and verify them. For each model, an LLM generates and shortlists cross features from a combinatorial candidate space, and all features are then ranked by several importance methods. Feature blending mixes these rankings into a single selection, using Pareto fronts across tasks for multi-task models. An LLM Researcher agent then tunes the feature selection in 10--25 asynchronous trials under resource caps, and a trimming step keeps the pipeline cost within budget. Across production models, the system improves normalized entropy by 0.31% on average within each model's budget, and the trimming step cuts pipeline cost by 2.3% with no regression beyond measurement noise. The optimized models were launched, and in online A/B tests they improved a primary online metric by 0.20%.

3. 【2611.26521】he Window Lever: Longer Observation Windows Outweigh the Observed Effects of Data, Capacity, and Architecture in Customer Behavior Forecasting

链接:https://arxiv.org/abs/2611.26521

作者:Tara Sowrirajan,Yanqiao Wang

类目:Machine Learning (cs.LG); Information Retrieval (cs.IR)

关键词:long-horizon customer-behavior forecasting, observation window outweighed, longer observation window, customer-behavior forecasting, newer architectures

备注:

点击查看摘要

Abstract:In our experiments on long-horizon customer-behavior forecasting, a longer observation window outweighed the observed effects of more data, more capacity and newer architectures. The window is the weeks of history a model sees before the week it forecasts from. Going from an 8- to a 16-week window adds a raw +0.070 to 0.107 log-space $R^2$ in every family at 1M. Matched test cohorts, which score the same customer-weeks at every window, show this raw comparison understates the gain in every family they cover. The direction holds on five public datasets, four spanning over a year (ridge and our transformer on all five). On our internal data, longer windows admit up to 10% more and colder training customers, which matching does not fix. Our design does not separate window length from calendar coverage, because every test anchor (the week a forecast is made from) falls in one eight-week band. On training-period anchors, the weekly ridge probe's gain persists outside the holiday season, keeping about 72% of it. We therefore report a robust positive association with longer windows and do not identify the magnitude of a calendar-independent history effect. Of four internally registered predictions, the only fully blind one failed. An activity-mixture model predicted +0.014 from 24 to 32 weeks; the weekly-aggregate probe returned -0.0004. For the sequence families, the other three levers are about an order of magnitude smaller, bounded and unresolved. Their observed effects are at most 0.006 between the nominal 1M and 10M tiers and 0.0052 among five sequence families at 16 weeks with clocks as built, both single-seed on the full test split, and 0.006 for capacity on three-seed means. These are observed intervals, not equivalence bounds. Our claim is the order-of-magnitude contrast between the window and those bounds. A team building on this class of data should start by lengthening the window.

4. 【2611.26398】Democratizing Cross-Project Knowledge for Transforming Water Governance and Climate Resilience: an Agentic AI Ecosystem

链接:https://arxiv.org/abs/2611.26398

作者:Oriol Alàs,Ian Palacín,Iván Cester,Edgar Rubión

类目:Information Retrieval (cs.IR); Artificial Intelligence (cs.AI); Computation and Language (cs.CL)

关键词:projects produce extensive, produce extensive evidence, form of deliverables, case studies, EU-funded projects produce

备注: 8 pages, 2 figures. Accepted at WDSA/CCWI 2026. Proceedings publication pending

点击查看摘要

Abstract:EU-funded projects produce extensive evidence in the form of deliverables, reports, and case studies. However, this knowledge is typically scattered across projects and document structures, making it difficult to locate relevant, contextual information when making decisions. This paper introduces an Agentic AI Ecosystem that consolidates heterogeneous project documentation into a domain-aware knowledge base and provides adaptive access through an agentic Retrieval-Augmented Generation pipeline. Documents from three EU-funded projects (IMPETUS, NATALIE, and InnWater) are reconstructed into hierarchical section trees and indexed with structured metadata. Retrieval is supported by two domain-specific embedding models fine-tuned on domain data (water and climate). On top of this knowledge base, a ReAct-style controller iteratively orchestrates retrieval actions by allocating retrieval budget across domain collections and accumulating evidence until a stopping criterion is met. In a retrieval benchmark over the corpus, the proposed approach improves Mean Reciprocal Rank by 26.7% (relative) compared to a single-domain embedding baseline, indicating more consistent retrieval of relevant evidence to support cross-project knowledge synthesis and climate-resilient decision-making.

5. 【2611.26096】From Sessions to Serving: Integrating LLM-Based Embeddings into Retrieval for Production Content Recommendation

链接:https://arxiv.org/abs/2611.26096

作者:Zhixiong Hu,Meng Li,Maggie Zhang,Deborshi Goswami,David Bailey,James Bradley

类目:Information Retrieval (cs.IR); Machine Learning (cs.LG)

关键词:placing increasing demands, Personalized content recommendation, downstream ranking stage, reconcile heterogeneous candidate, large marketplace websites

备注:

点击查看摘要

Abstract:Personalized content recommendation on large marketplace websites must reconcile heterogeneous candidate content (e.g., customer stories, product pages, industry pages, and learning content) with sparse, fast-changing user behavior, placing increasing demands on the retrieval stage to represent user interest well, not just the downstream ranking stage. We present LSER (LLM Session Embedding Retrieval), a production content recommendation system for this http URL that derives session- and history-level embeddings from a pretrained large language model (LLM) and adds them as auxiliary user-side features in the retrieval-stage user tower alongside existing learned sequence encoders. To keep user-side embeddings fresh under real-time behavior, we build a nearline serving path that asynchronously refreshes sequential and LLM-derived user vectors for our main content recommendation server. We validate the incremental value of the LLM features through an ablation study across our full multi-content-type catalog. Our results indicate that integrating LLM-derived embeddings as additional retrieval signals, together with production-grade nearline serving, is technically feasible and can improve model performances.

6. 【2611.26019】A Task-Level Bias Taxonomy for Historical Image Retrieval

链接:https://arxiv.org/abs/2611.26019

作者:Omkar Gavali,Ratan Sebastian,Christoph Rippe,Ralph Ewerth,Anett Hoppe

类目:Information Retrieval (cs.IR)

关键词:cultural heritage material, digitised vast amounts, GLAM institutions, including historical imagery, recent decades

备注: Presented at Bias In GLAM workshop at TPDL 2026

点击查看摘要

Abstract:In recent decades, galleries, libraries, archives, and museums (GLAM institutions) have digitised vast amounts of cultural heritage material, including historical imagery. A major challenge in making these collections publicly accessible is enabling effective search and discovery. Many historical images contain incomplete, sparse, or no descriptive metadata. Consequently, machine learning (ML) methods are increasingly being used to generate or enrich metadata to improve the searchability of such collections. However, artificial intelligence (AI)-based systems can reproduce or amplify existing biases, particularly when applied to historical imagery. Factors such as differences between the data used to train the models and the historical images on which they are applied, degraded image quality, monochrome photographs, and historically biased or incomplete descriptions can reduce model performance and introduce bias into search and retrieval results. In this paper, we examine different forms of bias in historical image collections and review current methods to mitigate them. We investigate how these biases arise and how they influence AI-supported search and retrieval systems.

7. 【2611.25845】LINEAGE: A Benchmark for Empirical Lineage Retrieval in AI Scientific Literature

链接:https://arxiv.org/abs/2611.25845

作者:Pangjing Wu,Xu Yuan,Wenqi Fan,Qing Li

类目:Information Retrieval (cs.IR); Artificial Intelligence (cs.AI)

关键词:Current research agents, Current research, Lineage, research agents, moving beyond academic

备注:

点击查看摘要

Abstract:Current research agents are moving beyond academic search toward literature synthesis and experiment planning. These workflows increasingly rely on literature retrieval systems, which serve as standalone systems or as components of research agents. However, existing scientific literature retrieval benchmarks mainly evaluate these methods using topical relevance or citation links as supervision, which cannot capture which prior methods are used for empirical comparison. To address this gap, we introduce Lineage, a benchmark for empirical lineage retrieval in AI scientific literature. An empirical lineage relation links a prior paper to a source paper that explicitly uses its method as an empirical comparison baseline. These relations form lineage graphs whose multi-hop neighborhoods provide retrieval ground truth for structural evaluation. To construct these graphs at scale, we develop Lineage-Constructor, which converts heterogeneous paper records and full-paper empirical baseline evidence into calibrated lineage graphs. Built from 121,048 papers across 17 top AI venues from 2018 to 2025, Lineage contains 3,950 graph components, 67,502 paper nodes, 164,948 lineage edges, and 80,000 multi-facet retrieval queries. Experiments with 19 representative retrieval systems show that empirical lineage retrieval remains challenging. Performance drops sharply with lineage distance, while structural metrics reveal that isolated retrieval hits often fail to recover closely connected empirical comparison neighborhoods.

8. 【2611.25738】CACO: Coverage-Aware Coordinated Optimization for Continuous Generative Retrieval

链接:https://arxiv.org/abs/2611.25738

作者:Pan Zhang,Nan Zhuang,Yongkang Zhang,Kun Xú,Zhongtao Jiang,Zhiwei Wang,Kun Xǔ,Youhuan Li

类目:Information Retrieval (cs.IR)

关键词:Continuous generative retrievers, parallel approximate nearest-neighbor, fixed retrieval budgets, Continuous generative, diffusion sample multiple

备注: 21 pages, 16 figures, 12 tables

点击查看摘要

Abstract:Continuous generative retrievers powered by flow matching and diffusion sample multiple queries in item embedding space for parallel approximate nearest-neighbor (ANN) search under fixed retrieval budgets. However, optimizing distribution diversity or target-level interest separation ignores set-level retrieval geometry: sampled queries collapse onto overlapping ANN neighborhoods, while discrete top-K lookup and single-target supervision block end-to-end coverage optimization. We introduce CACO (Coverage-Aware Coordinated Optimization), a lightweight self-supervised post-training framework that coordinates queries from frozen flow-matching and diffusion retrievers toward complementary item coverage with negligible additional inference latency. CACO uses empirical retrieval frequencies across sampled reference queries as a relevance proxy, formulating a label-free coverage objective that improves recall across all 30 backbone-dataset settings in offline greedy selection experiments. To avoid the inference overhead of generating a large query pool, CACO employs a coordinated query adjustment network that combines inter-query bidirectional attention with gated relative directions, finally updating queries by bounded tangent-space projection and spherical retraction. During training, CACO applies tangential antithetic perturbations and measures each query's marginal gain via counterfactual single-query replacement, yielding an unbiased gradient estimator of smoothed set coverage across non-differentiable ANN search. On eight public benchmarks and an in-house 24-partition streaming industrial dataset, CACO improves Recall@100 by up to 23.7% and Recall@400 by up to 23.9% over the strongest baseline, cuts inter-query overlap by 53.3%-64.9%, and adds only 0.5%-3.0% inference latency.

9. 【2611.25733】Can LLMs Resolve Entity Conflicts in Domain Specific NLU Pipelines?

链接:https://arxiv.org/abs/2611.25733

作者:Akansha Bhardwaj,Mingbo Cui,Victor Guimaraes,Dominik Kohler,Diana Nicoleta Popa,Meghana M. Reddy,Paolo Rosso

类目:Information Retrieval (cs.IR); Machine Learning (cs.LG)

关键词:Pipeline-based NLU architectures, Resolving entity conflicts, multiple extractors provide, extractors provide overlapping, step in Pipeline-based

备注: This work was completed in February, 2025

点击查看摘要

Abstract:Resolving entity conflicts is a critical step in Pipeline-based NLU architectures, especially when multiple extractors provide overlapping or conflicting outputs. Traditional rule-based entity conflict resolution methods often lack flexibility and are unscalable. In this work, we aim to study if the broad knowledge of LLMs can be leveraged to resolve entity conflicts in domain specific NLU pipelines. We present an experimental comparison between a traditional rule-based conflict resolution method, and two LLM-based approaches: a multi-shot prompting method using the Llama3.1-8B-Instruct model and a LoRA fine-tuned Llama3.1-8B-Instruct model. The evaluation was conducted under two setups: (1) a broad assessment of LLM's capability to match existing NLU pipeline output, and (2) a more controlled scenario assessing the LLM accuracy on human-labeled data. Results show that while multi-shot prompting offers ease of adaptation and rapid behavioral changes, the fine-tuned model delivers superior performance on complex, ambiguous cases. Ultimately, the best approach appears to be a combination of both strategies, leveraging multi-shot prompting's flexibility alongside the broad accuracy gains of LoRA fine-tuning for a more comprehensive performance.

10. 【2611.25705】STARK: Moving towards Personalized Agents That Truly Learn and Remember

链接:https://arxiv.org/abs/2611.25705

作者:Aryan Jain,Abhishek Jha,Anish Nediyanchath,Promod Yenigalla

类目:Computation and Language (cs.CL); Artificial Intelligence (cs.AI); Information Retrieval (cs.IR)

关键词:long time periods, personalized agentic systems, Long-term memory, time periods, underlying LLMs.

备注:

点击查看摘要

Abstract:Long-term memory is essential for personalized agentic systems that interact with users over long time periods, yet current architectures only capture what happened while leaving the reasoning of why it happened to be derived later by underlying LLMs.%, and silo knowledge within individual users. These events can, however, convey not only what a user experienced but also how that experience shaped their behavioral patterns. We introduce STARK, a dual-graph memory architecture which treats these events not just as nodes but also as edges between behavioral insights. STARK maintains an Episodic Graph which records interactions as temporally, causally, and semantically linked episodes; and an Insight Graph which represents LLM-extracted behavioral signals scored by how many events support them, connected by the very events that led to them. At inference, behavioral knowledge is retrieved directly from insight nodes grounded by event-as-edges, eliminating repeated episodic traversal. The event-as-edge structure also naturally enables projecting the knowledge graph into a relational embedding space where cross-user transfer becomes possible under shared contextual circumstances, allowing the system to infer missing insights for cold-start users. Evaluations show that STARK improves by 1.48 judge score on LongMemEval information extraction and 0.93 on LoCoMo open domain questions over state of the art. On the public LongLaMP abstract-generation benchmark, cross-user transfer improves open-ended judge scores by 0.7 points and ROUGE-L by 3 at 5% graph sparsity.

11. 【2611.25691】Exploring Semantic Table Search in Data Lakes

链接:https://arxiv.org/abs/2611.25691

作者:Martin Pekár Christensen,Matteo Lissandrini,Katja Hose

类目:Information Retrieval (cs.IR)

关键词:search, table search, key challenge, remains a key, semantic

备注: 29 pages, 13 figures, GitHub repository at [this https URL](https://github.com/dkw-aau/Thetis-extended)

点击查看摘要

Abstract:Discovery of relevant datasets within data lakes remains a key challenge. Union and join search are effective techniques for discovering data when augmenting an existing table. In exploratory search, instead, keyword search is commonly adopted to find a relatively small set of highly relevant tables matching some search terms. In this context, semantic search is a new technique that exploits the presence of named entities in exemplar tuples used as queries and in target tables. Hence, keyword search and entity search are able to find a diverse set of tables. In this work, we propose \texttt{Thetis++}, which combines keyword and semantic table search to discover relevant tables that combine the semantic signal with the presence of exact keyword matches. Incorporating multiple paradigms and sources of information allow us to combine the best of multiple worlds. Therefore, we study many alternative techniques for combining rankings and evaluate these for the table search task in data lakes. We explore both unsupervised and supervised techniques, including TOPSIS. Finally, alternative to Thetis++ combining keyword and semantic relevance signals, we study a variant of learning-to-rank for the table search task in data lakes. Our evaluation highlights both the limitations of many standard approaches along with some promising directions offered by learned approaches. Furthermore, while indexes for keyword search are quite efficient, here we provide insights into how to improve the scalability of prefiltering techniques for semantic search, comparing LSH indexes with Hierarchical Navigable Small Worlds. Our experiments show that HNSW drastically improves the runtime of semantic table search by 4.7X on average whilst retaining the high-quality tables. Together, our findings plot the line for effective and efficient data discovery techniques for semantic data lakes.

12. 【2611.25659】DNA: Unlocking Depth Scaling in CTR Models

链接:https://arxiv.org/abs/2611.25659

作者:Caiyuan Li,Wenjie Wu,Yuheng Huang,Xiao Zhou,Silu Zhou,Ke Chen,Zinan Zhao,Xinrui Zeng,Hanmo Xu,Chaoqun Song,Hua Zong,Zongyuan Wu,Hongbin Zhou,Huimin Yi,Zhixuan Zhang,Dan Ou,Haihong Tang

类目:Information Retrieval (cs.IR)

关键词:Long user behavior, Long user, click-through rate, strongest signals, signals for click-through

备注: 10 pages, 5 figures

点击查看摘要

Abstract:Long user behavior sequences are among the strongest signals for click-through rate (CTR) prediction. Cross-attention makes these sequences practical for industrial ranking by keeping compute linear in sequence length. Yet the gains from stacking cross-attention layers taper off: on Taobao search, growing a cross-attention stack from 16 to 28 layers adds only 0.02 percentage points (pt) of AUC. We argue that the key-value side is the bottleneck. Each layer projects the same behavior sequence into keys and values, so adding depth gives the model more reads of a fixed sequence. We propose DNA (Dual-stream Nexus Attention), which lets the sequence itself evolve with depth. At each layer, the request representation (the search query, user features and scene) first reads the sequence. Each behavior token then attends to the updated request representation and uses what it reads to update itself, so the next layer reads a sequence shaped by the current request. The update runs once per request, all candidates share the resulting sequence, and both the reads and the update keep linear complexity in sequence length. At comparable forward FLOPs, DNA improves AUC over cross-attention stacks by 0.08 to 0.26 pt and keeps improving in the depth range where those stacks level off. DNA-4 is deployed in Taobao search. In an online A/B test against the production model, it improves CTR by 0.8% and GMV by 0.5%.

13. 【2611.25627】Uncontrolled source titles: When Nature, Science, and PNAS meet Scopus

链接:https://arxiv.org/abs/2611.25627

作者:Weishu Liu,Ruifeng Zhang,Haifeng Wang

类目:Digital Libraries (cs.DL); Information Retrieval (cs.IR)

关键词:bibliographic data, Bibliographic databases, Science Core Collection, systematic reviews, bibliometric analyses

备注: Forthcoming in Journal of Information Science

点击查看摘要

Abstract:Bibliographic databases such as Scopus and Web of Science play a critical role in literature discovery, systematic reviews, bibliometric analyses, and research evaluation, yet their reliability depends on effective control of the quality of the bibliographic data. This study examines the problem of uncontrolled source titles in Scopus by comparing five possibly adopted journal-level retrieval strategies across three flagship journals-Nature, Science, and PNAS. Substantial discrepancies across retrieval results indicate persistent fragmentation in the source title field. A comparative analysis using the Web of Science Core Collection shows that no comparable degree of uncontrolled source-title variation is observed for the same journals. The study further explores possible mechanisms, including inheritance of different title forms from (different) upstream providers, automated parsing and OCR-related artifacts, and insufficient cross-validation. Based on these findings, the study proposes recommendations at both the user and database-provider levels to improve source-title normalization and strengthen source-level metadata governance, with implications for the accuracy and reproducibility of journal-level retrieval and evaluative uses.

14. 【2611.25606】Generative News Recommendation Reproduces Popularity: A Memorization-Based Analysis

链接:https://arxiv.org/abs/2611.25606

作者:Shotaro Ishihara

类目:Information Retrieval (cs.IR); Computation and Language (cs.CL)

关键词:directly generating item, generating item titles, ranking existing items, Text-based generative recommenders, existing items

备注: The 11th International Workshop on Application of Big Data for Computational Social Science (ABCSS2026 @ IEEE BigData 2026)

点击查看摘要

Abstract:Text-based generative recommenders can go beyond ranking existing items by directly generating item titles or other textual representations. In the news domain, this raises a societal concern that models may repeatedly surface already popular articles and thereby narrow the diversity of information to which users are exposed. We investigate training-data memorization as one mechanism behind this concentration. Using approximately 20 million browsing sessions from a Japanese news service, we continually trained an instruction-tuned Llama 3 8B model to generate the next article title in a session. The generated-title distribution was more concentrated and less diverse than the distribution of titles observed in the browsing logs. Frequently repeated target titles exhibited greater verbatim and approximate memorization, and more strongly memorized titles were generated in response to a wider range of prompts. These findings support a pathway in which popularity in behavioral logs becomes repeated training text, is preferentially memorized, and is subsequently reproduced by a generative recommender. An exploratory target-level deduplication condition reduced both memorization and output concentration, while substantially reducing the training-set size and recommendation accuracy. Our results suggest that popularity bias in generative recommendation may arise not only from user preferences or ranking objectives, but also from the model's tendency to reproduce frequently encountered strings, with important implications for information diversity on news platforms.

15. 【2611.25582】AutoRAGBench: A Benchmark for LLM Agents Automating RAG System Development

链接:https://arxiv.org/abs/2611.25582

作者:Hang Zhang,Yuzhi Pan,Ziyao Zhong,Chaokun Wang,Shuo Cao,Zeyu Huang,Yue Xue,Fang Niu,Bofan Xie,Guanchen Ge,Zecheng Li,Leqi Zheng,Ziyang Liu,Wei Wang,Miao Gu

类目:Computation and Language (cs.CL); Artificial Intelligence (cs.AI); Information Retrieval (cs.IR)

关键词:large language models, Retrieval-augmented generation, augmenting large language, RAG systems, RAG

备注:

点击查看摘要

Abstract:Retrieval-augmented generation (RAG) is a well-established paradigm for augmenting large language models (LLMs) with information retrieved from external corpora. However, developing effective RAG systems requires substantial engineering effort and iterative experimentation, as effective design choices vary across tasks and domains. With rapid advances in LLM agent capabilities, we ask how well these agents can autonomously build RAG systems and how effectively they can improve them through iteration. In this paper, we propose AutoRAGBench, the first benchmark to evaluate how well LLM agents develop RAG systems from scratch in an open code space. It comprises nine RAG system development tasks. For each task, agents build RAG systems from scratch without starter code or prescribed strategies, under a six-hour budget on a single H800 GPU. They analyze task data, implement RAG pipelines, and iteratively refine their designs based on experimental feedback. We evaluate nine agent configurations and analyze both the effectiveness of the resulting RAG systems and their development trajectories. GPT-6.1 Sol with Codex CLI achieves the highest average score of 56.1. The composite Human-SOTA baseline averages 44.4. These results show that LLM agents can automate effective RAG system development. When starting from a human-designed system on each of seven tasks, GPT-6.1 Sol with Codex CLI improves the score by 16.6 points on average.

16. 【2611.25527】Learning Generative Ranking Policies for Whole-Page Optimization

链接:https://arxiv.org/abs/2611.25527

作者:Chen Qiu,Zhigang Hua,Qi Xu,Jizhe Zhang,Sha Hua,Ren Mao,Luke Simon

类目:Information Retrieval (cs.IR); Artificial Intelligence (cs.AI); Machine Learning (cs.LG)

关键词:compete for attention, attention and complement, simply the individually, page, pages

备注:

点击查看摘要

Abstract:A recommendation page is consumed as a whole: the items on it compete for attention and complement one another, so the best page is not simply the individually best items placed side by side. We study whole-page optimization, which selects and orders the items on a page jointly to maximize the score of the page. The number of possible pages grows combinatorially, so good pages must be generated rather than enumerated. Some generative re-ranking methods use the same page-level training signal for every choice in a sampled page. Others imitate target pages built before training, which remain largely fixed as the generator improves. We address both limitations with one local page-level objective, which repeatedly moves the current generator toward pages with higher score. This objective splits exactly into one training target per position, and each target compares candidates at the same partial page by the scores of the complete pages they lead to. Every position therefore learns to choose the item that is better for the whole page, rather than the item that looks best on its own. We call this procedure whole-page policy improvement. On two public datasets, KuaiRand-1K and RL4RS-SeqSlate, which cover multi-objective item selection and constrained ordering, and on an industrial dataset of logged pages, our method outperforms the evaluated generative re-ranking baselines. On a synthetic benchmark with a known optimal page, it also comes closest to the optimum, showing that its improvements come from better whole-page decisions.

17. 【2611.25377】VideoTrail: Interactive Video Foraging Through Hierarchical Visualization

链接:https://arxiv.org/abs/2611.25377

作者:Catherine Yeh,Bryan Wang,Fernanda Viégas,Martin Wattenberg

类目:Human-Computer Interaction (cs.HC); Information Retrieval (cs.IR); Multimedia (cs.MM)

关键词:People often watch, locating relevant moments, difficult and time-consuming, locating relevant, People

备注: 19 pages, 13 figures

点击查看摘要

Abstract:People often watch videos to find information, but locating relevant moments can be difficult and time-consuming. Existing strategies like scrubbing rely primarily on visual scanning and linear search, providing limited support for goal-directed navigation. We present VideoTrail, an AI-powered browser extension that supports information seeking in videos through a hierarchical visualization. Inspired by information foraging theory, VideoTrail surfaces semantic and structural cues as information scent, creating an explorable roadmap of segments. The system is multimodal and adaptive, prioritizing visual or textual scent based on the video's content. In a user study (N=16), VideoTrail helped participants efficiently locate key moments and enabled new navigation strategies compared to traditional timelines, highlighting how exposing an adaptive video structure can enhance goal-directed viewing.

18. 【2611.25256】LQ-Rec: Late Quantization for Sequential Recommendation

链接:https://arxiv.org/abs/2611.25256

作者:Satoshi Koide,Yueqi Wang,Julian McAuley

类目:Information Retrieval (cs.IR)

关键词:sequential recommendation compresses, generate next-item codes, Generative retrieval, user history, sequential recommendation

备注:

点击查看摘要

Abstract:Generative retrieval (GR) for sequential recommendation compresses each item into a short semantic ID and trains an autoregressive decoder to generate next-item codes from a user history. While appealing for cold-start handling and storage, au- toregressive GR suffers a structural scaling limit: typically, longer semantic IDs reduce quantization error in principle but degrade ranking quality and quadratic memory usage against semantic ID length. We propose Late Quantization (LQ), a non-autoregressive paradigm that confines discretization to the output (item) side of the pipeline. The encoder and scoring rule operate in continuous space, while items are stored and matched as semantic IDs produced by an end-to-end-trained residual quantizer. This architectural shift permits an additive error decomposition and a pairwise ordering theorem that jointly bound the quantization- induced ranking drop, a structural guarantee absent in prior semantic-ID recommenders. Across four public benchmarks, the resulting recommender, LQ-Rec, delivers three capabilities simultaneously: ranking quality improves monotonically with the semantic-ID length and outperforms existing baselines; a single trained model supports anytime compression, tuning ranking quality against inference cost without retraining; and LQ-Rec shows superior generalization on a memorization/generalization partition.

19. 【2611.25247】Retrieving Patristic References and Modelling Topics in Bullinger's Reformation Correspondence

链接:https://arxiv.org/abs/2611.25247

作者:Lena Pickartz,Philipp Schaer,Phillip B. Ströbel

类目:Information Retrieval (cs.IR); Computation and Language (cs.CL); Digital Libraries (cs.DL)

关键词:Swiss Reformer, Antistes of Zurich, Reformer and Antistes, Church Fathers, including extensive appeals

备注: Accepted for the Seventh Conference on Computational Humanities Research 2027. To be published in the Anthology of Computers and the Humanities

点击查看摘要

Abstract:The correspondence of Heinrich Bullinger (1504-1575), Swiss Reformer and Antistes of Zurich, documents the political and religious debates of the Reformation, including extensive appeals to the Church Fathers as authoritative sources. Detecting such references computationally remains largely unexplored. We investigate dense and sparse retrieval for patristic reference detection, and metadata-informed topic modelling as a proxy for the topics and ideas discussed. Prior work on Latin intertextuality has fine-tuned dense retrievers for their respective target domain, and we test how well these models transfer to Bullinger's correspondence. We compare lexical (BM25), dense (bi-encoders), and hybrid methods on explicit and implicit references. A Latin encoder fine-tuned on biblical reuse performs best among the bi-encoders (82%, Recall@20 on explicit and 16% on implicit references), while BM25 outperforms dense approaches on explicit citations (94%, R@20). Because BM25 and dense retrieval recover disjoint sets of implicit references, fusing their rankings raises implicit R@20 to 22%, close to cross-encoder reranking of the same candidates (26%) but without its inference cost. For topic modelling, we extend FASTopic with a seed-guided loss function and editorial abstract embeddings. Measurable coherence gains over the baseline remain minimal, but qualitative and temporal analysis shows that some topics track documented historical events. However, dominant topic weights barely exceed a uniform distribution, making topics unusable to inform retrieval. A pilot application of LLM-based concept induction (LLooM) further points towards a more human-interpretable alternative. Together, these results present a first approach to retrieving patristic references and modelling fine-grained topics in Bullinger's correspondence.

20. 【2611.25229】LSP-RAG: Latent Scoring Perspectives for Knowledge Graph Retrieval-Augmented Generation

链接:https://arxiv.org/abs/2611.25229

作者:Haochen Liu,Guanghui Min,Song Wang,Yada Zhu,Chen Chen,Jundong Li

类目:Information Retrieval (cs.IR)

关键词:demonstrated strong capabilities, Large Language Models, Large Language, Language Models, training corpora

备注:

点击查看摘要

Abstract:Large Language Models (LLMs) have demonstrated strong capabilities in open-domain question answering, but often struggle with factual accuracy and multi-hop reasoning due to the incompleteness of the training corpora. A promising solution is Knowledge Graph Retrieval-Augmented Generation (KG-RAG), which supplements LLMs with structured knowledge retrieved from external knowledge graphs (KGs). However, existing KG-RAG methods either rely on large-scale language models (e.g., ChatGPT) to guide the retrieval process, which leads to high computational costs, or suffer from limited retrieval quality when using lightweight language models, particularly under multi-hop scenarios. We propose LSP-RAG (Latent Scoring Perspectives for Retrieval-Augmented Generation over Knowledge Graphs), a novel KG-RAG framework that enhances hop-wise KG knowledge retrieval through multi-perspective prompted scoring. Each latent scoring perspective is guided by a combination of two types of soft prompts: a perspective-specific soft prompt induces a distinct prompt-conditioned scoring function over candidate triplets, while a contextual soft prompt evolves with each reasoning hop by encoding the query and previously explored KG triplets, enabling the model to preserve consistency and relevance across multi-hop retrieval. Rather than assuming manually defined semantic roles, these latent perspectives are adaptively aggregated to capture heterogeneous relevance signals during retrieval. This design allows LSP-RAG to perform accurate and robust retrieval using lightweight language models. LSP-RAG achieves superior performance across multiple KG-based Question Answering benchmarks compared to existing retrieval systems, including those that rely on much larger language models, demonstrating its effectiveness under limited computational budgets.

21. 【2611.25071】FinVeriQA: Financial Research Agents for Question Answering: Routing, Retrieval, MCP Tool Use, and Provenance-Aware Evidence Verification

链接:https://arxiv.org/abs/2611.25071

作者:Meenakshi Rajpurohit,Anna Phan,Ravikumar Komandur Narayanan,Gunanidhi Ramakrishnan,Nathan Howland

类目:Computation and Language (cs.CL); Artificial Intelligence (cs.AI); Information Retrieval (cs.IR); Multiagent Systems (cs.MA)

关键词:Large language models, Model Context Protocol, fluent financial analyses, recall stale prices, Large language

备注: 15 pages, 4 figures, 11 tables. Code: [this https URL](https://github.com/SJSU-CMPE259/AI-Stock-Research-and-Earnings-Assistant)

点击查看摘要

Abstract:Large language models (LLMs) write fluent financial analyses but miscompute ratios, recall stale prices, and route questions to the wrong tool with high confidence. We ask which component should do each job in a financial question-answering assistant, and study it with FinVeriQA, an open-source multi-agent research prototype over SEC filings and live market data: a compact typed-decision model (Laya) routes each question and screens out advice requests, Python computes every number, retrieval and tools fetch evidence, Mistral-7B writes a cited answer, a code validator checks that cited IDs exist and that numbers match computed notes, and a Model Context Protocol (MCP) server exposes the pipeline. In live experiments, Laya routed better than Mistral-7B (0.85 vs. 0.52; overall 0.86 vs. 0.68, McNemar p = 0.0003); a 12B Mistral decided no better, and DeepSeek-R1-8B decided better (0.79 vs. 0.68, p = 0.0192) but about 9.9 times more slowly. Mistral-7B computed 29% of 100 financial calculations correctly against 100% for Python, a live quote cut price error from 210% to 0.00%, and hybrid retrieval reached 76% Hit@5 on 50 filing questions. Yet in a live case study, both agent answers passed validation while describing AMD sample data under NVIDIA's and Alphabet's names. A controlled upper-bound study on the FinRank benchmark explains why: when one figure in the shared evidence is corrupted, self-consistency stays confident (agreement 0.55 to 0.59 while accuracy falls from 0.50 to 0.00), and a shared-evidence check (the kind our validator performs) catches 35% of the errors, whereas a check against the original passage catches 77% ($p = 5.96 \times 10^{-8}$); for DeepSeek-R1-8B, 68% vs. 90%. Verification must be independent of the evidence the writer saw.

计算机视觉

1. 【2611.26629】Posterior Matching Distillation

链接:https://arxiv.org/abs/2611.26629

作者:Hidir Yesiltepe,Julius Berner,Arash Vahdat,Pinar Yanardag

类目:Computer Vision and Pattern Recognition (cs.CV)

关键词:Diffusion distillation accelerates, Diffusion distillation, pretrained teacher iterative, iterative sampling process, compressing a pretrained

备注: Project Page: [this https URL](https://pmd-distillation.github.io/)

点击查看摘要

Abstract:Diffusion distillation accelerates image and video generation by compressing a pretrained teacher's iterative sampling process into a few network evaluations. Distribution-based methods achieve this by matching the student's output distribution to the teacher's. This matching is often driven by reverse-KL objectives, whose mode-seeking bias favors dominant teacher modes at the expense of diversity across samples. We introduce Posterior Matching Distillation (PMD), a new member of the distribution-based distillation family that combines mode-covering posterior sampling with any-step generation. At zero population loss, PMD recovers the teacher's full posterior of clean images at each trained noise level. Its stochastic student supports any-step sampling budgets within the trained timestep grid without further training. To learn these posteriors, PMD matches exponentially reweighted averages of student samples conditioned on a fixed noisy state to corresponding teacher posterior means. Under Gaussian corruption, exponentially tilted posterior mean is a single teacher evaluation at a shifted input. We prove that, matching tilted means over an open neighborhood uniquely identifies the full posterior. PMD thus supervises the entire conditional distribution through a family of mean constraints, without any auxiliary student-score network. Experiments on SDXL and FLUX.2 [klein] 4B demonstrate that PMD achieves the best FID among 4-step methods on both models, while maintaining competitive or leading performance across prompt-alignment and perceptual-quality metrics.

2. 【2611.26627】RL Post-Training in Diffusion Transformers Is Activation Steering

链接:https://arxiv.org/abs/2611.26627

作者:Tuna Han Salih Meral,Kaan Oktay,Alec Helbling,Atakan Kara,Duen Horng Chau,Pinar Yanardag

类目:Computer Vision and Pattern Recognition (cs.CV); Machine Learning (cs.LG)

关键词:Reinforcement learning, post-training substantially improves, visual quality, diffusion transformers, substantially improves

备注: Project page: [this https URL](https://rl-is-steering.github.io)

点击查看摘要

Abstract:Reinforcement learning (RL) post-training substantially improves the visual quality, text rendering, and compositional accuracy of diffusion transformers. Because capabilities such as rendering requested text or composing multiple objects are inherently prompt-dependent, a natural hypothesis is that improving them requires learning complex, input-dependent transformations. To isolate how much of these gains an input-independent shift explains, we probe each RL-tuned model with a deliberately simple parameterization: a fixed set of steering vectors added to the block outputs of the frozen base model. On SD3.5-M, these vectors hold 65x fewer parameters than the RL-tuned LoRA (0.29M against 18.8M). We fit them to match the velocity predictions of public RL-tuned checkpoints, with no reward in the objective. Although the vectors are identical for every prompt, token, and sampling seed, they recover 90.0-96.3% of the reward gains of six RL-tuned image models, and 89.8% and 111.9% on two RL-tuned video models. Initialized at zero and trained with online RL, the vectors alone exceed the reward gains of weight-based RL on text rendering (OCR) and human preference (PickScore) on SD3.5-M. A constant shift thus suffices for most of the gain of policy-gradient RL post-training: without new input-dependent transformations, it steers the pretrained generator toward what it can already compute.

3. 【2611.26624】PlayCraft: Designing Engaging Games from Agent Play Experience

链接:https://arxiv.org/abs/2611.26624

作者:Weihao Cheng

类目:Artificial Intelligence (cs.AI); Computer Vision and Pattern Recognition (cs.CV)

关键词:generate playable games, necessarily engaging, generate playable, Recent, games

备注:

点击查看摘要

Abstract:Recent AI agents can generate playable games, but these games are not necessarily engaging. Mastery is a key source of engagement, yet designing for it requires extensive human play and tuning. We propose PlayCraft, an automated framework that uses a vision-based agent to play games and draws on its detailed play experience to revise them for mastery. Starting from an initial game, PlayCraft alternates between playing and crafting. In the playing stage, a player agent watches the game screen, uses the controls, and pursues the game objective over repeated episodes while retaining memory. After each episode, it reports its progress, outcomes, decisions, and reflections. In the crafting stage, a crafter agent uses these reports to revise the game. It estimates player abilities, predicts performance at different proficiency levels, and compares those predictions with a target learning progression. It diagnoses the game and addresses the gaps by giving players meaningful opportunities to improve through better decisions and execution. We evaluate games in fresh runs with player agents across three intelligence settings, measuring learning gain and learning slope. Compared with the initial games, PlayCraft yields larger learning gains and steeper learning slopes in most games. It also outperforms the baselines on the macro-average of each metric at every intelligence setting.

4. 【2611.26617】One Prior, Two Dynamics: Repurposing Video Diffusion Transformers for Self-Supervised Latent Action Learning

链接:https://arxiv.org/abs/2611.26617

作者:Yuxin Jiang,Wenqiang Sun,Jiaqi Li,Yuchao Gu,Lvmin Zhang,Ivor Tsang,Mike Zheng Shou

类目:Computer Vision and Pattern Recognition (cs.CV); Artificial Intelligence (cs.AI); Machine Learning (cs.LG); Robotics (cs.RO)

关键词:dynamics, video, pretrained video, Videos reveal, action

备注: Project page: [this https URL](https://showlab.github.io/o2laf) Code: [this https URL](https://github.com/showlab/O2LaF)

点击查看摘要

Abstract:Videos reveal how the world changes, but rarely the actions that cause those changes. Latent action learning recovers such supervision from visual transitions, yet often learns the underlying dynamics largely from scratch. We ask whether a pretrained video generator can instead provide the prior for both inverse and forward dynamics. We introduce O$^{2}$LaF, built on a simple principle: one prior, two dynamics. We initialize both models from the same pretrained video DiT and adapt them to their distinct roles: a feed-forward transition encoder with learnable action queries for inverse dynamics, and a causal, action-conditioned video generator for forward dynamics. By controlling information access and optimizing both dynamics end-to-end through a compact bottleneck, O$^{2}$LaF learns transferable actions from unlabeled videos. Across navigation and robotic manipulation, O$^{2}$LaF sets a new state of the art in few-shot action probing and enables stronger zero-shot action transfer across visual contexts and embodiments. Beyond generative video priors, we further extend our framework to predictive representation priors such as V-JEPA2, achieving competitive performance. Our findings suggest a broader paradigm for latent-action learning: repurposing pretrained video models to discover transferable actions and predict their consequences.

5. 【2611.26615】ScaleWorld: High-Resolution World Generation through Shared Prediction and Refinement

链接:https://arxiv.org/abs/2611.26615

作者:Yansong Qu,Wesker Zheng,Quanjian Song,Zhiyuan Zhang,Xiao Cao,Shengchuan Zhang,Liujuan Cao

类目:Computer Vision and Pattern Recognition (cs.CV)

关键词:render fine-grained scene, fine-grained scene structures, directly generating high-resolution, video incurs substantial, incurs substantial computational

备注:

点击查看摘要

Abstract:High-resolution generation enables video world models to render fine-grained scene structures and textures, but directly generating high-resolution video incurs substantial computational and memory costs. We present ScaleWorld, a causal video model that unifies low-resolution world prediction and high-resolution refinement within a single shared backbone. By refining its own predictions, ScaleWorld produces high-resolution world videos without requiring a separate refinement model. For each video chunk, the shared backbone first completes a low-resolution prediction and then performs one-step high-resolution refinement conditioned on that prediction. To support efficient streaming generation, we maintain separate causal states with a long low-resolution history for prediction and a short high-resolution history for refinement. To address task interference during distillation, we separate specialist optimization from capability consolidation and introduce task-asymmetric on-policy distillation (TA-OPD). This procedure transfers prediction and refinement behaviors from specialized teachers into a shared student through local velocity matching under their respective sampling and history policies. {Extensive experiments demonstrate that ScaleWorld achieves high-quality world generation with improved computational and memory efficiency.

6. 【2611.26613】AgentRoboCritic: Understanding the Capability of Frontier Agents on Direct Robot Control

链接:https://arxiv.org/abs/2611.26613

作者:Yiyang Du,Yuxuan Kuang,Qitao Zhao,Yixuan Wang,Zhanqiu Guo,Yi Xu,Xin Li,Wenbin He,Liu Ren,Chenyan Xiong

类目:Robotics (cs.RO); Computation and Language (cs.CL); Computer Vision and Pattern Recognition (cs.CV)

关键词:Astra have demonstrated, Frontier multimodal language, controlling robots directly, multimodal language agents, low-level numerical actions

备注: 28 pages. Project page: [this https URL](https://adu2021.github.io/blog/AgentRoboCritic/)

点击查看摘要

Abstract:Frontier multimodal language agents such as GPT-6 Astra have demonstrated strong capabilities in controlling robots directly via low-level numerical actions without high-level primitives. However, existing benchmarks report only episode-level success rates, which show neither how these agents succeed nor where they fail. In this paper, we present a systematic action-level analysis of frontier agents on direct robot control across 12 manipulation benchmarks. We introduce AgentRoboCritic, an intent-grounded critique framework in which the policy agent states a local intent alongside each action, and a critic evaluates the action's outcome and failure mode against this intent. Our action-level analysis demonstrates the promising capabilities of frontier agents on direct robot control, including closed-loop recovery from local execution failures, reliable semantic grounding across diverse scenes, and strong zero-shot manipulation without task-specific adaptation. It also identifies their current limitations, including task-relevant physical execution bottlenecks such as grasping and alignment, and uneven recovery across failure modes. Finally, we validate our critic on a 400-action human-annotated benchmark and release a dataset of 12,844 annotated actions alongside a fine-tuned open-source critic, Muse-Glimmer-30B, whose failure-mode accuracy rises from 32.58% to 74.24%, enabling low-cost action-level analysis.

7. 【2611.26603】Depth Pro 2: Sharp and Temporally Consistent Metric Monocular Depth

链接:https://arxiv.org/abs/2611.26603

作者:Peiyun Hu,Marko Mihajlovic,Lyndon Duong,Lahav Lipson,Wei Dong,Amaël Delaunoy,Stephan R. Richter,Vladlen Koltun

类目:Computer Vision and Pattern Recognition (cs.CV); Machine Learning (cs.LG)

关键词:present Depth Pro, estimating dense metric, Depth Pro, approach to estimating, estimating dense

备注: Code and weights available at [this https URL](https://apple-aiml-research.github.io/ml-depth-pro-2/)

点击查看摘要

Abstract:We present Depth Pro 2, an approach to estimating dense metric depth for images and video at full resolution. By analyzing prior methods for monocular depth estimation, we find that they commonly trade off temporal consistency and boundary sharpness: multi-frame methods favor consistency, single-frame methods favor sharpness. To overcome this trade-off, we train a single, scalable network that processes both images and video and predicts depth maps at native resolution. Depth Pro 2 produces depth maps that are metrically accurate, sharp, and temporally consistent all at once, outperforming prior work in all of these dimensions. We release a family of models that define a new Pareto frontier in terms of metric accuracy, sharpness, temporal consistency, and runtime. We release code and weights at this https URL

8. 【2611.26595】Coda-DaVinci: Reimagining Video Generation with Code2Video

链接:https://arxiv.org/abs/2611.26595

作者:Yiyan Ji,Qixun Wang,Yang Shi,Zhuoran Zhang,Yifan Xu,Tengfei Liu,Yuqi Tang,Xiaohan Zhang,Shujie Li,Jiafu Tang,Xuanyu Zhu,Xinyu Liu,Yan He,Yushuo Guan,Zihao Wang,Xianghua Ying,Shixuan Liu,Haotian Wang,Wenjing Yang,Zhaoxiang Zhang,Yuanxing Zhang,Pengfei Wan,Jiaheng Liu

类目:Computer Vision and Pattern Recognition (cs.CV)

关键词:enabling explicit control, enabling explicit, explicit control, guide video generation, proxies favor

备注: 47 pages

点击查看摘要

Abstract:Code2Video uses agent-authored 3D proxies to guide video generation, enabling explicit control over structure and dynamics. However, how to construct proxies and prompts for optimal downstream quality remains underexplored, as do the capabilities and limitations of this paradigm. Through controlled experiments and blinded human evaluation, we study proxy granularity, physics simulation, and prompting. The results reveal task-dependent trade-offs: coarse proxies favor natural motion, while medium-grained proxies favor consistency. Physics simulation and proxy-aligned prompting further improve average quality. Guided by these findings, we introduce Coda-DaVinci, a task-adaptive agentic framework for Code2Video generation that selects between proxy-guided and direct generation, adapts modeling granularity to each entity, and coordinates simulation with proxy-grounded prompting. Its multi-agent harness preserves scene assets and guides repairs through visual inspection and independent review. Experiments demonstrate leading instruction following on VTR and scene consistency on R2M, improved physical plausibility over direct generation, and higher visual realism than the compared Code2Video baselines. Long-video evaluations further show gains in event structure and consistency across successive interactions, while revealing limitations in transition stability and adherence to scene settings.

9. 【2611.26590】Probing the "Conscious" Mind: Selective Access and Global Workspace in Vision-Language Models

链接:https://arxiv.org/abs/2611.26590

作者:Jiahao Huang,Jun Wang

类目:Artificial Intelligence (cs.AI); Computer Vision and Pattern Recognition (cs.CV); Machine Learning (cs.LG)

关键词:concept, Vision-language models, integrate visual, multimodal Jacobian lens, visual and textual

备注:

点击查看摘要

Abstract:Vision-language models (VLMs) integrate visual and textual information, but their outputs do not reveal internal concept representations. Global workspace theory (GWT) proposes that information is selected from parallel brain processes, integrated in a shared workspace, and made available to specialised processes. We introduce the Multimodal Concept Lens (MC-Lens), a multimodal Jacobian lens, to study these functions in VLM concept processing. Its vision encoder and language decoder lenses map intermediate states to a shared semantic space defined by the decoder's output vocabulary to compare concept accessibility across modalities. MC-Lens supports internal concept interventions to examine the role of accessed concepts in downstream computation. (i) Accessibility and selective access. Multimodal concept information is readable in intermediate states and selectively accessible as tasks require. We extend "readout carriers" from one name token to a token set to better capture and express concept information and support counterfactual experiments. (ii) Concepts in downstream computation. Counterfactual experiments show that accessed concept information participates in downstream computation and reasoning. (iii) Development of cross-modal alignment and selective access. We develop two J-space metrics tracking the development of cross-modal concept alignment more clearly than hidden state cosine in controlled VLM training. The experiment also shows that reliable selective access develops later than concept accessibility. (iv) Shared functions across models. Across VLM families and sizes, we find shared patterns of concept accessibility, query-dependent selectivity and cross-task intervention effects despite differences in internal organisation. These findings offer an experimental basis to study GWT in multimodal artificial systems and how it can guide cross-modal alignment improvements.

10. 【2611.26569】xt-Aware, Vision-Blind: How Multimodal Image Editing Systems Handle Unsafe Inputs

链接:https://arxiv.org/abs/2611.26569

作者:Ziqing Yang,Yixin Wu,Michael Backes,Yang Zhang

类目:Cryptography and Security (cs.CR); Computer Vision and Pattern Recognition (cs.CV); Machine Learning (cs.LG)

关键词:Multimodal image generation, made rapid progress, increasingly emphasize content, Multimodal image, image generation systems

备注:

点击查看摘要

Abstract:Multimodal image generation systems have made rapid progress and increasingly emphasize content safety. However, their safety mechanisms tend to rely on a single modality and lack joint consideration of both visual content and text prompts, particularly in instruction-guided image editing, where the uploaded image may itself carry risk. We thus introduce EditSafetyBench, a benchmark that measures system-level safety awareness of advanced image editing systems, containing 5,940 image-instruction pairs, covering safe and unsafe images across 11 risk categories and three edit intents. We then probe the benchmark dataset on four systems, including Gemini, GPT, Qwen API, and Qwen Local. Based on our evaluation, we observe a systematic text-vision safety mismatch: Gemini and GPT refuse 87.1\% and 81.6\% cases containing unsafe-intent editing prompts, while only refusing 24.2\% and 22.4\% cases with both unsafe images and neutral instructions. At the category level, most risks are considered inconsistently by different systems, e.g., GPT allows 70.4\% editing attempts of unsafe political content while Gemini only allows 34.8\%. Notably, sexual content shows the strongest visual sensitivity, likely benefiting from mature NSFW identification works. Our findings highlight the need for safety guardrails that jointly reason over both visual and textual modalities in image editing systems.

11. 【2611.26562】Multi-institutional grading of Barrett's esophagus dysplasia without stain normalization using a pathology foundation model and multiple instance learning

链接:https://arxiv.org/abs/2611.26562

作者:Amirali Khosravi,Jaidip M. Jagtap,D. Chamil Codipilly,Catherine E. Hagen,Jason T. Lewis,Rickey E. Carter,Shahriar Faghani,Vani J. A. Konda,Fouad Otaki,Shajan Peter,Nicholas Shaheen,Sachin Wani,Prasad G. Iyer,Bradley J. Erickson

类目:Computer Vision and Pattern Recognition (cs.CV); Artificial Intelligence (cs.AI)

关键词:complicating clinical decision-making, substantial inter-observer variability, suffers substantial inter-observer, grading suffers substantial, dysplasia grading suffers

备注:

点击查看摘要

Abstract:Barrett's esophagus dysplasia grading suffers substantial inter-observer variability, particularly for low-grade dysplasia, complicating clinical decision-making. We conducted a retrospective multi-institutional diagnostic accuracy study of a foundation model-based deep learning system for dysplasia grading, using 969 H\E-stained whole slide images from four institutions. Patch-level classifiers trained on expert annotations identified diagnostically relevant regions, followed by attribute-aware multiple instance learning for slide-level classification as non-dysplastic Barrett's esophagus (NDBE), low-grade dysplasia, or high-grade dysplasia. On a held-out test set (n = 130), macro-averaged AUC was 0.96 (95\% CI 0.93--0.98). For binary dysplasia detection, sensitivity was 98\% (80/82) and specificity 81\% (39/48), with a negative likelihood ratio of 0.030 and quadratic-weighted \kap{} of 0.83 against expert consensus. No high-grade dysplasia case was misclassified as NDBE. Performance was consistent across institutions without stain normalization, indicating that stain-robust foundation-model representations may remove the site-specific preprocessing that has constrained prior systems to institution-specific deployment.

12. 【2611.26540】AURA: AUdio-dRiven streaming Avatar

链接:https://arxiv.org/abs/2611.26540

作者:Nathaniel Cohen,Nicolas Dufour,Amélie Royer,Alasdair Newson,Patrick Pérez

类目:Computer Vision and Pattern Recognition (cs.CV); Artificial Intelligence (cs.AI)

关键词:generally require dozens, Current audio-driven video, synthesize realistic talking-head, required properties, Current audio-driven

备注: BMVC 2026. Project page: [this https URL](https://kyutai.org/aura)

点击查看摘要

Abstract:Current audio-driven video diffusion models can synthesize realistic talking-head videos but generally require dozens of diffusion denoising steps over multi-second video chunks, making them unsuitable for streaming applications such as voice assistants and virtual avatars, where low-latency and online causal generation are required properties. AURA generates video chunk by chunk with a rolling KV cache, requiring only the audio of the current chunk to start generating, rather than a multi-second audio look-ahead like bidirectional models. It comes in two sizes: AURA 1.3B streams 576p video in real time on a single GPU (20.7 fps for a 16 Hz playback rate), making it suited to interactive avatar applications, and AURA 5B streams 720p video at 9.7 fps. Our two-stage distillation pipeline first aligns the causal student with the teacher, by training it to match the teacher's trajectories with fewer diffusion steps and a chunk-wise sliding window causal attention mask, then applies Distribution Matching Distillation (DMD) with backward simulation to close the remaining quality gap. On standard talking-head benchmarks, AURA 1.3B runs 23x faster than its own bidirectional teacher and reaches a quality comparable to TalkVerse at a lower resolution, and AURA 5B matches its TalkVerse teacher at 720p while running 17x faster. To support reproducibility and further research on streaming avatar generation, we release the weights of both models along with a dataset of 30K synthetic videos and their precomputed teacher trajectories.

13. 【2611.26522】Deterministic Temporal Embedding for Calcium Imaging Trace Extraction

链接:https://arxiv.org/abs/2611.26522

作者:Qi Gao,Julius Beichert,Miriam Ratliff,Karl Rohr

类目:Computer Vision and Pattern Recognition (cs.CV); Machine Learning (cs.LG); Image and Video Processing (eess.IV)

关键词:sequences requires identifying, requires identifying pixels, fluorescence intensity reflects, underlying cellular activity, imaging sequences requires

备注: 5 pages. Submitted to IEEE ISBI 2027

点击查看摘要

Abstract:Reliable extraction of calcium-dependent fluorescence traces in calcium imaging sequences requires identifying pixels whose fluorescence intensity reflects the same underlying cellular activity. We present a deterministic temporal embedding framework that maps pixel-wise fluorescence traces to a low-dimensional representation in which temporally coherent sources form spatially coherent regions. Starting from a fuzzy-neighborhood graph and spectral initialization, we optimize a class-balanced cross-entropy objective using exact sparse attraction and a deterministic particle-mesh decomposition of the dense repulsive interaction with sparse edge compensation, avoiding stochastic negative sampling. The resulting latent representation is spatially decoded into reliable source supports for trace extraction without imposing an explicit cellular shape prior. Experiments on synthetic and in vivo two-photon calcium imaging data demonstrate robust recovery of irregular and spatially touching sources together with efficient deterministic embedding optimization.

14. 【2611.26516】Plan4D: Generative Plannable 4D Worlds

链接:https://arxiv.org/abs/2611.26516

作者:Hangwei Zhang,Rujing Dang,Ruimin Lin,Jiahong Wu,Xiangxiang Chu,Chuanxia Zheng

类目:Computer Vision and Pattern Recognition (cs.CV)

关键词:prioritize aestetic quality, weak physical grounding, prediction prioritize aestetic, limited geometric consistency, future prediction prioritize

备注: NeurIPS 2026, Project Page: [this https URL](https://fogradio.github.io/plan4d-project)

点击查看摘要

Abstract:Most world models for future prediction prioritize aestetic quality of generated videos, but offer only weak physical grounding and limited geometric consistency over time, due to the lack of explicit 4D modeling. In this paper, we introduce plan4d, a possible next milestone towards building a generative plannable 4D world model that predicts future RGB videos together with dense 4D geometry, including depth, cameras, point clouds, and scene flow from an observed scene context and text-specified actions. Since training data with paired actions and dense 4D annotations is inherently scarce, plan4d leverages the complementary priors of foundational video generators and 3D reconstructors. Specifically, we propose a model-stitching architecture that couples Depth Anything 3 with LongCat-Video and augments it with an auxiliary DPT head for scene-flow prediction. The resulting model allows us to plan and generate complex, long-horizon, and fine-grained actions in a dynamic 3D scenes, while substantially improving both visual fidelity and geometric accuracy of the generated future videos. Extensive experiments on both synthetic and real-world datasets demonstrate that plan4d achieves state-of-the-art performance in generative 4D modeling of environmental reactions. Project Page: this https URL

15. 【2611.26511】Direct3D-V3: Latent-Free Diffusion Transformers in the Native Voxel Domain

链接:https://arxiv.org/abs/2611.26511

作者:Shuang Wu,Jiachen Qian,Youtian Lin,Feihu Zhang,Philip Torr,Yao Yao,Xun Cao

类目:Computer Vision and Pattern Recognition (cs.CV)

关键词:stronger variational autoencoders, recent advances concentrating, elaborate latent representations, increasingly elaborate latent, structured sparse voxels

备注: The Project Page is: [this https URL](https://dreamtechai.github.io/Direct3D-V3) . The Paper is published in ACM TOG (SIGGRAPH Asia 2026)

点击查看摘要

Abstract:Native 3D generation has largely been built around latent diffusion, with recent advances concentrating on stronger variational autoencoders (VAE) and increasingly elaborate latent representations, from vecset and triplanes to structured sparse voxels. Despite their architectural differences, these methods all share the same premise: geometry is first compressed into a latent space and only then modeled by the generator. We take a different route and propose Direct3D-V3, a latent-free diffusion transformer (DiT) framework that discards the VAE altogether and operates in the native voxel domain, directly modeling raw SDF voxel grids during both training and sampling. To make this direct voxel formulation geometrically reliable, we introduce an implicit manifold regularization over neighboring voxels to suppress non-watertight surfaces, and convert predicted SDF grids into meshes via Marching Cubes so that rendered normal losses can supervise the model directly from its predicted SDF. To improve alignment between the conditioning image and generated geometry, Direct3D-V3 also predicts the camera parameters of the conditioning image as an auxiliary task, encouraging the DiT to learn the mapping from image observations to 3D shape. This latent-free design removes the reconstruction bottleneck, brings training under an objective in the native voxel domain, and yields meshes with substantially richer local detail. The predicted camera and native SDF voxel output further make Direct3D-V3 extensible to generative reconstruction, where diffusion is interleaved with normal-guided refinement to improve alignment with the observed view while the generative prior completes unseen regions. Project page: this https URL.

16. 【2611.26508】Visual Attribution Limits and Graded Provenance Auditing in Commercial Video-Generation APIs

链接:https://arxiv.org/abs/2611.26508

作者:Haocheng Mei,Yuchuan Tian,Yingte Shu,Chao Xu,Kai Han,Hanting Chen,Yunhe Wang

类目:Cryptography and Security (cs.CR); Computer Vision and Pattern Recognition (cs.CV)

关键词:substitute cheaper models, video-generation APIs bill, Commercial video-generation APIs, cheaper models, cached clips

备注: 23 pages, 2 figures, 13 tables

点击查看摘要

Abstract:Commercial video-generation APIs bill by model and tier, while intermediaries could substitute cheaper models, lower tiers, or cached clips. What can a delivered file establish about its origin? We combine measurements of passive attribution and cryptographic provenance into a graded audit. We test attribution probes and zero-shot detectors under re-encoding, downscaling, and frame-rate changes, and survey C2PA and GB 45438 metadata across 19 commercial endpoints. Only authenticated provenance supports accusations: tier contradictions or duplicate signed instances under fresh-generation contracts, excluding retries and idempotent returns. Unverifiable signed-endpoint deliveries trigger original-file demands under pass-through clauses; archive-reuse and delivery-parameter screens support review only. Six-model container and bitstream accuracy falls from 83.9% on raw files to 45.3% after uniform re-encoding, using different feature sets. Subsampling real references from 24 to 3 fps raises pooled D3 AUROC from 0.845 to 0.988. Only BytePlus authenticates tier claims under our pinned policy, including 187/187 audited deliveries across three Seedance tiers; tested remuxing and re-encoding leave processed Seedance files unverifiable. The audit identifies authenticated contradictions in 100/100 raw 720p signed tier-swap cases, with zero observed false flags on the tested honest pools.

17. 【2611.26493】SGAR: Feed-Forward 3D Reconstruction via Sparse Geometry-Guided Attention

链接:https://arxiv.org/abs/2611.26493

作者:Aoxiang Fan,Maria Pilligua,Corentin Dumery,Oliver Heinimann,Pascal Fua

类目:Computer Vision and Pattern Recognition (cs.CV)

关键词:view patches densely, hoc attention designs, patches densely, designs that relate, attention

备注:

点击查看摘要

Abstract:Feed-forward models have been very successful in 3D reconstruction with ad hoc attention designs that relate all view patches densely. However, the cost of this dense global attention grows quadratically with the number of views, preventing the model from applying to larger-scale reconstruction settings. In this paper, we argue that most dense attention computations are unnecessary from a classical structure-from-motion perspective: 3D structures stem from geometrically matching regions in 2D. We bring this principle to learning-based feed-forward reconstruction with our Sparse Geometric Attention Reconstruction (SGAR) method. In SGAR, each patch token attends to only a small set of patch tokens from other views, chosen by a lightweight learned key selection module trained to predict cross-view 3D proximity of tokens. Thus, dense global attention is replaced by much more efficient geometric-similarity-guided sparse attention. SGAR is a general method that can be applied to different feed-forward models by fine-tuning. In our experiments, we show that SGAR is the fastest among both state-of-the-art dense and sparse methods and the most accurate sparse one. It runs up to 12-times faster than dense attention at 1,000 views with stable pose accuracy where existing sparse methods fail, and shows the best performance in reconstructing more complete geometries on the same time budget.

18. 【2611.26478】SHUSH: Emission-Aware Reward Design for Proactive Video Assistants

链接:https://arxiv.org/abs/2611.26478

作者:Shawn Liang,Soamar Homsi,Gourav Datta

类目:Computer Vision and Pattern Recognition (cs.CV)

关键词:Proactive video assistants, Proactive video, Proactive, reward, video assistants

备注: under review at ICASSP

点击查看摘要

Abstract:Proactive video assistants must decide not only what to say, but also when to speak. Existing reinforcement-learning objectives can reward timely, correct responses without accounting for response volume, thereby encouraging unnecessary or repeated emissions. We identify this failure mode in MMDuet2, whose proactive area-under-the-curve (PAUC) reward promotes timely responses but does not price the number of emissions. We show analytically that this makes additional responses weakly penalized or effectively free, systematically encouraging over-emission. We then propose an emission-aware F1 reward for interval-valued annotations, using exclusive span--emission matching to price false emissions and missed spans. Under MMDuet2's training recipe, our policy on the visual subset of OmniPro replies 2.5x less often than MMDuet2 while raising timing F1 from 0.259 to 0.396 and content-aware F1 from 0.103 to 0.154; a more conservative variant cuts replies by 82.5% and raises precision from 0.161 to 0.418. On ProactiveVideoQA, the same reward adds replies where the baseline is nearly silent, and PAUC ranks a heavily over-emitting policy second despite near-last F1, because it is insensitive to response count. Our results indicate that proactive assistants should be trained and evaluated with objectives that price emissions.

19. 【2611.26456】Fixing the Last Mile: Failure-Aware Image Restoration with Region-Specific Guidance

链接:https://arxiv.org/abs/2611.26456

作者:Hongyu An,Fanghua Yu,Kainan Yan,Luc Van Gool,Chao Dong,Jinjin Gu

类目:Computer Vision and Pattern Recognition (cs.CV)

关键词:producing visually plausible, Recent advances, visually plausible results, greatly improved global, producing visually

备注:

点击查看摘要

Abstract:Recent advances in image restoration have greatly improved global image fidelity, producing visually plausible results across most regions of an image. Nevertheless, even high-quality restorations often exhibit localized failures, including artifacts in semantically important regions such as faces and text, structural inconsistencies, and texture mismatches. Although these errors typically affect only a small portion of the image, they often dominate the overall perceptual quality of the result. In this work, we propose Failure-Aware Image Restoration (FAIR) to address the "last-mile" problem in image restoration. Rather than applying one uniform generative prior over the whole image, we condition a single-pass restoration model on both where local failures are likely to occur and what corrective guidance each region requires. This region-bound conditioning enables fine-grained and controllable restoration that improves semantic accuracy and stylistic consistency in critical regions while maintaining faithful, high-quality restoration elsewhere. Our method complements existing restoration pipelines by shifting the emphasis from global fidelity to local perceptual acceptability. Extensive experiments show that the proposed approach resolves common failure cases and improves perceptual quality, bridging the gap between technical accuracy and practical usability.

20. 【2611.26444】Mind2Act World: A Diagnostic Benchmark for Embodied Mind, Physical Action, and Mind-Action Composition in Robotic Manipulation

链接:https://arxiv.org/abs/2611.26444

作者:Yuzhi Huang,Jie Wu,Fanding Huang,Junchao Lin,Weichen Zhang,Ziyi Xiong,Yang Li,Zhengwei Xie,Zhimin Wang,Zhen Liu,Jiazhen Huang,Yuzheng Zhang,Yuche Liu,Hongyi Yang,Chen Tang,Shixiang Tang,Hakan Aktas,Adam Wu,Duo Wu,Kangye Ji,Shuzhao Xie,Huanan Liu,Weijue Bu,Chenxin Li,Xinyuan Wang,Tianyi Bai,Yinan Mao,Jiawei Wang,Minglei Li,Heng Yu,Ehsan Adeli,Minchen Li,Yong-Lu Li,Cengiz Öztireli,Huazhe Xu,Chao Yu,Jing Zhang,Shanghang Zhang,Weitao Zhou,Wenwu Zhu,Cheng Chi,Jingyan Jiang,Zhi Wang

类目:Robotics (cs.RO); Artificial Intelligence (cs.AI); Computer Vision and Pattern Recognition (cs.CV)

关键词:Embodied intelligence aims, perform complex physical, robots perform complex, history-based cognitive reasoning, Embodied intelligence

备注:

点击查看摘要

Abstract:Embodied intelligence aims to let robots perform complex physical tasks with human-like adaptability and reliability, which requires three complementary capabilities: history-based cognitive reasoning, precise and timely physical execution, and the composition of reasoning and execution through environmental feedback. Yet existing benchmarks either evaluate reasoning and execution within the same tasks or assess them separately without their composition, so they reveal little about whether a failure comes from reasoning, execution, or their composition. We introduce Mind2Act World, a diagnostic benchmark of 30 simulated manipulation tasks from five real-world application domains. Each task has three difficulty levels and two scene settings (Clean and Random), yielding 180 configurations. The tasks form three views: Mind tests decisions based on past observations, Act tests precise and timely execution, and Mind2Act combines both in closed-loop tasks where action outcomes guide later decisions. Comparing the three views shows whether a system lacks a capability or fails to compose capabilities it already has. A diagnosis harness further attributes the failed episodes of three representative systems to Mind, Act, or their composition and summarizes each system in a profile card. Our evaluation of multimodal agents, vision-language-action models (VLAs), and world-action models (WAMs) shows that competence in a single view does not carry over to Mind2Act tasks. Even the strongest agent succeeds in 52.7% of Mind and 39.0% of Act episodes but in only 12.0% of Mind2Act episodes. The strongest agent retains task state but struggles with sustained contact, whereas end-to-end policies execute practiced motions but retain little task state. These findings point to composing reasoning and execution through environmental feedback as the main open problem.

21. 【2611.26438】Risk-Controlled Test-Time Adaptation for Vision-Language-Action Models

链接:https://arxiv.org/abs/2611.26438

作者:Zixuan Liu,Hashem Haghbayan,Zizhan Zheng,Siavash Khajavi

类目:Computer Vision and Pattern Recognition (cs.CV)

关键词:updating model parameters, models enable efficient, model parameters, efficient test-time adaptation, updating model

备注:

点击查看摘要

Abstract:Retrieval-augmented vision-language-action (VLA) models enable efficient test-time adaptation by leveraging expert demonstrations as additional context without updating model parameters. However, even when retrieved demonstrations are relevant to the current task, the VLA may fail to effectively utilize the retrieved information, ultimately leading to task failure. To address this challenge, we propose R2-VLA, a risk-controlled retrieval-augmented VLA framework that enables reliable test-time adaptation by satisfying a user-specified task success rate requirement with minimal expert assistance. Specifically, R2-VLA learns an uncertainty predictor that estimates the uncertainty of the VLA's predicted actions given the retrieved demonstrations, guiding when to act autonomously or request expert assistance. We further introduce a task-level risk-control procedure that calibrates the uncertainty threshold to satisfy a user-specified task-failure risk requirement. Moreover, R2-VLA continually incorporates expert corrections into its retrieval memory to improve future adaptation while preserving the prescribed task-level risk guarantee. Experiments across three VLA backbones and four LIBERO benchmark suites demonstrate that R2-VLA improves average task success by 9.2--11.2 percentage points, achieving the desired target with fewer expert requests. These results highlight the effectiveness of connecting the usefulness of retrieved demonstrations with task-level risk control to enable successful robotic manipulation.

22. 【2611.26437】Asuka-FM: A System-Architectural Co-Design Framework for Efficient Text-to-Image Pretraining on Consumer Hardware

链接:https://arxiv.org/abs/2611.26437

作者:Víctor Rodrigo González,Pablo M. Olmos,Ignacio Peis,Alejandro Lancho

类目:Computer Vision and Pattern Recognition (cs.CV); Artificial Intelligence (cs.AI)

关键词:parameter Diffusion Transformers, Diffusion Transformers, massive compute clusters, models predominantly rely, multi-billion parameter Diffusion

备注: 40 pages, 19 figures, 17 tables. Project page: [this https URL](https://deeplearningcafe.github.io/projects/asuka-fm) ; Code: [this https URL](https://github.com/deeplearningcafe/asuka-fm)

点击查看摘要

Abstract:Frontier text-to-image (T2I) foundation models predominantly rely on multi-billion parameter Diffusion Transformers (DiTs) pretrained across massive compute clusters, creating an accessibility barrier for academic laboratories. When training DiTs from scratch on single consumer GPUs (e.g., an NVIDIA RTX 4090 with 24\,GB VRAM), practitioners encounter three primary bottlenecks: quadratic attention sequence costs, Just-In-Time (JIT) compilation graph breaks induced by variable aspect-ratio bucketing, and an autoencoder throughput wall where frozen Variational Autoencoder (VAE) operations dominate step latency. In this work, we introduce \textbf{Asuka-FM}, an end-to-end system and architectural framework demonstrating that competitive dual-stream DiTs ($\sim$206M parameters) can be pretrained from scratch on a single consumer GPU in $\sim$160 GPU hours for under \$55. Asuka-FM integrates four core designs: (i) an AdaLN-free dual-stream DiT backbone reducing parameter count; (ii) structured sequence sparsification via Sparse-Dense Residual Fusion (SPRINT) to cut attention FLOPs by over 50\%; (iii) continuous aspect-normalized Rotary Position Embeddings (RoPE) coupled with shifted square crop training, guaranteeing static tensor shapes while unlocking zero-shot virtual camera control; and (iv) a pixel adaptation phase utilizing a lightweight (10M parameter) Patch Detailer Head operating under velocity regression ($v$-prediction). Evaluated on open-domain benchmarks without benchmark-specific fine-tuning, Asuka-FM achieves GenEval 0.588 / DPG 78.96 scores at 256px and GenEval 0.609 / DPG 78.51 scores at 512px, setting a new Pareto-optimal efficiency standard for democratic foundation modeling.

23. 【2611.26434】HumanCOIN: A 12,500-Hour Dataset for Human Intent in Egocentric Manipulation

链接:https://arxiv.org/abs/2611.26434

作者:Shihan Wu,You Liu,Chunyu He,Minghui Qin,Longhan Jiao,Jinbang Guo,Shaoxuan Xie,Kaixi Tian,Wenjie Chen,Jingrui Pang,Yuanhang Zhang,Yichi Zhang,Miao Liu,Tengfei Pan,Yance Jiao,Yunyan Zhang,Yihang Li,Yongjian Shen,Li Liu,Kai Zhu,Chen Fang,Jianxing Chen,Sihan Liu,Xiaofeng Wang,Jie Zhao,Jing Li,Yueqi Wang,Yichen Tang,Wei Li,Zhexuan Yang,Yue Cao,Ji Zhao,Kai Wang,Dongsheng Jiang,Yongwei Wang,Wei Mu,Jiangmiao Pang,Maoqing Yao,Wenchao Ding,Zheng Zhu,Zhongpu Xia,Dalong Du,Jianwei Niu,Dahui Gao,Zhiwei Yu,Bo Lei,Huan Yang,Yonghua Lin,Tiejun Huang,Zhongyuan Wang,Guocai Yao

类目:Computer Vision and Pattern Recognition (cs.CV); Robotics (cs.RO)

关键词:policy network weights, manipulation datasets treat, First-person manipulation datasets, first-person manipulation dataset, First-person manipulation

备注: 41 pages, 10 figures, Project page: [this https URL](https://flagopen.github.io/HumanCOIN)

点击查看摘要

Abstract:First-person manipulation datasets treat the observation-action pair as the atomic unit and compress the generation and execution of an action into a policy network's weights, rendering the intent state behind the action unobservable. Human motor control research shows that intent mediates between observation and action and leaves observable traces along the intent pathway from decision to effect: gaze reaches the target before the hand, muscle load builds before contact, and cortical activity precedes the motor command. To record this pathway at scale, we present HumanCOIN, a first-person manipulation dataset built around human intent: 12,500 hours across 38 scene classes with three tiers of proxies. At the output end, the L1 attention tier (10,000 h) infers gaze from headband camera video, exposing the operator's next object of attention and reach before the hand moves. Midway, the L2 exertion tier (2,000 h) records surface EMG from armbands, making load expectations readable before execution is complete. At the origin, the L3 cognition tier (500 h) records scalp EEG, giving early readouts of task decisions and monitoring. Each tier inherits all proxies from the tiers below. The dataset further provides multi-view RGB images, depth maps and point clouds, 6-DoF head and wrist pose, and step-level action and object annotations. The controlled L3 setting additionally measures gaze, hand pose and finger kinematics directly. Clock and calibration data are released with the dataset so that any read rate can be re-aligned. The three tiers cover three stations, from visual orienting to cortical preparation, making the process behind visual-action pairs available for modeling at dataset scale. Project page: this https URL.

24. 【2611.26433】AdaPixFlow: Learning Continuous-Time 3D Dynamics Across Scenes from Sparse Multi-view Observations

链接:https://arxiv.org/abs/2611.26433

作者:Hrishikesh Viswanath,Aniket Bera

类目:Computer Vision and Pattern Recognition (cs.CV)

关键词:scenes requires recovering, unobserved times, Adaptive Pixel Flows, requires recovering, Understanding

备注:

点击查看摘要

Abstract:Understanding how objects move in dynamic scenes requires recovering their 3D motion from visual observations and predicting their positions at unobserved times. Existing methods either optimize scene-specific velocity fields or impose predefined physical and kinematic assumptions. We instead study learning a continuous-time dynamics operator across scenes, mapping multi-view visual observations to 3D object kinematics. We introduce Adaptive Pixel Flows (\ours{}), which learns this operator from multi-view RGB-D observations without per-scene optimization or predefined motion models. The method fuses observations from all cameras into a shared voxel grid in world coordinates, from which the network predicts a continuous-time velocity field. Because the operator is trained across scenes, it learns motion priors that carry over to unseen scenes. The field can then be integrated from observed states to predict object positions between observations and beyond the observed sequence. On held-out PhysInOne scenes, \ours{} improves F1 over the strongest competing method by 2.5\% for in-between prediction and 15\% for future prediction. Compared with per-scene velocity-field methods, the improvements are 30\% and 85\%, respectively. Applied zero-shot to Spring-Gaus, it improves future-prediction F1 by 12\%.

25. 【2611.26431】FFGaussType: Feed-Forward Conversion of Glyph and SVG Geometry into Compact Gaussian Assets

链接:https://arxiv.org/abs/2611.26431

作者:Haoan Feng,Tianfu Wang,Peimeng Guan,Dae Yeol Lee,Guan-ming Su

类目:Computer Vision and Pattern Recognition (cs.CV); Graphics (cs.GR)

关键词:graphical elements, flat overlays, text and graphical, displayed as flat, Immersive

备注: 20 pages. Project page: [this https URL](https://fengyee.github.io/FFGaussType/)

点击查看摘要

Abstract:Immersive 3D content allows text and graphical elements, such as subtitles, annotations, and icons, to be placed inside a scene rather than displayed as flat overlays. Placed this way, they respect occlusion and perspective and take on user-assigned materials and lighting while preserving their original shapes. In Gaussian scenes, however, each new glyph or icon requires its own iterative optimization, which limits scalable content creation. We present FFGaussType, which converts extruded glyph and SVG outlines into Gaussian assets in a single forward pass. A point encoder maps anchors sampled on the known surface to a single feature plane, from which a decoder shared by all assets predicts neural Gaussians carrying surface normals. This separates geometric conversion from material and lighting choices, and the feature plane supports image and video coding for compact delivery. We construct a paired dataset of 27,221 glyph and SVG icon assets with anchors and calibrated normal and silhouette renderings, holding out font families for evaluation. Conversion from prepared anchors takes 9.1 ms per asset, at a fraction of the compute and memory of prior baselines. On unseen font families and test icons, FFGaussType achieves $1.62^\circ$ normal error and $0.985$ silhouette IoU, the best among the compared feed-forward converters, matching the fidelity that per-asset fitting needs thousands of steps to reach. The code and dataset will be released after publication.

26. 【2611.26427】MemFolder: Structure-Guided Memory Folding for Efficient Agent Memory Management

链接:https://arxiv.org/abs/2611.26427

作者:Xuyang Liu,Chenwen Lu,Junxi Wang,Siyu Ren,Xinying Lin,Siteng Huang,Hongli Sun,Linfeng Zhang,Honggang Chen

类目:Computation and Language (cs.CL); Computer Vision and Pattern Recognition (cs.CV)

关键词:Long-term agents accumulate, agents accumulate large, accumulate large external, Long-term agents, large external memory

备注: Code is available at [this https URL](https://github.com/xuyang-liu16/MemFolder)

点击查看摘要

Abstract:Long-term agents accumulate large external memory stores as interactions unfold, increasing the costs of storage, retrieval, and response generation. Efficiency is constrained by two coupled factors: a growing record count increases retrieval overhead, while longer retrieved contexts increase prefill computation and decoding-time attention costs. Existing methods prune or consolidate records, but pruning can discard useful evidence, whereas consolidation may produce fewer yet longer retrieval units. We introduce MemFolder, a structure-guided memory folding framework that sequentially reduces record count and context length. The framework comprises two core components that share memory groups, using cross-record structure to guide both evidence consolidation and compact representation. RecordFolder grounds entity mentions, progressively selects representative yet complementary anchors, and combines semantic and entity affinity with temporal chaining to consolidate compatible records into fewer retrieval units. ContextFolder encodes the original records within these groups into compact text and uses verification feedback to revise encodings when key facts cannot be recovered. Experiments on the three text-based memory benchmarks LoCoMo, PersonaMem, and LongMemEval and the two multimodal benchmarks M3-Bench-web and M3-Bench-robot demonstrate the strong performance of our methods. Notably, on Mem0, RecordFolder retains 99.0% of the original average performance across the three text-based benchmarks with 50% fewer records, exceeding the strongest baseline by 1.9 percentage points in performance retention. Combined with ContextFolder, existing record compression methods further shorten their output records while retaining most of their performance.

27. 【2611.26420】Awaken-HI: Scaling Humanoid Intelligence with Whole-Body Human Experience

链接:https://arxiv.org/abs/2611.26420

作者:EngineAI

类目:Robotics (cs.RO); Artificial Intelligence (cs.AI); Computer Vision and Pattern Recognition (cs.CV)

关键词:everyday work people, homes and workplaces, central goal, perform the everyday, everyday work

备注: project page: [this https URL](https://engineai-robotics.github.io/Awaken-HI)

点击查看摘要

Abstract:A central goal of humanoid robotics is for robots to perform the everyday work people do in homes and workplaces. Existing egocentric video datasets cover many of these activities but provide little synchronized whole-body motion data for learning loco-manipulation at scale. We build Awaken-Human, which pairs approximately 50,000 hours of egocentric stereo video with synchronized whole-body human motion and covers more than 100 skills. Building on this, we also introduce Awaken-HI (Humanoid Intelligence), a vision-language-action model for humanoid loco-manipulation that jointly predicts original human motion and retargeted robot actions through a shared action expert. We show that loco-manipulation skills transfer more effectively from people to humanoids when human trajectories directly supervise the shared action expert. Increasing human pretraining data reduces held-out robot action prediction error according to an empirical scaling law and raises the mean success rate on downstream real-robot tasks. After full-corpus pretraining and T800 adaptation, Awaken-HI achieves a mean success rate of 93.75% across eight individual tasks and outperforms GR00T N1.7 and $\pi_{0.5}$ in both individual and long-horizon evaluations. The same policy generalizes to objects unseen during robot post-training and to new pairings of familiar objects and destinations.

28. 【2611.26409】WeMM-Video: Pushing the Frontiers of Open-Source Omni-Modal Video Understanding

链接:https://arxiv.org/abs/2611.26409

作者:Changli Tang,Jie Yang,Tianyi Wang,Lei Li,Daoze Zhang,Zijie Xin,Zhan Liu,Li Li,Jinchang Xu,Fengyun Rao,Chao Zhang,Jing Lyu

类目:Computer Vision and Pattern Recognition (cs.CV); Audio and Speech Processing (eess.AS)

关键词:remains a central, central challenge, accompanying audio, omni-modal video understanding, multimodal intelligence

备注:

点击查看摘要

Abstract:Video understanding, particularly with accompanying audio, remains a central challenge in multimodal intelligence. In this report, we introduce \textbf{WeMM-Video}, a family of Chinese-English bilingual omni-modal foundation models spanning 2B, 4B, 9B, and 35B-A3B variants. Centering on joint audio-visual perception and reasoning, WeMM-Video unifies text, image, audio, and video processing within a hybrid-thinking paradigm, seamlessly supporting both a non-thinking instruct mode for fast, direct responses and a thinking mode for explicit step-by-step reasoning. The models are trained on a large-scale multimodal corpus of approximately 1.2 trillion tokens through progressive pre-training and a multi-stage post-training pipeline incorporating multi-teacher OPD, anti-repetition MrDPO, and specialist RL. Evaluated across more than 120 benchmarks, WeMM-Video demonstrates superior performance in omni-modal video understanding and reasoning, while maintaining highly competitive capabilities across general audio, silent video, image, and text tasks. Specifically, in omni-modal video evaluations, our flagship WeMM-Video-35B-A3B establishes a new open-source state of the art, substantially outperforming proprietary models such as Qwen3.5-Omni-Flash and Seed2.0-Mini-0428 while narrowing the gap with the strong Gemini-3.0-Flash, and our compact WeMM-Video-9B surpasses the larger open-source Qwen3-Omni-30B-A3B-Instruct. We publicly release the WeMM-Video family along with our evaluation and reproduction pipeline to foster further research and development in the open-source community.

29. 【2611.26408】raining Vision Models for LLM-Driven Weight-Space Programming

链接:https://arxiv.org/abs/2611.26408

作者:Timo Ropinski,Sebastian Kranz

类目:Computer Vision and Pattern Recognition (cs.CV)

关键词:large language models, weight-space programming, large language, directly modify, perform weight-space programming

备注: Project page: [this https URL](https://viscom.uni-ulm.de/wsp)

点击查看摘要

Abstract:It has been demonstrated that, in certain settings, large language models can perform weight-space programming to directly modify the weights of neural networks. However, it is unknown how dedicated training of these networks can contribute to the successes of weight-space programming. We present a benchmark of LLM-driven weight-space programming on vision models, in which five LLMs edit convolutional networks and vision transformers on relabeling and error-correction tasks on Waterbirds, CIFAR-100, PACS and Entity-30. We report more than 2,600 such edit processes, whereby each edit is scored on held-out images for success, side effects and accuracy loss. We compare standard training with label smoothing, gradient-based editable training, and our Edit-Ready Training (ERT), a novel method that rehearses the direct, closed-form weight edits LLMs perform. We show that preparing networks for editing raises the success of LLM edits substantially where standard networks are hard to edit. ERT further achieves the highest mean score for GPT-6 Astra and Claude Opus 5.5 across all evaluated baselines. We will release our LLM testing framework alongside all tasks, editing trajectories and scores. Project page: this https URL

30. 【2611.26405】acInno: Teaching Vision-Language-Action Models the Contact Dynamics Beyond Vision and Action

链接:https://arxiv.org/abs/2611.26405

作者:Weile Li,Yunfan Lou,Muqing Shi,Zihao Zhang,Zhenbiao Huang,Boxian Yao,Xiyu Zhu,Zhibo Pang

类目:Robotics (cs.RO); Computer Vision and Pattern Recognition (cs.CV)

关键词:perceive visually subtle, visually subtle contact, subtle contact events, VLA models rely, visuo-tactile VLA models

备注: 11 pages, 4 figures, 4 tables. Project page: [this https URL](https://tacinno.github.io)

点击查看摘要

Abstract:Vision-Language-Action (VLA) models have recently been equipped with tactile sensing to perceive visually subtle contact events in contact-rich manipulation. However, adding tactile input yields inconsistent gains, and some visuo-tactile VLA models rely little on tactile input, because much of the tactile change, such as the gel deformation during gripper closure, is predictable from visual observations and executed actions. To address this problem, we introduce TacInno, which isolates the contact dynamics that vision and executed actions cannot explain, termed tactile innovation, and uses tactile innovation only as a training target. Specifically, the TacInno tokenizer, conditioned on visual observations and executed actions, compresses the future tactile change into a discrete code, and the policy, TacInno-VLA, learns to predict the code, adding no module for the code at deployment. TacInno-VLA achieves average success rates of 87.5% across six simulation tasks and 70.5% across four real-world tasks, outperforming the strongest visuo-tactile baseline by 10.7 and 9.0 percentage points, respectively. Ablation studies validate the contributions of visual and action conditioning, multi-frame encoding, and auxiliary code prediction. Together, these results demonstrate the effectiveness of tactile innovation as a supervision signal for contact-rich manipulation.

31. 【2611.26389】ViDecide: Teaching Video Diffusion Models to Reason and Plan Step by Step

链接:https://arxiv.org/abs/2611.26389

作者:Kaiwen Zhang,Yiming Gao,Zeqi Xiao,Xintao Wang,Pengfei Wan,Kun Gai,Xingang Pan

类目:Computer Vision and Pattern Recognition (cs.CV)

关键词:Current leading video, large-scale reasoning data, synthesize highly realistic, frequently violating physical, Current leading

备注: Project page: [this https URL](https://kevin-thu.github.io/ViDecide)

点击查看摘要

Abstract:Current leading video generation models can synthesize highly realistic videos, while still frequently violating physical and logical constraints in multi-step reasoning and planning, even after training with large-scale reasoning data or existing diffusion RL methods. We attribute these failures to the conventional video training paradigm, which treats a solution as a fixed-length clip rather than a chain of decisions. We present ViDecide, a decision-centric framework that trains video diffusion models to predict state transitions causally and decide when to stop, with a decision-focused training schedule. We further propose Corrective Fine-tuning (CFT), which corrects failed decisions by mapping their noise to the model's own successful continuations, achieving stable improvement without policy-gradient optimization. Across abstract games, planning tasks, and a new benchmark of multi-step robot planning, ViDecide achieves an average strict success rate of 83.8%, far above 23.3% for standard fine-tuning and 67.8% for the strongest prior method, and generalizes to larger puzzles and unseen layouts. Our framework therefore unlocks greater potential for video generation models as reliable reasoners and planners in the visual world. Project page: this https URL.

32. 【2611.26388】GateTrace: Where Do Crowd Crossings Go Missing Under Occlusion?

链接:https://arxiv.org/abs/2611.26388

作者:Ao He,Qianchen Yu,Vivian W. H. Wong

类目:Computer Vision and Pattern Recognition (cs.CV)

关键词:tracking supports entrance, pedestrian tracking supports, large events, passageway counts, tracking supports

备注: 23 pages, including supplementary material; 3 figures

点击查看摘要

Abstract:At large events, pedestrian tracking supports entrance and passageway counts, but totals hide which crossings were missed. In a stadium study, the system recorded about four in five clearly visible crossings but fewer than one in three mostly obscured crossings. The videos lacked trajectories annotated across frames, so we turned to MOT20, which provides pedestrian trajectories and visibility. We introduce GateTrace to locate missed crossings and assess changes to counting rules. It derives reference crossings from annotations under fixed rules, matches reports by person, direction, time and position, and traces misses through saved boxes and counting decisions. With a shared detector, ByteTrack, OC-SORT and BoT-SORT recover 38.9-46.7% of low-visibility reference crossings on development sequences, versus 98.1-99.0% at high visibility. The gap recurs on two further sequences under a protocol fixed in advance. Published outputs of MOT20-tuned BASE recover 63-78% at low visibility across both sequence groups. Losses stop at different stages across systems. Across four systems and both groups, 47-79% of low-visibility misses have no nearby output linked to the person or no possible crossing in the saved tracks. Removing later checks from BASE on development sequences leaves most misses unresolved and lowers precision against the fixed reference list from about 96% to 77%. Misses also remain among sampled reference crossings that two human annotators supported after checking the videos. GateTrace shows where crossings are lost and measures the recovery and precision costs of changing later checks. We will release the code and per-crossing records.

33. 【2611.26386】SEEC: A Large-Scale Dataset and Benchmark for Self-Supervised External-Eye Representation Learning

链接:https://arxiv.org/abs/2611.26386

作者:George R. Nahass,Michelle Fleishaker,Sanjay Ganesh,Sasha Hubschman,R. V. Paul Chan,Chad A. Purnell,Pete Setabutr,Ann Q. Tran,Sathya N. Ravi

类目:Computer Vision and Pattern Recognition (cs.CV)

关键词:low acquisition cost, large-scale standardized resource, standardized resource exists, photography captures clinically, captures clinically relevant

备注: 36 Pages, 10 Figures, 9 Tables

点击查看摘要

Abstract:External-eye photography captures clinically relevant periocular anatomy at low acquisition cost, yet no large-scale standardized resource exists for evaluating self-supervised external-eye representations. We introduce a 4.35-million-image Standardized External-Eye Corpus (SEEC) derived from fourteen heterogeneous public face datasets, together with a benchmark for external-eye representation learning. For evaluation, we provide held-out evaluation distributions, landmark-annotated anatomical probe sets, and clinical disease-classification experiments spanning ptosis, thyroid eye disease, and craniofacial syndromes. We evaluate representations at two complementary levels: (a) the representation level, by assessing embedding geometry across multiple data distributions, and (b) the functional level, by assessing anatomical recovery, out-of-distribution anatomical transfer, and disease classification. Our models include ResNet-101 and ViT-B encoders pretrained using InfoNCE, VICReg, and LeJEPA objectives and compared with DINOv2 and MAE. Our results indicate that representation-level and functional-level evaluations are dissociable: broader embedding-space utilization does not consistently correspond to anatomical transfer or disease-classification performance, and no single encoder is universally optimal. Domain-specific external-eye encoders outperform DINOv2 and MAE on cross-dataset anatomical transfer despite being pretrained at substantially smaller scale, while external-eye-pretrained ViT models match or outperform these general-purpose baselines on frozen disease-classification probes. We release benchmark code, evaluation scripts, and preprocessing pipelines to reproduce SEEC at this https URL

34. 【2611.26380】EgoCaptioner: Towards Comprehensive and Fine-Grained Egocentric Video Captioning

链接:https://arxiv.org/abs/2611.26380

作者:Yuqi Tang,Yang Shi,Yuran Wang,Tengfei Liu,Rundong Wang,Zongyang Qiu,Qixun Wang,Zhuoran Zhang,Xuanyu Zhu,Yifan Xu,Yiyan Ji,Shujie Li,Shixuan Liu,Haotian Wang,Yuanxing Zhang,Pengfei Wan,Chengwei Qin

类目:Computer Vision and Pattern Recognition (cs.CV)

关键词:captioning requires fine-grained, requires fine-grained detail, current multimodal large, multimodal large language, video captioning requires

备注: 17 pages

点击查看摘要

Abstract:Egocentric video captioning requires fine-grained detail and accurate timing, yet current multimodal large language models (MLLMs) often miss brief actions, confuse their order, or misinterpret the wearer's motion. Existing caption formats also offer limited support for combining detailed narration with precise timing. We present EgoCaptioner, which pairs a video-level overview of the setting, participants, and the wearer's overall activity with timed steps whose narration adapts to varying perceptual demands and marks key moments at specific timestamps. We develop a coarse-to-fine annotation pipeline that establishes event structure before expanding step descriptions, yielding 50,000 training examples from eight egocentric video sources. Complementary rewards assess factual accuracy and coverage, measure step-level temporal alignment, and penalize fragmented or redundant narration, guiding caption refinement through Group Relative Policy Optimization (GRPO). To improve the reliability of factual supervision, we supplement caption references with pose-derived motion facts when sensor data are available. Across five egocentric benchmarks, EgoCaptioner achieves leading performance among evaluated open-source models in caption quality, caption-conditioned question answering, and temporal alignment. It also surpasses Gemini 3.1 Pro on PhysCapBench and all evaluated proprietary baselines on Minerva-Ego under the caption-to-QA protocol. Ablations support the complementary roles of factual and temporal rewards and show that coarse-to-fine annotation provides more effective supervision than single-pass annotation.

35. 【2611.26370】Fast Self-Supervised Generative CT Denoising via Few-Step Sampling

链接:https://arxiv.org/abs/2611.26370

作者:Xingyi Yao,Zhengtao Ma,Zhennong Chen

类目:Computer Vision and Pattern Recognition (cs.CV)

关键词:obscures anatomical detail, thin-slice computed tomography, computed tomography, anatomical detail, suffer from elevated

备注: 30 pages, 4 figures, 2 tables. Xingyi Yao and Zhengtao Ma contributed equally to this work. Zhennong Chen is the corresponding author

点击查看摘要

Abstract:Low-dose and thin-slice computed tomography (CT) images suffer from elevated noise that obscures anatomical detail, while clean references for supervised denoising are often unavailable. Although Noise2Noise permits self-supervised training from noisy pairs, direct regression tends to oversmooth fine structure. Averaging multiple independently generated samples mitigates this blur, though repeating multi-step diffusion sampling is costly. We introduce Noise2Noise with Improved Mean Flow (N2N-iMF) to accelerate this process via few-step sampling. N2N-iMF learns to generate one noisy image conditioned on its paired counterpart; at inference, independent outputs are sampled and averaged to suppress noise. We evaluated image quality and sampling efficiency on simulated thin-slice brain and low-dose abdominal CT across four sampling methods and averaging counts from 1 to 20, and validated the approach on real-world photon-counting CT (PCCT) brain scans. With ten samples per conditioning input and three network evaluations per sample, N2N-iMF achieved a mean absolute error of 2.60 HU on brain CT and 14.02 HU on abdominal CT, outperforming 50-evaluation denoising diffusion implicit model (DDIM) sampling (2.91 and 14.53 HU). On the same GPU, per-output generation time dropped from 7.29 to 0.43 s. Across eight PCCT cases, N2N-iMF reached a mean gray-white matter contrast-to-noise ratio of 1.341, versus 0.916 for Noise2Noise and 1.176 for DDIM. N2N-iMF accelerates self-supervised generative CT denoising without distillation while keeping the number of averaged outputs adjustable at inference.

36. 【2611.26367】Contactless Estimation of Respiratory Frequency Using 3D-CNN on Thermal Images

链接:https://arxiv.org/abs/2611.26367

作者:Federica Gioia,Filippo Pura,Alberto Greco,Dario Piga,Arcangelo Merla,Marco Forgione

类目:Computer Vision and Pattern Recognition (cs.CV)

关键词:Monitoring physiological parameters, respiratory rate, pathological conditions, physiological parameters, essential for diagnosing

备注:

点击查看摘要

Abstract:Monitoring physiological parameters such as respiratory rate ($f_R$) is essential for diagnosing and managing various pathological conditions. Thermal imaging offers a promising contactless alternative to traditional methods, which often rely on partially invasive sensors or obtrusive wearable systems. However, existing approaches for $f_R$ estimation from thermal signals typically require extensive pre-processing and manual or semi-automatic region-of-interest (ROI) tracking, limiting their practical applicability. This study proposes a deep learning-based method for estimating $f_R$ directly from thermal videos, eliminating the need for complex pre-processing and ROI tracking. A 3D Convolutional Neural Network (3D-CNN) is developed to operate on raw thermal video data. To address challenges related to small datasets, the model is trained using data augmentation and transfer learning from synthetic datasets. Experimental results demonstrate that the proposed approach achieves a validation $R^2$ score of approximately 0.61 on both pre-processed and raw thermal videos. By simplifying the workflow, this method holds promise for enhancing the feasibility of thermal imaging in real-world applications, such as remote healthcare and driver monitoring in automotive applications.

37. 【2611.26347】OffBeat: Multicoset Sampling for 4D Capture Beyond the Camera Frame Rate

链接:https://arxiv.org/abs/2611.26347

作者:Chi-Wei Lee,An-Kai Chen,Cheng-Han Chiang,Yi-Chun Chang,Po-Chih Kuo

类目:Computer Vision and Pattern Recognition (cs.CV); Image and Video Processing (eess.IV)

关键词:PSNR, dynamic-region PSNR, Spacetime Gaussians, frame, synchronized

备注: Project page: [this https URL](https://arthur031221.github.io/offbeat/)

点击查看摘要

Abstract:For 4D reconstruction, a synchronized multi-camera rig records all views at the same timestamps and misses the motion between them. Staggered triggers spread the same frames over more timestamps. However, when the cameras along each row fire one after another, as in 4DSloMo's schedule on the N3DV rig, the row acts as one moving camera that confounds depth with motion. We introduce OffBeat, which models a staggered rig as a spatially distributed multicoset sampler: shifted copies of one sampling grid. From the calibration alone, OffBeat derives camera groups that fire together and their firing order. Since one group records each timestamp, a short-lived primitive can fit an error its cameras share, such as a color offset, and flicker in novel views. We show that a primitive's opacity over time weights all groups equally, for every number of groups, only if it lasts at least one camera frame interval. On Spacetime Gaussians, a minimum lifetime approximates this bound, and a closed-form smoothed box of that length satisfies it exactly, with less flicker than a Gaussian in its place at nearly equal PSNR. In simulated staggered capture on N3DV, OffBeat outperforms the full 4DSloMo pipeline with video diffusion on every test frame, by 3.04 dB in mean dynamic-region PSNR. Between the first and last synchronized frame, it outperforms the best synchronized method of each scene, with or without frame interpolation, by 1.98 dB in mean dynamic-region PSNR, while synchronized capture is better at its own frames. On PanopticSports, in this frame range, OffBeat is comparable in PSNR to synchronized capture with interpolation. On every N3DV scene, replacing 4DSloMo's schedule with ours raises the dynamic-region PSNR of Spacetime Gaussians, 4DSloMo without diffusion, and our reconstruction. Multi-camera rigs could follow faster motion at the same frame count by changing only when each camera fires.

38. 【2611.26334】Online Temporal Rotations for 2 Bit KV-Cache Quantization in Video Diffusion Transformers

链接:https://arxiv.org/abs/2611.26334

作者:Ryan Lucas,Petros Prastakos,Rahul Mazumder

类目:Computer Vision and Pattern Recognition (cs.CV); Artificial Intelligence (cs.AI)

关键词:transformers reuse key, diffusion transformers reuse, long video generation, Video diffusion transformers, reuse key

备注:

点击查看摘要

Abstract:Video diffusion transformers reuse key--value (KV) caches across denoising steps, which quickly becomes a bottleneck for long video generation. We find that quantizing temporally distant cache vectors leads to a large range and high quantization error, but that nearby video tokens in the KV-cache share a persistent pattern per channel. These patterns also vary across videos, motivating rotations that are adapted to each video's unique cache. We propose Temporal Rotation Quantization (TRQ), a method that rotates frames in the cache that are adjacent in time before quantization. For each block of temporally adjacent cache vectors, TRQ fits an orthogonal transform that maps the block's per-channel means to a vector that is close to a constant across channels. To keep the quantization format lightweight, we parameterize this rotation as a tree of $2 \times 2$ planar rotations, each stored with only two bits, followed by a fixed Hadamard transform shared across all blocks. The rotations are fit without gradients or changes to model weights, and we prove that a simple greedy fit is globally optimal for our 'mean-flattening' objective. The transforms are fit online for each new video, and we release fused Triton kernels to make this process efficient at inference. At 2-bit precision and matched compression, TRQ improves VBench subject consistency over the state-of-the-art method QVG from 90.38 to 92.02 on LongCat-Video-13B and from 86.33 to 87.61 on HY-WorldPlay-8B, as well as improving other metrics.

39. 【2611.26321】FirstCross: Convex Single-Photon 3D Reconstruction from Sparse Views

链接:https://arxiv.org/abs/2611.26321

作者:Chi-Wei Lee,Cheng-Han Chiang,Yi-Chun Chang,An-Kai Chen,Po-Chih Kuo

类目:Computer Vision and Pattern Recognition (cs.CV); Image and Video Processing (eess.IV)

关键词:Single-photon lidar records, lidar records, records the arrival, photons, Single-photon lidar

备注: Project page: [this https URL](https://arthur031221.github.io/firstcross/)

点击查看摘要

Abstract:Single-photon lidar records the arrival time of each returning photon, enabling 3D reconstruction from few views and scarce photons. However, leading neural methods such as Transientangelo require over an hour of non-convex per-scene optimization, and their accuracy degrades as photon flux decreases. We propose FirstCross, a network-free approach that instead solves a short sequence of convex problems on a voxel grid. For a solid object, the Poisson likelihood of a ray with given signal and background rates depends on the scene solely through its first crossing, the point where the ray first enters the object. Decomposing this likelihood yields two tractable terms. The first penalizes surfaces in front of detected photons and admits a tight convex closure, namely volume rendering with a running-maximum transmittance in place of NeRF's product transmittance. The second accounts for photons that no surface explains and admits a convex upper bound once the object is assumed to be thick behind each detected surface point. Compared with Transientangelo under its own real-data protocol, FirstCross achieves lower depth error across every test scene, view count, and flux level. With three views and 100 times fewer photons than at the protocol's lowest flux, our mean object depth error stays under half of Transientangelo's with all photons. Given our detected surface points, the best classical fusion has 1.3 to 1.8 times our three-view error across flux levels. On real room-scale captures with 1% of their photons, TransientNeRF has 2.8 to 7.7 times our error. FirstCross lets few-view single-photon scanners cut exposure to 1% and reconstruct an object in about a minute on one GPU.

40. 【2611.26318】ResVision: Spectrum-Matched Resizing and Context-Preserving Inspection for Visual Reasoning

链接:https://arxiv.org/abs/2611.26318

作者:Guojian Zhan,Jiahao Xu,Yahui Liu,Yao Hu,Chuan Mu

类目:Computer Vision and Pattern Recognition (cs.CV)

关键词:observations stay grounded, structure remains accessible, selectively enlarging evidence, fine structure remains, improves vision-language models

备注:

点击查看摘要

Abstract:Agentic visual reasoning improves vision-language models by selectively enlarging evidence without adding new scene content, but the benefit depends on whether fine structure remains accessible and whether local observations stay grounded in the full scene. We identify three recurring bottlenecks: high-frequency detail can be attenuated during visual-token formation; spatially distributed evidence can remain encoded yet difficult for the language model to retrieve; and isolated crops can weaken whole-detail correspondence through scale and positional mismatch. ResVision addresses these limitations with two complementary components. Spectrum-Matched Resizing (SMR) estimates image complexity and selects an adaptive global sampling scale, avoiding unnecessary enlargement on simple inputs while allocating more visual tokens to detail-rich images. Context-Preserving Inspection (CPI) returns each inspected crop together with the complete working view, preserving local detail and global context across tool turns. We evaluate Direct and Agentic variants on four frontier VLMs and 29 benchmarks spanning six visual-reasoning families. Complete ResVision (Agentic Base plus SMR plus CPI) is best for every model, improving by 3.36 to 4.94 points over Direct Base and 1.87 to 2.94 points over Agentic Base. These results show that adaptive visual sampling and context-preserving inspection address distinct, complementary constraints in multimodal reasoning.

41. 【2611.26316】C-LiTE: A Lightweight Foundation Model for Cellular feature learning in Histopathology

链接:https://arxiv.org/abs/2611.26316

作者:Esha Sadia Nasir,Jiaqi Lv,Gozde N. Gunesli,Shan E Ahmed Raza

类目:Computer Vision and Pattern Recognition (cs.CV); Artificial Intelligence (cs.AI)

关键词:Characterising individual cells, morphology informs tasks, Characterising individual, cellular morphology informs, tumour grading

备注:

点击查看摘要

Abstract:Characterising individual cells is fundamental to computational pathology, as cellular morphology informs tasks such as tumour grading, mitotic assessment and immune profiling. However, most pathology foundation models are trained on tissue patches and capture cellular features only implicitly, while existing cell-level models are either restricted to specific cell populations or rely on computationally expensive architectures with hundreds of millions of parameters. Applying ViTs directly to small cellular crops also presents an architectural challenge: upsampling increases computational cost without adding morphological information and may introduce interpolation artefacts. To address these limitations, we introduce C-LiTE, a lightweight foundation model for cellular representation learning in HE-stained histopathology, with only 22 million parameters. C-LiTE employs a native-resolution tokenisation strategy tailored to cellular inputs, processing 64 x 64 px cell-centred crops using a 4 x 4 patch embedding to produce a 16 x 16 token grid without image upsampling. To transfer visual knowledge from large-scale pre-training, we adapt the patch embedding weights of a pre-trained ViT-5-S using a pseudoinverse-based transformation, enabling weight initialisation at the finer patch resolution. We further assemble a pre-training dataset of 60 million cell-centred crops from TCGA, balanced across 19 cancer types and 16 tissue types. To our knowledge, this represents the largest public dataset for self-supervised pre-training on HE images at the cellular level. We evaluate C-LiTE on seven public datasets spanning mitotic figure recognition, nucleus and cell-type classification, and immune cell phenotyping. Under frozen-feature evaluation, C-LiTE achieves a mean macro-F1 of 0.670, approaching CytoFormer (0.682) with approximately 30x fewer parameters.

42. 【2611.26296】SKILL.PNG: Learning Agent Skills in Pixel Space

链接:https://arxiv.org/abs/2611.26296

作者:Maximilian Seeth,Chengzhi Martin Hu,Marie Kempkes,Paul Röttger,Leo Schwinn

类目:Machine Learning (cs.LG); Computer Vision and Pattern Recognition (cs.CV)

关键词:natural language instructions, http URL, natural language, commonly specialized, tasks and domains

备注:

点击查看摘要

Abstract:AI agents are commonly specialized for new tasks and domains using skills, i.e. natural language instructions placed in their context. However, the discrete nature of textual skills prevents continuous optimization. Gradient-based approaches such as finetuning, on the other hand, are typically model-specific, preventing skill transfer across different agents. Since many agents today are built on vision-language models (VLMs), we can address these limitations by introducing this http URL: a method for obtaining skill-like image inputs whose pixels are optimized with gradient descent through the vision encoder of a frozen VLM. Image skills are compatible with any VLM, and a single image can be optimized jointly for an ensemble of models. We show that this http URL significantly outperforms state-of-the-art textual skills optimized using frontier LLMs in 9 of 16 settings (across 4 VLMs and 4 benchmarks) while requiring only 64 to 256 image tokens instead of roughly 900 to 2000 text tokens at inference time. Hybrid skills, i.e. text and image skills jointly optimized, further improve agent performance. Moreover, an image optimized with an ensemble of VLMs can successfully transfer across the models in the ensemble. this http URL thus presents the first agent skill that can be optimized with gradient descent while retaining transfer benefits of text skills. Our code is available at: this https URL

43. 【2611.26294】BADAS-3.0: Real-Time Driving Perception and Planning from One Camera

链接:https://arxiv.org/abs/2611.26294

作者:Roni Goldshmidt,Hamish Scott,Lorenzo Niccolini

类目:Computer Vision and Pattern Recognition (cs.CV); Computation and Language (cs.CL)

关键词:real-time latency, current speed, Abstract, camera, latency

备注:

点击查看摘要

Abstract:We ask how far a driving model can get with one camera and real-time latency. BADAS-3.0 is a 1B-parameter video encoder with four small heads. From two seconds of one forward camera, it predicts the current speed, the metric path just driven, a collision-risk score and six candidate futures over 6.4 seconds, in under 50 ms. Given the car's current speed, BADAS-3.0 has 44% lower minADE$_6$ over five trajectory benchmarks than the 34B Alpamayo 2 Super, which reads six cameras and runs over 100x slower. The gain comes from camera rigs that neither model was trained on. On the rig they share, the two are on par. Existing benchmarks are mostly nominal driving, so we add two open-loop safety benchmarks, one of real dashcam collisions and near-misses and one of Euro NCAP scenarios. On both, BADAS-3.0 has a higher pass rate than every published planner we tested.

44. 【2611.26269】racking Objects Through Assembly and Decomposition

链接:https://arxiv.org/abs/2611.26269

作者:Cheryl Lam,Yihong Sun,Bharath Hariharan

类目:Computer Vision and Pattern Recognition (cs.CV)

关键词:understanding real-world dynamics, real-world dynamics, essential for understanding, understanding real-world, traditional object tracking

备注:

点击查看摘要

Abstract:Tracking objects through transformations is essential for understanding real-world dynamics. Recent work has extended traditional object tracking to detect state changes. However, existing methods require an initial object prompt and detect only its decompositions, missing assembly events, which begin with multiple objects rather than one. We therefore introduce a prompt-free formulation of this task, where the goal is to recover all object tracks and detect both decomposition and assembly events given only a video. Our key insight is that assembly is decomposition in reverse. Building on this insight, we present Atlas, a zero-shot system that constructs and jointly reasons over forward and reverse spatiotemporal partitions to recover object tracks and their transformations. We further introduce OmniST, a benchmark with dense annotations of objects and both transformation types. On OmniST, Atlas recovers more than twice as many state changes as prior methods while remaining competitive on existing benchmarks that only evaluate decomposition.

45. 【2611.26262】Digitization of Analogue Gauges using Computer Vision

链接:https://arxiv.org/abs/2611.26262

作者:Erico Kayumba Mutonkole,Olasupo Ajayi,Omowunmi Isafiade

类目:Computer Vision and Pattern Recognition (cs.CV); Artificial Intelligence (cs.AI)

关键词:measuring physical quantities, Analogue gauges remain, operators introduces human, introduces human error, gauges remain prevalent

备注: 9 pages, 12 figures. Presented at SATNAC 2025

点击查看摘要

Abstract:Analogue gauges remain prevalent in manufacturing industries for measuring physical quantities, yet manual monitoring by operators introduces human error and operational inefficiency. This research developed and evaluated an automated gauge reading system using computer vision and deep learning techniques. Three hybrid approaches were implemented and compared: OpenCV with Scikit-Image (traditional computer vision), OpenCV with ResNet18, and OpenCV with VGG19 (deep learning approaches). A comprehensive dataset of 1,848 pressure gauge images (0-10 BAR range) was curated through web scraping, synthetic generation, and augmentation techniques. The experimental results demonstrate that ResNet18 significantly outperforms the alternatives, achieving an accuracy of 97.3%, compared to VGG19's 64.4% and Scikit-Image's 63.8%. The study validates the effectiveness of residual learning architectures for industrial gauge digitization, providing a robust foundation for Industry 5.0 automation applications.

46. 【2611.26254】Prism: Hierarchical Text Detection with Minimalist Design

链接:https://arxiv.org/abs/2611.26254

作者:Yuchen Su,Xuesheng Wang,Zhilong Ji,Jinfeng Bai,Shijian Lu,Zhineng Chen

类目:Computer Vision and Pattern Recognition (cs.CV)

关键词:aims to localize, Hierarchical text, Hierarchical text detection, text detection aims, text

备注:

点击查看摘要

Abstract:Hierarchical text detection aims to localize words, text lines, and paragraphs while parsing their structural relationships. It provides essential structural cues for visual text understanding. To maximize accuracy, existing methods typically rely on complex multi-scale feature fusion necks and heavy-weight heads, resulting in bloated architectures and low inference efficiency. In this paper, we argue that a sufficiently pre-trained backbone can effectively replace these sophisticated external modules, while simple geometric inclusion suffices for hierarchical text structural parsing. Thus, we propose Prism, a minimalist framework for hierarchical text detection. Prism performs multi-granularity detection by attaching parallel lightweight heads to a single-scale backbone feature map, covering words, text lines, and paragraphs, and reconstructs the structural hierarchy via an efficient geometric inclusion algorithm. In addition, we systematically compare representative lightweight head archetypes and show that pixel-level regression offers the best accuracy-efficiency trade-off in this setting. Experiments on multiple public benchmarks show that our significantly simplified architecture achieves state-of-the-art (SOTA) performance with remarkable efficiency. Specifically, it runs $1.7\times$ faster than LRANet on CTW1500 while achieving higher accuracy, and surpasses the current SOTA method, OmniParser V2, by over 6 PQ points on the hierarchical HierText dataset. Code will be released soon.

47. 【2611.26249】Beyond Covers and Titles: Evidence-First Ranking for Robust Cold-Start Recommendation

链接:https://arxiv.org/abs/2611.26249

作者:Fuyi Wang,Yuxin Cao,Yifan Liao,Renyang Liu,Lin Li,Wei Song

类目:Computer Vision and Pattern Recognition (cs.CV)

关键词:Short-video platforms rank, Short-video platforms, title and frames, Short-video, video cover

备注:

点击查看摘要

Abstract:Short-video platforms rank new uploads before anyone has watched them, using content towers that read a video's cover, title and frames. Existing attacks on content-aware recommenders perturb pixels or features and ignore platform re-encoding, and existing defenses train against norm-bounded perturbations that neither match real edits nor preserve recommendation quality. We show with REAP, a realizable attack through {a simulated platform channel (H.264 transcoding, rescaling, cropping and JPEG)}, that packaging edits raise a cold video by 32.0 rank-percentile points in unrelated audiences. We then observe that cold-start ranking does not need the packaging at all: a tower that sees only the video body ranks cold items best. Building on this, we propose Evidence-First Ranking (EFR), which scores cold videos from their body, admits packaging only through a support-gated residual bounded by $\gamma$, and merges cold and warm rankings by rank percentile, yielding an exact certificate against any packaging edit. On MicroLens-100K, EFR cuts the attack gain by 81% and raises cold-start Recall@100 from 0.1133 to 0.170, while keeping the merged NDCG@10 within 4% of the undefended system.

48. 【2611.26234】Attention Drift Regularization with Adaptive Initialization for Domain-Robust Visual Recognition

链接:https://arxiv.org/abs/2611.26234

作者:Shivangi Rai,Vinod K Kurmi

类目:Computer Vision and Pattern Recognition (cs.CV)

关键词:Source-free domain adaptation, modes remain unaddressed, critical failure modes, failure modes remain, source model

备注: Accepted in WACV 2027

点击查看摘要

Abstract:Source-free domain adaptation (SFDA) adapts a pretrained source model to unlabeled target domains without source data access, yet two critical failure modes remain unaddressed. First, attention drift: source-learned attention maps shift toward spurious or background features under distribution shift, destabilizing feature representations. Second, domain-gap-agnostic initialization: applying a single fixed source model uniformly across all target domains ignores domain gap severity, causing premature overfitting to noisy pseudo-labels under large shifts. We propose DGA-CAFC to address both failures. The Counterfactual Attention-based Feature Consistency (CAFC) module enforces feature stability by minimizing the discrepancy between representations computed under learned attention and those under counterfactual-inspired normalized perturbations, thereby promoting domain-robust features independent of spurious attention shifts. The Domain-Gap Aware Source Model Selection (DGASS) module estimates domain gap severity via target prediction entropy and selects the source model variant whose confidence level best matches the estimated gap, reducing negative transfer and stabilizing pseudo-label quality. Crucially, CAFC transfers beyond SFDA: applied as a lightweight regularizer to standard domain generalization objectives without target data. Extensive experiments across SFDA and DG benchmarks demonstrate that our method consistently improves over recent baselines, demonstrating the effectiveness and generality of the proposed framework. Project page is available at this https URL

49. 【2611.26218】MixReact: Physics-Based Reactive Humanoid Control from Multimodal Cues via a Discrete-Continuous Mixture of Transformers

链接:https://arxiv.org/abs/2611.26218

作者:Bin Li,Xing Zhu,Jingya Wang

类目:Computer Vision and Pattern Recognition (cs.CV)

关键词:History-aware Action Transformer, agents in everyday, everyday environments, environments are expected, expected to react

备注:

点击查看摘要

Abstract:Humanoid agents in everyday environments are expected to react to diverse cues as humans do, with semantically appropriate and physically executable behaviors. Existing cue-driven reaction generation methods support only a narrow range of cue modalities and are limited to purely kinematic generation, leaving the resulting reactions susceptible to physical artifacts. To this end, we present MixReact, the first unified framework for physics-based reactive humanoid control from text, audio, and video cues, integrating a Discrete--Continuous Mixture of Transformers (DisCo-MoT) for state prediction with a History-aware Action Transformer (HAT) for physical execution. DisCo-MoT couples a discrete masked-modeling branch with a continuous autoregressive diffusion branch via one-way mixture attention within a MoT architecture. Guidance from discrete context mitigates collapse into averaged behaviors in one-to-many reaction generation, while direct prediction in continuous state space bypasses the decoding bottleneck of discrete representations. HAT then maps the predicted next state and observed execution history to executable actions, enabling robust closed-loop control in physics-based simulation. Experiments across all three modalities demonstrate consistent improvements over the baselines in fidelity, diversity, and physical consistency, while ablations validate the complementary contributions of the components. Our code will be released to facilitate future research. Project page: this https URL

50. 【2611.26211】SpecWM: Turning Verification into Refinement for Streaming World Models

链接:https://arxiv.org/abs/2611.26211

作者:Jiaming Li,Lingzhen Li,Xiaze Yang,Chenyu Zhu,Kaiqi Liu,Xiaotong Zhao,Songlian Li,Toyota Li,Alan Zhao,Haobin Zhong,Yue Ma,Zhiyuan Ma

类目:Computer Vision and Pattern Recognition (cs.CV)

关键词:advanced interactive generation, large models offering, models offering higher, smaller models enabling, models enabling faster

备注:

点击查看摘要

Abstract:Recently, video world models have advanced interactive generation, with large models offering higher quality and smaller models enabling faster inference. Existing speculative methods combine their strengths through fast draft generation and selective target regeneration guided by rewards on decoded frames. However, verification covers only the external scorer's evaluation dimensions and adds decoding and scoring costs. Moreover, these scores provide no correction, requiring costly target regeneration for rejected drafts. To address these limitations, we propose Speculative World Model (SpecWM), a unified, training-free inference framework for streaming world models. It verifies draft chunks using the target's latent velocity field. Verification also yields a reusable correction; two thresholds select Accept, Refine, or Reject. Specifically, one-step refinement reuses it to improve perceptual quality, while three-step refinement strengthens camera following and trajectory consistency. These findings motivate transition-aware refinement. On WBench Navigation, SpecWM improves visual quality at 1.63$\times$ speedup over the target and raises NavScore from the draft's 75.99 to 85.63, achieving the best quality-speed-control trade-off. Additionally, our LongSpecBench shows sustained quality and control over long, multi-turn rollouts, while experiments on Hy-Nomas validate SpecWM across backbones and denoising budgets.

51. 【2611.26201】Learn, Freeze, and Transfer an Activation Function Across Architectures

链接:https://arxiv.org/abs/2611.26201

作者:Arun Prakash A

类目:Machine Learning (cs.LG); Computer Vision and Pattern Recognition (cs.CV)

关键词:typically reuses knowledge, reuses knowledge acquired, trained from scratch, learning typically reuses, Transfer learning typically

备注: 17 pages, 5 figures, 10 tables. Code: [this https URL](https://github.com/AI4Bharat/LeFT-activation-transfer) , Interactive figures: [this https URL](https://arunprakash-a.github.io/papers/learn-freeze-transfer/)

点击查看摘要

Abstract:Transfer learning typically reuses knowledge acquired by a source network through its learned weights. We investigate whether a learned activation function can also be transferred and remain useful in networks trained from scratch. We study this complementary form of transfer through a learn-freeze-transfer protocol, in which the activation is optimized jointly with a source network in a single training run, and is then frozen and reused in target networks trained from scratch. A pointwise activation with globally shared parameters is independent of layer dimensions and can therefore be incorporated into networks of different architectures trained from scratch. We instantiate this protocol with FAct, a five-parameter activation represented by a two-harmonic truncated Fourier series, initialized from a Fourier approximation of GELU and learned in a Vision Transformer (ViT) on ImageNet-1K. When reused in a ViT of the same architecture trained from scratch on ImageNet-1K, the frozen activation (Frozen FAct) improves top-1 accuracy by 1.46 percentage points over GELU. Across four smaller target architectures (standard ViT, Swin Transformer, linear-attention ViT, and MLP-Mixer) and four image-classification datasets, it achieves the highest mean accuracy among eleven activation functions in 15 of 16 settings under a common training configuration, with improvements of up to 1.71 percentage points over the best conventional activation. When the learning rate and weight decay are tuned separately for each activation under the same search budget, Frozen FAct still ranks first in 11 of the 16 settings, second in four, and fourth in one. These results provide empirical evidence that a learned activation function can retain its usefulness when transferred across vision architectures, independently of the weights with which it was learned.

52. 【2611.26199】Learning to Think with Images Faithfully via Adaptive Data Selection

链接:https://arxiv.org/abs/2611.26199

作者:Yu Zhao,Jiaxin Zhang,Rohit Saxena,Fazl Barez,Pasquale Minervini,Yiwen Guo

类目:Computer Vision and Pattern Recognition (cs.CV); Computation and Language (cs.CL)

关键词:overcome perceptual bottlenecks, empowers vision-language models, Thinking with images, empowers vision-language, promising both higher

备注:

点击查看摘要

Abstract:Thinking with images (TwI) empowers vision-language models (VLMs) to overcome perceptual bottlenecks by actively cropping and zooming into fine-grained visual details, promising both higher accuracy and interpretable, region-grounded reasoning. However, our analysis shows that the improved accuracy of TwI models does not guarantee faithful tool use; they frequently invoke tools without actually relying on the zoomed region, undermining the reliability of tool-assisted reasoning. We trace this unfaithful behaviour to the training data: when questions are answerable without zooming, the model can answer correctly while ignoring the returned evidence, and this problem is exacerbated during training as the model becomes better at answering without tools. To keep zooming useful throughout training, we adapt the training data to the model's evolving capability and train on questions where correct zoomed evidence can still help the model answer correctly. This selection is supported by a data synthesis pipeline that localises small objects and constructs verifiable questions about their fine details, and complemented by an adaptiveness reward that favours tool-free answers when zooming is unnecessary. The experimental results show that our method substantially improves reasoning faithfulness and achieves state-of-the-art performance on visual search benchmarks (+11.6% on V*, +7.9% on HR-Bench 4K, and +9.4% on HR-Bench 8K). On multimodal hallucination benchmarks, our model also outperforms the compared TwI baselines. These results suggest that improving TwI faithfulness does not sacrifice accuracy and that both can be learnt from the right training data without directly supervising the reasoning process.

53. 【2611.26195】A version of the Cloude/van Zyl decomposition for full/quad pol synthetic aperture radar data

链接:https://arxiv.org/abs/2611.26195

作者:Allan Aasbjerg Nielsen,Paul Jacques Connetable,Henning Skriver,Knut Conradsen

类目:Computer Vision and Pattern Recognition (cs.CV)

关键词:symmetric polarimetric SAR, polarimetric SAR data, van Zyl decomposition, reflection symmetric polarimetric, van Zyl

备注: 11 pages, 5 figures

点击查看摘要

Abstract:Based on the Cloude/van Zyl decomposition of reflection symmetric polarimetric SAR data we present a version for full/quad polarisation data. We demonstrate both versions on X-band F-SAR data over Vejers, Denmark. Differences between the original and our version of the decomposition occur especially for very bright man-made targets. These differences come with lower entropy for our version giving an improved focus on a more pronounced dominant scattering mechanism for these objects. Interpretation of the scattering mechanism related to the full/quad pol version of the decomposition is facilitated by means of the Cloude/Pottier $\bar{\alpha}$ angle.

54. 【2611.26191】Backbone Matters: Benchmarking Encoder-Decoder Networks for Semantic Segmentation of Driving Scenes on BDD100K

链接:https://arxiv.org/abs/2611.26191

作者:Ehsan Rassekh

类目:Computer Vision and Pattern Recognition (cs.CV); Machine Learning (cs.LG)

关键词:Semantic segmentation, vehicle a dense, pixel-level description, scene understanding, autonomous vehicle

备注: 6 pages, 3 figures, 4 tables

点击查看摘要

Abstract:Semantic segmentation gives an autonomous vehicle a dense, pixel-level description of its surroundings and is a core part of scene understanding. Most segmentation networks consist of an encoder (the backbone) and a decoder, yet the backbone is often chosen without a systematic comparison. This work studies this choice on the BDD100K driving dataset. Four widely used decoders (U-Net, FPN, LinkNet and PSPNet) are combined with 13 backbones from six families (ResNet, VGG, DenseNet, Inception, MobileNet and EfficientNet), and all 52 models are trained with one common protocol. The best model, an FPN decoder with an EfficientNet-B4 backbone, reaches 62.6% mean intersection over union (mIoU), 89.8% accuracy and a loss of 0.092. Only two of the 52 models, both FPN with EfficientNet, exceed the 56.9% mIoU reported with the BDD100K benchmark. The effect of the backbone depends strongly on the decoder: the same EfficientNet-B4 gives 42.6% mIoU with PSPNet but 62.6% with FPN, and the gap between the best and the worst backbone ranges from 12.0 points (U-Net) to 42.5 points (FPN). The results and the scope of this study are discussed.

55. 【2611.26187】Invertible Transport Between Sample-Only Distributions via Projected Characteristic Functions

链接:https://arxiv.org/abs/2611.26187

作者:Juan Gutiérrez,Victor Gutiérrez-García,José Luis Blanco-Murillo

类目:Machine Learning (cs.LG); Computer Vision and Pattern Recognition (cs.CV)

关键词:PCF, latent distributions, tractable density, MMD, kernel MMD

备注:

点击查看摘要

Abstract:Many useful latent distributions can be sampled but do not have a tractable density. Such distributions may be represented by empirical collections of real-world observations or generated by scientific simulators and black-box generative models. These sample-only priors cannot be used directly in likelihood-based normalizing flows, which require evaluating the prior density. We propose Projected Characteristic-Function matching (PCF), a density-free framework for learning an invertible map between a data distribution and a prior specified only through samples. Our method aligns distributions using projected empirical characteristic functions, eliminating the need for density estimation, adversarial training, or paired observations. To address the non-uniqueness of the transport map, we introduce an optional transport regularizer that selects lower-displacement transformations while preserving endpoint alignment. We benchmark PCF on controlled transports, sample-efficiency studies, and high-dimensional alignment tasks. PCF matches the accuracy of kernel MMD on the controlled benchmark and on raw-pixel MNIST--USPS at a cost linear in the batch size, and it is more sample-efficient, achieving lower endpoint error than MMD in 13 of 15 paired runs across library sizes. By avoiding pairwise kernel evaluation, it reduces discrepancy memory by 29.8% and trains at batch sizes where kernel MMD exceeds GPU memory. These results show that PCF provides accurate and efficient invertible transport, particularly in sample- and memory-limited settings.

Subjects:

Machine Learning (cs.LG); Computer Vision and Pattern Recognition (cs.CV)

Cite as:
arXiv:2611.26187 [cs.LG]

(or
arXiv:2611.26187v1 [cs.LG] for this version)

https://doi.org/10.48550/arXiv.2611.26187

Focus to learn more

              arXiv-issued DOI via DataCite (pending registration)</p>
56. 【2611.26179】RegionAgent: Two-Stage Preference Optimization for Regional Image Restoration

链接:https://arxiv.org/abs/2611.26179

作者:Weiyu Zhou,Jiacong Tang,Tao Hu,Axi Niu,Jinqiu Sun,Peng Wu,Wei Dong,Qingsen Yan

类目:Computer Vision and Pattern Recognition (cs.CV); Artificial Intelligence (cs.AI)

关键词:non-uniform coupled degradations, diverse restoration capabilities, restoration, non-uniform coupled, coupled degradations require

备注:

点击查看摘要

Abstract:Images under non-uniform coupled degradations require diverse restoration capabilities for different degraded regions. Existing methods employ globally uniform operations, potentially overlooking regional differences and causing incomplete restoration or even damage to image content. Moreover, a degraded region may require successive restoration actions, and earlier actions may alter residual degradation characteristics or introduce artifacts, potentially hindering subsequent restoration. To address these challenges, we propose RegionAgent, an adaptive restoration framework that trains a Vision-Language Model (VLM) through two-stage preference learning to generate a region-specific restoration action at each iteration. Specifically, we first train the VLM to select an action suited to the current region through step-level Direct Preference Optimization (DPO), using preferences derived from immediate restoration feedback to supervise only the differing component in each action pair. To account for the effects of current actions on subsequent restoration, we then apply trajectory-level DPO, using preferences derived from complete trajectory scores to supervise the sole differing action in each trajectory pair. Furthermore, as conventional image-pair datasets do not directly provide preferences over regional restoration decisions, we construct a regional restoration preference dataset comprising 10K action pairs and 20K trajectory pairs for two-stage VLM training. Experiments across multiple datasets demonstrate strong restoration performance under both mixed and single degradations. Ablation studies further validate the effectiveness of two-stage preference learning in improving restoration quality.

57. 【2611.26178】FlexVSR: Continuous-Scale Super-Resolution for Streaming Video Generation

链接:https://arxiv.org/abs/2611.26178

作者:Ji Wang,Haonan An,Jiasheng Zhang,Renlong Chen

类目:Computer Vision and Pattern Recognition (cs.CV)

关键词:gained rapid momentum, live content creation, interactive media, generation has gained, gained rapid

备注: 23 pages, 5 figures

点击查看摘要

Abstract:Real-time streaming video generation has gained rapid momentum in live content creation and interactive media. Deploying such systems in practice requires flexible scaling factors to accommodate diverse source resolutions, target displays, and computational budgets. However, extending fixed-scale generative video super-resolution (VSR) to varying target scales poses two challenges: downsampling from a larger fixed scale incurs redundant cost, while re-optimizing the diffusion transformer (DiT) is costly and alters the shared generative prior. To address both, we propose FlexVSR, a decoupled framework for continuous-scale streaming generative VSR with supervision at only a few scales. Our key insight is that naively resampling fixed-scale latents onto a new target grid induces scale-dependent input mismatch, which can be effectively addressed through parameter-efficient, scale-aware adaptation of the latent interface without updating the DiT. To this end, FlexVSR introduces ReLSA (Refined Latent Scale Adaptation), a lightweight latent adapter that constructs and refines target-grid latents before they enter the DiT, enabling a shared frozen VSR prior to generalize across continuous scales with only 0.274M trainable parameters and an adaptation cost of 7.3 H100 GPU-hours, while introducing no additional temporal dependencies and preserving the native causal streaming protocol. Extensive experiments demonstrate competitive or superior reconstruction and temporal quality, lower latency than most native-scale streaming baselines and offline methods, and effective generalization to unseen continuous scales, filling the continuous-scale gap in real-time streaming generative VSR.

58. 【2611.26177】SpatialScout-V2: Can VLM Agents Actively Explore Real-World 3D Scenes and Localize Objects?

链接:https://arxiv.org/abs/2611.26177

作者:Zhaoxi Zhang,Hao Liang,Linxuan Wu,Yiming Xu,Yifu Yuan,Kun Liang,Chenming Tang,Meiyi Qiang,Wentao Zhang,Weikang Li,Yunfang Wu

类目:Computer Vision and Pattern Recognition (cs.CV)

关键词:Spatial intelligence requires, coherent three-dimensional representations, acquire visual evidence, Spatial intelligence, intelligence requires agents

备注:

点击查看摘要

Abstract:Spatial intelligence requires agents to acquire visual evidence and integrate partial observations into coherent three-dimensional representations. We introduce SpatialScout-V2, which extends SpatialScout's active mapping task from simulation to real residential scenes through three advances: real visual observations, preservation of original scene content, and interactive exploration of recorded views. Built from 100 homes in Realsee3D, the benchmark covers 3,171 target objects across 29 categories. Given a fixed target inventory, agents autonomously select recorded viewpoints and viewing directions, integrate calibrated RGB observations, and iteratively estimate and revise object centers within a fixed interaction budget. We evaluate the localization accuracy and completeness of object-center maps using mean IoU, IoU-threshold hit rates, and target coverage. Evaluating seven frontier vision-language models (VLMs), we find that six achieve an all-target mean IoU below 20\%. Even the best-performing model, Claude Opus 5.5, reaches only 22.52\%, with just 18.36\% of targets exceeding an IoU of 0.5. Further analysis shows that broad submission coverage can coexist with limited localization accuracy, while target omissions further reduce overall scores. These results highlight substantial challenges in constructing accurate and complete object-center maps through active exploration of real scenes.

59. 【2611.26165】StreamEgo: Streaming Exocentric-to-Egocentric Video Generation

链接:https://arxiv.org/abs/2611.26165

作者:Tianyi Xu,Boyang Lyu,Yunkai Tang,Kairan Ding,Jiahang Li,Weiyu Guo,Yandong Guo,Shuchen Weng,Boxin Shi

类目:Computer Vision and Pattern Recognition (cs.CV)

关键词:transforms third-person recordings, scalable demonstration synthesis, generation transforms third-person, video generation transforms, first-person viewpoints

备注:

点击查看摘要

Abstract:Exocentric-to-egocentric (Exo2Ego) video generation transforms third-person recordings into first-person viewpoints, unlocking immersive AR/VR experiences and scalable demonstration synthesis for embodied robotics. Extending it to a streaming setting is expected to further enable live applications. However, existing methods suffer from degraded manipulation regions, overfitting and confinement to fixed-length clips with prohibitive latency. To address these limitations, we present StreamEgo, a real-time framework for unbounded, generalizable Exo2Ego video generation. To resolve intricate manipulation dynamics while supporting the streaming pipeline, a streaming dual-branch prior provides pixel-aligned guidance for translating appearance and skeletal kinematics. We also introduce a region-aware loss curriculum to improve generalization and stabilize multi-objective diffusion training. Furthermore, we distill the model into a chunk-wise autoregressive model, augmented with a region-aware corruption mechanism and a first-chunk curriculum, enabling high-quality, generalizable real-time streaming Exo2Ego generation at 24.3 FPS on a single H100 GPU.

60. 【2611.26164】MultiBinder: Preventing Multi-Subject Identity Leakage in Personalized Video Generation

链接:https://arxiv.org/abs/2611.26164

作者:Wei Li,Yiheng Zhang,Zhaofan Qiu,Ting Yao,Xiaoyan Sun,Tao Mei

类目:Computer Vision and Pattern Recognition (cs.CV)

关键词:significant scalability challenge, Personalized video generation, custom subjects increases, video generation faces, severe identity leakage

备注: ACM Multimedia 2026

点击查看摘要

Abstract:Personalized video generation faces a significant scalability challenge: as the number of custom subjects increases, existing methods suffer from severe identity leakage and semantic entanglement, resulting in videos with blended attributes and incoherent narratives. We identify that this stems from a lack of explicit binding between textual entity phrases and their corresponding visual patterns prior to fusion in diffusion models. To address this, we propose MultiBinder, a novel framework built on a ``bind-then-narrate'' paradigm. Our approach introduces a two-stage conditioning process before video diffusion. First, Cross-modal Entity Binding devises an entity-aware Q-Former to extract identity-centric features from each subject's reference image and tightly align them with its unique entity phrase, yielding disentangled ``identity tokens.'' Second, a Multimodal Large Language Model (MLLM) acts as a ``narrative director'' in the Narrative-Aware Subject Arrangement stage, capturing the action of each subject from the text prompt and synthesizes contextualized subject features that encapsulate ``who is doing what'' and preserve role-specific distinctions. These features are then injected into a video diffusion model via dedicated conditioning pathways, ensuring precise attribute allocation. Extensive experiments on both general and newly curated multi-subject benchmarks show that MultiBinder significantly outperforms existing methods in multi-subject identity fidelity.

61. 【2611.26159】When Foundation Models Disagree: Representational Heterogeneity for Calibrated Uncertainty in Whole-Slide Image Classification

链接:https://arxiv.org/abs/2611.26159

作者:Yekai Shen,Yuqian Wang,Zishun Liao,Rongsheng Wang,Wenting Chen,S. Kevin Zhou,Qingsong Yao

类目:Computer Vision and Pattern Recognition (cs.CV); Quantitative Methods (q-bio.QM)

关键词:supply strong representations, models supply strong, Pathology foundation models, foundation models supply, predictions remains hard

备注: accepted by IEEE BIBM 2026

点击查看摘要

Abstract:Pathology foundation models supply strong representations for whole-slide image classification, yet the confidence attached to their predictions remains hard to trust. Prevailing uncertainty estimators act within a single feature space and thereby overlook the uncertainty that the choice of foundation-model representation itself induces. This work introduces CALIFUSE, a calibration-oriented framework that fuses several pathology foundation models for the specific purpose of uncertainty estimation. Departing from accuracy-driven fusion, CALIFUSE reads representational heterogeneity across encoders as an explicit confidence cue. Slide-level decision consensus is coupled with chunk-level evidence agreement quantified through Jensen-Shannon divergence, so that both prediction stability and fine-grained evidence consistency inform the estimate. Foundation-model encoders stay frozen. A task-specific nnMIL classifier is still trained for each FM; CALIFUSE then estimates confidence at inference without training an extra fusion module. Evaluated on seven WSI classification tasks under several foundation-model combinations, the framework sharpens calibration, misclassification detection, and selective prediction relative to both the strongest single-encoder baseline and naive ensemble averaging, all while preserving competitive balanced accuracy.

62. 【2611.26146】A Practical Proprioceptive Data Collection Utility for the Unitree GO1

链接:https://arxiv.org/abs/2611.26146

作者:Jack Anders Smitterberg,Tan Chen

类目:Robotics (cs.RO); Computer Vision and Pattern Recognition (cs.CV)

关键词:Robot Operating System, Unitree SDK, Robot Operating, Operating System, legged robotics research

备注:

点击查看摘要

Abstract:The Unitree Go1 is a widely used platform for legged robotics research, yet collecting proprioceptive data from it is unnecessarily difficult: the Unitree SDK is sparsely documented, and ROS1 (Robot Operating System), the Go1's native middleware, is fully deprecated. We present an open-source graphical utility that enables researchers to reliably record proprioceptive data from the Go1, including joint states, IMU (Inertial Measurement Unit) readings, foot force sensor data, and gyroscope measurements, without requiring ROS expertise. The tool provides a simple GUI (Graphical User Interface) for starting, stopping, and labeling recordings in real time. The contribution of this paper is the collection utility. We characterize the acquisition quality of 51 recordings collected with it, including the achieved sampling rate and repeated samples. As a secondary, deliberately modest check that the data is useful, we train a terrain classifier on it. The results are weak: a Temporal Convolutional Network reaches about 38% accuracy on held-out recordings across ten terrain classes (10% would be chance), and a window-level split that ignores recording boundaries inflates the same measurement to about 99%. We report this as a demonstration that the logged signals carry terrain information, not as a competitive classifier. The utility is designed to lower the barrier to collecting Go1 proprioceptive data without ROS.

63. 【2611.26145】Can World Models Predict Rice Growth? Toward Agricultural World Models

链接:https://arxiv.org/abs/2611.26145

作者:Wenkai Ji,Zeteng Li,Yinkun Guo,Jie Zhou,Liang He,Zhenong Jin

类目:Computer Vision and Pattern Recognition (cs.CV); Computation and Language (cs.CL)

关键词:physical scene evolves, World models, models can imagine, physical scene, scene evolves

备注:

点击查看摘要

Abstract:World models can imagine how a physical scene evolves over the next few seconds, but growers need to know how a crop will develop over weeks to months: will this rice plant have headed a month from now, and how will it look at harvest? We introduce agricultural world modeling, the task of predicting how the same plant will look at a requested horizon of days to months, from one photo of it, its cultivar, and its camera view. In preliminary analysis, even state-of-the-art image generators get this wrong: they often grow the plant too little or give it the color and form of the wrong growth stage. We argue that they do not work out the plant's developmental state, which the photo reveals only indirectly. We therefore propose AgroWM, which reasons about development before it draws growth. Its reasoning module, PhenoClock, works out the current growth stage, predicts the future one, and writes a Growth Script with numeric targets for height, plant area, and color; its generation module, the Rooted Renderer, then edits the photo to follow this script, adding new organs and colors to the photographed plant. Because pixel scores can prefer an unchanged copy of the photo, we instead check each forecast's traits and developmental timing against the plant's own later photos. On rice, 54.6% of the forecasts of AgroWM are matched to the target stage (Stage R@1), against 34.8% for GPT Image 2.5, 24.0% for Qwen Image 2.1, and 47.4% for the strongest domain-specific baseline, the same backbone fine-tuned without Growth Scripts; 9 to 20 weeks ahead, it still reaches 46.4%, against 35.7% for that baseline. On maize, a crop with a different body plan and stage scale, the same design again scores best on all four measures.

64. 【2611.26128】CryoGaussian: Heterogeneous Cryo-EM Reconstruction Made Explicit with Gaussian Splatting

链接:https://arxiv.org/abs/2611.26128

作者:Ziwen Wang,Jiajun Fan,Chaoran Cheng,Ruihan Guo,Yanru Qu,Juexiao Zhou,Ge Liu

类目:Computer Vision and Pattern Recognition (cs.CV)

关键词:Single-particle cryo-electron microscopy, determines macromolecular structures, Single-particle cryo-electron, noisy particle images, particle images

备注: 59 pages, 25 figures, 15 tables

点击查看摘要

Abstract:Single-particle cryo-electron microscopy (cryo-EM) determines macromolecular structures from noisy particle images. However, rare, transient compositional states and subtle, continuous conformational variation are difficult to resolve. We present CryoGaussian, a Gaussian splatting method for heterogeneous cryo-EM reconstruction that fits an explicit Gaussian density representation directly to the particle images through a differentiable rasterizer rendering orthographic line integrals modulated by the contrast transfer function. The Gaussians are initialized randomly, with their number derived from the estimated spatial support of the molecular density and their minimum size bounded by the Nyquist frequency of the particle images. To capture the compositional and conformational variation across particle images, we learn particle-image and per-Gaussian latents that jointly modulate the base position, amplitude, and scale of each Gaussian. Each structural state is then reconstructed by fitting the Gaussians conditioned on its inferred latent. On CryoBench, CryoGaussian achieves state-of-the-art reconstruction accuracy for both compositional and conformational heterogeneity. On experimental data, it resolves a rare compositional state of the 50S ribosomal subunit (class C4, 1.3% of particles) with rRNA helix 68 docked in the absence of the central protuberance, and continuous conformational motion of the SF3b and helicase subcomplexes of the pre-catalytic spliceosome. We further optimize the rasterizer for additive cryo-EM projection, reducing training time by 12.8-28.0$\times$.

65. 【2611.26120】Better Retrieval, Not Better Binding: Pretraining Exposure in Egocentric Video Retrieval

链接:https://arxiv.org/abs/2611.26120

作者:Yuqi Guo,Michael Wray

类目:Computer Vision and Pattern Recognition (cs.CV)

关键词:achieve strong performance, Video-language models, achieve strong, exposure, strong performance

备注: 17 pages, 4 figures, 3 tables

点击查看摘要

Abstract:Video-language models (VLMs) achieve strong performance on video-text retrieval and are increasingly used for egocentric video understanding. Yet, despite gains in zero-shot retrieval, they have shown limitations in compositional understanding, and it remains unclear whether they understand compositional relationships between verbs and nouns. In this work, we introduce Ego-ExBind, a compositional diagnostic protocol for analyzing pretraining exposure and verb-noun binding in egocentric video-text retrieval. Ego-ExBind consists of an Exposure Ledger built from the EgoClip pretraining corpus used by EgoVLPv2, which defines Seen Composition, Unseen Composition, and Unseen Atom splits for EPIC-KITCHENS-100 multi-instance retrieval (MIR). We further probe noun and verb binding channels through controlled noun-swap and verb-swap negatives. We find that zero-shot retrieval is positively associated with pair exposure within seen compositions, even after controlling for the marginal frequencies of verbs and nouns. However, higher pair exposure is associated with larger noun-swap margins and slightly smaller verb-swap margins. This asymmetry motivates us to test whether exposure reweighting can improve retrieval with little change in the measured swap margins. To isolate the effect of exposure input, we keep the trained checkpoint fixed and replace only the exposure input at inference. We demonstrate that this exposure reweighting can improve video-text retrieval without stronger verb-noun compositional understanding, and that the gain depends on the correct sample-level exposure correspondence. By separating exposure-driven retrieval gains from verb-noun binding, Ego-ExBind provides a diagnostic framework for evaluating whether future improvements in egocentric MIR reflect compositional understanding rather than pretraining familiarity.

66. 【2611.26116】Learning Reconstruction Errors as Spatiotemporal Evidence for AI-Generated Video Detection

链接:https://arxiv.org/abs/2611.26116

作者:Jiahua Jiang,Jinglai Li,Jianru Zhang,Jin Zhu

类目:Computer Vision and Pattern Recognition (cs.CV)

关键词:potential societal risks, raised growing concerns, concerns about misinformation, societal risks, creating an urgent

备注: 22 pages

点击查看摘要

Abstract:The proliferation of AI-generated videos has raised growing concerns about misinformation, fraud, and potential societal risks, creating an urgent need for detection. Reconstruction-error-based methods have shown strong empirical performance for AI-generated image detection by exploiting discrepancies between an input and its reconstructed version. Their extension to video detection, however, faces two challenges: (i) a fixed reconstructor is not optimized for the detection objective and may yield weak discriminative signals; and (ii) an additional raw-video feature stream is often introduced to model temporal dependencies, increasing architectural overhead. To address these challenges, we develop a detector that learns and exploits reconstruction errors as a unified source of spatiotemporal evidence. First, we jointly optimize a reconstructor and the downstream classifier to produce reconstruction errors that are more informative for detecting AI-generated videos. Second, we show that consecutive reconstruction errors alone carry temporal evidence, and thus a 3D convolution over them captures spatiotemporal evidence without a raw-video stream. Extensive experiments on over 30 existing AI-video generators demonstrate competitive performance against baselines.

67. 【2611.26107】HiUni: Harmonizing Understanding and Generation in Shared Visual Space

链接:https://arxiv.org/abs/2611.26107

作者:Jiahao Hu,Jingdong Du,Yichen Zhang,Yi Song,Zhenyu Zhang,Jiaxin Niu,Ruihang Li,Xiangzhao Hao,Yiqian Zhang,Yong Huang,Zefeng Zhang,Zheyuan Gu,Shuohuan Wang

类目:Computer Vision and Pattern Recognition (cs.CV); Artificial Intelligence (cs.AI)

关键词:multimodal models aim, models aim, Unified multimodal models, single framework, Unified multimodal

备注:

点击查看摘要

Abstract:Unified multimodal models aim to support visual understanding and generation within a single framework. However, prevailing approaches often separate their visual representations or backbone computation to accommodate the different requirements of these two tasks, which limits the degree of unification. In this paper, we present HiUni, which revisits this design choice and explores whether a pretrained vision language model can be directly extended into a unified model with extensive parameter sharing. Specifically, HiUni reuses the same language-aligned ViT and dense Transformer backbone for both capabilities, while confining task-specific computation to the output side. Yet such a design introduces two key challenges. Generation in pixel space is difficult to optimize, and semantic abstraction in pretrained ViT discards fine spatial details required for synthesis. To address these challenges, HiUni introduces a \emph{Pixel Diffusion Head} tailored to direct pixel-space optimization and a lightweight \emph{HiGate} pathway that provides fine-grained spatial details near the output stage. Extensive experiments show that HiUni achieves a similar average understanding score to the pretrained VLM while attaining competitive performance on image generation and editing. Controlled comparisons and analyses further demonstrate that a single dense backbone can effectively support both capabilities without task-specific backbone computation, highlighting a simpler path toward unified multimodal modeling.

68. 【2611.26103】UniVEP: Unifying Velocity and Endpoint Prediction for Robot Manipulation

链接:https://arxiv.org/abs/2611.26103

作者:Zhengtu Liang,Qiwei Liang,Tao Lin,Shihang Weng,Boyang Cai,Xiangyu Chen,Yikeng Chen,Jiaming Liang,Bohan Tan,Yu Yan,Renjing Xu

类目:Robotics (cs.RO); Computer Vision and Pattern Recognition (cs.CV)

关键词:Flow-matching robot policies, Flow-matching robot, clean actions bring, bring different challenges, robot policies

备注: 43 pages, 8 figures

点击查看摘要

Abstract:Flow-matching robot policies usually use one prediction target throughout action generation, even though recovering actions from noise and polishing nearly clean actions bring different challenges. We systematically compare velocity prediction (V), endpoint prediction (X), and velocity-weighted endpoint prediction (X--V) across multiple robot policies, benchmarks, and inference budgets. On StarVLA and LiLa-WAM, X--V works best for few-step generation, while V gains more from iterative refinement. We further uncover their complementary strengths across noise levels: clean actions have concentrated covariance spectra, endpoint predictions stay more stable under high noise, and velocity predictions offer more accurate corrections near clean actions. Inspired by these findings, we introduce UniVEP, which learns both prediction targets together with consistency supervision and switches from endpoint recovery to velocity refinement as noise drops. Each step only needs one network run, and joint training also boosts single-step endpoint generation. Tests on simulated and real-world manipulation show better success rates and efficiency. On RoboTwin, UniVEP hits 61.36% success with one StarVLA run, beating the best single-target baseline at sixteen runs (59.96%). With LiLa-WAM, two runs outperform the best sixteen-step single-target baseline and cut inference latency by 77.1%.

69. 【2611.26098】Frequency-Synchronized Boundary Coupling for Training-Free Multi-Prompt Long Video Generation

链接:https://arxiv.org/abs/2611.26098

作者:Yipan Xu,Shuyong Gao,Jiyuan Fu,Keliang Yin,Jiayu Chen,Qianyu Guo,Yang Liu,Wenqiang Zhang

类目:Computer Vision and Pattern Recognition (cs.CV)

关键词:descriptions remains challenging, made remarkable progress, evolving textual descriptions, textual descriptions remains, transformer-based video architectures

备注: Accepted at NeurIPS 2026. Project page: [this https URL](https://yipanxu-22.github.io/FreqSync-project)

点击查看摘要

Abstract:Text-to-video generation has made remarkable progress with diffusion models and transformer-based video architectures, yet generating coherent long videos from evolving textual descriptions remains challenging. Training long-video models from scratch requires substantial computation and large-scale multi-event data, while training-free multi-prompt generation must activate new prompt semantics without breaking temporal continuity. We identify a frequency-agnostic boundary coupling problem in existing transition strategies: weak coupling causes abrupt changes, flicker, and background jumps, whereas overly strong or coarse coupling may over-preserve source semantics and produce ghosting or duplicated subjects around prompt boundaries. To address this problem, we propose FreqSync, a training-free framework that formulates prompt transitions as frequency-selective boundary coupling. FreqSync combines Frequency-Synchronized Source-Conditioned Attention (FS-SCA) with Residual-Clipped Spectral Stitching (RCSS), enabling adjacent segments to share stable low- and mid-frequency structure and motion cues while keeping high-frequency prompt-specific details target-dominant under explicit residual constraints. Extensive experiments show that FreqSync achieves state-of-the-art transition smoothness and prompt alignment while maintaining strong perceptual video quality, demonstrating its effectiveness for coherent multi-prompt long video generation.

70. 【2611.26072】DfD-GS: Defocus Image Prior for Metric-Scale Single-View Depth Estimation and View Synthesis

链接:https://arxiv.org/abs/2611.26072

作者:Shiori Ueda,Rinto Yagawa,Hideo Saito,Shohei Mori

类目:Computer Vision and Pattern Recognition (cs.CV)

关键词:Recovering metric-scale geometry, remains fundamentally ill-posed, single viewpoint remains, viewpoint remains fundamentally, Recovering metric-scale

备注: Project page: [this https URL](https://sh1027.github.io/dfd-gs/)

点击查看摘要

Abstract:Recovering metric-scale geometry from observations at a single viewpoint remains fundamentally ill-posed, limiting the reliability of novel view synthesis. Recent methods rely on learned monocular depth priors, which provide strong structural estimates but can suffer from scale ambiguity and dataset-dependent biases. We propose DfD-GS, a feed-forward framework that predicts a metric-scale 3D Gaussian scene representation from a single-view all-in-focus--defocused image pair. The key idea is to exploit defocus blur as a physical metric depth cue. DfD-GS first estimates a defocus-based metric disparity map from the all-in-focus--defocused image pair and the corresponding defocus strength. This estimate is used to align and refine a monocular depth prior to metric scale. Visible and hidden 3D Gaussians are then predicted for occlusion-aware novel view synthesis. Experiments on synthetic-defocus datasets constructed from Booster and Middlebury show that DfD-GS outperforms single-image baselines in metric depth estimation and novel view synthesis. On WildRGB-D, DfD-GS achieves lower AbsRel and LPIPS but higher depth RMSE than the single-image baselines. Additional analyses show that DfD-GS maintains a more stable metric scale under field-of-view changes. Qualitative results on a real dataset show recovery of the main scene structure under optical defocus.

71. 【2611.26065】Domain-Aware Conformal Prediction for Trustworthy Spacecraft Pose Estimation under Sim-to-Real Shift

链接:https://arxiv.org/abs/2611.26065

作者:Nidhal Eddine Chenni,Arunkumar Rathinam,Djamila Aouada

类目:Computer Vision and Pattern Recognition (cs.CV)

关键词:proximity operations require, Autonomous rendezvous, accurate estimate, target spacecraft, rendezvous and proximity

备注:

点击查看摘要

Abstract:Autonomous rendezvous and proximity operations require not only an accurate estimate of the pose of the target spacecraft, but also a reliable indication of when this estimate can be trusted. Conformal prediction provides such an indication with a formal guarantee, by producing pose uncertainty sets that contain the true pose with a specified probability. Such a guarantee, however, assumes that calibration and test data follow the same distribution, an assumption that is violated in orbital perception, where labelled calibration images are available only from synthetic renderings while operational images are real. In this work, we show on the SPEED+ dataset that conformal pose estimation calibrated on synthetic images loses its coverage guarantee on real images, and that many of the resulting failures are silent: the predicted pose set appears valid but excludes the true pose. To address this, we calibrate on synthetic images translated to the appearance of real images. Since translation alters appearance while preserving geometry, the synthetic pose labels remain valid and no labelled real images are required. Experiments with two translation models show that this calibration restores coverage above the specified level for miscoverage levels of 0.1 and above, and reduces silent failures from up to 43.5% to at most 3.3% of the test images, bringing conformal pose estimation a step closer to trustworthy deployment in space.

72. 【2611.26058】One-to-Many: Diverse and Bidirectional Image-to-Image Translation via Rectified Flow

链接:https://arxiv.org/abs/2611.26058

作者:Sho Shimotsumagari,Seiichi Uchida,Shumpei Takezaki

类目:Computer Vision and Pattern Recognition (cs.CV)

关键词:tasks admit multiple, admit multiple valid, translation tasks admit, multiple valid target, latent-conditioned Rectified Flow

备注: 12 pages, 13 figures, 4 tables. Includes supplementary material

点击查看摘要

Abstract:Many paired image-to-image translation tasks admit multiple valid target appearances, while training provides only one target per source. We propose One-to-Many Flow (O2MFlow), a latent-conditioned Rectified Flow (RF) model for one-to-many generation and backward translation (target$\rightarrow$source) within a single transport framework. Conditioning the RF dynamics on an appearance latent turns transport into a family of latent-indexed bijections: a fixed latent gives a mapping that is inverted by time-reversed integration, and sampled latents produce diverse outputs for the same input. We formalize reusable appearance variations as Shared Latent Transport and show that Kullback--Leibler (KL) regularization directly controls the source--latent distribution gap between training and generation. Across three benchmarks, O2MFlow offers dataset-dependent forward diversity--fidelity trade-offs and the lowest backward LPIPS-Dist among evaluated methods, with the lowest backward FID on two benchmarks.

73. 【2611.26056】Less Than Meets the Eye: Action-Set Identifiability and Principled Abstention for 3D Physical Geometry

链接:https://arxiv.org/abs/2611.26056

作者:Reshad Ul Karim,Abrar Samin,Sammam Mahdi,Syeda Maliha Tabassum,Md Rashad Al Hasan Rony

类目:Computer Vision and Pattern Recognition (cs.CV)

关键词:physical surfaces lie, mirror reflection, surfaces lie, physical surface, physical

备注:

点击查看摘要

Abstract:A corridor, its mirror reflection, and its image on a display can look identical, yet their physical surfaces lie at different depths. We test whether bounded camera motions distinguish these explanations. Intervene3D takes an image and an initial geometry estimate, predicts the view under each hypothesis and candidate motion, and selects a motion by maximin separability. It reports the mechanism and first physical surface only if the available motions separate the hypotheses still in contention. Otherwise it abstains. An adequacy test also rejects observations that no hypothesis explains. On a matched synthetic benchmark with exact landmarks, disabling abstention gives confident answers on 76.4% of unresolvable scenes. With abstention, none receives a confident answer under exact or noisy landmarks. At 100% accuracy on answered scenes, maximin selection answers 61.0% of scenes and summed selection 43.1%. The adequacy test lowers wrong answers on strongly curved mirror scenes from 56.8% to 12.4%. Real display experiments compare two hypotheses using one frozen depth-network estimate. The second image supplies photometric evidence and is never passed through the depth network. One executed motion reduces the pooled median surface error of twelve networks by 31.5%. These networks have larger error inside the annotated panel than outside on 33.4% of illusion-benchmark images. They place the first physical surface behind the glass on 36.9% of Mirror3D frames.

74. 【2611.26045】PaveSplat: Resolution-Robust Feed-Forward Gaussian Splatting with Residually Corrected Surface-Aligned Covariances

链接:https://arxiv.org/abs/2611.26045

作者:Janghun Kim,Hyun-Cheol Kim,Eunbin An,Myeonghun Oh,Hyon-Gon Choo

类目:Computer Vision and Pattern Recognition (cs.CV)

关键词:feed-forward Gaussian splatting, Gaussian splatting method, Pixel-aligned feed-forward Gaussian, keeping surfels paving, Gaussian splatting predicts

备注: Accepted to ACCV 2026. 16 pages, 4 figures, 4 tables. Project page: [this https URL](https://janghunn.github.io/pavesplat/) ; code: [this https URL](https://github.com/janghunn/pavesplat)

点击查看摘要

Abstract:We present PaveSplat, a feed-forward Gaussian splatting method that adds a learned residual to the analytic surface-aligned covariance, keeping surfels paving the visible surface across rendering resolutions. Pixel-aligned feed-forward Gaussian splatting predicts one primitive per pixel from a few images in a single pass, and its rendering quality depends on how well these primitives cover the surface. Recent surfel-based methods compute the surface-aligned covariance analytically from predicted depth. However, this covariance has no way to correct the surfel orientation where the depth-derived frame is ill-posed. The surfels then cover the surface loosely; at the training resolution each surfel's one-pixel minimum footprint covers the gaps, but they show up as holes when the same prediction is re-rendered at a higher resolution. To address this, we keep the analytic covariance as an initialization, with its axes aligned to the surface tangents, and add an identity-initialized unit-quaternion residual that reorients a surfel only where this improves rendering, turning surfels that the analytic frame mis-aims back onto the surface. On RealEstate10K, PaveSplat achieves the best PSNR, SSIM, and LPIPS among single-Gaussian-per-pixel methods at every scale (1x-8x). The native-resolution gain is small; the benefit appears when the same prediction is rendered at higher display resolutions without re-running the network. On zero-shot DL3DV, it leads PSNR within this group at every scale and ranks first or second on SSIM/LPIPS. A coverage-hole analysis shows fewer uncovered pixels than both baselines under high-resolution re-rendering.

75. 【2611.26040】Streaming-AAA: Streaming AAA Game Video Generation at High Resolution, High Frame Rate, and Low Latency

链接:https://arxiv.org/abs/2611.26040

作者:Zhihao Hu,Jianqi Ma,Jun Zhang,Jiayang Bai,Haifeng Shi,Yanxiang Zhuang,Zhongqian Sun,Weidong Zhang,Wei Yang

类目:Computer Vision and Pattern Recognition (cs.CV)

关键词:existing methods remain, Controllable streaming video, offers a promising, promising route, existing methods

备注:

点击查看摘要

Abstract:Controllable streaming video generation offers a promising route toward interactive game simulation, yet existing methods remain far from practical gameplay. They often fail to jointly achieve high visual fidelity, high frame rates, and low interaction latency, resulting in poor responsiveness and degraded user experience. We present Streaming-AAA, a new paradigm for controllable infinite generation of AAA-quality game videos. Our system generates 1080p gameplay videos at 43 FPS with 198 ms end-to-end latency interaction on merely NVIDIA A800 GPUs, while supporting long-horizon play without severe temporal degradation or collapse. Streaming-AAA is built upon three key designs. First, we redesign the VAE architecture for efficient high-resolution decoding, enabling real-time synthesis at 1080p. Second, we pretrain the diffusion model using diffusion forcing and significantly extend the temporal training horizon, allowing the model to capture long-term dependencies and maintain persistent memory. Third, we perform post-training distillation and memory compression to improve inference efficiency while retaining visual quality and temporal consistency. Experiments show that Streaming-AAA substantially improves visual fidelity, frame rate, latency, and long-term stability over prior controllable video generation approaches, enabling a significantly more playable experience for interactive AAA game video generation.

76. 【2611.26035】Fixing the Twin, Not the Detector: Sim-to-Real Roadside LiDAR Detection from Digital-Twin Data

链接:https://arxiv.org/abs/2611.26035

作者:KuanLin Chen,Feng-jui Kuo

类目:Computer Vision and Pattern Recognition (cs.CV); Machine Learning (cs.LG)

关键词:digital twins promise, expensive to train, promise to replace, Roadside LiDAR detectors, real labels

备注: 8 pages, 1 figure, 5 tables. Accepted to the IEEE BigData 2026 Cup (UCF UrbanTwin Sim2Real LiDAR Challenge); first place, LUMPI track. Code: [this https URL](https://github.com/dipper13579/urbantwin-sim2real-lidar)

点击查看摘要

Abstract:Roadside LiDAR detectors are expensive to train on real labels, and digital twins promise to replace these labels, but a detector trained on a twin meets a different sensor, different objects and different annotation conventions on the real site. We describe the pipeline that placed first in the LUMPI track of the UrbanTwin Sim2Real LiDAR Challenge (IEEE BigData 2026 Cup, 0.5541 combined score) and fifth on its V2X-Real track (0.8102). Every detector is a CenterPoint model trained only on synthetic data, on a single 8 GB laptop GPU. Most of the gain came from correcting differences between the twin and the real site rather than from the detector. We align the twin's coordinates, point density and box convention with the test sensor, generate the pedestrians and riders that the twins lack, plus street clutter as unlabeled negatives, and rebuild twin objects at real size and real point density. A non-learned resampling to the per-cell point density of unlabeled real frames lifts LUMPI realism from 0.589 to 0.947, against 0.949 between real frames. Label-free post-processing then exploits what a fixed roadside sensor guarantees: static structures recur in every frame, and real objects have characteristic point statistics. Reporting the validation and test gain of each component side by side, we find that corrections of systematic sim-to-real differences transfer to the test set, typically keeping a quarter to three quarters of their validation gain, while choices that fit the few objects of a validation set do not.

77. 【2611.25980】Learning What Actions Change: Action Effect Matching for Dynamics-Faithful World Models

链接:https://arxiv.org/abs/2611.25980

作者:John Won,Yonghoon Dong,Jinwoo Shin

类目:Robotics (cs.RO); Computer Vision and Pattern Recognition (cs.CV)

关键词:predict environment dynamics, World models aim, enabling the prediction, aim to predict, predict environment

备注: 27 pages, preprint

点击查看摘要

Abstract:World models aim to predict environment dynamics under actions, enabling the prediction of action consequences without executing them in the real world. Yet despite producing increasingly realistic videos, current world models often fail to capture how small action changes affect future observations. This limits their use in tasks such as robotic policy evaluation and model-based planning, where nearly identical actions can lead to qualitatively different outcomes. To address this, we propose AFFECT, an auxiliary training objective for learning the distribution of visual changes induced by action perturbations. Specifically, we inject temporally correlated noise into the actions of a policy trained on task-agnostic human teleoperation. We then train the world model so that its generated videos respond to this noise as the recorded videos do, without requiring repeated executions from the same initial state. Across OGBench simulation and real-world DROID and ALOHA-2 setups, AFFECT makes world-model rollouts more faithful to real outcomes. On DROID, the Pearson correlation between policy success rates in the world model and on the real robot rises from 0.47 to 0.79, alongside up to 23% lower FVD. These gains carry over to planning, where selecting actions with AFFECT-trained world models improves task success by up to 8.3%.

78. 【2611.25979】Multi-View Transformer with Geometry-Guided Multi-Modal ControlNet for 3D Virtual Try-On

链接:https://arxiv.org/abs/2611.25979

作者:Sarthak Batra,Armin Mustafa,Adrian Hilton

类目:Computer Vision and Pattern Recognition (cs.CV); Artificial Intelligence (cs.AI)

关键词:Gaussian Splatting, Editing, Gaussian, Splatting, garment details

备注: 20 pages, 2 figures, 3 tables

点击查看摘要

Abstract:Editing a 3D Gaussian Splatting (3DGS) capture of a person so that they wear a new garment requires every rendered view to agree on the same garment details. Existing 3D virtual try-on (VTON) pipelines, however, edit training views largely independently and fuse them with hand-designed heuristics such as manually chosen reference views and fixed attention-blending weights. The resulting inconsistencies are baked into the 3D representation and surface as texture drift, blurred prints, and implausible geometry in novel views. We present GS-VTON-MVT, a three-stage framework that combines implicit and explicit 3D reasoning for consistent 3DGS try-on. (i) A multi-view inpainting transformer jointly edits a set of training views with full cross-view self-attention, so that cross-view correspondence is learned rather than imposed. (ii) A geometry-guided multi-modal ControlNet conditions the diffusion editor on the garment image together with depth and surface normals rendered from the current 3DGS state, supplying explicit structure where visual evidence is weakest. (iii) A pose-adaptive persona-aware attention replaces the fixed blending weight between current-view and reference-view features with weights derived from camera-pose proximity. On the 3D-VTONBench benchmark, GS-VTON-MVT improves FID, KID, LPIPS-based cross-view consistency, and user preference over general-purpose 3D editors and prior 3D try-on methods.

79. 【2611.25975】Sample-efficient end-to-end Optical Music Recognition through Active Learning

链接:https://arxiv.org/abs/2611.25975

作者:Eric Ayllon,Francisco J. Castellanos,Antonio Rios-Vila

类目:Computer Vision and Pattern Recognition (cs.CV)

关键词:Optical Music Recognition, Music Recognition, recently shown promising, Optical Music, approaches to Optical

备注:

点击查看摘要

Abstract:End-to-end approaches to Optical Music Recognition (OMR) have recently shown promising results, but their performance often degrades on unseen data. A common solution is to pretrain and fine-tune models with annotated samples from the target domain, improving adaptation to new distributions. However, this process typically requires extensive manual annotation to yield competitive models, making it costly and time-consuming. In this work, we explore Active Learning to reduce this burden by selecting informative samples for labeling in low-resource scenarios, where the amount of manual annotation that can be performed is limited. We evaluate both uncertainty- and diversity-based sampling strategies on staff-level and full-page corpora. Results show that confidence-based sampling performs comparably to oracle selection (based on true error), and that each scenario benefits from a tailored strategy.

80. 【2611.25963】MorphTwin: Enriching Deformable Material Responses from Videos for Morphable and Interactive Physical Twins

链接:https://arxiv.org/abs/2611.25963

作者:Haoqin Hong,Ding Fan,Fubin Dou,Tianyu Huang,Yuzhuo Chen,Zhemeng Zhang,Hanxiao Jiang,Congcong Zhu,Gerhard Petrus Hancke,Jingrun Chen,Rynson WH Lau

类目:Computer Vision and Pattern Recognition (cs.CV)

关键词:requires accurately capturing, video requires accurately, Reconstructing interactive physical, maintaining efficient simulation, requires accurately

备注:

点击查看摘要

Abstract:Reconstructing interactive physical twins from video requires accurately capturing how objects deform while maintaining efficient simulation. We observe that reproducing motion in a video does not guarantee correct responses to new forces when the physical representation restricts how an object can deform. In spring-based twins, coupling between volume and shape responses creates such a limitation. We present MorphTwin, which enriches local deformation modes to capture and independently control these responses while retaining standard spring forces and explicit simulation. To support interpretable material editing and transfer, we use offline calibration to map shared relative response changes to stiffness updates on each reconstructed graph. Under identical inverse settings on synthetic scenes, MorphTwin reduces the mean absolute error of Poisson's ratio by 75.8% and the mean relative error of Young's modulus by 22.1% compared to PhysTwin. We introduce an egocentric stereo and RGB-D benchmark covering interactions with objects of varying stiffness. Real-world experiments demonstrate improved reconstruction accuracy and up to 14.4% lower future tracking error than PhysTwin at interactive speeds.

81. 【2611.25946】Point Dynamics as Proxy for Robotic World Models

链接:https://arxiv.org/abs/2611.25946

作者:Dong An,Tianyu Liu,Haoran Ding,Wen Yang,Anqing Duan,Zhengzhong Liu,Eric P. Xing,Guangyi Liu

类目:Computer Vision and Pattern Recognition (cs.CV)

关键词:models predict future, predict future video, Robotic world models, world models predict, predict future

备注: 24 pages, 7 figures, 11 tables. Project page: [this https URL](https://marsaki.github.io/point-dynamics-proxy/)

点击查看摘要

Abstract:Robotic world models predict future video frames from the current observation and a sequence of robot actions. While recent models follow the given actions closely, manipulated objects often collapse or move inconsistently with the action. We attribute this to the conditioning: an action, or a rendering of the robot, specifies where the robot goes but not how the scene responds, leaving a single video model to infer the interaction dynamics implicitly while also rendering appearance. We therefore separate dynamics prediction from visual rendering, with 3D point dynamics as the proxy between them. Specifically, we lift the first frame into 3D points, simulate how they respond to the robot's motion with a learned point dynamics model, and render the simulated points back into pixel-aligned videos that any video model can read. In controlled experiments on the DROID dataset with two video renderers of different architectures, this point dynamics proxy reduces object motion error by 8.6% and 6.2% without degrading image quality, and by 8.1% on a real bimanual robot. Moreover, the video follows the simulation: more accurate dynamics yield more accurate object motion, and editing the points steers the video. This suggests that advances in dynamics modeling can translate into more faithful robotic world models.

82. 【2611.25935】winD: Generalizing Deepfake Detection from Blending to Entire Face Synthesis

链接:https://arxiv.org/abs/2611.25935

作者:Andrii Yermakov,Jan Cech,Mario Fritz,Jiri Matas

类目:Computer Vision and Pattern Recognition (cs.CV)

关键词:real frame, Facial deepfake detectors, facial deepfake detector, Facial deepfake, entire face

备注:

点击查看摘要

Abstract:Facial deepfake detectors are usually trained on blending fakes, where a generated face is pasted back into a real frame. Recent generators synthesize the entire face and leave no traces of blending. On a benchmark of 19 datasets released from 2019 to 2026, the best prior facial deepfake detector reaches a mean AUROC of 96.2% on ten blending datasets, but none exceeds 78.1% on five datasets of entire face synthesis (EFS). AI-generated content (AIGC) detectors do not close this gap. We propose TwinD, a detector trained on FaceForensics++ fakes and two kinds of twins: fakes made from a real frame. A reconstruction twin is a real frame reconstructed by an autoencoder, with the real pixels put back in a random region. It targets the neural traces of a decoder. A blob twin blends an altered copy of a real frame with the frame itself, targeting blending artifacts. It needs no generator and no face-shaped mask. Together, these twins provide two complementary forensic cues that generalize to unseen manipulations. TwinDreaches a mean AUROC of 96.3%, 8.9 points above the prior state-of-the-art, with 98.1% on the blending datasets and 95.6% on the EFS datasets. Trained on twins alone, without a deepfake dataset, single models still reach 94.3% on average over three seeds. For reproducibility, the code, trained model, and training data are at: this https URL

83. 【2611.25934】AGSR: Aperture-Guided Super-Resolution for Gaussian Splatting

链接:https://arxiv.org/abs/2611.25934

作者:Ayman Alkhateeb,Hadi Amirpour,Christian Timmerer

类目:Computer Vision and Pattern Recognition (cs.CV); Graphics (cs.GR)

关键词:attenuates fine-scale scene, fine-scale scene structure, Gaussian Splatting, rasterization attenuates fine-scale, Gaussian rasterization attenuates

备注: Accepted at the 2026 IEEE International Workshop on Multimedia Signal Processing (MMSP 2026). Recipient of the Best Paper Award

点击查看摘要

Abstract:3D Gaussian Splatting (3DGS) enables real-time novel view synthesis, but Gaussian rasterization attenuates fine-scale scene structure through the projected covariance of each primitive. We show that contracting these projected covariances at inference time can mitigate the effective bandwidth limitation induced by finite Gaussian footprints, motivating a multi-aperture rendering framework for super-resolution. Based on this insight, we propose AGSR (Aperture-Guided Super-Resolution), a lightweight inference-time framework that progressively fuses multi-aperture renderings using uncertainty-guided feature aggregation and geometric conditioning from transmittance and depth cues. AGSR operates as a plug-in enhancement for pre-trained 3DGS models. Experiments on the Mip-NeRF 360 benchmark show that AGSR outperforms lightweight image-space super-resolution baselines, with gains increasing as the underlying Gaussian representation deteriorates. The results provide evidence that covariance contraction exposes bandwidth-sensitive structural cues attenuated by native Gaussian rasterization. AGSR achieves these improvements while maintaining real-time performance with up to 139k parameters.

Comments:
Accepted at the 2026 IEEE International Workshop on Multimedia Signal Processing (MMSP 2026). Recipient of the Best Paper Award

Subjects:

Computer Vision and Pattern Recognition (cs.CV); Graphics (cs.GR)

Cite as:
arXiv:2611.25934 [cs.CV]

(or
arXiv:2611.25934v1 [cs.CV] for this version)

https://doi.org/10.48550/arXiv.2611.25934

Focus to learn more

              arXiv-issued DOI via DataCite (pending registration)</p>
84. 【2611.25930】Why Flat Drafts Stay Flat: Region-Mean Latents Govern Texture in Flow-Based Image Generation

链接:https://arxiv.org/abs/2611.25930

作者:Zihan Gu,Jiangcan Jia,Hua Zhang,Lulu Zhao

类目:Computer Vision and Pattern Recognition (cs.CV)

关键词:interior detail unspecified, leaving interior detail, pretrained flow-based generators, detail unspecified, photorealistic prompts

备注:

点击查看摘要

Abstract:Flat color-block drafts specify layout and regional color while leaving interior detail unspecified, yet pretrained flow-based generators can preserve their flat appearance even under photorealistic prompts. We trace this behavior to a cross-scale mechanism linking fine pixel variation to regional means in variational autoencoder (VAE) latent space. Flattening photo regions systematically shifts their latent means, with offsets concentrated along a shared channel direction within each backbone. Controlled field exchanges in FLUX.1-dev and Qwen-Image establish the effect of this displacement on texture generation: adding the photo-derived regional mean field to a flat latent restores texture, whereas adding its complementary field alone largely preserves flatness. A second-order encoding expansion explains how encoder curvature can convert zero-mean perturbations into mean shifts. An analytically solvable conditional Gaussian flow shows how regional noise averaging enables near-unit endpoint sensitivity at high signal-to-noise ratios. This mechanism motivates training-free rendering through pixel dither before encoding, mean-field scaling, and early boundary holding. Paired low-rank adaptation (LoRA) with draft conditioning improves content recovery and produces photo-like regional means under the standard velocity objective. Explicit regional mean guidance remains effective after adaptation, continuously controlling texture and improving color fidelity. These findings connect encoding shifts to persistent generation behavior and support interventions evaluated for texture, spatial fidelity, and semantic recovery.

85. 【2611.25925】SpatialMem: Building Quantitative Spatial Reasoning Agents with Hierarchical Memory

链接:https://arxiv.org/abs/2611.25925

作者:Zinan Zheng,Nuo Chen,Jia Li

类目:Computer Vision and Pattern Recognition (cs.CV)

关键词:Vision Language Models, Language Models, shown remarkable capabilities, Vision Language, spatial reasoning

备注:

点击查看摘要

Abstract:Vision Language Models (VLMs) have shown remarkable capabilities in qualitative spatial reasoning, but remain limited in quantitative spatial reasoning. Current work leverages the VLM agent to invoke external vision tools to perform computation. However, these methods usually bind each functional requirement to a single predefined vision specialist, implicitly resulting in fixed and rigid tool pipelines. In practice, addressing diverse spatial scenarios and tasks demands a broader diversity in pipeline orchestration and vision specialist selection to achieve robust spatial reasoning. However, such diversity leads to a combinatorial execution space, making it challenging for the agent to orchestrate the optimal pipelines and specialists for different scenarios. In this work, we propose SpatialMem, a memory framework that enables a VLM agent to accumulate tool-use experience and leverage that memory to orchestrate vision specialists to solve the spatial reasoning problem. SpatialMem builds a two-level memory structure, with task memory capturing tool orchestration and scene memory capturing tool selection. It further introduces a dual-state trajectory optimization process to preserve successful tool experiences and refine suboptimal ones. Moreover, existing spatial reasoning benchmarks suffer from quality issues and limited coverage of scene types and tasks. To address these gap, we propose MetricBench, a benchmark covering both indoor and outdoor scenes and three quantitative spatial reasoning tasks. Our method achieves state-of-the-art results on MetricBench, Q-Spatial, and SpatilRGPT-Bench, outperforming best VLM baselines by 32% on MetricBench. Additionally, the memory design yields a 9.1% absolute gain over a memory-free agent on MetricBench.

Subjects:

Computer Vision and Pattern Recognition (cs.CV)

Cite as:
arXiv:2611.25925 [cs.CV]

(or
arXiv:2611.25925v1 [cs.CV] for this version)

https://doi.org/10.48550/arXiv.2611.25925

Focus to learn more

              arXiv-issued DOI via DataCite (pending registration)</p>
86. 【2611.25904】Multiple-Camera Tracking of Rigid Objects

链接:https://arxiv.org/abs/2611.25904

作者:Frédérick Martin,Radu Horaud

类目:Robotics (cs.RO); Computer Vision and Pattern Recognition (cs.CV)

关键词:tracking rigid objects, paper we describe, tracking rigid, rigid objects, image contours

备注:

点击查看摘要

Abstract:In this paper we describe a method for tracking rigid objects using one or several cameras. The tracking process consists of aligning a 3-D model representation of an object with image contours by measuring and minimizing the image error between predicted model points and image contours. The tracker behaves like a visual servo loop where the internal and external camera parameters are updated at each new image acquisition. We study in detail the Jacobian matrix associated with this minimization process in the presence of both point-to-point and point-to-contour matches. We establish the minimal number of matches that are needed as well as the singular configurations leading to a rank-defficient Jacobian matrix. We find a mathematical link between the point-to-point and point-to-contour cases. Based on this link we show that the latter has the same kind of singularities than the former. Moreover, we study multiple camera configurations which optimize the robustness of the method in the presence of single-camera singularities, bad, noisy, or missing data. Extensive experiments done with a complex ship part and with up to three cameras validate the method. In particular we show that the tracker may well be used as a camera calibration procedure.

87. 【2611.25902】Does It Depend on the Split? Frozen Satellite Features vs. Fine-Tuned CNNs for Tree Species Mapping

链接:https://arxiv.org/abs/2611.25902

作者:Dharani Deivasihamani,Nandakishore Puttashamachar

类目:Computer Vision and Pattern Recognition (cs.CV)

关键词:Self-supervised foundation models, Self-supervised foundation, split, spatial split, Self-supervised

备注:

点击查看摘要

Abstract:Self-supervised foundation models are often said to need fewer labels than supervised training. We test this on a forestry task: classifying 20 tree species classes in 0.2 m aerial images from the TreeSatAI dataset. We compare a frozen DINOv3 model pretrained on 493 million satellite images, with only a linear classifier trained on top, against an ImageNet ResNet-50 that is fine-tuned end to end. On the official split, fine-tuning wins clearly (macro F1 0.764 vs. 0.685). However, 28.8% of the official test patches overlap a training or validation patch, and copying the label of the nearest such patch is correct for 83.6% of them. We therefore build a spatial split that holds out whole 2 km squares with a 200 m buffer. On this split the comparison becomes inconclusive (0.648 vs. 0.640, paired 95% CI of the difference [-0.054, 0.016]), and the learning curves of the two models overlap at every label budget from 343 to 33,126 training patches. On the five rarest classes, the frozen model achieves a higher mean F1 (0.566 vs. 0.445), although the confidence interval for this difference includes zero ([-0.05, 0.23]). Diagnostics show that the frozen features mainly miss differences between closely related species, such as the two native oaks. Choosing the input size and layer on validation data (224 px, block 16) raises the frozen model to 0.662 on the spatial split. A comparison that looks decisive on the official split is thus inconclusive under geographically separated evaluation. We release the spatial split and our code.

88. 【2611.25889】EvoMem: Self-Evolving Skills for Time-Aware Long Video Memory Agents

链接:https://arxiv.org/abs/2611.25889

作者:Yutong Guan,Meixing Shi,Haoran Sun,Bing Sun,Yuxiang Cai,Yankai Jiang

类目:Computer Vision and Pattern Recognition (cs.CV)

关键词:global semantic matching, coherent evidence chain, state-driven temporal control, lack state-driven temporal, large vision-language models

备注:

点击查看摘要

Abstract:Although memory-augmented architectures have expanded the ability of large vision-language models (LVLMs) to process long videos, existing retrieval processes still largely rely on global semantic matching and lack state-driven temporal control, often returning fragmented or redundant evidence and failing to maintain a coherent evidence chain. To address this challenge, we propose EvoMem, a time-aware long-video memory agent guided by self-evolving retrieval skills. EvoMem constructs a hierarchical time-indexed memory, where different types of evidence are aligned onto a unified video timeline; during retrieval, temporal cues from acquired evidence are further converted into explicit signals that constrain subsequent memory access. To address recurring procedural errors in long-video question answering, such as unclear retrieval targets, improper memory access paths, or premature stopping, EvoMem introduces retrieval skills to provide process-level guidance, enabling the agent to dynamically adjust retrieval content, memory operations, and target temporal ranges according to the current evidence state, while deciding when to stop. Furthermore, inspired by reinforcement learning, we formulate skill evolution as a reward-guided policy improvement process. Each candidate skill is treated as an external policy patch, and rewards are assigned according to answer-correctness state transitions on the validation set. In this way, effective skills are accepted and low-quality skills are rejected without updating model parameters. Experiments on four long-video question answering benchmarks show that EvoMem significantly outperforms existing baselines, achieving an average accuracy of 60.1% and an average improvement of 8.6 percentage points over the previous best methods, demonstrating its effectiveness for long-video reasoning.

89. 【2611.25873】WHU-Road3D: A dataset and benchmark towards fine-grained road surface mapping

链接:https://arxiv.org/abs/2611.25873

作者:Xiaoxin Mi,Zhen Cao,Chong Liu,Yuhao Li,Zhen Dong,Bisheng Yang,Yang Jia,Shengwu Xiong

类目:Computer Vision and Pattern Recognition (cs.CV)

关键词:intelligent transportation systems, road surface elements, Accurately extracting, road asset management, road surface

备注: 16 pages

点击查看摘要

Abstract:Accurately extracting and mapping road surface elements is crucial for applications such as high-definition maps, road asset management, and intelligent transportation systems. This study first provides a comprehensive review of the benchmark datasets for extracting and mapping road elements, encompassing overhead-to-perspective data, large-to-local scales, and coarse-to-fine details. Subsequently, we thoroughly review the relevant approaches for extracting road surface elements. However, we argue that existing datasets still require enhancements in semantic richness, data modality diversity, and scenario complexity. To bridge these gaps, this paper presents WHU-Road3D, a large-scale multimodal dataset for fine-grained road surface mapping to address these shortcomings. WHU-Road3D provides approximately 100 km of diverse Chinese highways and urban streets. It features a rich combination of MLS point clouds, associated panoramic images, vehicle trajectories, detailed vectorized annotations, and pointwise instance annotations for road surface elements. The dataset carefully annotates 15 distinct categories of road elements, including common markings like road boundaries, zebra crossings, lane dividers, stop lines, as well as 10 types of direction symbols. To our knowledge, WHU-Road3D stands as the first multimodal dataset for mapping such a comprehensive range of road surface elements, designed to facilitate cutting-edge research in instance segmentation, full-element extraction and mapping on road surfaces. Beyond these immediate applications, WHU-Road3D also provides a valuable source for hierarchical road topology reasoning, map updating, and street scene generation. The dataset, along with the code for our benchmark methods, will be made publicly available at \href{this https URL}{this https URL}.

90. 【2611.25865】hinking in Streams: Proactive Spatial Reasoning with Vision-Language Models

链接:https://arxiv.org/abs/2611.25865

作者:Junyoung Hong,Minsu Kim,Jaehyun Kim,Jeonghwan Cho,Jiwoo Lee,Seon Joo Kim

类目:Computer Vision and Pattern Recognition (cs.CV)

关键词:requested spatial condition, spatial condition holds, updating spatial knowledge, condition holds, video can require

备注: 11 pages, 8 figures

点击查看摘要

Abstract:Recognizing when a requested spatial condition holds in streaming video can require maintaining and updating spatial knowledge across changing viewpoints. We study this capability through proactive spatial reasoning, where a model receives a query in advance and determines whether and when to respond as the stream unfolds. We introduce ProSpace-Bench, comprising nine tasks across object, camera, and scene reasoning, and ProSpace-Train for supervised training. To support spatial decisions, we propose ProSpace-VLM, which injects explicit geometric encodings estimated causally from incoming frames into intermediate VLM layers, using the existing KV cache as spatial memory and spatial token selection for efficient streaming. Experiments reveal a substantial gap between VLMs and humans, while ProSpace-VLM improves timely responses and correct abstention over fine-tuned baselines. Analyses examine how ProSpace-VLM represents previously observed spatial information across changing viewpoints.

91. 【2611.25849】EgoWitness: Benchmarking Instance-Level Spatial Memory and Reasoning in Egocentric Video Streams

链接:https://arxiv.org/abs/2611.25849

作者:Zhengru Fang,Zehao Liu,Mingyu Tsoi,Yaoyu Zhu,Junlong Li,Fengmao Lv,Sijie Cheng,Yuanliang Sun,Fang Yang,Jun Zhang

类目:Computer Vision and Pattern Recognition (cs.CV)

关键词:instance-level spatial memory, days and people, leaving instance-level spatial, spatial memory largely, instance-level spatial

备注: 43 pages, including supplementary material. Zhengru Fang and Zehao Liu contributed equally. Project page: [this https URL](https://egowitness.github.io/) ; Code: [this https URL](https://github.com/egowitness/EgoWitness)

点击查看摘要

Abstract:Recalling where and when an object was last seen and who saw it requires recognizing it across days and people. Existing long-video benchmarks focus on events, object categories, or relative locations, leaving instance-level spatial memory largely untested in multimodal models. To address this gap, we present EgoWitness, a benchmark for instance-level spatial memory and reasoning in egocentric video. Its 8,453 human-verified questions span approximately 324 hours of week-long and short-horizon recordings and test seven abilities, from recognizing an object or room to recalling who saw it and reasoning about room visits. Over one million reviewed masks of 34 registered landmarks and 21 movable-object labels are linked through room timelines. Among twelve direct vision-language models (VLMs), open-weight models reach at most 46.97% accuracy, and none exceeds 35% on identifying who saw an object last. We therefore propose EgoClue, a spatial-memory agent that retrieves candidate intervals, verifies instance identity, and computes spatial and temporal relations with five tools. Two-stage training with supervised fine-tuning (SFT) and Group Relative Policy Optimization (GRPO) raises its 4B controller from 23.64% to 72.65% accuracy on 1,320 held-out questions. Evidence-trace synthesis matches the best direct VLM at 74.85% on week-long questions and achieves 80.56% in environments excluded from training.

92. 【2611.25848】Interpretable Prototype-Conditioned Adaptation for Weather Forecasting

链接:https://arxiv.org/abs/2611.25848

作者:Zhixuan Li,Hao Jiang,Chen Hui,Kai Geng,Weisi Lin

类目:Machine Learning (cs.LG); Computer Vision and Pattern Recognition (cs.CV)

关键词:requires distinguishing local, distinguishing local states, local, forecasting requires distinguishing, requires distinguishing

备注: 38 pages, including supplementary material. Project page: [this https URL](https://zhixuanli.github.io/projects/li2026protocast)

点击查看摘要

Abstract:Data-driven weather forecasting requires distinguishing local states and their temporal evolution. However, learned patterns remain implicit in neural forecasters, making their influence difficult to interpret. Failing to distinguish local evolution can lead to similar forecasts for locations that develop differently, increasing prediction error. This motivates explicit references for distinguishing and interpreting local patterns. Prototypes provide explicit pattern references, but sample- or task-level conditioning leaves open how to adapt to distinct locations and variable groups. To deal with this problem, we design ProtoCast, which uses group-specific prototype banks to adapt local features while accounting for cross-variable dependencies. Specifically, (1) we combine each group's historical features and temporal change with forecast context from cross-variable attention for evolution-aware prototype matching. (2) Sparse mixtures of prototype-specific low-rank operators tailor corrections to each group and location. A gated attention block then exchanges the corrected features across groups before prediction. Each group's correction decomposes into explicit prototype contributions before this fusion. (3) We jointly train the forecasting network and operators from scratch and refit each group's prototypes before every training epoch. Slot tracking preserves prototype-operator associations, making each prototype's contribution to local corrections more interpretable. Experiments on GlobalForecast and TyphoonTracking show improved forecasting accuracy, while prototype deletion links local feature contributions to changes in prediction error. This work advances the development of more accurate and interpretable data-driven weather forecasting models. Project page: this https URL

93. 【2611.25847】rajCal: Trajectory-Calibrated Time Schedules for Few-Step Flow Sampling

链接:https://arxiv.org/abs/2611.25847

作者:Junchi Zhang,Zihan Gu,Jiangcan Jia,Lulu Zhao,Wenbo Su,Hua Zhang

类目:Computer Vision and Pattern Recognition (cs.CV)

关键词:Flow matching enables, large neural networks, matching enables high-quality, process requires repeated, Flow matching

备注:

点击查看摘要

Abstract:Flow matching enables high-quality image and video generation, but its iterative sampling process requires repeated evaluations of large neural networks. Existing few-step methods can distort the output and reduce its agreement with a many-step reference. We propose TrajCal, a training-free calibration framework for few-step sampling. Based on scatter search, a well-established metaheuristic for difficult optimization problems, TrajCal starts with diverse seed schedules, deforms and combines these candidates, and selects one fixed schedule by final output fidelity for reuse on new inputs. For both image and video generation, TrajCal achieves higher reference fidelity than all compared baselines at low evaluation budgets while maintaining competitive visual quality. The search is effective even with randomly generated seeds, while trajectory-based seeds further improve fidelity. For cross-style generalization, photography-calibrated schedules also improve fidelity on unseen image styles.

94. 【2611.25840】Harnessing Medical Visual-Language Models with Agentic Evidence Management

链接:https://arxiv.org/abs/2611.25840

作者:Qiming Huang,Hanze Dong,Jipeng Zhang,Mike Zheng Shou,Xinxing Xu,Furu Wei

类目:Computer Vision and Pattern Recognition (cs.CV); Artificial Intelligence (cs.AI)

关键词:vision-language agents increasingly, agents increasingly rely, increasingly rely, recent work, work has focused

备注: 22 pages, 3 figures. Qiming Huang, Hanze Dong, and Jipeng Zhang contributed equally

点击查看摘要

Abstract:Medical vision-language agents increasingly rely on segmentation tools, and recent work has focused on acquiring the right mask, often treating a good mask as the end point of tool use. But do these agents use the resulting evidence effectively? To answer this question, we need a benchmark that separates evidence use from segmentation quality, spans multiple medical image types, and allows the same mask to be presented in different ways. Existing medical VQA datasets do not jointly provide these properties. We therefore construct Mask-Evidence MedVQA, a 2.5K-example benchmark spanning four image types, where each question is paired with an answer-relevant ground-truth mask and multiple possible renderings. Our study reveals two limitations of direct tool use. First, evidence presentation matters. Different renderings of the same fixed mask can substantially change model behavior. Second, masks introduce an evidence trade-off. Highlighting a relevant region can expose diagnostic evidence the model previously missed, but the added visual cue can also induce incorrect associations. As a result, even a ground-truth mask can correct some answers while turning initially correct ones into incorrect ones. Together, these findings show that acquiring the right mask is not enough. An agent must also decide how to present the evidence and whether it should change the answer. We therefore introduce TUNE (Tool Use is Not Enough), an evidence-management harness that explicitly manages what evidence to acquire, how to present it, and whether to accept the resulting answer update. Across three public medical VQA benchmarks, evidence management improves the fine-tuned Qwen3.5-9B backbone's mean accuracy by 4.0 percentage points. Full TUNE improves over direct tool use by 9.1 points on average across seven benchmarks and achieves competitive results against existing medical VLMs and agents.

95. 【2611.25838】ContextRenderer: Dynamic-Context Video Generation for Interactive Game Worlds

链接:https://arxiv.org/abs/2611.25838

作者:Runze Wang,Fei Zha,Bing He,Ke Xu,Zhongyuan Wu,Yuxuan Xie,Wenzheng Wu,Zhongyuan Liu,Youcheng Cai,Ligang Liu

类目:Computer Vision and Pattern Recognition (cs.CV); Graphics (cs.GR)

关键词:models separate executable, render engine-provided scene, world models separate, game world models, engine-provided scene representations

备注: 10 pages, 7 figures, 3 tables

点击查看摘要

Abstract:Interactive game world models separate executable world evolution from visual generation, using video models to render engine-provided scene representations into detailed observations. During sustained gameplay, however, existing generative renderers struggle to follow changing conditions while preserving visual continuity across diverse proxies and target styles. We therefore introduce ContextRenderer, an autoregressive video diffusion framework designed for both dynamic conditioning and cross-proxy generalization. It couples Dynamic-Context Learning (DCL) for chunk-level interaction supervision with Recurrent History Compression (RHC) to follow condition updates while preserving visual continuity within a bounded token budget. Across proxy representations, Cross-Representation Flow Alignment (CRFA) connects proxy reconstruction with appearance and interaction variation through a shared high-fidelity reconstruction. WorldPrism, our interaction-centric dataset constructed from agent-built worlds, AAA gameplay, and open-source videos, supplies the corresponding renderings and chunk-level annotations for these learning tasks. Experiments on GWM-Bench and Solar-Bench show improved reconstruction and visual quality, while qualitative results demonstrate minute-long interactive rendering under changing prompts and subject references. Project page: this https URL

96. 【2611.25834】Rank2Replace: VLM-Ranked Slot Replacement for Out-of-Distribution Generalization

链接:https://arxiv.org/abs/2611.25834

作者:Bo-Han Feng,Keng-Te Liao,Shou-De Lin

类目:Computer Vision and Pattern Recognition (cs.CV); Machine Learning (cs.LG)

关键词:high in-distribution accuracy, Spurious correlations, predictors high in-distribution, distribution shift, correlations can give

备注:

点击查看摘要

Abstract:Spurious correlations can give image predictors high in-distribution accuracy but poor generalization under distribution shift. We introduce $\textbf{Rank2Replace}$, which transfers knowledge of task-relevant evidence from pretrained vision--language models (VLMs) to small predictors through training-time augmentation for out-of-distribution (OOD) generalization. The VLM ranks object-centric slots by task relevance and guides rank-dependent replacement with same-rank slot embeddings from another image. This augmentation retains the original image's label, encouraging reliance on preserved task evidence as incidental cues change, and supports combination with existing OOD methods. Across four datasets and six splits under shared frozen slot embeddings, the default cross-entropy configuration reaches macro-averaged average-group and worst-group accuracies of $80.43\%$ and $65.28\%$ without environment labels or dataset-specific replacement tuning. Slot corruption improves both macro-averaged metrics for all fourteen OOD training methods. Combining it with slot ranking and IRM reaches $81.96\%$ and $72.68\%$. Controlled ablations support the ranking and replacement design, with gains across five ranking VLMs and six extractor--predictor combinations. Permutation-invariant predictors further improve both macro-averaged metrics over their unaugmented counterparts while using VLM guidance only during training for efficient inference.

97. 【2611.25824】Fusing Multimodal CNN Features with Structurally-Masked Transformers for Hierarchical Sub-Pixel Tree Species Mapping

链接:https://arxiv.org/abs/2611.25824

作者:Mattia Ferrari,Lorenzo Bruzzone

类目:Computer Vision and Pattern Recognition (cs.CV)

关键词:natural forest stands, inherent sub-pixel heterogeneity, Precise forest species, high spectral similarity, spaceborne hyperspectral data

备注: 19 pages, 6 figures

点击查看摘要

Abstract:Precise forest species mapping using spaceborne hyperspectral data is challenging due to high spectral similarity and the inherent sub-pixel heterogeneity of natural forest stands. While traditional deep learning models exploit spatial context through convolutional architectures, these approaches can introduce noise in fragmented or mixed-species forests. In this paper, we propose a hierarchy-aware, multimodal deep learning framework to resolve sub-pixel forest composition by integrating PRISMA (PRecursore IperSpettrale della Missione Applicativa) hyperspectral imagery, high-resolution panchromatic data, and digital elevation models. The architecture features a multi-branch encoder combining an EfficientNetV2-based one-dimensional convolutional neural network (CNN) for spectral signatures and twin two-dimensional CNNs for spatial and topographic features, coupled with a novel structurally-masked Transformer. By using a binary transition tensor to condition the Transformer slot embeddings and constrain inference, the model enforces taxonomic consistency between broad forest categories and fine-grained species distributions. The framework is validated across three diverse Italian biogeographical regions (the Alps and the Central and Southern Apennines) using field-surveyed plots. The proposed method consistently outperforms state-of-the-art multi-label and ranking-based paradigms, particularly in identifying subordinate species and their relative dominance. The bidirectional learning mechanism allows sub-pixel compositional features to refine macroscopic forest type predictions, while the integration of topographic priors effectively resolves spectral ambiguities in complex terrains. This knowledge-driven method supports regional-scale biodiversity monitoring and forest characterization within the evaluated PRISMA scenarios.

98. 【2611.25822】Limits of local curvature for constructing search subspaces in segmentation models

链接:https://arxiv.org/abs/2611.25822

作者:Maxim Bolshim(1),Alexander Kugaevskikh(1) ((1) ITMO University, Saint Petersburg, Russia)

类目:Computer Vision and Pattern Recognition (cs.CV); Machine Learning (cs.LG)

关键词:semantic segmentation models, gradient update field, field change, semantic segmentation, initial gradient

备注: 10 pages, 3 figures, 4 tables. Includes an appendix

点击查看摘要

Abstract:We examine how well subspaces constructed from local curvature represent the change in the gradient update field after a finite step in semantic segmentation models. The local response is the action of the curvature operator on the actual input displacement. In a study of 50 images and two models, we measure the ratio of the loss increase attained in a fixed subspace to that attained in the full space. This ratio decreases as the search radius increases. On ten images from the same sample, we compare the one-dimensional subspace spanned by the initial gradient with two rank-16 subspaces for both models. The two equal-rank subspaces are assessed at the same initial states, using the same actual steps and measured vectors. Adding generalised Gauss-Newton directions to the initial gradient reduces the mean relative error in representing the field change from 0.922 to 0.874 for the first model and from 0.819 to 0.763 for the second. At the same rank, including the response direction for the initial gradient nearly eliminates the error in representing the local response. The mean error in representing the field change does not decrease. A decomposition with fixed pixel weights shows that the finite-step remainder is comparable in norm to the field change and makes the largest diagonal contribution to the squared projection error of the field after the step. Evaluation of a search subspace should measure how accurately it represents the field change and the local response.

99. 【2611.25819】SCCV-6D: Sparse Canonical Coordinate Voting for Holistic Multi-View RGB-D 6D Pose Estimation in Robotic Bin Picking

链接:https://arxiv.org/abs/2611.25819

作者:Nico Leuze,Samed Dogan,Maximilian Hoh,Henry Schaub,Alfred Schöttl

类目:Computer Vision and Pattern Recognition (cs.CV)

关键词:picking requires pose, bin picking requires, partial occlusion, accurate under clutter, texture-less nature

备注: to appear in Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision (WACV), 2027

点击查看摘要

Abstract:Industrial bin picking requires pose estimates that are accurate under clutter, partial occlusion, and the reflective, texture-less nature of manufactured parts. Most recent approaches to 6D pose estimation follow multi-stage pipelines. These achieve high accuracy, but incur memory and runtime footprints that do not cope with the operational constraints of many real-world industrial deployments. Holistic pipelines estimate poses in a single pass, with cost controlled through indiscriminate subsampling or lowered resolution. Continuing this line, we present SCCV-6D, a fully sparse multi-view RGB-D estimator. We reconstruct the 3D space at millimeter precision, prune background via a staged heatmap mechanism, and predict class, translation offset and rotation at highest resolution for an arbitrary number of objects simultaneously. Rotation is formulated as canonical coordinate voting. Points vote for their canonical object coordinates with an associated confidence, enabling closed-form pose recovery with a symmetry-aware weighted Procrustes solve. Along an in-cluster point recovery induced by offset votes, this removes time-consuming post-processing stages. We reuse the task-supervised attention priors from the heatmap formulation for a novel attention-gated distillation objective that reweights teacher-student feature alignment. On the IPD test split we report competitive 0.826 AP at 1.13 s per scene, significantly faster than the leading entries. On MV-YCB our approach reaches 96.1 ADD-S, and remains competitive even on the densely packed bins of ROBI.

100. 【2611.25815】What Places an Action Boundary? Structure-Aware Attribution for Temporal Action Localization

链接:https://arxiv.org/abs/2611.25815

作者:Ruoshan Lan,Zongru Shao,Chieh Tsai,Mihai Surdeanu

类目:Computer Vision and Pattern Recognition (cs.CV)

关键词:Temporal action localization, identify actions, detection errors, remains a major, major source

备注:

点击查看摘要

Abstract:Temporal action localization (TAL) detectors must identify actions and estimate their temporal boundaries, but localization remains a major source of detection errors. While understanding what determines boundary placement requires faithful attribution of temporal evidence, existing attribution methods have not sufficiently addressed the temporal structure of TAL or the need to tailor attribution to boundary localization. We introduce a structure-aware attribution framework that averages contributions across random evidence-restoration orders following the Shapley-Shubik principle. We make it tractable by using the detector's receptive field to limit which time steps are ordered individually, and by approximating each order with a soft gradient path. For boundary predictions, we further introduce cell-structured restoration orders and boundary-anchored references that selectively remove adjacent evidence. We also adapt faithfulness evaluation to assess the joint effects of contiguous temporal evidence and the direction of boundary displacement. Experiments on THUMOS14 and EPIC-KITCHENS-100 with TriDet and ActionFormer demonstrate improved attribution faithfulness over existing methods. Attribution analysis and controlled interventions further reveal that class-score changes do not reliably track boundary displacement, providing input-level evidence that classification relies on content less informative about boundary placement. Code and evaluation artifacts will be publicly released.

101. 【2611.25814】What Does Anti-Hallucination Training Buy? Saying Less, Cheap Pruning, and the Rest

链接:https://arxiv.org/abs/2611.25814

作者:Lingkai Bu,Qian Gao

类目:Computer Vision and Pattern Recognition (cs.CV); Computation and Language (cs.CL)

关键词:reducing object hallucination, Preference training, lower CHAIR score, standard recipe, recipe for reducing

备注:

点击查看摘要

Abstract:Preference training is the standard recipe for reducing object hallucination in vision-language models, and each new method reports a lower CHAIR score. We ask what that reduction consists of. Every trained system is compared with its base model's caption truncated to the same object recall, which removes the part of the gain that comes from saying less; the remainder is compared with what a deliberately weak, frozen, eleven-feature post-hoc probe fitted on 500 annotated images can reproduce. All criteria were specified in the analysis code and frozen before the outputs they judge existed. For eight public methods on LLaVA-1.5-7B, the probe reproduces at least two thirds of what training removes beyond saying less for 7 of 8 methods on COCO, 8 of 8 on AMBER and 6 of 8 on Object HalBench, with RLAIF-V the consistent exception; a later public method released for both LLaVA-1.5 and Qwen2.5-VL (SENTINEL) meets the same bar on both. On our own DPO checkpoints the accounting holds on Qwen2.5-VL but fails on InternVL2.5 on COCO (2 of 4 training designs). At matched recall, the base model plus the probe beats truncation for 8 of 9 LLaVA-1.5 systems on COCO and 9 of 9 on AMBER, on the two newer backbones, and when realized as actual sentence deletion. The same measure prices training signals: DPO on the model's own samples labelled by the probe or by an object-level CLIP score recovers 0.70 and 0.87 of the beyond-brevity gain of ground-truth-labelled DPO on Qwen2.5-VL (the probe 0.73 on InternVL2.5), both beat length-matched random labels, and CLIP labels buy more than probe labels on COCO. Human judgements show that the sentences the probe deletes are wrong more often than those truncation would delete; a human check of the CHAIR evaluator's precision falls below our pre-set bar, which we report as a limitation.

102. 【2611.25802】World-delta: Continually Integrating Observations for Incremental 3D Scene Generation

链接:https://arxiv.org/abs/2611.25802

作者:Haonan Wang,Hanyu Zhou,Xiaobin Hu,Luxin Yan

类目:Computer Vision and Pattern Recognition (cs.CV)

关键词:methods synthesize visually, synthesize visually compelling, generation methods synthesize, visually compelling scenes, fixed inputs

备注:

点击查看摘要

Abstract:Existing 3D generation methods synthesize visually compelling scenes from fixed inputs, yet struggle to continually expand and revise a 3D world as new observations arrive. Iterative approaches often fuse real observations and generated content into a single representation, treating generated content as established structure rather than revisable predictions. Consequently, early generation errors can persist and accumulate despite new evidence. We propose World-delta, an incremental 3D scene generation framework combining uncertainty-aware persistent world modeling with delta-guided diffusion for continual scene expansion and correction. Our framework comprises three components: 1) Visual Evidence Encoding integrates geometric and semantic cues from incoming images into unified world tokens. 2) Observation-Calibrated World Modeling maintains a persistent latent world state that distinguishes observation-supported content from provisional completion hypotheses. New evidence drives selective world deltas to revise conflicting hypotheses while preserving well-supported content. 3) Incremental World Diffusion Generation uses delta-derived masks to restrict diffusion to regions requiring updates, freezing unchanged regions. Only affected 3D chunks are decoded and integrated into the existing scene. Together, these components enable continual world expansion and revision while reducing redundant generation and preserving scene consistency. Experiments demonstrate the effectiveness of World-delta in incremental 3D scene generation, observation-driven correction, scene consistency, and update efficiency over successive scene updates.

103. 【2611.25801】Fast and Accurate Body Estimation in Pixels, World Space and Scenes

链接:https://arxiv.org/abs/2611.25801

作者:Jinkun Cao

类目:Computer Vision and Pattern Recognition (cs.CV); Robotics (cs.RO)

关键词:Accurate Body estimation, jointly improves inference, improves inference efficiency, Momentum Human Rig, human body estimator

备注: 28 pages, 9 pages of main text. 16 figures

点击查看摘要

Abstract:We present FAB (Fast and Accurate Body estimation), a human body estimator that jointly improves inference efficiency and body-recovery fidelity. Rather than predicting only parametric mesh-model parameters, FAB directly predicts human skeleton joints and dense surface points. In images, it predicts dense 2D and 3D body points alongside a parametric Momentum Human Rig (MHR) head, then fits SMPL in one learned pass followed by two Gauss-Newton steps. Its ViT-L student, FAB-L, distilled from a fine-tuned SAM 3D Body, runs in real time for one person on an RTX 4070 Ti SUPER, from detection to a fitted body, at 4.6x the speed of SAM 3D Body's eager-PyTorch body-only path, while improving accuracy on most camera-space benchmarks. The same network predicts heel, toe, hand, and knee contacts, which connect the body to the scene. FAB also estimates world-space motion, fuses static-scene depth around the visible person, and aligns body and scene using predicted contacts. For human-to-humanoid real-to-sim transfer, an in-model G1 head and FAB-Retarget, a motion-level retargeter that reads dense points or the fitted mesh, map motion to a Unitree G1. FAB-Retarget produces references that follow the person more closely than specialized retargeters including SOMA, GMR, UMR, and OmniRetarget, and substantially more closely than Fast SAM 3D Body retargeted with our sequence IK. Project page: this https URL.

104. 【2611.25787】GeoPillars: A Large-Kernel Pillar Detector for Single-Sweep LiDAR Under Point Sparsity, and a Decomposition of Its Gain

链接:https://arxiv.org/abs/2611.25787

作者:Chaitanya Dumbre,Shubham Sarkar,Akash Mohapatra,Ghanasham Patil,Sheetal Girase

类目:Computer Vision and Pattern Recognition (cs.CV)

关键词:detector for single-sweep, geometric pillar features, hard budget, in-range points, adds geometric pillar

备注: 16 pages, 13 figures, 8 tables

点击查看摘要

Abstract:GeoPillars is a pillar-based 3D detector for single-sweep LiDAR under a hard budget of eight points per 0.4 m pillar, a budget that discards 46.6% of in-range points. It adds geometric pillar features, grouped vector attention within each pillar, a learned absolute positional prior on the bird's-eye-view (BEV) grid, a large-kernel backbone, auxiliary surface-normal supervision and a full-resolution detection head. Trained on 123k synthetic CARLA scans and evaluated on cars, it reaches 81.33 BEV AP@0.7, 13.76 points above CenterPoint at the same input budget and 1.91 points above CenterPoint at its native, denser budget. These are in-distribution figures, because the validation split is per-frame rather than scene-disjoint. A rung-by-rung decomposition shows that the new components explain little of the gain. Raising the detection-head resolution alone recovers 10.19 of the 13.76 points, the joint change of encoder, backbone, neck and positional prior adds 3.37, and the normal-supervision branch adds 0.20, which is within run-to-run variation. With heading reversals folded out, orientation error is similar for all seven models, although GeoPillars reverses headings about half as often as a resolution-matched baseline. It is also the slowest of the seven models. Evaluated zero-shot on 6,743 real nuScenes keyframes, no model transfers, and each retains 3.7% to 12.2% of its CARLA accuracy. A collection defect left LiDAR intensity constant, so every model learned from geometry alone. Code and checkpoints are publicly available.

105. 【2611.25776】Suppressing the Shortcut Hidden Behind the Gains of VideoLLM Fine-Tuning

链接:https://arxiv.org/abs/2611.25776

作者:Youngwoo Shin,Jongsuk Kim,Junmo Kim

类目:Computer Vision and Pattern Recognition (cs.CV)

关键词:Video Large Language, Large Language Models, Large Language, Video Large, Language Models

备注: 21 pages, 6 figures

点击查看摘要

Abstract:Supervised fine-tuning (SFT) is the stage in which Video Large Language Models (VideoLLMs) advance their capability to reason over video. However, the VideoLLM also learns to answer from textual regularities of its training data, a shortcut that bypasses the video. The regularities behind the shortcut are hard to define at the data level, and the shortcut forms regardless of training epoch, data size, or backbone. The shortcut is hidden behind a rise in accuracy within the training distribution, yet it is not required for the rise and even costs SFT the gain the video offers under distribution shift. We therefore propose Video-Evidence Training via Orthogonal projection (VETO), which intervenes on the fine-tuning update where the shortcut is written rather than on the data. VETO calibrates the subspace that the text alone occupies in the activations of the base model and projects the update onto its orthogonal complement. The VideoLLM is thus left to ground its answer in the full input, with the shortcut suppressed as it forms and never defined. Across seven VideoLLMs, VETO preserves the gain of SFT on the training distribution and suppresses the shortcut. On six benchmarks under shift it gains more than SFT and answers from the full input rather than the shortcut. Code is available at this https URL.

106. 【2611.25771】Driver Gaze Target Identification via Gaze Estimation and Structured Driving Scene Understanding

链接:https://arxiv.org/abs/2611.25771

作者:Xinze Cao,Ke Wang,Deyong Guan,Yuanquan Li,Ke Zhang

类目:Computer Vision and Pattern Recognition (cs.CV); Machine Learning (cs.LG)

关键词:intelligent vehicles, visual attention monitoring, visual attention, gaze, Identifying

备注: 14 pages, 6 figures

点击查看摘要

Abstract:Identifying a driver's gaze target provides object-level information for visual attention monitoring in intelligent vehicles. Gaze-zone classification and continuous attention localization provide limited information about the identity of the attended scene target. This paper proposes a modular framework that identifies object- or region-level gaze targets from synchronized in-cabin driver images and forward-view road images. The framework comprises four components: MFGaze, which combines a Vision Transformer pretrained with a masked autoencoder (MAE) and head-pose conditioning through feature-wise linear modulation (FiLM); panoptic scene understanding, which constructs a structured candidate set; polynomial ridge regression, which maps gaze features to 2D scene-image coordinates through individualized calibration; and entropy-driven target inference, which adaptively fuses boundary-distance geometry with class-level semantic priors. Experiments evaluate the framework using three public gaze datasets, a laboratory test using Cityscapes road scenes and self-collected gaze data, and the Look Both Ways (LBW) dataset. The complete framework achieves an overall Top-1 gaze target identification accuracy of 96.27% in the laboratory experiment. After sequence-specific calibration, it achieves Top-1 accuracies of 79.92% for task-relevant objects and 85.58% for the road class on LBW. MFGaze achieves the lowest mean angular errors among the compared methods, with 4.06$^\circ$, 4.23$^\circ$, and 10.52$^\circ$ on MPIIFaceGaze, ETH-XGaze, and Gaze360, respectively, and demonstrates strong cross-dataset generalization. Sensitivity analyses show stable performance under hyperparameter perturbations. These results support the framework as a source of object-level gaze information for driver monitoring.

107. 【2611.25761】AdaptReg: Skip-level FiLM Adapter for Task-Specific Adaptation of a Frozen Registration Foundation Model

链接:https://arxiv.org/abs/2611.25761

作者:Shuyue Shi,Rong Yuan,Lin Tian

类目:Computer Vision and Pattern Recognition (cs.CV); Machine Learning (cs.LG)

关键词:reflect treatment response, assessing treatment response, Longitudinal PSMA-PET, metabolic tumor volume, generic alignment accuracy

备注: 9 pages, 2 figures, 3 tables. Paper for the Learn2Reg 2026 PSMAReg challenge (PSMA-PET/CT longitudinal registration), 4th place among 16 teams on the hidden test set

点击查看摘要

Abstract:Longitudinal PSMA-PET/CT registration is a key step in assessing treatment response in prostate cancer, and it requires more than generic alignment accuracy: the deformation must preserve lesion volume, so that measured changes in metabolic tumor volume (MTV) and total lesion glycolysis (TLG) reflect treatment response rather than registration artifacts. We propose AdaptReg, which adapts a frozen registration foundation model to such task-specific requirements through a lightweight trainable adapter branch with identity-preserving initialization. Keeping uniGradICON fully frozen, we apply FiLM-style per-voxel affine modulation ($\gamma$, $\beta$) to its decoder skip connections; the modulation is generated from anatomical semantic features of a frozen TotalSegmentator encoder and supervised by a lesion volume-preservation loss. On the official Codabench validation set (20 pairs), AdaptReg achieves DSC 0.7585, HD95 6.33 mm, MTV/TLG errors of 1.68%/2.11%, and 0.0022% non-diffeomorphic voxels, improving on the uniGradICON + instance optimization baseline by $+0.05$ DSC and reducing the MTV/TLG errors by 70.9%/67.7%. On the hidden official test set (160 pairs), AdaptReg achieves CT DSC 0.816, PET DSC 0.598, and MTV/TLG errors of 4.02%/3.46%, ranking fourth among 16 teams in the final PSMAReg evaluation. Code: this https URL

108. 【2611.25747】SAM 3 Sees More Than It Detects: Promptable Concept Maps Beyond Categories

链接:https://arxiv.org/abs/2611.25747

作者:Yu-Huan Wu,Yun Liu,Liangli Zhen

类目:Computer Vision and Pattern Recognition (cs.CV); Artificial Intelligence (cs.AI)

关键词:SAM, Promptable segmenters answer, Promptable, Promptable segmenters, UNVEIL

备注: 18 pages, 5 figures, 15 tables

点击查看摘要

Abstract:Promptable segmenters answer what a thing is. Dense prediction tasks also ask class-agnostic questions, such as what stands out or hides, how light is blocked, transmitted or reflected, and where structure lies. Robots, surveillance systems and image editors build on their answers. We study promptable concept maps (PCM), which extend SAM 3's promptable concept segmentation from noun instances to maps of concepts named by short noun phrases such as `"transparent object", with "none" as an answer. PCM-Bench groups the standard test sets of seven such tasks by the question they ask. On it, we find that SAM 3 sees more than it detects. Its semantic path responds to many of these concepts, yet a low image-level presence score can block every detection of its instance path. We present UNVEIL, which trains a copy of SAM 3's semantic path for these maps, with LoRA on copies of its last blocks. A separate presence decoder learns from verified negatives whether a concept occurs. With one text phrase per query, UNVEIL maps region concepts such as salient and camouflaged objects, shadows, transparent objects and mirrors. The same weights map thin structures such as edges and skeletons as well as object categories. Run next to SAM 3, it leaves SAM 3's own outputs unchanged. On each of the seven tasks, UNVEIL reaches the highest score among the compared promptable models, including SAM 3's official fine-tuning and pipelines built on multimodal LLMs. Code is available at this https URL.

109. 【2611.25731】owards High-Fidelity Learned Video Compression

链接:https://arxiv.org/abs/2611.25731

作者:Bumyoon Kim,Yongseong Kim,Inhyuk Jeong,Byeungwoo Jeon

类目:Computer Vision and Pattern Recognition (cs.CV)

关键词:Professional video workflows, video workflows operate, Professional video, video workflows, learned video codecs

备注: 11 pages, 5 figures

点击查看摘要

Abstract:Professional video workflows operate at qualities that learned video codecs do not reach. We measure how much of a professional quality range a codec covers, from the lowest quality of the standard test conditions, VVC at QP 37, to the highest 4:2:2 grade of the production codec ProRes. To compare inter-frame coding alone, every codec receives original frames as references at the intra positions. On 10-bit YCbCr 4:2:2 test sequences, four recent learned codecs cover 24 to 38% of this range. Their encoders are feed-forward networks with a fixed latent capacity, the number of latent values per sample, of at most 0.25, and removing their quantization raises the quality by at most 1 dB. We propose a learned inter-frame codec whose encoder searches, as the encoders of conventional codecs do. It searches block motion, which a learned network refines, and it searches among learned block transforms whose latent capacity ranges up to 1, selecting the partition, the skip decisions, and the quantized coefficients by a rate-distortion cost. Our codec reaches the upper end of the range on all six sequences. With all residual blocks fixed to 16x16, whose latent capacity is 0.25, the same codec covers 55% of the range, with or without quantization.

110. 【2611.25730】CGM: Chaos-Guided Graph Attention and Expert Routing for Autism Stereotypy Detection

链接:https://arxiv.org/abs/2611.25730

作者:Hafsa Aslam,Yue Li,Saba Aslam,Abdur Rasool,Qiao Pan,Muhammad Rizwan,Ovidiu Bagdasar

类目:Computer Vision and Pattern Recognition (cs.CV); Machine Learning (cs.LG)

关键词:autism video analysis, Skeleton graph attention, video analysis weights, windowed Sample Entropy, Skeleton graph

备注:

点击查看摘要

Abstract:Skeleton graph attention for autism video analysis weights joints by content alone, giving an erratic wrist the same standing as a still one. Stereotypy, however, is defined by how unpredictably movement unfolds rather than how much of it occurs: in our cohort, every amplitude-based kinematic descriptor falls in stereotyped clips, while windowed Sample Entropy of angular velocity alone rises (Hedges' g = +1.05). We compute this irregularity as a per-joint, per-frame tensor that structurally drives computation. The proposed chaos-guided graph attention and mixture-of-experts model (CGM) couples two mechanisms. First, a learned pairwise term added to the graph-attention logit concentrates attention on irregular joints and re-ranks the remaining edge weight toward jointly irregular pairs; its unconstrained coefficient neither vanishes nor inverts, and zeroing the tensor realigns attention in 94% of clips. Second, a gate conditioned on the same signal routes each frame to two of four experts, whose weights track frame-level irregularity. Under subject-grouped five-fold cross-validation on 116 clips, CGM reaches 0.930 balanced accuracy, exceeding the graph-attention baseline by +0.187; the component ordering replicates on the public MMASD dataset, and on held-out Panoptic Studio sessions, CGM keeps 0.86 specificity without retraining, against at most 0.35 for chaos-free baselines. In ablation, routing without irregularity yields no reliable gain, whereas chaos modulation alone contributes +0.132. A severity index from the same tensor reaches 0.87 split-half reliability, separates the cohorts with Hedges' g = +1.08, and stays consistent within children across therapy recordings, supporting video-only monitoring of stereotypy during intervention.

111. 【2611.25728】What's in the box? Revealing Object Layouts Inside Opaque Boxes via Contactless Vibration Excitation and Sensing

链接:https://arxiv.org/abs/2611.25728

作者:Ethan Turok,Matan Kichler,Shai Bagon,Mark Sheinin

类目:Computer Vision and Pattern Recognition (cs.CV)

关键词:observing its surface, box, opaque box, surface vibrations, vibrations

备注:

点击查看摘要

Abstract:Can we reveal what is inside an opaque box by observing its surface vibrations? Objects inside a box alter its mechanical response, providing cues about their hidden locations. We introduce a method that uses these cues to predict a box's 2D occupancy layout entirely remotely. A nearby speaker excites vibrations, which we measure on one visible wall using a grid of laser points and a camera measuring each point's speckle-pattern shifts. Our model, WhisperBox, learns to encode these vibrations into a latent representation and decode it into an occupancy map of the box's contents. The method requires no physical contact to excite or sense vibrations. We demonstrate layout recovery across various boxes and single- and multiple-object scenes, with generalization to unseen object instances, positions, orientations, and weights, as well as changes in laser placement and excitation source. Our analysis reveals that informative frequency bands need not coincide with the box's resonances and that the learned latent space captures object placement and supports arithmetic operations. These results demonstrate the potential of remote vibration sensing to infer spatial layouts hidden from view.

112. 【2611.25724】Verifying Before Enhancing: Target Evidence Chain Verification for Infrared Small Target Detection

链接:https://arxiv.org/abs/2611.25724

作者:Jiayi Zuo,Songwei Pei,Shuhuai Wang,Yifei Wu,Shangguang Wang

类目:Computer Vision and Pattern Recognition (cs.CV); Artificial Intelligence (cs.AI)

关键词:Infrared small, Infrared small target, infrared small targets, space-based target perception, remote sensing

备注:

点击查看摘要

Abstract:Infrared small target detection is essential for early warning, surveillance, remote sensing, and space-based target perception. However, infrared small targets are typically tiny, weakly textured, and boundary-blurred, making them difficult to distinguish from complex backgrounds. Moreover, target-like background structures can produce salient responses across multiple scales, making response strength alone insufficient for reliable target discrimination. To address this challenge, we propose VBE-UNet, a Target Evidence Chain Verification Network that assesses candidate reliability beyond response magnitude through complementary evidence cues. Specifically, counterfactual background explanation models local target evidence, while joint evidence verification integrates cross-scale support and perturbation-based contribution verification to guide feature recomposition. The resulting reliability estimates guide selective feature refinement, promoting target-relevant responses while reducing unsupported background activations. Extensive experiments on IRSTD-1K, NUDT-SIRST, and NUAA-SIRST demonstrate that VBE-UNet achieves superior detection accuracy and lower false-alarm rates compared with state-of-the-art methods. Ablation studies and evidence reliability analysis further validate the effectiveness of the proposed verification framework.

113. 【2611.25722】DACA: Dual-Anchor Collaborative Agent for Zero-Shot 3D Visual Grounding

链接:https://arxiv.org/abs/2611.25722

作者:Jiaming Liu,Linghe Kong,Guixu Zhang,Huaxi Huang

类目:Computer Vision and Pattern Recognition (cs.CV)

关键词:human-computer interaction, task that bridges, bridges the natural, natural language, foundational capability

备注:

点击查看摘要

Abstract:3D visual grounding (3D-VG) is a task that bridges the natural language and 3D space, serving as a foundational capability for embodied intelligence and human-computer interaction. Although recent 3D-VG agents overcome the reliance on pre-processed 3D point clouds, they essentially employ a unidirectional, pipeline-style tool-calling architecture. This architecture separates semantic understanding from geometric reconstruction, leading to error accumulation and an inability to autonomously recover from failures. To address this flaw, this paper proposes the Dual-Anchor Collaborative Agent (DACA), which reinterprets 3D-VG as a semantic-geometric dual-anchor collaborative reasoning paradigm. DACA comprises two complementary specialized tools: the semantic anchor tool holds the visual semantic features of the target and achieves efficient target tracking via optical flow and VLM local verification; the geometric anchor tool holds the 3D spatial geometric features of the target and performs global spatial reasoning via voxelization and multi-keypoint projection. The two components exchange information bidirectionally via a standardized tool call protocol, autonomously deciding on the next perception-action step to form a closed-loop optimization. Experiments on the ScanRefer and Nr3D benchmarks demonstrate that zero-shot DACA achieves an overall Acc@0.25 of 73.6\% on ScanRefer and 70.4\% overall accuracy on Nr3D, far surpassing the latest zero-shot 3D-VG methods. The code and models are available at this https URL

114. 【2611.25719】SteerTrace: Tracing How Target Attacks Steer World Action Models

链接:https://arxiv.org/abs/2611.25719

作者:Yukun Dai,Tianshi Wang,Fengling Li,Jingjing Li,Xin Li,Lei Zhu

类目:Computer Vision and Pattern Recognition (cs.CV); Artificial Intelligence (cs.AI)

关键词:World action models, World action, targeted effect, safety signal, effect

备注: 10 pages, 5 figures, 5 tables

点击查看摘要

Abstract:World action models (WAMs) predict future videos and actions with one backbone, and their imagined future is often seen as a safety signal. We introduce SteerTrace, a targeted universal attack on WAMs that also traces its effect inside the model. One $\ell_\infty$-bounded (4/255) perturbation per model, shared across frames, tasks, and camera views, pushes the executed action toward an attacker-chosen target. On eight WAMs from four families (FastWAM, Faster-WAM, OpenWAM and LingBot-VA) on LIBERO and RoboTwin, it drives task success close to zero and moves the attacked action toward the target. Same-budget noise and BadWAM produce little action shift in the target direction. We use the signed targeted effect to measure how much of the effect each intervention removes or reproduces. Under the standard attack, restoring future tokens removes 61-94% of the targeted effect in the five models whose action experts attend to them. When the attack is constrained to keep the predicted future close to the clean prediction, the targeted effect can become more dependent on current-frame tokens. In three of these five models, restoring current-frame tokens removes 87-99% of the effect, compared with 9-15% for future tokens. In six models, restoring just 0.36% of the FFN neurons in layers 10-29 of the video expert removes 72-96% of the targeted effect. Residual directions 2-16 hold only 11-18% of the shift energy, yet removing them cancels most of the targeted effect.

115. 【2611.25713】Interpreting DeepStack-Style Vision-Language Models: Do Input or Deeper-Layer Visual Tokens Drive Answers?

链接:https://arxiv.org/abs/2611.25713

作者:Yucheng Du,Yi Nian,Ruixue Lian,Yue Zhao,Xiyang Hu

类目:Computer Vision and Pattern Recognition (cs.CV)

关键词:feed visual tokens, DeepStack-style vision-language models, main entry, vision-language models feed, models feed visual

备注: Includes the supplementary material. Code and numerical results: [this https URL](https://github.com/yucheng-du/interpreting-deepstack-style-vlms)

点击查看摘要

Abstract:DeepStack-style vision-language models feed visual tokens to the language model at its input, and again at deeper layers. We call these the main entry and the later entries. What do the later entries add when the model already has the image? We give the entries versions that differ in one object's color and see which one the answer follows. On photographs, the original DeepStack follows the later entries, while Qwen3-VL-8B/32B follow the main entry. Does weakening the main entry give the later features more influence? On synthetic images, we keep this conflict and reduce color saturation at the main entry only. The unchanged later features gain a stronger effect on the relative scores of the two colors, in all four models, including Granite Vision. This holds even under mild desaturation, when accuracy without the conflict stays at 100%. To test whether later features can also correct errors, we degrade both entries and restore only the later ones. The same blur costs both Qwen models 50 percentage points of accuracy on stripe orientation. The 32B model recovers 48 points and the 8B model only 8. Accuracy with one clean entry predicted which entry each model can lose, and five of six tested predictions held on new GQA attribute questions. With the other entry zeroed, Qwen3-VL-8B loses only 0.3 points on its main entry alone, and DeepStack at most 3.45 points on either entry alone. In split inference on these questions, DeepStack can then send only its main-entry features at higher precision. This takes 49% fewer bytes than all entries at a shared precision, at similar accuracy.

116. 【2611.25712】SkillLens: Action-Guided Dense Visual Dynamics for Proficiency Estimation

链接:https://arxiv.org/abs/2611.25712

作者:Deokhyun Ahn,Hyukjin Kim,Jae-Ho Han,Bumsub Ham,Heeseung Choi,Ig-Jae Kim,Haksub Kim

类目:Computer Vision and Pattern Recognition (cs.CV)

关键词:Human proficiency estimation, Human proficiency, challenging because skill, subjective property, abstract proficiency labels

备注: 6 pages, 4 figures, 6 tables, Project page: [this https URL](https://skill-lens.net/)

点击查看摘要

Abstract:Human proficiency estimation is challenging because skill is a high-level and often subjective property that is not captured by action identity alone. Existing approaches leverage multimodal observations such as ego-exo video, gaze, pose, and inertial signals, but the gap between sensor-level evidence and abstract proficiency labels remains difficult to bridge. We present SkillLens, a proficiency estimation framework that uses atomic action descriptions as temporal guidance, rather than direct semantic supervision, to localize proficiency-relevant windows for dense visual-dynamics extraction. Each window is summarized into an action-dynamics feature and scored by a high-low prototype margin against higher- and lower-proficiency executions from the same activity-action category. Fusing these ego/exo PDM cues with a global ego-exo-gaze baseline through CORAL ordinal calibration, our method achieves 0.542 accuracy on Ego-Exo4D. Our project page is available at this https URL

117. 【2611.25711】Calibrate in Context: Efficient Data-Free Quantization from Small Synthetic Pools

链接:https://arxiv.org/abs/2611.25711

作者:Haowen Hua,Sen Lin,Yaokun Huang,Zeyi Shao,Hao Zhang,Wei Sui,John See,Zeyd Boukhers,Cong Yang

类目:Computer Vision and Pattern Recognition (cs.CV)

关键词:enable accurate low-bit, accurate low-bit recovery, small synthetic pools, enable accurate, accurate low-bit

备注: 21 pages, 5 main figures. Supplementary material included

点击查看摘要

Abstract:We enable accurate low-bit recovery from small synthetic pools by improving both image synthesis and student preparation. A grid with lower local response error can incur greater error in the student's block context. Functional Grid Calibration (FGC) selects activation scales and integer offsets against full-precision responses constructed from the student's current block input. With weights fixed during calibration, FGC evaluates candidates only at their consuming operators, bringing block context to calibration at nearly local-scoring cost. For DeiT and ViT, progressive inversion updates the same images through increasingly deep teacher prefixes. With 1,000 images, the complete pipeline exceeds MimiQ's reported accuracy in all 18 W4A4/W5A5 settings across nine backbones. With 256 images, DeiT-S and ViT-S reach 68.20% and 66.90% ImageNet top-1 accuracy at W4A4 in 26-27 minutes on one RTX 5090. The pipeline outperforms strengthened 1K-image MimiQ by 13.03 and 24.36 percentage points at lower preparation cost and the same recovery-update count. The same low-budget checkpoints retain their advantage on ImageNetV2.

118. 【2611.25708】Slide2PPT: Native PowerPoint Reconstruction with Localization-Guided Refinement

链接:https://arxiv.org/abs/2611.25708

作者:Baoliang Chen,Xinlong Bu,Lingyu Zhu,Xiangjie Sui

类目:Computer Vision and Pattern Recognition (cs.CV)

关键词:native object structure, structure for subsequent, object structure, visual appearance, direct code generation

备注:

点击查看摘要

Abstract:Image-to-slide reconstruction must recover both visual appearance and native object structure for subsequent editing. Yet visually faithful charts and tables may lack editable series and cells, and direct code generation couples content prediction to rendering implementation. We present Slide2PPT, a model-agnostic, training-free framework that equips both open- and closed-source vision-language models with native PowerPoint reconstruction capabilities. Through a unified typed interface, the model predicts object records while a fixed renderer materializes their native properties, enabling integration with different backbones without task-specific training or architectural changes. Localization-guided refinement then connects visual discrepancies to object records for correction. Regional updates preserve unselected records, while whole-slide updates address cross-region relationships. Record refinement and targeted object editing support structural and attribute-level changes, with rendered feedback governing iterative acceptance. To evaluate this framework across diverse reconstruction demands, we introduce Slide2PPT-Bench, covering controlled elements, real slides, and scientific figures. Evaluation separately measures file production, rendered agreement, and native-object availability. Experiments across open- and closed-source backbones demonstrate substantially more reliable PPTX generation than direct code generation, improved layout and text recovery on scientific figures, and native chart and table interfaces beyond geometric primitives.

119. 【2611.25696】LIFT: Lifting Language-Controlled Image Fusion to Free-Viewpoint 3D Scenes

链接:https://arxiv.org/abs/2611.25696

作者:Jingning Tian,Hao Zhang,Yumin Huang,Zizhuo Li,Xingyu Jiang,Erdun Zhao,Jiayi Ma

类目:Computer Vision and Pattern Recognition (cs.CV)

关键词:diverse perceptual objectives, limiting comprehensive scene, comprehensive scene perception, fusion traditionally generates, perceptual objectives

备注:

点击查看摘要

Abstract:Multimodal image fusion traditionally generates a single fused image from paired observations at a specific viewpoint, limiting comprehensive scene perception and adaptability to diverse perceptual objectives. To overcome these limitations, we present LIFT, a novel framework for our proposed language-controlled 3D image fusion (L3D-IF) paradigm that extends multimodal image fusion from 2D images to free-viewpoint 3D scenes, enabling language-guided fusion tailored to diverse perceptual objectives. First, LIFT formulates Semantic Modality Binding (SMB), which interprets language instructions to identify semantic targets and their associated modality preferences and grounds these targets in corresponding regions of a shared Gaussian scene, establishing persistent semantic-to-spatial correspondences for consistent fusion control across viewpoints. Second, Language-Conditioned Gaussian Appearance (LCGA) is developed to achieve scene-wide multimodal appearance modeling by combining adaptive multimodal blending with language- and viewpoint-conditioned appearance refinement, enabling spatially adaptive realization of diverse fusion appearances while preserving source fidelity at pure-modality endpoints. Finally, we construct a Multi-View Language-Supervision (MVLS) dataset that associates language instructions and calibrated viewpoints with modulated fused images, providing effective supervision for learning instruction-dependent fusion from finite multi-view observations. Extensive experiments demonstrate that LIFT achieves coherent multimodal fusion across continuous viewpoints, accurate semantic target localization, and consistent regional fusion control under diverse language instructions.

120. 【2611.25689】D3-Cache: Accelerating Diffusion Models with Dynamic Mode Decomposition

链接:https://arxiv.org/abs/2611.25689

作者:Yanbo Li,An Xiang,Xiaoxin Hu,Feng Wang,Cheng-zhong Xu,Kejiang Ye,Xitong Gao

类目:Computer Vision and Pattern Recognition (cs.CV)

关键词:Transformers generate high-quality, Diffusion Transformers generate, full network evaluation, Transformers generate, Diffusion Transformers

备注:

点击查看摘要

Abstract:Diffusion Transformers generate high-quality images and videos, but every sampling step requires a full network evaluation. Training-free caching reduces this cost by reusing or forecasting network outputs across steps, but common approaches use per-model calibrated skip criteria or maintain per-layer prediction histories. We present D3-Cache, which applies Dynamic Mode Decomposition (DMD) to Diffusion sampling: it fits DMD to a short history of exact velocities and forecasts only the velocity that the scheduler consumes. A horizon-aware gate decides when to skip: whenever the network is evaluated, its output is compared with the forecast for that step, and the larger the error, the sooner the network is evaluated again, up to a cap on consecutive skipped steps. D3-Cache requires no training or per-model offline calibration curve and adds negligible memory. It shares one history length and risk tolerance across the four evaluated models. On four state-of-the-art image and video models, D3-Cache achieves speedups of 2.6-3.8x with high fidelity to the base model output.

121. 【2611.25680】Beyond Entity Recognition: Vision-Centric Knowledge Visual Question Answering

链接:https://arxiv.org/abs/2611.25680

作者:Jiatong Ma,Yuchen Liu,Longteng Guo,Yichen Yuan,Yuze Wu,Dongze Hao,Ru Zhen,Yanhao Zhang,Haonan Lu,Jing Liu

类目:Computer Vision and Pattern Recognition (cs.CV); Artificial Intelligence (cs.AI)

关键词:Multimodal Large Language, Large Language Models, Large Language, Multimodal Large, Language Models

备注: Accepted to EACL2027(Main)

点击查看摘要

Abstract:We propose VC-VQA, a large-scale benchmark for vision-centric knowledge reasoning, aiming to systematically evaluate the capabilities of Multimodal Large Language Models (MLLMs) in fine-grained visual knowledge acquisition and complex multi-stage reasoning. Unlike existing benchmarks that primarily treat visual information as entity recognition and rely on text-based external knowledge, VC-VQA explicitly models the role of visual information by considering both its source (internal vs. external images) and stage of involvement (pre-retrieval vs. post-retrieval). Specifically, VC-VQA requires models to retrieve external visual evidence, identify visual clues before retrieval, and localize, verify, or match retrieved knowledge against the internal image. We further construct a controlled multimodal knowledge base with traceable textual and visual evidence for each sample. We evaluate 21 representative MLLMs under four settings: no external knowledge, with provided evidence, knowledge base retrieval, and open-web search. Experimental results lead to three key findings: current models still struggle with vision-centric knowledge reasoning; fine-grained visual knowledge acquisition remains the primary bottleneck; and agentic retrieval highlights the value of multi-step planning and cross-modal knowledge integration. Overall, VC-VQA provides a more challenging benchmark for evaluating vision-centric knowledge reasoning in MLLMs. Our code and dataset are available at this https URL.

122. 【2611.25675】Minimal Memory for Correctable Visual Continuation

链接:https://arxiv.org/abs/2611.25675

作者:Xiyang Zhang,Yuanhe Tian,Hongzhi Wang,Yan Song

类目:Computer Vision and Pattern Recognition (cs.CV)

关键词:evolving scene state, visual continuation generates, generates future frames, continuation generates future, generates future

备注:

点击查看摘要

Abstract:Given past images and known actions, visual continuation generates future frames from one evolving scene state. Keeping this state revisable matters when an object reappears or a later view changes an earlier interpretation. Once past images cannot be replayed, a summary sufficient for current rendering can lose candidates needed after correction. We establish matching sparse and dense bounds on the bits needed for later correction under stable observations and bounded disagreement. For static streams at the stated smoothness and margin scales, correction incurs at least logarithmic overhead in one dimension and polynomial overhead in higher dimensions. Coherent Archive Memory (CAM) stores geometric summaries with exact updates under clipped expansion and coordinate assignments, then uses historical existence constraints to select one state for all requested frames. Exhaustive photographic tests show strict worst-truth image-risk gains from joint witnesses, while controlled neural continuations recover positions hidden in the current view.

123. 【2611.25672】SEAF: Selective Semantic Enrichment and Visual Evidence Fusion for Multimodal Sarcasm Detection

链接:https://arxiv.org/abs/2611.25672

作者:Faxian Wan,Xiaocui Yang,Yifan Cao,Shi Feng,Daling Wang,Yifei Zhang

类目:Computer Vision and Pattern Recognition (cs.CV); Computation and Language (cs.CL)

关键词:Multimodal sarcasm detection, detection requires textual, provide complementary cues, sarcasm detection requires, multiple images provide

备注:

点击查看摘要

Abstract:Multimodal sarcasm detection requires textual, visual, and contextual evidence, particularly when multiple images provide complementary cues. Multimodal large language models (MLLMs) can generate auxiliary information from a post's text and images, but their outputs may mix useful background knowledge with redundant visual descriptions. We propose SEAF, a dual-branch framework for selective semantic enrichment and visual evidence fusion. SEAF assigns generated fields to distinct roles: entity background enriches semantic input, while image descriptions supply alignment statistics. The semantic branch combines the original post text, background from a frozen MLLM, and optical character recognition (OCR) text. The aligned visual branch combines global and local visual features with statistics of cross-modal and inter-image similarity. Text-Anchored Smoothed Aggregation (TASA) weights images by their similarity to the original post text while preserving contributions from images with low literal similarity. Summing the branch classification scores integrates contextual semantics with visual evidence. Experiments on the multi-image MMSD3.0 benchmark show that SEAF outperforms all compared methods.

124. 【2611.25670】Efficient Latent Reasoning for Vision-Language Model via Dynamics-Aware Quantization and Distillation

链接:https://arxiv.org/abs/2611.25670

作者:Yanjing Li,Sheng Xu,Baochang Zhang,Xianbin Cao,Yawei Li

类目:Artificial Intelligence (cs.AI); Computer Vision and Pattern Recognition (cs.CV)

关键词:intermediate reasoning steps, requires reducing, reducing both intermediate, execution cost, vision-language models

备注:

点击查看摘要

Abstract:Efficient chain-of-thought (CoT) reasoning in vision-language models (VLMs) requires reducing both intermediate reasoning steps and their execution cost. Latent CoT methods compress textual rationales into continuous thoughts, yet their full-precision (FP) models retain substantial storage and computation costs. Lowering numerical precision could reduce these remaining costs, yet direct quantization severely degrades reasoning accuracy, particularly with 4-bit weights and activations. Operator and perturbation analyses reveal key sensitivity and direction-dependent predictive effects of continuous-thought errors, motivating preservation of information retrieval and subsequent reasoning beyond reconstruction accuracy. We therefore introduce QL-VLM, a dynamics-aware Quantization framework for efficient Latent reasoning in Vision-Language Models (VLMs). First, Dynamics-Guided Quantization (DGQ) uses nonuniform codebooks that preserve rotary-key structure, mitigating quantization-induced distortion in vision-language information retrieval. Second, Dynamics-Preserving Distillation (DPD) aligns continuous thoughts, latent-query readouts, and predictions with a full-precision teacher, maintaining predictive consistency through subsequent low-precision reasoning. On ScienceQA-IMG with Qwen2-VL-2B-Instruct, 4-bit QL-VLM achieves 90.43% accuracy and improves baseline by 40.33%, with 16.64x fewer bit-operations and 6.40x lower measured latency than explicit CoT.

125. 【2611.25667】CauSyncVLA: Causal Latent World Modeling for Asynchronous Robot Control

链接:https://arxiv.org/abs/2611.25667

作者:Xu Pan,Chubin Zhang,Xingrui Yu,Runmin Lan,Zihao Wang,Wenkai Guo,Bofang Jia,Bo An,Ziwei Wang

类目:Robotics (cs.RO); Computer Vision and Pattern Recognition (cs.CV)

关键词:policies must react, VLA, Causal Latent World, Latent World Model, Abstract

备注:

点击查看摘要

Abstract:Vision-Language-Action (VLA) policies must react while visual inference is still running. Overlapping computation and execution reduces waiting, yet leaves perceptual memory stale and successive action chunks disconnected. We present CauSyncVLA, an asynchronous dual-system controller that aligns perceptual-memory prediction and trajectory correction with actual robot execution. Rather than forecasting pixels or physical states, a Causal Latent World Model predicts current perceptual memory from delayed visual anchors and executed state-action history, supervised through action-sensitive distillation to retain the frozen expert's control response. Conditioned on this updated memory, Predictive Flow Warm-Starting (PFWS) aligns the unexecuted action horizon for single-step flow correction without extra training. On LIBERO-10, CauSyncVLA reaches 93.8% success at 5.04$\times$ inference throughput, improving closed-loop robustness and robot motion smoothness.

126. 【2611.25650】Extending Rademacher Complexity-Based Generalization Theory to Domain Generalization

链接:https://arxiv.org/abs/2611.25650

作者:Xi Wang,Liang Bai,Xian Yang,Richard Yi Da Xu

类目:Machine Learning (cs.LG); Computer Vision and Pattern Recognition (cs.CV)

关键词:ensure robust performance, multiple source domains, unseen target domains, Domain generalization, domain space

备注:

点击查看摘要

Abstract:Domain generalization (DG) aims to train models on multiple source domains with distinct distributions to ensure robust performance on unseen target domains. In doing so, it elevates the concept of generalization from the traditional sample level to the distribution level. Existing theoretical frameworks introduce a domain space endowed with a meta-distribution to extend classical generalization analysis to this new setting. However, these approaches typically treat the domain space as devoid of geometry. In reality, the domain space is an inherently infinite-dimensional, nonlinear abstract space composed of probability distributions, and its complex geometric properties fundamentally determine generalization behavior. Consequently, existing geometry-free theories of domain generalization have a fundamental limitation. They lack a formal characterization of what distinguishes distribution-level generalization from sample-level generalization, leaving the inherent difficulty of generalizing across domains largely unexplained. To bridge this gap, we formalize the domain space as the Wasserstein space over the representation space. Building on the geometric properties of this space (specifically the $\mathrm{CD}(K,\infty)$ curvature theory) we derive a novel, geometry-dependent DG error bound. By introducing an additional curvature-dependent confidence term, this bound formally disentangles the two levels of generalization effects and reveals the essential nature that domain generalization is affected by both the invariance and the curvature of the representation space. Finally, we experimentally validate the effect of curvature on domain generalization.

127. 【2611.25648】RACE: Attribute Execution for Sequential Image Editing

链接:https://arxiv.org/abs/2611.25648

作者:Xiyang Zhang,Yuanhe Tian,Hongzhi Wang,Yan Song

类目:Computer Vision and Pattern Recognition (cs.CV)

关键词:successive user instructions, preserves earlier requirements, remain active, editing should return, return an updated

备注:

点击查看摘要

Abstract:Given an image and successive user instructions, sequential image editing should return an updated image that satisfies each new request and preserves earlier requirements that remain active. This lets users develop a result without repeatedly restating or repairing their decisions. The difficulty is that a retrieved image can contain a valid color on an obsolete surface, while the current image may lack the newly requested material. Neither reference selection nor a precise object mask alone specifies how to reconcile attributes occupying the same region. We present TRACE, which separates the continuing attribute from its visual carrier and prioritizes local appearance operations under complete-request checks. Active-goal binding identifies what should persist, carrier selection supplies suitable current or newly generated pixels, and tonal calibration reduces tinting near highlights and shadows. Experiments across frozen editors, independent visual readers, new source images, and complete sequential benchmarks support joint color-material recovery and show how local execution preserves the surrounding scene.

128. 【2611.25644】MambaLIFT: Reference-Based Mechanism-Aware Attribution for VMamba

链接:https://arxiv.org/abs/2611.25644

作者:Jintu Kakati,Niladri B. Puhan

类目:Artificial Intelligence (cs.AI); Computer Vision and Pattern Recognition (cs.CV); Image and Video Processing (eess.IV)

关键词:couple feature selection, visual state-space models, state-space models remains, models remains challenging, input-dependent selective dynamics

备注:

点击查看摘要

Abstract:Attributing predictions of visual state-space models remains challenging because input-dependent selective dynamics couple feature selection, recurrent state propagation, and shared computational pathways. We introduce MambaLIFT, a reference-based, multiplier-first attribution framework for VMamba that propagates contributions through its computational graph using interaction-aware rules for selective-scan injection, recurrent memory, LayerNorm, and shared pathways. MambaLIFT decomposes the reference-relative class-logit difference, \(f_c(x)-f_c(x^0)\), by spatial position, VSS block, scan direction, and computational mechanism. Experiments on ImageNet-1K evaluate VMamba-T and VMamba-B, with hyperparameters selected on a calibration set and faithfulness assessed on a disjoint held-out evaluation set. MambaLIFT achieves numerical conservation to floating-point precision across audited attribution interfaces in both configurations. Under mean-colour pixel flipping, it achieves significantly higher mean faithfulness scores than MambaLRP on both configurations and the highest scores among the evaluated methods, including Integrated Gradients with matched references. Under blurred replacement, MambaLIFT is non-inferior to MambaLRP on VMamba-T and equivalent on VMamba-B, indicating that its advantage depends on the replacement protocol. Ablations on VMamba-T show that interaction allocation, temporal propagation, and pathway composition affect attribution maps. Mechanism-level analyses further show that exact relevance conservation does not imply additive effects under finite interventions: the mechanism decomposition accounts for the reference-relative logit difference but does not estimate intervention effects or establish causal validity.

129. 【2611.25640】Erasing the Wrong Thing: Conditional Attribute Leakage and Concept Erasure in Shared Foundation-Model Embeddings

链接:https://arxiv.org/abs/2611.25640

作者:Haijun He,Fei Li,Donghong Ji,Junjun He

类目:Cryptography and Security (cs.CR); Artificial Intelligence (cs.AI); Computer Vision and Pattern Recognition (cs.CV); Machine Learning (cs.LG)

关键词:authorized downstream tasks, Medical-image embeddings, increasingly shared, shared in place, place of raw

备注:

点击查看摘要

Abstract:Medical-image embeddings are increasingly shared in place of raw images for authorized downstream tasks, yet they may reveal sensitive attributes. Standard concept-erasure methods target population-level guardedness. However, we prove that, when binary task and sensitive labels are associated, global and task-conditional linear guardedness cannot both hold while a nonzero linear task signal is preserved, so global erasure must either eliminate that signal or retain task-conditional linear leakage. We further show how global erasure affects linear readers' discrimination between positive and negative samples with different sensitive attributes, deriving an approximate closed-form prediction of its utility cost from pre-erasure statistics. Motivated by this conflict, we introduce Task-Conditional LEACE (TC-LEACE), a closed-form variant of LEACE that enforces linear guardedness within each task stratum of its fitting data. We evaluate four vision foundation models on 85 CelebA attribute pairs spanning a broad range of associations. After LEACE, conditional leakage persists across all four encoders, the discrepancy between marginal and conditional audits increases with association strength, and predicted utility costs track observed losses. We then examine hospital chest radiographs using three foundation models, including two biomedical models, in which chest-tube status is strongly associated with diagnosis. Although a marginal attack on LEACE-erased embeddings underperforms a diagnosis-only reference, chest-tube status remains detectable within diagnosis strata. TC-LEACE recovers most of LEACE's linear task-utility loss while keeping task-conditional leakage within a prespecified margin of LEACE's.

130. 【2611.25639】LG-FiLM: Local-Global Mixture-of-Experts Modulation for RAW-to-RGB Reconstruction

链接:https://arxiv.org/abs/2611.25639

作者:Fei Li,Xilei Cao,Shichang Zhai,Kun Zhan

类目:Computer Vision and Pattern Recognition (cs.CV)

关键词:image signal processor, image signal, reconstruct an image, signal processor, image

备注:

点击查看摘要

Abstract:An image signal processor does more than reconstruct an image. It also has to decide how strongly to denoise, how to balance color, and which tone curve maps the scene onto the display, and these settings depend on the scene itself, the curve that suits a backlit sky is not the one that suits an indoor portrait. Most neural ISPs learn a single fixed transformation and apply it to every scene. We instead make the adaptation an explicit part of the network. Our model, LG FiLM, keeps one shared encoder decoder and places a small mixture of affine experts at its bottleneck; the experts scales, shifts, and mixing weights are predicted from local neighborhood evidence together with a global scene descriptor. Reconstruction stays in a single shared pathway and only the processing mode is switched. We evaluate LG FiLM on three datasets, Fujifilm UltraISP, Zurich RAW to RGB, and a COCO derived benchmark built with an inverse ISP. The model reaches the best reconstruction fidelity of the compared entries on the primary benchmark and transfers to the other two.

131. 【2611.25635】AesDistill: Distilling Generative Aesthetics for Image Color Enhancement

链接:https://arxiv.org/abs/2611.25635

作者:Chuanzhi Xu,Chengkun Yue,Huiqi Liang,Haoyang Li,Zihan Deng,Qingwen Zeng,Pengfei Ye,Weidong Cai

类目:Computer Vision and Pattern Recognition (cs.CV); Artificial Intelligence (cs.AI)

关键词:improve visual appeal, reproduce ground-truth expert, aims to improve, reproduce ground-truth, Image

备注:

点击查看摘要

Abstract:Image color enhancement aims to improve visual appeal, but previous methods often learn to reproduce ground-truth expert targets. These targets reflect the preferences of a few experts and cover a limited range of aesthetic choices. Matching them does not ensure stronger aesthetic quality across different tastes and scenes. By contrast, large generative models learn rich visual knowledge from large image collections and often produce vivid, appealing image edits. However, direct generation can change the source image's semantic structure along with its colors. In this paper, we introduce AesDistill, a framework for distilling generative aesthetics into image color enhancement. We use 4 leading open-source large generative models to produce 64,644 edited images as teacher targets at a cost of about 3,360 H100 GPU-hours, forming the AesDistill Dataset. AesDistill also includes AesDistill Student, a 1.13M-parameter model that learns teacher color changes through confidence-weighted supervision while preserving the source image's semantic structure. We evaluate AesDistill with 12 aesthetic and 4 image-quality assessment models. Every teacher's targets rank above the FiveK and PPR10K expert target means in the joint assessment. Across both datasets, AesDistill Student outperforms all 20 baseline methods and ranks directly behind the 4 teachers. These results demonstrate that distilling aesthetics from generative models enables a compact student to outperform methods trained on expert targets.

132. 【2611.25632】What One Sample Cannot See: Why Fewer Denoising Steps Can Score Higher on Physics Benchmarks

链接:https://arxiv.org/abs/2611.25632

作者:Yutao Li,Chen Wang

类目:Computer Vision and Pattern Recognition (cs.CV); Machine Learning (cs.LG)

关键词:north or south, plausible path, storm can head, head north, squared error

备注:

点击查看摘要

Abstract:When a storm can head north or south, a forecast down the middle may score better under squared error than either plausible path. The same paradox appears when video generators are tested as world models: fewer denoising steps can raise physics scores even as videos blur or stop moving early. Why? Physics-IQ compares each generated video with one recording, although a scene can have multiple valid futures. An exact decomposition shows its scores can reward reduced variation among generated futures, even when their distribution becomes less accurate. We introduce Kernel Physics-IQ to remove this incentive in expectation, needing only two independent samples per condition. Two-step sampling lowers Kernel Physics-IQ in all five image-to-video models. For CogVideoX, the official score rises by 0.055 while ours falls by 0.131. Changing the sampling schedule reveals further differences official scores miss. In controlled physical systems with known outcome probabilities, our metric favors the true distribution over collapsed predictions. In short, a world model should be judged not only by how closely one video matches what happened, but by how faithfully its predictions represent what could have happened.

133. 【2611.25628】RelGeo: Pair-Conditioned Relational Calibration for Frozen Diffusion Correspondence

链接:https://arxiv.org/abs/2611.25628

作者:Zixuan Dai,Zhiliang Zhang,Daosong Hu,Tang Zheng,Xinkai Liu,Tianyu Lei,Wei Zhan

类目:Computer Vision and Pattern Recognition (cs.CV)

关键词:valid high-ranking alternatives, retrieve strong semantic, strong semantic correspondence, select incorrect locations, semantic correspondence candidates

备注: 22 pages, 13 figures, including supplementary material

点击查看摘要

Abstract:Frozen diffusion descriptors retrieve strong semantic correspondence candidates, yet cosine nearest-neighbor matching can select incorrect locations despite valid high-ranking alternatives. Across DIFT-SD2.1 and DiTF-FLUX on SPair-71k, AP-10K, and SOCO (CC, SOC, Cross-SOC), 41-69% of Top-1 errors retain a PCK@0.10-valid match among the original Top-5 spatial peaks, revealing a persistent candidate-selection bottleneck. We find that descriptor rankings and current-pair relational evidence can favor different candidates: reciprocal diffusion Query/Key associations and pretrained RoMa transport exhibit distinct, partly complementary preferences. To incorporate this evidence into matching decisions, we introduce RelGeo, a Pair-Conditioned Relational Calibration framework expressing heterogeneous relations as shared source-target coordinates. Spectral Relational Coordinates (SRC) encode reciprocal diffusion affinities, while Warp Identity Coordinates (WIC) encode reliability-weighted bidirectional transport endpoints. Their inner products augment frozen descriptor similarity while preserving full-domain cosine nearest-neighbor matching without additional semantic-correspondence training. RelGeo improves all ten evaluations by an average of 5.51 PCK@0.10 points; fixed-candidate reranking and pair-misalignment controls corroborate improved selection and dependence on current-pair relational alignment.

134. 【2611.25626】Physical Laws without Physics Loss? Probing Learned Dynamics in Video World Models through Forced Response

链接:https://arxiv.org/abs/2611.25626

作者:Yutao Li,Chen Wang,Wangmeng Zuo,Hui Li

类目:Computer Vision and Pattern Recognition (cs.CV); Machine Learning (cs.LG)

关键词:world models learn, models learn physical, learn physical laws, video world models, world models

备注:

点击查看摘要

Abstract:Do video world models learn physical laws without a physics loss, or merely imitate motion? Overall motion and appearance scores cannot settle it. We investigate this through nonlinear forced response: an excitation shakes the support of a suspended object. We introduce dual visual-numeric conditioning and adapt six image-to-video backbones to predict this motion using only the flow-matching loss. Wan predicts motion accurately and generalizes to unseen excitations, objects, poses, and scenes. Counterfactual tests suggest capabilities beyond replaying training videos. But what laws have they learned? We design three tiers of tests using Taylor expansions of the dynamics and geometry. All six reproduce the linear response, including resonance; five capture higher-order dynamics, including nonlinear softening. Remarkably, softening is barely present in training: only 0.74% of the swing's squared displacement. Extensions to a double pendulum and a rocking block show sustained dynamics are learned more faithfully than brief impacts. Yet learned dynamics may fail to appear when state estimation or propagation goes wrong. We turn these tests and findings into strategies to improve video training and generation.

135. 【2611.25621】DAWN: Robust Neural Blind Deconvolution via Alternating Wiener Guidance

链接:https://arxiv.org/abs/2611.25621

作者:Yida Pan,Yixuan Jia,Chaoyan Huang,Qing Qu,Ismail Alkhouri

类目:Computer Vision and Pattern Recognition (cs.CV); Artificial Intelligence (cs.AI)

关键词:Blind image deconvolution, single degraded observation, ill-posed inverse problem, Blind image, degraded observation

备注:

点击查看摘要

Abstract:Blind image deconvolution (BID) is an ill-posed inverse problem that aims to jointly recover a sharp image and blur kernel from a single degraded observation. Under nontrivial and high noise, which is common in photography and scientific imaging such as microscopy, BID remains particularly challenging, as existing methods suffer from noise overfitting and inaccurate kernel estimation. Existing training-data-free learning methods for BID, such as those based on the deep image prior (DIP), often utilize external guidance by including model-based estimates as additional loss terms, which underexploits the information these estimates carry especially under nontrivial noise. To address these challenges, we propose DAWN, a training-data-free DIP framework that alternates between injecting model-based estimates directly into the model input and refining this imperfect estimate with the network's own prior. To provide more reliable kernel estimates for these injections, we apply autoencoding regularization to both the image and kernel networks. On natural and scientific images under varying levels of Gaussian and Poisson noise, DAWN outperforms state-of-the-art methods in both image and kernel quality (achieving up to +4.5 dB over the next best baseline on the Levin dataset at $\sigma = 0.1$). Without any training images, DAWN remains highly competitive with a state-of-the-art diffusion model on FFHQ, despite the latter being trained on a massive number of in-distribution images.

136. 【2611.25620】CaT-Movie: Balancing Global Planning and Local Rendering for Long Multi-Shot Video Generation

链接:https://arxiv.org/abs/2611.25620

作者:Jinpeng Lu,Haoyuan Shi,Yuanshen Guan,Jiyuan Xia,Bin Wu,Yinda Chen,Zhiwei Xiong

类目:Computer Vision and Pattern Recognition (cs.CV)

关键词:Current multi-shot video, visual identity drift, video generators render, requiring long movies, multi-shot video generators

备注: 31 pages

点击查看摘要

Abstract:Current multi-shot video generators render only limited durations per pass, requiring long movies to be stitched from local segments across which narrative progression and visual identity drift. To synthesize long-horizon movies with Coherence across Time, we present CaT-Movie, a multi-shot generation system consisting of a global story planner and a local window renderer. The planner expands user requests via a large language model to preserve, infer, and complete story details, while deterministic code maintains the resulting storyboard across shots to compile self-contained shot prompts and index reference frames from generated history. To execute this guidance, the renderer applies reference-routed sparse self-attention and shot-wise cross-attention to route identities from the indexed frames while isolating per-shot textual conditions. Given the scarcity of high-quality open-source multi-shot data, we construct a dataset of over 350K film shots annotated with semantic state transitions, entity grounding, and multi-granularity requests. We further build CaT-MBench, a benchmark on held-out films with the same annotations. It gives every baseline its native format and counts cross-shot consistency only when a cut opens a new camera setup, preventing repeated framing from passing as consistency. On CaT-MBench, CaT-Movie attains the highest average score and the most consistent characters among all methods. The project page is available at this https URL.

137. 【2611.25595】LoopedWAM: Looped World Action Models for Elastic Inference

链接:https://arxiv.org/abs/2611.25595

作者:Anh Dao,Xinxin Liu,Hang Yu,Jeffrey A Chan Santiago,Yuzhang Shang

类目:Computer Vision and Pattern Recognition (cs.CV); Robotics (cs.RO)

关键词:limiting deployment efficiency, World action models, couple generative video, generative video modeling, robotic action prediction

备注:

点击查看摘要

Abstract:World action models (WAMs) couple generative video modeling with robotic action prediction but typically require large video and action transformers, limiting deployment efficiency. We present LoopedWAM, a compact WAM that recurrently applies a shared six-block core in each expert, executing 30 transformer layers while storing only 12 blocks per expert. To account for their asymmetric inference costs, the video and action experts can operate at different depths: visual features are computed once per query, whereas action computation repeats at every sampling step. We further propose elastic self-distillation, which trains a single elastic checkpoint, LoopedWAM-E, across action-depth and sampling-step budgets. With only 0.58B policy parameters and no robot pretraining, LoopedWAM achieves 96.1% success on LIBERO, 91.9% on RoboTwin 2.0, and 52.2% on RoboCasaGR1, while using $7.4\times$ less peak GPU memory than Fast-WAM under matched inference measurements. On LIBERO, LoopedWAM-E achieves 96.0% success in 27.6 ms or 95.2% in 24.7 ms on an H100, providing multiple accuracy-latency operating points from a single checkpoint; with two sampling steps and four action loops, it stays within 0.5 points of LoopedWAM on all three benchmarks. Ablations demonstrate that weight-shared depth preserves competitive performance, while aggressive action-depth reduction requires explicit budget-aware training. On a real AgileX PiPER arm, LoopedWAM succeeds in 71.3% of 80 trials across four tasks.

138. 【2611.25594】Bradley-Terry Training Can Distort Outcome Probabilities of Initially Accurate Language World Models

链接:https://arxiv.org/abs/2611.25594

作者:Xinze Shi,Litian Zhang,Zhenjie Su,Zelong Yu,Yuting Zhang,Qihua Feng,Zifeng Kang,Chaozhuo Li,Zhengxuan Hu

类目:Machine Learning (cs.LG); Artificial Intelligence (cs.AI); Computer Vision and Pattern Recognition (cs.CV)

关键词:train language world, Comparative feedback, language world models, train language, language world

备注: 10 pages, 6 figures, 1 table

点击查看摘要

Abstract:Comparative feedback, in which a judge says which of two candidate outcomes is closer to what happened, is a practical way to train language world models that assign probabilities to an action's outcomes. We show that, fitted with a Bradley-Terry objective, it need not teach the true outcome probabilities and can distort accurate ones. On a simulated robot push with enumerated outcome distributions, this objective takes a model that is accurate on unseen layouts (hazard error 2.4 percentage points, pp) and moves 33 to 43 pp of its probability onto outcomes that none of a condition's enumerated branches produces. The cause is the objective, not the comparisons: its optimum, computed before training, predicts where the model ends up, and a likelihood that knows the judge's rule learns from the same comparisons without the shift. The shift is robust: it recurs on unseen conditions, under five candidate laws and a three-fold denser enumeration, and about 29 pp remain without the physically impossible events. Its cost is conditional: with those events it inflates hazard probabilities, and a threshold inherited from the pre-feedback model costs about 3 pp of utility per decision on trained and 1 to 2 pp on unseen conditions, through false rejects; without them the utility loss averages below 1 pp over six seeds, though a small regret in action choice remains. Audits of readout, response and decision trace the effect from probabilities to actions.

139. 【2611.25593】Characterizing Age-Conditioned Affective Reasoning in Vision-Language Models

链接:https://arxiv.org/abs/2611.25593

作者:Shahriyar Zaman Ridoy,Azmine Toushik Wasi,Lin Gu,Ruogu Fang

类目:Computer Vision and Pattern Recognition (cs.CV)

关键词:emotion evoked, age, images, prompt, models

备注: Preprint. Under review

点击查看摘要

Abstract:An age prompt can change how a vision-language model (VLM) rates the emotion evoked by an image. Does the change predict how people of different ages rate that same image? We evaluate 17 VLMs on 900 OASIS images using young-, middle-, and older-adult prompts and estimate image-specific age differences from ratings by 822 participants. Without an age cue, models closely track average human valence (median Pearson $r=0.891$). Under an older-adult prompt, 16 of 17 models assign lower valence on average than under a young-adult prompt. Yet after adjusting for image pleasantness, correlations between model and human age effects range from $-0.030$ to $0.093$, against an estimated ceiling of $0.595$. The discrepancy is clearest on 54 sensitive images rated as pleasant. Across models, the older prompt reduces valence by $0.845$ points on average, while older participants rate these images $0.443$ points higher than younger participants. Providing human ratings from other images improves agreement on the direction of age differences, although image-specific fidelity remains modest on 600 held-out images ($r_{\mathrm{part}}=0.105$). These results show why evaluations of demographic personalization in foundation models need human comparisons at the level of individual stimuli. Project page available at this https URL

140. 【2611.25578】QFT-Based Hybrid PointNet-Quantum Neural Networks for LiDAR Point-Cloud Classification

链接:https://arxiv.org/abs/2611.25578

作者:Anurag Kulkarni,Nouhaila Innan,Muhammad Shafique

类目:Computer Vision and Pattern Recognition (cs.CV); Quantum Physics (quant-ph)

关键词:complex data-processing tasks, Quantum Fourier Transform, data-processing tasks, Fourier Transform, provide a promising

备注: 14 pages, 3 figures, 6 tables

点击查看摘要

Abstract:Hybrid quantum-classical learning architectures provide a promising framework for integrating quantum feature transformations into complex data-processing tasks. Among these transformations, the Quantum Fourier Transform (QFT) offers a structured approach to feature processing, motivating an investigation of how its circuit configuration influences predictive performance, quantum resource requirements, and noise robustness. We present a hybrid PointNet-quantum neural network architecture for LiDAR point-cloud classification, combining classical feature extraction, amplitude encoding, and a variational quantum circuit. Five Fourier-processing configurations are systematically evaluated: No-QFT, Full-QFT-After, QFT-no-swap, IQFT-After, and Trainable-QFT. Experiments on KITTI, nuScenes, and PandaSet across three random seeds assess classification performance, class-level behavior, circuit complexity, and sensitivity to the IBM FakeManilaV2 noise model. Under ideal simulation, IQFT-After achieves the highest weighted F1-score on KITTI (0.857 $\pm$ 0.033), compared with 0.838 $\pm$ 0.023 for No-QFT. Removing the final SWAP gates in the QFT circuit reduces circuit complexity while preserving the Pauli-Z measurement features up to a fixed permutation that can be absorbed by the classification head. Under quantum noise, the circuit ranking changes, with IQFT-After exhibiting greater performance degradation and No-QFT and Trainable-QFT showing stronger robustness. Per-class analysis further identifies dataset-dependent differences in minority-class performance. These results demonstrate that Fourier-processing configuration influences both classification behavior and noise sensitivity, highlighting the role of circuit structure and hardware-aware evaluation in hybrid quantum learning for three-dimensional point-cloud data.

141. 【2611.25575】rajectory-Calibrated Merging of Embodied Policies

链接:https://arxiv.org/abs/2611.25575

作者:Wenshuo Chen,Honglei Jin,Le Yang,Jianfei Song,Shaofeng Liang,Lei Wang,Junze Zhu,Kailong Wang,Zhengxi Jin,Haozhe Jia,Honglin Chu,Songning Lai,Xiucheng Wang,Yutao Yue

类目:Robotics (cs.RO); Computer Vision and Pattern Recognition (cs.CV)

关键词:Robot foundation models, increasingly adapted, expand their embodied, Robot foundation, TCR

备注:

点击查看摘要

Abstract:Robot foundation models are increasingly adapted into task specialists to expand their embodied capabilities. Yet these capabilities remain fragmented across separate policies, and existing model-merging methods do not reliably preserve them during closed-loop execution: parameter-space alignment provides only an indirect proxy for behavior, while regression may fail to preserve behavior at states not covered by its calibration inputs. We present Trajectory-Calibrated Regression (TCR), an execution-aware, gradient-free framework that consolidates robot specialists into a single policy. Across two complementary timescales, TCR collects calibration inputs from expert rollouts spanning different stages of task execution and from intermediate states within iterative action generation. To account for representation shifts introduced by merging, TCR recomputes each block's input through the already merged blocks. It then uses each expert's computation within the block to form module inputs and fits merged weights to match the experts' linear outputs. A two-pass calibration procedure further refines the merged policy using separate rollout caches while retaining the frozen specialists as teachers. The resulting model preserves the original inference graph and requires neither expert routing nor additional gradient training. On LIBERO, TCR merges four $\pi_{0.5}$ specialists and achieves 77.83% success across 1,200 evaluation episodes, outperforming FeatCal, RegMean, and direct weight averaging by 9.50, 24.17, and 33.00 percentage points, respectively.

142. 【2611.25574】VidToC: Rethinking Video Token Compression for Multimodal Large Language Models

链接:https://arxiv.org/abs/2611.25574

作者:Zihan Xiao(1),Boshen Xu(1),Pei Fu(2),Jianzhong Ju(2),Zhenbo Luo(2),Jian Luan(2),Qin Jin(1) ((1) AIM3 Lab, Renmin University of China, (2) MiLM Plus, Xiaomi Inc.)

类目:Computer Vision and Pattern Recognition (cs.CV)

关键词:Extending multimodal large, multimodal large language, faces substantial memory, latency challenges due, large language models

备注: 24 pages, 7 figures

点击查看摘要

Abstract:Extending multimodal large language models (MLLMs) from images to videos faces substantial memory and latency challenges due to the massive number of visual tokens. These challenges have driven the development of numerous visual token compression methods and more efficient LLM architectures. However, existing compression methods are often evaluated with different training setups, tasks, and models, making fair comparison difficult. Meanwhile, the growing use of hybrid Mamba-Transformer architectures raises questions about whether compression strategies transfer across backbones. We present VidToC, a controlled evaluation framework for representative compression algorithms, LLM backbones, and time encoding strategies, covering training-free and fine-tuned settings on three complementary video understanding tasks. Our results reveal task-dependent compression behavior and characterize the effects of compression placement, time encoding, and backbone choice. These findings provide empirical guidance for designing and selecting video token compression strategies across tasks and training settings.

143. 【2611.25567】Online Mistake Detection in Instruction-Conditioned Procedural Video

链接:https://arxiv.org/abs/2611.25567

作者:Amit Feldman,Ilan Shimshoni,Rotem Dror

类目:Computer Vision and Pattern Recognition (cs.CV); Artificial Intelligence (cs.AI)

关键词:assistant monitoring, monitoring a person, person performing, performing a procedure, time to recover

备注:

点击查看摘要

Abstract:An assistant monitoring a person performing a procedure is most useful when it can detect mistakes while there is still time to recover. This requires judging actions against task instructions as the video unfolds, identifying the attempted step, locating it in time, and assessing its execution. Existing online detectors do not use the instruction text, while instruction-conditioned methods operate offline. In this paper, we study online, instruction-conditioned mistake detection in egocentric videos. Using only observed video, task instructions, and prior predictions, our model identifies and localizes completed action attempts and classifies them as correct, modified, slip, corrections, or additions; omitted steps are determined after the video ends. We compare a modular pipeline, with separate step identification, temporal localization, and execution verification, to a joint model that predicts all outputs in a single step. To support supervised learning from mistakes, we repartition EgoPER to include erroneous videos in all splits. The approaches achieve similar step-segmentation F1@0.5 (93.6% and 92.1%), while the joint model achieves higher binary mistake-detection F1@0.5 (84.1% vs. 80.7%), with its largest gain on slips (56.4% vs. 37.5%). Component-level analysis suggests that error propagation between stages contributes to the modular approach's lower detection score. On test videos containing mistakes, both designs outperform released AEM and ESTANet checkpoints under our evaluation protocol.

144. 【2611.25555】UniReFP: Robust Unified Fingerprinting for Vision Models against Cross-Task Repurposing Attacks

链接:https://arxiv.org/abs/2611.25555

作者:Ziye Geng,Guang Yang,Yihang Chen,Lingfeng Yao,Xing Shi,Miao Pan,Xin Fu,Changqing Luo

类目:Computer Vision and Pattern Recognition (cs.CV); Cryptography and Security (cs.CR)

关键词:fingerprinting approaches inapplicable, existing model fingerprinting, model fingerprinting approaches, approaches inapplicable, render fingerprints generated

备注: NeurIPS 2026 Spotlight

点击查看摘要

Abstract:We identify, for the first time, a new model modification attack--the cross-task model repurposing attack--that can render fingerprints generated by existing model fingerprinting approaches inapplicable. To address this challenge, we propose UniReFP--a novel unified model fingerprinting framework applicable to diverse vision models across different tasks--that utilizes ownership evidence in a shared class-level semantic space rather than original task-specific output space, enabling effective ownership verification when a protected model is repurposed across tasks with heterogeneous output formats. Specifically, UniReFP constructs a surrogate classifier by repurposing the protected model, enabling fingerprints to be generated independently of the original task-dependent output space of the protected model. Moreover, it optimizes grouped fingerprints to encode ownership evidence: fingerprints within the same group are encouraged to induce consistent semantic responses and intermediate representations on the surrogate classifier, while producing dispersed responses on independently trained reference classifiers. During verification, UniReFP uses a task-aware semantic projection mechanism to map heterogeneous suspect-model outputs into unified semantic-presence vectors, and determines ownership by measuring inner-group consistency. Extensive experimental results validate the effectiveness of UniReFP, consistently showing superior performance under cross-task repurposing attacks, commonly studied model modification attacks, and even their combinations.

145. 【2611.25553】woPass: Draft-Guided Few-Step Video Prediction for Robotic Manipulation

链接:https://arxiv.org/abs/2611.25553

作者:Yijun Liu,Shijia Ge,Tongtong Feng,Xin Wang,Yuan Meng,Yuwei Liu,Zhi Wang,Wenwu Zhu

类目:Computer Vision and Pattern Recognition (cs.CV)

关键词:Action-conditioned video prediction, Action-conditioned video, robots anticipate, anticipate the visual, visual consequences

备注:

点击查看摘要

Abstract:Action-conditioned video prediction lets robots anticipate the visual consequences of their actions for planning and policy evaluation. Existing predictors denoise each future from noise without using their own estimate of it, which limits the quality of the predicted future, in particular of the manipulated objects, and requires many network calls. We present TwoPass, a self-conditioned training-and-inference framework that keeps the standard flow-matching objective. Self-Generated Draft Training supplies a detached clean-video estimate from the shared network as conditioning at the same noisy state. A Refinement-Only Late Adapter applies a dedicated residual transformation when a draft is present. At inference, Draft-Reuse Iterative Refinement obtains both a solver update and the next draft from each velocity prediction, requiring as few as 4 function evaluations (NFE) per video chunk. Training samples discrete solver-query noise levels and updates the pretrained backbone. Extensive experiments on Bridge and Language Table show that TwoPass achieves state-of-the-art results with 4 NFE, far fewer than the 50-59 calls of existing predictors. Code: this https URL.

146. 【2611.25551】Anchored Inertial Proposals with Objective Verification for Diffusion-Prior Inverse Problems

链接:https://arxiv.org/abs/2611.25551

作者:Minwoo Kim,Seunghyeok Shin,Dabin Kim,Hongki Lim

类目:Computer Vision and Pattern Recognition (cs.CV)

关键词:Training-free diffusion priors, inverse problems, aggressive updates, disrupting later denoising, Training-free diffusion

备注: Accepted to NeurIPS 2026

点击查看摘要

Abstract:Training-free diffusion priors are powerful for inverse problems, but measurement guidance during reverse sampling is local: aggressive updates can improve immediate data fit while disrupting later denoising. We introduce an anchored inertial correction rule that separates proposal generation from acceptance. At each corrected step, the denoiser prediction anchors a frozen clean-space objective combining measurement fit with a noise-level-dependent proximal term. Heavy-ball dynamics with step size and momentum computed from the measured curvature generate a clean-state candidate; a separate verifier evaluates it with the same objective, with the denoiser prediction retained as fallback; both stages are derived from one objective for the whole trajectory, considered one step at a time. The accepted clean state is re-noised to continue sampling. The rule applies to pixel and latent diffusion with differentiable measurement operators, requires no retraining, and adds no denoiser calls inside the correction loop. We analyze how objective verification yields local nonincrease and displacement control. Across natural-image benchmarks, the method achieves competitive reconstruction quality. Code is available at this https URL.

147. 【2611.25537】Query-Aware vs. Query-Agnostic Training-Free Video Token Pruning in Multimodal LLMs

链接:https://arxiv.org/abs/2611.25537

作者:Divya Kothandaraman,Timo Kumkel,Jaclyn Pytlarz

类目:Computer Vision and Pattern Recognition (cs.CV)

关键词:Large Language Models, Language Models process, Models process large, Video Large Language, process large numbers

备注:

点击查看摘要

Abstract:Video Large Language Models process large numbers of visual tokens, making token pruning important for efficient inference. Training-free pruning methods differ in whether and how they use the user query, and the value of query conditioning remains unclear. We present a controlled study of training-free, pre-LLM video token selection. Using a shared Qwen-3-VL backbone, token budgets, and decoding settings, we compare 21 pruning principles spanning query-agnostic, shallow query-conditioned, structured query-conditioned, and holistic strategies. To instantiate the last two categories, we introduce Structured Query Pruning, which decomposes the question into semantic anchors, and a Pareto variant, which combines anchor relevance with query-agnostic visual diversity. Across Video-MME, MVBench, and LongVideoBench, relevance computed from the raw question embedding is unreliable (8--16 points below the best methods), whereas anchors derived from query decomposition avoid this failure. However, query access is not inherently advantageous: strong query-agnostic coverage methods remain competitive or superior in the full-field comparison. Relative to a redundancy-only baseline, structured selection helps most on evidence-specific and long-context tasks at tight budgets; on broad tasks, the holistic selector improves over either relevance or redundancy alone. Further, pruning reduces cost and memory. These results support a practical design rule: preserve visual coverage by default, and add structured relevance when the task and token budget demand selective evidence retrieval.

148. 【2611.25535】Support-Guided Triadic Decoding for Hallucination Mitigation in Multimodal Dialogue with LVLMs

链接:https://arxiv.org/abs/2611.25535

作者:Yiheng Li,Wenhao Wang,Xingyi Li,Zhilin Tang,Zichang Tan,Zhen Lei,Yang Yang

类目:Computer Vision and Pattern Recognition (cs.CV)

关键词:multimodal multi-turn dialogue, reliable responses provide, dialogue history plays, subsequent turns, dual role

备注: preprint

点击查看摘要

Abstract:In multimodal multi-turn dialogue, dialogue history plays a dual role: reliable responses provide useful context for subsequent turns, whereas hallucinated responses can propagate errors across turns. Existing training-free methods globally adjust the effect of dialogue history on generation, making it difficult to suppress misleading information without weakening useful context. To address this trade-off, we introduce Support-Guided Triadic Decoding (SGTD), a training-free framework that selectively controls the influence of unreliable history and fuses complementary predictions during decoding. Specifically, Historical Answer Support Estimation measures the visual support of historical responses against type-matched alternatives and identifies low-support content that may mislead later generation. Support-Targeted Attention Intervention then weakens the influence of this content while preserving the surrounding dialogue, producing a support-refined prediction. Triadic Decoding combines this prediction with the full-history prediction and answer-free visual evidence, while the Support-Adaptive Plausibility Constraint filters implausible tokens using visual agreement and historical support. Experiments on MMHalSnowball and VisDiaHalBench across representative LVLMs show that SGTD consistently achieves the highest accuracy under hallucinated histories while preserving the benefits of factual context.

149. 【2611.25524】Contrast Where It Matters: Visual Evidence-Guided Decoding for Mitigating Hallucinations in LVLMs

链接:https://arxiv.org/abs/2611.25524

作者:Yiheng Li,Wenhao Wang,Xingyi Li,Mingda Jia,Zichang Tan,Yinhang Tang,Zhen Lei,Yang Yang

类目:Computer Vision and Pattern Recognition (cs.CV)

关键词:large vision-language models, linguistic priors outweigh, strong multimodal capabilities, misleading image regions, priors outweigh visual

备注: preprint

点击查看摘要

Abstract:Despite strong multimodal capabilities, large vision-language models (LVLMs) often hallucinate when linguistic priors outweigh visual evidence or when predictions are grounded in irrelevant or misleading image regions. Training-free contrastive decoding mitigates hallucinations by contrasting the original prediction with an auxiliary one that exposes linguistic priors. Yet existing methods focus on altering the auxiliary input or computation and do not ensure that either branch uses query-relevant visual evidence during decoding. We propose Visual Evidence-Guided Decoding (VEGD), a training-free framework that identifies query-relevant visual evidence and guides its use throughout contrastive decoding. VEGD locates this evidence by aggregating localization heads with strong visual responses and coherent spatial structure, then selectively weakens it in the auxiliary image. During generation, VEGD directs visual-carrier heads in both branches toward the same evidence location, strengthening access to intact evidence in the main branch while anchoring the auxiliary branch to its weakened counterpart. An adaptive distribution fusion strategy then combines or contrasts their predictions according to distribution divergence. Experiments across diverse LVLMs and benchmarks demonstrate that VEGD consistently reduces hallucinations.

150. 【2611.25523】Multi-View Relighting with Mergeable Radiance Fields

链接:https://arxiv.org/abs/2611.25523

作者:Yinan Wang,Rohan Kadkol,Hyunjin Yoo,Jean-François Lalonde,James J. Clark

类目:Computer Vision and Pattern Recognition (cs.CV)

关键词:radiance fields, Neural Radiance Fields, propose MergeableRF, model parameter merging, radiance fields trained

备注:

点击查看摘要

Abstract:We propose MergeableRF, a method for radiance field relighting through model parameter merging. Given multi-view images of a scene under known basis lights, our goal is to relight the scene by directly combining the parameters of radiance fields trained under these lights. This is motivated by the linearity of light transport: the image under a mixture of lights is the same weighted combination of the images under each light. However, combining the parameters of independently trained radiance fields does not produce the corresponding combination of images. To address this, we represent a scene with shared geometry and a separate appearance network for each basis light. We start these networks from the same initialization and introduce merge-aware training, which teaches their parameter combinations to reproduce the corresponding combinations of images. Given a target HDR environment map, we compute its basis-light coefficients and use them to merge the appearance networks into a single relit model. Our method supports both Neural Radiance Fields (NeRFs) and 3D Gaussian Splatting (3DGS), without material estimation, additional relighting networks, or target-specific optimization. Unlike image-space blending, it requires only one rendering pass per viewpoint. Experiments on synthetic Neural Gaffer objects and real OLATverse captures demonstrate relighting quality competitive with methods trained on the same multi-light observations.

151. 【2611.25513】Mesh-Free Keypoint Learning for Category-Level 6D Object Pose Estimation

链接:https://arxiv.org/abs/2611.25513

作者:Junghoon Seo,Hakjin Lee,Jaehoon Sim

类目:Computer Vision and Pattern Recognition (cs.CV); Robotics (cs.RO)

关键词:estimation relates partial, relates partial RGB-D, partial RGB-D observations, pose estimation relates, canonical frame shared

备注:

点击查看摘要

Abstract:Category-level 6D pose estimation relates partial RGB-D observations of unseen instances to a canonical frame shared across a category. Object-aware reconstruction helps adaptive keypoints retain the geometry of the visible surface. Common reconstruction targets select observed points by their distance to a registered instance surface. We present MORSEPose, which derives this support directly from standard pose and metric-size annotations. We map observed 3D points to normalized object coordinates and retain those inside a tolerancedilated pose-and-size envelope. This construction is consistent across camera frames and metric scales. When the annotated extent bounds the object surface, the envelope conservatively contains the support of a conventional surface-distance selector. We integrate this target with multi-level RGB-D features, adaptive keypoints, and learnable category context. Experiments on REAL275 and HouseCat6D show that the envelope closely matches surface-based supervision at both point and task levels, remains stable under boundary perturbations and viewpoint changes, and supports competitive category-level pose accuracy across benchmarks.

152. 【2611.25508】Persona Lenses: What Makes a Vision-Language Model See Through Its Persona's Eyes?

链接:https://arxiv.org/abs/2611.25508

作者:Yoonho Shin

类目:Computer Vision and Pattern Recognition (cs.CV); Artificial Intelligence (cs.AI); Human-Computer Interaction (cs.HC)

关键词:Vision-language models, lens, lens separates, separates the personas, model

备注: 29 pages, 9 figures, 14 tables, including supplementary material

点击查看摘要

Abstract:Vision-language models have been used to simulate where a user looks on a web page. However, a persona in the prompt reaches the simulated gaze only partly. We measure this reach as the difference in one model's gaze between two MatrAIx personas that differ in a single attribute. We introduce persona lenses, which place the persona between the model and the page: a text lens (a two-step prompt in which the model first writes what the persona would look at), a pixel lens (marks on the region matching the persona's interests) and a weight lens (a dwell reward gated to keep the scanpath natural). Using Qwen3-VL (4B and 32B) and Qwen3.5-9B on generated and live pages, we compare the lenses with one another and with scale, the 32B model with no lens. It has been confirmed that the text lens separates the personas strongly but is not robust to profile changes that should not move gaze. The pixel lens separates the personas less but is unaffected by attributes unrelated to gaze. At 4B and 9B, the weight lens separates the personas on interests held out from training, with no extra step on a new page. The 32B model separates the personas with the base prompt alone. Therefore, the choice of a lens depends on cost and model size.

153. 【2611.25506】Carry the Fine Print: Budget-Aware Visual Token Compression for Document OCR

链接:https://arxiv.org/abs/2611.25506

作者:Yixuan Huang,ShengFan Zhang,Jiawei Chen

类目:Computer Vision and Pattern Recognition (cs.CV)

关键词:Visual token compression, compression often treats, treats salience, salience or similarity, token compression

备注: 11 pages; supplementary material is included as an ancillary file

点击查看摘要

Abstract:Visual token compression often treats salience or similarity as a proxy for transcription importance. Complete document OCR breaks this assumption: small symbols and repeated entries can be indispensable to the transcript despite being visually inconspicuous. We present Select and Compensate, a two-stage framework built on a simple idea: choosing where to retain a token need not determine all the evidence it carries. Selection combines adaptive encoding resolution with global importance and local contrast to choose which tokens to retain as anchors. Compensation enriches these anchors with nearby discarded content and its relative position, without adding tokens. The OCR backbone remains frozen, and native coordinates and structural tokens are preserved. We evaluate complete page transcription across five backbones on OmniDocBench and document reconstruction on MPDocBench. With half the original visual tokens, our input-adapted configuration reaches 95.42 OmniDocBench Overall on Ovis versus 96.09 for the uncompressed model, and paired evaluations show gains over simply reducing the input resolution on both benchmarks. With the original input grid, our method stays within 0.17 points of the uncompressed model at 75% retention and exceeds a learned global selector by 4.70 points at 25%. On a fixed 200-page Ovis workload at the same 50% visual-token budget, only 3 of its outputs hit the length limit, versus 30-34 for the evaluated RTPrune, DivPrune, and HiPrune adaptations, and it finishes the batch 35-40% sooner. Code will be made publicly available.

154. 【2611.25500】PRIO: Pre-trained Representation for Interactive Objects in 4D Worlds

链接:https://arxiv.org/abs/2611.25500

作者:Shixun Huang,Lifu Wei,Yue Chang,Chen Geng,Heng Zhang,Jinhyung Ahn,Caoliwen Wang,Siyuan Chen,Donglai Xiang,Jiajun Wu,Peter Yichen Chen

类目:Computer Vision and Pattern Recognition (cs.CV)

关键词:pressed pillow, pulled door, world models requires, PRIO, user actions

备注:

点击查看摘要

Abstract:3D worlds can now be captured, reconstructed, and generated at scale, yet the objects in them stay still: a pressed pillow does not compress, and a pulled door does not swing. Bringing these objects into interactive 4D world models requires a representation of how they can move, deform, and respond physically to user actions in real time. We present PRIO, a pre-trained representation for interactive objects. Given static geometry, PRIO predicts an object-specific configuration space in a single forward pass. This space provides generalized coordinates for Lagrangian mechanics, allowing physical laws and user actions to govern motion. Configurations are linear in these coordinates, allowing the quantities needed by the dynamics to be precomputed. Dynamics then run at interactive rates without further network inference. Pre-trained on diverse 4D trajectories, PRIO learns priors over object configurations that generalize to unseen shapes. It supports meshes, point clouds, and 3D Gaussians, bringing objects in reconstructed and generated scenes into interaction. We demonstrate its use in real-to-sim workflows and robot manipulation. In addition, compared with existing 4D generation methods, PRIO generates dynamics faster and follows user actions more faithfully.

155. 【2611.25478】AP: Tapping the Discriminative Potential of Frozen Representations for Class-Incremental Learning

链接:https://arxiv.org/abs/2611.25478

作者:Zihao Wang(1),Yu-Wei Zhan(2),Xiao-Qian Liu(1),Zhen-Duo Chen(1),Weitong Liu(1),Xin Luo(1),Xin-Shun Xu(1) ((1) Shandong University, (2) Zhejiang University of Technology)

类目:Computer Vision and Pattern Recognition (cs.CV)

关键词:Pre-trained models provide, models provide transferable, Pre-trained models, provide transferable representations, models provide

备注: 10 pages, 3 figures. Code is included as an ancillary file

点击查看摘要

Abstract:Pre-trained models provide transferable representations for class-incremental learning (CIL). However, sequentially adapting these representations introduces optimization costs and may disrupt old-class feature structure. This motivates us to explore whether the discriminative potential of these representations can be further exploited by continually accumulating class information while keeping the encoders frozen. To this end, we propose TAP, a new and novel framework for tapping the discriminative potential of frozen representations in class-incremental learning. It accumulates visual and semantic class information in the frozen CLIP embedding space with only lightweight optimization. TAP comprises three modules: class-conditional visual distribution modeling, class-adaptive semantic calibration, and visual-semantic posterior fusion. The first module retains class means and within-class scatters, combining class-specific and shared covariances to stabilize distribution estimation. The second module learns class-specific weights over text templates from current-task images, adapting text representations to downstream classes. The third module combines the visual and semantic posteriors through a weighted product of experts for prediction over all seen classes without retaining images from previous tasks. Across nine datasets under two incremental protocols, TAP achieves state-of-the-art performance, attaining the highest average incremental and final-stage accuracies among the compared methods. The code is provided as an ancillary file.

156. 【2611.25475】SegAnyFruit: Learning Generalizable and Edge Deployable Fruit Instance Segmentation at Scale

链接:https://arxiv.org/abs/2611.25475

作者:Yanan Wang,Yuzheng wang,Yibin Ying,Zhenghao Fei

类目:Computer Vision and Pattern Recognition (cs.CV)

关键词:Deploying fruit instance, settings requires generalizing, strict onboard compute, real agricultural settings, agricultural settings requires

备注: 11 pages, 6 figures

点击查看摘要

Abstract:Deploying fruit instance segmentation in real agricultural settings requires generalizing across unseen species, orchards, and imaging conditions under strict onboard compute and annotation budgets. We introduce SegAnyFruit (SAF), a data construction and progressive training framework that distills foundation-model priors into lightweight, prompt-free segmenters. SAF combines generic mask pre-training, broad mid-training on automatically annotated web imagery from our curated WebFruits dataset, and human-verified agricultural refinement post-training on PaperFruits, together spanning 733K images and 9.66M whole-fruit masks across 165 fruit types. Across seven benchmarks, zero-shot SAF-RF-DETR-XL reaches 69.06 average mask AP50:95 (+4.76 over SAM 3) while remaining deployable on edge devices. SAF initialization also boosts few-shot to full-shot adaptation across five agricultural domains while mitigating catastrophic forgetting. Finally, we systematically analyze training stages, data scale, model capacity, and generative augmentation, and validate our pipeline's transferability to underwater, cell, and human segmentation. We will release our datasets, code, and SAF models.

157. 【2611.25474】uringPAI: A Foundation for On-Device Physical AI

链接:https://arxiv.org/abs/2611.25474

作者:Qiman Wu,Hanlin Chen,Yuhua Wei,Mingyuan Wang,Shaman Tang,Minghao Li,Rui Xin,Xingye Liu,Zhichao Yang,Yiqi Yu,Hongyan Pei,Xiaowen Tian,YingYing Liu,Shiwen Liu,Jiahui Hu,Yizhao Wang,Yuheng Zhang,Anhua Liu,Yue He,Shuang Su,Da Zhu,Hua Zhou,Pengfei Diao,Haojie Yang,Weichao Huang,Hang Zhang,Honggou Yang,Xianming Liu

类目:Computer Vision and Pattern Recognition (cs.CV); Robotics (cs.RO)

关键词:limited onboard compute, understand spatial relationships, Physical AI requires, onboard compute, relationships and dynamic

备注:

点击查看摘要

Abstract:Physical AI requires models that understand spatial relationships and dynamic environments and translate observations into actions under limited onboard compute. We present TuringPAI, a family of models for on-device Physical AI comprising TuringPAI-VL for shared vision-language understanding, TuringPAI-Drive for autonomous driving, and TuringPAI-Embodied for robotic manipulation. TuringPAI-VL is a 0.65B model built on HybridViT-MLQ, which combines convolutional local aggregation with Transformer-based global modeling. Multi-scale Latent Queries integrate spatial features and observation history within the same visual Transformer, providing a compact, structured interface to the language model. Training combines general vision-language data with geometric, cross-view, and temporal supervision. TuringPAI-VL supports general multimodal understanding and improves quantitative geometric perception over Qwen2-VL-2B, while providing a shared initialization for the two downstream VLA models. For autonomous driving, TuringPAI-Drive maintains all evaluated ADE/FDE metrics within 1.2\% of a full-token HybridViT baseline using the MLQ interface and achieves comparable aggregate scores in an in-house simulation evaluation. Using XPeng Native Runtime (XNRT) on a single NVIDIA Orin-X SoC, TuringPAI-Drive generates trajectories at 12\,Hz in on-vehicle operation. For robotic manipulation, TuringPAI-Embodied achieves a 96.8\% average success rate across the four LIBERO suites. These results demonstrate a shared vision-language foundation for driving and robotic manipulation under practical on-device constraints.

158. 【2611.25469】A Wolf in Sheep's Pixels: Visual Camouflage Attacks via HDR Gain Maps

链接:https://arxiv.org/abs/2611.25469

作者:Qi Shen,Yujia Gong,Yue Zhao,Tong Liu

类目:Computer Vision and Pattern Recognition (cs.CV); Artificial Intelligence (cs.AI)

关键词:showing harmful content, pass automated safety, showing harmful, harmful content, automated safety checks

备注:

点击查看摘要

Abstract:An image can pass automated safety checks while showing harmful content to its human viewer. We introduce DRJanus, a visual camouflage attack that exploits the gap between standard dynamic range (SDR) processing and high dynamic range (HDR) display. Automated systems often read only an image's SDR base, whereas HDR viewers reconstruct its appearance using an auxiliary gain map. DRJanus inverts this reconstruction to embed two independently chosen color images in a single JPEG, without querying or optimizing against the target model. It preserves the chosen SDR base exactly and reconstructs HDR targets at an average PSNR of 43.03 dB across 400 images. On a benchmark of 100 sexual-content images, DRJanus evades detection by all six tested commercial moderation services and nine vision-language models. Across five agents, each evaluated on 120 task cases per threat setting, attack success ranges from 31.67% to 100% for moderation evasion and from 95% to 100% for task manipulation. We also explore cross-view consistency checks as a mitigation. These findings expose a gap in visual trust: securing what a machine reads does not secure what a user sees. A demo is available at this https URL

159. 【2611.25461】ActMem: From Latent Memory Reasoning to Grounded Robotic Action

链接:https://arxiv.org/abs/2611.25461

作者:Linfeng Dong,Jianwei Niu,Jiuyi Zhao,Ruibing Feng,Xinghao Wu,Wu Yang,Shuai Tao,Wulong Liu,Mu Li

类目:Computer Vision and Pattern Recognition (cs.CV); Machine Learning (cs.LG)

关键词:real-world long-horizon manipulation, shown promising results, Toggle, memory, Toggle Hugging Face

备注: 17 pages, 11 figures, 6 tables. Code and checkpoints: [this https URL](https://github.com/LigphiDonk/ActMem)

点击查看摘要

Abstract:Vision-Language-Action (VLA) models have shown promising results in real-world long-horizon manipulation, where memory plays a crucial role. However, existing memory-augmented VLAs either retrieve past observations without reasoning about what they imply, or reason over the full history in a shared context where it crowds out current perception. Cognitive science treats them as distinct systems: memory reconstructs past experience to decide what to do, while perception guides how to act. We therefore propose ActMem, an in-context memory framework that decouples memory reasoning from perceptual grounding within a single vision-language backbone. Built on a vision-language model with native video understanding, ActMem infers the current subtask from a timestamped interaction history and keeps the hidden states formed during this inference as a Latent Memory Chain-of-Thought (Memory CoT), which conditions a flow-matching action expert. Perception Attention further lets the expert attend directly to the current view. To keep inference fast under long histories, Efficient Memory CoT Inference reuses computation across consecutive decisions without changing the output. ActMem achieves 71.1% task success and 80.8% completion success on RoboMemArena, surpassing the strongest baseline by 7.5 and 8.7 percentage points, and 73.8% average success on MIKASA-Robo, the best among all compared methods. In the real world, it reaches 68.8% average success on exogenous, endogenous, and traceless memory tasks, 10.0 percentage points above the strongest memory-augmented VLA. Code and checkpoints will be released at this https URL and this https URL.

Comments:
17 pages, 11 figures, 6 tables. Code and checkpoints: this https URL

Subjects:

Computer Vision and Pattern Recognition (cs.CV); Machine Learning (cs.LG)

Cite as:
arXiv:2611.25461 [cs.CV]

(or
arXiv:2611.25461v1 [cs.CV] for this version)

https://doi.org/10.48550/arXiv.2611.25461

Focus to learn more

              arXiv-issued DOI via DataCite (pending registration)

Submission history From: Linfeng Dong [view email] [v1]
Thu, 19 Nov 2026 03:46:39 UTC (11,167 KB)

Full-text links:
Access Paper:

View a PDF of the paper titled ActMem: From Latent Memory Reasoning to Grounded Robotic Action, by Linfeng Dong and 8 other authorsView PDFHTML (experimental)TeX Source

view license

Additional Features

Audio Summary

Current browse context:
cs.CV

prev

|
next

new
|
recent
| 2026-11

Change to browse by:

cs
cs.LG

References Citations

SciXGoogle Scholar
Semantic Scholar

export BibTeX citation
Loading…

BibTeX formatted citation

loading…

Data provided by:

Bookmark

checked="checked"class=“labs-tab-input”>
Bibliographic Tools

Bibliographic and Citation Tools

Bibliographic Explorer Toggle

Bibliographic Explorer (What is the Explorer?)

Connected Papers Toggle

Connected Papers (What is Connected Papers?)

Litmaps Toggle

Litmaps (What is Litmaps?)

scite.ai Toggle

scite Smart Citations (What are Smart Citations?)

Code, Data, Media

Code, Data and Media Associated with this Article

alphaXiv Toggle

alphaXiv (What is alphaXiv?)

Links to Code Toggle

CatalyzeX Code Finder for Papers (What is CatalyzeX?)

DagsHub Toggle

DagsHub (What is DagsHub?)

GotitPub Toggle

Gotit.pub (What is GotitPub?)

Huggingface Toggle

Hugging Face (What is Huggingface?)

ScienceCast Toggle

ScienceCast (What is ScienceCast?)

Demos

Demos

Replicate Toggle

Replicate (What is Replicate?)

Spaces Toggle

Hugging Face Spaces (What is Spaces?)

Spaces Toggle

TXYZ.AI (What is TXYZ.AI?)

Related Papers

Recommenders and Search Tools

Link to Influence Flower

Influence Flower (What are Influence Flowers?)

Core recommender toggle

CORE Recommender (What is CORE?)

Author
Venue
Institution
Topic

    About arXivLabs

arXivLabs: experimental projects with community collaborators
arXivLabs is a framework that allows collaborators to develop and share new arXiv features directly on our website.
Both individuals and organizations that work with arXivLabs have embraced and accepted our values of openness, community, excellence, and user data privacy. arXiv is committed to these values and only works with partners that adhere to them.
Have an idea for a project that will add value for arXiv’s community? Learn more about arXivLabs.

Which authors of this paper are endorsers? |
Disable MathJax (What is MathJax?)

mathjaxToggle();

    We gratefully acknowledge support from our major funders,
    member institutions, ,
    and all contributors.

About
Help
Contact
Subscribe
Copyright
Privacy
Accessibility
Operational Status (opens in new tab)

160. 【2611.25458】RemoteVLPG: How to Obtain a Specific VLM for Language-Guided Path Generation on Remote Sensing Images?

链接:https://arxiv.org/abs/2611.25458

作者:Junzhe Dang,Mengmeng Zhang,Yuxiang Zhang,Wei Li

类目:Computer Vision and Pattern Recognition (cs.CV)

关键词:task-specific Vision-Language Model, remote sensing images, Vision-Language Model, sensing images, obtain a task-specific

备注: 18 pages, 10 figures

点击查看摘要

Abstract:We study how to obtain a task-specific Vision-Language Model (VLM) for language-guided path generation between start and end points on remote sensing images. We explore this question from three aspects. First, we implement three widely used architectural approaches and compare their performance. Second, we design a scalable method for constructing pre-training data and build an effective pre-training dataset with a degree of diversity. Methods trained on this dataset achieve promising zero-shot performance. Third, we construct three benchmarks that support stratified evaluation and design three metrics for path generation. We investigate this problem through extensive experiments. The results show that the zero-shot performance of methods trained on the constructed pre-training dataset can exceed that of some advanced commercial VLMs. Code and Dataset can be available at this https URL

161. 【2611.25457】CANDOR: Contrastive Self-Denoising Rewards for Label-Free Text-to-Image Post-Training

链接:https://arxiv.org/abs/2611.25457

作者:Junhyeok Lee,Byung-Hoon Kim,Kyu Sung Choi

类目:Computer Vision and Pattern Recognition (cs.CV)

关键词:reinforcement learning, learning usually relies, auxiliary evaluators, Post-training, reward hacking

备注: 10 pages, 3 figures, 5 tables

点击查看摘要

Abstract:Post-training text-to-image generators with reinforcement learning usually relies on external reward models that need annotated preferences or auxiliary evaluators and are prone to reward hacking. We show that the generator's own denoising accuracy, used by a recent intrinsic reward, is a poor proxy for prompt fidelity. Within a same-prompt group, this absolute self-confidence is dominated by image complexity and ranks correct samples above incorrect ones near chance. We introduce CANDOR (Contrastive Alignment from Null-referenced Denoising lOg-Ratio), the log-ratio of unconditional to conditional denoising errors under shared noise probes, which measures how much the prompt helps the model denoise its own sample. It cancels prompt-independent difficulty that scales both errors and ranks the better of two same-prompt samples higher in 60% and 63% of pairs on compositional generation and text rendering. As the only Flow-GRPO reward for a flow-matching model, CANDOR improves text rendering, aesthetic and human-preference scores over the base model and a prior absolute self-confidence reward, with text-rendering and preference gains that stronger classifier-free guidance does not reproduce. On public Flow-GRPO adapters trained on external rewards, CANDOR further raises PickScore and ImageReward, so it can complement those rewards. Code is available at this https URL.

162. 【2611.25454】Lossless Latent Diffusion Models

链接:https://arxiv.org/abs/2611.25454

作者:Tongda Xu,Yu Zhang,Jun Hu,Huijuan Huang

类目:Computer Vision and Pattern Recognition (cs.CV)

关键词:preserving pixel-exact reconstruction, pixel-exact reconstruction, preserving pixel-exact, diffusion models, diffusion

备注:

点击查看摘要

Abstract:We study the problem of training image diffusion models while preserving pixel-exact reconstruction. Prior works typically address this problem by directly modeling images in pixel space, requiring modifications to the diffusion training objective or model architecture. In this work, we introduce a different paradigm: rather than modifying diffusion models, we construct a lossless transformation that maps pixels into a representation more amenable to diffusion modeling. We term our approach the Lossless Latent Diffusion Model (LLDM). We construct LLDM using residual latent transformation and uneven latent scaling, and explain its effectiveness through a proposed metric, semantic rank priority (SRP). Empirically, LLDM substantially outperforms prior pixel-space diffusion models in generation quality and even surpasses several widely used lossy VAEs. Furthermore, we show that SRP is strongly correlated with downstream generative performance.

163. 【2611.25453】Beyond Resemblance: Image Fusion Reimagined

链接:https://arxiv.org/abs/2611.25453

作者:Hao Zhang,Yumin Huang,Linfeng Tang,Junjun Jiang,Jiayi Ma

类目:Computer Vision and Pattern Recognition (cs.CV)

关键词:forcing existing deep, simultaneously resemble multiple, resemble multiple source, lacks ground-truth fused, multiple source images

备注:

点击查看摘要

Abstract:Image fusion often lacks ground-truth fused images, forcing existing deep methods to indirectly define fusion objectives by requiring the fused image to simultaneously resemble multiple source images. However, the distinct imaging characteristics of different modalities can impose competing constraints on the fused result, potentially limiting the effective integration of their complementary information. To address this long-standing challenge facing the image fusion community, we propose ApexFuse, a novel generative image fusion paradigm that moves beyond source-resemblance constraints toward learning a general distribution of high-quality scene representations. Specifically, we develop a human-LMM-in-the-loop pipeline that leverages large multimodal generative models and iterative human feedback to translate abstract fusion objectives into reliable visual references, thereby defining the target distribution of ideal fusion outcomes. Through target-distribution-guided adaptation, we transfer the rich generative priors of pretrained FLUX to image fusion, enabling high-quality scene reconstruction conditioned on multimodal observations. Extensive experiments demonstrate that ApexFuse harmoniously integrates salient thermal targets, natural colors, and fine textures, achieving superior visual quality as well as semantic segmentation and object detection performance over previous source-resemblance methods. Moreover, ApexFuse exhibits remarkable cross-task generalization, extending from infrared-visible fusion to multi-exposure fusion, multi-focus fusion, and visible-depth fusion without additional fine-tuning.

164. 【2611.25445】Stereo Visual Odometry with In-the-Loop Geometric Feed-Forward Models Towards Dense Metric Mapping

链接:https://arxiv.org/abs/2611.25445

作者:Fahira Afzal Maken,Sagun Man Singh Shrestha,Edward Vanderfeen,Lucas Carvalho de Lima,Rongxin Li,Changming Sun,Milad Ramezani

类目:Robotics (cs.RO); Computer Vision and Pattern Recognition (cs.CV)

关键词:geometric feed-forward models, integrates geometric feed-forward, stereo visual odometry, dense metric mapping, dense GFM predictions

备注: 8 PAGES, 6 FIGURES

点击查看摘要

Abstract:We present a stereo visual odometry (SVO) framework that integrates geometric feed-forward models (GFMs) for dense depth prediction with factor-graph-based visual odometry to achieve accurate localisation and dense metric mapping. Our key idea is to exploit sparse but reliable metric geometry estimated from stereo and temporal multi-view observations to calibrate dense GFM predictions, while retaining the robustness and efficiency of model-based SVO. Rather than directly conditioning the GFM on potentially noisy stereo depth, optimised 3D landmarks are used to jointly estimate a scale-shift transformation over all keyframes within each local submap, yielding metrically consistent dense depth predictions. To balance accuracy and computational efficiency, we introduce a hierarchical factor-graph architecture. At the local level, sliding-window optimisation estimates high-frequency camera poses and multi-view-refined 3D landmarks. Dense predictions are generated only for selected keyframes and registered within each submap using SE(3) Generalised ICP. Consecutive submaps are further aligned in Sim(3) using their shared geometry, allowing residual scale inconsistencies to be corrected. These intra- and inter-submap constraints are incorporated into a global factor graph to produce a consistent trajectory and dense metric reconstruction. Extensive experiments on public and in-house datasets demonstrate that the proposed framework achieves competitive odometry accuracy against geometric and learning-based baselines while providing accurate dense metric reconstruction and improved robustness to failures in learned depth prediction.

165. 【2611.25444】VisualTalk-Bench: Evaluating Question-Driven Avatar-Led Video Explanations

链接:https://arxiv.org/abs/2611.25444

作者:Xin Zhang,Tujie Xu,Zhiyi Chen,Xiaodong Zhang,Sicheng Yang,Minglei Li

类目:Computer Vision and Pattern Recognition (cs.CV)

关键词:answer user questions, avatar synthesis enable, synthesis enable, user questions, answer user

备注: 10 pages, 3 figures, 4 tables

点击查看摘要

Abstract:Advances in video generation and avatar synthesis enable new ways to answer user questions through visual explanations. This task requires a system to formulate a correct answer, translate relevant information into visual material, and deliver the explanation through an avatar's speech and actions. Existing benchmarks rarely assess whether avatars point to the visual content being discussed. We introduce VisualTalk-Bench for question-driven avatar-led video explanations in finance and Chinese and English education. It includes 800 curated reference videos and 100 independent evaluation requests, assessing Content Factuality, Speech-Visual Consistency, Visual Quality, Motion Smoothness, and Avatar-Content Pointing. Yet end-to-end video generators often struggle to align gestures with specific visual targets. To explore this challenge, we develop Avatar-O Agent, a modular reference implementation with explicit interaction planning. Experiments across six agents show that factual precision and visual quality do not guarantee accurate pointing. A 26-participant study comparing four agents finds that Avatar-O Agent receives the highest average ratings for visual guidance, perceived pointing accuracy, and explanation effectiveness. We will release the data and code.

166. 【2611.25437】Wearable Logo, Silent Hijack: Stealthy Cross-Modal Semantic Attacks on Visible--Infrared Vision-Language Models

链接:https://arxiv.org/abs/2611.25437

作者:Qike Zhang,Jiaju Han,Xuelian Shi,Xiang Chen,Dehe Zhao,Banglong Nong,Wenhao Zheng,Lei Wang,Jiahuan Long,Jiujiang Guo,Chengyin Hu

类目:Computer Vision and Pattern Recognition (cs.CV)

关键词:semantic associations make, daily necessities, targeted semantic attacks, naturally on clothing, clothing and daily

备注:

点击查看摘要

Abstract:Logos appear naturally on clothing and daily necessities, where their recognizable shapes and semantic associations make them familiar visual cues. This familiarity helps them blend into real-world environments and remain visually inconspicuous. We ask whether an existing logo can serve as a carrier for coordinated targeted semantic attacks on visible-infrared (VIS-IR) vision-language models (VLMs) without changing its original outline. This setting requires expressive adversarial variation within a fixed shape while accounting for different appearance characteristics and model responses across modalities. We propose Logo-guided Semantic Attack (LoSA), a score-based black-box framework that optimizes a logo's interior appearance and placement while preserving its silhouette. LoSA uses a contour field and low-frequency graph spectral modes to structure the interior color layout, a frozen appearance adapter to couple VIS and IR appearances, and particle swarm optimization to jointly optimize shared design parameters. Experiments across four datasets demonstrate that LoSA consistently outperforms the evaluated baselines in simultaneous Top-1 targeted success. Cross-architecture experiments, visual naturalness ratings, and captioning/VQA transfer analyses further characterize its effectiveness and limitations. These findings reveal that familiar visual symbols from real-world environments can serve as shape-preserving carriers for coordinated targeted semantic attacks across imaging modalities.

167. 【2611.25436】Raptor: Rejection-aware Adaptive Tree based on Speculative Decoding

链接:https://arxiv.org/abs/2611.25436

作者:Qishu Hu,Haozhao Wang,Zicong Hong,Yufeng Zhan,Yuanqing Xia

类目:Computer Vision and Pattern Recognition (cs.CV); Artificial Intelligence (cs.AI)

关键词:accelerates vision-language models, verifying multiple candidate, multiple candidate continuations, Tree-structured speculative decoding, Tree-structured speculative

备注: 18pages, 7figures, 3tables

点击查看摘要

Abstract:Tree-structured speculative decoding accelerates vision-language models (VLMs) by verifying multiple candidate continuations in parallel. However, long visual contexts increase the cost of speculative computation, making candidate coverage and redundant expansion important considerations beyond tree topology adaptation. We propose Raptor, a training-free framework that uses rejection feedback to jointly regulate candidate coverage and drafting effort. Our key insight is that acceptance length alone cannot distinguish insufficient candidate coverage from excessive deep expansion, and these conditions require different allocation decisions. Raptor therefore extracts complementary rejection-aware signals to characterize speculative progress, frontier coverage, and redundant expansion. These signals jointly guide bounded, momentum-smoothed updates of tree depth, branching width, and drafting budget, together with execution control that limits unnecessary drafting and progressively restores expansion when appropriate. Extensive experiments with two VLMs across seven multimodal benchmarks demonstrate the effectiveness of Raptor, achieving up to 2.39x average speedup over autoregressive decoding.

168. 【2611.25419】DiLE: Discrimination-Aware Latent-Space Learning for Universal Multimodal Embeddings

链接:https://arxiv.org/abs/2611.25419

作者:Ben Kang,Junda Wang,Yuxuan Yuan,Hang Li,Yuhan Liu,Xin Chen,Wanting Geng,Tong Lin,Pei Fu,Dong Wang,Jian Luan,Huchuan Lu

类目:Computer Vision and Pattern Recognition (cs.CV)

关键词:Multimodal large language, greatly improved input, universal multimodal embedding, improved input understanding, Multimodal large

备注:

点击查看摘要

Abstract:Multimodal large language models have greatly improved input understanding in universal multimodal embedding models. However, this understanding does not ensure that their embeddings preserve the evidence needed to distinguish relevant items from confusable alternatives. To preserve essential discriminative evidence in embeddings, this paper proposes DiLE, a discrimination-aware latent-space learning framework that combines task guidance with on-policy retrieval feedback. Specifically, a few learnable queries are used to encode task-relevant information in a compact latent space. Reasoning instructions and auxiliary answer-generation supervision shape these representations to support both task understanding and retrieval discrimination. To address task-specific discriminative needs, we design shared-private expert adaptation that integrates cross-task knowledge sharing with task-specific representation learning. Building on this, the on-policy hard-negative distillation is introduced: the student retrieves confusable candidates, and a reverse KL divergence objective aligns its embedding-based similarity distribution with the teacher's fine-grained relevance distribution over these candidates. Candidates are periodically refreshed using updated student snapshots, tying teacher supervision to the student's evolving retrieval behavior and confusions. Across four model configurations that span two backbone generations, DiLE achieves state-of-the-art performance on MMEB-v2, UVRB, and MRMR, demonstrating its effectiveness across modalities and tasks.

169. 【2611.25418】Bilateral Evidence Selection and Reasoning for Mammography Vision--Language Models

链接:https://arxiv.org/abs/2611.25418

作者:Zhengbo Zhou,Dooman Arefan,Shandong Wu

类目:Computer Vision and Pattern Recognition (cs.CV); Artificial Intelligence (cs.AI)

关键词:redundant visual information, extensive normal anatomy, Medical vision, identify small, visual information

备注:

点击查看摘要

Abstract:Medical vision--language models (VLMs) often need to identify small, subtle lesions within high-resolution images containing extensive normal anatomy and redundant visual information. Visual-token compression improves efficiency, but preserving salient regions alone can discard the context needed for diagnostic interpretation. This context can extend across images: in bilateral mammography, the opposite breast provides a patient-specific anatomical reference. Yet retaining such evidence does not ensure that the model uses it effectively. We propose Bilateral Evidence Selection and Reasoning (BESR), which connects evidence preservation with comparative assessment through a bilateral-guided token selector and a bilateral reward. Under a shared token budget, the selector jointly prioritizes lesion appearance and its local and bilateral context, while the reward encourages assessments that benefit from the preserved comparison. Experiments on a bilateral dataset constructed from CBIS-DDSM and on VinDr-Mammo demonstrate improved evidence retention and diagnostic performance under constrained visual-token budgets. Inference benchmarking further shows reduced prefill latency.

170. 【2611.25417】MIR-Depth: Multi-Range Refinement for Single-View RGB-D Depth Completion of Transparent Objects

链接:https://arxiv.org/abs/2611.25417

作者:Xinzhe Fu,Qi Shao,Yuanfei Xie,Taoyu Wu,Zhihao Cao,Long Hu,Hong Liang,Xianzhi Li,Anh Nguyen,Baoru Huang,Rongsheng Li

类目:Computer Vision and Pattern Recognition (cs.CV); Robotics (cs.RO)

关键词:single RGB-D image, Recovering accurate visible-surface, RGB-D image, single RGB-D, object mask

备注: 9 pages

点击查看摘要

Abstract:Recovering accurate visible-surface depth from a single RGB-D image and an object mask is essential for robotic grasping and manipulation of transparent objects. Specular reflections cause missing depth, while refraction corrupts nonzero returns with background depth, making raw measurements unreliable as surface anchors. Under a fixed sampling budget, refining an initial single-view prediction within a single search interval either misses large refraction offsets or sacrifices fine local resolution. To resolve this single-view trade-off, MIR-Depth constructs Coordinate-Aware Multi-Range Hypotheses: a 3D-aggregated global inverse-depth volume and depth-axis-filtered pixel-wise local volumes around a prior-guided initial depth. Selective Recurrent Refinement then introduces out-of-bounds lookup masking and validity-masked softmax fusion to iteratively update inverse depth from these volumes using a multi-scale ConvGRU. Finally, Spatially Gated Correction applies a learned per-pixel spatial gate to the full-resolution residual, preserving accurate initial geometry without imposing raw sensor readings as hard constraints. Experiments on TransCG, TODD, TRansPose, ClearPose, and HouseCat6D show that MIR-Depth achieves lower error than ReMake, DFNet, BP-Net, and PSD. Real-robot evaluations confirm accurate surface geometry for 6-DoF grasp pose prediction. Code and pretrained models will be publicly available.

171. 【2611.25414】Watertight Mesh Optimization of Textured Objects

链接:https://arxiv.org/abs/2611.25414

作者:Nathaniel Burgdorfer,Philippos Mordohai

类目:Computer Vision and Pattern Recognition (cs.CV)

关键词:reconstructing high-quality, existing approaches, reconstructed surfaces, approaches often excel, objective while compromising

备注:

点击查看摘要

Abstract:When reconstructing high-quality 3D assets from images, existing approaches often excel at one objective while compromising another, and frequently overlook the intrinsic quality of the reconstructed surfaces. We present MODL, a framework that jointly optimizes geometry and view-dependent appearance directly on a watertight triangle mesh. Our approach combines Laplacian-preconditioned optimization for smooth vertex updates with curvature-adaptive remeshing to preserve fine details and well-shaped triangles. The triangle mesh is optimized subject to multi-view normalized cross-correlation constraints, but without segmentation, geometric supervision or alpha-blending. Evaluations on object reconstruction datasets demonstrate that MODL achieves competitive novel-view synthesis while producing geometrically accurate models with superior intrinsic mesh quality. Our results highlight the potential of direct mesh optimization to unify photorealistic rendering, geometric reconstruction, and practical 3D asset creation.

172. 【2611.25408】FG-GraphBench: Benchmarking and Enhancing Fine-Grained Visual Understanding with Multimodal Scene Graphs

链接:https://arxiv.org/abs/2611.25408

作者:Sicheng Liu,Xunkai Li,Ziyu Han,Xingnan Zhang,Ru Zhang,Rong-Hua Li

类目:Machine Learning (cs.LG); Artificial Intelligence (cs.AI); Computer Vision and Pattern Recognition (cs.CV)

关键词:large language models, recognize small visual, requires multimodal large, multimodal large language, small visual entities

备注:

点击查看摘要

Abstract:Fine-grained visual understanding requires multimodal large language models (MLLMs) to recognize small visual entities and understand their relationships. Scene graphs offer a natural way to represent this information, organizing visual entities as nodes and their relationships as edges. However, we identify two challenges: existing scene graph benchmarks provide insufficient coverage of fine-grained entities and the relationships among them, and existing methods for fine-grained visual understanding have yet to effectively leverage such graph-structured information in MLLMs. To address these challenges, we introduce Fine-Grained Multimodal Scene Graph Benchmark (FG-GraphBench) and FiGRA. FG-GraphBench is a benchmark comprising three complementary tasks: fine-grained visual question answering (FG-QA), image captioning (FG-Caption), and image description generation (FG-Description). We develop a two-stage construction pipeline that discovers and grounds fine-grained entities and establishes their relations to construct the fine-grained multimodal scene graphs (MSGs) in FG-GraphBench. Building on these MSGs, we propose FiGRA, a Fine-grained Graph Representation Augmentation method for MLLMs. FiGRA adopts a two-stage training strategy. It encodes multimodal entity and relation features as continuous graph tokens, and integrates them into the MLLM. Experiments show that FiGRA significantly outperforms strong baselines across three tasks of FG-GraphBench. Further analyses confirm the effectiveness of FG-GraphBench and highlight the importance of preserving fine-grained entity and relation information for visual understanding.

173. 【2611.25406】Decision Boundaries Matter: Graph-Regularized Residual Adaptation for Cross-Domain Fake Image Detection

链接:https://arxiv.org/abs/2611.25406

作者:Shichuang Xie,Qishuai Ding,Tong Qiao,Liquan Shen,Guorui Feng

类目:Computer Vision and Pattern Recognition (cs.CV)

关键词:mismatched decision boundaries, fake image detectors, fake image, due to mismatched, learning transferable features

备注:

点击查看摘要

Abstract:Despite learning transferable features, fake image detectors may struggle on unseen domains due to mismatched decision boundaries. Our analysis shows that features extracted by the frozen forensic encoder remain discriminative, but the source-trained decision boundary does not fully exploit this information. However, correcting this boundary without source data or target labels remains challenging, as unreliable source predictions may lead to incorrect adaptation. To address this issue, we propose Graph-Regularized Residual Adaptation (GraRA), which refines the source decision boundary using only unlabeled target data. To determine which classifier directions to update, GraRA constructs a compact residual subspace from uncertainty-weighted target features and keeps it orthogonal to the source classification direction. To guide these updates, it uses source-score anchors to generate soft labels through target-graph diffusion, while neighborhood consistency regularizes residual learning within the constrained subspace. The learned correction is merged into the source classifier, requiring only 32 optimized coefficients and no additional inference branches. On diverse unseen OpenMMSec domains, GraRA achieves 84.36\% accuracy and 91.34\% AUC, improving over the source detector by 4.33 and 3.50 percentage points, respectively. Further analysis shows that reliable source predictions and locally consistent neighborhoods help guide residual learning, allowing GraRA to correct ranking errors that a simple threshold shift cannot address. \textcolor{cvprblue}{Code will be made publicly available.

174. 【2611.25391】HOPD: Historical On-Policy Distillation for SAR MLLM Continual Learning

链接:https://arxiv.org/abs/2611.25391

作者:Yuxuan Li,Derrick Goh Xin Deik,Zheng-Peng Duan,Qianru Sun,Weijie Li

类目:Computer Vision and Pattern Recognition (cs.CV)

关键词:multimodal large language, large language models, interpretation requires multimodal, requires multimodal large, learn incoming sources

备注:

点击查看摘要

Abstract:Continual SAR interpretation requires multimodal large language models to learn incoming sources while retaining earlier task abilities. This is difficult when acquisition and geographic shifts alter visual evidence, regionally concentrated categories bias answers, and historical images are scarce. We address this problem with Historical On-Policy Distillation (HOPD), which reuses source-acquisition LoRA states as persistent teachers for one evolving student. Archiving each state before consolidation preserves a stable source reference. The teachers supervise student-generated prefixes, transferring historical behavior to the student's changing generation contexts. To enrich sparse replay, original-view teachers guide augmented historical student views while preserving spatial coordinates. This knowledge is consolidated into a single deployed adapter that requires no source identity. On SARVQA-CL, our source-incremental evaluation protocol, HOPD achieves the highest final mean score of 54.51 among the compared methods, exceeding the strongest baseline by 2.96 points. It also improves the final source-macro score in every evaluated task over shared-memory Replay-SFT, with gains across source orders. These results support source-acquisition history as supervision for retaining earlier task performance during continual SAR adaptation. Code and data: this https URL.

175. 【2611.25389】Zero to Four Views: Unified Panorama Generation and Preference RL for Unobserved Regions

链接:https://arxiv.org/abs/2611.25389

作者:Rui Hong,Xugang Ye,Jana Košecká,Kai Fan

类目:Computer Vision and Pattern Recognition (cs.CV)

关键词:multi-shot video generation, shared scene backgrounds, existing panorama generators, single input image, video generation

备注:

点击查看摘要

Abstract:Panoramas can serve as shared scene backgrounds for multi-shot video generation, but most existing panorama generators take text alone or a single input image. We present a single model that generates a full equirectangular panorama from a text caption and K in {0,...,4} perspective conditioning views taken by a four-camera rig at one viewpoint. We fine-tune a pretrained text-to-image rectified-flow transformer in two stages. In the first stage, each conditioning view is appended to the token sequence together with the ID of its camera slot, and the number of views varies during training, so one set of weights serves every input setting. In the second stage, we improve the region not covered by any conditioning view. Supervised training computes its loss on noisy latents of the ground truth and never scores a panorama the model has generated and decoded, so we propose Uncovered-Region GRPO (UR-GRPO), a reinforcement-learning recipe that rewards the candidates a vision-language model prefers in pairwise comparisons on crops around this region. Because the slot ID alone does not tell the model which way the rig points, we also offer an optional variant that writes each view's camera direction into its position encoding, which lets the user decide where each view appears in the output. With one conditioning view, our model outperforms the previous state of the art, 360Anything, trained on the same data and evaluated with the same code, on Laval Indoor and gives mixed results on an indoor subset of SUN360; the second stage improves the results for every number of conditioning views.

176. 【2611.25382】xt Lives in the Guidance: Preventing Silent Text Rewriting in Cached Diffusion Transformers

链接:https://arxiv.org/abs/2611.25382

作者:Kun Zhou,Wenmian Yang,Zhensheng Wang,Weijia Jia

类目:Computer Vision and Pattern Recognition (cs.CV)

关键词:Diffusion transformers, text, Diffusion, feature caching, legible text

备注:

点击查看摘要

Abstract:Diffusion transformers (DiTs) now render long, legible text in images, and feature caching is a standard way to accelerate them. Caches are judged by image-level metrics, which do not check whether the rendered text is still correct. We find that caching silently rewrites this text. On Z-Image, TaylorSeer at 2.9x fewer FLOPs lowers word accuracy from 62% to 46%, while ImageReward and CLIP Score barely change. Sampling with fewer steps at the same compute keeps almost all of this text, so the loss comes from how the cache predicts the skipped steps. Decomposing the cache error along classifier-free guidance (CFG), we trace this loss mainly to the guidance term, the difference between the conditional and unconditional predictions, which caches extrapolate worst on text. Correcting the guidance inside the text alone recovers most of the lost text. We therefore propose GuideCache, which recomputes both CFG branches only on the tokens with the largest predicted guidance error, using the guidance terms that the cache already stores. At 2.4x fewer FLOPs, GuideCache restores full-compute prompt fidelity on Z-Image and Qwen-Image, and on Z-Image it renders text as well as or better than ToCa, TeaCache, and FasterCache while staying much closer to the full model's output. GuideCache works with caches of any order, and on top of fewer sampling steps it keeps more text than step reduction at a higher speedup. These results suggest that acceleration methods for text-rendering models should be evaluated on the text they render.

177. 【2611.25371】A Unified Framework for Image Manipulation Localization Under Multi-Type Weak Supervision

链接:https://arxiv.org/abs/2611.25371

作者:Songlin Li,Hui Ma,Zhiqing Guo,Dan Ma,Gaobo Yang,Xin Liao

类目:Computer Vision and Pattern Recognition (cs.CV)

关键词:Existing weakly supervised, weakly supervised image, annotation-specific training strategies, Existing weakly, image manipulation localization

备注:

点击查看摘要

Abstract:Existing weakly supervised image manipulation localization (IML) methods typically rely on annotation-specific training strategies, hindering the joint use of diverse annotations within a single model. This limits the scale of usable training data and constrains further progress. We propose UniIML, a teacher-student framework that maps diverse weak annotations into a unified representation to guide the Segment Anything Model (SAM) in generating multiple candidate masks. However, SAM is designed for general-purpose segmentation and has limited sensitivity to manipulation traces; its predicted quality scores alone may therefore fail to reliably identify the candidate that best delineates the manipulated region. We introduce Ordinal Safe Consensus (OSC), which first filters candidates using trusted weak-annotation constraints encoded in the unified representation. It then ranks the remaining candidates separately by consistency with weak annotations, SAM-predicted quality scores, and cross-view stability, aggregating these rankings for an initial selection. To assess how changes in the evaluation criteria affect candidate selection, OSC repeats ranking and selection while omitting one criterion at a time. The most frequently selected candidate is adopted as the final mask, and its relative selection frequency quantifies selection stability. We construct soft pseudo-labels from the selected candidate's predicted probabilities and estimate pixel-level supervision weights based on selection stability, pixel-level agreement among candidates, and predictive certainty. The student receives direct supervision in regions defined by weak annotations and learns from soft pseudo-labels elsewhere through reliability-weighted distillation, enabling unified training across weak annotation types. Experiments show that UniIML achieves superior average performance compared with state-of-the-art IML methods.

178. 【2611.25369】Fold the Loop: Decoding-Time KV Memory for Looped Vision-Language Models

链接:https://arxiv.org/abs/2611.25369

作者:Kun Zhou,Wenmian Yang,Zhensheng Wang,Yanzeng Li,Weijia Jia

类目:Computer Vision and Pattern Recognition (cs.CV)

关键词:Looped vision-language models, repeatedly applying weight-tied, Looped vision-language, deepen visual reasoning, applying weight-tied modules

备注:

点击查看摘要

Abstract:Looped vision-language models (VLMs) deepen visual reasoning by repeatedly applying weight-tied modules. However, their decoding memory grows with recurrent depth and limits the image size and batch size that fit in a fixed memory budget. Each recurrent step produces its own key-value (KV) states, so the cache holds several copies of the same image. We find that, after an exact prefill, earlier steps still need access to visual information but can reuse the final visual KV within the same module. We propose FoldCache, which compresses visual KV along the recurrent axis while preserving the full prefill computation. FoldCache partitions recurrent steps within each module into groups and retains the final visual KV of each group for its members to share during decoding. Sharing stays within module boundaries, while question and answer KV remain separate for each step. The grouping granularity controls the degree of compression, and folding can be combined with token eviction and quantization to further reduce memory use. On LoopVL, FoldCache approximately halves prefix KV memory while matching full-cache accuracy on five open-ended benchmarks. These memory savings support batch sizes up to 1.7 times those of a full cache under the same GPU memory budget, and up to 2.6 times with the more aggressive FoldCache-Lite.

179. 【2611.25347】Search or Scoring? Diagnosing Why Visual World-Model Planners Fail

链接:https://arxiv.org/abs/2611.25347

作者:Haoxuan Jia,Yang Liu,Shang Luo,Chenhao Wu,Zixiang Ni,Yingguang Yang,Feng Zhang,Chongyang Zhang,Hao Zheng,Kefu Xu,Congjing Ran,Jifeng Zhu,Bin Chong

类目:Computer Vision and Pattern Recognition (cs.CV)

关键词:Visual world models, JEPA-WM plan, Visual world, DINO-WM and JEPA-WM, cost

备注: 43 pages, 10 figures, 30 tables; supplementary material included

点击查看摘要

Abstract:Visual world models such as DINO-WM and JEPA-WM plan with an optimizer that searches for actions whose predicted outcome has the lowest cost, its distance to a goal image. A plan can thus fail in two ways: the optimizer never tries a good action that the cost would prefer (a search failure), or the cost, computed on predictions, ranks a bad action first (a scoring failure). The two need different remedies, more search or a better model, but a planner does not record which one occurred. We tell them apart by executing the optimizer's candidates, scoring a known solution with the planner's own cost, and comparing costs on predicted and real outcomes. An error model explains when each arises and so how to use retrieved demonstration windows (DemoProp): start the search from them (a warm start) when prediction errors are small, and average them without the model when large errors make the optimizer pick overrated actions. Across four simulated environments and three world models, on short tasks whose goals the planner can represent, PushT and Wall fail mostly by search, while Metaworld and RoboCasa fail by scoring because the cost, sound on real outcomes (AUROC 0.94 on Metaworld), is at chance on predictions (0.54). The diagnosis tells which remedy pays off: a four times larger search lowers the real-outcome cost only where failures come from search (clearly for DINO-WM), and where they come from scoring, averaging the windows without the model solves 127 of 168 new Metaworld tasks, against at most 87 when the model chooses among them. Within PushT, degrading only the model flips which use is better, and our rule, frozen in advance, follows the flip and solves 678 of 960 new episodes, against 530 for the best fixed use.

180. 【2611.25340】SceneAnything: Agentic Reconstruction of Large-Scale Mesh Scenes from a Single Video

链接:https://arxiv.org/abs/2611.25340

作者:Bowen Xue,Saeed Hadadan,Fabrice Rousselle,Zahra Montazeri,Ravi Ramamoorthi,Milos Hasan

类目:Computer Vision and Pattern Recognition (cs.CV); Graphics (cs.GR)

关键词:requires matching visible, matching visible structure, video requires matching, completing unseen surfaces, plausibly completing unseen

备注:

点击查看摘要

Abstract:Reconstructing an editable 3D scene from video requires matching visible structure while plausibly completing unseen surfaces. Agents can write scene-construction programs, but direct reconstruction attempts often produce errors in layout, scale, and lighting. We introduce SceneAnything, an automated system for reconstructing mesh scenes from real and generated videos, enabling text-to-scene construction when paired with video generation. A software harness coordinates agents, pretrained vision models, and graphics tools through an explicit reconstruction protocol. Shared camera, depth, and segmentation estimates ground component construction: a scene-owner agent plans the environment and briefs authors, who write parameterized Blender programs. A visibility- and fit-based level-of-detail rule selects the construction level, while unseen regions are completed and marked as inferred. Assembled scenes use fitted lighting and supported motion, and revisions undergo programmatic checks and independent visual comparisons. Accepted programs and lessons are reused across scenes. We evaluate on 100 scenes; subset comparisons show improved original-view fidelity over direct-agent reconstruction and higher novel-view content similarity than the compared video-to-3D methods. We will release the harness, agent instructions, reusable libraries, and scenes.

181. 【2611.25338】VideoRNG: Relightable Neural Gaussians from Generative Video Models

链接:https://arxiv.org/abs/2611.25338

作者:Bowen Xue,Saeed Hadadan,Fabrice Rousselle,Ravi Ramamoorthi,Zahra Montazeri,Miloš Hašan

类目:Computer Vision and Pattern Recognition (cs.CV); Graphics (cs.GR)

关键词:existing Gaussian-splatting generators, Gaussian-splatting generators typically, compact physically based, typically bake fixed, physically based materials

备注:

点击查看摘要

Abstract:Generating realistic 3D assets remains constrained by classical representations: surface meshes and compact physically based materials struggle to express objects whose appearance arises from fine geometry or complex light transport, while existing Gaussian-splatting generators typically bake fixed illumination into appearance. We introduce VideoRNG, a two-stage method that generates complete, relightable 3D assets from a text prompt or a single image without prescribing a mesh or analytic material model. A video generator fine-tuned from a pretrained video diffusion backbone first produces a structured RGBA turntable sequence under a known pose, camera, and illumination. A feed-forward large reconstruction model initialized from a pretrained video diffusion backbone then maps the video-VAE latent frames of selected turntable frames directly to millions of anisotropic Gaussians with learned neural appearance features. A shared directional decoder and stochastic Gaussian ray tracer turn this representation, which we call ray-traced Relightable Neural Gaussians (rtRNG), into radiance under novel viewpoints and illumination. To preserve high-frequency appearance while learning geometry, we introduce depth-anchored appearance training, and we make million-Gaussian reconstruction practical through foreground token selection and cost-aware distributed sampling. VideoRNG generates complex assets such as fur, feathers, and spider webs that can be interactively rendered, relit, and integrated into scenes.

182. 【2611.25333】MoGloss: Joint Motion and Pseudo-Gloss Pretraining for American Sign Language Understanding

链接:https://arxiv.org/abs/2611.25333

作者:Rui Hong,Jana Košecká

类目:Computer Vision and Pattern Recognition (cs.CV)

关键词:American Sign Language, shown to benefit, Sign language, Sign language translation, ASL STEM Wiki

备注: 15 pages, 4 figures. Project page: [this https URL](https://hongrui16.github.io/MoGloss)

点击查看摘要

Abstract:Sign language translation and recognition have been shown to benefit from various pretraining approaches. We propose a novel two-stage pretraining approach for the motion encoder, which first learns to reconstruct masked motion, followed by joint masked prediction of pseudo-gloss units and motion. Pseudo-glosses serve as an intermediate structured representation of natural language text and are obtained from the captions by a large language model. We curate a corpus of about 1,100 hours of American Sign Language from five public datasets, estimate 2D pose for all clips and define an evaluation protocol. We then reproduce state-of-the-art translation methods and evaluate them using the same train and test splits. We propose a translation model built on our pretraining strategy, and this model outperforms the previous state-of-the-art methods Uni-Sign and SHuBERT, as well as the YouTube-ASL baseline, on the How2Sign, ASL STEM Wiki and OpenASL datasets. Furthermore, the gain from the joint pretraining using motion and pseudo-gloss grows as translation data shrinks. A classifier built on the same encoder also achieves state-of-the-art accuracy on ASL Citizen isolated sign recognition and outperforms our reproduction of Uni-Sign on WLASL2000.

183. 【2611.25309】VoxForge: A Unified Foundation Model for Dense 3D Features Across MRI, CT and PET

链接:https://arxiv.org/abs/2611.25309

作者:Rushil Anirudh,Jayaraman J. Thiagarajan

类目:Computer Vision and Pattern Recognition (cs.CV); Machine Learning (cs.LG)

关键词:medical foundation model, foundation model promises, model promises reusable, wide variety, specialist models

备注: 34 pages (8-page main paper, references, and 23-page supplementary material), 7 figures. Rushil Anirudh and Jayaraman J. Thiagarajan contributed equally

点击查看摘要

Abstract:A unified 3D medical foundation model promises reusable, frozen features that serve a wide variety of tasks such as segmentation, classification and registration on MRI, CT and PET as well as specialist models do. Generalist models still need separate backbones for localized and global tasks, specialist models are confined to a single modality, with the concern that spanning modalities under-emphasizes fine anatomy, and both typically rely on end-to-end fine-tuning to be competitive. We present VoxForge, a single encoder that integrates recent advances in 3D architecture, self-distillation and cross-modal curricula to learn transferable features from permissively licensed MRI and CT volumes. We evaluate it strictly frozen, the test a reusable foundation model must pass, against state-of-the-art generalist and specialist models on segmentation, classification and registration across MRI and CT, and on PET, which it never sees. VoxForge has the best or tied-best average rank in every task group, serves localized and global tasks across modalities at the level of state-of-the-art specialists, and is extremely label-efficient, matching top-performing models on TotalSegmentator CT with a tenth of the labels.

184. 【2611.25297】ScaleDiag: Diagnosing the Limits of Test-Time Scaling in GUI Grounding

链接:https://arxiv.org/abs/2611.25297

作者:Chen Liu,Zhen Fang,Xin Yu,Dadong Wang,Ling Chen

类目:Computer Vision and Pattern Recognition (cs.CV)

关键词:intended interface element, GUI grounding maps, natural language instruction, GUI grounding, interface element

备注: Accepted by ICDM@D2AI 2026 Best Paper Award

点击查看摘要

Abstract:GUI grounding maps a screenshot and a natural language instruction to the coordinates of the intended interface element. Existing approaches improve grounding through visual scaling, instruction scaling, and sampling scaling, yet it remains unclear which grounding bottlenecks these strategies address and where their gains saturate. To address this gap, we introduce ScaleDiag, a controlled diagnostic framework that isolates the effects of visual input, instruction information, and prediction sampling on GUI grounding by independently varying target scale and surrounding context, target-specific information and auxiliary descriptions, and candidate coverage and selection, respectively. We evaluate ScaleDiag across five benchmarks and up to eight vision-language models and find that each scaling strategy operates within a much narrower regime than commonly assumed. (1) Visual scaling is governed by target occupancy rather than a fixed crop ratio, with full-screen errors typically rescued once the target occupies only 0.32% of the cropped view, while further zooming eventually becomes harmful. (2) Instruction scaling is driven by information that identifies the target rather than description richness itself, as even ground-truth-derived attributes such as color, size, and shape are often neutral or harmful, while appearance and spatial descriptions account for over 90% of the achievable gain. (3) Repeated sampling exposes a different limitation, as correct coordinates are frequently generated but remain difficult to select, leaving 5.1--11.0 points of oracle headroom while geometric consensus provides almost no improvement. Together, these results show that test-time scaling is constrained by distinct operating regimes rather than benefiting monotonically from more visual focus, richer instructions, or additional samples. Our code is available at this https URL.

185. 【2611.25296】CalibCache: Cache-to-Full Feature Dynamics Calibration for Diffusion Transformers

链接:https://arxiv.org/abs/2611.25296

作者:Mingyu Liang,Dingkun Xu,Jingwei Xu

类目:Computer Vision and Pattern Recognition (cs.CV)

关键词:Diffusion Transformers generate, Transformers generate high-quality, Diffusion Transformers, generate high-quality images, remains computationally expensive

备注:

点击查看摘要

Abstract:Diffusion Transformers generate high-quality images, but iterative sampling remains computationally expensive. Feature caching reduces the number of full network evaluations, yet its approximation error can become more severe under short sampling schedules, where fewer full-compute steps and a coarser sampling grid can make history-based extrapolation less reliable. In this paper, we propose CalibCache, a lightweight offline calibration method that corrects systematic discrepancies between cached and full-compute feature dynamics. Specifically, CalibCache calibrates feature acceleration, defined as the second-order finite difference across sampling steps. Using only a single pair of full-compute and naive-cache trajectories at each setting, CalibCache fits scalar acceleration maps offline by centered least squares. These maps are then applied during sampling without additional full-network evaluations or online optimization. Across FLUX.1-dev and Qwen-Image on DrawBench and DiT-XL/2 on ImageNet-256, CalibCache consistently improves fidelity or generation quality over TaylorSeer under 20-step sampling at cache intervals $N=3,4,5$. At $N=5$, it reduces FLUX.1-dev LPIPS from $0.563$ to $0.429$ while achieving a $2.86\times$ denoising speedup over full-compute sampling, and lowers DiT-XL/2 FID from $16.39$ to $10.37$ relative to TaylorSeer.

186. 【2611.25283】Persistent Identity in Person Tracking: Online Enrollment and Registered Recognition

链接:https://arxiv.org/abs/2611.25283

作者:Ratha Siv,Matei Mancas,Bernard Gosselin

类目:Computer Vision and Pattern Recognition (cs.CV)

关键词:online appearance-bank matching, appearance-bank matching, extend PoseTReID, PoseTReID with online, online appearance-bank

备注: 7 pages, 2 figures, 7 tables. An earlier version of this paper was submitted to an IEEE conference

点击查看摘要

Abstract:We extend PoseTReID with online appearance-bank matching (Gallery) and a residual-conflict guard for registered SVM recognition. All methods use supplied ground-truth (GT) person boxes. We average all-box assessment IDF1/HOTA equally across videos. On six SportsMOT clips, StrongSORT leads standalone trackers (79.44/84.72%) and exceeds both Gallery variants. Over eleven videos, Gallery AIN (83.29/85.44%) and COPE (84.66/86.38%) exceed standalone trackers with complete results. This lead depends on two retrospectively added GTA development compilations and is absent on the original nine videos. Registered SVMs lead both cohorts among complete results, with AIN at 95.72/94.82% over eleven videos. They use every GT crop from approximately 20% of frames, including future appearances. On MOT20-03, registered AIN reached 96.80% IDF1 and assigned the correct enrolled name to 90.72% of assessment GT observations. Gallery and standalone trackers receive no per-video labelled enrollment. Both identity routes restore identities across internal title cards, but Gallery trails SORT on three videos (four in HOTA) and both SVMs trail SORT on MOT16-04/MOT20-03. Under six-clip persistent-only scoring, native Gallery gains 2.39 percentage points in mean IDF1 over a matched control, with lower correct-P coverage and 5.02 times as many encoder attempts. Pose gating is mixed; the guard's metric effect is not isolated.

187. 【2611.25279】Accord: Universal Online Metric Localization and Dense Mapping from one Camera and an IMU

链接:https://arxiv.org/abs/2611.25279

作者:Jaafar Mahmoud,Arthur Movsesyan,Cuong Huynh,Mikhail Iumanov,Sergey Zagoruyko,Sergey Kolyubin

类目:Computer Vision and Pattern Recognition (cs.CV)

关键词:IMU remains challenging, predict dense structure, IMU remains, visual-inertial odometry, remains challenging

备注:

点击查看摘要

Abstract:Dense metric mapping from a single camera and IMU remains challenging: visual-inertial odometry (VIO) provides metric motion but sparse geometry, whereas feed-forward depth models predict dense structure without ensuring metric scale. We present Accord, which constructs an accurate online dense metric map from any camera with an IMU. A VIO filter publishes metric poses, a depth foundation model conditioned on them predicts depth for short keyframe windows, and a factor graph fuses the resulting submaps. Recent work has explored several approaches to this problem. What is essential is the multi-view agreement (Accord), which is considered in SLAM as a source of geometric self-consistency. Accord anchors each keyframe window to metric 3D points triangulated from the filter poses, then exploits bidirectional dense depth agreement between co-visible images to estimate and correct per-image scale errors while preserving the metric scale established by the anchors. Context keyframes further improve depth prediction, and the resulting submaps are fused in a factor graph. We evaluate a single mapping configuration across five public camera-IMU setups and a new Phone-rig/GoPro dataset. With nothing tuned per dataset/setup and every map aligned only by Accord's own trajectory, Accord achieves consistently best/second-best mean F-score at 5 cm on every platform. Accord is open-source along with the phone-rig dataset for the benefit of the research community

188. 【2611.25265】Real Cameras are Radiometric

链接:https://arxiv.org/abs/2611.25265

作者:Ties Harder,Felix Seegräber,David Nakath,Hendrik Sauer,Kevin Köser,James Pritts

类目:Computer Vision and Pattern Recognition (cs.CV)

关键词:radiometric, geometric camera model, model, camera model, geometric calibration

备注: 22 pages, including supplementary material

点击查看摘要

Abstract:We propose the radiometric camera, a model that lifts any central geometric camera model to a radiometric one. The lifted model keeps the projection of the geometric model and adds a finite aperture, a free-form focus surface and a generic vignetting function. We calibrate it by inverse rendering directly on the relative irradiance of the sensor, from the same images of standard calibration targets used in geometric calibration. Geometric calibration absorbs the bias of its feature detections. The proposed radiometric calibration uses no features, and our radiometric cameras have lower reprojection errors on held-out test sets. The focus surface extends defocus modeling from the thin lens to any central camera, and we use it in a first multi-view radiometric reconstruction, which recovers the in-focus reflectance of a texture from defocused images. The vignetting is estimated jointly and accurately reproduces a flat field measured through a diffuser. Because every standard camera model lifts to a radiometric one, the radiometric camera naturally and directly extends generative scene models beyond multi-view geometry to multi-view radiometric ones, where phenomena such as lighting and attenuation can be accurately estimated.

189. 【2611.25257】Keel: A Frontier Model Teaches a Vision-Language Model by Rewriting Its Weights

链接:https://arxiv.org/abs/2611.25257

作者:Jiajun Fan,Neil He,Bangji Yang,Qinglong Zheng,Ziwen Wang,Chaoran Cheng,Changyi Yang,Hanqi Mao,Ge Liu

类目:Computer Vision and Pattern Recognition (cs.CV); Artificial Intelligence (cs.AI)

关键词:weights, frontier model repair, Small reasoning VLMs, vision-language model, VLM

备注:

点击查看摘要

Abstract:Can a frontier model repair a vision-language model (VLM) by rewriting its weights? Small reasoning VLMs such as Qwen3.5 need such a repair: in thinking mode they fall into loops on many image questions, repeating themselves until the token limit, spending the whole budget and losing the answer. The usual remedies act on the decoding, leaving the looping model as it was, or must be told what to change. Yet the cause is written in the weights. Keel, the first agentic weight-editing method for VLMs on image tasks, finds and removes it there. Told only the symptom, an editor, a frontier multimodal LLM acting as an agent, locates the cause, edits the weights and tests them, with a failure census and instruments showing it the failing images and where the VLM attends; among several editors, the harness keeps only edits that clearly reduce the failure without lowering accuracy on questions no editor saw. The repair needs no training, keeps the model's size and per-token cost, and applies to already-edited weights. The editors expose where the loops live: from the original weights, independent editors on both models trace them to the attention outputs of the middle layers, and the smaller VLM stops consulting the image inside a loop. On Qwen3.5-0.8B and Qwen3.5-2B, across six image benchmarks from visual mathematics to GUI grounding, the edit removes most loops on every benchmark and shortens answers, and the edited models are significantly more accurate than the thinking models on average; the larger one gains most on GUI grounding, a task absent from the practice questions. Because the repair lives in the weights, it combines with budget forcing where held-out questions favor it: on the best available weights, Keel has the best average accuracy on both models, significantly above every training-free baseline we compare on the larger one.

190. 【2611.25248】Branch Caching After Step Distillation in Four- and Eight-Step Image Generators

链接:https://arxiv.org/abs/2611.25248

作者:Mingyu Liang,Dingkun Xu

类目:Computer Vision and Pattern Recognition (cs.CV)

关键词:remaining denoising steps, generator save time, distilled image generator, image generator save, generator save

备注:

点击查看摘要

Abstract:Can a distilled image generator save time by reusing features between its few remaining denoising steps? We study a fixed branch cache that replaces selected attention and MLP evaluations while preserving the student's noise schedule. One full trajectory and one raw-cache trajectory from the same initial noise supply a scalar correction for each branch type and reuse position. At $1024^2$, one internal reuse reduces four-step Qwen-Image-Lightning denoising time from 1.279 to 0.977 seconds. Three reuses yield $1.51\times$ speedup on eight-step Lightning and $1.53\times$ on eight-step Hyper-FLUX. We evaluate image quality on all 200 DrawBench prompts. With the reuse positions held fixed, correction improves mean same-seed image fidelity over raw reuse by 0.66 and 0.87 dB on the two Lightning students. On Hyper-FLUX the PSNR difference is 0.01 dB, while LPIPS improves from 0.1625 to 0.1546. These results measure the effect of branch caching and correction while retaining each student's original sampling grid.

191. 【2611.25239】Beyond Restoration: Content-Degradation Factor Unfolding for Real-World Concealed Object Segmentation

链接:https://arxiv.org/abs/2611.25239

作者:Rihan Zhang,Fengyang Xiao,Yuhan Chen,Sina Farsiu

类目:Computer Vision and Pattern Recognition (cs.CV); Artificial Intelligence (cs.AI)

关键词:requires distinguishing task-relevant, distinguishing task-relevant content, degradation requires distinguishing, concealed object segmentation, Reliable visual perception

备注:

点击查看摘要

Abstract:Reliable visual perception under degradation requires distinguishing task-relevant content from degradation-induced variations, especially in concealed object segmentation (COS), where foreground cues are inherently weak. The central challenge is not merely improving visibility, but identifying reliable content evidence amid degradation-induced variations. RGB-space segmentation often entangles content and degradation cues, while restoration-based methods primarily focus on image recovery and task interaction. This motivates explicit factorization to distinguish content evidence from degradation effects and assess its reliability. Retinex provides a spatially aligned basis for this separation but remains illumination-centric. We therefore extend it to content-degradation formation with task-asymmetric factor roles. The Factor Unfolding Network (FUN) uses the content factor R as the sole source of direct semantic evidence for segmentation, while the degradation factor L captures degradation-related variations and generates a spatial reliability map to calibrate this evidence. A unified energy couples observation formation with reliability-weighted foreground-background separation, and its alternating updates yield a coarse-to-fine network without clean-image supervision. In this way, FUN factorizes degradation for perception rather than removing it. Extensive experiments across clean and degraded conditions and diverse segmentation tasks demonstrate robust performance and generalization. Code will be released.

192. 【2611.25228】MKA: Evaluating Multimodal Knowledge Agents on end-to-end professional tasks

链接:https://arxiv.org/abs/2611.25228

作者:Anima Singh,Suhas Yogin,Weicheng Kuo,Ivor Rendulic,Ivan Jurin,Antoine Yang,Oran Kelly,Rohan Doshi,Hammad Ayyubi,Ioana Bica,Sergi Caelles,Matthew Conniff,Dan Graur,Anastasija Ilić,Ahmet Iscen,Chao Jia,Tomasz Kępa,Chunyuan Li,Noah Ó Donnaile,Filip Pavetic,Goran Petrovic,Khalid Salama,Zhicheng Wang,Polina Zablotskaia,Yiqian Zhang,Ming Zhao,Benjamin Caine,Mario Lučić,Melvin Johnson,Radu Soricut,Jean-Baptiste Alayrac

类目:Artificial Intelligence (cs.AI); Computer Vision and Pattern Recognition (cs.CV)

关键词:professional knowledge work, autonomous agents perform, Multimodal Knowledge Agent, professional knowledge, knowledge work

备注:

点击查看摘要

Abstract:We introduce the Multimodal Knowledge Agent (MKA) benchmark, which measures how well autonomous agents perform end-to-end professional knowledge work that depends on visual understanding. Each task requires an agent to extract and combine information from multiple visually dense files, such as CAD blueprints, electronics schematics, radiographs, complex charts and geospatial maps, and to produce a standalone deliverable in .xlsx, .pptx, .pdf, .docx, .html or .png format. Existing benchmarks leave this capability largely untested: removing image-viewing tools leaves Gemini 4 Carbon's score on GDPval unchanged, and single-format evaluations such as this http URL and this http URL do not require reasoning across files or modalities. MKA contains 200 tasks across 10 economic sectors. Deliverables are graded against expert-weighted rubrics by a sandboxed agentic autorater that agrees with human domain experts on 96.2% of criterion judgments in our expert-graded study (Cohen's $\kappa = 0.91$), scored by weighted satisfaction rate (WSR) overall and on frontier criteria unsaturated by prior-generation models (Climb-WSR). Across the frontier models we evaluate (Gemini 4 Carbon, Claude Fable 5.2, Claude Opus 5.5, and GPT-6 Astra), Claude Fable 5.2 (56.6% Climb-WSR, 73.3% Overall WSR) and Gemini 4 Carbon (54.3% Climb-WSR, 71.9% Overall WSR) achieve the highest scores, while Carbon uses $5.7\times$ fewer tokens. Even for the strongest models, Climb-WSR remains below 57%, leaving substantial headroom for multimodal agents.

193. 【2611.25226】Reject and Report: Forgery-Evident Undetectable Watermarks for Generative Image Models

链接:https://arxiv.org/abs/2611.25226

作者:Yihan Wang,Haochen Sun,Xi He,Yaoliang Yu,N. Asokan

类目:Cryptography and Security (cs.CR); Artificial Intelligence (cs.AI); Computer Vision and Pattern Recognition (cs.CV)

关键词:Provenance schemes, traced back, Provenance, generative model, watermarking scheme

备注:

点击查看摘要

Abstract:Provenance schemes such as those based on content watermarks let images from a generative model be traced back to it. A practical watermarking scheme must be accurate at a low false-positive rate (FPR), robust under content-preserving processing, and undetectable. In addition, it must resist forgery; otherwise, even from a single watermarked image and a surrogate model, an adversary can imprint the watermark onto an arbitrary image or regenerate new content that carries it, falsely attributing harmful content to a model or making authentic photographs appear AI-generated. We argue that a watermarking scheme should not only reject forgeries but also report them: a forgery rejected as unwatermarked leaves no trace of the attempt. State-of-the-art forgery-resilient undetectable watermarking schemes, such as SemBind, fall short on both counts. External anomaly detectors that do report forgeries either miss most of them or falsely flag genuine images after common processing. We propose Tripwire-WM, the first forgery-evident framework for undetectable watermarks. It commits the semantics of the generated image to the magnitudes of the initial latent noise, which the sign-based watermarks leave unused, through a distribution-preserving embedding; a forged image carries a valid watermark but with a commitment that does not match its content, so the detector both rejects it and reports it as forged. Built on the pseudorandom code (PRC) watermark and Gaussian Shading++, Tripwire-WM provably preserves their undetectability. On MS-COCO, with the owner's own model as the surrogate, it accepts no imprinting forgery, compared with up to 55.0% for SemBind, and reports all of them as forged; it also accepts at most 1.8% of reprompting forgeries, against up to 46.2% for SemBind. Meanwhile, it accepts at least 99% of genuine watermarked images at 10^{-6} FPR and matches the robustness of the base schemes.

194. 【2611.25215】Bridging vision-language reasoning and learned control for legged navigation in the wild

链接:https://arxiv.org/abs/2611.25215

作者:Takahiro Miki,Robin Schmid,Manthan Patel,Turcan Tuna,Julia Richter,Julian Nubert,Joonho Lee,Matthias Müller,Marco Hutter

类目:Robotics (cs.RO); Artificial Intelligence (cs.AI); Computer Vision and Pattern Recognition (cs.CV)

关键词:Open-world Autonomous Mobility, Robust Open-world Autonomous, Autonomous Mobility, Open-world Autonomous, VLM

备注: 42 pages, 6 figures, 15 supplementary figures. Author's version of the article published in Science Robotics. Project page: [this https URL](https://roam-hiking.github.io)

点击查看摘要

Abstract:Autonomous navigation in diverse real-world environments remains one of the greatest challenges in robotics. It demands both robust low-level motion control to handle diverse terrain and high-level reasoning to interpret environmental cues. We present ROAM (Robust Open-world Autonomous Mobility), a system that enables legged robots to perform kilometer-scale navigation in complex alpine and urban settings. ROAM combines the world knowledge of a Vision-Language Model (VLM), pre-trained on internet-scale data, with robust locomotion and local planning policies learned in simulation via reinforcement learning (RL). The VLM provides common-sense reasoning and semantic interpretation of the scene, assessing risks, interpreting signs, and producing semantic labels with associated navigation costs. An open-vocabulary segmentation module converts these labels into a dense, real-time risk map, which conditions two pre-trained RL policies responsible for agile locomotion and local navigation, effectively coupling high-level VLM reasoning with low-level physical control. Real-world experiments showed the system's ability to autonomously navigate a wide range of complex real-world environments. The robot completes kilometer-long hikes with minimal intervention, interprets signs to avoid danger, and evaluates its environment to identify safe areas on which to step. This work represents a step toward the reliable deployment of systems in unstructured environments. However, it also underscores the challenges that remain in VLM reliability and spatial reasoning, indicating that foundational models are a crucial component but not yet a complete solution for navigation in the wild.

195. 【2611.25210】Recurrent Vision Language Models Solve the Shell Game

链接:https://arxiv.org/abs/2611.25210

作者:Dingqiang Ye,Jiacong Xu,Wen Zhang,Yiqun Mei,Vishal M. Patel

类目:Computer Vision and Pattern Recognition (cs.CV)

关键词:vision language models, effortless for humans, remains a bottleneck, bottleneck for vision, vision language

备注:

点击查看摘要

Abstract:Tracking a hidden object through a video is effortless for humans, yet it remains a bottleneck for vision language models (VLMs). In the shell game, where a ball is hidden under one of several identical cups that are then swapped, VLMs that answer directly stay near chance, thinking models fail on long games, and fine-tuning learns only the lengths it has seen. We argue that the limit is serial computation: each swap must be composed with the last result, and a VLM of fixed depth has only a fixed number of state transitions for it. To address this, we propose RecurrentVLM, which makes every attention layer of a pretrained VLM recur over time by computing the keys and values of finished frame groups from the layer's own output. With a single answer token, RecurrentVLM reaches 100% on VET-Bench and 99.6% on MET-Bench, and 99% on 50-swap games after training on at most 30 swaps. We also observe several interesting phenomena: length generalization emerges suddenly at an aha moment; without recurrence, each swap costs an attention layer; the ball's state forms in the first four attention layers; and recurrence keeps the latest state at the cost of details of earlier moments. These results suggest that recurrent VLMs are a potential solution for tracking the states of unobserved objects in video. Code will be released.

196. 【2611.25204】Identity World Model: Recognizing People in an Imagined Latent World

链接:https://arxiv.org/abs/2611.25204

作者:Dingqiang Ye,Kartik Narayan,Vishal M. Patel

类目:Computer Vision and Pattern Recognition (cs.CV)

关键词:Whole-body human recognition, Whole-body human, large vision models, advanced rapidly, rapidly with large

备注:

点击查看摘要

Abstract:Whole-body human recognition has advanced rapidly with large vision models. However, existing methods remain fragile under real-world variations, as they tie identity to transient cues such as clothing, texture, pose, or viewpoint. We argue that a robust recognizer should not merely learn to ignore these cues, but imagine how a person would be represented if they changed. To this end, we propose IDWorld, an identity world model: just as a latent world model predicts the features of a scene after an action, IDWorld predicts how a person would be represented under a changed condition. In a single forward pass, IDWorld imagines these counterfactual features under shared canonical conditions, such as the same outfit, a fixed view, or a bare silhouette, without rendering any pixels. Compared with explicit imagination in pixels, this latent imagination keeps nearly the same accuracy while being over 11x faster, at 5.9 ms per frame. Since each imagined condition has its own strengths, IDWorld then fuses them adaptively, weighting each condition for every probe-gallery pair. Extensive experiments show that IDWorld achieves state-of-the-art results on CCPG and CCGR-MINI, outperforms the compared methods on MEVID, and generalizes best to CASIA-B* and SUSTech1K, with the best mAP on CCVID. These results suggest identity world models offer a simple yet effective path to robust recognition. Code will be released.

197. 【2611.25179】Phantom Success, Silent Failure: Auditing the Success Checkers of Simulated Manipulation Benchmarks

链接:https://arxiv.org/abs/2611.25179

作者:Zirui Song,Jiadong Zhang,Zheng Lu,Xiaosong Ma,Xiaojun Chang,Xiuying Chen

类目:Computer Vision and Pattern Recognition (cs.CV); Robotics (cs.RO)

关键词:measure policy performance, manipulation benchmarks measure, benchmarks measure policy, measure policy, policy performance

备注:

点击查看摘要

Abstract:Simulated manipulation benchmarks measure policy performance using hand-written success checkers that label each episode as a success or failure. Passing an expert demonstration shows that a checker accepts one way of completing the task. Learned policies, however, may stop after partial progress or complete the task in a different way. A checker may then accept an incomplete execution (phantom success) or reject an execution that satisfies the instruction (silent failure). In this paper, we introduce SimAudit, an LLM-based pipeline that audits success checkers against task instructions. SimAudit analyzes the instruction and checker code to identify scenarios where the two may disagree, recreates those scenarios in simulation, and runs the original checker. Multi-view renderings are then used to verify whether the observed behavior satisfies the instruction. Across RoboTwin 2.0, RoboCasa365, and RoboDojo, 183 simulated test cases produced the predicted checker verdict. Visual verification and deduplication yielded 61 distinct defects across 49 tasks, including 40 phantom successes and 21 silent failures. Most defects in RoboCasa365 accepted incomplete executions, most in RoboDojo rejected valid executions, and both error types were common in RoboTwin 2.0. These findings suggest that validating success checkers requires testing both incomplete executions and valid alternatives to expert demonstrations, and motivate more rigorous evaluation protocols for future benchmarks.

198. 【2611.25172】MV-JEPA: Post-Training Video Encoders for Cross-View Understanding

链接:https://arxiv.org/abs/2611.25172

作者:Yangchen Zhao,Ali Payani,Caiwen Ding

类目:Computer Vision and Pattern Recognition (cs.CV)

关键词:learn strong representations, encoders learn strong, learn strong, JEPA-based video encoders, video encoders learn

备注:

点击查看摘要

Abstract:JEPA-based video encoders learn strong representations for video and image understanding through prediction in latent space. However, their standard pretraining predicts features within individual images or video clips, without explicitly exploiting paired camera views of the same object or moment. Consequently, the learned representations may not reliably support matching across different camera viewpoints. To solve this problem, we introduce multi-view predictive post-training to strengthen cross-view instance and temporal correspondence while retaining independent single-view inference. A shared encoder processes masked context views independently, and a training-time predictor jointly predicts latent tokens for masked context regions and a fully held-out view. We alternate cross-view prediction with single-view masked prediction on general image and video data to help retain the encoder's pretrained capabilities. After only 9,000 post-training updates on a single GPU, MV-JEPA improves cross-view instance retrieval and coarse cross-camera moment retrieval on various datasets, with further gains in zero-shot action-phase retrieval and frozen-feature classification on AE2. Frozen-probe evaluations further indicate that post-training largely preserves motion recognition, image recognition, and dense semantic segmentation performance on the evaluated benchmarks. Together, these results demonstrate that multi-view predictive post-training can strengthen instance and coarse temporal correspondence in JEPA video encoders without modifying the encoder architecture or single-view inference interface.

199. 【2611.25168】What a Calibrated Probability Hides: Belief-Level Post-Training for Decision Models

链接:https://arxiv.org/abs/2611.25168

作者:Xinye Li,Zirui Song,Huaxing Liu,Xiang Wang,Shuai Li,Zixiang Xu,Yinjun Liu,Zhewen Zhang,Xiaojun Chang,Xiuying Chen

类目:Computer Vision and Pattern Recognition (cs.CV)

关键词:answer is chosen, BGPO, Policy, choice, calibrated decision model

备注:

点击查看摘要

Abstract:A calibrated decision model can predict how often each answer is chosen. The same prediction can reflect disagreement among people who are confident in their choices or uncertainty among people who share a belief. Disagreement calls for identifying whose perspective applies, while uncertainty calls for gathering more evidence. Standard post-training scores a single prediction against one observed choice at a time, learning the pooled distribution without distinguishing these two sources of uncertainty. We introduce Belief-Grouped Policy Optimization (BGPO). The policy proposes a list of beliefs, estimates their prevalence, and predicts a choice distribution for each. During training, BGPO uses Bayes' rule to estimate how likely each belief was to generate an observed choice. These estimates weight the choice's contribution to each belief's log-score reward. To discover rare beliefs, the policy also proposes lists after seeing a few observed choices and retains them only if they predict held-out choices well. At inference, the policy uses only the question and combines the belief-specific distributions by prevalence to produce a calibrated aggregate prediction. On benchmarks covering opinions, conversation safety, emotions elicited by artwork, and photo aesthetics, BGPO achieves the lowest conditional, worst-group, and overall Jensen-Shannon divergence across model initializations. It improves conditional divergence on both text and image tasks. On a synthetic image task with known beliefs, BGPO recovers the underlying groups more accurately than baselines trained with belief labels.

200. 【2611.25166】CoRelate: Coordinated Human-Object Motion Synthesis through Interaction Relations

链接:https://arxiv.org/abs/2611.25166

作者:Zhiwen Qiu,Daisy Lin,Diyu Zhou,Jialin Wu,Rosita Ji,Saleh Kalantari

类目:Computer Vision and Pattern Recognition (cs.CV)

关键词:synthesis requires coordinated, motion synthesis requires, requires coordinated human, Human-object motion synthesis, synthesis requires

备注: 11 pages, 4 figures

点击查看摘要

Abstract:Human-object motion synthesis requires coordinated human participation as objects move toward specified goals. However, reaching object targets does not ensure that contact is established and maintained, while adjustments that improve contact may interfere with task completion. We introduce CoRelate, a diffusion framework that coordinates full-body human-object motion through explicit interaction relations and task feedback for rigid and articulated objects. We represent intended participation through hand-part relations that specify the participating hand, its part-relative reference position, and participation interval. These relations serve both as generation conditions and as references for measuring interaction realization. At each denoising step, a temporal refiner trained jointly with the generator combines relation deviations, task residuals, and motion features from the current prediction to jointly update human and object motion across the sequence. After generation, temporal motion correction addresses remaining contact failures by optimizing local motion segments while preserving object trajectories and protected human motion. Experiments demonstrate improved object-goal satisfaction, contact F1, and complete interaction success over the evaluated baselines. Ablation studies support the benefits of relation conditioning and the complementary roles of task and relation feedback.

201. 【2611.25134】GEMS: Generation and Experience Memory for Subgoal-Guided Long-Horizon World Model Planning

链接:https://arxiv.org/abs/2611.25134

作者:Yangchen Zhao,Ali Payani,Caiwen Ding

类目:Robotics (cs.RO); Computer Vision and Pattern Recognition (cs.CV)

关键词:enable effective short-horizon, effective short-horizon goal-conditioned, Visual world models, models enable effective, reaching nearby targets

备注:

点击查看摘要

Abstract:Visual world models enable effective short-horizon goal-conditioned planning, such as reaching nearby targets and pushing objects over short distances, by predicting candidate action outcomes and comparing their representations with a goal representation. However, extending this approach to long-horizon tasks is challenging: prediction errors accumulate over extended rollouts, while latent-space similarity to distant goals provides limited guidance for ranking candidate outcomes. Existing methods address these limitations by introducing intermediate subgoals, either generated by learned models or selected from experience memory. Generated subgoals can contain hallucinations or be inaccurate in aspects such as object positions and timing. Memory-based approaches are constrained by experience coverage and may lack precise guidance for selecting appropriate intermediate states from stored observations. We introduce GEMS, which generates temporally structured subgoals to guide action search and state selection from Latent Experience Memory (LEM). A distance gate determines whether predicted outcomes are evaluated against the generated subgoal or the selected memory state. Experiments across six tasks and task suites spanning manipulation, navigation, and reaching demonstrate the effectiveness of GEMS for long-horizon planning. GEMS outperforms state-of-the-art world-model planning baselines across different manipulation settings, achieving success rates of 92.8% on Cube, 68.4% on Push-T, and 54.1% on Meta-World. It improves over the strongest evaluated baseline on these tasks by 24.0, 7.3, and 22.2 percentage points, respectively. These results demonstrate that generated subgoals and experience memory can jointly strengthen long-horizon planning with a frozen world model.

202. 【2611.25126】ALD: Transferability-Aware Logit Disentanglement for Cross-Modal Knowledge Distillation

链接:https://arxiv.org/abs/2611.25126

作者:Hui Li,Pengfei Yang,Xinyi Feng,Junhong Liu,Tianyang Zheng,Hui Zeng,Le Dong,Kaibing Zhang,Mengting Zhao,Quan Wang

类目:Computer Vision and Pattern Recognition (cs.CV)

关键词:modality discriminative characteristics, source-modality supervision conflicts, target modality discriminative, incur negative transfer, discriminative characteristics

备注:

点击查看摘要

Abstract:Cross-modal knowledge distillation can incur negative transfer when source-modality supervision conflicts with the target modality's discriminative characteristics. We propose TALD, a two-stage framework for Transferability-Aware Logit Disentanglement in class-aligned logit space. Guided by task-dependent self-benefiting and cross-modal exclusivity criteria, a Residual Disentangler decomposes logits from a teacher and an independently trained target-modality reference student into modality-general and modality-specific components. A Sample-Adaptive Modulator regulates decomposition constraints and sample-specific fusion weights. The learned modality-general components are then adaptively fused to supervise the final student. Across five benchmarks spanning vision, audio, and text, TALD achieves the largest mean accuracy gain and highest positive-transfer frequency among the compared methods. Across ten transfer directions with five runs each, TALD improves over training without distillation in 84% of evaluations, compared with at most 70% for the baselines, while offering a favorable accuracy-efficiency trade-off. We plan to release the code.

203. 【2611.25113】Do Deep Stereo Models Learn Shape Priors?

链接:https://arxiv.org/abs/2611.25113

作者:Linbo Tang,Aneel Damaraju,Todd Zickler

类目:Computer Vision and Pattern Recognition (cs.CV)

关键词:embodied visual systems, Localizing object boundaries, Localizing object, visual systems, physical surfaces

备注:

点击查看摘要

Abstract:Localizing object boundaries is important for embodied visual systems, since small boundary errors can change the inferred geometry of physical surfaces. We study whether deep stereo models produce boundaries consistent with simple shape priors when local image and disparity cues are weak. We create a synthetic stereo dataset consisting of objects drawn from a highly constrained family of shapes, with randomized depth, slant, color and texture. Despite the simplicity of this shape space, we find that several state-of-the-art stereo models retain systematic boundary errors, even when finetuned or trained from scratch on the dataset. We then introduce a simple generative stereo model that jointly predicts disparity and a distance field representing object boundaries. The model improves boundary localization and produces contours that adhere more closely to the underlying shape space. These results expose limitations in the boundary predictions of current stereo architectures when local evidence is weak, and motivate further study of whether generative models can better combine regional grouping evidence with global geometric structure.

204. 【2611.25108】Long-Range Video Presentation Attack Detection: A Three-Class Benchmark Across 1-100 m and Two Sensor Types

链接:https://arxiv.org/abs/2611.25108

作者:Lambert Igene,Mohammad Zahir Uddin Chowdhury,Afzal Hossain,Sandip Purnapatra,Peter Johnson,Stephanie Schuckers

类目:Machine Learning (cs.LG); Computer Vision and Pattern Recognition (cs.CV)

关键词:Face presentation attack, presentation attack detection, Face presentation, subjects are captured, close range

备注: 10 page, IJCB

点击查看摘要

Abstract:Face presentation attack detection (PAD) systems deployed in border control and airport surveillance are constrained by a critical assumption: subjects are captured at close range, typically under one meter. No existing benchmark systematically addresses multi-distance capture, diverse environmental conditions, and concurrent spoofing and occlusion-based attacks, leaving deployed models underprepared for real-world scenarios. We introduce a video-based PAD dataset of 374,000+ frames from 261 unique identities collected across five standoff distances (1 m-100 m) under indoor and outdoor conditions using a DSLR camera (C) and a mobile sensor (M), with three-class labels: live, spoof, and props (occlusions); models are trained and evaluated on video clips rather than single frames. We fine-tune and benchmark four CNN architectures: MobileNetV3, MobileNet, EfficientNet, and ResNet-50. The higher-resolution DSLR maintains near-perfect spoof classification rate across all distances, while the lower-resolution mobile sensor degrades sharply beyond 50 m, with props classification rate dropping to 55% at 75 m for MobileNetV3. EfficientNet achieves the best overall performance (98.48% accuracy, ACER 2.01%). Generalization tests on unseen attack datasets yield 89-94% accuracy, validating cross-domain robustness. These results establish capture distance and sensor resolution as critical but underexplored axes of PAD difficulty. The dataset will be publicly released under an End-User License Agreement to support reproducible long-range PAD research.

205. 【2611.25105】MAWAM: A Multi-Agent World-Action Model for Coordinated Manipulation

链接:https://arxiv.org/abs/2611.25105

作者:Xianzhe Fan,Chi-Pin Huang,Fu-En Yang,Jing Wang,Yecheng Wu,Yu-Cheng Chou,Johan Bjorck,Linxi "Jim" Fan,Jan Kautz,Yu-Chiang Frank Wang,Zhiding Yu,Hengshuang Zhao,Zhiqi Li

类目:Robotics (cs.RO); Artificial Intelligence (cs.AI); Computer Vision and Pattern Recognition (cs.CV)

关键词:teammates must act, MAWAM, Abstract, robot, act

备注:

点击查看摘要

Abstract:Every move in a robot team changes the world in which its teammates must act. Coordinated manipulation therefore calls for predicting the team's actions together with the physical future they jointly create. We introduce MAWAM, a multi-agent world-action model that jointly generates each robot's future views and executable action chunks in a single sample. Built from a pretrained single-robot world-action backbone, MAWAM jointly denoises all robots' action and future-view tokens through cross-agent attention, while preserving a separate control interface for each robot. MAWAM achieves the highest success on all three real-robot tasks (laundry-bag collection, bath-towel folding, and umbrella handover), with 67.8% mean success, 18.9 percentage points above MA-VLA trained on the same demonstrations and cameras. In simulation, MAWAM reaches 90.3% mean success on eight tasks with teams of up to four arms. Ablations show that cross-agent attention and team-stage future-video supervision both improve coordinated control: without attention, a four-arm relay falls from 95.0% to 39.0%, and without this supervision, Place Food falls from 78.0% to 47.0%.

206. 【2611.25069】VisAlloc: Allocating Visual Supervision for On-Policy Self-Distillation

链接:https://arxiv.org/abs/2611.25069

作者:Haoran Xu,Tian Chen,Yuhe Wu,Yanbo Zhao,Junchi Liao,Hongyu Wang,Jiaze Li,Wentao Wang,Yanfeng Li,Rui Qian

类目:Computer Vision and Pattern Recognition (cs.CV); Artificial Intelligence (cs.AI)

关键词:multimodal large language, On-policy self-distillation, improving visual reasoning, large language models, offers a promising

备注:

点击查看摘要

Abstract:On-policy self-distillation (OPSD) offers a promising approach to improving visual reasoning in multimodal large language models (MLLMs). However, not all samples benefit equally from privileged visual guidance, and uniform token supervision can dilute visually grounded signals. We propose VisAlloc, a dual-level framework combining sample-level visual filtering with token-level counterfactual reweighting. A fixed paired-teacher filter retains samples answered correctly from an evidence-centered positive crop but incorrectly from a spatially displaced negative crop. At the token level, both EMA teachers score the same student-generated response, and their absolute log-probability differences measure view sensitivity. Additional supervision targets tokens admitted by sensitivity or teacher-distribution disagreement, with shared frozen-prior, template-bigram, and student--teacher divergence gates. Inverse-frequency redistribution preserves the total additional weight within each micro-batch, while the positive teacher remains the sole distillation target. Inference requires only the full-image student. Across eight benchmarks, our 4B and 9B models achieve macro-averages of 81.44 and 84.92, outperforming Vision-OPD reproductions by 0.97 and 0.52 points, respectively. Code on GitHub: this https URL

207. 【2611.25061】Infer3D: Inverse Generative Modeling Enables Robust 3D Reconstruction

链接:https://arxiv.org/abs/2611.25061

作者:Amani Kiruga,Mark Hamilton,Joshua B. Tenenbaum,William T. Freeman,Vincent Sitzmann,Yilun Du,Ayush Tewari

类目:Computer Vision and Pattern Recognition (cs.CV); Machine Learning (cs.LG); Robotics (cs.RO)

关键词:extreme viewpoints, highly unfamiliar, unfamiliar or extreme, generative, Abstract

备注: 23 pages, 12 figures, 15 tables. Includes references and appendices. Project page: [this https URL](https://amanikiruga.github.io/infer3d-site/)

点击查看摘要

Abstract:Humans can understand the 3D structure of scenes even from highly unfamiliar or extreme viewpoints. Cognitive science suggests this generalization stems from a top-down generative inference strategy. In contrast, state-of-the-art 3D reconstruction neural networks typically rely on amortized, feedforward inference. While fast, this pattern-matching approach degrades when faced with out-of-distribution (OOD) scene conditions unseen during training. Inspired by the generative nature of human vision, we propose Infer3D, an inverse generative modeling framework that recovers the latent variables of a generative model to best match the monocular input. To prevent degenerate flat outputs, we couple the generative image prior with a frozen 3D lifting network that constrains the optimization space to a valid, multi-view-consistent 3D manifold. Across a variety of controlled synthetic and real-world tasks, Infer3D demonstrates superior OOD reconstruction accuracy compared to feedforward networks trained on equal or larger datasets. Furthermore, mirroring how human perception requires more time to process confusing stimuli, we introduce an adaptive inference strategy that expends more test-time compute on ambiguous OOD examples while efficiently routing in-distribution inputs. Project website: this https URL

208. 【2611.25005】Inhabitant: An Agent Framework for Long-Horizon Human-Scene Interaction with Spatial Control and Quantitative Feedback

链接:https://arxiv.org/abs/2611.25005

作者:Yao Zhang,Zhuchenyang Liu,Yu Xiao

类目:Robotics (cs.RO); Computer Vision and Pattern Recognition (cs.CV)

关键词:Recent work, language-model agent plans, generation moves, work on human-scene, moves from single

备注: [this https URL](https://ryenhails.github.io/Inhabitant/)

点击查看摘要

Abstract:Recent work on human-scene interaction (HSI) generation moves from single interactions to long-horizon interaction, where a language-model agent plans and executes a sequence of interactions in a 3D scene under one instruction. However, existing agents struggle in challenging everyday scenarios (e.g., picking up a toy and throwing it into a basket) because they lack spatial control over the motion and quantitative feedback on its execution, and they often fail to complete the interaction sequence. To address these gaps, we present Inhabitant, a training-free agent framework in which the agent specifies each motion through positions, paths and poses, measures how the executed interaction deviates from its intent, and revises the control specification accordingly. For an extensive evaluation in such scenarios, we build Inhabit, a benchmark of 75 household tasks in 31 furniture layouts of an apartment, in which every task chains one to three goals on movable objects and is carried out by the whole body, excluding fingers. On Inhabit, Inhabitant completes 98.7\% of the tasks with Claude Opus 5.5, and 93.3\% and 82.7\% with Claude Sonnet 5.5 and Gemini 4 Argon, respectively. Ablations confirm that without spatial control the agent fails most tasks, and without quantitative feedback its motion becomes less precise. Code is available on the project page.

209. 【2611.25001】Density Feedback Control for Transferability of Generative Sparse Adversarial Attacks

链接:https://arxiv.org/abs/2611.25001

作者:Dongsu Song,Jay Hoon Jung

类目:Machine Learning (cs.LG); Computer Vision and Pattern Recognition (cs.CV)

关键词:Generator-based sparse adversarial, produce transferable sparse, black-box robustness evaluation, Generator-based sparse, transferable sparse perturbations

备注: preprint

点击查看摘要

Abstract:Generator-based sparse adversarial attacks train encoder--decoder networks to produce transferable sparse perturbations for black-box robustness evaluation. Their perturbation density, however, is set only indirectly by a fixed sparsity coefficient whose mapping to density is unknown and shifts during training, so reaching a target density requires re-tuning; this fixed pressure also yields little transferability gain once density reduction begins. We propose Density Feedback Control (DFC), which tightens or relaxes this coefficient at the end of each epoch according to the gap between the observed hard-mask density and the target, without changing the generator architecture or loss. At a 10\% target density, DFC brings the mean density of three base attacks within one percentage point of the target in a single run and improves the transfer attack success rate (ASR) of two of them on CNN and Transformer targets, including under per-image $\ell_0$ budgets, while perturbing fewer pixels. Tracing GPG's training, we observe that as DFC relaxes the coefficient, the transfer ASR recovers with the density and overtakes the fixed-coefficient baseline with fewer pixels. For EGS-TSSA, which restricts perturbations to source-model feature-importance regions, DFC yields no consistent gain; we analyze how this restriction interacts with density feedback.

210. 【2611.24995】EgoPass-Bench: Identifying Objects That May Affect the Wearer's Passage in Egocentric Video

链接:https://arxiv.org/abs/2611.24995

作者:Hongyu Wang,Chuhan Li,Congjin Xu,Xuandong Zhao,Hua Shen

类目:Computer Vision and Pattern Recognition (cs.CV)

关键词:environment requires reasoning, egocentric environment requires, surrounding entities affect, environment requires, requires reasoning

备注:

点击查看摘要

Abstract:Understanding an egocentric environment requires reasoning about how surrounding entities affect the observer's actions. Existing evaluations of multimodal foundation models (MFMs) examine observer localization, motion trajectories, and action feasibility, but leave the discovery and interpretation of evolving observer-entity interactions insufficiently assessed. We introduce EgoPass-Bench, a benchmark for observer-relative interaction understanding in real-world egocentric video. EgoPass-Bench comprises 284 manually reviewed evaluation clips recorded using Ray-Ban Meta (Gen 2) smart glasses and 2,201 distinct questions across five tasks assessing motion perception, interaction judgments about specified entities, and relevant-entity discovery without target cues. Our evaluation of eight frame-based and six native-video configurations reveals a pronounced discrepancy between motion perception and interaction understanding. The strongest evaluated model, GPT-6 Astra, correctly recognizes object motion in 91.37% of 394 comparisons involving the same targets, but correctly identifies changes in passage constraints in only 66.50%. Across all 574 entity-discovery questions, it trails the two-person human baseline by $23.65 \pm 7.45$ percentage points (paired 95% CI), while exceeding the always-empty baseline (36.76%) by at least 18.9 percentage points. Further analysis reveals frequent inclusion of entities that neither pose a plausible physical conflict nor constrain passage, showing that accurate motion recognition does not ensure reliable relevant-entity discovery.

211. 【2611.24960】ETHER: Training-Free, Elastic, ToC-Anchored Hierarchy Extraction and Recovery in Design-Driven Long Documents

链接:https://arxiv.org/abs/2611.24960

作者:Yu Zou,Yan Chen,Jialei Zeng,Xiaorui Zhou,Aixi Zhong,Qi Zhou,Haoran You,Ke Jiang,Changyu Hu

类目:Computation and Language (cs.CL); Computer Vision and Pattern Recognition (cs.CV)

关键词:deploying Large Language, Large Language Models, Converting complex documents, structured knowledge constitutes, critical infrastructure urgently

备注: EMNLP 2026 Main

点击查看摘要

Abstract:Converting complex documents into structured knowledge constitutes the critical infrastructure urgently required for deploying Large Language Models (LLMs) in downstream research. However, blind to the Table of Contents (ToC) as a macro-structure, existing bottom-up paradigms rely on localized visual cues, inducing heuristic fragility in high-entropy layouts where visual prominence decouples from logical depth; meanwhile, direct LLM extraction lacks spatial grounding, triggering hallucinations in long contexts. To address this, we introduce TETHER, a training-free framework that elevates the ToC from discarded noise to a vital structural prior, enabling robust adaptation across design-driven long documents. TETHER operates via a two-stage neuro-symbolic mechanism: first, it constructs a document skeleton from the raw ToC using a Large Multimodal Model (LMM) empowered by spatio-semantic constraints; second, it employs an anchor-based alignment strategy to tether content blocks to this skeleton, effectively resolving boundary ambiguities. Evaluated across a multilingual benchmark of ESG reports, product manuals, and annual reports, TETHER delivers highly precise, state-of-the-art (SOTA) zero-shot hierarchy recovery, outperforming deep hierarchical recovery models, dedicated parsers, and general-purpose LMMs. Moreover, stress tests highlight TETHER's resilience to sparse omissions and textual noise, demonstrating that its precise semantic alignment is fundamentally anchored in topological and depth fidelity. Crucially, ablations confirm the framework's architectural irreducibility, while achieving 90% token reduction and drastic speedups over end-to-end baselines, establishing a compelling equilibrium between structural precision and deployment scalability.

212. 【2611.25510】Life-IQA+: Boosting Blind Image Quality Assessment via GCN-enhanced Layer Interaction and Flexible Hierarchical Expert Specialization

链接:https://arxiv.org/abs/2611.25510

作者:Long Tang,Guoquan Zheng,Huiyu Duan,Jie Hao,Yucheng Zhu,Liang Yuan,Xiongkuo Min,Guangtao Zhai

类目:Image and Video Processing (eess.IV); Computer Vision and Pattern Recognition (cs.CV)

关键词:Blind image quality, optimizing visual experience, Blind image, image quality assessment, Existing BIQA methods

备注:

点击查看摘要

Abstract:Blind image quality assessment (BIQA) is essential for evaluating and optimizing visual experience. Existing BIQA methods typically aggregate features from multiple backbone stages without accounting for their unequal contributions to quality prediction. Furthermore, uniformly supervising all experts may constrain their flexibility and limit the diversity of the learned quality representations. To address these issues, we present Life-IQA+, a quality representation decoding framework that combines graph convolutional network (GCN) enhanced layer interaction and flexible hierarchical expert specialization. Specifically, the layer interaction module constructs quality queries from GCN-refined deepest-stage features and uses penultimate-stage features as keys and values, thereby enabling cross-layer representation interaction via the cross-attention mechanism. Building on these representations, we further introduce a flexible hierarchical mixture-of-experts (MoE) architecture comprising a densely activated shared expert layer and a sparsely activated routed expert layer. To enhance the representation diversity and ability of experts, we apply auxiliary classification supervision to a subset of shared experts using distortion-type labels for synthetic datasets and discretized quality-level labels for authentic datasets. Experiments on synthetic and authentic distortion benchmarks, together with controlled ablations of expert configurations and supervision strategies, show that Life-IQA+ achieves competitive prediction accuracy with a favorable balance between accuracy and computational cost. The code is available at: this https URL.

213. 【2611.25335】Breaking Noise Shortcuts in Self-Supervised Learning Via Noise-Aligned View Generation

链接:https://arxiv.org/abs/2611.25335

作者:Oankar R. Patil,May D. Wang

类目:Image and Video Processing (eess.IV); Artificial Intelligence (cs.AI); Computer Vision and Pattern Recognition (cs.CV)

关键词:Self-supervised learning, SEANA, increasingly applied, applied to scientific, labels are scarce

备注: 40th Conference on Neural Information Processing Systems (NeurIPS 2026)

点击查看摘要

Abstract:Self-supervised learning (SSL) is increasingly applied to scientific imaging domains, such as microscopy, where labels are scarce but noisy data is abundant. These domains exhibit significant signal-dependent noise, which can create a representational shortcut: since both augmented views derive from the same noisy image, the encoder can boost view agreement by also encoding noise patterns rather than semantic content. To enable noise-robust representation learning without knowledge of underlying noise models, we introduce Self-Aligned Noise Augmentation (SEANA), a drop-in module that generates noise-aware SSL views. SEANA learns an invertible variance-stabilizing transform (VST) from noisy data, then estimates the clean signal and resamples independent noise in learned VST space, requiring no clean targets, no dataset-specific denoiser training or use at inference, and no changes to the SSL objective or encoder. On CIFAR-10 and ImageNet-100 under synthetic Gaussian, Poisson-Gaussian, and multiplicative noise, SEANA improves clean-test linear-probe accuracy across contrastive/non-contrastive SSL methods, indicating higher-quality representations. On real fluorescence imaging, SEANA improves Jurkat cell-cycle stage classification by up to 28.6pp over standard SSL and 27.5pp over denoiser-preprocessed baselines. In all settings, SEANA outperforms the denoiser-preprocessed SSL pipelines, demonstrating that learned VST-space noise-aligned resampling yields more robust representations than denoising alone. Code is available at: this https URL