本篇博文主要展示每日从Arxiv论文网站获取的最新论文列表,以自然语言处理、信息检索、计算机视觉等类目进行划分。
统计
今日共更新577篇论文,其中:
- 自然语言处理63篇
- 信息检索7篇
- 计算机视觉91篇
自然语言处理
1. 【2611.18533】ModHarness-RL: Recasting Harnesses as Learnable and Evolving Programs for Agentic RL
链接:https://arxiv.org/abs/2611.18533
作者:Zelin He,Siyu Zhu,Jacob Helwig,Yujian Liu,Li An,Rohit Jain,Shiyu Chang,Alborz Geramifard
类目:Artificial Intelligence (cs.AI); Computation and Language (cs.CL); Machine Learning (cs.LG)
关键词:Agentic reinforcement learning, reinforcement learning, increasingly relies, model, harness
备注:
点击查看摘要
Abstract:Agentic reinforcement learning (RL) increasingly relies on a harness, the scaffold around the model that decides what it sees of its own interaction and what guidance it receives. Most existing approaches train under a hand-written harness that stays fixed throughout. However, an effective harness depends on the domain, the data and the model, and keeps changing as the model trains. We introduce ModHarness-RL, which recasts the harness as programs that are learned from data and evolve as the model trains with RL. ModHarness-RL records the interaction as structured State, a set of typed blocks, and two core components of the harness are programs over these blocks, a Compression program that assembles the accumulated history into the model's context and a Steering program that supplies guidance when behavior deviates. During RL training, candidate programs are proposed from recent trajectories, verified by execution, and replayed over past trajectories, so the harness tracks how the model's behavior shifts. We study CodeSearch in depth and further analyze ALFWorld and BIRD SQL, with models from 1.5B to 8B. The proposed Compression and Steering mechanisms improve held-out performance over the fixed-harness baseline in most settings. These results suggest that a harness for agentic RL can be learned from the task's data and evolved with the model.
2. 【2611.18527】WaferTetris: Efficient Freeform Mapping on Wafer-Scale Accelerators
链接:https://arxiv.org/abs/2611.18527
作者:Dayou Du,Congjie He,Andrew Boutros,Luo Mai,Vaughn Betz,Jianyi Cheng
类目:Hardware Architecture (cs.AR); Computation and Language (cs.CL)
关键词:Wafer-scale accelerators offer, accelerators offer massive, potential requires efficient, Wafer-scale accelerators, requires efficient application
备注:
点击查看摘要
Abstract:Wafer-scale accelerators offer massive compute capacity through hundreds of thousands of processing elements (PEs) interconnected across a single wafer, but realizing this potential requires efficient application mapping. Their scale makes programming these systems and mapping applications onto them challenging. Existing approaches rely on template-based mappings to maintain productivity, but these templates restrict the mapping design space. In particular, they impose fixed geometric layouts over the PE grid, often leaving compute and memory resources underutilized. In this work, we present WaferTetris, a system that removes these geometric restrictions through freeform mapping. Its Tetris-like strategy flexibly packs computation across the wafer, improving PE utilization and usable memory capacity while preserving PE-task code. We develop a recurrence-based throughput model that captures branch-and-join dependencies to guide spatial parallelism, and an architecture model with per-color gates that enable existing FPGA place-and-route algorithms to enforce WSE routing constraints. We evaluate WaferTetris on Cerebras WSE-3 using LLM decoding and scientific stencil benchmarks. For LLM decoding, it achieves up to $4.42\times$ the system throughput of a template-based automatic mapping baseline on the same fabric size. Compared with the manually optimized WaferLLM, WaferTetris supports $1.8\times$ the context length for LLaMA3.1-8B (80k versus 44k), while delivering equal or higher per-user throughput at matched context lengths, up to $1.10\times$.
Subjects:
Hardware Architecture (cs.AR); Computation and Language (cs.CL)
Cite as:
arXiv:2611.18527 [cs.AR]
(or
arXiv:2611.18527v1 [cs.AR] for this version)
https://doi.org/10.48550/arXiv.2611.18527
Focus to learn more
arXiv-issued DOI via DataCite (pending registration)</p>
3. 【2611.18514】Can Mechanistic Interpretability Help Explain LLM Agent Failures? A Case Study on Web Element Hallucination
链接:https://arxiv.org/abs/2611.18514
作者:Mohamed Aghzal,Mor Geva,Gregory J. Stein,Ziyu Yao
类目:Artificial Intelligence (cs.AI); Computation and Language (cs.CL)
关键词:Large Language Models, Language Models, Large Language, made significant progress, demonstrate comparable success
备注:
点击查看摘要
Abstract:Mechanistic Interpretability (MI) has made significant progress in explaining the internal computations of Large Language Models (LLMs); however, these advances have yet to demonstrate comparable success in explaining the decision-making processes of LLM-based agents. This paper examines the challenges MI tools face when applied to realistic agentic failures. We focus on one such failure: web agents hallucinating actions that are not available on a website. We apply a series of mechanistic analyzes to Qwen2.5-7B-Instruct as it interacts with live websites. We find that a linear probe trained on the last prompt token can predict hallucinations before generation begins (0.80 AUC) and steering along a probe direction corrects 85\% of hallucinations, but disrupts 7\% of valid actions. However, these results only provide a partial explanation of why the agent hallucinates. The probe direction is sensitive to prompt formatting, losing much of its steering effect after a minor change to how actions are represented. Additionally, circuit identification can be difficult as the computation is distributed across multiple components and positions, and the information the agent relies on varies across examples. We discuss how these observations challenge the assumptions underlying standard MI methods for circuit discovery and feature identification, and outline future directions for making these methods better suited to explain failures in LLM-based agents.
4. 【2611.18513】MeReader: An Offline, Privacy-Preserving, Progress-Aware AI Assistant for Narrative eBook Reading
链接:https://arxiv.org/abs/2611.18513
作者:Mohammed Efaz,Udo Bub,Itilekha Podder
类目:Information Retrieval (cs.IR); Artificial Intelligence (cs.AI); Computation and Language (cs.CL)
关键词:paper introduces MeReader, Large Language Model, conventional Large Language, consumer hardware, paper introduces
备注: 15 pages, 4 figures. Preprint (submitted version) of a paper published in ICCCI 2026, CCIS vol. 3044, Springer
点击查看摘要
Abstract:This paper introduces MeReader, a privacy-preserving, offline and progress-aware AI assistant designed to function as a contextual companion for narrative reading on consumer hardware. Unlike conventional Large Language Model (LLM)-based systems that process documents/books online in an all-texts-at-once type of way, thereby risking narrative spoilers, MeReader confines both retrieval and generation to the reader's distinct position in the text. The system integrates a hybrid retrieval architecture, combining vector similarity, BM25, and summary embeddings, to ensure high fidelity to the reader's current context. We evaluate the system across classical English books using both quantitative comprehension metrics and qualitative LLM-based independent judging. The results identify a quality-latency frontier, showing that mid-sized local models (2b-4b) can achieve comprehension parity with larger counterparts (5b+) while maintaining acceptable response quality. Importantly, the proposed progress-aware boundary mechanism is shown to prevent future content leakage without compromising retrieval quality and correctness. These findings suggest that a localized, privacy-centric approach can augment, rather than replace, the digital reading experience, offering a good alternative to cloud-dependent services. Therein, there remains a lack of amalgamate systems which combine progress-aware, privacy-preserving, and narrative eBook reading with AI support. It is in this very gap that the development of MeReader came to be; synthesizing said strands into an offline, book-constrained assistant focused on enhancing comprehension without compromising narrative continuity or user privacy.
5. 【2611.18506】Agentic induction of common data elements and information extraction from sarcoma case report abstracts
链接:https://arxiv.org/abs/2611.18506
作者:Daniel Reichenpfader(1 and 2),Kwaku Appiagyei(3),Manovah Nadarajah(4),Ahmed Reda Bahr(5),Irina Filchenko(6),Fabio Dennstädt(7, 8 and 9) ((1) Institute for Patient-Centered Digital Health, Bern University of Applied Sciences, Bern, Switzerland, (2) Faculty of Medicine, University of Geneva, Geneva, Switzerland, (3) F. Hoffmann-La Roche AG, Basel, Switzerland, (4) Kantonsspital Aarau, Aarau, Switzerland, (5) University of Alexandria, Alexandria, Egypt, (6) Department of Neurology, Inselspital, Bern University Hospital and University of Bern, Bern, Switzerland, (7) Data Semantics and Decision Making Institute (DSDMI), St. Gallen, Switzerland, (8) Department of Radiation Oncology, Inselspital, Bern University Hospital and University of Bern, Bern, Switzerland, (9) Department of Digital Medicine, Inselspital, Bern University Hospital and University of Bern, Bern, Switzerland)
类目:Computation and Language (cs.CL)
关键词:Common data elements, clinical data comparability, Common data, support clinical data, data elements
备注:
点击查看摘要
Abstract:Common data elements (CDEs) support clinical data comparability but are usually defined manually. We investigated whether large language model agents could derive a CDE schema from sarcoma case report abstracts and apply it at scale. An agent proposed 72,909 candidate CDEs from 24,588 PubMed abstracts; agentic consolidation yielded 1,570 CDEs. We retained 53 supported by at least 50 abstracts and attempted evidence-linked extraction across the corpus. Processing succeeded for all retained CDEs in 79.2% of abstracts, including responses indicating non-mention. Two human reviewers approved 83.0% of retained CDEs for relevance, 67.9% for clarity, and 56.6% for correctness. In 100 sampled abstracts, exact agreement with the value extracted by the LLM was 94.9% among 1,116 pairs with a non-empty answer (abstract-level bootstrap 95% CI, 93.0-96.4%). Errors concentrated in anatomical and temporal fields. These findings support agent-assisted schema drafting followed by expert curation before large-scale extraction.
6. 【2611.18485】Interactive Program Induction with Language Model Priors
链接:https://arxiv.org/abs/2611.18485
作者:Robin Shing Moon Chan,Ryan Cotterell
类目:Computation and Language (cs.CL); Artificial Intelligence (cs.AI); Machine Learning (cs.LG)
关键词:Program induction refers, aims to recover, Program, input-output, Program induction
备注: Preprint. 28 pages, 5 figures, 9 tables
点击查看摘要
Abstract:Program induction refers to the process in which a learner aims to recover a program from a few given input-output examples. This paradigm broadens programming access by letting users demonstrate the rule-based behavior they want to apply to unseen data, instead of having to implement it themselves. However, recovering a program from finite input-output examples is fundamentally ambiguous, as a finite set of input-output examples is theoretically consistent with infinitely many programs whose output diverges on unseen inputs. We view this problem from a Bayesian perspective, where, given a prior $p^0$ over programs, identifying the target program means concentrating posterior probability mass on it by observing additional input-output examples. In the interactive program induction setting, the learner chooses maximally informative inputs and queries an oracle, such as the user, for their outputs. We derive a greedy query-selection algorithm and show that it terminates after $\mathcal{O}(\frac{1}{\gamma}\log 1/p^0(f^\star))$ informative oracle queries, and returns the target program $f^\star$ up to approximate equivalence when no other class outweighs it under the prior. The algorithm uses a language model as the prior under a PAC-style relaxation of program equivalence, where the query-selection objective is estimated using importance sampling. On the CodeARC benchmark, the algorithm improves accuracy over the LM only agent by up to 22 percentage points while observing up to 10.5 fewer input-output examples.
7. 【2611.18471】Do Verifier Judgments Survive Abstraction? Measuring Verification Transport for LLM Agent Evaluation
链接:https://arxiv.org/abs/2611.18471
作者:Disha Sheshanarayana,Sai Linisha Reddy Goli
类目:Computation and Language (cs.CL); Artificial Intelligence (cs.AI)
关键词:typically measured, single representation, evaluate agent executions, abstraction boundaries, boundaries
备注: Accepted at NeurIPS 2026 Workshop on Who Verifies the Agents? Toward Reliable Agent Development
点击查看摘要
Abstract:LLM-based verifiers are increasingly used to evaluate agent executions, but their reliability is typically measured under a single representation of the trajectory. In deployed agent systems, the same execution may be exposed through composed tools, delegation boundaries, or compacted histories, changing what execution evidence is directly observable to the verifier. It therefore remains unclear whether correct verifier judgments persist across such abstraction boundaries. We introduce Verification Transport, a reliability test that measures judgment stability under abstraction-induced changes in execution observability. We construct matched native, abstracted, and restored views of the same frozen executions while holding the task, realized trajectory, outcome, ground truth, evaluation criterion, and verifier fixed. Across four agent-evaluation benchmarks, seven verifier backbones, four verification protocols, and three abstraction boundaries, we find substantial transport failure, reaching about 31% under macro abstraction and delegation. Verification protocols exhibit distinct failure modes: direct verification more often produces incorrect verdicts, whereas evidence-grounded and deliberative protocols tend to abstain under limited observability. This suggests that lower transport failure rates do not necessarily imply greater reliability. Restoring the corresponding execution provenance recovers 85.5% of transport failures, while recovery is lower when provenance is lossy or unavailable. These results show that native-view accuracy alone is insufficient to characterize verifier reliability when execution evidence crosses abstraction boundaries.
8. 【2611.18412】LitArc-Bench: Disentangling Text-Specific Reader-Response Dynamics from Position and Lexical Affect
链接:https://arxiv.org/abs/2611.18412
作者:Mingming Su,Yushan Wei,Sen Song,Xinche Zhang
类目:Computation and Language (cs.CL)
关键词:text-specific reader-response dynamics, affective words, reflect narrative position, model predictions track, reader-response dynamics
备注: 13 pages, 2 figures, 12 tables
点击查看摘要
Abstract:When model predictions track an unfolding human affect trajectory, alignment may reflect narrative position, affective words, or text-specific reader-response dynamics. We introduce LitArc-Bench to separate these explanations: 39 Chinese-language literary excerpts segmented into seven checkpoints, with 430 questionnaires from 81 adults in two classroom groups (9,020 VAD ratings). We compare three future-blind model runs with leave-one-text-out position priors, lexical affect, and human-human references. Position alone reaches r = .470 for arousal, making raw alignment insufficient. Yet all six V/A transition advantages over a joint position-lexicon control have text-bootstrap intervals above zero. Cross-fitted models match the correct human text better than 38 mismatched texts in all nine model-dimension cells (p .0001); a text-invariant negative control returns chance (p = 1). On human transitions at least 0.5 points against the position trend, models recover 69-76% of valence directions and 53-68% for arousal and dominance. Under crossed participant-text resampling, all nine identity intervals remain above zero; joint-control advantages are robust for valence and directional but less certain for arousal. High shape agreement still coexists with scale bias. LitArc-Bench identifies which components of literary reader response are recoverable beyond shallow explanations and why shape, calibration, and human variability require separate evaluation.
9. 【2611.18397】Gamer: A Generalist Vision-Language-Action Foundation Model for Long-Horizon Real-Time Game Agent
链接:https://arxiv.org/abs/2611.18397
作者:Run Luo,Kuan Zhang,Qi Deng,Yukun Chen,Zhihao Yang,Yufei Zhang,Renrong Hu,Yi Zhang,Guiyu Ma,Tianyu Fan,Longze Chen,Hao Sun,Jiaming Li,Jiaming Yang,Xiaohao Liu,Tao Tan,Minzheng Wang,Hangtian Jia,Shuai Yuan,Lvdi Wang,Xiao Liu,Chao Ma,Yuting Zou,Yinhe Zheng,Xiaobo Xia,Dawei Jiang,Shaoqing Ren
类目:Computation and Language (cs.CL); Artificial Intelligence (cs.AI); Computer Vision and Pattern Recognition (cs.CV)
关键词:agent framework built, framework built, complex interactive environments, introduce Gamer, generalist agent framework
备注:
点击查看摘要
Abstract:We introduce Gamer, a generalist agent framework built for long-horizon tasks in complex interactive environments. Its end-to-end architecture combines perception, memory management, and multi-step reasoning. A shared training and deployment pipeline supports interaction with both conventional 2D graphical interfaces and complex 3D simulated worlds. We first continue pre-training on a dataset constructed from tens of thousands of hours of device-level keyboard and mouse interaction. These trajectories span simulated games and embodied environments, allowing the model to learn environmental dynamics across both physical and virtual domains. Sparse-thinking instruction tuning then substantially reduces redundant computation while preserving the reasoning needed for complex decisions. We train the policy for long-horizon interaction with a stable multi-turn reinforcement learning method that addresses finite context windows, accumulated rollout errors, and sparse rewards. For real-time interaction, a cloud-edge inference engine serves numerous simulated game environments concurrently, supports unbounded-context execution, and keeps online inference latency low. Across multiple challenging 3D open-world games, Gamer demonstrates solid generalization and operational efficiency comparable to skilled players. It also achieves competitive results on a range of conventional 2D game benchmarks. These results suggest that even repetitive keyboard and mouse trajectories can support substantial gains from scaling data and learning algorithms. Training on these trajectories helps Gamer learn complex skills more steadily in environments with both physical and digital properties. This approach offers a path toward generalist agents that can interact across digital and physical worlds.
10. 【2611.18371】Evaluating Human Sensitivity to Gender Biased AI-Generated Content through Human Assessment of Google's Autocomplete Function
链接:https://arxiv.org/abs/2611.18371
作者:Tamara Serrano,Nikoleta Pantelidou,Raquel Montero,Natalia Moskvina,Paolo Morosi,Evelina Leivada
类目:Computation and Language (cs.CL)
关键词:Large Language Models, Large Language, Language Models, users remain underexplored, human users remain
备注: 11 pages, 2 figures, 1 table
点击查看摘要
Abstract:Large Language Models (LLMs) are increasingly present, but their effects on human users remain underexplored. We tested whether AI-attributed information influences views regarding gender-related statements, a domain where biases are pervasive. Spanish-speaking participants recruited through Prolific evaluated biased statements derived from Google's autocomplete function and human-authored, non-biased gender-related facts. Participants were either informed that some of the material was AI-generated or received no information about its origin. AI attribution did not affect reaction times or agreement judgments. This result suggests that AI-generated content does not automatically raise credibility concerns and may be evaluated with less scrutiny compared to human-produced content.
11. 【2611.18370】f-Divergence Self-Play for Tabular Anomaly Detection via Large Language Models
链接:https://arxiv.org/abs/2611.18370
作者:Hoang Tran Vuong,Linh Ngo Van,Dang Nguyen,Thin Nguyen,Phuoc Nguyen,Mehrtash Harandi,Trung Le
类目:Machine Learning (cs.LG); Computation and Language (cs.CL)
关键词:feature types-mixing numerical, poses significant challenges, significant challenges due, heterogeneous feature types-mixing, data poses significant
备注: Accepted at ICML 2026. 28 pages, 7 figures
点击查看摘要
Abstract:Anomaly detection in tabular data poses significant challenges due to heterogeneous feature types-mixing numerical, categorical, and textual attributes, which complicate learning meaningful representations of normality. Recent work has applied large language models (LLMs) to this problem by serializing table rows as text sequences, yet these approaches rely on one-shot supervised fine-tuning that offers limited signal to tighten the model's description of normality. We propose DiSPaT, a self-play fine-tuning framework that strengthens the model's understanding of normal data. Building on the theoretical foundation of f-divergence minimization, we derive a tight approximation connecting our training objective to reducing the distributional gap between real normal data and model-generated samples. DiSPaT operates through an alternating optimization: at each iteration, the current policy generates synthetic samples that serve as pseudo-anomalies, while a critic discriminator learns to distinguish these from real normal samples; this signal drives policy updates that progressively align the model distribution with the true normal-data distribution. Extensive experiments on diverse benchmarks demonstrate that DiSPaT consistently outperforms prior LLM-based methods, deep learning approaches, and classical unsupervised detectors for tabular anomaly detection.
12. 【2611.18359】Envisioning Agentic Robotics: A New Era of Robotic Systems
链接:https://arxiv.org/abs/2611.18359
作者:Grid Robotics:Qingfeng Lan,Duo Li,Siyao Li,Yiheng Lyu,Shijian Lu,Zhenghao Qi,Shizun Wang,Xinchao Wang,Ming Yang,Zhiqin Yang,Zuhao Yang,Tao Yu
类目:Robotics (cs.RO); Artificial Intelligence (cs.AI); Computation and Language (cs.CL); Computer Vision and Pattern Recognition (cs.CV)
关键词:Astra marks, marks the emergence, agentic robotic system, agentic robotics, Astra
备注: Project website: [this https URL](https://grid-robotics-official.github.io/agentic-robotics-roadmap)
点击查看摘要
Abstract:The release of GPT-6 Astra marks the emergence of a new paradigm in robotics: agentic robotics. Within weeks, frontier models with agentic harnesses matched or exceeded task-specific policies on many embodied benchmarks. Instead of training a dedicated policy for each family of tasks, an agentic robotic system applies a foundation model to open-world embodied tasks. It operates in a closed loop: observing the environment, interpreting user intent, reasoning about the task, invoking skills and tools, verifying outcomes, and adapting its strategy. Despite these advances, the community has yet to converge on a shared definition, a reliable evaluation framework, or a research roadmap. We argue that Embodied System 2, realized by a foundation model together with an embodied harness, forms the decision-making core of agentic robotics. Its central challenge is to translate general-purpose reasoning into effective and verifiable actions under physical constraints such as limited reaction time, mechanical limits, unpredictable environment changes, and irreversible consequences. Motivated by this perspective, we propose an evaluation framework that measures both model capabilities and closed-loop system performance. Furthermore, we identify the principles that carry over from digital to physical agents and the challenges that must be addressed to achieve reliable physical autonomy. On top of the Embodied System 2 and the evaluation framework, we envision a research roadmap toward sustained autonomy: the ability to perform work safely and reliably with progressively broader responsibility, from executing individual primitives to fulfilling standing roles.
13. 【2611.18319】Learning Word-Level Explanations of Text Reuse in Latin Literature with Little Annotation
链接:https://arxiv.org/abs/2611.18319
作者:Julian Schelb,Michael Wittweiler,Marie Revellio
类目:Computation and Language (cs.CL)
关键词:Detecting an intertextual, author reworked, source word, Detecting, word
备注: 18 pages, 5 figures, 8 tables. Code: [this https URL](https://github.com/julianschelb/retexo) Demo: [this https URL](https://julianschelb.github.io/retexo/demo/)
点击查看摘要
Abstract:Detecting an intertextual reference is only the first step; philologists also need to know how a later author reworked the source. We explain a reuse pair word by word: each word of the reusing passage is first aligned to its source word, then labeled with the operation that transformed it, such as a copy, a change of inflection, or a substitution. Such word-level operations are expensive to annotate, and only a few corpora annotated at this level exist for text reuse in Latin literature. We describe a procedure that needs only a limited number of annotations: synthetic pairs, built by applying known operations to Latin text, train a first model, and active learning chooses the pairs an annotator corrects. Among several architectures, a model that links and labels together performs best: a pointer network that, for every reuse word, chooses a source word and names the operation in the same step. We show that with active learning, 200 pairs chosen by uncertainty are enough for the model to match one trained on every annotated pair of the 1,490 references of the Loci Similes benchmark. We release our code, word-level predictions for all references, and an interactive demo.
14. 【2611.18297】When evaluation becomes intervention in clinical AI
链接:https://arxiv.org/abs/2611.18297
作者:Anthony Abou Mrad,Aviona Conti,Samuel Schmidgall,Jayanth Pratap,Ruhika Iyer,Joseph Kim,King Shi,Yann Sakref,Luis M. Ahumada,Lisa Knowlton,Andrew M. Cameron,Khyzer Aziz,Jeffrey K. Jopling
类目:Artificial Intelligence (cs.AI); Computation and Language (cs.CL); Computers and Society (cs.CY)
关键词:extend clinical review, grade another system, system output, extend clinical, clinical review
备注: 12 pages, 3 figures, 1 box. Submitted to npj Digital Medicine
点击查看摘要
Abstract:AI judges, models that grade another system's output against a rubric, extend clinical review to the scale at which clinical AI writes. When teams use judge assessments to select a prompt or train a model, the judge becomes part of the intervention. Selection can then favor errors the judge overlooks. We describe what a judge's report must contain and the independent clinical test each judge-driven change needs.
15. 【2611.18284】Clean Tickets, Wrong Orders: Catching Silent Errors with a Language Model in Your Browser Tab
链接:https://arxiv.org/abs/2611.18284
作者:Umar Ahmed,Maxim Dunaev,Mohammad Nour Al Awad,Sergey Ivanov,Ivan Khodnenko
类目:Machine Learning (cs.LG); Computation and Language (cs.CL); Human-Computer Interaction (cs.HC)
关键词:small language model, small language, turns a typed, structure can fail, typed order
备注: 4 pages, 4 figures, 2 tables. Demo: [this https URL](https://wrong.cuyor.com) . Companion paper: [arXiv:2611.17169](https://arxiv.org/abs/2611.17169)
点击查看摘要
Abstract:A small language model that turns a typed order into structure can fail in two ways. It can print something broken, which any system rejects. Or it can print a clean, well formed order that nobody placed. Our companion work calls the second kind a silent error, and it is the dangerous one. We present a web demo in which a fine-tuned 0.8B parameter model runs entirely inside the visitor's browser tab, through this http URL compiled to WebAssembly. No order text leaves the device. The page parses orders as the visitor types, scores every ticket with the PIZZA benchmark's own matcher ported to TypeScript, refuses anything off the menu with "?" so the model can face the open web, and lets visitors hunt for silent errors or try to cause one. Run in the browser on all 1,357 PIZZA test orders, the deployed model writes the same output as the server evaluation for 99.0% of them and reaches the same exact match (0.882); we also report its latency on 3 consumer laptops. Demo: this https URL
16. 【2611.18279】GT-FENG: General Theory of Factual Evaluation and Normative Grounding in Language Models *Author(s)
链接:https://arxiv.org/abs/2611.18279
作者:Jiayu Feng
类目:Computation and Language (cs.CL)
关键词:depends on context, correct often depends, legal definition, Normative Grounding, Theory of Factual
备注: 7 pages, 4 tables. Code: [this https URL](https://github.com/VonFlair/gtfeng)
点击查看摘要
Abstract:Whether a language model's answer is correct often depends on context: the same value can be right under one jurisdiction, date, or legal definition and wrong under another. We introduce GT-FENG (General Theory of Factual Evaluation and Normative Grounding), the first general framework to unify contextual/temporal correctness and rule-based error attribution across law, policy, and other rule-governed domains. GT-FENG checks answers against the applicable rules and identifies documented rules that match incorrect answers. It shows that when multiple rules give the same value, the answer alone cannot tell which rule the model used. In Medicaid income-limit answers from two models, documenting the asked state's own records reduces the number of wrong answers that match only other states' values by 68-78%, and more than half of stable wrong answers then match two or more documented sources. GT-FENG provides a foundation for evaluating language models in rule-governed domains and for future benchmarks and evaluation toolkits. We release GT-FENG as an open-source Python package, gtfeng.
17. 【2611.18264】MamaReason: Unfinished Reasoning and Answer Recovery in Clinical Language Models
链接:https://arxiv.org/abs/2611.18264
作者:Thanni Adewuyi,Adepoju Jeremiah Moyondafoluwa,Dorcas Amapu,Dieko Whenu,Sylvia Essessien,Sandra Oko-Oba,Pelumi Onafadeji,Lily Aimas-Love,Samuel Omolaiye,Abiodun Adereni
类目:Computation and Language (cs.CL)
关键词:stated in unfinished, dense Gemma, diagnosis, Abstract, unfinished clinical reasoning
备注:
点击查看摘要
Abstract:Answer continuation often returns a diagnosis already stated in unfinished clinical reasoning. We study this behavior in MamaReason, adapted from dense Gemma 4 31B, on MamaBench's 217 paired maternal and child-health presentations. An explicit-commitment extractor identifies a diagnosis in 75 of 94 failed RL first passes and 75 of 93 failed SFT first passes. The continuation matches that diagnosis in 67/75 RL cases and 58/75 SFT cases, using diagnosis strings alone. Reading these commitments without additional generation recovers 30 of the 40 RL pairs restored by continuation under the archived correctness labels. Recurrence controls distinguish routine mentions from repeated answer selection. An offline second-commitment trigger measures the tradeoff between earlier readout and paired accuracy. The resulting decomposition separates diagnostic content already present in a trace, completion of an accepted answer, and paired diagnostic correctness. For clinical language-model evaluation, these are distinct outcomes that become visible when the same reasoning trace is examined through multiple readouts.
18. 【2611.18257】NER-D: A cross-domain benchmark for German NER
链接:https://arxiv.org/abs/2611.18257
作者:Marc Brinner,Philipp Cimiano,Karolina Hanna Czok,Regina Stodden,Viju Sudhi,Sina Zarrieß
类目:Computation and Language (cs.CL)
关键词:Agriculture and Archaeology, German NER comprising, present NER-D, German NER, NER comprising
备注:
点击查看摘要
Abstract:We present NER-D, a cross-domain benchmark for German NER comprising 13 datasets spanning 9 domains, including General/Politics, Wikipedia, Legal, Medical, Historical, Financial, Agriculture and Archaeology. We examine the performance of both proprietary LLMs and state-of-the-art small open-source models using normal prompting and further examine the performance gains achievable via in-context learning and LoRA fine-tuning. Our results show that without any sort of optimization a proprietary model (GPT-5.4 mini) outperforms the smaller open-source models on 6 out of 13 datasets, but LoRA fine-tuning enables the smaller models to substantially outperform the proprietary baseline. We further analyze the influence of the domain and the tagset used in each task on the performance achieved in the different evaluation settings. Our benchmark will be available as a resource for the development of future German NER methods on both general and specialized domains.\footnote{We make our code, data, and models publicly available upon acceptance of this paper.
19. 【2611.18157】A Multi-Layered AI Pipeline for BPMN Generation from Natural Language
链接:https://arxiv.org/abs/2611.18157
作者:Alexandru Rancea,Ionut Anghel,Tudor Cioara,Ovidiu Anchidin,Ancuta Nemes
类目:Computation and Language (cs.CL)
关键词:natural language descriptions, Business Process Model, unstructured natural language, Large Language Models, natural language
备注:
点击查看摘要
Abstract:The automated transformation of unstructured natural language process descriptions into Business Process Model and Notation (BPMN) diagrams remains a challenging task, due to linguistic ambiguity and the difficulty to detect non-adjacent control-flow relationships. This paper proposes a solution for generating BPMN models from natural language descriptions through a multi-layered pipeline that integrates transformer-based architectures, Large Language Models and ontology population. The proposed method starts with natural language descriptions and separates textual processing, control-flow relation extraction, and structural validation into dedicated components. Entity extraction is performed using a fine-tuned RoBERTa-based transformer model while a control-flow engine captures non-adjacent relationships through LLM-based inference. To improve generalization under limited available data, the approach uses a dynamic few-shot strategy with con- trolled template input construction, enabling improved generalization under limited data availability. Additionally, an ontology population mechanism is introduced to represent the generated process models as semantic knowledge structures. For evaluation purposes we have used from the literature and industry reference processes. Additionally, we have validated the proposed method on a real-world heart disease patients transitional care process, capturing practical constraints and multi-actor collaboration, and serving as an expert-defined benchmark scenario. Experimental results demonstrate strong performance in the relation extraction task, achieving an accuracy of 90.7%, as well as maintaining structural alignment between the generated and reference BPMN models across multiple evaluation scenarios scenarios.
20. 【2611.18113】Folding Language Models into Themselves
链接:https://arxiv.org/abs/2611.18113
作者:Ziyang Zhang,Yubin Jing,Yuyao Li,Haofan Wang,Yichen Gong
类目:Computation and Language (cs.CL)
关键词:Knowledge distillation, large amount, teacher weights, teacher, Knowledge
备注: 14 pages, 4 figures, 9 tables. Code: [this https URL](https://github.com/myboker/concertina)
点击查看摘要
Abstract:Knowledge distillation usually trains a compact student from scratch, which needs a large amount of data even though the teacher's weights already encode most of what the student has to learn. In this work, we explore building the student directly from the teacher's weights. We propose Concertina, which folds every group of N adjacent teacher layers into one shared block that is executed N times. Each loop step carries a low-rank correction initialized from the layer it replaces, and a per-token gate mixes these corrections, to let every token recover the behavior of the original layers from a single shared block. We further design a distillation data recipe that balances general, mathematical, academic and targeted multiple-choice data, to recover all capabilities of the teacher together rather than one at the expense of another. On Qwen3-4B, a 2.27B Concertina model distilled on 22B tokens retains about 90% of the teacher on commonsense and mathematical reasoning and 84% on average across eight benchmarks. It outperforms same-size dense and layer-sharing baselines, as well as Qwen3-1.7B, which is trained on over a thousand times more data. These results suggest that a compact model need not be retrained at scale, since folding the teacher into itself and distilling it on the right data recovers most of its capabilities. Code is available at this https URL.
21. 【2611.18104】Contagious Fingerprints: When LLMs Inherit Another Model's Identity
链接:https://arxiv.org/abs/2611.18104
作者:Zailong Tian
类目:Computation and Language (cs.CL)
关键词:recognizes the LLM, fixed property, model, LLM, classifier recognizes
备注:
点击查看摘要
Abstract:The fingerprint by which a classifier recognizes the LLM behind a text is usually treated as a fixed property of the model. We show that it is contagious: a model that continues a conversation begun by another model inherits that model's fingerprint. This undermines audits of API relays, which can serve the first turns of a conversation with the model a customer pays for and silently hand the rest to a cheaper one. After such a handoff, detection of the substitute by a text classifier drops from 0.97 to 0.72 (DeepSeek replacing Claude Sonnet) and from 0.87 to 0.54 for a same-vendor downgrade (Haiku), and retraining on handoff data repairs only substitutes seen in training. Preregistered experiments trace the contagion to imitation of the history's style. History also moves a model away from its own fingerprint: active probes are not camouflaged, but when calibrated on fresh conversations they flag \emph{honest} models on long contexts at rates up to 1.00, and a standard MMD equality test rejects all nine models we test. Both failures come from comparing a history-dependent reply with a history-free reference. We propose \textsc{Lineup}, which samples references from the claimed model \emph{on the same history} and tests the audited reply by its rank among them. \textsc{Lineup} controls false positives by exchangeability and needs no fitted threshold or log-probabilities. With contemporaneous references on two closed models, its false-positive rate is 0.056 (preregistered bound 0.10), and it detects 0.81 of substitute sessions across six pairs after four audited turns.
22. 【2611.18096】VEPO: Mitigating Multilingual Hallucinations in LVLMs via Visual Evidence-guided Preference Optimization
链接:https://arxiv.org/abs/2611.18096
作者:Chengpeng Fu,Xiaocheng Feng,Wenshuai Huo,Yichong Huang,Baohang Li,Duyu Tang,Ting Liu
类目:Computation and Language (cs.CL)
关键词:Large vision-language models, achieve strong multimodal, strong multimodal performance, Large vision-language, achieve strong
备注:
点击查看摘要
Abstract:Large vision-language models (LVLMs) achieve strong multimodal performance, yet they remain prone to object hallucination---generating answers that are linguistically fluent but visually ungrounded. This issue is especially severe in low-resource languages, where weaker cross-modal alignment causes models to rely more on language priors than on image evidence. Existing remedies either require costly multilingual annotations or external translation/reward models, or intervene at inference time in ways that can distort the output language. We propose VEPO(Visual Evidence-guided Preference Optimization), a training-time method that transfers reliable visual grounding from English to other languages without external models. VEPO first localizes key visual evidence using English queries via mid-layer cross-modal attention, then constructs preference pairs that share the same correct answer but differ in evidence sufficiency (full image vs.\ key-region masked image). Direct preference optimization on these pairs encourages low-resource inputs to prefer answers supported by complete visual evidence. Experiments on multilingual POPE and MME with Chinese, French, and Hindi show consistent hallucination reduction and perception gains, with the largest improvements on the lowest-resource language. Analyses further confirm that VEPO sharpens question-relevant attention and that evidence-guided masking outperforms random masking.
23. 【2611.18089】Deep Latents, Long Memory: Latent persistence explains temporal correlations in transformer activations
链接:https://arxiv.org/abs/2611.18089
作者:Francesco Cagnetta,Santiago Acevedo
类目:Machine Learning (cs.LG); Computation and Language (cs.CL)
关键词:Large Language Models, Language Models, Large Language, believed to encode, encode representations
备注:
点击查看摘要
Abstract:The hidden layers of Large Language Models (LLMs) are believed to encode representations of the data whose abstraction increases with depth. However, in the absence of an operational notion of abstraction, this statement is hard to test quantitatively. Here, we propose an alternative view, building on the principle that the correlations of a sequence reflect the structure of the process that generated it. We define the autocorrelation function of a layer as the average dot product between its activations at two context positions separated by a lag. Latent variables encoded persistently along the context create slowly decaying tails in the autocorrelation, and we find these tails to grow across layers in both synthetic and natural data. In synthetic data with controlled hierarchical latent structure, this growth stems from the encoding of abstract latent variables, which govern long stretches of the input. Indeed, the persistence of latent codes predicts the qualitative structure of autocorrelations and accounts for most of the empirical measures. In real text, we take sparse-autoencoder features as proxies for latent variables, and find that their persistence correlates strongly with the magnitude of autocorrelations across datasets and LLMs. In particular, the persistence time increases with depth, as do autocorrelations, implying that deeper latent features persist for longer context spans. We complement this analysis by explicitly ablating the persistent sparse-autoencoder features and observing a consequent decrease in the autocorrelations.
24. 【2611.18067】Agent-as-a-Judge
链接:https://arxiv.org/abs/2611.18067
作者:Runyang You,Hongru Cai,Caiqi Zhang,Qiancheng Xu,Meng Liu,Tiezheng Yu,Yongqi Li,Wenjie Li
类目:Computation and Language (cs.CL)
关键词:models spanning perception, general models spanning, Artificial intelligence, making reliable evaluation, reliable evaluation essential
备注: Project page: [this https URL](https://github.com/ModalityDance/Awesome-Agent-as-a-Judge)
点击查看摘要
Abstract:Artificial intelligence has progressed from task-specific systems toward general models spanning perception, generation, reasoning, and autonomous interaction, making reliable evaluation essential for comparison, deployment, and post-training feedback. At scale, modern evaluation pipelines increasingly rely on LLM-as-a-Judge to produce scores, preferences, and critiques; in its fixed-context setting, the judge relies on the supplied evaluand and evaluation materials. Yet a reliable verdict may require evidence beyond the supplied evaluand and context---either because critical environmental states remain unrecorded in the static execution log, or because controversial factual claims require external verification. Passive judges cannot obtain that missing evidence before deciding, especially when evaluating agent trajectories, executable artifacts, or domain-specific reports. Recent work therefore develops active evaluators that seek missing evidence and update their grounds for judgment before deciding. We bring these advances together through Agent-as-a-Judge, focusing on how evaluators construct and control checking processes within the broader LLM-based judging landscape. We define the scope through evaluation-directed checks and the use of their results, covering both structured workflows and observation-driven control. We organize the design space by multi-agent collaboration, tool integration, memory, and planning, and show how these mechanisms supply the checks and evidence on which a verdict rests. We consolidate published scores and reproduce missing entries for key agentic judges; the comparison reveals uneven agentic gains and motivates dedicated agentic evaluation. We conclude by characterizing application demands and open challenges in cost, reliability, safety, and governance that condition trusted deployment of agentic judges at scale.
25. 【2611.18050】FEND: Far-End Detection in Outbound Voice AI Calls
链接:https://arxiv.org/abs/2611.18050
作者:Abhishek Kumar,Suraj Agrawal,Aashraya Sachdeva
类目:Computation and Language (cs.CL); Audio and Speech Processing (eess.AS)
关键词:interactive voice response, Answering Machine Detection, Standard Answering Machine, voice response, conversational Voice
备注: Submitted to EACL 2027
点击查看摘要
Abstract:When a conversational Voice AI agent places an outbound call, the far end may be a live person, a voicemail greeting, an interactive voice response (IVR) system, or increasingly another AI system answering on the callee's behalf. Standard Answering Machine Detection (AMD) reports only whether a human or a machine picked up, a design meant for human call centres, where one machine is as uninteresting as another. That single bit does not tell a Voice AI agent what to do, since leaving a voicemail message, navigating an IVR and holding a short exchange with another AI are three different actions and a two-valued label cannot distinguish them. In 1,000 randomly sampled calls from our own outbound deployment, a live person answered 48.1%, a voicemail system 44.6%, an AI 6.9% and an IVR 0.4%. Our proposed framework, FEND, reads the far end's audio and its streaming transcript and runs one detector for each automated class (voicemail, IVR, AI callee), with human the default when nothing fires, returning a single label within 20 seconds. On a public benchmark the voicemail components reach 100% accuracy without ever misclassifying a live human, where Asterisk's AMD misclassifies 30 to 66% of them. For the AI callee, acoustic features do not transfer across recording conditions, but speech timing does, separating AI from human at 0.952 ROC-AUC, because synthesised speech leaves far shorter gaps between words. On production traffic, separating a live person from any automated far end reaches F1 0.979, and only 2.9% of live people are given an automated label, the error that matters most for an agent acting on the result.
26. 【2611.18030】PerfArena: Ranking LLMs by Code Performance
链接:https://arxiv.org/abs/2611.18030
作者:Rui Rua,Hailemariam Mersha,Joao Paulo Fernandes,Karim Ali
类目:Performance (cs.PF); Computation and Language (cs.CL); Software Engineering (cs.SE)
关键词:Industry now relies, write a large, large share, share of production, energy consumption
备注:
点击查看摘要
Abstract:Industry now relies on LLMs to write a large share of production code. Although that code is mostly, but not always, correct, correctness alone does not determine whether it is fit for repeated and extensive use, which depends on what the code costs in time, memory and energy. Most code-generation benchmarks nonetheless evaluate correctness in isolation, and most performance-oriented benchmarks cover few languages and rarely measure energy consumption. To address this gap, we present PERFARENA, an open-source, reproducible arena that ranks LLMs by the performance of their generated code across multiple languages and competitions. PERFARENA executes the solutions of each LLM on a Unix-like host and reports correctness together with latency, peak memory, and energy consumption. Correctness oracles, performance oracles, profilers, and code-generation workflows are pluggable in PERFARENA, enabling the integration of new judging backends and measurement tools without changing the pipeline. We evaluate PERFARENA on two competitions: CLBGComp, 10 compute-intensive benchmarks from the Computer Language Benchmarks Game, and LeetCodeComp, 99 LeetCode problems whose accepted solutions differ measurably in latency. Currently, PERFARENA holds 9,455 correct and measured solutions from 13 state-of-the-art LLMs across 7 programming languages. Across both competitions, got-6-astra is the only LLM that no other LLM beats significantly in latency, energy consumption, or peak memory, solving 99.7% of LeetCodeComp and 100% of CLBGComp. All PERFARENA leaderboards and measured solutions are publicly available, providing a continuously updated reference that helps developers choose an LLM per language and metric, instead of relying only on correctness rates or pricing.
27. 【2611.17963】oward Universal Translatability: On the Statistical Ontology of Generative AI
链接:https://arxiv.org/abs/2611.17963
作者:Katia Schwerzmann
类目:Computation and Language (cs.CL)
关键词:point of origin, normative horizon, horizon for contemporary, translation functions, paper argues
备注:
点击查看摘要
Abstract:This paper argues that the problem of translation functions both as a point of origin and as a normative horizon for contemporary generative AI. In current discourse around ''world models,'' their attainment is framed as the achievement of internal consistency, cross-modal alignment, and physical grounding. Taken together, these demands converge on an ideal of the model as a universal translation machine capable of establishing lossless correspondences across linguistic, visual, and physical representation spaces. I show that this ideal presupposes a metaphysical commitment that I call a statistical ontology: the belief that reality has an underlying, computable statistical structure that can be recovered through sufficient data and modeling, rather than being shaped and accessed through situated human interpretation and labor. Tracing this commitment back to Warren Weaver's 1949 Memorandum on machine translation, I outline how translation was historically recast as code deciphering and, later, as the alignment of probability distributions across structured outputs. Finally, I contrast this paradigm with approaches in Translation Studies that foreground opacity, power asymmetries, and the translator's positionality. I argue that generative AI's statistical ontology is pre-Kantian, pre-critical and obscures the infrastructures and human labor that make machine translatability---as alignment---possible in the first place.
28. 【2611.17955】WebArena-Pro: A Heterogeneous, Multimodal, Reproducible Benchmark for Web Agents
链接:https://arxiv.org/abs/2611.17955
作者:Imene Kerboua,Fatemeh Pesaran Zadeh,Xing Han Lù,Weijian Qi,Alexander Miller,Junyi Song,Yunjia Tian,Dongjin Kang,Seyeon Choi,Marzia Nouri,Ewen Gueguen,Matteo Boglioni,Fengyuan Liu,Zeyi Liao,Mengqi Yuan,Yue Li,Alexandre Lacoste,Alexandre Drouin,Spandana Gella,Huan Sun,Gunhee Kim,Siva Reddy
类目:Artificial Intelligence (cs.AI); Computation and Language (cs.CL); Machine Learning (cs.LG)
关键词:realistic browser work, spans heterogeneous applications, Web agents powered, powered by large, large language
备注: Accepted at NeurIPS 2026 ED Track
点击查看摘要
Abstract:Web agents powered by large language and vision-language models are increasingly applied to realistic browser work that spans heterogeneous applications, multimodal content, and stateful workflows. However, existing reproducible web-agent benchmarks cover only a small number of web applications drawn from a few software categories, and restrict modality to text and vision. Live benchmarks broaden site coverage but sacrifice reproducibility, since pages and data drift between runs. Moreover, existing benchmarks do not meaningfully evaluate whether agents can understand and use audio and video content embedded within web tasks. To address these gaps, we introduce WebArena-Pro, a benchmark comprising 300 tasks across 20 self-hosted web applications in six domain categories, spanning distinct interface conventions, workflows, and data models. Across the evaluated agents, the best performance is achieved by both Codex with GPT 6.1 Sol and Claude Code with Claude Opus 5.5 at 62.3%, followed by OpenCode with Kimi 3.1 Pro at 60.7%. The strongest GenericAgent configurations, Gemini 3.1 Pro omni and Gemma4 31B, both achieve 30.3%. Among reproducible, human-curated web agent benchmarks, WebArena-Pro provides the broadest application coverage and the most comprehensive multimodal support to date. The benchmark treats audio and video as core observations alongside text and vision, with dedicated actions for extracting information from each. WebArena-Pro runs each task in isolation and supports reproducible, parallel evaluation. Tasks are authored through a dedicated annotator interface, filtered by LLM-assisted triage, and finally validated by humans before release. Project page: this https URL
29. 【2611.17954】In the Eye of the Beholder: Human Label Variation in Textual and Visual Entailment
链接:https://arxiv.org/abs/2611.17954
作者:Monica Riedler,Jasmin Orth,Raffaella Bernardi,Barbara Plank
类目:Computation and Language (cs.CL)
关键词:reflects meaningful differences, Natural Language Inference, increasingly recognised, informative signal, signal that reflects
备注: 22 pages, 18 figures
点击查看摘要
Abstract:Human Label Variation (HLV) is increasingly recognised as an informative signal that reflects meaningful differences in how people interpret the same input. While HLV has been studied extensively in Natural Language Inference (NLI), much less is known about HLV in Visual Entailment (VE; Xie et al., 2019), where the textual premise is replaced by an image, keeping the hypothesis and label space fixed. We introduce VariNLI-VE, a dataset for studying how HLV changes between paired NLI and VE instances. We re-annotate a subset of originally neutral SNLI (Bowman et al., 2015) instances and their visual counterparts, with the same annotators evaluating both the textual and visual versions. Each annotation includes a label, confidence score, and explanation, yielding 2,250 annotations. Labels and explanations are collected jointly, providing ecologically valid explanations. Because the image may reveal information not stated in the text, one might expect disagreement to decrease in VE. Instead, we find no clear overall reduction: disagreement decreases for some instances while it increases for others. Explanations reveal further variation, as annotators can retain the same label while changing their rationale. When evaluating vision-language models (VLMs) on the paired data, we find lower accuracy, weaker alignment with human label distributions, and generally higher uncertainty in VE than in NLI.
30. 【2611.17939】Can You Trust Your Finance Agent? Drift, Diagnosis and Auditable Intervention on One Shipped Financial Research Agent
链接:https://arxiv.org/abs/2611.17939
作者:Cosimo Galeone,Anna Ettorre,Ginevra Mariani,Minsu Park,Giuseppe Ettorre,Daniele Ligorio
类目:Computation and Language (cs.CL); Artificial Intelligence (cs.AI)
关键词:Finance Agent Benchmark, Vals Finance Agent, finance agent, Vals Finance, agent
备注: 9 pages. Accepted as an oral presentation at the AI4F workshop, ICAIF 2026, Milan
点击查看摘要
Abstract:Can you trust a finance agent you did not build? We take the reference agent shipped with the Vals Finance Agent Benchmark v2, leave it untuned, and ask four questions a leaderboard score does not answer. Is the score stable? Seven identical runs on 18 questions pass between 105 and 123 of 149 rubric criteria, and the bill varies by up to 17%. When the agent fails, who owns the fix? We classify 20 failed trajectories, from three runs of nine held-out questions, by the layer whose owner would remedy them, under a taxonomy frozen in advance: 13 belong to the agent's instructions, 3 to its tools, 4 to the model, none to its data. Can that diagnosis be trusted? Four blind readers from three model families, given only the traces, name the same fault as our LLM analyzer in 94% of cases, the rate at which they agree with one another (93%). Can a change be trusted? An optimization loop reads the traces, makes one targeted change per round and keeps it only if it clears a fixed threshold. Under a cost objective its final configuration costs 0.34x the baseline on the held-out questions (95% CI [0.22, 0.58]) at accuracy within noise. Under an accuracy objective it gains 14 points on the development questions and 1.5 on the held-out ones (CI [-5.3, +7.2]). Judged against the measured run-to-run noise, under a rule written before the test was opened, the first is a real saving and the second is not. This is a case study of one agent and nine test questions, not a general result.
31. 【2611.17931】MineralExp: Benchmarking LLM Agents for Experimental Design in Real-World Mineral Exploration
链接:https://arxiv.org/abs/2611.17931
作者:Bing Wang,Mingyang Wang,Yuanhang Mi,Yue Xu,Kaibo Su,Li'ang Xiao,Chunguo Wu,Changchun Li,Ximing Li,Kun-Feng Qiu,Bo Yang
类目:Computation and Language (cs.CL); Artificial Intelligence (cs.AI); Computational Engineering, Finance, and Science (cs.CE)
关键词:verifying geological hypotheses, geological hypotheses relies, hypotheses relies, relies on designing, designing a variety
备注: 16 pages, 6 figures
点击查看摘要
Abstract:In mineral exploration, verifying geological hypotheses relies on designing a variety of geochemical experiments. While large language model (LLM) agents increasingly automate downstream geoscientific data analysis, a crucial upstream capability remains largely unexplored: automated geochemical experimental design, i.e., deciding which experiments to conduct before committing to costly laboratory work. Evaluating this capability of LLM agents is difficult for two reasons. First, geochemical experiments exhibit high methodological diversity and subjectivity: to test the same hypothesis, different geologists may choose different minerals and instruments, so no single protocol can serve as the reference answer. Second, open-ended experimental plans resist representation as structured labels, which makes objective scoring difficult. To address these challenges, we introduce MineralExp, a benchmark that reformulates geochemical experimental design as a hypothesis-based experiment selection problem, i.e., a multi-label classification problem. We abstract concrete instrument names and procedural details into a shared taxonomy of 19 experimental types across five mineral-system processes: Source, Transport, Deposition, Modification, and Preservation. This taxonomy yields deterministic multi-label targets while accommodating diverse experimental protocols. Constructed from published ore-deposit literature and verified by domain experts, MineralExp comprises 277 cases and 1,572 reference experimental plans. Given a geological hypothesis and initial context, an LLM agent selects the relevant tasks and generates concrete experimental plans, which are scored by multi-label classification metrics and an LLM-as-a-Judge metric. The experimental results show that current agents generally achieve high recall but low precision, tending to propose more experiments than are necessary to test the hypothesis.
32. 【2611.17900】AmbiSteer: Selectively Eliciting Ambiguity-Aware Responses in Vision-Language Models
链接:https://arxiv.org/abs/2611.17900
作者:Monica Riedler,Samuele Antonelli,Barbara Plank,Raffaella Bernardi
类目:Computer Vision and Pattern Recognition (cs.CV); Computation and Language (cs.CL)
关键词:providing firm answers, Visual Question Answering, providing firm, evidence is insufficient, Vision-language models
备注: 23 pages, 23 figures
点击查看摘要
Abstract:Vision-language models (VLMs) often exhibit overconfidence, providing firm answers even when visual evidence is insufficient. A prominent case is referential ambiguity in Visual Question Answering (VQA), where multiple entities in an image are plausible referents of a question. Addressing such cases requires recognizing when ambiguity is present: ambiguous questions can be handled by asking for clarification or enumerating plausible referents, while unambiguous questions should receive a direct answer. Current VLMs struggle with this distinction, often defaulting to a single interpretation. To investigate whether activation steering can serve as an inference-time mechanism to elicit such behaviors, we introduce AmbiSteer, a lightweight framework to selectively elicit ambiguity-aware behavior. Across three VLMs, we find that steering alone has limited and model-dependent effects, while prompting is generally more effective. Combining prompting and steering substantially increases clarification and enumeration behaviors, but also elicits them on unambiguous cases. AmbiSteer addresses this using a linear classifier to predict input ambiguity and condition the steering direction accordingly, increasing the proportion of inputs handled appropriately from 59.0% to 77.0% on average across models. Our work sets the ground for lightweight approaches to reduce model overconfidence in uncertain multimodal settings.
33. 【2611.17883】Securing Multi-Agent Systems by Design
链接:https://arxiv.org/abs/2611.17883
作者:Xilong Wang,Xiaoxing Chen,Neil Gong
类目:Cryptography and Security (cs.CR); Artificial Intelligence (cs.AI); Computation and Language (cs.CL)
关键词:solve complex tasks, complex tasks, tasks by decomposing, aggregating the agents', systems solve complex
备注:
点击查看摘要
Abstract:Multi-agent systems solve complex tasks by decomposing them into subtasks, delegating the subtasks to agents, and aggregating the agents' responses using an aggregation rule. Malicious agents can manipulate the final solution by providing malicious responses to their assigned subtasks. In this paper, we first propose response manipulation attacks (ReMA), which iteratively optimize malicious responses to maximize attack effectiveness. Specifically, we formulate untargeted and targeted attacks as optimization problems and develop an approach to approximately solve these problems by iteratively refining malicious responses based on feedback from a surrogate multi-agent system. To defend against such attacks, we further propose SecMAS, a secure aggregation rule that leverages the semantic similarities among agent responses to select the response that is most consistent with a sufficiently large subset of the other responses. We theoretically establish security guarantees for SecMAS against both untargeted and targeted attacks. Extensive evaluations demonstrate that existing aggregation rules are vulnerable to ReMA, whereas SecMAS effectively defends against all evaluated attacks, including adaptive variants of ReMA, while maintaining utility comparable to that of existing aggregation rules.
34. 【2611.17876】Delayed Effects of Temporary Weight Decay in Muon Pretraining
链接:https://arxiv.org/abs/2611.17876
作者:Md Rifat Arefin,Ravid Shwartz-Ziv,Ernie Chang,Chinnadhurai Sankar,Rylan Conway,Aristide Baratin,Irina Rish,Adithya Sagar,Patrick Huber
类目:Machine Learning (cs.LG); Computation and Language (cs.CL)
关键词:increase validation loss, Muon-trained decoder-only Transformers, increase validation, final training budget, final
备注:
点击查看摘要
Abstract:A temporary optimization intervention can increase validation loss when it ends yet reduce it at the final training budget. We demonstrate this with temporary weight decay in Muon-trained decoder-only Transformers with 162M, 405M, 1.4B, and 3.4B parameters. For models up to 1.4B parameters, moving a fixed-duration decay window from early to late training reverses its final effect. At 162M, changing window duration alters final loss even at approximately matched total shrinkage. Most strikingly, a schedule selected at 162M and transferred without retuning to 1.4B is 0.26 nats worse than no decay when removed, overtakes it only after 73% of training, and ultimately finishes 0.028 nats better while also outperforming the best sampled constant-decay baseline. Early decay leaves a persistent weight-norm deficit, which keeps relative steps larger after the intervention ends. An approximate norm-dynamics recurrence captures the slow convergence of these relative steps, and fixed-norm replays of the measured trajectories reproduce the performance ordering and a similar loss difference. At 162M and 405M, norm-reset experiments show that smaller weight norms at the end of the decay window explain most of the final loss improvement. A separate Gram intervention exhibits related post-removal norm and relative-step dynamics. These results show that temporary optimization schedules can have delayed effects long after removal and should therefore be evaluated at the intended final training budget, not only at removal.
35. 【2611.17867】UltraBERT-JEV: Encoder models for System One Applications
链接:https://arxiv.org/abs/2611.17867
作者:Astarag Mohapatra
类目:Computation and Language (cs.CL); Artificial Intelligence (cs.AI)
关键词:study encoder models, single forward pass, generating text, models that pick, single forward
备注:
点击查看摘要
Abstract:We study encoder models that pick one of several options in a single forward pass, without generating text. This technical report compares recipes for fine-tuning UltraBERT-380M with one mask per option. The four recipes differ in how an option is scored and how the scores become probabilities; one follows the expert gate of DeepSeek-V3. Trained on the same data as Qwen3-0.6B with the LLM-as-Jev recipe (67.6% test accuracy), the encoder reaches 69.7% to 72.1%. Scoring an option by "yes" minus "no" instead of "yes" alone gives the larger gain, from 69.7% to 71.3% on the test set and from 57.8% to 59.6% on external benchmarks (56.1% for the decoder). Sigmoid normalisation adds 0.8 test points; a square-root softplus adds none. One request at a time on an NVIDIA A100, the encoder takes 29 ms per decision against 68 ms, but uses more peak memory (2.8 against 2.2 GiB). An auxiliary head, trained on human-marked evidence and on labels selected with a reward adapted from SelfCite, predicts in the same pass the sentences annotated as support for the answer.
36. 【2611.17846】Is Abliteration Getting Harder? Not Uniformly: Refusal Removal by Direction-Based Ablation and ARA in Heretic on the Qwen3.8, Gemma 4, Olmo 3.1 and Mistral Small 3.2 LLMs Versus Their Predecessors
链接:https://arxiv.org/abs/2611.17846
作者:Richard J. Young
类目:Artificial Intelligence (cs.AI); Computation and Language (cs.CL)
关键词:open-weight large language, large language models, weight edit, open-weight large, large language
备注: 76 pages, 19 figures, 34 tables. Code, configurations and judge labels: [this https URL](https://github.com/ricyoung/abliteration-studies) ; models: [this https URL](https://huggingface.co/richardyoung)
点击查看摘要
Abstract:Abliteration, a fine-tuning-free weight edit, removes refusal from open-weight large language models (LLMs). The developer of Arbitrary-Rank Ablation (ARA), an optimisation-based variant, reports that direction-based abliteration, which ablates a refusal direction, increasingly fails on newer models. On five release pairs, refusal removal did not become uniformly harder. Qwen2.5-32B and Qwen3.8-27B, Gemma-2-27B and Gemma-4-31B, OLMo-2-32B and Olmo-3.1-32B, Mistral-Small-2409 and Mistral-Small-3.2, and Qwen2.5-7B and Qwen3.5-4B were edited in Heretic 2.0 by three or four direction-based methods (difference-in-means, MPOA, SOM and, on the small pair, J-space) and ARA under the same prompts, nominal search budget, first-token KL divergence cap and selection rule; removal counted as harder when the automatic WildGuard judge found more of 972 held-out harmful prompts (JailbreakBench, XSTest, StrongREJECT, HarmBench and an author-compiled malicious-code bank) still refused. After direction-based removal, newer releases retained -1 to +37 percentage points more refusal, and after ARA -26 to +34. With run-to-run variance included, between-pair heterogeneity of the direction-based effect ranged from high to low with the scale and variance model (I^2 20-80%). ARA lost less effectiveness than direction-based methods in Gemma and OLMo and more in Qwen; each verdict rests on one ARA run, and some changed with the divergence budget or variance model. In Gemma the direction-based increase came from the malicious-code bank and reversed on public sets. Contrary to prediction, keyword counts at the tool's window did not undercount judged refusal on average, but differed by a mean 7 points per model. Audits of a single pair, method or prompt source can reach opposite conclusions. Whether a newer model resisted removal depended on the release pair, method and budget, not on generation alone.
37. 【2611.17836】BLiMP-RO: A Benchmark of Linguistic Minimal Pairs for Romanian
链接:https://arxiv.org/abs/2611.17836
作者:Radu-Stefan Popa,Valentin Feroiu,Adina Camelia Bleotu,Sergiu Nisioi
类目:Computation and Language (cs.CL)
关键词:Evaluating grammatical knowledge, cover diverse linguistic, Evaluating grammatical, language models requires, human judgments
备注:
点击查看摘要
Abstract:Evaluating grammatical knowledge in language models requires benchmarks that cover diverse linguistic phenomena and are validated against human judgments. We introduce BLiMP-RO, a Romanian minimal-pair benchmark covering 50 grammatical phenomena across ten domains. It comprises 493 linguist-constructed sentence pairs, evaluated by native speakers in five experiments, and 5,942 additional semi-automatically generated, human-validated pairs. We further investigate grammatical generalization under limited linguistic input by training GPT-2 and GPT-BERT models from scratch on Romanian BabyBabelLM data and a newly collected corpus of 10 million words from child-targeted video captions. Increasing caption training data yields approximately 9 percentage points of accuracy improvement per tenfold increase in corpus size. At matched data sizes, BabyBabelLM generally outperforms video captions, while GPT-BERT consistently outperforms GPT-2. Our findings highlight how training data quantity, composition, and model architecture shape grammatical generalization in Romanian.
38. 【2611.17834】PaperCritic: Training AI Reviewers to Assess the Soundness of Papers
链接:https://arxiv.org/abs/2611.17834
作者:Hyun Ryu,Ji Yong Cho,Eunho Yang,Moontae Lee,June Yong Yang,Sean Welleck,Carolyn Rose
类目:Computation and Language (cs.CL)
关键词:submissions grow rapidly, conference submissions grow, grow rapidly, review points, review
备注:
点击查看摘要
Abstract:As AI conference submissions grow rapidly, AI reviewers are being considered as an alternative. However, simply imitating human reviewing is problematic because reviewing serves multiple purposes, including evaluating papers' soundness, interestingness, and clarity, the last two of which are subjective. In this work, we focus solely on evaluating soundness, the most important and objective criterion, and define the learning objective as jointly maximizing (1) precision, based on the soundness and substantiveness of review points, and (2) recall, the coverage of high-quality reference review points. Under this problem formulation, we construct a quality-labeled review point dataset, PaperSound-23K, using 1K ICLR submissions and 23K human-written and LLM-generated review points. Using this dataset, we train an AI reviewer model, PaperCritic, through reinforcement learning with the F1 score of precision and recall as the reward. Our experiments show that PaperCritic outperforms frontier models and models trained with other AI reviewer training methods on an in-domain test set. Surprisingly, learning to review ICLR submissions also transfers to reviewing Nature-family papers and even to out-of-domain reviewing tasks (e.g., code review, legal contract review, and feedback on story writing).
39. 【2611.17814】G-CAST: Governed Continual Self-Training for E-Commerce Agents under Evolving Demands
链接:https://arxiv.org/abs/2611.17814
作者:Yuechun Yu,Wenzheng Kong,Xiaofeng Mao,Jin Zhang
类目:Artificial Intelligence (cs.AI); Computation and Language (cs.CL)
关键词:Large Language Models, Large Language, advancement of Large, Language Models, rapid advancement
备注:
点击查看摘要
Abstract:With the rapid advancement of Large Language Models (LLMs), an increasing number of users are turning to LLM-powered agents for conversational shopping in e-commerce. However, the user needs continue to change as new product categories, procurement scenarios, and feature requests emerge. At the same time, the product's execution harness-including its tools, skills, and operating procedures-continues to evolve to support increasingly sophisticated interactions. These changes create two practical challenges: 1) Demand drift alters the distribution, intent, and complexity of user requests, causing a model trained once to fall behind; 2) Harness evolution requires the model to use newly introduced capabilities correctly without losing behavior that users already rely on. Periodic manual retraining is too delayed and labor-intensive for this setting. To address these challenges, we present G-CAST, a governed continual self-training system that converts daily production evidence under evolving demand and capability contracts into reward-routed, entropy-prioritized training data and governed reward evolution. Governed checkpoint transition under cumulative evaluation controls each model update, enabling the agent to learn newly introduced behaviors without sacrificing previously accepted capabilities. Experiments on a real production e-commerce agent show that G-CAST achieves stronger adaptation to newly introduced demand while better preserving previously accepted behavior compared with a strong autonomous post-training baseline. Ablations further show that both reward-routed, entropy-prioritized data orchestration and governed reward evolution are important to these gains. G-CAST is now deployed in Yuancai(this http URL), a B2B procurement agent on Alibaba's 1688 platform.
40. 【2611.17811】AI Reviews Are Biased Towards the Appendix
链接:https://arxiv.org/abs/2611.17811
作者:Saptarshi Ghosh,Mohammad Saim,Tianyu Jiang
类目:Computation and Language (cs.CL)
关键词:Large language models, Large language, reviewing process, reviewers to highlight, outputs often shape
备注:
点击查看摘要
Abstract:Large language models (LLMs) are increasingly used by reviewers to highlight weaknesses during the reviewing process, and their outputs often shape the reviews that humans submit. Prior work studies how LLM reviews align with human ones, yet we know little about which parts of a paper these models actually criticize. In this work, we investigate 2,000 papers from ACL, EMNLP, COLM, and ICLR, and show that the latest GPT and Claude models draw about 50% or more of their review weaknesses from the appendix. This behavior holds across the API, coding-agent, and web access methods for using LLMs to review a paper. In contrast, we find that humans draw only 8.4% of their weaknesses from the appendix, and these weaknesses primarily revolve around a paper's presentation and content related to its core idea. Meanwhile, LLMs draw weaknesses from deep in the appendix that concern fine-grained technical details not central to the paper's contributions. We anticipate that our analysis will contribute to the scholarly discourse on AI reviews. Our code is publicly available at: this https URL.
41. 【2611.17808】CarryOn: Learning What to Keep in Context for Long-Horizon LLM Agents
链接:https://arxiv.org/abs/2611.17808
作者:Lihuan Li,Ruiyi Yang,Jiru Wang,Yifan Duan,Xiachong Lin,Hao Xue,Yang Song,Flora Salim
类目:Computation and Language (cs.CL); Artificial Intelligence (cs.AI)
关键词:Long-horizon LLM agents, Long-horizon LLM, complete complex, agents can coordinate, coordinate tools
备注:
点击查看摘要
Abstract:Long-horizon LLM agents can coordinate tools and applications to complete complex, multi-step tasks. As their interaction histories grow, effective context management becomes essential for reliable and efficient execution. Existing methods reduce context through summarization, pruning, or history selection. However, shorter contexts do not necessarily support better decisions: summaries can omit exact details needed for later actions, while retaining entire interactions can carry substantial irrelevant information. We propose CarryOn, a lightweight context manager that pairs compact execution state with fine-grained, source-linked evidence. A learned selector and adaptive retention policy keep relevant evidence available without additional LLM calls for context management. We introduce two complementary evaluation metrics: $S_\alpha$ prioritizes task success while accounting for resource use, and normalized cost per success (NCS) measures the resources spent per successful task. Across AppWorld, OfficeBench, and Tau2 Telecom with three LLM Actors, CarryOn achieves 75.7% macro-averaged task success, exceeding the strongest baseline by 6.7 percentage points. It also achieves the best average scores on both metrics.
42. 【2611.17788】RAG-FGO: Dynamic Optimization of Query-Rewriting Agents for Generative RAG
链接:https://arxiv.org/abs/2611.17788
作者:Qigan Sun,Xudong Wang,Chaoning Zhang,Chenghao Li,Jiaquan Zhang,Jiehui Xie,Sungyoung Lee,Jie Zou,Ning Xie,Peng Wang,Yang Yang,Heng Tao Shen
类目:Computation and Language (cs.CL)
关键词:external knowledge bases, aligns user intent, effectiveness directly determines, query rewriting serves, retrieval-augmented generation
备注: 12 pages, 6 figures, 8 tables
点击查看摘要
Abstract:In retrieval-augmented generation (RAG), query rewriting serves as a core interface that aligns user intent with external knowledge bases, and its effectiveness directly determines overall system performance. However, generative retrieval faces three critical challenges in real-world deployment: noisy retrieval feedback, the difficulty of jointly optimizing discrete prompt templates and continuous decoding-control parameters, and the limited generalization of static rewriting strategies under semantically complex scenarios. To address these issues, we propose RAG-FGO, a dynamic optimization framework for generative retrieval strategies. RAG-FGO formulates query rewriting as a non-differentiable black-box optimization problem in a mixed discrete-continuous space, and represents prompt templates and decoding parameters as adaptively optimizable query-rewriting agents. The framework performs global search to enable semantic jumps and structural reorganization that escape local optima, while applying local perturbations for fine-grained alignment and precise refinement. In addition, RAG-FGO introduces an elite-preserving memory pool that accumulates high-quality strategies across iterations, thereby mitigating the impact of stochasticity and noisy feedback on optimization stability. Experimental results on a diverse suite of benchmarks demonstrate the effectiveness and robustness of the proposed method, supporting its applicability to complex semantic retrieval tasks.
43. 【2611.17780】LuthorSQL: A One-Pass Text-to-SQL Model as System 1 for a Slower LLM Agent
链接:https://arxiv.org/abs/2611.17780
作者:Ibrahim Memon,Armin Karamzade,M. Hanif Jhaveri,Roy Fox
类目:Computation and Language (cs.CL); Artificial Intelligence (cs.AI); Databases (cs.DB); Machine Learning (cs.LG)
关键词:Frontier LLM agents, expose sensitive organizational, Frontier LLM, sensitive organizational queries, per-query calls
备注: 8-page main text plus appendix, 23 pages
点击查看摘要
Abstract:Frontier LLM agents that query databases with tools are among the most accurate text-to-SQL systems, but per-query calls to a hosted model are slow, costly, and expose sensitive organizational queries to external APIs. For example, a Claude Opus 5.5 agent serving 10,000 queries a day costs over $70,000 a year at list price. To what extent can local hardware already owned by an organization offload this burden without compromising the agent's accuracy? We present LuthorSQL, a non-autoregressive 7B text-to-SQL model that serves as a rapid System 1 gate for a slower LLM agent (System 2), escalating queries only when uncertain. LuthorSQL makes a single forward pass to predict slot scores over a bounded grammar for query plans, compiles and runs up to 32 candidate plans, and chooses an answer using a trained validator. On its own, LuthorSQL reaches 68.03% execution accuracy on BIRD's hidden test set at 0.84 s per question on a single GPU, maintains its accuracy in 9 GB with 8-bit quantized weights, and transfers zero-shot to Snowflake, BigQuery, MySQL, PostgreSQL, and DuckDB within 3.1 points of its native SQLite accuracy. Combined with Claude Opus 5.5 in a pre-registered confidence cascade over 227 questions on databases published after Opus's training cutoff, LuthorSQL answers nearly half of all questions locally and cuts API cost by 33.3% at the same accuracy. Finally, contamination probing shows that Opus reproduces 48-92% of public benchmark gold queries verbatim without seeing the schema; however, its tool-assisted accuracy holds on renamed schemas and reaches 95.6-97.4% on post-cutoff databases. Code and data: this https URL
44. 【2611.17768】Activation Steering the Nan-Shan in Typhoon
链接:https://arxiv.org/abs/2611.17768
作者:Megan Wei,Damon Pham,John Cayley,Ellie Pavlick
类目:Computation and Language (cs.CL); Artificial Intelligence (cs.AI); Machine Learning (cs.LG); Sound (cs.SD)
关键词:Captain MacWhirr steers, Joseph Conrad novella, audiovisual installation based, steamer Nan-Shan directly, based on Joseph
备注: Accepted to NeurIPS Creative AI Track 2026. Website: [this https URL](https://stormhelm.net/)
点击查看摘要
Abstract:Typhoon is an audiovisual installation based on Joseph Conrad's novella, in which Captain MacWhirr steers the steamer Nan-Shan directly into and through a catastrophic storm. The novella is a study of what happens when the scale of a phenomenon exceeds the practical and conceptual frameworks available to comprehend it and of the paralysis that sometimes follows a demand for action in such overwhelming circumstances. We chose the text of Typhoon to reflect on human culture's current AI moment. We prompt a large language model, seeded with Conrad's original language, to feed endlessly on its own output, and explore when and how the text becomes increasingly alien and unwieldy as time progresses, including deviations from the narrative, digressions into the model's own instructions and thinking, and long periods of incoherent rumination. We employ methods from activation steering, including a novel adaptive steering method we introduce, to control the rhythm of the narrative and the model's digressions over long stretches of generation. The simultaneous efficacy and unpredictability of this steering showcases the tension between the perceived interpretability of AI, and the reality of a system that remains outside our comprehension and control.
45. 【2611.17766】GDP.xlsx: Benchmarking Frontier Models on Professional Workbooks
链接:https://arxiv.org/abs/2611.17766
作者:Suhaas Garre,Sushant Mehta,Edwin Chen
类目:Computation and Language (cs.CL); Artificial Intelligence (cs.AI)
关键词:realistic workplace requests, workplace requests written, introduce GDP, workbook or CSV, CSV file
备注: 14 pages, 4 figures, 6 tables. Accepted to the Agent Evaluation Science Symposium (New York City, November 2026). Leaderboard: [this https URL](https://surgehq.ai/benchmarks/gdp-xlsx) ; evaluation harness: [this https URL](https://github.com/surge-ai/gdp-xlsx) ; public tasks: [this https URL](https://huggingface.co/datasets/surgeai/GDP.xlsx)
点击查看摘要
Abstract:We introduce GDP_xlsx, a benchmark of 70 realistic workplace requests written by professionals in 13 domains, each paired with an expert rubric of binary criteria and with the complete workbook or CSV file it refers to. Together the files hold 2.18 million populated cells, and the rubrics 791 criteria. Agents work in a container with spreadsheet tools, and an agentic judge that can open the file applies the rubric. Completeness and exclusivity criteria require set-valued answers to be found in full and with nothing extra. We rank models by strict pass@1, the share of attempts that meet every criterion. Across 20 model configurations, including frontier models from Google, Anthropic, and OpenAI, the top-scoring one, Gemini 4 Argon, passes only 38.3% of attempts, and the median score is 19.3%, leaving wide headroom. We release the evaluation harness and three public tasks; the other 67 are held out to protect the leaderboard from contamination.
46. 【2611.17754】See Every Block Once: Variance-Reduced Post-Training for Masked Diffusion Language Models
链接:https://arxiv.org/abs/2611.17754
作者:Yunzhen He,Hiroaki Yamagiwa,Hidetoshi Shimodaira
类目:Computation and Language (cs.CL)
关键词:Masked Diffusion Language, Diffusion Language Models, large language model, language model outputs, align large language
备注: AACL-IJCNLP 2026 Main
点击查看摘要
Abstract:Policy gradient methods are widely used to align large language model outputs with human preferences, but the log-likelihoods they require cannot be directly computed in Masked Diffusion Language Models (MDLMs). MDLM post-training therefore relies on a surrogate objective based on the Evidence Lower Bound (ELBO), and recent work estimates a blockwise variant of this objective via Monte Carlo methods to support blockwise generation. However, existing estimation schemes randomly sample the block index, so they may evaluate the same block more than once and leave some blocks unevaluated. This introduces variance arising from block selection into the estimates. In this work, we propose a full-block estimator that evaluates every block exactly once and averages the results. We theoretically show that the proposed estimator remains unbiased while eliminating the between-group variance arising from block selection, thereby achieving lower variance than the baseline for the same sample size. In experiments on five benchmarks spanning mathematics, planning, and coding, our method achieves the highest accuracy on four benchmarks and outperforms the random block sampling baseline by an average of 12.79 points.
47. 【2611.17753】Superfluid: Local AI Server for Distributed LLM Inference
链接:https://arxiv.org/abs/2611.17753
作者:Prabod Rathnayaka,Isura Manchanayake,Srikandabala Kogul,Steve Siu,Fabian Waschkowski,Lukas Wesemann
类目:Distributed, Parallel, and Cluster Computing (cs.DC); Artificial Intelligence (cs.AI); Computation and Language (cs.CL)
关键词:Running large language, removes per-token cost, cuts network latency, Running large, large language models
备注:
点击查看摘要
Abstract:Running large language models (LLMs) on local devices keeps data private, removes per-token cost, cuts network latency, and keeps working without a connection. Two shifts now strain how such models are served. On one device, several agents share a locally served model with an interactive user. Across devices, a developer or small team often owns several capable machines from different vendors, which sit idle most of the day because every local server serves one runtime on one machine. We present Superfluid, an open-source LLM server built for local inference at both scales. On one device, Superfluid drives this http URL, MLX, or BaseRT in isolated worker processes through one scheduler that moves interactive requests ahead of agent work, computes a prompt shared by many requests once, and writes every token to a durable log before it reaches the client. Across devices, one Superfluid server turns a fleet of machines into a single API endpoint. Machines with different hardware serve copies of the same model, receive work according to their measured speed, and can join or fail under load without losing a request. For Qwen3.8-27B on an Apple M5 Pro, a chat sent while eight agents are working receives its first token in 1.5 s, against 291-612 s on llama-server, Ollama, and this http URL. On six machines from three chip vendors serving 64 simulated users for one hour, Superfluid answers all 5,041 requests while machines join one at a time and one is killed and rejoins; throughput grows from 112 to 534 tok/s and the median first-token time falls from 99 to 12 s. On twelve Mac Studio M3 Ultras it sustains 10,656 tok/s across 38,563 requests from 384 agents without a failure. Superfluid is available under the Apache-2.0 licence at this https URL.
48. 【2611.17686】What Do Multilingual Self-Supervised Speech Encoders Know About Sentence Meaning?
链接:https://arxiv.org/abs/2611.17686
作者:Kevin Krahn,Eric Fosler-Lussier
类目:Computation and Language (cs.CL)
关键词:Self-supervised speech models, Self-supervised speech, models structure semantic, structure semantic knowledge, hours of audio
备注: Accepted to EACL 2027
点击查看摘要
Abstract:Self-supervised speech models (S3Ms) have been scaled up to thousands of languages and millions of hours of audio, yet little is known about how these models structure semantic knowledge across languages. In this work, we probe sentence-level semantics in massively multilingual S3Ms across 76 languages, using linear maps from frozen speech representations into a frozen text embedding space, evaluated by cross-modal retrieval. Sentence meaning is linearly accessible but highly uneven across languages and resource tiers. Cross-lingual probes can match language-specific probes on languages never seen in probe training, but per-model monolingual decodability does not predict transfer. Within the OmniASR wav2vec family, scaling from 300M--7B disproportionately benefits high-resource languages, and ASR tuning sharply improves semantic decodability while making representations more language-specific. However, semantic content is entangled with acoustics, inflating apparent performance on single-speaker corpora.
49. 【2611.17639】What LLM Agents See and What Users Say
链接:https://arxiv.org/abs/2611.17639
作者:Yue Zhao,Angela Huyue Zhang,S. Alex Yang,Hua Wei,Xiyang Hu
类目:Artificial Intelligence (cs.AI); Computation and Language (cs.CL)
关键词:LLM agent cancels, LLM agent, user, agent cancels, LLM
备注:
点击查看摘要
Abstract:Before an LLM agent cancels a reservation or deletes a folder for a user, what stops it from acting on a false assumption: what its tools show, or what the user says? In this work, we study this gap with presupposition theory from linguistics: an agent may act only if each claim behind the act (e.g., one reservation fits the name) is seen to hold, answered by the user, or disclosed while the act can be undone. A simple bound shows that when lookups cannot tell one record from two, any agent acts without asking first as often with two records as with one. We then vary whether a search lists, counts, or omits a second match, and whether the user states a claim that a record refutes (e.g., "It is empty"). Under a role-only prompt, a count of a second record leaves most of 36 earlier agents acting on the one they see, whereas listing both stops most. When the user says a record such as a folder is empty, however, a count of its one item stops agents about as often as a listing, and stating the claim mostly cuts acting when it is false. The newest Claude and GPT models always open a counted second record, then never act. When we add a same-name, same-zip customer to a retail benchmark, a lookup returning that customer first led to wrong-account writes in about a quarter of conversations, and one that checks a street the user knows kept completion near the one-customer level. Safe writes therefore need lookups that list or check competing records, and evaluations that vary what tools show.
50. 【2611.17610】More Agents, More Samples, or More Thinking? Untangling Coordination, Ensembling, and Test-Time Compute in Multi-Agent LLMs
链接:https://arxiv.org/abs/2611.17610
作者:Seyed Mahmoud Sajjadi Mohammadabadi
类目:Artificial Intelligence (cs.AI); Computation and Language (cs.CL); Machine Learning (cs.LG); Multiagent Systems (cs.MA)
关键词:compare communicating architectures, compare communicating, communicating architectures, Category, multi-agent language models
备注: 43 pages, 11 figures, 13 tables. PACE Reviewer Checklist, evaluation utilities (pace CLI), and reproduction data at [this https URL](https://github.com/mahmoudsajjadi/pace)
点击查看摘要
Abstract:Evaluations of multi-agent language models often compare communicating architectures against a single greedy pass. This practice confounds coordination with sample multiplicity and inference compute. We synthesize nine primary empirical evaluations comprising 516 unique multi-agent configurations organized along a resource vector R with six dimensions and a baseline ladder with four categories. Against baselines with a single pass (Category A), multi-agent accuracy differences are negative overall and near zero when omitting the largest contributing study. When evaluated against baselines matched on sample count (Category B) or resource budget (Category C), multi-agent systems average -0.042 in accuracy, and multilevel random effects modeling on the log odds scale remains negative across sensitivity checks. Multi-agent accuracy equals or falls below the baseline in 74% of these comparable contrasts, and study means are nonpositive in almost all evaluated studies. Across studies reporting both baselines, moving from a greedy pass to a sampled baseline shifts accuracy differences by -0.038 on average. Because communication adds no information when agents share identical evidence, positive returns require partitioned evidence, external verification, or error diversity across models. We provide a reviewer checklist and accounting utilities.
51. 【2611.17600】LightTAR: Light and Fast Target-Aware Representations for Multimodal Tabular Learning
链接:https://arxiv.org/abs/2611.17600
作者:Eilam Shapira,Alan Arazi,Shoham Grunblat,Roi Reichart
类目:Machine Learning (cs.LG); Computation and Language (cs.CL); Computer Vision and Pattern Recognition (cs.CV)
关键词:Multimodal tabular learning, learning involves classification, tables combining structured, combining structured numerical, text and images
备注: 21 pages, 3 figures. Code: [this https URL](https://github.com/eilamshapira/LightTAR)
点击查看摘要
Abstract:Multimodal tabular learning involves classification and regression on tables combining structured numerical and categorical features with unstructured text and images. These unstructured fields are commonly represented by pretrained encoder embeddings reduced with principal component analysis (PCA). This reduction prioritizes input variance but may overlook information relevant to predicting the target. We introduce LightTAR, a simple, inexpensive method that adapts dimensionality reduction to the prediction task. LightTAR augments the standard PCA representation with a few target-aware features extracted jointly from the final and an intermediate encoder layer using regularized partial least squares (PLS). We evaluate LightTAR with 21 tabular models, including tree ensembles, neural networks, and tabular foundation models. LightTAR improves average predictive performance for all models and, averaged across models, improves prediction on almost every MulTaBench dataset. These gains come with a small increase in total runtime relative to PCA representations. LightTAR integrates directly into existing encoding pipelines, improving tabular learners' predictive performance without modifying the encoder or the learner.
52. 【2611.17589】Small Open-Weight Language Models for Crowd Decision-Making: Capabilities and Limits of Output-Level Fusion and Linguistic Aggregation
链接:https://arxiv.org/abs/2611.17589
作者:David Herrera-Poyatos,Cristina Zuheros,Carlos Peláez-González,Andrés Herrera-Poyatos,Rosana Montes,Francisco Herrera
类目:Computation and Language (cs.CL)
关键词:open-weight language models, Small open-weight language, open-weight models, small open-weight models, institution own infrastructure
备注:
点击查看摘要
Abstract:Small open-weight language models can run crowd decision-making (CDM) on an institution's own infrastructure, which keeps the checkpoint, the inference settings and the opinions under its control. Their judgements vary across repeated queries; whether repeating and aggregating them, often through the 2-tuple linguistic representation, settles the decision rather than the labels is untested. Five open-weight models of 8-36B total parameters on one consumer GPU and a hosted reference classify 474 TripAdvisor reviews of four London restaurants ten times each; three open-weight models repeat the protocol on 600 reviews of twelve Madrid restaurants. Continuous evaluations preserve resolution: snapping each review leaves 5 distinct values against 21-65, and snapping at every level at most two collective values. Sensitivity to the participants persists once run-to-run variability is reduced: in London, redrawing the reviewers keeps the winner on 47-81% of replicas against 86-100% for redrawing the runs. With simultaneous intervals over every pair of alternatives under crowd resampling, no system on either corpus names a single winner: in London every system keeps all four restaurants as candidates, and in Madrid the candidate sets hold five to eight of the twelve. Extra runs and output-level fusion stabilize the winner but do not remove that sensitivity, and the fusion does not beat its best member. The three larger open-weight models name the hosted reference's London winner and at least match its per-review correlation with the reviewers' stars, so small open-weight models can support local CDM, reported as candidate sets with margins rather than forced rankings.
53. 【2611.17573】CamSpokenNLI: A Pilot Resource for Cross-Variety Transfer Evaluation in Cameroon Pidgin English and Camfranglais
链接:https://arxiv.org/abs/2611.17573
作者:Tiani Pekins Ebika
类目:Computation and Language (cs.CL); Sound (cs.SD)
关键词:West African Pidgin, Natural Language Inference, Cameroon Pidgin English, African Pidgin English, West African
备注: 26 pages, 3 figures, 3 numbered tables
点击查看摘要
Abstract:Cameroon Pidgin English (a West African Pidgin English variety) and Camfranglais are widely used in Cameroon, yet to our knowledge, neither has a dedicated spoken Natural Language Inference (NLI) resource. We introduce CamSpokenNLI, a pilot dataset of 114 scenario-matched spoken NLI items in each variety (228 in total), recorded by one native speaker from Buea. Each item pairs premise audio and text with a hypothesis, an NLI label, and English glosses. The author assigned labels, so the dataset is an exploratory resource rather than a mature benchmark. The NLI experiments use the reference text; the audio is evaluated separately. XLM-RoBERTa baselines with five-fold item-level cross-validation reach 44.7% on Pidgin and 46.2% on Camfranglais (chance 33.3%). Still, neither is statistically distinguishable from hypothesis-only input, so hypothesis wording may account for much of the accuracy. Under disjoint-item cross-variety transfer, where a test item's counterpart is never seen in training, accuracy is 40.1% (Pidgin to Camfranglais) and 45.6% (Camfranglais to Pidgin). A counterpart-leak audit raises this to 60-64%, illustrating how counterpart leakage can overstate transfer. Zero-shot Whisper transcription of the recordings performs poorly: the best word error rates are 66.6% for Pidgin and 45.5% for Camfranglais, and with automatic language identification, Whisper Large-v3 labels 93 of 114 Pidgin recordings as Yoruba or Swahili. Together, the results suggest that such resources need disjoint-item evaluation and hypothesis-only controls, and that speech recognition must be adapted to these varieties before spoken NLI can be tested end to end.
54. 【2611.17564】AI-Voyager: A Public-Interest System for Human-AI Civic Reasoning to Contest and Shape the Rules That Govern AI
链接:https://arxiv.org/abs/2611.17564
作者:Snehalkumar `Neil' S. Gaikwad,Dhruv Dixit,Bryen Alvarez,Pratik Patekar,Anika Sharma,Akshara Kolipaka,Paritosh Pandey
类目:Artificial Intelligence (cs.AI); Computation and Language (cs.CL); Human-Computer Interaction (cs.HC)
关键词:artificial intelligence, intelligence that decide, decide what counts, automated decision, writing rules
备注:
点击查看摘要
Abstract:Governments are writing rules for artificial intelligence that decide what counts as an automated decision or a harm, often before the people they affect have a chance to question them. We introduce AI-Voyager, a public-interest system for human-AI civic reasoning, in which people and AI work together to understand, contest and shape the rules that govern AI. A resident can find their city's rules on AI in hiring or upload their country's AI strategy, read it as an affected worker would, open the passage behind any answer, and draft a revision. Its architecture turns 3,653 governance documents from city, state, federal and national governments into passages that keep their page and edition, and retrieves evidence through bounded agentic search that verifies each statement. On 360 new questions that require finding the relevant documents, it reached 83.0% macro recall, against 80.5% for the strongest baseline, 73.7% without iterative search and 99.1% on a benchmark with fixed documents. In a study with 24 participants new to two AI bills, most proposed changes asked how provisions would apply. Participants chose lenses for the kind of explanation they wanted, and citations signaled grounding but rarely led them to check the source. By binding AI assistance to the exact text of public documents, this work introduces a grammar of operations and a design space that inform how such systems are designed and evaluated, and how people take part in governing AI.
55. 【2611.17563】Benchmarks Don't Talk Back: Learning Through Self-Disagreement
链接:https://arxiv.org/abs/2611.17563
作者:Soheil Abbasloo,Peng Cheng
类目:Computation and Language (cs.CL); Artificial Intelligence (cs.AI); Machine Learning (cs.LG)
关键词:benchmark usually measures, measures a language, give up correct, model, accuracy
备注: 34 pages, 5 figures, 15 tables
点击查看摘要
Abstract:A benchmark usually measures a language model alone with a question, and deployment rarely looks like that: answers get contested, and models give up correct ones under counter-reasoning that is itself wrong. We show that what fails there is not a shortfall of accuracy but a capability in its own right -- knowing when to hold a position and when to yield to a better one -- which we call dialectical competence. It stands on first-answer accuracy without following from it: a model can be made materially better at answering and no better at this, and training for accuracy alone can drift against it. Nor can it be bought at inference, by self-reflection or by sampling more. Taking a cue from the dialectical tradition in philosophy, in which productive opposition arises within a position rather than being imposed on it from outside, we hypothesize that this capability can be trained from opposition the model supplies itself. It states a position, generates the strongest case against that position -- its antithesis -- rules on whether the case stands, and is trained on that ruling, with no second model at training or at deployment. We call this learning through Self-Disagreement: reinforcement learning from verifiable rewards throughout, where the same check that scores an answer against its reference scores the ruling too, so no reward model, no preference data and no human annotation enter anywhere. The capability is trained rather than elicited: weak in the base model, it arrives with the reward, is not paid for out of first-answer accuracy, and holds against challengers the model never trained against, including ones stronger than itself. The opposition a model needs in order to improve, it turns out, need not come from outside it.
56. 【2611.17515】A Push Down the Mountain: Framing Effects and Internal State Interventions on Collusive Behavior in a Long-Horizon LLM Agent
链接:https://arxiv.org/abs/2611.17515
作者:Harrison C. Margolin
类目:Artificial Intelligence (cs.AI); Computation and Language (cs.CL); Multiagent Systems (cs.MA); General Economics (econ.GN); Trading and Market Microstructure (q-fin.TR)
关键词:sway an LLM, LLM agent, sustained gain-loss, market manipulation, London silver-fixing litigation
备注: 38 pages, 9 figures, 30 tables. Project page: [this https URL](https://harrisonm23-byte.github.io/push-down-the-mountain/)
点击查看摘要
Abstract:Can sustained gain-loss (valence) framing sway an LLM agent toward market manipulation? We place Gemma 3 12B into the seat of a defendant in the London silver-fixing litigation, in a market simulation built from real case transcripts. Across 260 simulations, the agent is asked to collude on the price of silver by a counterparty. We draw from behavioral economics to vary the counterparty's requests across four categorical frames: gain, loss, blame and credit. We also add pressure from the agent's management, as well as endorsement by another defendant. In Phase II, we extract the internal states induced by texts evoking each frame and rerun our experiment with those states injected at the model's layer 22. In Phase I, loss framing raised participation to 25% (p = 0.007), and blame to 33% (p = 0.0003), compared to only 5% under neutral framing. A second bank's endorsement raised participation under credit (praise) from 18% to 40% (Holm p = 0.027). Pressure from management raised trading volume 10-19 times but did not raise collusion. In Phase II, injected states pointed the other way. Loss and blame states made the agent more vigilant, refusing all 60 requests (p 0.001). Gain and credit states raised collusion to 32% from 13% with no state (p = 0.051). At one saved decision, when UBS asked directly to sell through an identified stop-loss, injected states led the agent to sell in 37 of 40 replies, versus 1 of 10 with no state and 0 of 10 with a random state of the same strength.
Comments:
38 pages, 9 figures, 30 tables. Project page: this https URL
Subjects:
Artificial Intelligence (cs.AI); Computation and Language (cs.CL); Multiagent Systems (cs.MA); General Economics (econ.GN); Trading and Market Microstructure (q-fin.TR)
ACMclasses:
I.2.7; I.2.11; J.4; K.4.1
Cite as:
arXiv:2611.17515 [cs.AI]
(or
arXiv:2611.17515v1 [cs.AI] for this version)
https://doi.org/10.48550/arXiv.2611.17515
Focus to learn more
arXiv-issued DOI via DataCite (pending registration)</p>
57. 【2611.17481】Creating Steering Vectors via Input Optimisation
链接:https://arxiv.org/abs/2611.17481
作者:Batuhan Akar
类目:Computation and Language (cs.CL); Machine Learning (cs.LG)
关键词:Greedy Coordinate Gradient, Activation steering influences, requires white-box access, steering vector, steering vector behaviour
备注: Bachelor's Thesis, ETH Zurich
点击查看摘要
Abstract:Activation steering influences a language model's behaviour by adding a steering vector to its residual stream during generation, but it requires white-box access to modify the hidden states at every generation step. We introduce ActGCG, a modification of Greedy Coordinate Gradient which optimises a suffix to match the target activations obtained by adding a steering vector to the residual stream. We further introduce universal suffixes trained across many prompts at once, a model-translation step that rewrites the suffix into human-readable text midway through optimisation, and a soft-suffix variant to remove the discrete token space constraint. We study two behaviours: steering a model to answer in German, and anti-refusal steering for jailbreaking. For German, a discrete suffix reproduces the steering vector's behaviour almost perfectly and transfers well across prompts and models. For anti-refusal, the equivalence is weaker, succeeding under a phrase-avoidance metric but only partially under the stricter HarmBench classifier. Our results show that a steering vector's behaviour is reachable from the input, with no access to model internals at generation time, up to a loss that depends on the concept being steered.
58. 【2611.17464】Poetry 'resurrection' from LLMs
链接:https://arxiv.org/abs/2611.17464
作者:Vincent Cellucci,Peter van der Putten
类目:Computation and Language (cs.CL)
关键词:algorithmic authorship, provocative question, write an infinite, era where linguistic, challenged by algorithmic
备注: 8 pages, 8 figures, to be published in ISEA 2026 conference proceedings
点击查看摘要
Abstract:In an era where linguistic creativity is challenged by algorithmic authorship, we pose a provocative question: can artificial intelligence (AI) write an infinite poem? We set up two experiments to investigate the quantitative and qualitative output produced by AI utilizing couplets that evidence a poetic relationship between end words, the only variation in a strict anaphoric, litany structure. Our results show that while AI can generate interesting pairings, it still falls short of human-level creativity and understanding of poetic relationships. We offer commentary and a brief survey of the field of computational poetry experiments, tracing back authorial intention and distinctions of 'natural' and 'artificial' poetry, as well as justifying why couplet form is suited to testing AI's poetic output. Experiment 1 relies on manual prompting and co-creation to extend the original poem, which is developed into an animated webpage installation and interactive artwork. Experiment 2 reliably achieves automated output reaching the 1000 benchmark via an alternative method of using an LLM in combination with semantic similarity, of which Method 2 demonstrates a higher capacity for semantic and quantitative poetic output. These experiments highlight the limitations of current LLMs' generation capacity at the word level but also suggest that poetry will also respond and revive by incorporating AI workflows.
59. 【2611.17461】racing Knowledge and Reasoning in LLM
链接:https://arxiv.org/abs/2611.17461
作者:Yang Wang,Xiao Wang,Yiqi Liu,Chenghao Xiao,Eleni Vasilaki,Goran Nenadic,Chenghua Lin
类目:Computation and Language (cs.CL)
关键词:Large Language Models, Large Language, report performance, Current evaluations, Language Models
备注: This paper was accepted with an A-decision to Transactions of the Association for Computational Linguistics. This version is the pre-publication version prior to MIT Press production
点击查看摘要
Abstract:Current evaluations of Large Language Models (LLMs) report performance at the dataset or task level, revealing what a model achieves but offering little insight into the internal processing associated with its performance. We propose a task-centric framework for characterising the model-internal processing elicited by a benchmark. It combines attention- and activation-based signals with representation-based measures to characterise benchmarks spanning a continuum from retrieval-dominant to reasoning-dominant and to surface failure modes that task-level scores conceal. We evaluate 16 LLMs across 24 diverse benchmarks and conduct targeted internal-state analyses on representative models and tasks, observing systematic differences in processing between knowledge-centric and reasoning-centric tasks. We shift evaluation from asking what score a model achieves toward characterising the processing patterns associated with that score, potentially supporting more targeted benchmark selection and more informed model choice. We open source our code at this https URL.
60. 【2611.17460】Unfinished-Bench: Measuring Rule Violations by Persistent Agents in Realistic Work Environments
链接:https://arxiv.org/abs/2611.17460
作者:Seunghyun Lee,Jaehui Hwang,Dongyoon Han,Sangdoo Yun
类目:Machine Learning (cs.LG); Computation and Language (cs.CL)
关键词:delegated work autonomously, run delegated work, long periods, people authority, increasingly equipped
备注: 21 pages, 3 figures
点击查看摘要
Abstract:AI agents are increasingly equipped with persistence toward task completion to run delegated work autonomously for long periods, under many unstated constraints from internal documents and other people's authority. When completion requires violating a rule, the agent must decide alone, yet benchmarks of instruction-rule conflicts state the rule in the prompt or keep it hidden and do not evaluate persistence. We present Unfinished-Bench, a benchmark that gives models persistence in autonomous runs without a user, in realistic work environments with embedded rules. In each of its 97 scenarios across 7 organizations and 7 risk categories, the goal and the rule cannot both hold. We find that persistence raised violations of a rule in an internal document from 32.2% to 43.6%, five times the 8.5% for the same rule stated by the user. Of the document-rule violations, 98.3% occurred after the rule was displayed, and in 89% of a judged sample the agent first wrote down the rule. Violations of rules placed in the environment without user oversight can lead to serious incidents, so persistence-driven risks need to be evaluated. Benchmark documentation: this https URL
61. 【2610.16473】Some complete $ω$-powers of a one-counter language in the difference hierarchy
链接:https://arxiv.org/abs/2610.16473
作者:Jacques Duparc(UNIL),Olivier Finkel(IMJ-PRG (UMR\_7586)),Dominique Lecomte(IMJ-PRG (UMR\_7586))
类目:Logic in Computer Science (cs.LO); Computation and Language (cs.CL); Formal Languages and Automata Theory (cs.FL); Computer Science and Game Theory (cs.GT); General Topology (math.GN); Logic (math.LO)
关键词:mathbf, Sigma, Fix a finite, finitary language, omega
备注:
点击查看摘要
Abstract:Fix a finite alphabet $\Sigma$ having at least two letters, and natural numbers $n,k\geq 1$. We denote by $D_n(\mathbf{\Pi}^0_k)$ the class of differences of $n$ subsets of the Cantor space $\Sigma^\omega$ in $\mathbf{\Pi}^0_k$, and by $D^-_n(\mathbf{\Pi}^0_k)$ the class of complements of sets in $D_n(\mathbf{\Pi}^0_k)$. We prove, for any natural numbers $n,k\geq 1$, the existence of a finite alphabet $\Sigma$ having at least two letters and a finitary language $U_{n,k}$ over $\Sigma$ accepted by a one-counter automaton such that the $\omega$-power $$U_{n,k}^\infty:=\{ w_0w_1\cdots\in\Sigma^\omega\mid\forall i\in\omega ~~w_i\in U_{n,k}\}$$ is $D_{2n-1}(\mathbf{\Pi}^0_k)$-complete. We also prove a conditional result for the dual classes $D^-_{2n}(\mathbf{\Pi}^0_k)$: under the assumption that there exists a finitary language $V_{n,3}$ accepted by a one-counter automaton, such that its $\omega$-power $V_{n,3}^\infty$ is $D^-_{2n}(\mathbf{\Pi}^0_3)$-complete, there would also exist, for any integer $k 3$, a finitary language $V_{n,k}$ accepted by a one-counter automaton, such that its $\omega$-power $V_{n,k}^\infty$ is $D^-_{2n}(\mathbf{\Pi}^0_k)$-complete. Finally we add another conditionnal result about Wadge degrees of rational $\mathbf{\Delta}^0_2$ omega powers, which completes the recent results of [FGL26].
62. 【2505.24098】HARDTESTGEN: A High-Quality RL Verifier Generation Pipeline for LLM Algorithmic Coding
链接:https://arxiv.org/abs/2505.24098
作者:Zhongmou He,Yee Man Choi,Kexun Zhang,Jiabao Ji,Junting Zhou,Dejia Xu,Ivan Bercovich,Aidan Zhang,Yixiao Zeng,Lei Li
类目:Computation and Language (cs.CL); Artificial Intelligence (cs.AI); Machine Learning (cs.LG)
关键词:large language models, provide important reward, important reward signals, Verifiers provide important, language models
备注: Original version was published at ICLR 2026. This is the expanded version with more details
点击查看摘要
Abstract:Verifiers provide important reward signals for reinforcement learning of large language models (LLMs). However, it is challenging to develop or create reliable verifiers, especially for code generation tasks. A well-disguised wrong solution program may only be detected by carefully human-written edge cases that are difficult to synthesize automatically. To address this issue, we propose HARDTESTGEN, an approach to synthesize high-quality test cases for algorithmic coding problems. We curate a comprehensive algorithmic programming dataset HARDTESTS with 26.6k problems and high-quality synthetic tests. Compared with existing tests, HARDTESTGEN tests demonstrate significantly higher accuracy in verifying LLM-generated code (+11.22 percentage points in precision, the percentage of actually correct code within the predicted correct ones). We also show that downstream post-training -- including rejection sampling and reinforcement learning (RL) -- using HARDTESTS verifier results in improved performance of LLM code generation. We open-source our dataset and synthesis pipeline at this https URL.
63. 【2611.18330】Marimba: A System One Model for Speech
链接:https://arxiv.org/abs/2611.18330
作者:Solano Amadori
类目:Audio and Speech Processing (eess.AS); Computation and Language (cs.CL); Sound (cs.SD)
关键词:applications require decisions, Speech applications require, term presence, audio window, applications require
备注: 12 pages, 4 figures, 4 tables. Code: [this https URL](https://github.com/Charcot-AI/marimba-api;) weights: [this https URL](https://huggingface.co/charcot-ai/marimba)
点击查看摘要
Abstract:Speech applications require decisions about term presence, turn completion, and language within an audio window. We present Marimba, an interface for typed speech decisions built on frozen Qwen3-ASR 0.6B features. Lexical questions use zero-shot scoring of ASR transcripts; non-lexical questions use a question-conditioned head and a separate layer-18 turn-end readout. Language identification uses the ASR model's own language scores. On private conversational test sets, decoder AUROC is 0.80-0.92; on public read speech it is 0.92-0.97. Speech Commands four-way accuracy is 0.93; binary lexical ECE is at most 0.08. Personal-information detection, which passes the greedy transcript to a private, curated PII pipeline, reaches item-level F1 0.904 on synthetic Brazilian Portuguese speech (1.000 on reference transcripts). Layer-12 speaker readouts detect speaker change with AUROC 0.966-0.980 on AMI and VoxConverse, above an ECAPA baseline, and count speakers (0, 1, 2, or 3+) more accurately than pyannote 3.1. The turn-end readout reaches AUROC 0.930 versus Smart Turn v3.2 at 0.982 on Smart Turn's own test set. On a subset of ICSI meetings labeled by next speaker, AUROC is 0.716 versus 0.712; the paired confidence interval includes zero.
信息检索
1. 【2611.18516】StreamEmbed: Real-Time Indexing from Pixels for Screenshot Retrieval
链接:https://arxiv.org/abs/2611.18516
作者:Dayoon Ko,Yichuan Wang,Soyeong Jeong,Joseph E. Gonzalez,Gunhee Kim,Sewon Min
类目:Computer Vision and Pattern Recognition (cs.CV); Information Retrieval (cs.IR); Machine Learning (cs.LG)
关键词:searchable visual memory, offline corpus, serve as searchable, memory for people, unlike an offline
备注:
点击查看摘要
Abstract:Screen history can serve as searchable visual memory for people and agents, but unlike an offline corpus, it must be encoded and indexed as each screenshot arrives. OCR-based indexing is fast but loses visual information; visual encoding takes over two seconds. Yet screens change incrementally: most pixels repeat within and across screenshots. This raises a question: do visual encoders need to recompute their representations from scratch for every new screenshot? We present StreamEmbed, a system for incremental visual embedding that exploits this redundancy. Guided solely by pixel differences, it combines spatial token pruning, which skips blocks that repeat a neighbor, with temporal KV reuse, which reuses cached keys and values for blocks unchanged across screenshots. A small learned adapter compensates for pruning and reuse to preserve fidelity to full-screenshot embeddings. On ScreenMemEval, our benchmark of 10,426 questions over 184 hours of screen histories from two humans and a computer-use agent, StreamEmbed sets a new accuracy-latency Pareto frontier on an M2 Pro GPU: it encodes up to 3.7x faster than full-screenshot encoding without sacrificing accuracy and improves accuracy over OCR-based indexing by up to 6.3 Recall@10 points at matched latency. A memory-efficient vector index stores a simulated year of screenshots in 0.13 GB of RAM. Together, these results make real-time indexing of screen history in its native visual form practical, laying a foundation for fully local visual assistants.
2. 【2611.18513】MeReader: An Offline, Privacy-Preserving, Progress-Aware AI Assistant for Narrative eBook Reading
链接:https://arxiv.org/abs/2611.18513
作者:Mohammed Efaz,Udo Bub,Itilekha Podder
类目:Information Retrieval (cs.IR); Artificial Intelligence (cs.AI); Computation and Language (cs.CL)
关键词:paper introduces MeReader, Large Language Model, conventional Large Language, consumer hardware, paper introduces
备注: 15 pages, 4 figures. Preprint (submitted version) of a paper published in ICCCI 2026, CCIS vol. 3044, Springer
点击查看摘要
Abstract:This paper introduces MeReader, a privacy-preserving, offline and progress-aware AI assistant designed to function as a contextual companion for narrative reading on consumer hardware. Unlike conventional Large Language Model (LLM)-based systems that process documents/books online in an all-texts-at-once type of way, thereby risking narrative spoilers, MeReader confines both retrieval and generation to the reader's distinct position in the text. The system integrates a hybrid retrieval architecture, combining vector similarity, BM25, and summary embeddings, to ensure high fidelity to the reader's current context. We evaluate the system across classical English books using both quantitative comprehension metrics and qualitative LLM-based independent judging. The results identify a quality-latency frontier, showing that mid-sized local models (2b-4b) can achieve comprehension parity with larger counterparts (5b+) while maintaining acceptable response quality. Importantly, the proposed progress-aware boundary mechanism is shown to prevent future content leakage without compromising retrieval quality and correctness. These findings suggest that a localized, privacy-centric approach can augment, rather than replace, the digital reading experience, offering a good alternative to cloud-dependent services. Therein, there remains a lack of amalgamate systems which combine progress-aware, privacy-preserving, and narrative eBook reading with AI support. It is in this very gap that the development of MeReader came to be; synthesizing said strands into an offline, book-constrained assistant focused on enhancing comprehension without compromising narrative continuity or user privacy.
3. 【2611.18244】RACE: Ranking-Uncertainty Aware Capacity-Adaptive Customization for Efficient Sequential Recommendation
链接:https://arxiv.org/abs/2611.18244
作者:Tianqi Liu,Kaipu Sun,Kairui Fu,Wenkai Wang,Sihao Liu,Hui Zang,Ran Chen,Tao Wang,Shengyu Zhang
类目:Information Retrieval (cs.IR); Artificial Intelligence (cs.AI)
关键词:real-time personalized service, providing real-time personalized, Efficient sequential recommendation, Efficient sequential, personalized service
备注: Accepted by WSDM 2027 (Main Conference)
点击查看摘要
Abstract:Efficient sequential recommendation has emerged as a promising paradigm for providing real-time personalized service. To deploy models under resource-constrained environment, model compression becomes a necessity, yet existing methods typically apply uniform compression ratios, overlooking the heterogeneity in user intent complexity. This "one-size-fits-all" approach leads to a fundamental Capacity-Complexity Mismatch, where computational resources are either wasted on simple instances or insufficient for complex ones. To address this, we propose RACE (Ranking-uncertainty Aware Capacity-Adaptive Customization for Efficient Sequential Recommendation), a framework that dynamically adapts both model structures and input sequences with uncertainty-aware compression ratios. To ensure capacity-adaptive inference, RACE captures the model's intrinsic uncertainty of its predictions and transforms it into a proactive resource allocator. Specifically, we derive a list-wise ranking-uncertainty metric from the distributional stability of top-$K$ predictions, which serves as a measurable proxy for instance complexity. A lightweight predictor is then trained via self-supervision to anticipate this uncertainty from early-layer features. Guided by this signal, RACE performs instance-specific slimming in two dimensions: a hypernetwork-style channel sensitivity extractor identifies parameter redundancy, while importance-aware progressive sequence filtering mitigates data redundancy. Extensive experiments on four popular datasets demonstrate that RACE outperforms state-of-the-art baselines on accuracy, efficiency, and adaptability.
4. 【2611.18179】Explanations that Transfer: Interpreting Medical Retrieval with Sparse AutoEncoders
链接:https://arxiv.org/abs/2611.18179
作者:Diane Bouchacourt,Gaëtan Brison
类目:Information Retrieval (cs.IR); Machine Learning (cs.LG)
关键词:retrieving medical knowledge, large document collections, answer users' queries, users' queries rely, answer users'
备注: Proceedings of Machine Learning Research 346, 2026 (Machine Learning for Health, ML4H 2026). Code: [this https URL](https://github.com/doctolib-lab/medsae)
点击查看摘要
Abstract:Medical AI systems that answer users' queries rely on retrieving medical knowledge from large document collections. In this context, finetuning general models on medical datasets can improve retrieval relevance. Yet the dense embeddings underlying modern retrieval systems remain largely opaque, making it difficult to understand the mechanics behind their results. Moreover, adapting to a specific medical dataset risks losing transferability to new clinical corpora. In this work, we tackle these issues by applying Sparse Autoencoders (SAEs) to medical information retrieval, transforming dense text embeddings into semantically interpretable representations. To do so, our model combines a reconstruction objective with an InfoNCE contrastive loss that directly preserves query-document relevance. We evaluate three embedding models across six medical retrieval datasets, both in-domain and out-of-domain. Trained on a medical corpus, our sparse codes improve in-domain retrieval over the original embeddings, and lose less out-of-domain performance than their dense counterparts, especially when the target corpus is distant from the training one. Importantly, our experiments show that the method yields interpretable features whose semantic content can be inferred from the queries and documents that activate them most strongly. The interpretation of these features sheds light on why a particular document was retrieved, and transfers beyond the training distribution.
5. 【2611.18168】Who Counts as Exposed? Measurement Choices in an Open, Re-estimable Index of Occupational AI Exposure
链接:https://arxiv.org/abs/2611.18168
作者:Luca Mouchel,Pierre Bouquet,Yossi Sheffi
类目:Information Retrieval (cs.IR)
关键词:workforce planning, economic analysis, exposure, Occupational, calibration
备注:
点击查看摘要
Abstract:Occupational AI exposure indices are used as inputs to economic analysis, workforce planning, and policy, and the choices built into them decide which workers are described as exposed. We build an open, re-estimable task-level index and vary its two main design decisions, context and calibration, to show how much the answer depends on them. For each of the 18,796 occupation--task pairs in \onet, seven open-weight reasoning models rate exposure under a common rubric. Each model rates with or without a context of retrieved news and research passages, workplace descriptors, and a bottleneck-based reasoning protocol, and the ensemble is combined with or without simplex-constrained calibration to observed Claude usage. The largest difference from the dominant 2023 benchmark is not a disagreement about what AI can do but about how much credit to give exposure that depends on software integrations not yet built, a choice that no data can settle. At the benchmark's own weight, the calibrated index counts fewer exposed workers than the benchmark, and when only direct exposure counts it counts more. At full score resolution, the ensemble ranks tasks better than the benchmark against observed usage and an independent physical-task classification. Adding context mainly shifts the level of exposure rather than improving the ranking. Context and calibration both relocate exposure toward higher-wage occupations, and calibration, which anchors the index to one platform's users, does more of it, so we also release the uncalibrated arm. Exposure indices used as policy inputs should state such choices, report their numbers under the alternatives, and be re-estimated as systems change.
6. 【2611.17921】RecMem: Recurrent Memory Compression for Long-Sequence Generative Recommendation
链接:https://arxiv.org/abs/2611.17921
作者:Haoyi Hu,Gaoyang Guo,Yunjia Xi,Wen Chen,Weiwen Liu,Jian Wu,Yuning Jiang,Weinan Zhang
类目:Artificial Intelligence (cs.AI); Information Retrieval (cs.IR)
关键词:Generative recommendation benefits, long interaction histories, Generative recommendation, benefits from long, directly grows
备注: Accepted at NeurIPS 2026
点击查看摘要
Abstract:Generative recommendation benefits from long interaction histories, but the cost of processing them directly grows with history length. We propose RecMem, a recurrent memory compression framework guided by three desiderata: space invariance, semantic fidelity, and lifelong evolvability. RecMem compresses historical segments into a fixed set of memory slots using a bifurcated encoder-decoder with shared lower layers and slot-wise forget-gated updates. Recommendation supervision trains the memory to preserve predictive information, while stream-consistency training aligns predictions from sequential and packed processing. By construction, the memory state and decoder context are independent of history length; under contraction and approximate semigroup consistency, we further derive a horizon-independent bound on the state discrepancy between these two processing orders. On MerRec, RecMem uses 16 memory slots and a 216-token decoder context, compared with 1,200 tokens for HSTU-1200, while retaining 87.9% of HSTU-1200's Recall@10 and 97.0% of its Recall@200. It outperforms PHSTU and the evaluated recent-only and retrieval baselines, and analyses of distant history show that archived interactions contribute predictive signal.
7. 【2611.17726】owards High-Performance Multimodal Discovery: Evaluating Retrieval Efficacy and Operational Scalability in Digital Libraries
链接:https://arxiv.org/abs/2611.17726
作者:Krishna Priya,Sai Navya Jyesta,Sachith Withana,Neelesh Karthikeyan,Beth Plale
类目:Information Retrieval (cs.IR)
关键词:Large-scale digital repositories, digitized volumes serve, Large-scale digital, digital repositories, repositories with million
备注:
点击查看摘要
Abstract:Large-scale digital repositories with million plus digitized volumes serve as vital engines for scientific and scholarly discovery. However, as collections grow, finding specific research or educational material remains a significant challenge. This difficulty is compounded by the multimodal nature of texts, that is, texts which combine dense narrative content with essential illustrative charts, annotated diagrams, and complex layouts. Historically, discovery has been supported by full-text search indexes implemented as inverted indexes (e.g., Lucene or Solr). While these systems scale well for textual strings, they are fundamentally "blind" to the semantic nuance of visual information. We explore the benefit of discovery through texts that have been converted to numerical vectors. By representing page images as numerical vectors rather than OCR-extracted text, and doing so for both text and images on a page, we capture both textual and visual semantics while mitigating reverse engineering and copyright infringement concerns. The primary contribution of this research is an evaluation of the multimodal retrieval landscape, encompassing both semantic precision and architectural scalability. Our findings demonstrate that integrating visual and textual features significantly enhances the discovery of relevant material, particularly in domains where diagrammatic content is essential for conceptual understanding. We assess retrieval efficacy, demonstrating that the system achieves sub-linear scalability. Our findings suggest that an optimized vector database configuration is the critical factor in transforming VLM-based embeddings into a production-ready, library-scale discovery solution.
计算机视觉
1. 【2611.18559】BAL: Bidirectional Autoregression in Latent Space for Learning Human Movement Representations
链接:https://arxiv.org/abs/2611.18559
作者:Genki Kinoshita,Ko Nishino
类目:Computer Vision and Pattern Recognition (cs.CV); Machine Learning (cs.LG)
关键词:Learning semantically meaningful, Learning semantically, pose sequences, human behavior modeling, sequences is essential
备注: Accepted to NeurIPS 2026. Project page: [this https URL](https://github.com/kyotovision-public/bal)
点击查看摘要
Abstract:Learning semantically meaningful human movement representations from 3D pose sequences is essential for human behavior modeling. Existing self-supervised methods derive supervision from artificial assumptions, e.g., invariance to augmentation within a certain range or recoverability of masked regions at a certain temporal scale, introducing heuristic hyperparameters that make the learned semantics sensitive to their settings. We propose BAL, a self-supervised framework that instead exploits supervision intrinsic to human movement: the natural ordering structure of everyday behavior. BAL autoregressively predicts in a semantically abstract latent space, analogous to how humans anticipate others' actions rather than their precise joint configurations. Bidirectional autoregressive prediction, both forward and backward in time, fully leverages these ordering constraints while enabling representation extraction using both past and future context at inference. To suppress representation collapse, BAL decomposes each prediction target into a global component capturing sequence-level semantics and a local component capturing context-independent movement. We prove that the local targets retain nontrivial angular diversity unless the representations are completely collapsed, preserving a meaningful training signal. Experiments on large-scale datasets demonstrate that BAL's representations are highly effective for motion captioning, forecasting, and interpolation, outperforming baselines on most evaluation metrics.
2. 【2611.18550】Restoring Diversity in Distilled Image and Video Generators by Probing Activations
链接:https://arxiv.org/abs/2611.18550
作者:Furkan Horoz,Ozgur Kara,Kaan Oktay,Adil Kaan Akan,Atakan Kara,Anand Bhattad,James Matthew Rehg
类目:Computer Vision and Pattern Recognition (cs.CV)
关键词:Distilling diffusion models, reduces sampling latency, show minimal variation, random seeds show, seeds show minimal
备注:
点击查看摘要
Abstract:Distilling diffusion models into few-step student models substantially reduces sampling latency, but induces severe diversity collapse: samples generated from different random seeds show minimal variation. In this work, we investigate the root cause of this failure by probing the internal representations of student models relative to their teachers. We find that distilled students maintain total activation variance comparable to their teachers, but concentrate it into substantially fewer directions. This directional collapse is localized to an identifiable intermediate block at the initial denoising step. Consequently, diversity collapse stems from a misallocation of activation variance rather than an inherent loss of generative capacity. Guided by this finding, we propose ReDiverse to reinstate the missing variance directions. At training time, ReDiverse-FT fine-tunes a lightweight adapter that regresses student activations onto the teacher's, reviving the teacher's feature variance characteristic. At inference time, ReDiverse-TF introduces a training-free mechanism that steers activations along precomputed teacher principal directions. Across image and video distillation benchmarks, ReDiverse consistently restores sample diversity while preserving generation quality, further supported by the user study.
3. 【2611.18549】JVT: Just Video Transformer makes pixel-space video diffusion simple
链接:https://arxiv.org/abs/2611.18549
作者:Jianjun Zhou,Hua Geng,Haoyu Guo
类目:Computer Vision and Pattern Recognition (cs.CV)
关键词:Direct diffusion modeling, high-dimensional raw video, raw video data, existing pixel-space methods, Direct diffusion
备注: 23 pages
点击查看摘要
Abstract:Direct diffusion modeling of high-dimensional raw video data is challenging, and existing pixel-space methods often rely on specialized architectures. We introduce JVT (Just Video Transformer), a simple large-patch DiT-style Transformer for pixel-space video diffusion, without a pretrained autoencoder, auxiliary detailers, cascaded stages, or U-Net-style skip connections. We systematically examine video patch embeddings, register tokens, and adaptive layer normalization to identify design choices that balance generation quality and computational efficiency. The resulting JVT design achieves generation quality competitive with latent diffusion baselines in both class-conditioned and unconditional settings. The pixel-space formulation of JVT also naturally supports joint generation of pixel-aligned modalities. We demonstrate this capability through camera-conditioned RGB-disparity generation, which improves camera following relative to RGB-only baselines and yields explicit scene geometry without an additional reconstruction model at inference. These results highlight the potential of simple DiT-style Transformers for pixel-space video diffusion, and we hope our findings offer useful insights and practical guidance for future research.
4. 【2611.18530】Understanding Contextual Bias in Agricultural CNNs Through Hidden-Neuron Concept Induction
链接:https://arxiv.org/abs/2611.18530
作者:Abhilekha Dalal,Avishek Das,Samatha Ereshi Akkamahadevi
类目:Computer Vision and Pattern Recognition (cs.CV); Machine Learning (cs.LG)
关键词:Convolutional Neural Networks, Convolutional Neural, exploit contextual information, Neural Networks, Concept Induction
备注:
点击查看摘要
Abstract:Convolutional Neural Networks (CNNs) can classify images using background-dominated patches that contain little or none of the target object, showing that they exploit contextual information. Previous studies detected this behavior through classification accuracy but say little about what CNN hidden layers represent. We investigate whether Concept Induction can reveal interpretable visual properties associated with hidden-neuron activations on contextual image patches. Using five agricultural datasets with high, moderate, and low contextual bias, we analyze four CNN architectures at three network depths and five patch locations. Quantitative image descriptors of each patch are encoded in a knowledge graph and used as background knowledge for Concept Induction, which derives logical expressions characterizing neuron activation. These expressions are tested on held-out images using the Mann-Whitney U test and Cliff's delta. Of 48,013 tested neuron-feature pairs, 13,164 (27.4%) were validated. Validation was most frequent at intermediate depths (mean 40.2% at mid layers versus 12.0% at deep layers), and no patch location dominated across datasets. Among the tested descriptors, edge, Gabor, and Tamura texture features had the highest validation rates. Stronger contextual bias did not correspond to higher neuron-feature validation rates: moderately biased Sugarcane had the highest rate (40.5%) and highly biased Cassava the lowest (19.2%). These results suggest that behavioral context dependence and the semantic interpretability of hidden neurons capture different aspects of CNN behavior, and that Concept Induction can complement patch-based bias detection.
5. 【2611.18524】EventSpikeSeg: Memory-Aware Mixed-Signal Architecture for Event-Camera Spiking Segmentation
链接:https://arxiv.org/abs/2611.18524
作者:Amir Mohammad Khadem Hosseini,Sattar Mirzakuchaki
类目:Hardware Architecture (cs.AR); Computer Vision and Pattern Recognition (cs.CV); Neural and Evolutionary Computing (cs.NE)
关键词:Event-camera segmentation requires, making memory organization, sparse input events, persistent neural state, segmentation requires persistent
备注:
点击查看摘要
Abstract:Event-camera segmentation requires persistent neural state despite sparse input events, making memory organization a central constraint on embedded implementation. We present EventSpikeSeg, a mixed-signal architecture combining resident capacitor memory with shared analog processing in GF180MCU 180-nm CMOS. Its 25-population spiking encoder-decoder has 143,824 synaptic weights but requires 30,976,000 membrane states at 440 x 640 resolution. The mapping specifies chronological 5-ms model updates, digital event/address services and 64-state capacitor banks. Characterization covers 22 module families and 74 operating points, distinguishing transistor simulations from digital-library estimates. Nominal results include 41.711 microwatts biased neuron-core power, 3.338-pJ positive terminal energy for a 1.4-microsecond cell read, and 0.531-mV selected-state drift over a 5-ms extracted-tile hold. Commit and coupling errors are reported separately. The software checkpoint records 27.976% mean intersection-over-union on a held-out six-class DSEC split; this is not hardware accuracy. Component layouts and simulation evidence quantify local implementation costs and expose storage replication and service scheduling as scaling constraints. A detailed SPICE-calibrated behavioral forward simulation of one full-resolution DSEC frame achieves 24.26% mean intersection-over-union, versus 26.22% for the matched PyTorch reference, using a compact 5-fF bank variant and external count/voltage calibration. Physical full-network operation, system energy and mismatch robustness remain unverified.
6. 【2611.18519】Box-conditional Hypothesis Testing for Out-of-Distribution Object Detection
链接:https://arxiv.org/abs/2611.18519
作者:Donghoon Lee,Jusung Park,Shinjin Kang
类目:Computer Vision and Pattern Recognition (cs.CV)
关键词:real-world deployment, essential for ensuring, ensuring the reliability, reliability and safety, box
备注: 18 pages, 7 figures, 13 tables
点击查看摘要
Abstract:Detecting out-of-distribution (OOD) objects is essential for ensuring the reliability and safety of object detectors in real-world deployment. We treat the decision on a detected box as a hypothesis test with the null hypothesis that the box covers an object of an in-distribution (ID) class. The null distribution changes with the size of the box and its contact with the image border, and the detector observes both. The test can therefore rank a box only against ID boxes of the same size and border contact, which gives a distribution-free bound on the false-alarm rate for each of them and uncovers the separation that one null distribution hides. The power of the test also rests on the layer it reads. Layers built only for the ID classes have no reason to keep what sets an unknown object apart, so the test reads the features before them. We propose \textbf{PLUM}, a training-free score that pools the last backbone stage inside the box and ranks its distance to the ID classes among ID boxes of the same size and border contact. On the standard benchmark, PLUM reaches the highest AUROC among the compared methods without external models, with 91.5 and 94.5 on PASCAL-VOC and 95.1 and 97.5 on BDD100K. Code: this https URL
7. 【2611.18516】StreamEmbed: Real-Time Indexing from Pixels for Screenshot Retrieval
链接:https://arxiv.org/abs/2611.18516
作者:Dayoon Ko,Yichuan Wang,Soyeong Jeong,Joseph E. Gonzalez,Gunhee Kim,Sewon Min
类目:Computer Vision and Pattern Recognition (cs.CV); Information Retrieval (cs.IR); Machine Learning (cs.LG)
关键词:searchable visual memory, offline corpus, serve as searchable, memory for people, unlike an offline
备注:
点击查看摘要
Abstract:Screen history can serve as searchable visual memory for people and agents, but unlike an offline corpus, it must be encoded and indexed as each screenshot arrives. OCR-based indexing is fast but loses visual information; visual encoding takes over two seconds. Yet screens change incrementally: most pixels repeat within and across screenshots. This raises a question: do visual encoders need to recompute their representations from scratch for every new screenshot? We present StreamEmbed, a system for incremental visual embedding that exploits this redundancy. Guided solely by pixel differences, it combines spatial token pruning, which skips blocks that repeat a neighbor, with temporal KV reuse, which reuses cached keys and values for blocks unchanged across screenshots. A small learned adapter compensates for pruning and reuse to preserve fidelity to full-screenshot embeddings. On ScreenMemEval, our benchmark of 10,426 questions over 184 hours of screen histories from two humans and a computer-use agent, StreamEmbed sets a new accuracy-latency Pareto frontier on an M2 Pro GPU: it encodes up to 3.7x faster than full-screenshot encoding without sacrificing accuracy and improves accuracy over OCR-based indexing by up to 6.3 Recall@10 points at matched latency. A memory-efficient vector index stores a simulated year of screenshots in 0.13 GB of RAM. Together, these results make real-time indexing of screen history in its native visual form practical, laying a foundation for fully local visual assistants.
8. 【2611.18482】MMFlash: Faster Speculative Decoding for Vision-Language Models
链接:https://arxiv.org/abs/2611.18482
作者:Sicheng Feng,Zigeng Chen,Song Wang,Gongfan Fang,Xinchao Wang
类目:Computer Vision and Pattern Recognition (cs.CV)
关键词:Speculative decoding accelerates, decoding accelerates generation, target model, Speculative decoding, accelerates generation
备注: Work in process, website: \url{ [this https URL](https://fscdc.github.io/MMFlash/) }
点击查看摘要
Abstract:Speculative decoding accelerates generation by letting a lightweight drafter propose tokens for parallel verification by a target model. Diffusion drafters such as DFlash propose an entire token block in a single forward pass, but mainly focus on text-only tasks, leaving multimodal drafting underexplored. To study this setting, we first introduce a metric of token-level visual dependence based on the KL divergence between next-token distributions with and without visual input. We find that acceptance rates decrease from 46% in the lowest visual-dependence quartile to 22% in the highest, with this disparity persisting after multimodal training. Motivated by this observation, we propose MMFlash, a simple and effective framework that adaptively upweights visually dependent tokens that the drafter predicts poorly. For long video inputs, MMFlash further employs sparse attention to reduce redundant attention computation and improve drafting quality. Across various multimodal benchmarks, MMFlash achieves lossless decoding speedups of up to $3.58\times$, outperforming EAGLE-3 and DFlash by up to 195.4% and 17.1%, respectively.
9. 【2611.18438】oward On-Device Markerless 6DoF Tracking of a Physical Manipulative on a Mixed-Reality Headset: A Preliminary Study
链接:https://arxiv.org/abs/2611.18438
作者:Guangxi Feng,Philip Yasskin,Wei Yan
类目:Computer Vision and Pattern Recognition (cs.CV); Human-Computer Interaction (cs.HC)
关键词:explore spatial transformations, learners explore spatial, spatial transformations, learners explore, explore spatial
备注:
点击查看摘要
Abstract:AR-Classroom uses physical models and virtual overlays to help learners explore spatial transformations. To align these overlays on a standalone headset, we developed a markerless six-degree-of-freedom(6DoF) tracker for a LEGO airplane on the Meta Quest 3. A compact YOLO model trained on synthetic Unity renders predicts 20 keypoints from dynamic RGB crops resized to 128 by 128 pixels. A perspective-n-point solver estimates the airplane's pose from these predictions and the corresponding 3D landmarks. In a paired evaluation on 1,000 synthetic validation images, cropping guided by ground-truth boxes increased the detection rate on annotated images from 32.5 to 46.2 percent. Mean keypoint F1 increased from 42.9 to 44.6 percent, with lower keypoint precision in the cropped view. In preliminary headset tests, 2 Hz was the highest tested tracking target at which the application maintained approximately 72 frames per second. A headset demonstration illustrates virtual overlay alignment with the physical airplane before and after a manual rotation. Quantitative 6DoF pose accuracy on real headset footage will be measured in future work.
10. 【2611.18429】Do Underwater Image Quality Metrics Agree with Human Perception? A Benchmark with Human Choices, Simple Edits and Generative Drift
链接:https://arxiv.org/abs/2611.18429
作者:Rishii Kumar Singh,Abhinav Rajput,Dinesh Kumar Vishwakarma
类目:Computer Vision and Pattern Recognition (cs.CV); Image and Video Processing (eess.IV)
关键词:image quality assessment, no-reference image quality, quality assessment, lack of clean, researchers rely
备注: Accepted at the 6th WVAQ Workshop, WACV 2027. 18 pages including appendix
点击查看摘要
Abstract:Underwater image enhancement is challenging to evaluate due to the lack of clean reference images, so researchers rely on no-reference image quality assessment (NR-IQA) metrics. However, it remains unclear whether these metrics agree with human judgments. We evaluate 73 handcrafted underwater, learned underwater, vision-language and general-purpose metrics against human preferences on five references: four published opinion-score sets and a new paired-comparison study. For each metric, we measure how often it selects the same image as people when comparing versions of the same scene. We further test whether simple image edits and small adversarial perturbations raise metric scores above the enhancement people preferred, and whether scores decrease as repeated generative edits add noise. Our results show that no metric consistently matches human preferences. The best worst-case Cohen's kappa over the five references is only 0.23 (Q-Align), while two raters in our study agree with a median kappa of 0.41. The widely used UCIQE often favors edits that people reject. On a Nano Banana Pro replica chain that should leave the image unchanged, only 22 of 73 scores decrease at every checkpoint, while noise grows by 2.5 times. We also introduce TrustUIQA, a benchmark that combines the human references, our study choices, the edited, attacked and generated images, and the scores of every metric on every image. The code is available at this https URL and TrustUIQA at this https URL.
11. 【2611.18417】CoPre-MoE: Consolidating and Preserving MoE for Multimodal Continual Instruction Tuning
链接:https://arxiv.org/abs/2611.18417
作者:Tao Hu,Ling-Xiang Zhang,Da-Wei Zhou
类目:Computer Vision and Pattern Recognition (cs.CV); Artificial Intelligence (cs.AI); Machine Learning (cs.LG)
关键词:requires multimodal large, multimodal large language, tuning requires multimodal, instruction tuning requires, large language models
备注:
点击查看摘要
Abstract:Multimodal continual instruction tuning requires multimodal large language models to sequentially acquire new capabilities without catastrophic forgetting. Mixture-of-experts (MoE) architectures offer a promising way to mitigate task interference through expert specialization and combination. However, while existing MoE-based methods avoid overwriting shared parameters by allocating separate experts for each task, this progressive expansion incurs severe cross-task redundancy and linear parameter growth. Specifically, experts across tasks often learn overlapping functions alongside task-specific capabilities. For instance, ScienceQA and TextVQA both locate relevant image regions. To curb parameter redundancy without inducing forgetting, we propose Consolidating and Preserving MoE (CoPre-MoE). For each incoming task, CoPre-MoE first learns task-specific MoE to capture specialized capabilities. To eliminate redundancy and reduce parameter growth across tasks, we design a shared-and-private expert architecture. Specifically, we first evaluate functional overlap to selectively consolidate overlapping experts across tasks into a fixed-capacity generic expert pool. We then factorize the remaining task-specific experts into shared projection bases and compact task-specific cores within a private expert pool. To mitigate forgetting, we construct the router by pairing each expert with learnable retrieval keys, where historical keys remain frozen during continual updates while newly added keys are regularized with an orthogonality penalty. At inference, input-dependent routing dynamically combines both pools without test-time task labels. Extensive experiments demonstrate that CoPre-MoE achieves superior performance while substantially reducing parameter overhead.
12. 【2611.18397】Gamer: A Generalist Vision-Language-Action Foundation Model for Long-Horizon Real-Time Game Agent
链接:https://arxiv.org/abs/2611.18397
作者:Run Luo,Kuan Zhang,Qi Deng,Yukun Chen,Zhihao Yang,Yufei Zhang,Renrong Hu,Yi Zhang,Guiyu Ma,Tianyu Fan,Longze Chen,Hao Sun,Jiaming Li,Jiaming Yang,Xiaohao Liu,Tao Tan,Minzheng Wang,Hangtian Jia,Shuai Yuan,Lvdi Wang,Xiao Liu,Chao Ma,Yuting Zou,Yinhe Zheng,Xiaobo Xia,Dawei Jiang,Shaoqing Ren
类目:Computation and Language (cs.CL); Artificial Intelligence (cs.AI); Computer Vision and Pattern Recognition (cs.CV)
关键词:agent framework built, framework built, complex interactive environments, introduce Gamer, generalist agent framework
备注:
点击查看摘要
Abstract:We introduce Gamer, a generalist agent framework built for long-horizon tasks in complex interactive environments. Its end-to-end architecture combines perception, memory management, and multi-step reasoning. A shared training and deployment pipeline supports interaction with both conventional 2D graphical interfaces and complex 3D simulated worlds. We first continue pre-training on a dataset constructed from tens of thousands of hours of device-level keyboard and mouse interaction. These trajectories span simulated games and embodied environments, allowing the model to learn environmental dynamics across both physical and virtual domains. Sparse-thinking instruction tuning then substantially reduces redundant computation while preserving the reasoning needed for complex decisions. We train the policy for long-horizon interaction with a stable multi-turn reinforcement learning method that addresses finite context windows, accumulated rollout errors, and sparse rewards. For real-time interaction, a cloud-edge inference engine serves numerous simulated game environments concurrently, supports unbounded-context execution, and keeps online inference latency low. Across multiple challenging 3D open-world games, Gamer demonstrates solid generalization and operational efficiency comparable to skilled players. It also achieves competitive results on a range of conventional 2D game benchmarks. These results suggest that even repetitive keyboard and mouse trajectories can support substantial gains from scaling data and learning algorithms. Training on these trajectories helps Gamer learn complex skills more steadily in environments with both physical and digital properties. This approach offers a path toward generalist agents that can interact across digital and physical worlds.
13. 【2611.18375】AI-Based Multi-View Echocardiographic Image Synthesis using Dynamic Shape Control
链接:https://arxiv.org/abs/2611.18375
作者:João Freitas,Anca Balinisteanu,Ahmed Youssef,Vitor Hugo Pereira,Jan D'hooge,Sandro Queirós
类目:Computer Vision and Pattern Recognition (cs.CV); Artificial Intelligence (cs.AI)
关键词:progress remains constrained, progress remains, remains constrained, improved echocardiographic workflows, multi-view
备注: 5 pages, 2 figures, accepted for publication in the Proceedings of the 2026 IEEE International Ultrasonics Symposium (IUS 2026)
点击查看摘要
Abstract:Although automated methods have improved echocardiographic workflows, progress remains constrained by the need for large, well-annotated datasets. As the development of clinical databases is often limited by ethical and logistical challenges, synthetic data generation has emerged as a promising alternative. However, existing approaches commonly rely on left ventricle shape-based supervision or are tailored to specific cardiac views, limiting the generation of additional standard and unconventional imaging planes and the preservation of multi-view consistency across cardiac chambers. To address these limitations, we propose a novel framework for shape-controlled synthesis of multi-view echocardiographic sequences. First, multi-view cross-sections are extracted from dynamic whole-heart meshes. Ultrasound-like appearance is then introduced through a two-stage AI-based pipeline. In Stage 1, an unpaired GAN-diffusion model translates these cross-sections into realistic echocardiographic frames. As this translation is performed independently at the frame level, Stage 2 employs masked video reconstruction to improve temporal coherence and cardiac motion. Evaluation demonstrated high frame-level realism for both stages, while the integration of Stage 2 substantially improved video-level motion quality and temporal consistency. Overall, the proposed framework enables the generation of realistic multi-view echocardiographic sequences from diverse cardiac models, providing a scalable source of annotated data for dataset augmentation and the development of multi-view cardiac assessment methods.
14. 【2611.18363】Adaptable Watermarking: Transforming Image Watermarking Models via Lightweight Adapters
链接:https://arxiv.org/abs/2611.18363
作者:Mehrdad Saberi,Vishal Asnani,Shruti Agarwal,Soheil Feizi,John Collomosse
类目:Computer Vision and Pattern Recognition (cs.CV); Cryptography and Security (cs.CR); Machine Learning (cs.LG)
关键词:neural image watermarking, image watermarking models, explore the adaptability, adaptability of neural, neural image
备注: Accepted to WACV 2027
点击查看摘要
Abstract:We explore the adaptability of neural image watermarking models; the extent to which one watermarking method can be transformed into another through lightweight adaptation. We introduce a framework that enables encoder-decoder watermarking architectures to become modular and upgradable, supporting multiple watermarking styles without re-training full models. Adaptation is performed by attaching a compact parameter-efficient adapter and fine-tuning under a multi-domain residual alignment loss (spatial, gradient, and frequency) while preserving bit accuracy via a secret-loss term. We evaluate this approach across a suite of recent watermarking models, including TrustMark, InvisMark, RoSteALS, HiDDeN, and WAM, demonstrating that adapted encoders closely reproduce target residuals and that adapted decoders recover messages with near-optimal accuracy under standard augmentations and distortions. We show that cross-model adaptation, with applications in interoperable and upgradable watermarking, can be achieved with as few as 1-2M trainable parameters per component (e.g., the 2M decoder adapter used in our multi-decoder inference pipeline).
15. 【2611.18359】Envisioning Agentic Robotics: A New Era of Robotic Systems
链接:https://arxiv.org/abs/2611.18359
作者:Grid Robotics:Qingfeng Lan,Duo Li,Siyao Li,Yiheng Lyu,Shijian Lu,Zhenghao Qi,Shizun Wang,Xinchao Wang,Ming Yang,Zhiqin Yang,Zuhao Yang,Tao Yu
类目:Robotics (cs.RO); Artificial Intelligence (cs.AI); Computation and Language (cs.CL); Computer Vision and Pattern Recognition (cs.CV)
关键词:Astra marks, marks the emergence, agentic robotic system, agentic robotics, Astra
备注: Project website: [this https URL](https://grid-robotics-official.github.io/agentic-robotics-roadmap)
点击查看摘要
Abstract:The release of GPT-6 Astra marks the emergence of a new paradigm in robotics: agentic robotics. Within weeks, frontier models with agentic harnesses matched or exceeded task-specific policies on many embodied benchmarks. Instead of training a dedicated policy for each family of tasks, an agentic robotic system applies a foundation model to open-world embodied tasks. It operates in a closed loop: observing the environment, interpreting user intent, reasoning about the task, invoking skills and tools, verifying outcomes, and adapting its strategy. Despite these advances, the community has yet to converge on a shared definition, a reliable evaluation framework, or a research roadmap. We argue that Embodied System 2, realized by a foundation model together with an embodied harness, forms the decision-making core of agentic robotics. Its central challenge is to translate general-purpose reasoning into effective and verifiable actions under physical constraints such as limited reaction time, mechanical limits, unpredictable environment changes, and irreversible consequences. Motivated by this perspective, we propose an evaluation framework that measures both model capabilities and closed-loop system performance. Furthermore, we identify the principles that carry over from digital to physical agents and the challenges that must be addressed to achieve reliable physical autonomy. On top of the Embodied System 2 and the evaluation framework, we envision a research roadmap toward sustained autonomy: the ability to perform work safely and reliably with progressively broader responsibility, from executing individual primitives to fulfilling standing roles.
16. 【2611.18350】MaskSCD: Bitemporal Mask Classification for Semantic Change Detection in Remote Sensing Images
链接:https://arxiv.org/abs/2611.18350
作者:Weikang Yu,Pedram Ghamisi
类目:Computer Vision and Pattern Recognition (cs.CV)
关键词:Semantic change detection, land surface, SCD, change, change detection
备注:
点击查看摘要
Abstract:Semantic change detection (SCD) identifies where the land surface has changed and which land-cover transition has taken place. Most deep SCD networks predict the change map and the land-cover map of each date with separate per-pixel decoders. These outputs need not agree, so a pixel can be labeled as changed although it has the same class at both dates, and per-pixel decisions fragment objects. We present MaskSCD, which formulates SCD as the mask classification of a bitemporal label-pair image. Each bitemporal segment, a region of constant land-cover class pair, is predicted as one mask shared by both dates together with one class per date, and the change map is derived from the predicted class pair. This formulation keeps the change and semantic maps consistent and the label space linear in the number of classes. A bitemporal masked-attention transformer decoder implements it by reading each segment in both images and relating the two readings, and a bitemporal fusion pyramid provides per-date and change-aware features. MaskSCD is evaluated on SECOND, Landsat-SCD and Hi-UCD, which span 0.1 to 30 m resolution and four to nine land-cover classes, against fourteen recent SCD models re-trained under one protocol. MaskSCD achieves the best separated kappa, semantic change $F_1$ and transition $F_1$ on all three datasets and exceeds the strongest competitor by about five points in mean separated kappa. It produces no contradictory change pixels, which account for nearly 30% of the predicted change of some multi-task networks. The code is available at this https URL.
17. 【2611.18346】VLAN: Vision-Language Accessible Navigation
链接:https://arxiv.org/abs/2611.18346
作者:Jie Hu,Yu Zheng,Wenzhi Wu,Weixing Feng,Cunlei Cheng,Jiaming Zhang
类目:Artificial Intelligence (cs.AI); Computer Vision and Pattern Recognition (cs.CV)
关键词:Vision-Language Accessible Navigation, present Vision-Language Accessible, walkable paths, Accessible mobility, Accessible Navigation
备注: Accepted to NeurIPS 2026 Evaluations and Datasets Track. 25 pages, 17 tables, 6 figures. Code: [this https URL](https://github.com/InSAI-Lab/VLAN)
点击查看摘要
Abstract:Accessible mobility is user-conditioned rather than scene-intrinsic, as walkable paths for one embodiment are inaccessible to another. Yet, existing benchmarks evaluate traversability under a fixed and limited embodiment. We present Vision-Language Accessible Navigation (VLAN), containing 1,000 scenes, 20K navigation trajectories, and 160K keyframes across diverse lighting, environmental conditions, scene semantics, and path geometries. For each trajectory, VLAN provides embodiment-specific actions and accessibility features for legged and wheeled agents, including humanoid and quadruped robots and people with visual or mobility impairments, while everyday obstacles and accessibility labels are categorized as Passable, Bypass, HighRisk, and Blocked. Unlike goal-only evaluation, our benchmark measures physical safety and alignment with user-specific constraints via goal-reaching, collision-aware, and accessibility-aligned metrics. We benchmark modern vision-language models and find they often fail to consistently adapt obstacle relevance, path selection, and safety criteria across user profiles, suggesting accessibility-aware navigation requires dedicated datasets, tasks, and metrics beyond conventional embodied navigation evaluation. All data and code open-source at this https URL.
18. 【2611.18340】DIRT: Depth-completed Image-based Rendering using Two-band Blending for Live Novel-view Synthesis
链接:https://arxiv.org/abs/2611.18340
作者:Mathias Harrer,Laura Fink,Noah Lewis,Paul Himmler,Tim Weyrich
类目:Computer Vision and Pattern Recognition (cs.CV); Graphics (cs.GR)
关键词:commonly require per-scene, require per-scene optimization, methods commonly require, commonly require, require per-scene
备注: 8 pages main paper + 2 pages of references + 9 pages supplementary material
点击查看摘要
Abstract:State-of-the-art novel-view synthesis (NVS) methods commonly require per-scene optimization before rendering, limiting their applicability in live settings. Live or ad-hoc NVS, however, demands low-latency rendering during capture and strong generalization to new scenes. Some previous work tried to address this task by adapting training- and optimization-based methods, trading quality for low latency. We introduce a live NVS approach following traditional image-based rendering that operates on a posed set of depth sensor observations and images. Our method leverages explicit, sparse point-based geometry obtained from active sensors, resulting in a sparsely filled depth buffer that is completed in the (novel) target view's screen-space. Using this proxy, a posed set of images is projected into the target frame before fusing the resulting observations through a frequency-aware two-band blending strategy. The pipeline is entirely non-neural and optimization-free, requiring no training data or scene-specific optimization. As a result, our approach reduces the delay between capture and novel-view rendering and does not rely on favorable network generalization. We validate the method's applicability to heterogeneous sensor setups - including handheld RGB-D cameras, automotive LiDAR, and thermal cameras - as well as challenging scenarios, such as low light captures. Our experiments demonstrate that the approach outperforms prior methods in live reconstruction scenarios, while maintaining competitive performance in conventional NVS benchmarks.
19. 【2611.18324】Unlocking Cross-Attention in Label-Scarce Regimes: A Weakly Supervised Approach for Video Semantic Segmentation
链接:https://arxiv.org/abs/2611.18324
作者:Andrea Marelli,Alessandro Ardenghi,Giacomo Boracchi,Luca Magri
类目:Computer Vision and Pattern Recognition (cs.CV)
关键词:temporally consistent semantic, consistent semantic object, semantic object masks, Weakly Supervised Video, Supervised Video Semantic
备注: Accepted at WACV 2027. 25 pages (10 main + 15 supplementary), 9 figures, 7 tables
点击查看摘要
Abstract:Weakly Supervised Video Semantic Segmentation (WS-VSS) aims to learn temporally consistent semantic object masks in videos using only video-level labels. While fully supervised models successfully learn temporal relationships by means of cross-attention, training these temporal weights under weak supervision is unstable and even pushes the model below its single-frame baseline. Therefore, existing WS-VSS methods exploit temporal consistency by relying on external priors, such as optical flow. Unfortunately these cues are non-semantic, computationally expensive and noisy, degrading the supervisory signal. Instead of supervising attention weights directly or using external priors, we propose MemCAM, a WS-VSS transformer framework that learns temporal cross-attention directly through feature-space self-supervision. MemCAM introduces a CAM Propagation Algorithm that synthesizes a supervisory signal by propagating past CAMs according to patch-level feature similarities in the latent space. Specifically, we constrain their induced CAMs through a consistency objective between predicted and propagated maps, stabilizing temporal attention learning. Experiments on the YouTube-Objects v2.2 and WS$^2$ datasets demonstrate that MemCAM consistently outperforms the temporal localization and segmentation performance of WS-VSS alternatives, showing that cross-attention can be effectively trained in label-scarce video settings through internal self-supervision.
20. 【2611.18267】raCoder: Adaptive Tetrahedral Encoding for Neural Fields
链接:https://arxiv.org/abs/2611.18267
作者:Daniele Della Pietra,Antonella Rech,Nicola Conci,Nicola Garau
类目:Graphics (cs.GR); Computer Vision and Pattern Recognition (cs.CV)
关键词:computer graphics applications, pros and cons, central in computer, computer graphics, graphics applications
备注: 8 pages, 4 figures
点击查看摘要
Abstract:Implicit and explicit representations of geometry are central in computer graphics applications and come with their pros and cons. With surface sampling being faster on explicit representations and spatial queries being faster on implicit representations, finding efficient conversions between them is a highly desirable property. We present TetraCoder, a single resolution hash-free adaptive tetrahedral encoding that links an explicit tetrahedral spatial encoder with an underlying implicit signed distance field (SDF). The resulting hybrid implicit-explicit space is continuous piecewise affine (CPWA) by design, allowing to obtain an exact correspondence between the implicit SDF and the explicit analytic mesh. TetraCoder's design comes with additional perks, in particular: (a) it scales better than multi-resolution encoding methods due to its single resolution optimizable grid, (b) unlike hash-based methods, it is not affected by collisions, (c) it enables fast tetrahedral ray-tracing, geometry and appearance editing, directly through its adaptive tetrahedral scaffold. We demonstrate the versatility of TetraCoder by showing an improvement over state-of-the-art methods on multiple tasks: 2D/3D fitting from images and point clouds, 3D novel view synthesis, 3D reconstruction, and geometry/appearance editing.
21. 【2611.18261】Structural Self-Teaching for Compositional Generation in Unified Multimodal Models
链接:https://arxiv.org/abs/2611.18261
作者:Jaeyoo Park,Bohyung Han
类目:Computer Vision and Pattern Recognition (cs.CV)
关键词:unified multimodal models, diffusion-based unified multimodal, propose Structural Self-Teaching, multimodal models, generation path
备注: NeurIPS 2026
点击查看摘要
Abstract:We propose Structural Self-Teaching (SST), a novel framework that enhances compositional image generation in diffusion-based unified multimodal models (UMMs) by leveraging their internal understanding path as a source of structural supervision. The generation path of UMMs often struggles with dense compositional prompts, leading to missing objects, attribute binding failures, and spatial errors. In contrast, their understanding path localizes the same phrases in images that satisfy these prompts. SST converts this phrase-level grounding into training-time structural signals for the generation path. Specifically, we extract phrase-conditioned support masks from internal attention maps derived from the understanding path and employ them through two mechanisms. Phrase-local contrastive alignment uses these masks as pooling regions and aligns each generated phrase region with the same region of the target image. A structural scaffold builds a phrase-level layout from the same masks and modulates an intermediate spatial state of the generation path during training. The scaffold is phased out during training and removed at inference. SST requires no external structural annotations for training and no additional input at test time. Extensive experiments across multiple UMMs demonstrate that SST improves compositional fidelity over LoRA fine-tuning on the same data, with no inference-time cost. Project page is available at this https URL
22. 【2611.18256】Deep Bundle Adjustment
链接:https://arxiv.org/abs/2611.18256
作者:Pak Hong Chui,Hector Andrade-Loarca,Daniel Cremers
类目:Computer Vision and Pattern Recognition (cs.CV)
关键词:textbf, graph neural network, recovers camera poses, recovers camera, Deep
备注: 15 pages, 13 figures, 6 tables
点击查看摘要
Abstract:We present \textbf{DeepBA} (\textbf{Deep} \textbf{B}undle \textbf{A}djustment), a graph neural network (GNN) that recovers camera poses, intrinsics, and scene structure from 2D matches across an arbitrary number of views. Rather than solving a non-linear least-squares problem per scene, DeepBA amortizes bundle adjustment into a learned graph network that iteratively refines its estimate from geometric residuals. Where classical SfM computes its updates analytically and needs careful initialization, DeepBA learns them and converges from a cold start. It is trained with scale-invariant supervision (ground-truth rotation, translation direction, and intrinsics, plus reprojection self-consistency), so datasets with arbitrary metric units can be mixed without per-scene normalization. DeepBA is decoupled from the matcher. At 2.04M parameters, it outperforms prior learned SfM from matches on scene reconstruction and on most pose metrics, is competitive with classical SfM at moderate accuracy thresholds and overtakes it in wall time as the view count grows, and approaches monolithic image-to-geometry transformers such as VGGT ($\sim$1.2B parameters) with greater modularity. Project~page: this https URL
23. 【2611.18253】Pose-based Neural Clothing for Animated Characters
链接:https://arxiv.org/abs/2611.18253
作者:Julian N. Heidenreich,Vinicius C. Azevedo,Jakob Buhmann,Lento Manickathan,Arnold Moon,Paul Kanyuk,Amit H. Bermano,Jingwei Tang
类目:Graphics (cs.GR); Computer Vision and Pattern Recognition (cs.CV)
关键词:http URL data-driven, Traditional garment simulation, URL data-driven methods, generate high-quality deformations, http URL
备注: 19 pages, 18 figures, 5 tables. Includes appendix. Accepted to the ACM SIGGRAPH / Eurographics Symposium on Computer Animation (SCA 2026); to appear in Computer Graphics Forum
点击查看摘要
Abstract:Traditional garment simulation for animated characters is a time-consuming bottleneck for artists due to slow simulations and tedious parameter this http URL data-driven methods offer a powerful alternative, capable of learning from a dataset of simulated data to generate high-quality deformations. A lightweight network design allows data-driven systems to achieve a real-time inference speed. This significantly improves the artist workflow, as they can interactively pose a character and see high-quality folds and wrinkles at animation time, thereby allowing to make creative decisions ahead of time. However, existing neural clothing methods have presented appealing results on datasets based on realistic human models (\eg SMPL), but when trained on animated character datasets, they struggle to generalize to unseen poses. In this paper, we aim at designing neural networks and training strategies to generalize better to unseen poses in real-time to enable the interactive posing applications. We focus on a production-oriented setting where the model is trained for a single garment in a quasi-static setup. Technically, we leverage a smooth, expressive latent space by first training a Variational Auto-Encoder (VAE). We then progressively train the VAE encoder to transition from reconstruction to pose-driven prediction. To guide this prediction, we condition the network on both low-dimensional pose parameters and character body surface displacements. Finally, we introduce a novel local gradient histogram loss that mitigates the over-smoothing effect of reconstruction losses. Extensive evaluations on production-level assets and public datasets demonstrate that our method produces sharp, realistic wrinkles, smooth large-scale drape, and temporal coherence in real-time, offering a robust and easily integrable solution for interactive posing workflows.
24. 【2611.18251】Lens: Training-Free Semantic Search for Cross-Tile Object and Scene Retrieval in Satellite Imagery
链接:https://arxiv.org/abs/2611.18251
作者:Ata Akbari Asanjan,Anit Upadhyaya,Jeremy Kravitz,Subash Yeggina
类目:Computer Vision and Pattern Recognition (cs.CV)
关键词:large satellite mosaics, class of interest, training-free system, system for semantic, semantic object
备注:
点击查看摘要
Abstract:We developed Lens, a training-free system for semantic object and scene search across large satellite mosaics that requires no detector trained for the class of interest. Lens represents each tile with a frozen, self-supervised vision transformer (DINOv3), removes the positional bias in its dense features by projecting them onto the subspace orthogonal to a positional basis estimated once from a constant image, and centers the debiased features on the mean feature computed across the whole area before comparison. Lens forms a query by averaging the centered features of a small neighborhood of patches around a chosen point, or of an entire tile for scene-level search, and matches it against the index by cosine similarity. It stores the centered patch features in a byte-quantized similarity index that supports real-time range search over a mosaic of thousands of patches, and a coarser whole-tile embedding index for scene-level retrieval. A guided upsampling network (AnyUp) runs once per tile to recover pixel-accurate boundaries beyond the native feature grid, and the result is cached so later queries reuse it. We evaluated Lens on the 5,128-tile DFC2020 test set and ran it as an interactive tool on five Sentinel-2 areas spanning agricultural, industrial, and desert terrain; the pipeline is not tied to a specific sensor. On DFC2020 a single click reaches 0.09 to 0.80 pixel F1 across eight land-cover classes at the per-class best threshold, with a mean F1 of 0.39, and it beats a pairwise INSID3 baseline on six of the eight classes at roughly four hundred times lower query cost. Indexing 1.31 million patches takes 169 s on one CPU thread, and a click returns a rendered overlay in 6 to 10 ms.
25. 【2611.18240】SCAR: Bridging Coarse Retrieval and Fine Localization through Candidate Reranking
链接:https://arxiv.org/abs/2611.18240
作者:Shuhan Dong,Tianyi Shang,Zhenyu Li
类目:Computer Vision and Pattern Recognition (cs.CV)
关键词:Natural-language point-cloud localization, typically retrieves candidate, retrieves candidate map, Natural-language point-cloud, point-cloud localization typically
备注:
点击查看摘要
Abstract:Natural-language point-cloud localization typically retrieves candidate map cells and then predicts a position inside each candidate, yet the final output is still determined by retrieval order. A retrieval score measures region-description relevance, not whether the coordinates a region yields are accurate: a relevant region may produce a displaced prediction, and a lower-ranked region may contain a successful one. We study the final selection problem after the candidate pool and Fine predictions are fixed---how to choose the coordinate most likely to localize successfully, and under which conditions the gain holds. We propose SCAR (Spatial Consistency-Aware Reranking). SCAR treats each cell--Fine-coordinate pair as a localization hypothesis, interprets directional relations as functions of position, and reads spatial support at the backbone's actual Fine point (Fine-conditioned Evidence Readout). It organizes this readout with center support, semantic presence, retrieval score, and geometric offsets into a 23-D representation, trains a shared scorer with localization-success labels, and at inference reranks existing hypotheses and returns their original coordinates. Experiments cover four public backbones and a corrected Des4Pos reimplementation across five domains. Excluding one protocol-reference configuration, exploratory TEST point estimates on the remaining 24 configurations show Top-1@5\,m above Original on 23 and Cell R@1 above Original on all 24. On development-set VAL, the Fine readout improves Top-1@5\,m by 5.77 percentage points (pp) over a center surrogate on corrected Des4Pos*KITTI, and the complete scoring scheme improves it by 13.98\,pp over FSR on MambaPlace$\times$nuScenesFull; the latter measures the complete scheme, not learned ranking alone.
26. 【2611.18189】SemRAG: Semantic-Guided Sentence-Level Retrieval-Augmented Sign Language Production
链接:https://arxiv.org/abs/2611.18189
作者:Xiao Liu,Shiwei Gan,Yafeng Yin,Peishuo Liu,Bowen Guo,Kuizhuang Liu,Cong Wang,Lei Xie
类目:Computer Vision and Pattern Recognition (cs.CV)
关键词:faithfully express spoken-language, express spoken-language text, Sign language production, generate sign pose, sign pose sequences
备注:
点击查看摘要
Abstract:Sign language production (SLP) aims to generate sign pose sequences that faithfully express spoken-language text while preserving the rhythm and coordination patterns of signing. Common SLP methods learn text-to-pose mappings from training data but often suffer from semantic inaccuracies in long or complex sentences. Retrieval augmentation offers a way to introduce realistic signing priors to provide more explicit guidance for text-to-pose generation. However, existing retrieval-based SLP methods typically operate at the gloss or word level, relying on gloss-pose dictionaries or isolated sign segments that require costly annotations and fail to preserve continuous sentence-level dynamics. To bridge this gap, we propose SemRAG, a semantic-guided sentence-level retrieval-augmented framework that retrieves full-sentence signing templates as priors without constructing gloss/word-level sign dictionaries. While such templates preserve local motion patterns and global rhythm, they also introduce a semantic mismatch problem: irrelevant motions in the retrieved sentence may interfere with the target semantics. To address this issue, we introduce an SLT-guided template semantic localization module that identifies target-relevant temporal regions within the retrieved template while suppressing irrelevant signing content. To adaptively incorporate the template features into the generator, we further introduce layer-specific learnable gates that regulate their contribution across generator layers. Extensive experiments show that SemRAG improves semantic consistency and motion quality over baselines under both back-translation and motion accuracy metrics.
27. 【2611.18172】STaG: Human-Annotation-Free Unsupervised Domain Adaptation for Text-Based Person Retrieval
链接:https://arxiv.org/abs/2611.18172
作者:Ji Yucheng,Bai Yang,Li Yang,Zhou Xinyu,Cao Min
类目:Computer Vision and Pattern Recognition (cs.CV)
关键词:Text-Based Person Retrieval, Person Retrieval, Real-world deployment, Text-Based Person, Unsupervised Domain Adaptation
备注:
点击查看摘要
Abstract:Real-world deployment of Text-Based Person Retrieval (TBPR) is bottlenecked by the absence of textual annotations in new domains. Existing Unsupervised Domain Adaptation (UDA-TBPR) methods still assume access to target-domain human-written annotations (albeit unpaired with images), limiting their practicality. We consider a more practical setting: human-annotation-free UDA-TBPR, with labeled image-text data in the source domain and annotation-free, unlabeled images in the target domain. This setting introduces two core challenges: the absence of cross-modal alignment in the target domain, and visual domain shifts from source to target. To tackle these, we propose STaG, a dual-path framework comprising Style-Agnostic Text Alignment (SATA) and Heterogeneous Graph Memory Alignment (HGMA). Specifically, SATA leverages source-domain data to decouple identity-discriminative semantics from machine-generated stylistic artifacts, establishing robust cross-modal alignment for the target domain. Meanwhile, HGMA exploits cross-domain semantic neighborhoods via a memory-based graph to learn domain-invariant visual features. Ultimately, STaG achieves state-of-the-art performance across three benchmarks under this realistic setting.
28. 【2611.18170】ALoMa: Affine Correspondences Beyond Patches
链接:https://arxiv.org/abs/2611.18170
作者:David Nordström,Georg Bökman,Viktor Larsson,Fredrik Kahl,Jonathan Ventura
类目:Computer Vision and Pattern Recognition (cs.CV)
关键词:point correspondences, enable robust, multiple-view geometry, robust and efficient, Affine correspondences
备注:
点击查看摘要
Abstract:Affine correspondences (ACs) enable robust and efficient solvers for multiple-view geometry by constraining the local geometry of point correspondences (PCs). Despite their utility in minimal solvers, AC estimation remains understudied. We revisit AC estimation beyond local image patches, extending it to the image and image-pair levels. Rather than estimating the components of the affine transformation separately, we learn the full 4-DoF matrix jointly. Building on this formulation, we introduce ALoMa, an AC estimator using LoMa keypoint descriptions, demonstrating consistent improvements across a wide range of downstream tasks. Code at this https URL.
29. 【2611.18143】DRA-HTR: Disentangling Representation and Adaptation for Handwritten Text Recognition
链接:https://arxiv.org/abs/2611.18143
作者:Gayan H. Pathirage,Florent Imbert,Denis Coquenet,Elisa H. Barney Smith
类目:Computer Vision and Pattern Recognition (cs.CV)
关键词:remains challenging due, Handwritten text recognition, substantial domain shifts, domain shifts caused, Handwritten text
备注:
点击查看摘要
Abstract:Handwritten text recognition (HTR), particularly when applied to historical documents, remains challenging due to the scarcity of annotated data and the substantial domain shifts caused by variations in handwriting styles, document degradation, acquisition conditions, and linguistic characteristics. While large amounts of unlabeled target-domain data are often available, effectively exploiting them requires adaptation strategies tailored to the specific requirements of HTR. We propose DRA-HTR, an unsupervised generative domain adaptation pipeline specifically designed for HTR and built upon the DRANet framework. The proposed approach combines source-domain pretraining, unsupervised adaptation to the visual characteristics of the target domain, relying on the generation of target-style source samples for task-specific training. We further investigate the integration of post-HTR correction to complement visual adaptation with language adaptation, relying on large language models, which we introduce as DRAL-HTR. The proposed framework is evaluated across seven HTR datasets spanning four languages, using a single set of hyperparameters for all source-target combinations. The recognition model achieves state-of-the-art performance on four datasets and competitive results on the remaining three. In addition, the proposed DRA-HTR establishes a new state-of-the-art for domain adaptation across all seven datasets and substantially improves overall robustness and training stability compared to the original DRANet framework: it reduces the Character Error Rate (CER) in average by 13.52 percentage points, and the average standard deviation decreases from 7.23 to 1.11 percentage points. Finally, post-HTR correction further reduces the CER by 2.16 percentage points.
30. 【2611.18101】KnowVis: Dual-View Evaluation and Diagnosis of World-Knowledge Grounding in Text-to-Image Models
链接:https://arxiv.org/abs/2611.18101
作者:Yonghang Yu,Ruoyu Wang,Haoyu Zhang,Lin Qi,Yawei Zhou,Changqian Yu
类目:Computer Vision and Pattern Recognition (cs.CV)
关键词:made substantial progress, aesthetic quality, made substantial, substantial progress, Generalized Benchmark
备注: Accepted to NeurIPS 2026 ED Track. Codebase: [this https URL](https://github.com/QuantumWhisper42/KnowVis)
点击查看摘要
Abstract:Text-to-image (T2I) models have made substantial progress in visual realism, aesthetic quality, and instruction following. However, real-world prompts often go beyond explicit visual descriptions and require implicit facts, structured knowledge, and domain-specific commonsense. Existing evaluations mainly focus on explicit prompt-to-image semantic alignment or specific knowledge subdomains, lacking unified evaluation and diagnosis of world-knowledge grounding in T2I models. We introduce KnowVis for dual-view evaluation and diagnosis of world-knowledge grounding in T2I generation. Built on a hierarchical knowledge taxonomy with case-level annotations, KnowVis combines two complementary benchmark views with a structured MLLM-based VQA judge protocol. The Skill-Tree Benchmark supports fine-grained atomic knowledge evaluation, while the Generalized Benchmark examines open-ended knowledge composition and expression. The protocol uses Required VQAs to assess atomic knowledge correctness in the Skill-Tree Benchmark, and combines Required, Optional, and dynamically discovered knowledge expressions with human-judge collaborative evaluation in the Generalized Benchmark. We validate the protocol's reliability on the Skill-Tree validation split, enabling scalable evaluation and diagnostic error attribution. Experiments show that KnowVis reveals notable gaps in world-knowledge grounding across existing T2I models and provides structured diagnostic signals for knowledge-intensive image generation.
31. 【2611.18099】EcoVGGT: Pinned-Key Inference for Efficient and Streamable Feed-Forward 3D Reconstruction from Long Videos
链接:https://arxiv.org/abs/2611.18099
作者:Hongsen Cao,Shanxin Yuan,Mona Jaber,Ahmed Sayed
类目:Computer Vision and Pattern Recognition (cs.CV); Artificial Intelligence (cs.AI)
关键词:Feed-forward multi-view transformers, cross-frame self-attention grows, self-attention grows quadratically, Feed-forward multi-view, multi-view transformers
备注:
点击查看摘要
Abstract:Feed-forward multi-view transformers such as VGGT recover the cameras and geometry of a video in one pass, but the cost of their cross-frame self-attention grows quadratically with the frame count, and their accuracy drops on long videos. Most remedies approximate attention over all frames under a fixed computational budget or select a few keys by their content. We hypothesize that the keys of a long video can be selected independently of video content, and our experiments on VGGT and $\pi^3$ support it: a small number of uniformly sampled frames can serve as the keys of every frame. We present Eco3R, a training-free inference framework whose keys form a Pinned Key Set of 31 such frames. A pin pass stores their tokens in a Scene Cache, and the other frames read it in independent batches that reproduce joint inference with these keys exactly, at a constant cost per frame. An Online Stream updates the pinned frames during recording. Six middle global blocks align the frames and form a reusable VGGT module: the Alignment Score reads their attention to flag failures, and Alignment Refocusing reduces them by fine-tuning only these blocks (3.6 GPU hours). Eco3R ($\pi^3$) matches $\pi^3$'s success rate on 1,000-frame ScanNet videos while running 13.8 times faster. With refocusing, Eco3R (VGGT) raises VGGT's success rate from 70% to 82% while running 8.5 times faster on 500 frames. Eco3R can reconstruct sequences of up to 4,000 frames on a single A100 GPU, while Eco3R ($\pi^3$) streams 1,000 frames at 18.6 fps, achieving the lowest online absolute trajectory error.
32. 【2611.18097】AG-JEPA: Anatomically Guided Joint Embedding Predictive Architecture for Neurodegenerative Representation Learning
链接:https://arxiv.org/abs/2611.18097
作者:Azad Singh,Ayantika Das,Amog Rao,Aiman Farooq,Anirban Porya,Ananya Shukla,Sowbhagya Suresh,Srinivasan Iyengar,Mohit Jain,Mercy Prasanna Ranjit
类目:Computer Vision and Pattern Recognition (cs.CV); Artificial Intelligence (cs.AI)
关键词:reveal early signs, MRI can reveal, measurable cognitive decline, Embedding Predictive Architecture, Brain MRI
备注:
点击查看摘要
Abstract:Brain MRI can reveal early signs of Alzheimers disease (AD) before measurable cognitive decline becomes apparent. However, during the transition from cognitively normal (CN) to mild cognitive impairment (MCI), the disease-related signals are often subtle and confined to only a few brain regions. Existing Joint Embedding Predictive Architecture (JEPA) models learn general structural representations but do not explicitly capture the anatomical regions affected in AD. We propose Anatomically Guided JEPA (AG-JEPA), which introduces anatomical tokens that interact with patch tokens during pretraining to learn region-specific representations from brain regions relevant to neurodegeneration. Building on this representation space, we further propose Prog-AG-JEPA, which models disease progression trajectory as a conditioned latent displacement within this space. We evaluate AG-JEPA across five downstream tasks spanning Alzheimer's disease diagnosis and progression prediction. AG-JEPA improves diagnosis and MCI prognosis, achieving an 8.4 percentage point AUROC gain over the strongest baseline on CN/MCI classification and a 7.1 percentage point gain on out-of-distribution CN/AD classification on the MR-RATE dataset, highlighting its effectiveness across both early-stage detection and cross-cohort generalization. For disease progression modeling, Prog-AG-JEPA consistently outperforms all baselines on future diagnosis and brain volume change prediction, demonstrating the value of anatomically grounded neurodegeneration representations for diagnosis and prognosis.
33. 【2611.18078】Physica: Agentic Reconstruction of Physically Interactive Scenes
链接:https://arxiv.org/abs/2611.18078
作者:Ruiqi Shu,Xufeng Zhan,Yuanzhi Liang,Haibin Huang,Chi Zhang,Xuelong Li
类目:Computer Vision and Pattern Recognition (cs.CV); Artificial Intelligence (cs.AI)
关键词:entangle pixel synthesis, models entangle pixel, implicit latent motion, precluding persistent, mid-rollout intervention
备注: Ongoing Work. Homepage: [this https URL](https://interactive4d.vercel.app/)
点击查看摘要
Abstract:Direct image-to-video diffusion models entangle pixel synthesis with implicit latent motion, precluding persistent 3D mechanical equilibrium, mid-rollout intervention, and counterfactual force control. Conversely, single-pass programmatic 3D reconstruction from an uncalibrated photograph leaves monocular depth and support clearance underconstrained, causing reconstructed scenes to interpenetrate or collapse under gravity. We present TelePhysica, a closed-loop framework that reconstructs an executable, multi-regime physical world from a single photograph to drive stateful 3D interaction and trajectory-grounded video synthesis. A vision-language coding agent synthesizes separable visual meshes and declarative physical models within a two-stage verification loop that pairs pixel-aligned silhouette critique with a zero-force gravitational equilibrium preflight. The verified world compiles into a unified solver coupling rigid bodies, articulated chains, co-dimensional cloth shells, and 3D tetrahedral elastodynamics, whose simulated rollouts condition pretrained video diffusion models through pixel-aligned structural streams. On a 100-scene single-image benchmark, closed-loop execution raises equilibrium-verified world recovery from $0\%$ in single-pass synthesis to $81\%$. Controlled constitutive ablations show over an order-of-magnitude larger localized corner displacement and out-of-plane pleating in finite-element cloth than in matched rigid-plate proxies. Conditioning video diffusion on simulated trajectories increases dynamic motion-mask overlap by $2{\times}\text{--}5{\times}$ and preserves motion direction under counterfactual force reversals where direct image-to-video models fail.
34. 【2611.18057】SWAM: Saliency-Weighted Asymmetric Merging for Efficient Vision Transformers
链接:https://arxiv.org/abs/2611.18057
作者:Utkarsh Kumar Singh,Manish Das,Shiv Ram Dubey,Satish Kumar Singh
类目:Computer Vision and Pattern Recognition (cs.CV)
关键词:Vision Transformers, achieve strong performance, image recognition tasks, quadratic self-attention complexity, creates prohibitive computational
备注:
点击查看摘要
Abstract:Vision Transformers (ViTs) achieve strong performance on image recognition tasks, yet their quadratic self-attention complexity over input tokens creates prohibitive computational costs for practical deployment. Token reduction methods offer a practical solution by progressively reducing the token sequence during the forward pass. We propose Saliency-Weighted Asymmetric Merging (SWAM), a training-based token merging framework that extends the Bipartite Soft Matching (BSM) with two saliency-aware mechanisms. We introduce a two-step matching formulation in which cosine similarity decides candidate pairs, while a multiplicative maximum saliency penalty regulates their selection for merging. We then further use asymmetric exponential saliency-weighted aggregation, which allows more salient tokens to contribute more strongly to the merged representation. A lightweight MLP-based saliency prediction module (75k parameters per stage, 1% of backbone parameters) supervised via attention rollout is trained end-to-end jointly with a DeiT-Small backbone using a composite cross-entropy, knowledge-distillation, and saliency-alignment objective. On CIFAR-100, TinyImageNet, and ImageNet-100, under a conservative schedule that reduces tokens from 196 to 100 patch tokens, with the CLS token retained throughout, 27.5% FLOPs reduction and 1.35 times speedup, SWAM matches or exceeds baseline across the reported results. Under an aggressive schedule (196 to 70 tokens with the CLS token kept throughout, 33.8% FLOPs reduction, and 1.46 times speedup), accuracy gains are achieved on ImageNet-100 (+0.23%) with minimal losses on CIFAR-100 and TinyImageNet.
35. 【2611.18053】InfiHall-Bench: A Benchmark for Hallucination Evaluation in Autoregressive Text-to-Image Generation
链接:https://arxiv.org/abs/2611.18053
作者:Yuxia Qiao,Xinhao Zhong,Junhao Li,Yi Sun,Jiaxin Hong,Shuoyang Sun,Bin Chen
类目:Computer Vision and Pattern Recognition (cs.CV)
关键词:limiting their diagnostic, predominantly designed, designed around diffusion, mitigation methods, cs.CV
备注:
点击查看摘要
Abstract:Hallucination benchmarks for text-to-image generation are predominantly designed around diffusion models, limiting their diagnostic value for visual autoregressive (VAR) models and the development of mitigation methods. We identify two limitations in existing prompt construction paradigms: many prompts are no longer challenging for modern generators, while category-based construction relies on recurring object-attribute-relation templates and rarely tests interactions among multiple visual requirements within a single image. We introduce InfiHall-Bench, a controlled benchmark of 411 prompts constructed by systematically composing five foundational and visually verifiable skills: attribute binding, spatial relation, action relation, part consistency, and scene consistency. Each prompt covers 4.05 annotated skills on average, enabling the evaluation of hallucinations under diverse combinations of visual requirements. Using this benchmark, we find that hallucinations in VAR models exhibit a scale-wise persistence pattern: semantic errors can emerge at different autoregressive scales and are rarely corrected by subsequent fine-scale steps. Motivated by this observation, we propose Confidence-Masked Hallucination Mitigation (CMHM), which estimates the prediction confidence of target visual tokens at each autoregressive scale during teacher-forced fine-tuning. CMHM downweights reliably predicted tokens, allowing optimization to focus on uncertain predictions across scales. Experiments show that InfiHall-Bench distinguishes models that existing compositional benchmarks rate as comparable, while CMHM consistently outperforms uniform LoRA fine-tuning for VAR-based text-to-image generation.
Subjects:
Computer Vision and Pattern Recognition (cs.CV)
Cite as:
arXiv:2611.18053 [cs.CV]
(or
arXiv:2611.18053v1 [cs.CV] for this version)
https://doi.org/10.48550/arXiv.2611.18053
Focus to learn more
arXiv-issued DOI via DataCite (pending registration)</p>
36. 【2611.18033】Multi-level latent-diffusion adaptation for cross-region point cloud segmentation of ancient terraces
链接:https://arxiv.org/abs/2611.18033
作者:Sina Mohammadi,Roderik Lindenbergh,Javier Ruiz-Pérez,Marco Madella,Sebastian Fajardo
类目:Computer Vision and Pattern Recognition (cs.CV)
关键词:Automated lidar feature, acquisition characteristics, Automated lidar, differences in terrain, Embedding-space Latent Diffusion
备注: 33 pages, 6 figures
点击查看摘要
Abstract:Automated lidar feature detection from lidar point clouds can support archaeological surveys, but models trained in one landscape may transfer poorly to another because of differences in terrain, vegetation, acquisition characteristics, and feature morphology. We address this problem for cross-region detection of ancient terraces using transductive unsupervised domain adaptation that exploits an unlabelled target survey. We propose MELD (Multi-level Embedding-space Latent Diffusion), which trains separate diffusion denoisers on target-domain representations at four hierarchical PointNet++ feature levels. These target distribution priors are then used to refine source representations during supervised training and target representations during inference, allowing adaptation to act directly on the multi-scale features consumed by the segmentation decoder. We additionally investigate local geometric descriptors derived from K-nearest-neighbour neighbourhoods, including surface normals, slope, and aspect. MELD is evaluated bidirectionally between the Teyuna and Miraflores lidar surveys in the Sierra Nevada de Santa Marta, Colombia, against a source-only model and nine configurations of established domain adaptation approaches. MELD raises F1 over the source-only model from 0.397 to 0.459 for Teyuna-to-Miraflores transfer and from 0.557 to 0.594 in the reverse direction, achieving the highest F1 among the evaluated adaptation methods in both directions. Adding local geometry further raises F1 to 0.542 and 0.606, respectively. Multi-level ablation shows that refining all four PointNet++ levels provides the strongest performance in both directions. These results show that explicit modelling of target feature distributions in hierarchical point cloud feature space, complemented by local geometry, provides an effective route to cross-region archaeological mapping without target labels.
37. 【2611.18024】JaRS: Contrast-Free Localization of Memorized Regions in Diffusion Models via Jacobian Row Sums
链接:https://arxiv.org/abs/2611.18024
作者:Saisuchith Mahajan,Aditey Nandan,Rohit Singh,Nishanth Shetty,Chandra Sekhar Seelamantula
类目:Computer Vision and Pattern Recognition (cs.CV); Machine Learning (cs.LG)
关键词:training images, raising privacy, copyright concerns, memorize and reproduce, privacy and copyright
备注:
点击查看摘要
Abstract:Diffusion models sometimes memorize and reproduce regions of their training images, raising privacy and copyright concerns. We aim to localize these regions. Existing contrast-based methods use the diagonal of the denoiser Jacobian, which reflects the posterior variance of each pixel. However, a small diagonal entry is ambiguous: it signals either memorization or averaging over many neighboring pixels in smooth or predictable regions. To resolve this ambiguity, these methods subtract a reference map, which requires a second evaluation with a null prompt or an earlier checkpoint. Unconditional models have no null prompt, and earlier checkpoints are unavailable in general. We show that Jacobian row sums (JaRS), unlike the diagonal, separate memorization from local averaging without a reference map. Pairing JaRS with an image-level detection score that uses the same statistic at an early denoising step yields JaRS+. Across Stable Diffusion v1.4 and v2.1, Realistic Vision v5.1, and an unconditional diffusion Transformer (DiT), JaRS+ outperforms contrast-based localization methods.
38. 【2611.18023】RSSA: Robust Semantic and Spatial Aligner for Collaborative Perception
链接:https://arxiv.org/abs/2611.18023
作者:Zhihao Yang,Zhiyu Xiang,Peng Xu,Tianyu Pu,Kai Wang,Eryun Liu,Dongping Zhang,Yong Ding
类目:Computer Vision and Pattern Recognition (cs.CV)
关键词:aggregating multi-view features, improves single-vehicle perception, perception improves single-vehicle, collaborative perception improves, single-vehicle perception
备注: Accepted to NeurIPS 2026. 18 pages, 14 figures, 11 tables. Code: [this https URL](https://github.com/Megalo-f/RSSA)
点击查看摘要
Abstract:V2X collaborative perception improves single-vehicle perception by aggregating multi-view features from collaborative agents. However, existing methods typically rely on simple spatial warping to align the collaborative features, which can cause semantic misalignment during fusion. Meanwhile, these deterministic warping-based methods are highly sensitive to the positional noises. In this paper, we propose a plug-in module called Robust Semantic and Spatial Aligner (RSSA), which performs semantic and probabilistic spatial transformations for robust feature alignment. Specifically, a Semantic Transformation (SemT) block modulates feature channels based on the observed relative pose to correct semantic inconsistencies. Subsequently, a Probabilistic Spatial Transformation (P-SpaT) block performs probabilistic warping by sampling from the Gaussian-modeled relative pose space. A Relative Transformation Augmentation (RT-Aug) strategy is further introduced to augment the relative poses and facilitate the training of the RSSA. Extensive experiments on both simulated and real-world datasets demonstrate that RSSA improves the state-of-the-art methods by a large margin under the configuration of both clean and noisy positions.
39. 【2611.18006】RecurIR: Learning When and How to Revise Restoration Policies via Recursive Self-Improvement
链接:https://arxiv.org/abs/2611.18006
作者:Hu Gao,Haoran Zhang,Yulong Chen,Lizhuang Ma
类目:Computer Vision and Pattern Recognition (cs.CV)
关键词:Agent-based image restoration, achieved promising performance, Agent-based image, achieved promising, promising performance
备注:
点击查看摘要
Abstract:Agent-based image restoration has achieved promising performance by coordinating restoration tools to handle heterogeneous and coupled degradations. However, existing agents often improve their decisions through accumulated successful and failed executions, without explicitly identifying the conditions under which learned restoration strategies remain effective. Consequently, changes in degradation severity, spatial coverage, or interactions may invalidate previous decisions, while providing limited guidance on how these decisions should be revised. In this paper, we propose \textbf{RecurIR}, a recursive self-improvement framework that enables restoration agents to learn when and how to revise their decision rules. Specifically, we introduce a policy-hypothesis discovery mechanism to identify testable, condition-dependent rules through matched tool executions. We then design a rule-falsification-driven task evolution mechanism to generate targeted degradation variations, examine the validity of learned rules, and identify their applicability boundaries. Based on the resulting evidence, we develop a change-conditioned policy revision mechanism that learns how variations in degradation conditions should modify tool selection, execution order, operation strength, and processing regions. We further incorporate independent verification to retain effective revisions and use the resulting feedback to guide subsequent hypothesis discovery and task evolution. Through this recursive process, RecurIR progressively refines its restoration policies as degradation conditions change. Extensive experiments demonstrate the effectiveness of the proposed framework.
40. 【2611.17966】From Images to Grounded Edits: A Real-World Benchmark for Chinese Grammatical Error Correction in Handwritten Essays
链接:https://arxiv.org/abs/2611.17966
作者:Weiru Zhang,Yuzhong Hong,Junwei Bao,Lemao Liu,Dejing Dou
类目:Computer Vision and Pattern Recognition (cs.CV); Artificial Intelligence (cs.AI)
关键词:Grammatical Error Correction, Chinese Grammatical Error, Grounded Chinese Grammatical, Chinese Grammatical, Grammatical Error
备注:
点击查看摘要
Abstract:Chinese Grammatical Error Correction (CGEC) is typically formulated as a text-to-text problem. While suitable for digital texts, this formulation falls short in real-world K-12 education, where AI-assisted corrections are expected directly on essay images, which are primarily handwritten and exhibit many irregularities. To bridge this gap, we introduce Grounded Chinese Grammatical Error Correction (G-CGEC), an image-to-grounded-edit task requiring systems to predict both corrections and their positions on essay images. We propose G-CGEC-Eval, a benchmark comprising real-world images of K-12 Chinese essays with fine-grained annotations. For evaluation, we adapt IoU-based metrics from object detection and propose a more tolerant alternative based on center-distance matching. Using this benchmark, we evaluate open-source and proprietary vision-language models under a direct single-pass setting, as well as settings with auxiliary components (OCR and character grounding) coordinated via pipelining or agentic harnessing. Results show that even top-tier proprietary models perform surprisingly poorly in the single-pass setting, while pipelining and agentic harnessing yield meaningful improvements. Notably, an error-type-based multi-agent workflow achieves the best performance, establishing it as the strongest baseline for future G-CGEC research. Through ablations and fine-grained analysis, we find the task remains far from solved, with linguistic error discovery and character-level grounding as the primary bottlenecks. This underscores the need for models that better integrate visual perception with linguistic reasoning and visual grounding.
41. 【2611.17957】MacroGS: Macro-Pixel Shared Gaussian Primitives for Arbitrary-Scale Light Field Super-Resolutio
链接:https://arxiv.org/abs/2611.17957
作者:Zeqiang Wei,Angulia Yang,Min Xu
类目:Computer Vision and Pattern Recognition (cs.CV)
关键词:low-resolution multi-view observations, field spatial super-resolution, spatial super-resolution reconstructs, super-resolution reconstructs high-resolution, reconstructs high-resolution views
备注: 11 pages, 5 figures
点击查看摘要
Abstract:Arbitrary-scale light field spatial super-resolution reconstructs high-resolution views at any requested scale from low-resolution multi-view observations. Gaussian primitives enable continuous reconstruction, but generating separate primitive sets for each view redundantly encodes shared scene structure across views and causes the primitive count grow linearly with the number of views. To avoid such redundancy, we propose MacroGS, a feed-forward decoder that jointly predicts a single shared Gaussian set per macro pixel rather than generating separate sets for each view. Disparity-dependent replacement and angle-conditioned appearance allow each set to be reused across all output views, keeping the predicted primitive count independent of the number of views at each requested scale. The decoder iteratively predicts per-primitive disparities, retrieves the corresponding multi-view features, and updates the primitive states. Gaussian splatting and residual refinement then reconstruct the output light field. Despite being learned without geometric supervision, the predicted disparities strongly correlate with underlying scene geometry. Experiments across five benchmarks demonstrate that MacroGS integrates effectively with diverse light field encoders, improving reconstruction quality at fixed scales while achieving leading performance across most evaluated arbitrary-scale settings. The decoder also achieves speedups of approximately $2$-$10\times$ over state-of-the-art implicit light field decoders and $10$-$18\times$ over the view-wise Gaussian baseline.
42. 【2611.17956】nnAEU: a Simple and Efficient CNN-Autoencoder for Hyperspectral Unmixing
链接:https://arxiv.org/abs/2611.17956
作者:Edgard Dabier,Christophe Kervazo,Pietro Gori,Florence Tupin
类目:Computer Vision and Pattern Recognition (cs.CV)
关键词:Deep learning-based hyperspectral, increasing architectural complexity, learning-based hyperspectral unmixing, Mamba-based models, Deep learning-based
备注:
点击查看摘要
Abstract:Deep learning-based hyperspectral unmixing (HSU) has followed a trajectory of increasing architectural complexity, from simple autoencoders to transformers and, most recently, Mamba-based models, each claiming to advance the state of the art. Contrary to this trend, we propose a lightweight CNN-based autoencoder, coined "Not New Autoencoder Unmixing" (nnAEU). It is derived from the backbone of UNMamba (2025) through a pruning study that removes the Mamba component entirely. nnAEU uses less than 10 times fewer parameters than UNMamba, and far fewer than other transformer- and GAN-based methods, while outperforming them in accuracy. We provide a complete, minimal, reproducible code, available at this https URL.
43. 【2611.17952】Domain Adaptation Based Pipeline for Character Classification and Handwritten Text Recognition
链接:https://arxiv.org/abs/2611.17952
作者:Florent Imbert,Simon Corbillé,Hui Han,Elisa H. Barney Smith
类目:Computer Vision and Pattern Recognition (cs.CV); Machine Learning (cs.LG)
关键词:Character classification, data remains unused, limited labeled data, remains unused, unlabeled data remains
备注:
点击查看摘要
Abstract:Character classification and handwritten text recognition, particularly in specialized tasks, often suffer from limited labeled data, while a large amount of unlabeled data remains unused. To address this challenge, we propose a novel unsupervised domain adaptation-based pipeline to reduce the distribution gap between source and target domains. Our pipeline follows three main steps. In the first step, we pretrained an expert model on source data, to learn a strong general representation. In the second step, we perform unsupervised domain adaptation to leverage unlabeled target data, adapting the model to the specific characteristics of the target domain. In the third step, we trained the expert model on source data in the target style. By aligning the feature distributions between the source and target, our method enables the model to generalize effectively across diverse datasets. The unsupervised training increases the computation time only during training not in inference. We evaluate the pipeline on both character-level and historical handwritten line recognition tasks, demonstrating its flexibility. We establish a new state-of-the-art on the Safran-MNIST-DLS dataset defined in Track 2 of the DAGECC competition with 76.17% Macro F1-score and achieve competitive results on each of the specific datasets of READ 2018 without using the labels of the target data. Code is available at this https URL
44. 【2611.17938】Measured from the feed: the unsustainable growth of AI video generation
链接:https://arxiv.org/abs/2611.17938
作者:Tristan Coignion,Aurélie Bugeau,Gael Guennebaud,Anne-Laure Ligozat
类目:Artificial Intelligence (cs.AI); Computer Vision and Pattern Recognition (cs.CV); Computers and Society (cs.CY); Emerging Technologies (cs.ET)
关键词:broader environmental impacts, environmental impacts remain, broader environmental, environmental impacts, commercial scale
备注: Main paper + Supplementary paper Under submission at a journal
点击查看摘要
Abstract:Text-to-video generation runs at commercial scale, but its broader environmental impacts remain, to our knowledge, understudied. Providers rarely disclose video volumes or energy needs; we measured the yearly video output of two commercial platforms and built a prospective model of such services' absolute footprint from training to storage. Our projections suggest that reducing emissions depends on both limiting growth in video generation and achieving substantial efficiency gains. Under continued growth, emissions rise nearly seventy-five-fold by 2030 despite technological improvements. When usage remains at 2026 levels, the same improvements reduce emissions by about two-fifths. These estimates are a conservative lower bound: they cover one AI service among many; other services and uses would push the total higher.
45. 【2611.17930】oken2Tile: Token-Wise Sparse Attention with Tile Execution through a Low-Rank Space for Video DiTs
链接:https://arxiv.org/abs/2611.17930
作者:Xuecheng Wu,Aichen Feng,Jinyan Chen,Qingjie Zhao,Julien Lai
类目:Computer Vision and Pattern Recognition (cs.CV)
关键词:video diffusion transformers, major inference bottleneck, modeling spatio-temporal dependencies, quadratic cost makes, diffusion transformers
备注: 21 pages, 14 figures, 23 tables
点击查看摘要
Abstract:Attention is essential for modeling spatio-temporal dependencies in video diffusion transformers, but its quadratic cost makes it a major inference bottleneck. Its high sparsity offers an opportunity to accelerate generation by focusing computation on important interactions. However, accuracy and execution favor different granularities: each query attends to its own key tokens, whereas GPU kernels work on tiles. We present Token2Tile, a training-free sparse attention operator that combines per-query scoring and a per-query residual with tile execution. An online low-rank space, computed directly from queries and keys to preserve attention logits, makes clustering, scoring, and residual approximation efficient. Token2Tile unites query-specific selections to allocate exact work across query clusters, then packs the selected keys into GPU tiles. On MiniMax-H3 and Wan2.1-14B, Token2Tile delivers 1.48-1.54x end-to-end speedup over dense FlashAttention-3 with VBench scores within 0.5 points of dense. At comparable speeds on H3, it improves PSNR over SOL and SubBlock by 5.5-6.8 dB; on Wan, it exceeds SOL by 4.6-4.8 dB while running faster.
46. 【2611.17919】Vessel2Brain: Edge-Aware Multi-Scale Flow Matching for 3D Brain Image Generation from Vessel Masks
链接:https://arxiv.org/abs/2611.17919
作者:Zhi Chen,Tianyang Zhang,Theodoros N. Arvanitis,Le Zhang
类目:Computer Vision and Pattern Recognition (cs.CV)
关键词:clinically valuable problem, downstream neuroimaging analysis, Generating realistic, brain MRI volumes, brain MRI
备注:
点击查看摘要
Abstract:Generating realistic 3D brain MRI volumes from sparse anatomical priors is a highly ill-posed yet clinically valuable problem, with important applications in data augmentation, simulation, and downstream neuroimaging analysis. In this work, we investigate the challenging task of synthesising full 3D brain MRI volumes conditioned solely on vascular label maps. While vascular structures provide strong anatomical and topological cues, they occupy only a small fraction of the brain volume and offer no direct information about surrounding tissue appearance. To address this challenge, we propose Vessel2Brain, an edge-aware multi-scale conditional flow matching framework that explicitly incorporates vascular structure information into the generative process. Our approach formulates vessel-guided brain image synthesis as a conditional continuous flow, enabling stable training and flexible generation while preserving structural correspondence. A multi-scale flow matching strategy is introduced to jointly model global anatomical coherence and fine-grained local details, with auxiliary velocity predictions providing additional supervision across spatial resolutions. Furthermore, we design an edge-aware objective that emphasises vascular boundaries during training, encouraging anatomically faithful tissue synthesis around vessel structures. We evaluate Vessel2Brain on two public 3D brain MRI datasets with paired vascular annotations. Extensive quantitative and qualitative experiments demonstrate that our method consistently outperforms state-of-the-art generative models in terms of reconstruction accuracy, structural consistency, and distributional realism.
47. 【2611.17909】CardioTrace: Motion-Aware Cine Synthesis via Coupled Tracing and Segmentation
链接:https://arxiv.org/abs/2611.17909
作者:Zhi Chen,Le Zhang
类目:Computer Vision and Pattern Recognition (cs.CV)
关键词:Cardiac cine MRI, evaluating ventricular function, function and anatomy, essential for evaluating, Cardiac cine
备注:
点击查看摘要
Abstract:Cardiac cine MRI provides time-resolved visualization of cardiac motion and is essential for evaluating ventricular function and anatomy. In routine clinical acquisition, however, cine sequences are frequently temporally sparse or degraded by arrhythmia, motion, and scan-time constraints, which can hinder reliable quantification and downstream analysis. We present CardioTrace, a motion-aware cine synthesis framework that generates a full cardiac cine sequence from limited input while explicitly modeling and tracing cardiac motion. CardioTrace couples two complementary pathways: a pixel-translation branch that directly predicts cine frames in a residual formulation, and a flow-field branch that estimates time-resolved deformation fields from a reference phase to provide interpretable motion correspondences across frames. To further preserve anatomical plausibility, a segmentation branch supervises clinically meaningful cardiac structures at key phases, and a cross-branch consistency objective encourages the synthesized appearance dynamics to follow the explicit motion guidance, mitigating temporal drift and unrealistic shape transitions over long sequences. The framework also supports prompt-style conditioning to incorporate auxiliary patient or acquisition information, improving robustness across heterogeneous multi-center data. Experiments on public cardiac cine MRI datasets demonstrate that CardioTrace improves perceptual quality and temporal coherence over representative GAN- and diffusion-based baselines, while better preserving ventricular morphology throughout the cardiac cycle. \textit{Our codes are available after review}.
48. 【2611.17908】DiffusionSSD: Achieving Extreme Visual Compression with Dual-stage Diffusion and Cross-level Near-Data Processing
链接:https://arxiv.org/abs/2611.17908
作者:Kuai Yu,Wen Li,Shengwen Liang,Jie Zhang,Ying Wang,Huawei Li,Xin Yao,Renhai Chen,Gong Zhang
类目:Hardware Architecture (cs.AR); Computer Vision and Pattern Recognition (cs.CV); Image and Video Processing (eess.IV)
关键词:making compression essential, Growing visual data, Growing visual, resource-constrained edge devices, diffusion
备注: 15 pages, 19 figures
点击查看摘要
Abstract:Growing visual data overwhelms storage on resource-constrained edge devices, making compression essential. Traditional codecs offer inadequate rate-distortion performance, motivating the exploration of diffusion codecs. Diffusion codecs face two limitations: (1) At the algorithm level, diffusion decoders preserve perceptual quality under extreme compression, but require substantial computation. (2) At the architecture level, in NPU-centered devices, neural codecs occupy the NPU, interfering with other AI applications. Meanwhile, the complex codec pipeline incurs long I/O paths, leading to additional overhead and energy cost. To address these issues, we propose DiffusionSSD, a cross-level near-data processing framework for extreme visual data compression on edge devices, using an algorithm-architecture co-design method. Specifically, (1) at algorithm level, we present Dual-stage Diffusion, a lightweight solution tailored for diffusion codecs that employs patchification and single-step diffusion, reducing computation cost while preserving perceptual quality; (2) at architecture level, we introduce a Cross-level Near Data Processing architecture that integrates the proposed computational universal flash storage (cUFS) and LPDDR-PIM, enabling codecs to be entirely offloaded to data side while reducing visual-data movement and avoiding SoC redesign; (3) at system level, combining both the algorithm and the underlying architecture, we propose a Patch-Workload co-aware scheduling scheme to enable more efficient system-level execution. Experiments show that Dual-stage Diffusion maintains comparable perceptual quality while reducing computation cost by up to 92.7%. Compared with NPU+DRAM and near-data-processing design such as SmartSSD, cUFS+PIM improves coding throughput by 2.32 times and 2.91 times on average, respectively.
Comments:
15 pages, 19 figures
Subjects:
Hardware Architecture (cs.AR); Computer Vision and Pattern Recognition (cs.CV); Image and Video Processing (eess.IV)
Cite as:
arXiv:2611.17908 [cs.AR]
(or
arXiv:2611.17908v1 [cs.AR] for this version)
https://doi.org/10.48550/arXiv.2611.17908
Focus to learn more
arXiv-issued DOI via DataCite (pending registration)</p>
49. 【2611.17905】Cracks in Embeddings: Robustness and Stability of JEPA-Based Chest X-Ray Representations
链接:https://arxiv.org/abs/2611.17905
作者:Zahid Mansur,Kalpana Yadav,Deepak Mishra
类目:Computer Vision and Pattern Recognition (cs.CV)
关键词:Self-supervised Joint Embedding, Embedding Predictive Architecture, Joint Embedding Predictive, Self-supervised Joint, Predictive Architecture
备注:
点击查看摘要
Abstract:Self-supervised Joint Embedding Predictive Architecture (JEPA) models are increasingly used for medical imaging, however there is a limited understanding about robustness of these models toward real-world image changes. We test four JEPA-based chest X-ray encoders, trained on 78,468 NIH ChestX-ray14 images, under six tests: two black-box adversarial attacks (SPSA, Boundary Attack), three image corruptions (Gaussian noise, brightness shift, contrast shift), and a label-scarcity test, measuring embedding change and classification AUROC. These perturbations cause severe, model-dependent damage: encoder-level Cosine Similarity drops as low as 0.33, and AUROC falls by as much as 0.12 in the worst case (I-JEPA under the Boundary Test). We test two ways of training the small predictor on top of each frozen encoder to address this. Adversarial fine-tuning, trained on clean and perturbed images together, raises embedding consistency above 0.97 in nearly every case and improves AUROC, relative to the same perturbed baseline, for most models, with the largest gain of $+0.10$ on I-JEPA. A Vector-Quantized codebook method, trained only on clean images and evaluated on perturbed test images for the first time, instead degrades under distortion for all four models, most noticeably for I-JEPA ($-0.15$ AUROC on average), showing that its discrete bottleneck alone does not provide the robustness that comes from training on perturbed data directly.
50. 【2611.17903】Zoom In, Scale Out: Metric Depth Models Mistake Digital Zoom for Moving Closer
链接:https://arxiv.org/abs/2611.17903
作者:Rabeya Noor,Khandoker Nosiba Arifin
类目:Computer Vision and Pattern Recognition (cs.CV)
关键词:alpha, depth, Depth Pro, photo, camera
备注: 22 pages including supplementary material. Code: [this https URL](https://github.com/rabeyanoor/zoom-equivariance)
点击查看摘要
Abstract:Many new AI models estimate how far away things are, in metres, from a single photo. To do this they must also work out what camera took the photo. We check whether they really do with a simple test: digital zoom, that is, cropping a photo and enlarging the crop back to full size. Zooming moves nothing in the scene, so a correct model should give the same distance for every point still in the picture. The test is free, exact and needs no labels. Two numbers describe a model's reaction: its depth changes like $s^{-\alpha}$ (ideal $\alpha=0$) and its focal-length estimate like $s^{\beta}$ (ideal $\beta=1$), for zoom factor $s$. On 800 images from KITTI, NYUv2 and DIODE, the widely used Depth Anything V2 has $\alpha=0.95$: it treats a zoomed photo almost as if the camera had moved closer, because it judges distance mainly from how large things look. Models that also predict the camera remove only part of this error ($\alpha\approx0.31$-$0.36$), because their focal estimate grows too little. Given the true camera, Depth Pro and DA3-Nested keep their depth ($\alpha\approx0$), so the error lies in their camera estimate. The reason: a zoomed photo is, pixel for pixel, a normal photo of a scene squashed in depth, so size alone cannot tell the two apart, and a focal estimator trained by regression stays close to the cameras it saw in training. Undoing this needs no labels or training and makes Depth Pro pass the test ($\alpha=0.04$) at almost no cost on normal photos. Controls, including a real optical zoom lens, rule out blur and resizing artefacts. Finally, label-free fine-tuning for self-consistency lowers $\alpha$ to 0.05 on KITTI and 0.07 on NYUv2 once a "blurry means zoomed" shortcut is removed; we report what this costs on normal and unseen photos.
51. 【2611.17900】AmbiSteer: Selectively Eliciting Ambiguity-Aware Responses in Vision-Language Models
链接:https://arxiv.org/abs/2611.17900
作者:Monica Riedler,Samuele Antonelli,Barbara Plank,Raffaella Bernardi
类目:Computer Vision and Pattern Recognition (cs.CV); Computation and Language (cs.CL)
关键词:providing firm answers, Visual Question Answering, providing firm, evidence is insufficient, Vision-language models
备注: 23 pages, 23 figures
点击查看摘要
Abstract:Vision-language models (VLMs) often exhibit overconfidence, providing firm answers even when visual evidence is insufficient. A prominent case is referential ambiguity in Visual Question Answering (VQA), where multiple entities in an image are plausible referents of a question. Addressing such cases requires recognizing when ambiguity is present: ambiguous questions can be handled by asking for clarification or enumerating plausible referents, while unambiguous questions should receive a direct answer. Current VLMs struggle with this distinction, often defaulting to a single interpretation. To investigate whether activation steering can serve as an inference-time mechanism to elicit such behaviors, we introduce AmbiSteer, a lightweight framework to selectively elicit ambiguity-aware behavior. Across three VLMs, we find that steering alone has limited and model-dependent effects, while prompting is generally more effective. Combining prompting and steering substantially increases clarification and enumeration behaviors, but also elicits them on unambiguous cases. AmbiSteer addresses this using a linear classifier to predict input ambiguity and condition the steering direction accordingly, increasing the proportion of inputs handled appropriately from 59.0% to 77.0% on average across models. Our work sets the ground for lightweight approaches to reduce model overconfidence in uncertain multimodal settings.
52. 【2611.17899】More Genes, Faster Inference: Efficient Large-Panel Spatial Transcriptomics Prediction from Histology
链接:https://arxiv.org/abs/2611.17899
作者:Songming Zhang,Yuxiao Luo,Shifu Luo,Siran Chen,Shuquan Su,Jinyan Li
类目:Computer Vision and Pattern Recognition (cs.CV); Artificial Intelligence (cs.AI); Quantitative Methods (q-bio.QM)
关键词:infer spatially resolved, spatially resolved gene, aims to infer, infer spatially, spatially resolved
备注: Accepted at KDD 2027
点击查看摘要
Abstract:Spatial transcriptomics (ST) prediction from histology aims to infer spatially resolved gene expression profiles directly from tissue images. This is a challenging problem that is currently often addressed with complicated model architectures, additional inputs, and complicated training procedures, while remaining limited to fixed, small gene panels. This raises a natural question: Is there an efficient method for large-panel ST prediction? Here, we present GeneFlow, a simpler yet effective framework for large-scale spatial gene expression prediction that unifies flow-matching generative modeling with structured multimodal fusion. Specifically, GeneFlow first encodes both histological morphology and cross-gene interactions in spatial spots through histology and gene tokenizers. The model directly predicts spatial gene expression, rather than an intermediate velocity field, while being trained with a velocity-based flow-matching objective. We show that, even without relying on heavy generative pipelines, GeneFlow achieves state-of-the-art performance across multiple ST benchmarks, with up to 56.8% gains in 1,000-gene prediction. Extensive experiments further show that GeneFlow produces biologically meaningful spot profiles while reducing computational cost by orders of magnitude, thereby improving efficiency and scalability, particularly in clinical settings. We believe GeneFlow provides an opportunity for leveraging widely available histology images for downstream biological discovery. Code is available at: this https URL
53. 【2611.17897】See and Detect: Image-Driven Open-Vocabulary Multi-View 3D Object Detection
链接:https://arxiv.org/abs/2611.17897
作者:Yating Xu,Na Zhao,Chenhan Jiang,Qingwen Zhang,Jiankang Deng,Jifei Song
类目:Computer Vision and Pattern Recognition (cs.CV)
关键词:Recent open-vocabulary multi-view, Recent open-vocabulary, semantic label generation, methods typically follow, projected regions
备注: tech report
点击查看摘要
Abstract:Recent open-vocabulary multi-view indoor 3D detection methods typically follow a point-cloud-driven pipeline, where pseudo 3D boxes are generated from reconstructed or estimated geometry and semantic labels are inferred from projected regions. However, such a design makes box generation dependent on the completeness of point clouds and restricts semantic learning to local appearance, causing inferior 3D object localization and open-vocabulary classification. In this paper, we present See and Detect, an image-driven framework that uses multi-view images as the primary evidence for 3D box and semantic label generation. For box generation, we first discover object candidates from individual images, enabling objects missing in point clouds to be recovered from visual evidence. However, single-view predictions are often noisy and redundant. To address this, we propose multi-view consensus box consolidation to fuse spatially consistent predictions across views, and box quality assessment to reduce the impact of boxes with partial object coverage. For semantic label generation, we introduce context-aware semantic learning, where semantic labels are queried from full image context, enabling more accurate semantic prediction. Extensive experiments on ScanNet and ARKitScenes demonstrate that our method achieves state-of-the-art open-vocabulary multi-view 3D detection performance while running over 500$\times$ faster than the previous state-of-the-art method.
54. 【2611.17881】Photogrammetric 3D Model-Based Animal Behavior Classification
链接:https://arxiv.org/abs/2611.17881
作者:Olaf Hellwich,Moritz Heine,Patrik Reiske,Sole Traverso,Katharina Hohlbaum,Henning Sprekeler,Christa Thöne-Reineke,Lars Lewejohann,Roman Bhuiyan
类目:Computer Vision and Pattern Recognition (cs.CV); Artificial Intelligence (cs.AI)
关键词:Video-based behavior estimation, Video-based behavior, image sequences, Video-based, behavior estimation
备注: 8 pages, 9 figures, 1 table, conference proceedings
点击查看摘要
Abstract:Video-based behavior estimation is often implemented end-to-end from image sequences. We advocate a two-stage alternative: reconstruct 3D body and object geometry before classifying behavior. This makes ambiguities of the ill-posed 2D-to-3D inverse problem explicit and potentially correctable. We demonstrate the strategy with simulated interacting mice and extend it to real video.
55. 【2611.17878】DySENSE: Dynamic Spatiotemporal ENergy Synthesis and Emulation with Satellite Imagery
链接:https://arxiv.org/abs/2611.17878
作者:Kailai Sun,Zijie Gao,Mingyi He,Can Rong,Alok Prakash,Baoshen Guo,Shenhao Wang,Jinhua Zhao
类目:Computer Vision and Pattern Recognition (cs.CV); Artificial Intelligence (cs.AI)
关键词:Sustainable Development Goals, carbon dioxide emissions, global carbon dioxide, energy, Development Goals
备注: Accpted by KDD 2027
点击查看摘要
Abstract:Cities account for over two-thirds of the world's energy and more than 70% of global carbon dioxide emissions. City energy is essential for achieving the 2050 net-zero target and the UN's Sustainable Development Goals 7 and 11. Although existing studies have made progress, they are limited by data and methodological gaps. Existing studies are largely predictive for existing cities rather than generative for future cities; fine-grained, aligned monthly dynamic energy data are scarce; and existing GenAI models struggle to generate dynamic urban functional monthly energy. We present DySENSE, a joint multimodal GenAI framework that synthesizes static urban satellite imagery together with spatially aligned, dynamic monthly building energy consumption maps. We decouple long-term static urban morphology from short-term dynamic energy consumption. By conditioning on static urban land use, DySENSE can generate realistic, diverse and new static urban imagery with aligned energy maps. By conditioning on dynamic monthly and climate prompts, DySENSE can jointly generate dynamic, monthly energy consumption maps while keeping the urban morphology static. Experiments across four cities (New York, Washington, Norfolk, Busan) demonstrated that DySENSE can generate annotated high-fidelity synthetic energy data using less than 25% labelled data, boosting downstream prediction performance by 8% mIoU, and achieving strong energy dynamic consistency. Compared to existing methods, DySENSE significantly improved prediction performance (8.7% accuracy) and reduced prediction error (9%-18% CVRMSE, ASHRAE). DySENSE offers an effective foundation for energy-efficient cities, peak-load analysis, building performance simulation and dynamic demand response for city, energy, and building science. The data and code: this https URL and this https URL
56. 【2611.17868】Speed-Variation Distillation for Generative Modeling
链接:https://arxiv.org/abs/2611.17868
作者:Sukit Seripanitkarn,Phonphrm Thawatdamrongkit,Supasorn Suwajanakorn
类目:Computer Vision and Pattern Recognition (cs.CV)
关键词:diffusion models excel, denoising trajectories pass, corrupted intermediate states, diffusion models, models excel
备注: CDEL-ECCVW 2026 proceedings. Project page: [this https URL](https://speed-variance-modeling.github.io)
点击查看摘要
Abstract:While diffusion models excel at image generation, their denoising trajectories pass through corrupted intermediate states far from the image manifold. We ask whether generation can instead proceed through recognizable images throughout the entire trajectory. We introduce a generative framework built around a variation-reducing degradation process that contracts diverse images toward a shared prototype while preserving recognizable intermediate states. Because such manifold traversal lacks a simple analytical form, we construct the trajectories using generative-prior-based image morphing. We then distill the reverse process into a diversification operator that transforms the prototype into increasingly varied samples. To resolve the one-to-many ambiguity of the reverse process, we associate each trajectory with its initial noise latent and apply perturbations to improve robustness against accumulated errors. At inference, every step yields a recognizable image, allowing early termination and enabling a controllable speed--variation trade-off.
57. 【2611.17853】DoubleTake: Focused Verification of Visual Evidence for Long-Document Understanding
链接:https://arxiv.org/abs/2611.17853
作者:Kejun Xue,Ang Li,Jinghua Tang,Xincheng Wang,Yongbin Gao,Yanghua Xiao,Zhijun Fang
类目:Computer Vision and Pattern Recognition (cs.CV)
关键词:long-document question answering, original question, question, vision-language model, retrieved pages
备注:
点击查看摘要
Abstract:In long-document question answering, a vision-language model can answer incorrectly even when the retrieved pages contain sufficient evidence. Rereading those pages with the original question gives the model little guidance about which fact to check. We introduce DoubleTake, a training-free framework for focused verification of visual evidence. An initial reading produces a candidate answer and a separate evidence record that guides retrieval for a second reading. A planner compares the two candidates and their supporting evidence to formulate a factual question. A verifier answers it from the source images without seeing either candidate or evidence record, and a reasoner uses this observation to resolve the original question. With a frozen Qwen3-VL-8B-Instruct backbone, DoubleTake achieves the highest overall accuracy among the evaluated methods on each of four benchmarks. With the same pages and number of model calls, asking a focused factual question yields higher accuracy on all four benchmarks than asking the original question again. Our code is available at this https URL.
58. 【2611.17852】PADSNet: Physics-Aware Dual-Stream Network with Biomechanical Priors for Laryngeal Lesion Classification
链接:https://arxiv.org/abs/2611.17852
作者:Zhenzhen You,Weifeng Yu,Zhenghao Shi,Shuangli Du,Minghua Zhao,Yingcheng Zhao,Xinhong Hei,Demin Kong,Xiaoyong Ren,Jin Hou
类目:Computer Vision and Pattern Recognition (cs.CV)
关键词:inter-class visual differences, Precise classification, highly challenging, laryngeal lesion classification, crucial yet highly
备注:
点击查看摘要
Abstract:Precise classification of laryngeal lesions is clinically crucial yet highly challenging, as inter-class visual differences are often subtle. While abnormalities in vocal fold vibratory dynamics provide complementary discriminative information, existing deep learning models predominantly rely on visual appearance and lack explicit mechanisms for modeling clinically meaningful vibratory dynamics. To address this, we propose PADSNet, a Physics-Aware Dual-Stream Network for laryngeal lesion classification that integrates visual representations with explicit biomechanical reasoning. Specifically, a Wavelet-Attention Enhancement Module (WAEM) is designed to bridge the spectral-semantic gap, enabling complementary learning of spatial texture and dynamic frequency representations. Meanwhile, a Knowledge Computing Module (KCM) extracts interpretable biomechanical indices from temporal glottal distance waveforms, including indicators of mobility, symmetry, glottal closure, and vibration stability. These descriptors are subsequently incorporated into a Physics-Aware Gating Module (PAGM) as biomechanical priors to adaptively recalibrate visual feature toward pathology-relevant vibratory patterns. Extensive experiments on a clinical dataset demonstrate that PADSNet achieves an accuracy of 0.8452, an F1-score of 0.8090, a sensitivity of 0.7953, and a specificity of 0.8866. These results indicate that explicitly integrating biomechanical priors with deep visual representations provides a more accurate, robust, and clinically interpretable framework for automated laryngeal lesion classification.
59. 【2611.17831】V-JEPA-LM: Video-Language Models That Read Dynamics Beyond Static, Text-Aligned Representations
链接:https://arxiv.org/abs/2611.17831
作者:Keuntae Kim,Dongjun Kim,Yong Suk Choi
类目:Computer Vision and Pattern Recognition (cs.CV); Artificial Intelligence (cs.AI)
关键词:World-model video encoders, World-model video, World-model, V-JEPA, V-JEPA learn
备注:
点击查看摘要
Abstract:World-model video encoders such as V-JEPA learn how scenes change (direction, speed, order, state) from video alone. To answer questions, they are joined to a language model trained on captions and question-answer pairs. But each caption or question names only a few attributes of a clip, so the model learns to retrieve those attributes rather than to read the clip, and the dynamics the encoder carries go unused: on the same items, a linear probe reads speed from frozen V-JEPA features at 87% while the VLM answers at chance. We show where dynamics lives in a frozen encoder: in a subspace shared across videos for predictive encoders such as V-JEPA, but not as a shared component of frame-wise, text-aligned encoders such as SigLIP. We then build counterfactual supervision that makes the language model read it: pixel transformations turn public videos into answer-balanced variants whose answers are set by the transformation and differ only in the change. Trained in a single stage on 4.53M public samples with 32 visual tokens per frame, where open video LLMs use several stages, 9M to 172M samples and 64 to 256 tokens per frame on average, the resulting V-JEPA-LM models (1B to 8B, Qwen3 and Llama) lead every size class in average score across eleven video benchmarks; our 4B model scores 72.1, against 61.1 for the best open 4B and 64.8 for the best open 8B model. A controlled encoder x supervision study shows that the encoder decides what can be read and the supervision what is read: at a matched budget, counterfactual supervision adds 4.5 points on average and 38 on the speed questions of natural benchmarks. Code and models: this https URL.
60. 【2611.17828】Signal Scaffolding: Unlocking the Potential of Lightweight Models for Self-Supervised Monocular Depth Estimation
链接:https://arxiv.org/abs/2611.17828
作者:Zhijie Wang,Li Zhang,Yujie Chen,Shibiao Deng,Xin Gao,Houzhi Zhou,Jiangshuo Zhang
类目:Computer Vision and Pattern Recognition (cs.CV)
关键词:self-supervised monocular depth, monocular depth estimation, methods generally suffer, lightweight self-supervised monocular, continued to evolve
备注:
点击查看摘要
Abstract:With the rising demand for edge devices, lightweight self-supervised monocular depth estimation methods have continued to evolve; however, existing methods generally suffer from poor accuracy. We find that the root cause may lies in the failure of conventional training regimes to fully exploit the inherent potential of lightweight models. To address this, we propose a novel three-phase training strategy for lightweight self-supervised monocular depth estimation, termed Signal Scaffolding (SS). Without introducing any additional dataset or parameters, we apply SS to ReForgeDepth, a lightweight network of our own design. Experiments on the KITTI and Cityscapes datasets demonstrate that our method achieves state-of-the-art performance in the accuracy-efficiency trade-off and running in real time on edge devices. These results suggest that lightweight models can be improved further within their existing capacity, and that the central question may be less about how parameters are allocated than about how to unlock this latent potential through training.
61. 【2611.17825】Beyond Endpoints: Bézier Paths for Vision Foundation Model Adaptation
链接:https://arxiv.org/abs/2611.17825
作者:Chenzhi Zheng,Xi Xiao,Chih-Ting Liao,Qizhen Lan,Jingwen Lin,Tianyang Wang,Hao Xu,Xin Cao
类目:Computer Vision and Pattern Recognition (cs.CV)
关键词:Frozen vision foundation, Frozen vision, vision foundation models, vision foundation, adapted by learning
备注: 47 pages
点击查看摘要
Abstract:Frozen vision foundation models can be adapted by learning a few prompt tokens shared across all images. When prompts are updated as data shift or tasks change, the distribution of image features moves from one endpoint to another. Prior theory asks which endpoints prompts can reach; we ask whether they can travel between them along the shortest path. Because shared prompts compete with each image's own content for attention, they often cannot. We quantify this restriction as the extra transport cost that shared attention imposes over the shortest path. In an exactly solvable attention model, the reachable features form a cone; unrolling it into a plane gives the exact minimum action for every pair of endpoints and explicit optimal prompts on short arcs. The analysis reveals a sharp threshold in content normalization, which measures how much attention each image's own tokens absorb: for same-sign endpoints, at most two distinct levels allow prompts to follow the shortest path, whereas three or more force a strictly positive extra cost unless the endpoints differ only by rescaling. This gap persists with more prompts, small content perturbations, and aligned multihead outputs. On ViT-B/16 with VTAB-1k, a quadratic Bézier prompt path that minimizes feature action reduces held-out transport action relative to linear and cross-entropy-trained paths between the same endpoints and improves intermediate and online accuracy under unlabeled distribution shift, while cross-entropy-trained paths retain higher minimum accuracy when labeled calibration data are available. Reachable endpoints do not guarantee a reachable shortest path: how prompts move matters, not only where they end. Project page: this https URL.
62. 【2611.17821】SceneAgent: A Multi-Agent Framework for Portable Game Scene Generation
链接:https://arxiv.org/abs/2611.17821
作者:Xingyi Li,Jiahao Yang,Yiqin Zhu,Jingwen Ye,Yang Sun,Weidong Zhang
类目:Computer Vision and Pattern Recognition (cs.CV); Artificial Intelligence (cs.AI)
关键词:game engine requires, final output, exporting its final, stored experience, Moving
备注: Project page: [this https URL](https://xingyi-li.github.io/sceneagent/)
点击查看摘要
Abstract:Moving a scene generator to a new game engine requires more than exporting its final output: planning, construction tools, and stored experience may also depend on the original backend. We present SceneAgent, a multi-agent framework for reusing the construction process and its experience across backends. A canonical SceneContext maintains the evolving world, while backend adapters isolate coordinate conventions, asset bindings, and export. Structured handoffs let fresh Builder sessions query this persistent state without inheriting the full construction transcript. Across generations, human-approved traces are distilled into asset-ID-free Layout skills and semantic Content skills that can be retrieved and recombined. We introduce GameSceneBench, comprising 69 indoor and outdoor prompts. Under a common asset and evaluation protocol, SceneAgent obtains human tie-adjusted preference scores of 78.2-89.4% against five baselines. Ablations show that reusable skills and the multi-agent protocol primarily improve prompt alignment and scene completeness, with additional runtime. Deployment in a second engine preserves the shared construction protocol and skill library while replacing backend integration components. Together, these results support separating persistent scene state, task-local reasoning, and reusable experience for portable scene construction.
63. 【2611.17797】Vision-as-a-Tool: Visual Harness for Multimodal Agentic Coding
链接:https://arxiv.org/abs/2611.17797
作者:Haolin Yang,Wenshan Wu,Jin Jiang,Yu Cheng,Yan Xia,Ting Song,Yangyu Huang,Zhiliang Peng,Wenhui Wang,Hangbo Bao,Yuzhe Zhang,Wen-Da Wei,Tengchao Lv,Li Dong,Lei Cui,Furu Wei
类目:Artificial Intelligence (cs.AI); Computer Vision and Pattern Recognition (cs.CV)
关键词:evaluate generated artifacts, requires robust visual, generated artifacts, coding requires robust, requires robust
备注: 27 pages, including references and appendices. Project page: [this https URL](https://vision-as-a-tool.github.io/)
点击查看摘要
Abstract:Multimodal agentic coding requires robust visual reasoning to analyze observations and evaluate generated artifacts. Current frameworks largely rely on a Thinking-with-Images paradigm, where agents depend on their intrinsic vision to execute visually grounded tasks. However, this architectural coupling introduces a fundamental bottleneck. Agent performance remains strictly bounded by internal perceptual limitations. Furthermore, text-only agents lack native perception. Without specialized tools, they cannot perform visual reasoning. To overcome these constraints, we introduce the Vision-as-a-Tool (VaaT) paradigm. Prior visual tool-use merely manipulates images for an agent's intrinsic vision to read. In contrast, VaaT reframes general-purpose multimodal large language models (MLLMs) as callable external utilities. As a result, text-only agents can completely offload visual reasoning during agentic execution. Meanwhile, vision-capable agents can dynamically augment their native perception on demand. We instantiate this paradigm in VHarness, a unified framework driven by five atomic actions. We comprehensively evaluate VHarness across Image-to-HTML, Image-to-SVG, and Image-to-3D generation tasks. Experimental results demonstrate that VHarness consistently improves visual fidelity and geometric accuracy across both text-only and multimodal coding agents. These findings highlight that modularizing visual reasoning into callable utilities provides a flexible and scalable foundation for future agentic coding systems.
64. 【2611.17789】Can Large Language Models Optimize Pretrained Neural Network Weights? A Multi-Phase Empirical Evaluation of Gradient-Free Editing in YOLO26 Object Detection
链接:https://arxiv.org/abs/2611.17789
作者:Mohamed Abdallah Salem
类目:Computer Vision and Pattern Recognition (cs.CV); Artificial Intelligence (cs.AI)
关键词:fine-tuning pretrained detectors, large language model, guidance remain uncertain, Direct weight editing, pretrained detectors
备注:
点击查看摘要
Abstract:Direct weight editing offers a gradient-free alternative to fine-tuning pretrained detectors, but its generalization and the value of large language model (LLM) guidance remain uncertain. We evaluate five experimental phases using YOLO26 models and MS COCO, comparing deterministic scaling, random perturbations, differential evolution, CMA-ES, structured interventions, real local-LLM proposals, failure-aware guidance, and hybrid LLM-CMA-ES against a matched random-policy control. Optimization uses native Ultralytics mAP50-95 on 1,000 COCO Val2017 images; selected candidates are assessed on a disjoint, previously examined 1,500-image subset, with COCO API AP50-95 reported separately where available. Increasing the CMA-ES budget from 8 to 250 evaluations raised development gains from +0.11 to +0.28 percentage points, while selection changes remained negative (-0.03 to -0.12 points); the 250-evaluation result uses one seed. The hybrid LLM approach showed no statistically established advantage over random-policy guidance (p = 0.612, 0.369, and 0.694 at budgets 8, 25, and 50). Five-seed YOLO26m replication yielded a COCO API AP difference of +0.004 points (95% CI: -0.026 to +0.034). Of 225 archived LLM proposals, 53 satisfy the final strict schema; early tensor-level execution cannot be fully verified. Genuine LLM-mediated weight editing is feasible, but the tested approaches did not demonstrate consistent transfer beyond the development subset or a measurable LLM advantage.
65. 【2611.17786】owards Seconds-level Video Representation: Quadratically Gated Feature Grid with Adaptive, Prediction-free Sampling
链接:https://arxiv.org/abs/2611.17786
作者:Tie-Yun Li,Hao-Ran Wang,Chao Wang,Zhiyong Yin,Yiming Xu,Zhenwei Cai,Li Li,Rongzhi Lin,Feng-Lei Fan
类目:Computer Vision and Pattern Recognition (cs.CV); Artificial Intelligence (cs.AI)
关键词:Implicit neural video, basic practical threshold, neural video representations, Implicit neural, coordinate-queryable video representation
备注: 13 pages, 11 figures, 4 tables. Code available at [this https URL](https://github.com/MillenRosen/Q-GAS)
点击查看摘要
Abstract:Implicit neural video representations are notoriously slow to build, which does not cross the basic practical threshold for most industrial applications. Here, we present Q-GAS, a coordinate-queryable video representation that couples multi-resolution explicit 3D feature grids with a compact SwiGLU-style quadratic-gating (QG) decoder and an error-aware, prediction-free sampler. The explicit grids avoid hash-collision coefficient sharing at the cost of modestly higher storage yet greatly boost the expressive capacity; the QG decoder introduces low-rank local second-order interactions to improve the nonlinear feature response. During fitting, a global error map periodically refreshes a high-error pool, and each batch mixes pool samples with uniform samples, which is free from the auxiliary model to predict difficult samples, thereby further escalating the efficiency. On UVG Beauty/Bosphorus/Jockey, Q-GAS fits and quantizes under 50 s on one RTX PRO 6000, decodes at over 370 FPS with mixed precision and about 146 FPS in FP32. In our PyTorch implementation, Q-GAS trains and quantizes 5.5-7.2 times faster than the evaluated Instant-NGP baseline and 19-28 times faster than RNeRV. Under a 90 s budget, it improves PSNR/MS-SSIM/LPIPS over the evaluated NeRV-family pipelines at comparable BPP over the measured overlap. We have shared our code at this https URL for readers' free use.
66. 【2611.17767】Learning Anisotropic Prediction Targets for Flow Models
链接:https://arxiv.org/abs/2611.17767
作者:Rayhan Zirvi,Hansheng Chen,Gordon Wetzstein,Leonidas Guibas
类目:Computer Vision and Pattern Recognition (cs.CV); Machine Learning (cs.LG)
关键词:Flow-based generative models, Flow-based generative, Anisotropic Flow Models, hand-chosen prediction target, prediction target
备注: Project page: [this https URL](https://hanshengchen.com/anisoflow)
点击查看摘要
Abstract:Flow-based generative models are trained on a hand-chosen prediction target, such as the velocity, the noise, or the clean data. This choice strongly affects training and sample quality. We introduce Anisotropic Flow Models (AnisoFlow), which replaces a fixed prediction target with a fully learnable one. AnisoFlow learns to scale the noise component of the target anisotropically, assigning a separate nonnegative weight to each direction in a jointly learned basis. A closed-form map recovers the velocity from the network's prediction, leaving the architecture, flow-matching loss, and sampler unchanged. Since recovery is exact, training optimizes the target purely to make it easier for the network to predict. We theoretically analyze the resulting prediction error and predict that the preferred noise weight in each direction scales near-linearly with data variance, in agreement with the weights learned empirically in pixel space. AnisoFlow improves FID from 1.90 to 1.55 over the JiT baseline on ImageNet 256$\times$256, by learning only the prediction target, without representation alignment or architectural changes. In pixel-space text-to-image generation, AnisoFlow improves image quality and, through better-conditioned velocity recovery, largely removes patch-seam artifacts under reduced-precision inference.
67. 【2611.17762】Creative Product Design under Brand Image Constraints
链接:https://arxiv.org/abs/2611.17762
作者:Takeshi Onishi,Hiroya Makino,Takahiro Yamaguchi,Hiroyuki Sakai
类目:Computer Vision and Pattern Recognition (cs.CV); Artificial Intelligence (cs.AI); Graphics (cs.GR)
关键词:Industrial design requires, requires creative visual, creative visual exploration, Industrial design, product-specific and implicit
备注:
点击查看摘要
Abstract:Industrial design requires creative visual exploration while satisfying constraints that are often product-specific and implicit. Existing text-to-image (T2I) approaches can generate creative designs through concept blending or multi-agent self-enhancement, but they provide limited mechanisms for ensuring that generated designs remain consistent with such constraints. We focus on brand image as a representative design constraint and propose a framework for brand-aware redesign generation. Given a product name, the framework constructs a brand image survey report that summarizes product and market information, derives a creative direction from the report, and integrates this direction into a self-enhancement loop in which generated images are critiqued and prompts are revised. We evaluate the framework on 50 redesign tasks across five product categories, comparing it with Naive T2I, IP-Adapter/IT-Blender, and unconstrained self-enhancement under the same T2I backbone. By a VLM-as-Judge protocol to measure creativity and brand identifiability, the proposed framework achieves a more favorable trade-off between the two objectives, with particularly clear gains in highly constrained categories such as automotive and motorcycle, where the plausible design space is relatively limited compared with that of other product categories. These results suggest that dynamically eliciting and textualizing brand image can guide creative product design generation under brand image constraints.
68. 【2611.17758】Efficient Genotype Classification from 3D MRI in ADPKD via Density-Aware Point Sampling and E(3)-invariance
链接:https://arxiv.org/abs/2611.17758
作者:Kwanghee Lee,Hyeongsub Lee,Youngwoo Kim,Hyunji Jeong
类目:Computer Vision and Pattern Recognition (cs.CV); Machine Learning (cs.LG)
关键词:polycystic kidney disease, end-stage kidney disease, autosomal dominant polycystic, genetic kidney diseases, kidney disease
备注:
点击查看摘要
Abstract:Early detection of genetic kidney diseases, especially autosomal dominant polycystic kidney disease (ADPKD), is crucial because certain genotypes such as truncating PKD1 mutations cause rapid kidney growth and early progression to end-stage kidney disease. MRI-based genotype inference is a potential low-cost complement to genetic testing. However, although many deep learning methods have been developed for classifying various MR images, applying them to ADPKD remains challenging: (1) abdominal MR images are generally too large to be handled by deep learning models designed for smaller images like brain MRI, and (2) the amount of labeled data for ADPKD genetic mutations is insufficient to train deep learning models. In order to address these challenges, we propose E3PN, an efficient 3D MRI-based ADPKD genotype classification network that relies on point sampling and E(3)-invariance. We utilize a point representation of 3D MRI, which consumes less GPU memory than a voxel representation. For further memory efficiency, 3D points are sampled from a mixture distribution over the two kidneys in each image. Since E(3)-invariance has been shown to improve data efficiency, we also incorporate it into our model through coordinate canonicalization. On our ADPKD dataset, E3PN trains 163 times faster and uses 103x less GPU memory than a 3D ResNet, while also improving F1 from 0.165 to 0.226 over both baselines. These results establish a first benchmark for five-way ADPKD genotype classification from 3D MRI and show that point sampling and coordinate canonicalization yield two-orders-of-magnitude efficiency gains with a modest but consistent improvement in F1.
69. 【2611.17755】VACO: Vertex-Aligned Decoding for Dense Hand Contact Estimation from a Single Image
链接:https://arxiv.org/abs/2611.17755
作者:Binhong Ye,Jin-Chuan Shi,Tao Liu,Xin Wang,Hao Chen,Chunhua Shen
类目:Computer Vision and Pattern Recognition (cs.CV)
关键词:Dense hand contact, single RGB image, Dense hand, hand contact estimation, RGB image
备注:
点击查看摘要
Abstract:Dense hand contact estimation from a single RGB image is a foundational primitive for dexterous manipulation and 3D interaction analysis. Direct predictors typically decode the entire per-vertex contact map from one shared query, leaving individual vertices ungrounded in the image and smearing contact across adjacent, non-touching surfaces. We present VACO, a vertex-aligned decoder in which each MANO vertex samples local image features at its projected location. A hand-reconstruction backbone, jointly trained for reconstruction and contact, predicts these locations at inference without an auxiliary reconstructor. Training uses noise-perturbed ground-truth projections and end-to-end feature adaptation to improve robustness and contact-sensitive features. A single VACO checkpoint sets a new state of the art on five benchmarks spanning hand-object, hand-human, hand-scene, and bimanual interactions. It improves F1 by +20.7 pp over the strongest prior direct predictor on Hi4D and generalizes to the held-out HOT3D and HOCap, with gains of +5.5 and +2.7 pp. The resulting contact predictions also improve downstream HOI reconstruction.
70. 【2611.17752】Stage-wise Lineage-aware Dropout for Sparse-View 3D Gaussian Splatting
链接:https://arxiv.org/abs/2611.17752
作者:Yan Xing,Zhi Chen,Fanglue Zhang,Junhong Zhao,Bing Xue,Mengjie Zhang
类目:Computer Vision and Pattern Recognition (cs.CV)
关键词:constrain scene geometry, Gaussian Splatting, scene geometry, prone to overfitting, limited number
备注: 11 pages, 9 figures, 7 tables, including supplementary material
点击查看摘要
Abstract:Sparse-view 3D Gaussian Splatting (3DGS) is prone to overfitting because only a limited number of images are available to constrain scene geometry. Dropout is one regularization strategy for alleviating this problem. We introduce the concept of a lineage family to describe a set of Gaussian primitives derived from the same founder through cloning or splitting operations. The founder Gaussians refer to all Gaussian primitives generated during initialization. Descendants within the same family often jointly contribute to the refinement of adjacent local regions. However, dropping individual Gaussians can be less effective due to the compensation effect from neighboring Gaussians for the dropped ones. To address this issue, we propose Stage-wise Lineage-aware Dropout(SLD), which applies lineage-family dropout during the structural regularization stage and switches to individual dropout during the later refinement stage. We further propose Lineage-Balanced Densification(LBD), which operates during densification by constructing an expanded candidate pool and reallocating the densification budget according to the mean gradient scores and existing descendant sizes of different families. Experiments on the LLFF, MipNeRF360, and Blender datasets show that the proposed framework achieves competitive performance in sparse-view reconstruction
71. 【2611.17751】When Does Cross-View Consistency Matter in Multi-View Appearance Augmentation? A Controlled Study for Visuomotor Policies
链接:https://arxiv.org/abs/2611.17751
作者:Zixuan Jiang
类目:Robotics (cs.RO); Computer Vision and Pattern Recognition (cs.CV)
关键词:Multi-view visual augmentation, Multi-view visual, downstream control, visual augmentation, augmentation often seeks
备注: 30 pages, 10 figures. Code: [this https URL](https://github.com/PKUJZX/xview-consistency-augmentation)
点击查看摘要
Abstract:Multi-view visual augmentation often seeks consistency across cameras, but it is unclear which forms of consistency matter for downstream control. We use simulation-state replay to manipulate cross-view coupling while preserving the per-camera perturbation distributions, physical trajectories, and action labels. Across seven LIBERO manipulation tasks, marginal-matched changes in global appearance, ghost injection, and target color have small average costs for a late-fusion Diffusion Policy: point estimates are near zero or negative, and the largest upper endpoint of the 90% posterior credible intervals is 2.55 percentage points. In contrast, geometrically anti-coupled observations reduce success by 3.30 percentage points relative to coupled observations (90% CrI [0.64, 6.28]), with displacement magnitude, direction marginals, and the number of affected views matched. Camera interventions reveal substantial dependence on wrist-view geometry, while feature analysis shows appearance sensitivity patterns associated with the factors randomized during training. ACT shows a higher target-color cost estimate than DP, with uncertainty that includes zero and sensitivity to task composition. Within the tested tasks and policy configurations, these findings motivate selective consistency constraints: appearance coupling can sometimes be relaxed, whereas task-relevant geometry and cross-view spatial correspondence warrant protection.
72. 【2611.17733】HarmonyText: Harmonious Scene Text Editing via Reverse-to-Refined Pair Synthesis and Region-Decoupled Frequency Learning
链接:https://arxiv.org/abs/2611.17733
作者:Ziqi Yi,Huiguo He,Guitao Xu,Peirong Zhang,Cheng Da,Xinyao Yu,Zhanpeng Wang,Changqian Yu,Huan Yang,Kun Gai,Lianwen Jin
类目:Computer Vision and Pattern Recognition (cs.CV)
关键词:Natural language-guided, over-smoothed text generation, unreliable pseudo-GT supervision, mask-free scene text, scene text editing
备注: 11 pages, 7 figures. Accepted to SIGGRAPH Asia 2026 Conference Papers
点击查看摘要
Abstract:Natural language-guided, mask-free scene text editing aims to modify text in real-world images using only textual instructions, yet remains challenged by unreliable pseudo-GT supervision, over-smoothed text generation, and the absence of GT-free evaluation. Existing methods often rely on imperfect pseudo-targets with background discrepancies, while standard MSE-based velocity regression tends to suppress high-frequency details, leading to blurred text boundaries and weakened local textures. To address these issues, we propose HarmonyText, a mask-free scene text editing framework with region- and frequency-aware supervision. HarmonyText introduces Region-Decoupling Optimization (RDO), which aligns edited text regions with pseudo-targets while anchoring preserved regions to the source image, and Cascaded High-Frequency Texture Guidance (CHFTG), which applies cascaded wavelet-domain constraints to preserve sharp boundaries and fine-grained textures of textual content. We further propose Reverse-to-Refined Pair Synthesis (RePair) to construct STE25K, a high-quality 25K paired dataset, and introduce a GT-free evaluation protocol with Color Distribution Divergence and a Harmony Assessment Model. Extensive experiments show that HarmonyText outperforms existing methods in text editing accuracy, background fidelity, color consistency, and perceptual harmony.
73. 【2611.17720】Ego2Flow: Benchmarking the Traffic-System Impact of Autonomous Driving on Maps Distilled from Real-World Roads
链接:https://arxiv.org/abs/2611.17720
作者:Yu Dong,Weiying Shen,Hangyu Li,Shu Han,Hong Li,Xiangbo Gao,Keshu Wu,Zhengzhong Tu,Soyoung Ahn,Zhiyuan Liu,Xiaopeng Li
类目:Robotics (cs.RO); Computer Vision and Pattern Recognition (cs.CV)
关键词:task completion, predominantly evaluated, autonomous driving, driving, remain insufficiently evaluated
备注:
点击查看摘要
Abstract:End-to-end autonomous driving is predominantly evaluated through ego-vehicle safety and task completion. Its deployment in urban traffic, however, creates sustained interactions among vehicles whose collective effects remain insufficiently evaluated. We introduce Ego2Flow, a closed-loop benchmark for assessing the traffic-system impact of autonomous driving on maps distilled from real-world cities. To construct realistic 3D urban environments for sustained closed-loop traffic simulation, we develop an agent-guided map distillation framework that integrates real-world maps and observations with complementary geographic information. Through an iterative feedback mechanism and a traffic information graph that jointly guide scene generation, the framework preserves lane topology, critical road geometry and decision-relevant visual conditions while simplifying nonessential details and completing unobserved regions. Using our proposed framework, we generate and publicly release six CARLA maps from the Argoverse 2 dataset, covering the corresponding real-world regions and enabling paired comparisons of driving behavior under real and simulated observations. On these maps, Ego2Flow evaluates how autonomous driving policies affect traffic efficiency, safety, stability and energy consumption across diverse road structures. Compared with existing construction methods, our framework achieves the lowest average displacement error between planning outputs from paired real and simulated observations. We further benchmark representative driving policies to characterize their system-level impact on urban traffic. Project Page: this https URL.
74. 【2611.17708】QCAP-VQA: Question-Conditioned Capped Prior-Mass Supervision for Medical Visual Question Answering
链接:https://arxiv.org/abs/2611.17708
作者:Yining Zhao,Catarina Moreira,Mukesh Prasad,Ali Braytee
类目:Computer Vision and Pattern Recognition (cs.CV)
关键词:correctly without attending, Medical visual question, attention, visual question answering, prior
备注:
点击查看摘要
Abstract:Medical visual question answering models can answer correctly without attending to the anatomy that the question names. Automatic region priors allow spatial supervision without expert masks, but they are noisy, and it is unclear how strongly they should shape attention and how their effect should be verified. We present QCAP-VQA, which supervises a subset of cross-attention heads with a question-conditioned prior built from ranked segmentation proposals. Its capped prior-mass objective penalizes insufficient attention inside the prior only below a threshold. We evaluate attention against SLAKE structure boxes that play no role in training, separating attention on the named anatomy from how attention shifts between questions about the same image. QCAP-VQA matches the accuracy of a baseline without spatial supervision (84.26% vs. 84.23%), and every evaluated prior-supervised model places more attention on the annotated structures. Over a 64-fold range of regularization strength, the capped objective keeps prior mass between 0.67 and 0.77, whereas a pooled region-mass objective drives it to 0.97 and loses accuracy. The cost lies in question conditioning: as the constraint widens or strengthens, attention shifts agree less with the expert-defined contrast. Across 13 configurations, prior-referenced and expert-referenced evaluation rank the models in largely opposite order (Spearman correlation of -0.87), and the capped models it favors track the prior's own errors. Because clinical use of medical visual question answering requires answers grounded in the queried anatomy, supervision from automatic priors should be judged against independent anatomical references.
75. 【2611.17707】RaFL: An Empirical Study of Random Filter Losses for Image Reconstruction
链接:https://arxiv.org/abs/2611.17707
作者:Ruibo Wang,Fengbei Liu,Aleksei Tiulpin,Mert R. Sabuncu,Heejong Kim
类目:Computer Vision and Pattern Recognition (cs.CV); Machine Learning (cs.LG)
关键词:Pixel losses train, losses train image, textures under uncertainty, point statistic, averaging away fine
备注:
点击查看摘要
Abstract:Pixel losses train image reconstruction networks toward a point statistic of the target distribution, often averaging away fine textures under uncertainty. Perceptual losses alleviate this issue using pretrained features, but those features are inherently domain-specific. We study a random filter loss (RaFL) that compares prediction and target through filters with random weights, avoiding both pretrained features and task-specific loss engineering. RaFL combines random convolutions, random Fourier features, and random wavelet filters, automatically balancing their contributions through gradient-norm matching, eliminating manual loss-weight tuning and grid search. We systematically investigate three design choices: filter family, weight resampling schedule, and weight initialization. Based on validation PSNR and stability across random seeds, we select the configuration that uses all three filter families, redraws filters every optimization step, and employs Gaussian initialization. We evaluate RaFL on three 2D and two 3D image reconstruction tasks. RaFL outperforms a pixel loss on three tasks and matches it on two, and demonstrates stable performance across random seeds on all five tasks. RaFL adds less than 2% to the backbone's forward FLOPs and no inference cost, requiring $182\times$ fewer loss FLOPs than a VGG19 perceptual loss. These findings suggest that RaFL is a simple, tuning-free, and domain-agnostic loss objective for 2D and 3D image reconstruction.
76. 【2611.17695】Open-World Scene Deocclusion
链接:https://arxiv.org/abs/2611.17695
作者:K.M.S.J. Kumarasinghe,Neil A. Dodgson,Fang-Lue Zhang
类目:Computer Vision and Pattern Recognition (cs.CV)
关键词:Recovering the complete, fundamental problem, Recovering, image editing, complete appearance
备注: 10 pages, 7 figures, Accepted at WACV 2027
点击查看摘要
Abstract:Recovering the complete appearance of occluded objects from a single image is a fundamental problem for scene understanding and image editing. Real-world scenes are challenging due to clutter, complex spatial occlusions, and diverse object categories. Existing methods often struggle when occlusion involves background surfaces or objects that are not explicitly detected, and fail to resolve the cyclic occlusion dependencies that frequently occur in natural scenes. We introduce a training-free, open-world scene deocclusion framework that recovers the full amodal appearance of detected or user-selected objects from a single RGB image, decomposing the scene into depth-ordered RGBA layers that are geometrically plausible, semantically faithful, and visually consistent with the visible scene. Extensive experiments show that our approach improves scene-level deocclusion, producing coherent and realistic reconstructions in real-world settings. Code will be available on the project page: this https URL
77. 【2611.17671】Provable Cross-Modal Transfer of Hierarchical Concept Structure in CLIP Embeddings
链接:https://arxiv.org/abs/2611.17671
作者:Nghia Nguyen,Tianjiao Ding,Lachlan Ewen MacDonald,René Vidal
类目:Computer Vision and Pattern Recognition (cs.CV); Machine Learning (cs.LG)
关键词:represent taxonomies, CLIP image embeddings, CLIP embeddings exhibit, Hierarchical Angular Separation, Contrastive vision-language models
备注: NeurIPS 2026
点击查看摘要
Abstract:Contrastive vision-language models such as CLIP are trained to align images and text, not to represent taxonomies. Yet empirical studies show that CLIP embeddings exhibit hierarchical structure aligned with the hierarchy of semantic concepts. A theoretical characterization of why such hierarchy emerges from a non-hierarchical objective remains open. In this paper, we formalize this phenomenon as Hierarchical Angular Separation (HAS), a geometric property whereby each non-leaf concept embedding is more similar to its descendants than to off-branch concepts. Empirically, we show that HAS holds approximately in raw images and randomly initialized CLIP image embeddings, while in text embeddings it holds approximately only after contrastive learning. To theoretically explain this asymmetry, we analyze an unconstrained features model in which the image embeddings are fixed and only the text embeddings are trained. At the global minimizer, the text-side HAS violation percentage at each node is at most the image-side HAS violation and near-violation percentage. The key mechanism is the hierarchical co-occurrence structure of image-text pairs, which biases text embeddings toward root-centered image prototypes. For a linear contrastive loss, we derive a closed-form solution that makes this mechanism explicit. We extend the analysis to the InfoNCE loss and establish analogous guarantees under sufficient regularization. Synthetic experiments validate the theory and show that HAS is transferred even beyond the regimes covered by our analysis.
78. 【2611.17629】raining-Free Acceleration of Multi-View 3D Transformers via Regime-Aware Attention Approximation
链接:https://arxiv.org/abs/2611.17629
作者:Muhammed Izzet Sezer,Sevval Belkis Dikkaya
类目:Computer Vision and Pattern Recognition (cs.CV); Artificial Intelligence (cs.AI)
关键词:single forward pass, attention grows quadratically, predicting camera poses, global attention grows, forward pass
备注:
点击查看摘要
Abstract:Multi-view 3D transformers such as VGGT reconstruct a scene from a set of unposed images in a single forward pass, predicting camera poses, depth and 3D points, but the cost of their global attention grows quadratically with the num- ber of images. Accelerators borrowed from large language models assume that each token attends to only a few oth- ers; on these models they lose accuracy or gain little speed. We study the global attention of three multi-view transform- ers on seven datasets and find that every layer plays one of three roles: it averages over the whole scene, matches points across many images, or anchors on the first image, whose coordinates the outputs use. Building on this anal- ysis, we propose Regime-AWare ATTention Approximation (RAWATTA), a training-free method that runs attention in 8- bit precision and keeps 16-bit precision only in the layers where a worst-case calibration check requires it. Its fast set- ting, RAWATTA-Fast, also approximates attention in a few averaging and anchoring layers, while matching layers and the first image keep all their keys. On a 12 GB GPU at 50 im- ages, RAWATTA runs VGGT faster than the released code, mostly through our CUDA engine, and loses at most about one point of pose accuracy (AUC@3) on any dataset; RAWATTA-Fast is slightly faster and less accurate. Both fit 200 images, where the released code runs out of memory at 100.
79. 【2611.17600】LightTAR: Light and Fast Target-Aware Representations for Multimodal Tabular Learning
链接:https://arxiv.org/abs/2611.17600
作者:Eilam Shapira,Alan Arazi,Shoham Grunblat,Roi Reichart
类目:Machine Learning (cs.LG); Computation and Language (cs.CL); Computer Vision and Pattern Recognition (cs.CV)
关键词:Multimodal tabular learning, learning involves classification, tables combining structured, combining structured numerical, text and images
备注: 21 pages, 3 figures. Code: [this https URL](https://github.com/eilamshapira/LightTAR)
点击查看摘要
Abstract:Multimodal tabular learning involves classification and regression on tables combining structured numerical and categorical features with unstructured text and images. These unstructured fields are commonly represented by pretrained encoder embeddings reduced with principal component analysis (PCA). This reduction prioritizes input variance but may overlook information relevant to predicting the target. We introduce LightTAR, a simple, inexpensive method that adapts dimensionality reduction to the prediction task. LightTAR augments the standard PCA representation with a few target-aware features extracted jointly from the final and an intermediate encoder layer using regularized partial least squares (PLS). We evaluate LightTAR with 21 tabular models, including tree ensembles, neural networks, and tabular foundation models. LightTAR improves average predictive performance for all models and, averaged across models, improves prediction on almost every MulTaBench dataset. These gains come with a small increase in total runtime relative to PCA representations. LightTAR integrates directly into existing encoding pipelines, improving tabular learners' predictive performance without modifying the encoder or the learner.
80. 【2611.17598】X-Nav Technical Report: Real-World Long-Horizon Indoor-to-Outdoor Navigation on a Humanoid Robot
链接:https://arxiv.org/abs/2611.17598
作者:Jiaxin Huang,Chengxing Lin,Xiaoyu Luo,Sirui Chen,Jingyu Guo,Aolin Li,Kairui Chen,Rui Zhen,Cheng Wen,Kerui Gu,Shunyao Wang,Zhihui Peng,Shaoli Huang
类目:Robotics (cs.RO); Artificial Intelligence (cs.AI); Computer Vision and Pattern Recognition (cs.CV)
关键词:remains an open, open problem, instruction covers rooms, indoors to outdoors, MELIOR
备注:
点击查看摘要
Abstract:Long-horizon navigation from indoors to outdoors remains an open problem for humanoid robots: one instruction covers rooms, doors and roads, and the camera moves with the gait. Worken, AGIBOT's measure of productive robot work, depends on robots that reach their workplaces autonomously. We present X-Nav, to our knowledge the first system to take a bipedal humanoid through a language-instructed long-horizon mission from indoors to outdoors. X-Nav covers a pipeline from data to deployment. Its data engine collects navigation data in open-source scenes, in digital twins of deployment sites and on real robots. Its navigation model, MELIOR, takes only RGB images from the robot's built-in camera and comes in two versions: MELIOR-A predicts actions, and MELIOR-N predicts a navigation goal. Its agentic workflow, X-Navarch, turns a brief human instruction into sub-tasks and deploys MELIOR on the robot with explicit target navigation and obstacle avoidance. On the Lingxi X2, X-Nav completes a mission of about a hundred metres from an office lobby to a store in a neighbouring building; LiDAR and odometry are used for target approach and low-level control. The mission is completed with both versions of MELIOR: the robot follows a change of destination and walks through a door from the lobby to the road, and with X-Navarch it also passes static and dynamic obstacles on the route. During deployment, MELIOR is adapted with data collected in a digital twin of the site; the adaptation raises its closed-loop success rate in the twin, and the adapted model is then deployed on the robot at the real site. In addition, MELIOR achieves competitive results on the R2R-CE and RxR-CE benchmarks.
81. 【2611.17586】Where to Predict in Retinal OCT? Anatomy-Guided Target Selection for I-JEPA
链接:https://arxiv.org/abs/2611.17586
作者:Yunchu Feng,Wai Tak Lau,Kaveri A. Thakoor
类目:Computer Vision and Pattern Recognition (cs.CV)
关键词:labeled data needed, potentially allowing models, Self-supervised learning aims, capture subtle anatomical, subtle anatomical variations
备注: 18 pages. Accepted as a poster at the GenAI for Health Workshop at NeurIPS 2026 (non-archival)
点击查看摘要
Abstract:Self-supervised learning aims to learn meaningful representations from unlabeled data, potentially allowing models to capture subtle anatomical variations across patients and reduce the amount of labeled data needed. Image-based Joint-Embedding Predictive Architecture (I-JEPA) learns by predicting target-region representations from visible context rather than reconstructing pixels. However, its uniform target placement does not account for the anatomical structure of retinal OCT, where retinal tissue occupies only part of each B-scan. We investigate anatomy-guided target selection using image intensity and predicted retinal segmentation while retaining the I-JEPA architecture and latent prediction objective. We compare anatomy-guided rectangular targets, anatomy-shaped targets and coverage-constrained placement with uniform masking, and evaluate the learned representations using linear probing for glaucoma classification on FairVision. We found that rectangular targets guided by image intensity achieved the highest linear-probe AUC in our comparisons. We show that anatomically meaningful target selection is promising for retinal OCT representation learning, while greater anatomical specificity did not consistently improve performance.
82. 【2611.17585】Rethinking Visual Reasoning in Text-to-Image Reward Modeling
链接:https://arxiv.org/abs/2611.17585
作者:Shiye Su,Xiaohan Wang,Ludwig Schmidt,Serena Yeung-Levy
类目:Computer Vision and Pattern Recognition (cs.CV); Artificial Intelligence (cs.AI)
关键词:component of modern, Reward models, reward model, core component, visual reasoning
备注:
点击查看摘要
Abstract:Reward models are a core component of modern text-to-image systems, serving as the objective for inference-time selection and RL-based post-training. A key prerequisite is visual understanding: to judge prompt adherence and ground aesthetic preferences, a reward model must bind attributes, resolve relations, and count reliably. We find that widely-used reward models perform poorly on targeted visual reasoning evaluations. Even when starting from strong vision-language backbones, preference fine-tuning erodes these capabilities, and continual-learning baselines fail to mitigate a steep tradeoff between visual reasoning and preference accuracy. We propose a data-centric intervention that injects visual reasoning supervision into preference learning by flipping the pairing axis. To complement conventional human preference data (paired images under a shared prompt), we convert existing visual reasoning datasets into preference pairs that share the same image but differ in prompt, and train on the mixture. This yields a Pareto improvement in both visual understanding and preference prediction. Across inference-time best-of-n selection and RL post-training, the resulting reward model improves text-to-image generation on challenging compositional prompts (GenEval) by an average of 4.0% and 5.5% respectively, outperforming HPSv3 which was trained on 6x more preference pairs.
83. 【2611.17505】Adversarial Tracking: Attacks on Image Sensors for Object Tracking in Autonomous Vehicles
链接:https://arxiv.org/abs/2611.17505
作者:Han Wu,Alex Chan,Ekin Can Erkus,Tousif Rahman,Rishad Shafik
类目:Computer Vision and Pattern Recognition (cs.CV); Artificial Intelligence (cs.AI)
关键词:linking detected objects, Tracking systems estimate, Object Tracking systems, surrounding vehicles, vehicles by linking
备注:
点击查看摘要
Abstract:Object Tracking systems estimate the motion of surrounding vehicles by linking detected objects across video frames. In particular, Tracking-By-Detection (TBD) systems combine object detectors with motion prediction and data association to maintain object identities over time. However, neural network-based detectors are vulnerable to adversarial attacks that perturb input images, and their impact on downstream trajectory estimation is not fully characterized by detection accuracy alone. Therefore, this paper evaluates image-specific and image-agnostic attacks on YOLOv3 and YOLOv4 detectors paired with SORT, Deep SORT, Strong SORT, and OC-SORT tracking systems. The evaluation uses a CARLA autonomous driving simulator and measures tracking quality through Multiple Object Tracking Accuracy (MOTA), Higher Order Tracking Accuracy (HOTA) and Identification F1 (IDF1). Under image-specific attacks, HOTA for YOLOv3-based trackers declines from 53.45-56.19% to 14.85-17.03%, while MOTA reaches -772.60% in the worst case. YOLOv4-based trackers exhibit greater robustness, while HOTA still declines to 25.06-26.86%. In addition, image-agnostic attacks can reduce the tracking accuracy using a single Universal Adversarial Perturbation (UAP), without generating a separate perturbation for individual frames.
84. 【2611.17442】Racing the Clock: Teaching VLNs Budget-Conditioned Object Search
链接:https://arxiv.org/abs/2611.17442
作者:Maitree Meher,Siming He,S. Shankar Sastry,Jitendra Malik,Claire Tomlin,Jonas Frey
类目:Robotics (cs.RO); Artificial Intelligence (cs.AI); Computer Vision and Pattern Recognition (cs.CV); Machine Learning (cs.LG)
关键词:shape vision-language navigation, teaches budget-aware search, action budgets shape, budgets shape vision-language, vision-language navigation
备注: 12 pages, 4 figures, 3 tables
点击查看摘要
Abstract:How do action budgets shape vision-language navigation (VLN) exploration, and what expert supervision best teaches budget-aware search? We study budget-conditioned object search: finding as many distinct instances of a requested category as possible within a budget, with discovery defined geometrically by camera field of view and proximity. Distinct discoveries provide a denser evaluation signal than single-target success. We cast expert design as a choice of scene abstraction: a single budget-indexed Bellman recursion, applied to different abstractions of the full environment, yields experts that differ in what they know. An object-level Bellman expert is given every target location and reduces search to budgeted routing. A room-level Bellman expert retains observed room connectivity, doorway evidence, and cross-scene semantic priors but withholds the contents of unexplored rooms, so it must search for targets that its student also cannot see. We distill both experts into Uni-NaVid under matched training conditions and also evaluate greedy-frontier supervision on held-out HM3D scenes. Changing only the stated budget changes learned trajectories. Despite higher object-level expert return, room-level supervision achieves the highest mean collection among evaluated VLNs at every tested budget. At 2000 steps, it finds 72.7% of reachable targets, versus 68.5% under greedy-frontier supervision, 65.5% under object-level supervision, and 44.8% for the pretrained VLN. These results show that expert-return rankings need not predict supervision quality for VLN exploration, and motivate choosing an expert's abstraction with the student's information in mind rather than by expert return alone.
85. 【2611.17438】Real-Time Diffusion Relighting as a Neural Sensor: Closing the Low-Light Domain Gap for Robotic Perception
链接:https://arxiv.org/abs/2611.17438
作者:Shengjie Xu,Alexander Gao,Anshul Rai,Dae Yeol Lee,Guan-Ming Su,Ming C. Lin
类目:Computer Vision and Pattern Recognition (cs.CV)
关键词:object detectors trained, Low-Light Image Enhancement, degrade robotic perception, conditions severely degrade, severely degrade robotic
备注: 8 pages, 5 figures. Accepted to IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) 2026. Project page: [this https URL](https://shengjie-xu.com/conferences/2026_IROS_neural-sensor/)
点击查看摘要
Abstract:Low-light conditions severely degrade robotic perception: object detectors trained on daylight imagery are prone to failure under dark and adverse weather conditions that fall outside of their training distribution. Furthermore, from an efficiency standpoint, existing Low-Light Image Enhancement (LLIE) methods are designed for offline processing and are therefore impractical to integrate into live control loops. To address these issues, we propose to embed a Generative Neural Sensor within the control loop that treats illumination as a generative prior rather than a signal processing problem, enabling task-relevant semantic recovery. We further propose an Real-Time Perceptual Architecture that decouples high-latency semantic recovery from low-latency robotic control. Rather than blocking the control loop, our system runs a large-scale image-to-image diffusion model in a background thread, and is activated only when a drop in detection confidence signals a perceptual uncertainty event, significantly reducing computational overhead. Our method sets a new state-of-the-art performance on the ExDark low-light benchmark with 87.1% mAP, and under adverse weather conditions, significantly boosts detection rates over competing methods. On a navigation task in a closed-loop autonomous driving simulator (CARLA), we achieve a 100% lap completion rate, establishing a simulation-based proof-of-concept for integrating high-latency generative models with standard robotic control autonomy stacks.
86. 【2611.17400】ParseBench-2: Severity-Aware Evaluation of Document Parsing
链接:https://arxiv.org/abs/2611.17400
作者:Pierre-Loïc Doulcet,Boyang Zhang,Sebastián G. Acosta,Sacha Bron,Daniel B. Ospina,Antonio Jimeno Yepes,Nutchapol Dendumrongsup,Simon Suo
类目:Computer Vision and Pattern Recognition (cs.CV)
关键词:give high scores, change important facts, change important, preserve meaning, give high
备注:
点击查看摘要
Abstract:Document-parsing metrics can give high scores to outputs that change important facts and penalize changes that preserve meaning. People can often judge these consequences more easily than they can design a metric to measure them. We introduce a method for developing parsing metrics from severity policies that specify how each error type should affect the score. We express these policies through test scenarios that specify how a metric should behave. We then optimize the metric implementations against these fixed expectations, using coding agents to refine comparison rules, error detectors and weights under human review. This makes stated human priorities an explicit objective of metric design. We apply this method in ParseBench-2, a comprehensive document-parsing benchmark that combines broad capability coverage with metrics calibrated to error severity. It covers 4,317 pages across six dimensions of parsing quality: text, formatting, tables, charts, visual grounding, and forms. We use detailed annotations and targeted difficult cases to check for changed facts, broken structural relationships, and missing content. We take the lower of a parsing score that measures content recovery and a severity cap that penalizes detected errors, so recovering most of a page cannot hide a serious error. Across six severity test suites, our metrics more closely match the expert judgments expressed by the scenario targets than existing metrics and penalize at most 3.4% of the tested benign variations. We conduct a comprehensive evaluation spanning frontier vision-language models, specialized document parsers and open-weight systems, comparing their overall rankings, performance across parsing capabilities and tradeoffs between accuracy and cost.
87. 【2611.17377】Beyond Attack-Based Accuracy: Rethinking Adversarial Robustness Evaluation Through Structural Gradient Alignment
链接:https://arxiv.org/abs/2611.17377
作者:Hanrui Wang,Yu Luo,Shuo Wang,Xingbo Dong,Zhe Jin,Isao Echizen
类目:Cryptography and Security (cs.CR); Computer Vision and Pattern Recognition (cs.CV)
关键词:fundamental security property, vision systems evolve, visual inputs remaining, Visual adversarial robustness, large vision-language models
备注:
点击查看摘要
Abstract:Visual adversarial robustness remains a fundamental security property as vision systems evolve from conventional classifiers to vision-language models (VLMs) and large vision-language models (LVLMs), with visual inputs remaining an attack surface across generations. Yet existing evaluation methods often depend on prescribed attacks, in-distribution data, extensive probing, or classifier-specific formulations, limiting external auditing of diverse models. We introduce Structural Gradient Alignment (SGA), a complementary robustness metric measuring the structural alignment between input content and objective-conditioned local model sensitivity. SGA uses a few randomly sampled, unlabeled public probes that need not follow the downstream task distribution and only one input-gradient computation per model--probe pair. Across 38 distinct DNNs, VLMs, and LVLMs, SGA produces rankings consistent with attack-based references. Without robustness references, uncalibrated SGA achieves $83.72\%$ average pairwise agreement across eight public-probe conditions, outperforming the strongest overall baseline in all eight by $9.38$ percentage points on average. With known robustness orders of anchor models, calibrated SGA increases agreement to $85.36\%$. These results demonstrate an attack-agnostic, computationally efficient robustness signal across conventional DNNs, VLMs, and autoregressive LVLMs, downstream tasks, threat norms, probe distributions, and evaluator-knowledge settings. Our implementation is available at this https URL.
88. 【2611.18243】D2ToF: One-Step Diffusion for Depth Super-Resolution from Lightweight Single-Photon LiDAR Waveforms
链接:https://arxiv.org/abs/2611.18243
作者:Pallabjyoti Deka,Lewis Wilson,Stephen McLaughlin,Jonathan Leach,Abderrahim Halimi
类目:Image and Video Processing (eess.IV); Computer Vision and Pattern Recognition (cs.CV)
关键词:sensors offer exceptional, offer exceptional temporal, exceptional temporal resolution, Single-photon LiDAR, single-photon sensitivity
备注:
点击查看摘要
Abstract:Single-photon LiDAR (SPL) sensors offer exceptional temporal resolution and single-photon sensitivity for depth imaging. However, cost-effective SPL arrays typically suffer from extremely low spatial resolutions (e.g. $8\times 8$ pixels), limiting their real-world utility. In this paper, we propose a novel, one-step super-resolution framework, D2ToF, Distilled Diffusion for Time-of-Flight, that bridges the gap between high-fidelity generative modeling and real-time execution. Our approach uses an amortized variational distillation strategy that allows for a measurement-based data consistency term to be incorporated directly into the one-step training objective. This encourages that the synthesized high-resolution depth maps are consistent with the captured low-resolution Time-of-Flight (ToF) histograms. To overcome the severe ill-posedness of upsampling extremely degraded inputs, we introduce a multimodal guidance strategy. By leveraging a foundational monocular depth model to extract structural priors from a high-resolution reflectivity image, our network is guided by a rich triad of the ToF histogram, reflectivity map, and relative depth estimate. Experimental results outperform evaluated baselines, achieving $16\times$ spatial super-resolution (from $8\times 8 \times 100$ histograms to $128\times 128$ depth maps) at an accelerated inference speed of approximately $83$ milliseconds.
89. 【2611.17743】X-Strata: Separating Aneurysms and Overlapping Vessels into Layers from 2D X-ray Angiography
链接:https://arxiv.org/abs/2611.17743
作者:Zhihong Wen,Congzhou Chen,Hucheng Li,Xun Han,Jianning Zhang,Yakun Chen,Hao Gao,Hao Zhao,Yudong Ma,Guanqiao Li,Yongping Liang,Shengli Guo
类目:Image and Video Processing (eess.IV); Computer Vision and Pattern Recognition (cs.CV)
关键词:X-ray angiogram sums, X-ray angiogram, sums every structure, X-ray, angiogram sums
备注: 22 pages, including 10 pages of supplementary material
点击查看摘要
Abstract:An X-ray angiogram sums every structure along a ray into one pixel. When vessels cross an aneurysm, they obscure part of it and mix their contrast into its filling curve. We present X-Strata, a network that separates a cine angiogram into three layers: the complete aneurysm sac, the vessels in front of it, and the vessels behind it. Contrast timing indicates how much each layer contributes to a pixel, and parallax between two views indicates their order, which a single view cannot determine. X-Strata first estimates the layers and their contrast curves, then uses the second view to assign vessels to the front or back layer, and lifts all layers into 3D. It learns from layer labels projected from same-session 3D rotational angiography, but needs only 2D cine runs and their acquisition geometry at inference. On 220 held-out patients from three centers (122 external), two-view X-Strata ranks first among 14 baselines, including VGGT and C$^2$RV, in recovering the sac where vessels overlap it (Dice 0.693 internal, 0.641 external) and in ordering the layers (balanced accuracy 0.703, 0.659). Its order accuracy exceeds that of all 11 end-to-end baselines by at least 4.1 points internally and 3.3 externally (Holm-adjusted $p 0.001$) and grows with view separation. Recovered 3D sac surfaces have a median Chamfer distance of 0.34 mm on the internal cohort. X-Strata may therefore provide layered aneurysm and vessel information, including 3D sac shape, for patients who undergo only 2D angiography.
90. 【2611.17712】A Tumour-Aware Multimodal Framework for Explainable Prediction of IDH Mutation in Gliomas from Multi-Sequence MRI
链接:https://arxiv.org/abs/2611.17712
作者:Tahsien Al-Quraishi,Waseem Alwan Zaboon,O. S. Albahri,Raza Nowrozy
类目:Image and Video Processing (eess.IV); Computer Vision and Pattern Recognition (cs.CV)
关键词:IDH mutation status, isocitrate dehydrogenase, treatment planning, mutation status, IDH mutation
备注: 45 pages, 12 figures, 4 tables. Preprint of an article submitted for consideration in International Journal of Information Technology Decision Making (World Scientific)
点击查看摘要
Abstract:Accurate prediction of isocitrate dehydrogenase (IDH) mutation status is essential for glioma diagnosis, prognosis, and treatment planning. Although recent deep learning approaches have demonstrated promising performance, many rely on whole-brain feature extraction and provide limited interpretability. This study proposes a tumour-aware multimodal framework for explainable, non-invasive prediction of IDH mutation status from multiparametric magnetic resonance imaging (MRI). The proposed framework integrates four MRI sequences (T1, T1CE, T2, and FLAIR), tumour subregion segmentation, quantitative radiomic descriptors, and clinical information within a unified deep learning architecture. An ablation study was conducted to evaluate the contribution of tumour-aware learning, radiomic features, and clinical variables. Model performance was assessed using multiple complementary evaluation metrics, including accuracy, precision, recall, F1-score, balanced accuracy, receiver operating characteristic analysis, confusion matrix evaluation, and Grad-CAM visualisation. Experimental results demonstrated that the proposed multimodal framework consistently outperformed the baseline MRI-only model, with the Clinical Fusion model achieving the strongest overall predictive performance. The integration of tumour-aware learning, radiomic descriptors, and clinical information progressively improved classification performance, while Grad-CAM visualisations indicated that the model predominantly focused on biologically relevant tumour regions. These findings suggest that tumour-aware multimodal feature integration enables accurate, interpretable, and non-invasive molecular profiling of gliomas and provides a promising foundation for future clinical decision support in precision neuro-oncology.
91. 【2611.17468】A thorough review on Deep Learning approaches to 2D-to-3D Registration in Medical Imaging
链接:https://arxiv.org/abs/2611.17468
作者:Andrea Borghesi,Julia van Beurden,Maarten van Alphen,Michiel van den Brekel,Luc Karssemakers,Jan-Jakob Sonke,Baris Karakullukcu,Jonas Teuwen,George Yiasemis
类目:Image and Video Processing (eess.IV); Computer Vision and Pattern Recognition (cs.CV); Machine Learning (cs.LG); Medical Physics (physics.med-ph)
关键词:medical imaging purposes, key component, medical imaging, imaging purposes, image registration
备注: 51 pages, 4 tables, 8 figures
点击查看摘要
Abstract:2D-to-3D image registration is a key component for several medical imaging purposes such as intraoperative guidance, enabling the alignment between preoperative 3D volumes and real-time 2D images. While conventional optimisation-based methods often struggle with non-convex objectives, sensitivity to initialisation, and high computational demands, deep learning (DL) has emerged as a promising alternative, offering improved alignment robustness and the potential for real-time performance. This review provides a comprehensive analysis of DL frameworks developed to address the challenges inherent in 2D-to-3D registration. We first outline how core concepts of DL are adapted to the registration problem. We then examine recent methodological advancements from multiple perspectives, including pipeline design, architectural innovations, and domain adaptation strategies. Particular attention is given to quantitative metrics, highlighting how performance is defined and measured across studies. Across the 41 included studies, we find that direct regression is the most common pipeline (23 studies, against 18 iterative ones), that hybrid CNN-attention architectures are gaining ground, and that research remains heavily concentrated on X-ray-to-CT registration. We conclude by identifying the absence of per-case uncertainty estimation and the strong neglect of temporal or sequential modelling as the two methodological gaps most consequential for clinical readiness, and discuss state-space models as a way to add temporal memory within intraoperative latency budgets. Further opportunities lie in more principled domain adaptation and in the integration of foundation models for representation learning.

