本篇博文主要展示每日从Arxiv论文网站获取的最新论文列表，以自然语言处理、信息检索、计算机视觉等类目进行划分。
统计
今日共更新659篇论文，其中：
- 自然语言处理108篇
- 信息检索27篇
- 计算机视觉107篇
自然语言处理
1. 【2608.26091】PlanSightRAG: A Visual-First Multimodal RAG for Automating Question Answering and Compliance Checking for Civil Standard Plans
链接：https://arxiv.org/abs/2608.26091
作者：Nabaraj Subedi,Shuvo Dip Datta,Ahmed Abdelaty,Shivanand Venkanna Sheshappanavar
类目：Information Retrieval (cs.IR); Computation and Language (cs.CL); Computer Vision and Pattern Recognition (cs.CV)
关键词：OCR-based automation strips, manually reading legacy, engineers manually reading, Civil infrastructure compliance, Civil infrastructure
备注： 32 pages, 9 figures, 25 tables. Preprint submitted to Automation in Construction
点击查看摘要
Abstract:Civil infrastructure compliance checking has long relied on engineers manually reading legacy 2D plans; however, OCR-based automation strips away the geometry and layout essential for interpreting these plans. We present a Visual-First Multimodal Retrieval-Augmented Generation (RAG) framework called PlanSightRAG. It indexes and reasons directly over plan imagery, integrates a ColNomic-3B multi-vector retrieval, an agentic Planner-Retriever-Auditor-Synthesizer, and MaxSim heatmaps as an evidence trail. We introduce a 4,056-pair benchmark from five state Departments of Transportation (DOT) standard plans (1,898 pages). PlanSightRAG achieves 91.47% Recall@5 on zero-shot retrieval, while on a held-out Michigan DOT corpus, it achieves 91.40%. On synthetic, parametrically-generated compliance drawings, our Qwen2.5-VL-72B pipeline reaches 100% verdict accuracy only when supplied a pre-resolved rule threshold, a controlled ceiling that a non-VLM OCR baseline already reaches at 76.4%. Finally, we demonstrate autonomous visual rule-grounding by extracting numeric limits directly from a specification corpus without any human-supplied rules.
2. 【2608.26081】SwarmWorld: Stigmergic technological evolution in societies of language-model agents
链接：https://arxiv.org/abs/2608.26081
作者：Subhadeep Pal,Fiona Y. Wang,Markus J. Buehler
类目：Artificial Intelligence (cs.AI); Materials Science (cond-mat.mtrl-sci); Computation and Language (cs.CL)
关键词：Collective intelligence, allowing local actions, durable social organization, allowing local, intelligence can emerge
备注：
点击查看摘要
Abstract:Collective intelligence can emerge when individuals coordinate through a shared environment, allowing local actions to accumulate into durable social organization. Language-model agents offer a new substrate for this process, yet most multi-agent systems rely on direct conversation, predefined roles, or centralized workflows. It remains unclear whether decentralized agents can build functional technologies and outperform independent search. Here, initially homogeneous LLM agents in SwarmWorld self-organize without assigned roles or recipes into evolving technological societies. Agents explore a spatial environment, process resources, test materials, construct persistent artifacts, and write executable controllers evaluated by a deterministic simulator under unseen disturbances after the agents are removed. SwarmWorld splits cognition from consequence: agents propose architectures and controllers within fixed action and material schemas, while the simulated world determines function. Shared societies develop broader, more resilient technological portfolios than a strong best-of-N isolated-search baseline, although isolated search remains competitive for the strongest artifact. Agents differentiate into exploration, construction, maintenance, and coordination behaviors, transitioning as the world matures. Technologies accumulate through collaborative construction, executable inheritance, and persistent agent-artifact networks, with most reuse beginning through physical observation rather than communication. Explicit cultural mechanisms amplify collaboration and organization, but functional benefits depend on outcome and timescale. Physical stigmergy alone supports capable societies, while interaction drives persistent technological ecologies rather than universally superior individual inventions.
3. 【2608.26070】Prefix Sliding for efficient test-time scaling
链接：https://arxiv.org/abs/2608.26070
作者：Niklas Muennighoff,Zhengyang Wang,Zeyi Chen,Weijia Shi,Binyuan Hui,John Yang,Dapeng Jiang,Mika Senghaas,Fares Obeid,Johannes Hagemann,Sami Jaghouar,Ludwig Schmidt,Percy Liang,Jason Wei,Andrew Y. Ng,Luke Zettlemoyer,Yejin Choi,Mike Lewis
类目：Computation and Language (cs.CL); Artificial Intelligence (cs.AI); Machine Learning (cs.LG)
关键词：extra test-time compute, letting language models, Prefix Sliding, solving a problem, compute to improve
备注： 28 pages (9 main), 22 figures, 3 tables
点击查看摘要
Abstract:Test-time scaling uses extra test-time compute to improve performance, such as letting language models reason longer when solving a problem. As models keep the entire reasoning trace in memory via full attention, hard tasks that need long thinking can be prohibitively expensive. However, we find most intermediate reasoning tokens lose importance as the model continues reasoning. This calls into question whether retaining them is worth the cost. Based on this insight, we propose Prefix Sliding, which discards tokens during reasoning that are not part of the prefix or the window of the last few thousand tokens. The prefix has key instructions and tools available to the model, while the most recent tokens are the current reasoning the model is working on. This caps the total memory requirement regardless of how long the model reasons, allowing for efficient long-horizon test-time scaling. Without training, Prefix Sliding can make existing models 3x faster while maintaining performance. Training with Prefix Sliding using reinforcement learning can achieve better performance by enabling scaling to reasoning traces beyond a hundred thousand tokens. Ablations show Prefix Sliding outperforms summarizing intermediate tokens or vanilla sliding window. Our code is at this https URL
4. 【2608.26060】Fine-Tuning Whisper for Automatic Speech Recognition in Baniwa: A Preliminary Study
链接：https://arxiv.org/abs/2608.26060
作者：Leonardo Duart,Tiago Fonseca,Thiago Chacón
类目：Computation and Language (cs.CL); Machine Learning (stat.ML)
关键词：Automatic Speech Recognition, Automatic Speech, Speech Recognition, Baniwa Automatic Speech, achieved remarkable performance
备注： 12 pages, 3 tables. Preliminary study
点击查看摘要
Abstract:Automatic Speech Recognition (ASR) technologies have achieved remarkable performance in recent years through the use of large multilingual foundation models. However, most advances remain concentrated on high-resource languages, while indigenous languages continue to suffer from a lack of speech resources and language technologies. This work presents a preliminary study on the adaptation of Whisper for Automatic Speech Recognition in Baniwa, an indigenous Arawakan language spoken in Brazil, Colombia, and Venezuela. The experiments were conducted using a corpus of 1,373 manually transcribed recordings obtained from a linguistic documentation project. The corpus contains approximately 0.54 hours of speech and consists primarily of isolated words and short elicited utterances. The Whisper Small model was fine-tuned using supervised learning and evaluated using Word Error Rate (WER) and Character Error Rate (CER). The best model achieved a WER of 37.5% and a CER of 7.45%, demonstrating that multilingual foundation models can be successfully adapted to extremely low-resource indigenous languages. The results establish an initial baseline for Baniwa Automatic Speech Recognition and provide a foundation for future research involving larger datasets, language-specific adaptation strategies, and post-processing techniques.
5. 【2608.26053】$R^3$: Training Robots to Reason in Natural Language via Reinforcement Learning
链接：https://arxiv.org/abs/2608.26053
作者：Lehong Wu,Yuxiao Qu,Zheyuan Hu,Ivan Zhang,Limin Wei,Zackory Erickson,Aviral Kumar
类目：Robotics (cs.RO); Artificial Intelligence (cs.AI); Computation and Language (cs.CL); Machine Learning (cs.LG)
关键词：hard problems, requiring decomposition, future consequences, foundation models, models to spend
备注： 42 pages, 23 figures
点击查看摘要
Abstract:Reasoning in language allows foundation models to spend more test-time compute on hard problems, such as those requiring decomposition, constraint tracking, and prediction of future consequences. Whether this mechanism can improve robotic manipulation remains unclear, where long-horizon tasks require tracking partial progress, reasoning about object relations, recovering from mistakes, and steering noisy low-level policies. In this paper, we study whether VLMs can be trained to reason directly in natural language to guide low-level manipulation policies. We introduce $R^3$, a simple post-training recipe that turns off-the-shelf VLMs into robotic reasoners: it first mid-trains a VLM on expert-generated reasoning traces to initialize the desired reasoning style, then improves the reasoner with single-step rubric-based RL from offline action data. Unlike prior robotic reasoning methods that mostly use structured traces as auxiliary supervision, $R^3$ trains free-form language reasoning to produce test-time guidance for action. We instantiate $R^3$ on Language Table and simulated bimanual grocery packing, two controlled testbeds for studying robotic reasoning and long-horizon manipulation. $R^3$ improves exploration and generalization across unseen tasks and significantly outperforms instruction-only imitation learning baselines on both benchmarks. Our analyses suggest that free-form language reasoning can function as a test-time compute mechanism for steering low-level policies. Our project page is available at this https URL.
6. 【2608.26052】How Much Rank Does LoRA Need? Rank-Error Bounds for Transformer Attention
链接：https://arxiv.org/abs/2608.26052
作者：Gerard Conangla Planes
类目：Machine Learning (cs.LG); Artificial Intelligence (cs.AI); Computation and Language (cs.CL)
关键词：low-rank adaptation, target attention, rank, attention, Choosing the rank
备注：
点击查看摘要
Abstract:Choosing the rank of a low-rank adaptation (LoRA) update is usually an empirical task. In this paper, we provide a task-dependent theory of the approximation error achievable at each LoRA rank for Transformer attention. We fix a pretrained attention head, a target attention function, and a distribution over inputs from the downstream task, and bound the smallest expected Kullback--Leibler (KL) error achievable by a rank-$r$ query LoRA update. When target attention probabilities are bounded away from zero, we prove a lower bound of the error proportional to $\psi(\|d\|_2)$, where $d$ is the difference between candidate and target attention scores and $\psi(t)=\min\{t^2,t\}$. We also prove an unconditional upper bound $\min\{\|d\|_2^2/4,\sqrt2\|d\|_2\}$. Under explicit realizability, geometry, and moment conditions, we then bound the best rank-$r$ error between an explicit multiple of $\psi(\sqrt{T_r})$ and $\min\{T_r/4,\sqrt{2T_r}\}$, where $T_r$ is the downstream-weighted tail energy of the target update. We also provide target-Fisher bounds when candidate scores remain within a fixed range of the target scores, and an unrestricted lower bound when a subset of tokens carries most of the probability mass. These spectral bounds describe finite-score approximation. We then construct explicit families in which softmax saturation makes the rank required to match the attention function strictly smaller than the rank required to match the finite logits. Finally, we extend the analysis to fused multi-head LoRA and joint query/key updates, exposing the effects of rank sharing and query/key factorization constraints.
7. 【2608.26036】race Integrity for LLM Data Agents: A Vision for Auditable Structured Reasoning in Real-World Systems
链接：https://arxiv.org/abs/2608.26036
作者：Srimonti Dutta,Akshata Kishore Moharir
类目：Artificial Intelligence (cs.AI); Computation and Language (cs.CL)
关键词：Trace Integrity, Trace Integrity Pass, trace, introduces Trace Integrity, Integrity Pass Rates
备注：
点击查看摘要
Abstract:Answer accuracy is an insufficient reliability signal for LLM data agents. In structured-data tasks, a benchmark-correct answer can be produced by an invalid trace. This paper introduces Trace Integrity, a deployment reliability criterion for evaluating whether the computation recorded behind an answer is explicit, executable, schema-valid, operator-faithful, replayable, answer-consistent, and auditable. We identify the Structure Gap as the deployment failure mode that makes Trace Integrity necessary: natural-language reasoning and free-form rationales do not reliably specify the operator-level programs required by real-world systems. We operationalize Trace Integrity with execution contracts, structured artifacts that bind user intent to schema elements, operator plans, assumptions, executable queries, verification status, and final-answer linkage. We also introduce CAIT (Correct Answer / Invalid Trace) Rate, which measures how often answer-only evaluation counts computationally unsupported outputs as successes. In an empirical demonstration on BIRD Mini-Dev, Direct SQL, Operation Summary + SQL, and Contract-First SQL achieve answer accuracies of 20%, 22%, and 24%, while their Trace Integrity Pass Rates are 39%, 43%, and 40% and their CAIT Rates remain high at 55%, 59.1%, and 45.8%, showing that answer accuracy, trace validity, and silent-failure risk are distinct evaluation signals. Real-world LLM data agents should, therefore, be evaluated not only by whether their outputs match a reference answer, but by whether those outputs are backed by auditable computation.
8. 【2608.26035】Beyond Local Surprise: Grounded Dialogue as Selective Belief Revision under Referential Uncertainty
链接：https://arxiv.org/abs/2608.26035
作者：Ziming Liu,Bhanu Chaitanya Jasti,Ziyang Xu,Hongyu Wu,Yi Wu,Jiqun Liu
类目：Computation and Language (cs.CL)
关键词：utterances arrive, speaker refers, understanding, current understanding, current understanding encourages
备注：
点击查看摘要
Abstract:When a speaker refers to a scene that the listener cannot directly see, the listener must decide whether to preserve its current understanding or revise it as new utterances arrive. Many language systems treat local mismatch as a cue for updating: divergence from the current understanding encourages adjustment. Yet conversational understanding may be more conservative, interpreting mismatching evidence relative to prior understanding rather than immediately revising it. We introduce a controlled, data-driven framework for turn-by-turn preserve/revise decisions in dialogue, where competing revision policies are learned under otherwise identical conditions. We compare four theory-driven revision strategies, each reflecting a different assumption about when listeners should preserve or revise. Two findings stand out. First, a mismatch-driven policy that updates solely based on local divergence reacts strongly to mismatch but destabilizes grounding and degrades retrieval. Second, an uncertainty-sensitive policy extends mismatch-based updating with accumulated evidence, preserving coherent understanding while maintaining strong retrieval performance. Surprisingly, coherent understanding emerges from a counterintuitive pattern: local mismatch promotes preservation, whereas accumulated uncertainty promotes revision, suggesting that listeners maintain prior understanding despite local mismatch and revise only when uncertainty sufficiently accumulates. This pattern is consistent with conceptual pact theory.
9. 【2608.26013】VISA: Agentic Self-Evolving Data Synthesis for Multimodal Instruction Following
链接：https://arxiv.org/abs/2608.26013
作者：Min Zeng,Guanxin Tan,Libin Cen,Yawei Wen,Rui Hu,Liuyang Bian,Xiaolong Chen,Xiaoxin Chen
类目：Computation and Language (cs.CL)
关键词：require training data, instruction-following models require, models require training, Visual Instruction Synthesis, Instruction Synthesis Agent
备注：
点击查看摘要
Abstract:Multimodal instruction-following models require training data that is accurate, diverse, verifiable, and challenging. Existing synthesis pipelines typically follow a one-pass generate-and-filter paradigm, discarding feedback from failed samples, verifier outcomes, and target-model errors. We present VISA (Visual Instruction Synthesis Agent), an agentic framework that reformulates multimodal instruction synthesis as a self-evolving loop. At each round, VISA analyzes an image to filter incompatible constraints and discover new verifiable ones, samples diversity- and difficulty-aware constraint sets from persistent memory, generates candidate instructions, and verifies the resulting samples with executable tools and structured large language model judges. Failed samples trigger diagnostic-guided recovery, while accepted samples are probed against the target model to estimate difficulty. The resulting verifier signals and target-model failure profiles are written back to memory, allowing subsequent rounds to adaptively expand the constraint space, reduce template repetition, and focus on unresolved model weaknesses. The same verifier contracts further provide reward signals for reinforcement learning without a separately trained reward model. Experiments on MM-IFEval show that VISA consistently improves multimodal instruction following over strong baselines, while preserving general multimodal capability across seven public benchmarks.
10. 【2608.26008】A Self-Evolving Multi-Agent Framework Defense against LLM Jailbreak Attacks
链接：https://arxiv.org/abs/2608.26008
作者：Tongyan Hu,Bryan Hooi
类目：Cryptography and Security (cs.CR); Computation and Language (cs.CL)
关键词：Large language models, elicit harmful outputs, Large language, code transformation, exploit techniques
备注： 8 pages (main), with appendix
点击查看摘要
Abstract:Large language models (LLMs) remain vulnerable to jailbreak attacks that exploit techniques such as role-playing, obfuscation, code transformation, and multi-step indirection to elicit harmful outputs. As jailbreak strategies keep emerging, defenses have proliferated in an ongoing cat-and-mouse game, yet most remain static: their safety behavior is fixed at deployment, so they cannot accumulate defensive experience or adapt to unseen strategies. We propose a self-evolving test-time defense built around a persistent, cross-interaction rule memory: when an attack succeeds, the framework abstracts that failure into a method-level rule capturing the structural attack wrapper rather than the harmful topic, and reuses it against future inputs. Because rules are method-level, one induced rule generalizes across an entire attack family, and the label space expands as novel wrappers appear. The mechanism operates entirely through external memory and prompting, with no parameter updates, and applies to both open-weight and black-box API models. We realize it as four cooperating modules, but the contribution is the memory-based adaptation mechanism, not the module decomposition. Across four black-box jailbreak families and multiple models, our method substantially reduces attack success rates while preserving benign utility, remains robust under an adaptive composite-wrapper attack, and does not increase over-refusal as the memory grows.
11. 【2608.26004】AsymSpec: Context-Asymmetric Speculative Decoding for Agentic LLMs
链接：https://arxiv.org/abs/2608.26004
作者：Sheng Liang,Yongyue Zhang,Nathanael Brian,Hang Lv,Hao Wang,Chen Zhang,Yong Liu
类目：Artificial Intelligence (cs.AI); Computation and Language (cs.CL)
关键词：LLM pipelines face, Agentic LLM pipelines, pipelines face escalating, face escalating inference, LLM pipelines
备注： EMNLP Main Conference 2026
点击查看摘要
Abstract:Agentic LLM pipelines face escalating inference costs as context accumulates across retrieval, tool use, and multi-turn interactions. To control latency, deployments routinely compress inputs, but this degrades task accuracy. Speculative decoding (SD) accelerates generation losslessly, yet it assumes the drafter and verifier share an identical context, preventing SD from resolving the accuracy-overhead trade-off. We propose AsymSpec, an asymmetric speculative decoding framework that breaks this symmetry: a lightweight drafter reads the full input while the large verifier operates on the compressed view. The drafter steers the verifier via a contrastive $\delta$-fusion of logits, modulated by a divergence-aware acceptance gate that preserves verification stability and high draft acceptance rates. Evaluated across four agentic capabilities and two end-to-end agent benchmarks, AsymSpec reaches $\approx 90\%$ of full-context accuracy on average, delivering $1.3$--$1.7\times$ throughput speedups at $0.2$--$0.3\times$ the compute cost on isolated text capabilities. These results show that asymmetric context access yields substantial gains precisely when compression discards critical reasoning signals.
12. 【2608.25999】Distinct dynamics of conceptual and referential disruptions in human reading and large language model processing
链接：https://arxiv.org/abs/2608.25999
作者：Rui He,Nihal Altay,Wolfram Hinzen
类目：Computation and Language (cs.CL)
关键词：Linguistic meaning, words enter discourse, referential disruption produced, enter discourse, disruption produced
备注：
点击查看摘要
Abstract:Linguistic meaning is grounded in conceptual content, from which reference to particular entities emerges as words enter discourse. To examine the processing dynamics associated with these two dimensions of meaning, we selectively disrupted conceptual or referential information in short narratives and traced the resulting effects in human self-paced reading and in the predictive and representational processing of large language models. In human reading, conceptual disruptions produced a strong but localized processing cost, emerging immediately after the distorted word, reaching an early maximum, and then declining rapidly. Referential disruptions produced weaker effects, which decreased more gradually across subsequent words, and were more strongly modulated by sentence boundaries. In the language model, both disruptions emerged immediately at the manipulated word. Contextual model surprisal showed a pattern closely paralleling human reading: conceptual disruption produced a larger, more locally concentrated effect that decayed rapidly, whereas referential disruption produced a smaller and more gradual downstream effect. Output-layer representations showed a different pattern: referential disruption produced a larger initial displacement, while both distortions were subsequently characterized by power-law decay. Together, these results provide convergent evidence for distinguishable processing dynamics of two types of meaning: conceptual information imposes a more locally concentrated integration cost, whereas referential information engages a more distributed process of maintaining discourse-level identity.
13. 【2608.25977】When Personality Meets Quantization: A Layer-wise MBTI Analysis of Quantized LLMs
链接：https://arxiv.org/abs/2608.25977
作者：Yao Fu,Lijia Huang,Xiaomin Li,Runchao Li,Yu Yin,Kenneth A. Loparo
类目：Computation and Language (cs.CL)
关键词：shapes users' trust, Briggs Type Indicator, large language models, users' trust, emotional experiences
备注：
点击查看摘要
Abstract:Personality is increasingly important in large language models (LLMs), as it shapes users' trust, engagement, and emotional experiences. While the Myers--Briggs Type Indicator (MBTI) has emerged as a common framework for assessing LLMs' personality, existing studies focus primarily on full-precision models and evaluate only final outputs. They overlook the widespread deployment of quantized LLMs requiring low memory footprints, whose personality traits remain underexplored. In this work, we present a systematic MBTI analysis of open-source LLMs across multiple precisions, including mainstream 4-bit methods (GPTQ, AWQ) and extreme 2-bit settings (AQLM variants). Beyond output-level evaluation, we examine how personality emerges across layers through option-level entropy and confidence-gap dynamics, and introduce Uncertainty-Amplified Layer Decoding (UALD) to study decoding-induced personality drift at inference time. Our results reveal a key insight: LLMs' personality is not a static property, but an emergent, layer-dependent decision process sensitive to quantization, prompting, and decoding. Specifically, we find that (1) ENFJ remains dominant across model families and precisions; (2) 4-bit quantization largely preserves coarse personality structure, while 2-bit quantization disrupts fine-grained prompt consistency and cross-precision agreement; (3) personality decisions emerges in upper layers, following substantial ambiguity in early layers; and (4) inference decoding can shift personality, while personality-aligned conditioning improves robustness. These findings provide a new perspective on the behavioral reliability of quantized LLMs and highlight the importance of considering internal dynamics and inference strategies in personality-sensitive chatbot applications.
14. 【2608.25976】Lost but not erased: Finding traces of a forgotten language in neural speech models
链接：https://arxiv.org/abs/2608.25976
作者：Peter Plantinga,Charlotte Moore,Peter W. Donhauser,Krista Byers-Heinlein,Denise Klein
类目：Computation and Language (cs.CL); Machine Learning (cs.LG)
关键词：persistence typically attributed, International adoptees retain, adoptees retain phonological, retain phonological traces, international adoptee experience
备注：
点击查看摘要
Abstract:International adoptees retain phonological traces of a birth language they can no longer speak or comprehend, a persistence typically attributed to a biologically-timed critical period. We asked whether it could instead reflect the ordinary dynamics of learning, using automatic speech recognition models that simulate the international adoptee experience without maturational confounds. Models were trained on one language and then abruptly switched to a second. We found that traces of the first language persisted throughout second-language training, but mainly in the lowest, pre-phonemic layers. These traces were functional, as models with early exposure re-learned their lost first language 14% faster than naive models; this advantage held even against models adopted early from a related language and disappeared when the earliest layers were substituted from a non-adopted model. We argue that these critical-period effects reflect entrenchment of foundational representations rather than a maturational loss of plasticity, and that experience plays a central role in critical periods in language acquisition.
15. 【2608.25944】Unveiling Spectral Mechanisms in Training-Free LLM Text Detection
链接：https://arxiv.org/abs/2608.25944
作者：Haitong Luo,Xuying Meng,Weiyao Zhang,Wenji Zou,Shengfeng Lou,Xuefeng Jiang,Chungang Lin,Yujun Zhang
类目：Computation and Language (cs.CL)
关键词：Large Language Models, Language Models, Large Language, advancement of Large, distinguish human writing
备注：
点击查看摘要
Abstract:The rapid advancement of Large Language Models (LLMs) makes it increasingly difficult to distinguish human writing from machine-generated text. Training-free detection offers a scalable solution, yet common confidence-based metrics mainly measure average token probabilities and often miss the signal fluctuations that characterize human writing, which we call "generative vitality". Spectral analysis offers a way to capture this vitality, but its mechanism and practical boundaries remain underexplored. In this paper, we analyze spectral detection from both theoretical and empirical perspectives. We connect spectral energy to variance in proxy log-probability trajectories and explain how broader human token choices create the fluctuations used by frequency-domain indicators. We further show that the strength of this signal depends on text length and sampling range: spectral evidence is clearest for long, continuous, constrained generation, while short, fragmented, mixed, and edited settings require complementary confidence and fluctuation views. These findings clarify when frequency-domain detection works and provide guidance for future multi-dimensional detector design.
16. 【2608.25936】One Symptom, Three Levers: A Critical Review of On-Policy Self-Distillation
链接：https://arxiv.org/abs/2608.25936
作者：Justin Robert,Raheel Qader
类目：Machine Learning (cs.LG); Artificial Intelligence (cs.AI); Computation and Language (cs.CL)
关键词：On-policy distillation trains, distillation trains, trains a language, On-policy distillation, teacher
备注： 30 pages, 4 figures. Survey / critical review
点击查看摘要
Abstract:On-policy distillation trains a language model on its own generations while a teacher scores them token by token. It combines the dense supervision of imitation learning with the on-policy sampling of reinforcement learning. But it requires a second, larger model to act as teacher. On-Policy Self-Distillation (OPSD) removes that cost. The teacher is the model itself, conditioned on privileged information the student will not have at test time, such as a reference solution, a plan, or environment feedback. The teacher is no stronger than the student, only better informed. Early results were promising, with accuracy comparable to reinforcement learning at a fraction of the generated tokens. But the same asymmetry that produces the signal also biases it. One failure mode now dominates the field: collapse, the progressive narrowing of the set of reasoning paths the model can produce. Collapse is not specific to OPSD, though privileged information aggravates it. This review treats collapse as a symptom governed by three levers: (i) where the signal is applied, that is, how tokens are weighted; (ii) what the teacher is shown, that is, the nature of the privileged information; and (iii) when the signal changes, that is, the teacher's dynamics and the decay of guidance. We restrict our scope to mathematical reasoning, where the method originated and where its failure modes are best documented. We report no new experiments. The contribution is structural: a shared vocabulary for phenomena named differently across papers, and a clear line between what is settled and what is still disputed.
17. 【2608.25927】Code World Model: Coding Agent as World Brain
链接：https://arxiv.org/abs/2608.25927
作者：Yiwen Chen,Guosheng Lin,Chi Zhang
类目：Computer Vision and Pattern Recognition (cs.CV); Artificial Intelligence (cs.AI); Computation and Language (cs.CL)
关键词：complex environments evolve, existing video-based world, mechanisms governing world, models primarily learn, primarily learn dynamics
备注： Project Page: [this https URL](https://buaacyw.github.io/cwm/)
点击查看摘要
Abstract:World models aim to simulate how complex environments evolve under actions and events, yet existing video-based world models primarily learn dynamics from visual observations, which reveal outcomes rather than the underlying knowledge, rules, and mechanisms governing world evolution. This makes it difficult to maintain persistent consequences and support coherent, open-ended evolution. We introduce Code World Model, a framework that separates world evolution from visual realization by combining the reasoning and coding capabilities of language models with the generative priors of video models. A coding agent serves as the world brain, reasoning about events and their consequences and generating executable code to maintain persistent world state and perform rule-consistent evolution. To connect executable state with visual generation, we introduce a proxy representation that encodes frame-wise spatiotemporal constraints and is compiled into a proxy video, which conditions a video model to render high-fidelity visual observations. We further develop data pipelines for constructing aligned proxy-observation pairs from gameplay and real-world videos. After fine-tuning on paired gameplay data, MiniMax-H3 follows proxy-based spatiotemporal specifications from simple interactive worlds built by the coding agent while preserving rich visual details and dynamics. These results demonstrate the potential of combining code for persistent world evolution with video models for flexible visual realization, providing a new path toward open-ended world models.
18. 【2608.25926】Formal, Executable and Explainable Runtime Monitoring of Spoken Air Traffic Control Operational Procedures
链接：https://arxiv.org/abs/2608.25926
作者：Roberto Luvini,Giacomo Longo,Alessandro Armando,Enrico Russo
类目：Artificial Intelligence (cs.AI); Computation and Language (cs.CL); Audio and Speech Processing (eess.AS)
关键词：Air traffic control, executed through spoken, traffic control procedures, control procedures, spoken exchanges
备注：
点击查看摘要
Abstract:Air traffic control procedures are executed through spoken exchanges between controllers and pilots. These interactions are essential to the safety of air transportation: failures in their execution can create severe operational hazards, as evidenced by past fatal accidents. Assessing whether an instruction has been followed requires relating what was said to the aircraft concerned, its state, and the obligations that pilots must meet. We present a runtime verification framework that monitors such procedures by checking controller-pilot exchanges, surveillance data, and onboard observations. The framework parses radio communications into events linked to the entities they concern and merges them with surveillance and onboard observations into a time-stamped trace. The ICAO-derived obligations as formalized as temporal formulas with explicit time bounds and evaluated over execution traces. Every violation is reported along with the breached obligations and the observations that support the verdict. With real traffic, the complete pipeline reaches an F1 of 0.85 against blind human-annotated violations; in 1,495 synthetic situations derived from two public corpora, the monitor logic returns the expected verdict in every case. In two historical accidents reconstructed from official investigation reports, the monitor identifies the same procedural deviations documented by the investigators.
19. 【2608.25922】Query-Side Attacks on GNN-Based KGQA: Tracing Failures from Entity Linking to Answer Generation
链接：https://arxiv.org/abs/2608.25922
作者：Pankaj Kumar,Subhankar Mishra
类目：Computation and Language (cs.CL); Artificial Intelligence (cs.AI); Information Retrieval (cs.IR)
关键词：Graph Question Answering, GNN-based Knowledge Graph, pipelines process queries, Question Answering, Knowledge Graph Question
备注：
点击查看摘要
Abstract:GNN-based Knowledge Graph Question Answering (KGQA) pipelines process queries through four discrete stages: entity linking, subgraph retrieval, GNN reasoning, and answer generation. Standard robustness evaluations conflate stage-level failures into a single end-to-end metric, obscuring both the source of brittleness and the appropriate mitigation target. We ask which stage fails, and why, when the pipeline is subjected to adversarial perturbations on the input question. We introduce a stage-isolation protocol with two answer-preserving adversarial perturbations verified against the knowledge graph: Compositional Restructuring (CR) and Relation Synonym Swap (RS) target distinct stages while leaving entity seeds intact. Evaluated across ComplexWebQuestions and WebQSP, the results run counter to prevailing assumptions: the GNN reasoning stage retains near-baseline accuracy when the subgraph is intact, while subgraph construction accounts for over 99\% of the end-to-end collapse under CR, occurring even when the gold answer is present in 74\% of retrieved subgraphs. This exposes a fundamental distinction between answer presence and answer reachability that end-to-end metrics cannot detect, and places the mitigation target firmly at the subgraph construction stage rather than the reasoning model. Perturbed datasets and evaluation infrastructure are released at this https URL .
20. 【2608.25910】SAMpLE: A SystemC-AMS Machine LEarning-based Framework for Virtual Prototyping
链接：https://arxiv.org/abs/2608.25910
作者：Andrei Mihai Albu,Sara Vinco
类目：Computation and Language (cs.CL); Machine Learning (cs.LG)
关键词：Machine Learning, capture analytically, prototypes of embedded, embedded systems, difficult to capture
备注：
点击查看摘要
Abstract:Machine Learning (ML) is increasingly used in virtual prototypes of embedded systems to model behaviors that are difficult to capture analytically. However, integrating ML models into virtual platform simulation is still typically done through ad hoc solutions, which limits reuse, comparability, and reproducibility. This paper presents \textbf{\textit{SAMpLE}}, an open-source SystemC-AMS-based framework that integrates ML models as first-class Timed Dataflow (TDF) components through a standardized plug-and-play interface. SAMpLE provides two execution backends: a native C++ backend for online training of lightweight models, and an offline backend for executing externally developed models without requiring re-implementation in C++ or manual integration steps. The framework uses ONNX as a standard model exchange format to enable integration of externally trained ML models into SystemC-AMS simulations, and allows the evaluation of different ML-based solutions within the same testbench, dataset, and simulation workflow. The modular design and unified and reproducible environment will allow future extensions of SAMpLE to new models, without modifying the SystemC-AMS structure.
21. 【2608.25904】One Form to Transfer Them All: Pretraining Multilingual Language Models Beyond Native Orthography
链接：https://arxiv.org/abs/2608.25904
作者：Muge Zhang,Aaron Jencks,Krishna Badikela,Yulia Tsvetkov,Sachin Kumar
类目：Computation and Language (cs.CL)
关键词：shared subword vocabulary, subword vocabulary, writing systems, shared subword, mechanism that breaks
备注： EMNLP 2026 (Main Conference). 9 pages, 6 figures (plus appendix)
点击查看摘要
Abstract:Multilingual language models transfer knowledge across languages through shared subword vocabulary, a mechanism that breaks down when related languages use different writing systems. Prior work addresses this via script equalization (romanization or IPA transcription), but direct comparisons are rare; the focus has been on encoder-only models, with most work adapting existing pretrained models. We systematically compare different input representations in autoregressive multilingual pretraining, comparing orthographic text, IPA, and romanization in a controlled setup across three scales (467M, 709M, and 1.03B) on eight languages in four typologically motivated pairs. Across a wide range of downstream tasks on seen and unseen languages, romanized pretraining yields the strongest cross-lingual transfer, and the advantage over text widens with scale. IPA improves over text in most settings but trails romanization. Surprisingly, finetuning a text-pretrained model on romanized data hurts performance on languages already covered by the base model, only marginally helping when the model lacks script coverage. Our results indicate that for multilingual models spanning typologically diverse scripts, to obtain maximum benefits, romanization should be treated as a core design choice applied at pretraining rather than a post hoc fix.
22. 【2608.25894】From Passive Response to Proactive Correction: Enhancing LLM Robustness Against Input Fact Perturbations
链接：https://arxiv.org/abs/2608.25894
作者：Ping Wang,Xiangguo Sun,Bingbing Xu,Guocong Li,Xiaofeng Meng
类目：Computation and Language (cs.CL)
关键词：Large language models, frequently produce confident, factually incorrect responses, Large language, frequently produce
备注： Accepted to the Main Conference of EMNLP 2026
点击查看摘要
Abstract:Large language models (LLMs) frequently produce confident yet factually incorrect responses when user inputs contain misleading premises, a phenomenon we attribute to fact perturbations in the input. Existing approaches to hallucination mitigation typically assume reliable user inputs, overlooking how such factual errors can actively mislead model reasoning. To address this vulnerability, we propose DEDUCE, a three-stage framework that transforms LLMs from passive responders into proactive error correctors. DEDUCE operates in three stages: (1) detect errors through fine-grained fact extraction and verification; (2) devise correction strategies via multi perspective deliberation; and (3) correct misconceptions while delivering reliable answers. We also present MisFactQA, a dataset containing factual errors of varying degrees, and propose new metrics for evaluating model robustness. Experiments on TruthfulQA, FalseQA, and our MisFactQA benchmark demonstrate that DEDUCE significantly improves both accuracy and error correction capability. Consistent gains across Qwen, LLaMA, and Gemma families confirm its effectiveness and scalability.
23. 【2608.25881】Loss-Based Active Learning for Neural Abstractive Summarization
链接：https://arxiv.org/abs/2608.25881
作者：Michail Ioannou,Tatiana Passali,George Michalopoulos,Grigorios Tsoumakas
类目：Computation and Language (cs.CL)
关键词：high-quality annotated data, requires high-quality annotated, Fine-tuning abstractive summarization, Fine-tuning abstractive, models requires high-quality
备注：
点击查看摘要
Abstract:Fine-tuning abstractive summarization models requires high-quality annotated data. However, obtaining such corpora is expensive and time-consuming, as it requires human annotators to read and comprehend long documents to create accurate summaries. Active learning mitigates this issue by selecting only the most informative instances for annotation, allowing models to achieve competitive results with significantly fewer labels. However, the application of active learning to summarization remains under-explored, and existing studies often suffer from instability and significant computational bottlenecks. To overcome these challenges, we propose LOBSTER (LOss-BaSed acTivE leaRning), a novel active learning framework designed specifically for abstractive summarization. LOBSTER improves performance by prioritizing unlabeled instances semantically similar to the model's current high-loss training examples, enabling the model to explicitly correct its specific weaknesses. Our empirical evaluation across three benchmark datasets and two summarization backbone models demonstrates that LOBSTER consistently matches or outperforms current state-of-the-art approaches while achieving a query selection speedup of up to 665x.
24. 【2608.25869】Anchoring Bias in LLM-as-a-Judge Systems: Prior Scores Compromise Evaluation Independence
链接：https://arxiv.org/abs/2608.25869
作者：Ante Kapetanovic,Kemal Altwlkany,Andro Mercep,Tomislav Duricic,Emanuel Lacic
类目：Computation and Language (cs.CL)
关键词：increasingly assess generated, Large language models, assess generated content, Large language, increasingly assess
备注： 10 pages, full research paper, to appear in proceedings of the 35th ACM International Conference on Information and Knowledge Management (CIKM '26), 2026
点击查看摘要
Abstract:Large language models (LLMs) increasingly assess generated content, giving rise to the LLM-as-a-Judge paradigm. These systems now score outputs, filter content, and gate iterative refinement in production pipelines, where each judgment is often assumed to be independent of earlier evaluations. We test this assumption using three prompt conditions: no metadata, revision framing, and anchored metadata containing revision, attempt, and prior-score fields. We show that prior scores, even when included only as context metadata, anchor judgments and systematically shift ratings toward their values. Across 192,000 attempted evaluations (185,271 successful), seven out of the eight evaluated models have 95% task-stratified bootstrap intervals below zero for the total anchored-metadata effect on 20 fixed texts. Cohen's $d$, a standardized measure of the difference between score distributions, reaches an absolute value of 0.71. Token-level analysis of selected model-task probes suggests a threshold-like response pattern: introducing anchored metadata produces a marked redistribution of output-score probabilities, while changing the anchor value within the tested below-threshold range produces comparatively little additional variation. On categorical industry data with human-labeled ground truth, anchored metadata blocks 48% of error corrections and flips 10.18% of correct judgments toward an assigned wrong label, demonstrating the bias extends beyond numerical scoring to categorical decisions. Neither Chain-of-Thought nor a metadata-disregard warning reduces the total effect, although the warning improves the paired accuracy effect relative to baseline in the industry experiment. Reliable LLM evaluation demands careful context engineering rather than an assumption of impartiality. Effective mitigation must be validated for the intended model and task or domain.
25. 【2608.25854】Key Point Analysis Needs Structure Recovery: Task Definition, Dataset Diagnosis, and a Structure-Aware Benchmark
链接：https://arxiv.org/abs/2608.25854
作者：Zhiqiang Shi,Oana Cocarascu
类目：Computation and Language (cs.CL); Machine Learning (cs.LG)
关键词：Key Point Analysis, Point Analysis, key points, KPA, aims to identify
备注：
点击查看摘要
Abstract:Key Point Analysis (KPA) aims to identify a concise set of key points that summarize a collection of arguments together with their prevalence. We argue that KPA is fundamentally a structured prediction problem that requires recovering semantic groupings, generating representative key points, ensuring coverage, and estimating prevalence. Under this formulation, we show that existing KPA benchmarks suffer from limitations in grouping quality, redundancy, coverage, and argument-key point mappings, causing ceiling violation and selection failure in reference-based evaluation. To support future research on true KPA, we introduce a structure-aware, distribution-sensitive benchmark built via a human-in-the-loop re-annotation. Human and LLM evaluations consistently show that the resulting structures yield more coherent groupings, higher-quality key points, better coverage, and more reliable prevalence estimates than existing annotations. We further release several annotation resources to support research on KPA evaluation, argument-key point matching, explainable KPA, and LLM-as-a-judge methodologies, and outline a research agenda for true KPA.
26. 【2608.25832】Skill Issue: Are Skills Language-Invariant in LLMs?
链接：https://arxiv.org/abs/2608.25832
作者：Bobby Cheng,Adam Gaber,Zhengyuan Liu,Catherine Arnett,Omer Goldman,Cheston Tan,Leshem Choshen
类目：Computation and Language (cs.CL); Artificial Intelligence (cs.AI); Computer Science and Game Theory (cs.GT); Machine Learning (cs.LG)
关键词：access knowledge inconsistently, Large language models, Large language, models access knowledge, Large
备注：
点击查看摘要
Abstract:Large language models access knowledge inconsistently across languages, but to what extent do they differ in their skill sets when interacting with different languages? This work quantifies cross-lingual skill inconsistency orthogonally from knowledge and general benchmark performance. We do this via multilingual self-play: two instances of the same model compete in a text-based game, each interacting through a different language interface. Since the model, opponent, rules, state space, and available actions remain fixed, this setting isolates the effect of language on the model's realized behavior. We build a multilingual extension to TextArena and evaluate three open-weight models across eight languages and six games covering spatial reasoning, imperfect information, resource allocation, and repeated interaction. We find that the same model can exhibit markedly different playing strength across languages, with systematic variation in win--loss margins, invalid actions, and strategic tendencies. Detailed analyses reveal language-specific failures in spatial reasoning, card-conditioned decisions, and optimal move selection. In some settings, changing only the intermediate reasoning language recovers much of the lost performance, suggesting that language can affect different stages of the decision process. These results show that skill discrepancies are a measurable major roadblock in the development of truly multilingual models. Better understanding these discrepancies can help us design models that perform more equitably across languages.
27. 【2608.25826】Unfolding Scientific Papers into Multi-Turn Generation Trajectories for Continued Pre-Training
链接：https://arxiv.org/abs/2608.25826
作者：Qiankai Xu,Qiguang Chen,Zixin Su,Wenhao Huang,Yue Gao,Jiaheng Liu,Ge Zhang
类目：Computation and Language (cs.CL); Artificial Intelligence (cs.AI); Machine Learning (cs.LG)
关键词：short web passages, synthetic-data work reconstructs, web passages, recovers only local, local thoughts
备注：
点击查看摘要
Abstract:A recent line of synthetic-data work reconstructs the thinking behind existing text rather than rewriting the text itself, but it operates on short web passages, recovers only local thoughts, and leaves the structure of whole documents untouched. Scientific papers are written to a clear and largely uniform structure and make a natural substrate for lifting this paradigm to the document level. We present a pipeline that unfolds each paper into a multi-turn generation trajectory in which a teacher model reconstructs the writing process of the whole paper: a writing request, a global plan, and pre-writing deliberation for each section. All section texts and the abstract are kept verbatim from the source paper. We apply the pipeline to quality-filtered arXiv papers and obtain a corpus for continued pre-training (CPT) that is roughly twice the size of the source text. The same reverse construction extends to instruction data and evaluation. Treating real paper text as the answer yields an SFT dataset. Anchoring tasks in held-out papers yields PAW-Bench, an academic-writing benchmark whose tasks carry their own rubrics and checklists. In controlled experiments CPT on our corpus followed by supervised fine-tuning on public datasets improves writing benchmarks broadly while preserving general reasoning and improving long-document reading. The writing gain persists even when every model is fine-tuned on a dedicated writing SFT dataset. Mixing our SFT data into that recipe lifts academic writing further.
28. 【2608.25824】Localize-Then-Decide Guarantees for LLM Judgments
链接：https://arxiv.org/abs/2608.25824
作者：Xinyu Li,Yi Zhou,Guanqun Cao,Zeyu Fu,Tianjin Huang,Gaojie Jin
类目：Computation and Language (cs.CL)
关键词：Large language models, judgments remains challenging, assess output quality, Large language, human judgments remains
备注： Accepted to Findings of the Association for Computational Linguistics: EMNLP 2026. Code: [this https URL](https://github.com/llm2409/Localize-Then-Decide)
点击查看摘要
Abstract:Large language models (LLMs) are increasingly used as evaluators to assess output quality and preference alignment, yet providing reliable guarantees of agreement with human judgments remains challenging. Recent work introduces confidence-thresholding methods that provide such guarantees for pairwise comparisons, relying on the assumption that higher estimated confidence implies lower disagreement risk with humans. However, this assumption can break down when the number of candidate responses increases, since distributing probability mass across many alternatives can distort confidence estimates. To address this issue, we propose a Localize-Then-Decide framework. First, conformal prediction localizes a small shortlist that contains the human-preferred response with high probability. Then, a calibrated confidence-based rule selectively chooses a single response from this shortlist or abstains. This design restores the monotonic relationship between confidence and disagreement risk and enables high-probability agreement guarantees. Experiments with multiple candidate sizes across several datasets and judge LLMs demonstrate that our framework consistently achieves higher guarantee success rates and substantially higher coverage than single-stage baselines.
29. 【2608.25768】MoganBert-TR: A Turkish Encoder Foundation Model Trained from Scratch with a CLM-to-MLM Curriculum
链接：https://arxiv.org/abs/2608.25768
作者：Furkan Yilmaz,Habibe Aleyna Tasdemir,Muhammed Faruk Gozay
类目：Computation and Language (cs.CL); Artificial Intelligence (cs.AI)
关键词：adopted modern architectures, pretraining objective fixed, masked language modelling, language modelling, Turkish encoder models
备注： 32 pages, 4 figures, 18 tables. Model weights, tokenizer, embedding model and evaluation code: [this https URL](https://huggingface.co/moganai)
点击查看摘要
Abstract:Turkish encoder models have adopted modern architectures while leaving the pretraining objective fixed at masked language modelling. This paper introduces MoganBert-TR, a 149M-parameter Turkish encoder foundation model trained from scratch on a language-specifically filtered corpus, together with an embedding model derived from it (MoganBert-Embed). MoganBert-TR is trained over 237.3B tokens with a two-stage CLM-to-MLM curriculum: causal language modelling first, masked language modelling for the remainder, with the transition made inside the stable phase of a WSD schedule. In a controlled ablation under an equal step budget, this design outperforms pure MLM by 2.7-3.7x on Turkish MS MARCO retrieval; the measured mechanism is embedding geometry, where a single direction absorbs 28.1% of the variance under pure MLM against 11.9% under the curriculum. Long-context extension and learning-rate decay are then split into two branches after a shared prefix: running the final portion of decay at 1024 context improves the TrGLUE average by 0.49 +/- 0.26 points across five paired seeds (p = 0.013) and beats a model-soup alternative by 0.75 points at ~4.3% additional cost. MoganBert-TR attains 78.41 on TrGLUE, the best among the Turkish ModernBERT models compared, and 77.73 on TabiBench, where it leads two of the eight categories with the largest margin on code retrieval (+3.62 points over TabiBERT). MoganBert-Embed, produced through teacher distillation and multi-signal contrastive fine-tuning, ranks first among student models on the MTEB(Turkish) overall average with 68.30 and reaches 99.5% of its 7.57B-parameter teacher's score with a 51x smaller backbone. The accompanying 50,048-token tokenizer outperforms all compared Turkish tokenizers on compression and fertility across two independent test sets. Weights, tokenizer, embedding model and evaluation code: this https URL
Comments:
32 pages, 4 figures, 18 tables. Model weights, tokenizer, embedding model and evaluation code: this https URL
Subjects:
Computation and Language (cs.CL); Artificial Intelligence (cs.AI)
MSC classes:
68T50, 68T07
ACMclasses:
I.2.7; I.2.6
Cite as:
arXiv:2608.25768 [cs.CL]
(or
arXiv:2608.25768v1 [cs.CL] for this version)
https://doi.org/10.48550/arXiv.2608.25768
Focus to learn more
arXiv-issued DOI via DataCite (pending registration)</p>
30. 【2608.25761】Beam Search, Self-Consistency, and the Limits of Inference-Time Scaling for Grammar-Constrained Text-to-SQL in Small Language Models
链接：https://arxiv.org/abs/2608.25761
作者：Ty Chermsirivatana,John MacCormick
类目：Computation and Language (cs.CL); Artificial Intelligence (cs.AI)
关键词：language models involves, models involves reducing, large language models, model size, model
备注： 7 pages, 2 figures
点击查看摘要
Abstract:One common trade-off in the use of large language models involves reducing the size of the model while increasing the amount of computation at inference time, for example by using a wider beam search. In this paper, we examine the constrained case of this "model size vs. inference compute" trade-off, in which the model outputs are constrained by a strict grammar at inference time. Our results demonstrate that the constrained trade-off behaves differently from the unconstrained trade-off. We investigate the task of converting a prose query into an equivalent SQL query (text-to-SQL). Performance is evaluated on the Spider text-to-SQL benchmark, using the Qwen2.5-Instruct model family ranging in size from 0.5B to 7B parameters, all at 4-bit precision. We experiment with two approaches to varying inference compute: (i) beam search with a variable number of beams; and (ii) sample+vote, i.e., sampling several constrained outputs and then voting on their execution results, where the number of samples is varied. On the 1034-example development set, we find that: (a) both beam search and sample+vote improve accuracy, especially on smaller model sizes; (b) the "model size vs.\ inference compute" trade-off is not advantageous in this experiment, because moving to a larger model size typically results in higher accuracy than increasing inference compute on the same model size; (c) beam search outperforms sample+vote at a matched inference budget. This latter result is of particular interest since it contrasts with the findings of the unconstrained trade-off.
31. 【2608.25741】Why Does Graph Learning Fail to Fully Benefit from a Text Teacher?
链接：https://arxiv.org/abs/2608.25741
作者：Fumiaki Kimino(1),Ryoma Sato(1 and 2) ((1) SOKENDAI, (2) National Institute of Informatics)
类目：Machine Learning (cs.LG); Computation and Language (cs.CL)
关键词：represent complex interactions, Graph neural networks, GNN encoder pretrained, E-step teacher, E-step
备注： 21 pages, 1 figure, 7 tables
点击查看摘要
Abstract:Graph neural networks (GNNs) are widely used to represent complex interactions and relationships among entities. We investigate a multimodal model that combines two complementary ideas: a self-supervised method that enables a GNN encoder pretrained on one dataset to operate directly on another dataset with a different node-feature dimensionality, without rebuilding the model or realigning the data; and an alternating optimization method that updates a language-model module in an E-step and a GNN module in an M-step, rather than jointly training a large language model and a GNN end to end on a large graph. Despite expectations, the combined model did not sufficiently improve predictive performance. We identify six factors: (1) an external anchor in the E-step has a strength-safety trade-off: a weak anchor has little effect, whereas an overly strong anchor can damage the graph representation; (2) the knowledge of the E-step teacher is not injected directly into the GCN embedding Z; (3) the representation space constructed in the M-step is not optimized for the same objective as the E-step teacher space, resulting in a compromise representation for target classification; (4) GCN propagation averages a node's own textual information with information from its neighbors; (5) cosine alignment does not guarantee axes that are discriminative for classification, so stronger geometric alignment with the E-step text anchor need not sufficiently improve the target decision boundary or classification performance; and (6) the force that preserves the source-side self-supervised geometry in the M-step conflicts with the force that moves the representation toward the E-step teacher. We support these observations through a staged set of experiments that varies the influence of the E-step.
32. 【2608.25717】When RAG Fails to Equalize: Geo-bias in Factual Question Answering over Public Companies
链接：https://arxiv.org/abs/2608.25717
作者：Abhinav Havaldar,Enrico Santus
类目：Computation and Language (cs.CL); Artificial Intelligence (cs.AI)
关键词：Retrieval-augmented generation, mitigate factual errors, retrieval uniformly compensates, large language models, widely assumed
备注： 10 pages, COLM
点击查看摘要
Abstract:Retrieval-augmented generation (RAG) is widely assumed to mitigate factual errors in large language models (LLMs), but it remains unclear whether retrieval uniformly compensates for missing knowledge. We study this question in a controlled factual QA setting over public companies, constructing a benchmark of approximately 2,000 firms across global equity indices. We evaluate six LLMs on four atomic attributes under four conditions: no-context, perfect context, misleading context, and distraction context. We find strong geographic disparities in no-context accuracy, indicating uneven parametric knowledge. While perfect context improves performance, it does not eliminate these gaps: gains are correlated with baseline accuracy, suggesting retrieval effectiveness is coupled to internal representations. Under misleading context, models frequently copy incorrect information. Larger models improve overall performance but do not remove these structural effects. These results challenge the view of RAG as a universal corrective and highlight the interaction between model knowledge, context quality, and entity representation.
33. 【2608.25677】Learning New Facts with QLoRA: An Acquisition-Retention Frontier
链接：https://arxiv.org/abs/2608.25677
作者：Estelle Zheng,Sébastien Warichet,Emmanuel Helbert,Christophe Cerisara
类目：Computation and Language (cs.CL); Artificial Intelligence (cs.AI); Machine Learning (cs.LG)
关键词：Parameter-efficient fine-tuning, number of parameters, small number, Parameter-efficient, Abstract
备注： accepted EMNLP 2026 Findings
点击查看摘要
Abstract:Parameter-efficient fine-tuning is often assumed to preserve pretrained capabilities because it updates only a small number of parameters. We show that this assumption depends strongly on adapter capacity. We study factual acquisition in a controlled OpenStreetMap-derived benchmark where Qwen3-4B must acquire anonymized geographic associations while retaining unrelated capabilities. Comparing full fine-tuning (FFT) with quantized low-rank adaptation (QLoRA) at ranks 8, 16, 32, and 64, we find that rank induces a clear acquisition--retention frontier. Low-rank QLoRA preserves out-of-domain (OOD) performance but acquires fewer facts, whereas higher ranks improve same-fact paraphrase generalization at an increasing cost in performance on unrelated benchmarks. FFT behaves as a conservative baseline: it retains general capabilities well, but does not reach the highest factual-acquisition regime. Distributional, weight-space, and spectral diagnostics mirror this behavioral trade-off, with higher-rank QLoRA moving farther from the pretrained model. A separate math adaptation experiment shows a weaker frontier, suggesting that the effect is most pronounced when adaptation must install new factual associations rather than reinforce skills already supported by pretraining. Code and data are available at this https URL.
34. 【2608.25662】Overview of SHROOM-Visions 2026: A Shared Task on Hallucination Detection in Large Vision-Language Models
链接：https://arxiv.org/abs/2608.25662
作者：Raúl Vázquez,Aman Sinha,Chuyuan Li,Claudio Savelli,Eduardo Calò,Emilio Raimond,Stella Frank,Hengyu Luo,Flavio Giobergia,Vincent Segonne,Lorenzo Vaiani,Jörg Tiedemann,Timothee Mickus
类目：Computation and Language (cs.CL)
关键词：UncertaiNLP Workshop co-located, textbf, SHROOM Shared Task, co-located with EMNLP, UncertaiNLP Workshop
备注： To appear at UncertaiNLP 2026 @ EMNLP 2026
点击查看摘要
Abstract:In 2026, we held the fourth iteration of the SHROOM Shared Task series: SHROOM-Visions (\textbf{S}hared-task on \textbf{H}allucinations and \textbf{R}elated \textbf{O}bservable \textbf{O}vergeneration \textbf{M}istakes in \textbf{Vision} language model\textbf{s}), which is hosted at the UncertaiNLP Workshop co-located with EMNLP 2026. Following the success of the 2024 and 2025 tasks, this time we aim to tackle hallucinations through a model-agnostic detection task focused on large vision-language models. Building on the recently introduced SHEEP dataset, designed for long-term evaluation across model generations, the task invites participants to detect and classify fine-grained hallucination spans in image-conditioned text generation (VQA, image captioning, etc.). The evaluation uses a five-class taxonomy of hallucinations spanning four languages: Chinese, English, French, and Italian. The shared task generated strong interest in the NLP community worldwide, with 27 teams contributing 600+ system submissions. The best systems achieve average scores of 0.58 in character-level correlation, 0.46 in label-conditioned correlation, and 0.51 in intersection-over-union (IoU) across four languages, outperforming the baselines by 30-40 points.
35. 【2608.25660】hink-Probe-Respond: Improving Large Language Models as Judges of Research Idea Novelty
链接：https://arxiv.org/abs/2608.25660
作者：Tim Schopf,Tobias Schreieder,Akiko Aizawa
类目：Computation and Language (cs.CL); Artificial Intelligence (cs.AI)
关键词：enabling efficient evaluation, accelerate scientific discovery, Automated novelty judgment, Automated novelty, efficient evaluation
备注： Accepted to EMNLP 2026 (Findings)
点击查看摘要
Abstract:Automated novelty judgment can accelerate scientific discovery by enabling efficient evaluation, refinement, and comparison of research ideas. While large language models are increasingly adopted for this task, we investigate a previously overlooked limitation in their judgment capabilities: despite generating reasoning rationales that closely mirror those of human experts, their final novelty judgments often diverge substantially. We demonstrate that this miscalibration stems from a systematic bias towards judging ideas as "medium novel". To mitigate this, we propose Think-Probe-Respond (TPR), a lightweight approach that probes latent novelty judgments from hidden states during the reasoning phase and uses the probed judgments to condition the final response. Across strong baselines, TPR improves novelty judgment performance by 22.30% and successfully mitigates the prevalent "medium novelty" bias.
36. 【2608.25655】Reconstructing the Right Episode: Evaluating Interleaved Conversational Memory Beyond Long Context
链接：https://arxiv.org/abs/2608.25655
作者：Zhexi Feng,Ruiyi Zhang,Yongbo Yang,Pengtao Xie
类目：Computation and Language (cs.CL); Artificial Intelligence (cs.AI)
关键词：chat assistants increasingly, assistants increasingly span, single long-running thread, challenging memory systems, chat assistants
备注： 19 pages, 6 figures, 30 tables. Accepted to the Main Conference of EMNLP 2026
点击查看摘要
Abstract:Conversations with chat assistants increasingly span many topics in a single long-running thread, challenging memory systems. Existing long-context and memory benchmarks often expose session or topic boundaries, or probe direct personal-memory questions. These settings understate a harder assistant-memory regime: a flat mixed-topic thread where the system must infer which earlier episode makes a later task decision valid. We introduce SCALE-QA, a constraint-grounded task QA benchmark for flat unsegmented threads targeting episode integrity failure. The dataset contains 3,000 audited questions across 10 domains, uses deterministic four-way multiple-choice grading, and includes a deterministic runtime builder; experiments use all 3,000 questions through 128k and a stratified 400-question diagnostic at 1M. SCALE-QA questions are ordinary task-oriented requests whose correct answer depends on causally related evidence introduced earlier in the conversation. We also propose Temporal-Semantic Interleaved Memory Reconstruction (TSIM), which segments the turn stream into coherent episodes and indexes them through a hierarchical multi-view memory stack with deterministic episode-level summary and cluster-routing views. Experiments show that SCALE-QA challenges strong RAG baselines and long-context LLMs alike; across three open-source and proprietary LLM backends, TSIM achieves the highest accuracy in every backend setting, gaining 5.6-17.6 accuracy points over the strongest corresponding baseline.
37. 【2608.25654】Unmatched Does Not Mean False: Incomplete Reference Sets Can Reverse Calibration Rankings in Open-Ended Theory-of-Mind Tracking
链接：https://arxiv.org/abs/2608.25654
作者：Zhexi Feng,Wuxi Chen,Bingrui Zhang
类目：Computation and Language (cs.CL); Artificial Intelligence (cs.AI)
关键词：trackers emit valid, emit valid beliefs, valid beliefs absent, trackers emit, emit valid
备注： Main paper: 9 pages, 1 figure, 5 tables. Supplementary material: 23 pages
点击查看摘要
Abstract:Open-ended Theory-of-Mind (ToM) trackers emit valid beliefs absent from finite references. A finite-reference-plus-matcher pipeline marks unmatched outputs false, creating proxy labels that can reverse proper-score model selection on fixed outputs. Holding 259 beliefs and paired scores fixed, reference recoding lowers weighted prevalence from 0.783 to 0.295 and reverses strictly proper Brier risk: a frozen source-prior rule leads native confidence by 0.227 under reference labels and trails by 0.152 under blinded adjudication, in all six authored scenarios. A reference-only Platt recalibrator reverses further. An ICE-specific reversal appears in a released 301-question NQ-open DPR-BERT pipeline: its average-confidence baseline improves instance-level calibration error by 0.045 under exact match but worsens it by 0.074 under human correctness, with both intervals excluding zero. On independently authored OpenToM narratives, 90-96% of audited unmatched beliefs are literally true and the paired direction again reverses. An exact decomposition attributes the distortion to omitted truths, and a closed-form criterion correctly classifies comparisons from twelve released systems. Frozen-audit retrospective replay shows 50 attempted annotations recover ranking direction with probability at least 0.996. TriSource-Restore anchors full-frame reference labels and frozen automatic judgments to a probability-sampled human pilot, maintains at least nominal coverage, narrows intervals, and repairs confidence subject to a base-rate deployment gate.
38. 【2608.25643】A Token-Level Analysis of Sampled-Token Reverse-KL On-Policy Distillation
链接：https://arxiv.org/abs/2608.25643
作者：Bing Shao,Jiazheng Zhang,Long Ma,Yujiong Shen,Senjie Jin,Xin Guo,Yuming Yang,Mingxu Chai,Zhiheng Xi,Tao Gui,Qi Zhang,Xuanjing Huang
类目：Machine Learning (cs.LG); Computation and Language (cs.CL)
关键词：remains poorly understood, loss allocates updates, On-policy distillation, sampled loss allocates, tokens remains poorly
备注： 16 pages, 7 figures
点击查看摘要
Abstract:On-policy distillation (OPD) supervises a student on its own trajectories with token-level signals from a frozen teacher, yet how a sampled loss allocates updates across tokens remains poorly understood. We analyze the gradient of the per-token K2 estimator of reverse KL with respect to the student logits. The $\ell_1$ norm of this gradient factorizes into the absolute teacher--student log-probability gap and a student-side softmax factor that grows as the sampled token becomes less likely under the student. In our math-distillation runs, these per-token norms are highly non-uniform: low-student-probability tokens account for a disproportionate share of their sum and are also enriched in large teacher--student gaps. As a lightweight intervention suggested by this analysis, we study Surprise-aware Reweighting (SuRe), a detached, bounded weighting rule that further amplifies this existing allocation. Across two Qwen3 student scales, SuRe improves several math metrics over vanilla OPD and shows no clear degradation on the selected out-of-domain benchmarks. Our primary contribution is therefore a gradient-level characterization of reverse-KL OPD trained with the K2 estimator, with SuRe as one empirical instantiation.
39. 【2608.25637】AutoVerifier: Residual-Guided Non-Parametric Optimization for Reference-Based Answer Verification
链接：https://arxiv.org/abs/2608.25637
作者：Zebei Zhao,Zhihao Shi,Minqi Shi
类目：Computation and Language (cs.CL)
关键词：accurate outcome rewards, evaluating reasoning models, providing accurate outcome, outcome rewards, verifiable rewards
备注： 8 pages of main text, 5 figures, with appendices
点击查看摘要
Abstract:Reference-based verifiers are important for evaluating reasoning models and providing accurate outcome rewards in reinforcement learning with verifiable rewards. To improve verification accuracy, prior work has explored rule-based, model-based, and tool-augmented verifiers for checking answer equivalence across diverse answer forms. However, the equivalence of answer forms such as $1+3.14$ and $1+\pi$ may depend on the question and scoring criterion. We frame such implicit assumptions as verifier inductive biases. To address this challenge, we propose AutoVerifier, a residual-guided non-parametric optimization method that learns these biases from recurring verifier errors. Specifically, AutoVerifier records these biases in rule cards and promotes them to code modules or prompt guidance only after replay validation detects no direct regressions, keeping accepted updates auditable, editable, and reusable. Experiments on four verifier benchmarks demonstrate that AutoVerifier outperforms state-of-the-art verifiers by a large margin.
40. 【2608.25622】Plans You Can Check: Verifier-Grounded Learning of an Open-Weight Planner for Executable Video-Editing
链接：https://arxiv.org/abs/2608.25622
作者：Haoyu Wang,Cheng Feng,Liuyang Bian,Ruiyang Huang,Lei Wei,Yafei Wen,Xiaoxin Chen,Xiaoying Tang
类目：Computer Vision and Pattern Recognition (cs.CV); Computation and Language (cs.CL)
关键词：Practical video editing, Practical video, pixel generation, editor must turn, hard constraints
备注： Accepted to the Main Conference of EMNLP '26
点击查看摘要
Abstract:Practical video editing is not only pixel generation: an editor must turn a brief, a clip pool, music metadata, and hard constraints into an executable timeline. We study this decision layer as \emph{executable video-editing planning} and introduce RefineCut, which, unlike workflow systems that wrap a prompted frontier model, trains a compact open-weight planner for it. The planner edits a typed timeline through structured patches covering clip selection, trimming, ordering, transitions, and duration and music alignment; a deterministic verifier applies each patch and checks it against an explicit constraint ledger. Because editing has no single ground-truth repair, we do not imitate teachers directly: RefineCut replays every multi-teacher branch through the verifier and keeps verifier-best repairs as supervision. A second stage, RefineCut-Evo, lets the student score its own repairs with the verifier and a task rubric and trains on high-margin preference pairs, so the final $8$B planner runs in a closed verifier loop with no teacher calls at inference. On RefineCut-Bench ($3{,}578$ tasks, $7{,}971$ captioned clips, $499$ music tracks, explicit ledgers), verifier-replayed distillation lifts the planner from $0.620$ to $0.858$ on the protocol-specific Video-Editing Score and RefineCut-Evo reaches $0.924$; the gain transfers to Llama-3.1-8B and GLM-4-9B, and in the same closed loop the $8$B planner matches or exceeds its frontier teachers. Code and RefineCut-Bench are publicly released; see the Data Availability statement.
41. 【2608.25618】AWM: Answerable Working Memory for Long-Document VQA Agents
链接：https://arxiv.org/abs/2608.25618
作者：Dongzhuoran Zhou,Yuqicheng Zhu,Yule Liu,Zhen Yang,Rui Lu,Yuxiao Dong,Jie Tang,Evgeny Kharlamov
类目：Computation and Language (cs.CL)
关键词：Long-document visual question, Long-document visual, working memory, relies on VLM, terminal working memory
备注： EMNLP 2026 Findings. 16 pages, 4 figures, 9 tables
点击查看摘要
Abstract:Long-document visual question answering increasingly relies on VLM agents that retrieve candidate pages, inspect page images, write findings to working memory, and synthesize answers. Working memory should carry answer-supporting evidence across page inspections for later grounded answering, yet existing evaluation mainly checks final-answer correctness and evidence-page access. This creates a memory-quality blind spot: an agent may reach the right page and answer correctly while leaving behind memory too generic or incomplete to support answering once page context is removed. We introduce \emph{memory-only answerability}, a diagnostic that asks whether a reader can answer from the question and terminal working memory alone. Building on this diagnostic, \emph{Answerable Working Memory} (AWM) treats terminal working memory as an answerable evidence artifact, and AWM-GRPO incorporates this signal into the GRPO reward while preserving final-answer priority. Under GRPO, this reward assigns higher advantages to answer-correct trajectories whose terminal working memory remains answerable. On \textsc{MMLongBench-Doc}, even when gold evidence pages are provided, 42.5\% of correct answers still cannot be answered from terminal working memory alone. AWM-GRPO improves final-answer accuracy over the RAG baseline by 8.1 and 11.9 points on \textsc{MMLongBench-Doc} and \textsc{LongDocURL} and reduces the memory-missing-correct rate by 2.7 points over answer-only GRPO.
42. 【2608.25605】From Specialization to Generalization: Instruction-tuned LLMs for Robust Harmful Content Mitigation
链接：https://arxiv.org/abs/2608.25605
作者：Lukas Edman,Daryna Dementieva,Alexander Fraser
类目：Computation and Language (cs.CL)
关键词：general NLP tasks, Large language models, NLP tasks, Large language, general NLP
备注：
点击查看摘要
Abstract:Large language models (LLMs) demonstrate impressive performance across a wide range of general NLP tasks; however, their effectiveness in sensitive domains, such as hate speech detection, remains less clear. Prior studies comparing prompted LLMs with state-of-the-art encoder-based models (e.g., BERT variants (Roy et al., 2023; Dönmez et al., 2024)) have shown only marginal gains, suggesting that LLMs may not excel in hate speech detection or mitigation. In this work, we revisit this question through the lens of instruction tuning. By thoroughly unifying 36 English hate speech datasets spanning multiple labeling schemes, we fine-tune a generalist LLM, based on Qwen3 (Qwen Team, 2025), specifically for hate speech mitigation. Our results demonstrate not only state-of-the-art performance on in-domain benchmarks but also substantial improvements in cross-domain and cross-lingual generalization--areas where encoder-based specialist classifiers often struggle.
43. 【2608.25593】JIT-Agent: Scaling Harness Intelligence via Just-in-Time Harness Evolution
链接：https://arxiv.org/abs/2608.25593
作者：Guibin Zhang,Leo Lu,Fangzhou Xie,Kang Zhu,Junhao Wang,Zhifei Xie,Zhaochen Yu,Zihang Liu,Zhongxiang Sun,Qiankun Li,Yue Liao,Heng Chang,Xiaobin Hu,Qibing Ren,Wangchunshu Zhou,Shuicheng Yan
类目：Computation and Language (cs.CL); Machine Learning (cs.LG)
关键词：harness, Agent, model, agent harness, Agent capability
备注：
点击查看摘要
Abstract:Agent capability is not determined by the model alone. The agent harness, encompassing memory management, planning strategy, action protocol, and tool/skill orchestration, can dominate the contribution of the underlying foundation model. Yet harness design remains manual, task-specific, and fundamentally unscalable. We present JIT-Agent, a harness intelligence model trained to synthesize task-adaptive agent harnesses on the fly for arbitrary off-the-shelf agentic LLMs. We formalize the agent harness as a composable, machine-generatable artifact governed by a fixed four-module protocol, and train JIT-Agent to customize harnesses for a given task at hand, repair harnesses for stable and reliable execution, and self-evolve by distilling performance signals from an expanding archive of prior harness configurations. Equipped with JIT-Agent as a harness helper, DeepSeek-V4-Flash surpasses GPT-5.6 on DeepSearchQA (+9.1) and OdysseyBench (+4.3), while the already strong GLM-5.2 gains up to +20.2 points. Across controlled evaluations, JIT-Agent-generated harnesses are performance-competitive with mature agent runtimes such as OpenCode and Claude Code and consistently improve multi-scale model families of DeepSeek V4, Mimo-V2.5, and Qwen3.6. To our knowledge, JIT-Agent is the first model purpose-built for just-in-time harness generation, establishing harness intelligence as a trainable, transferable, and compounding dimension of agent capability orthogonal to model scaling.
44. 【2608.25583】GRIP: Granular Reward-Guided Parameter Interpolation for Efficient Reasoning
链接：https://arxiv.org/abs/2608.25583
作者：Lam So,Canhui Wu,Han Lin
类目：Computation and Language (cs.CL)
关键词：Reasoning-oriented large language, strong problem-solving performance, generating long chains, behavior substantially increases, substantially increases inference
备注： 13 pages, 8 figures
点击查看摘要
Abstract:Reasoning-oriented large language models often achieve strong problem-solving performance by generating long chains of thought, but this behavior substantially increases inference cost and latency. In contrast, instruction-tuned models tend to answer more concisely, yet often lack comparable reasoning ability. This accuracy-efficiency mismatch motivates a lightweight approach that combines the strengths of both models without full model retraining. In this paper, we propose GRIP (Granular Reward-guided Interpolation of Parameters), a reward-guided parameter interpolation framework for efficient reasoning. Given a reasoning model and an instruction model with identical architectures, GRIP assigns learnable interpolation ratios to individual modules and optimizes only these ratios while keeping both source models frozen. The interpolation ratios are trained with a reward signal that favors responses that are both correct and concise. Experiments show that GRIP achieves a better accuracy-efficiency trade-off than fixed or search-based merging baselines and further reveals module-wise fusion patterns associated with efficient reasoning.
45. 【2608.25579】Cross-Dataset Stability of Expert-Informed Skill Prompting and Fine-Tuning for Chinese Metaphor Identification
链接：https://arxiv.org/abs/2608.25579
作者：Yufeng Wu,Meichun Liu
类目：Computation and Language (cs.CL)
关键词：annotation policy, Metaphor-identification performance, change markedly, differ in text, text distribution
备注： 6 pages, 1 figure, 5 tables
点击查看摘要
Abstract:Metaphor-identification performance can change markedly across datasets that differ in text distribution and annotation policy. We examine whether a fixed expert-informed procedure produces a more even cross-dataset profile than task-specific parameter adaptation. Four prespecified conditions are compared for Chinese sentence-level metaphor identification: BERT fine-tuning (BERT-FT), QLoRA-based large language model fine-tuning (LLM-FT), direct zero-shot LLM prompting (LLM-ZS), and zero-shot prompting with a frozen procedural Skill (Skill-ZS). The Skill operationalizes established criteria involving contextual meaning, basic meaning, contrast, and comparison. Evaluation covers CMRE Test and two external datasets, CCIME and CMC. Fine-tuned scores are means over three seeds, whereas each zero-shot score comes from one deterministic configuration. Fine-tuning remains strongest on the native test set: BERT-FT reaches 91.76 Macro-F1. LLM-FT has the highest external mean (83.52), while Skill-ZS is close at 82.92 and has both the highest external floor (82.64) and the smallest observed range across all three datasets (4.08 points). In the matched zero-shot comparison, adding the Skill reduces metaphorical predictions on every dataset. This sharply lowers false positives on CCIME but increases false negatives on CMRE Test and CMC. The results position expert-informed Skill prompting as a complementary route to more even observed cross-dataset performance, while fine-tuning retains its advantage in native-data accuracy. To our knowledge, this is the first study to compare an expert-informed procedural Skill with task-specific fine-tuning in the same cross-dataset evaluation of Chinese sentence-level metaphor identification.
46. 【2608.25574】Generative vs. Encoder Large Language Models for ASR Evaluation: A Comparative Study
链接：https://arxiv.org/abs/2608.25574
作者：Thibault Bañeras-Roux,Shashi Kumar,Driss Khalil,Sergio Burdisso,Petr Motlicek,Shiran Liu,Mickael Rouvier,Jane Wottawa,Richard Dufour
类目：Computation and Language (cs.CL)
关键词：Automatic Speech Recognition, Word Error Rate, Automatic Speech, Speech Recognition, reflects semantic similarity
备注：
点击查看摘要
Abstract:Automatic Speech Recognition (ASR) is typically evaluated using Word Error Rate (WER), which poorly reflects semantic similarity. While embedding-based metrics correlate better with human judgments, the respective roles of encoder and decoder-based Large Language Models (LLMs) remain underexplored. This paper presents a comparative study of both families for ASR evaluation. We analyze BERTScore and SemDist across different LLMs, layers, and pooling strategies, showing that both metrics can achieve strong correlation with human judgments when properly configured. For decoder models, we investigate generative LLMs in two settings: pairwise hypothesis selection via prompting and direct qualitative error classification. Our results show that encoder-based metrics remain highly competitive, while generative LLMs perform strongly in hypothesis comparison and improve the interpretability of ASR evaluation.
47. 【2608.25569】Controllable Affective Generation via Latent Vector Steering
链接：https://arxiv.org/abs/2608.25569
作者：Xixian Yong,Siyuan Chang,Yingying Zhang,Xian Wu,Xiao Zhou
类目：Computation and Language (cs.CL); Artificial Intelligence (cs.AI)
关键词：Large Language Models, Large Language, produce emotionally flattened, emotionally flattened responses, Language Models
备注：
点击查看摘要
Abstract:Large Language Models (LLMs) often produce emotionally flattened responses after alignment, limiting their effectiveness in affect-sensitive applications. In this paper, we propose EmoVec, a lightweight framework for controllable affective generation via latent vector steering. EmoVec extracts emotion-specific directions from paired neutral and emotion-conditioned responses using contrastive activation addition, and further refines them through task-specific debiasing and principal subspace removal. During inference, these vectors are injected into the final residual stream with static or scenario-adaptive scaling, enabling continuous control over emotional intensity without updating model weights. Experiments across three LLMs and eight emotions show that EmoVec consistently improves emotional salience while largely preserving semantic content, fluency, and coherence. Ablation studies and human evaluation further confirm the effectiveness of vector purification and adaptive scaling, establishing EmoVec as a practical inference-time method for affective control in deployed LLMs.
48. 【2608.25561】EgoArgus: Benchmarking VLMs as Situational Assistants for Modality-Grounded User Supports
链接：https://arxiv.org/abs/2608.25561
作者：Yu-Chien Tang,Yu-Hsiang Liu,An-Zi Yen
类目：Computation and Language (cs.CL)
关键词：follow user dialogue, perceive first-person environments, first-person environments, follow user, user dialogue
备注： Accepted by EMNLP 2026 main conference
点击查看摘要
Abstract:VLMs are increasingly positioned as daily assistants that perceive first-person environments, follow user dialogue, and decide how to help. Existing egocentric benchmarks mainly evaluate visual understanding in isolation, leaving open whether models can arbitrate between visual evidence and user-provided language when the two are helpful, irrelevant, or conflicting. We introduce EgoArgus, a human-annotated dataset for evaluating egocentric assistants on understanding and decision tasks in five dialogue-video daily scenarios. Our results demonstrate that it is still challenging for current VLMs as reliable egocentric assistants, which requires identifying which modality is trustworthy and deciding when intervention is warranted. Deeper analysis also shows that existing modality bias mitigation methods are quite restricted to enhance performance, providing insights to aid practioners into the deployment of current VLMs as daily assistants.
49. 【2608.25555】Virgil: Navigating Explainability for Transformer-based Language Models
链接：https://arxiv.org/abs/2608.25555
作者：Martino Ciaperoni,Sezer Kutluk,Benedetta Muscato,Marta Marchiori Manerba,Fosca Giannotti
类目：Computation and Language (cs.CL); Machine Learning (cs.LG)
关键词：high-stakes applications, transformer-based language models, deployed in high-stakes, explainability tools, language models
备注： ECML PKDD 2026, Demo Track
点击查看摘要
Abstract:Explainability for transformer-based language models is becoming crucial as these systems are deployed in high-stakes applications. As a result, the ecosystem of explainability tools is rapidly evolving, becoming richer, but also more fragmented and harder to navigate. To address this challenge, we present Virgil, an interactive system that lets practitioners and researchers, including non-experts, navigate explainability tools for transformer language models. Supported by a curated knowledge base, the system enables users to discover and compare explainability tools within a unified interface.
50. 【2608.25553】When Stale Constraints Go Unchecked: Budgeted Verification Failures in Inherited Agent Memory
链接：https://arxiv.org/abs/2608.25553
作者：Kazuki Nakayashiki
类目：Information Retrieval (cs.IR); Artificial Intelligence (cs.AI); Computation and Language (cs.CL)
关键词：newer authoritative record, inherits a consolidated, true when written, newer authoritative, authoritative record
备注： 18 pages, 3 figures, 7 tables. Manuscript, LaTeX source, all 5,400 episode files, frozen specifications, timestamp proofs, analysis and generator scripts are archived at Zenodo: doi: [https://doi.org/10.5281/zenodo.22108558](https://doi.org/10.5281/zenodo.22108558)
点击查看摘要
Abstract:An agent that inherits a consolidated memory may inherit a constraint that was true when written and has since been withdrawn by a newer authoritative record. Under a scarce verification budget, does the agent recover the withdrawal, and if not, is the error avoidable without spending more? We model supersession explicitly -- historical provenance is immutable; what changes is which record is current -- and assign by design the memory's form, the world's state (source current or superseded), and the verification policy at a fixed budget of two records: the agent's own allocation, or the same budget with one slot re-assigned to the critical provenance path or to a random record. With a constraint stated, agents inspected its provenance path in about one episode in five; when that constraint had been superseded, native allocation produced stale-consistent decisions in 77.3%, 74.7% and 74.7% of episodes across a primary run, a fresh-wording replication and a held-out domain. Re-assigning one slot to the critical path raised current-record-consistent decisions by +74.0, +72.7 and +61.3 points, positive in six of six models in each of those runs, and changed nothing when the record agreed with the memory. The held-out scenario was later found to contain a temporal inconsistency; a robustness replication with one sentence corrected, deposited externally before execution, gave +73.3 points and is reported alongside the original. The intervention uses knowledge of the critical path and is not a scheduler; it identifies that the share of stale-memory error attributable to verification allocation is close to its structural ceiling. Memory systems may need freshness or supersession signals separate from relevance.
51. 【2608.25542】Reflection Steering: Disentangling Reflection from Reasoning in Activation Space for Token-Efficient Inference
链接：https://arxiv.org/abs/2608.25542
作者：Jiarui Hu,Zhiyuan Wen,Xiaoyun Liu,Jiaxing Shen,Yu Yang
类目：Machine Learning (cs.LG); Computation and Language (cs.CL)
关键词：Large reasoning models, produce reasoning traces, Large reasoning, traces with verification, reflection steering
备注： 8 pages, 5 figures, 4 tables
点击查看摘要
Abstract:Large reasoning models often produce reasoning traces with verification, revision, and backtracking. When reflection merely re-checks established results, it wastes reasoning tokens and increases latency. Most existing reflection steering methods add a label-derived mean-difference direction across preset layers, but its entanglement with reasoning and length signals destabilizes the accuracy-efficiency trade-off. In this paper, we propose Reflection Steering, a training-free framework for controlling reflection-associated computation within LLMs by disentangling reflection-related activations from general reasoning. Specifically, we contrast reflective and non-reflective hidden states at each LLM layer, denoise the resulting reflection directions with PCA, and orthogonalize them against general-reasoning directions. To limit downstream amplification from early-layer interventions, we calibrate each layer across multiple intervention strengths on a small set, retain only stable layers, and apply bounded projection removal to their residual-stream activations. We conduct extensive experiments across two public benchmarks and three open-weight LLMs against state-of-the-art activation-steering baselines. Results show that Reflection Steering reduces reasoning tokens by 16.9% on average across six matched settings. Besides, our method further introduces a bounded reflection intervention-strength parameter $\alpha$, enabling deployment-time adjustment to balance token savings, accuracy, and generation stability.
52. 【2608.25531】ClueWeaver: Reward-Guided Dual-Agent Evidence Reasoning for Compact LLMs on Literary Long Narratives
链接：https://arxiv.org/abs/2608.25531
作者：Jihao Zhu,Zhiwei Yang,Wenxiao Zhang,Junqian Zhao,Qi You,Fangqi Wang,Zheyuan Deng,Hanzhe Yang,Yu Liu,Jin B. Hong
类目：Computation and Language (cs.CL)
关键词：social science research, science research requires, research requires close, requires close reading, Humanities and social
备注： Accepted by ICONIP 2026
点击查看摘要
Abstract:Humanities and social science research requires close reading of long narrative materials such as novels, scripts, archives, and case reports, yet many users have limited access to costly proprietary long-context models. Compact, locally deployable language models are a practical alternative, but directly feeding them an entire long context remains costly, hard to inspect, and prone to missing sparse evidence. We present ClueWeaver, an evidence-aware dual-agent framework for long-narrative question answering with compact local models. A Finder identifies passages containing answer-critical clues through retrieval-guided segmentation, while an Interpreter derives the answer from the selected evidence, produces rationales with paragraph-ID citations, and applies an internal self-calibration pass for high-risk questions. Both agents are optimized with reward-guided reinforcement learning: Finder rewards emphasize evidence retention and faithful paragraph-ID references, and Interpreter rewards emphasize correctness, grounding, and concise explanations. This decomposition makes evidence selection and reasoning more inspectable than end-to-end prompting. Experiments across multiple long-context narrative question answering and claim verification settings show that ClueWeaver substantially improves local end-to-end language models while providing evidence coverage and paragraph-referenced reasoning traces. Code is available at this https URL.
53. 【2608.25523】OPAS: Workflow-Aware Prefix-State Scheduling for Multi-Agent LLM Serving
链接：https://arxiv.org/abs/2608.25523
作者：Hongqiu Ni,Han Tian,Chi Zhang,Guopeng Li,Haisheng Tan
类目：Computation and Language (cs.CL)
关键词：large language model, long system-prompt key-value, accelerates future calls, multi-agent large language, Prefix caching introduces
备注： 8 pages
点击查看摘要
Abstract:Prefix caching introduces a fundamental tradeoff in multi-agent large language model (LLM) serving: retaining a long system-prompt key-value (KV) cache for an agent accelerates future calls, yet it reduces the GPU memory available for batching concurrent requests. In multi-stage workflows, existing schedulers tend to prioritize either immediate prefix locality or overall workflow progress. However, under a shared KV cache budget, optimizing either objective in isolation can prolong tasklevel job completion time (JCT) through downstream delays or frequent prefix replacement. To strike a balance, we here propose TOPAS, a Task-Oriented Prefix-Aware Scheduler that jointly decides which agent prefixes to keep in the cache and which requests to schedule for execution. TOPAS scores candidate post-decision states by trading off the expected reduction in each task's longest remaining service path against the near-term benefit of downstream prefix reuse, accounting for the costs of prefix movement and preemption. A task-level aging mechanism is also incorporated to prevent starvation. We implement TOPAS within the SGLang framework and assess its performance on three synthetic DAGs and two MetaGPT software-development workflows. Compared with the best performing baseline for each workload and metric, TOPAS reduces the mean/p99 JCT by up to 39.8%/49.4% on the synthetic workloads, while lowering mean JCT by 9.8% on MetaGPT-SOP and mean/p99 JCT by 22.0%/26.6% on MetaGPT-TL.
54. 【2608.25505】Conditional Total Correlation and the Serial Depth of Adaptive Parallel Sampling
链接：https://arxiv.org/abs/2608.25505
作者：Chuling Wen,Weijie Liang,Jian Lu
类目：Information Theory (cs.IT); Computation and Language (cs.CL)
关键词：study adaptive parallel, adaptive parallel sampling, selects unrevealed coordinates, deterministic policy selects, policy selects unrevealed
备注：
点击查看摘要
Abstract:Motivated by parallel decoding in masked diffusion models, we study adaptive parallel sampling of discrete vectors: in each round, a deterministic policy selects unrevealed coordinates on the basis of the values observed so far, and the selected coordinates are sampled independently from their exact conditional marginals. Approximation error is measured by forward Kullback-Leibler divergence, and serial depth is the minimum target-averaged number of rounds meeting a prescribed error budget. Our central result is an exact identity: the divergence of every policy equals the expected conditional total correlation accumulated over its reveal rounds, so conditional total correlation is the exact information cost of within-round parallelism. The identity yields zero-error schedules for finite-order Markov chains with round complexity proportional to the Markov order and logarithmic in sequence length, a matching logarithmic characterization of the Bernoulli walk at every fixed error budget, and a linear-versus-logarithmic separation between left-to-right and hierarchical reveal orders. Uniform random permutations require linearly many expected rounds at every fixed budget; their hard-cap round-error tradeoff is an exact integer-composition problem whose fixed-round asymptotics and joint-scaling frontier we determine. Uniform balanced binary strings have depth of order squared logarithm, and binary one-hot blocks have square-root depth, with rectangular versions realizing every polynomial exponent up to one half. These results separate serial depth from entropy and negative log-likelihood, and establish conditional-dependence structure as a fundamental determinant of parallelizability. Experiments with a masked diffusion language model show that the pseudo-cost distinguishes deployed decoding rules and that its policy rankings agree closely with the quality of self-sampled outputs.
55. 【2608.25500】CaSKG: Counterfactual-Causal Skill Graphs for Scalable Agent Skill Retrieval
链接：https://arxiv.org/abs/2608.25500
作者：Zhiyuan Li,Linyuan Gao,Xuechun Ding,Hongwei Chen,Yuan Wu,Yi Chang
类目：Artificial Intelligence (cs.AI); Computation and Language (cs.CL)
关键词：Reusable skill libraries, turn memory access, challenging retrieval problem, reuse procedural knowledge, large language model
备注： 11 pages
点击查看摘要
Abstract:Reusable skill libraries allow large language model (LLM) agents to reuse procedural knowledge across tasks, but they also turn memory access into a challenging retrieval problem. Full-library prompting preserves coverage at high context cost, vector retrieval returns compact neighborhoods but treats skills as independent text, and graph-based retrieval can recover workflow context only when the edges that carry relevance are reliable. We propose CaSKG, a counterfactual-causal skill graph framework that calibrates procedural relations before retrieval. CaSKG first builds a high-recall directed candidate graph from semantic, lexical, input/output, and structural evidence, with repair evidence and an optional LLM judge further refining candidate scores. It then applies direction-conditioned textual counterfactual probes that remove, substitute, and reorder skill pairs, aggregates the evidence with Bayesian smoothing, and publishes a state-filtered weighted graph for task-conditioned expansion. The graph is constructed offline and used without changing the downstream agent policy or task interface. Across six LLM backbones on ALFWorld ID-140 and ScienceWorld U211, CaSKG achieves the highest task score in all twelve combinations of model and benchmark. Relative to Graph-of-Skills (GoS), it improves the six-model macro-average ScienceWorld score from 72.62 to 80.50 and ALFWorld success from 80.01\% to 86.79\%, while reducing mean environment steps on both benchmarks. Qualitative and ablation analyses further show that calibrated edges help retrieval preserve prerequisites, state-changing actions, verification routines, and final completion steps. These results position edge-confidence calibration as an effective route to compact and executable skill retrieval at scale\footnote{Code is available at: this https URL }.
56. 【2608.25489】A Storage-Retrieval Gap in Parametric Knowledge Graph Memory
链接：https://arxiv.org/abs/2608.25489
作者：Martino M. L. Pulici,Cuong Xuan Chu,Evgeny Kharlamov,Volker Tresp
类目：Machine Learning (cs.LG); Computation and Language (cs.CL); Information Retrieval (cs.IR)
关键词：exposing source data, recurring token cost, generation places retrieved, model context window, places retrieved subgraphs
备注： 12 pages, 2 figures, 7 tables, accepted at SKGi 2026
点击查看摘要
Abstract:Graph retrieval-augmented generation places retrieved subgraphs into the model's context window at query time, paying a recurring token cost and exposing source data on every call. We study an alternative: compiling a knowledge graph offline into a bank of LoRA adapters, one per entity, that serve as a parametric knowledge layer queried by injecting weights rather than text, at zero query-time context cost. On the MetaQA dataset, we find that subgraph-trained adapters encode context-free factual knowledge that generalizes to unseen questions: on single-valued relations the adapter gains $+0.243$ exact-match score over a base model that is nearly blind closed-book ($0.007$), and only the correct adapter recovers this knowledge (an oracle gap of $+0.283$ over the base model). However, the stored knowledge is not recoverable by similarity: given a query with no subgraph, embedding-based and weight-space geometry retrieval both perform at chance, because a semantically neighbouring entity's adapter does not contain the answer - knowledge is stored locally and does not transfer. Weight geometry correlates with subgraph semantics ($\rho = +0.329$) but not with functional retrievability. We quantify the byte and context-token costs against graph retrieval-augmented generation and discuss deployment implications. Our results establish that parametric knowledge graph memory is feasible for storing knowledge, and identify selecting and composing the right adapters by a mechanism other than semantic similarity as the central open problem - motivating a learned, query-conditioned composition mechanism.
57. 【2608.25487】ReliableRAG: Combating Misinformation in Retrieval-Augmented Generation via Reliability-Guided Reasoning Chains
链接：https://arxiv.org/abs/2608.25487
作者：Jinpu Jiang,Xuan Wu,Wenhao Song,Bo Yang,You Zhou,Hongwei Ge,Heow Pueh Lee,Yanchun Liang,Chunguo Wu
类目：Computation and Language (cs.CL); Information Retrieval (cs.IR)
关键词：Large Language Models, Language Models, Large Language, Retrieval-Augmented Generation, Question Answering
备注：
点击查看摘要
Abstract:Retrieval-Augmented Generation (RAG) has emerged as a powerful architecture for Question Answering (QA) by integrating external information into Large Language Models (LLMs). However, false, inaccurate, and misleading information in news and social media poses a serious challenge to real-world RAG systems, especially in multi-hop QA, where complex multi-step reasoning can be misled by even a single deceptive misinformation segment in the retrieved documents. Existing approaches mainly rely on implicit alignment or explicit regulation, but their limited ability to assess fine-grained information reliability makes them vulnerable to deceptive misinformation that is semantically relevant to the question yet factually incorrect, leading to erroneous answers. To address this limitation, we propose ReliableRAG, which, to the best of our knowledge, is the first reliability-driven framework that mitigates deceptive misinformation in multi-hop QA through fine-grained evaluation of individual triples. ReliableRAG first extracts information segments from source documents and represents them as structured triples. It then quantifies triple reliability by combining query-triple semantic relevance with triple credibility, retaining only the top-$K$ reliable and non-redundant triples. Based on these refined triples, ReliableRAG autoregressively constructs robust reasoning chains to consolidate trustworthy evidence and filter deceptive misinformation, producing accurate answers faithful to reliable information. Experiments on three multi-hop QA datasets show that ReliableRAG outperforms existing methods, substantially improving the factual reliability and robustness of RAG systems under deceptive misinformation injection.
58. 【2608.25486】PonsRAG: A Pons-Inspired RAG Bridging Cognitive Islands for Coordinated Long Narrative Reasoning
链接：https://arxiv.org/abs/2608.25486
作者：Rongchen Zhao,Yu Chen,Juyuan Wang,Zhouting Mo,Jianxing Yu,Wenqing Chen,Jingping Liu
类目：Artificial Intelligence (cs.AI); Computation and Language (cs.CL)
关键词：Long Narrative Reasoning, Long Narrative, essential capability, capability for processing, Long
备注： Accepted to EMNLP 2026 (Main Conference)
点击查看摘要
Abstract:Long Narrative Reasoning is an essential capability for processing and reasoning over complex narratives. While retrieval-augmented generation provides a promising framework, existing methods still face two critical challenges: cognitive islanding and cross-layer evidence disconnection. To address these issues, we propose PonsRAG, a coordinated RAG framework inspired by the biological pons. PonsRAG consists of two key components: Triple-Layer Indexing, which organizes documents into a connected knowledge structure to bridge cognitive islands, and Coordinated Reasoning, which retrieves evidence across distinct layers and integrates cross-layer information into a unified context. We evaluate PonsRAG on four long-context narrative benchmarks, and experimental results show that it outperforms the strongest baseline, achieving a 11.56% relative improvement in average accuracy on multi-choice tasks.
59. 【2608.25478】VietAIDetector: An Open-Source Zero-Shot Detector for Vietnamese AI-Generated Text
链接：https://arxiv.org/abs/2608.25478
作者：Trieu Hai Nguyen,Van-Dung Hoang
类目：Computation and Language (cs.CL)
关键词：recent years, remained a challenge, Vietnamese AI-generated text, detecting Vietnamese AI-generated, AI-generated text
备注： 17 pages, 5 figures
点击查看摘要
Abstract:In recent years, distinguishing between AI-generated text and human-written text has remained a challenge. In this paper, we introduce VietAIDetector, an open-source tool designed specifically for detecting Vietnamese AI-generated text. It allows users to interact through a Gradio web interface with inputs ranging from raw Vietnamese text to common text file formats, including scanned documents and exceptionally long texts that exceed the context size of the employed Large Language Models (LLMs). The core component of the tool employs a Zero-Shot approach to detect AI-generated text without requiring domain-specific training data, building upon the previous VietBinoculars and Binoculars research. The tool is built upon a Vietnamese-specific language model and has been evaluated on out-of-domain datasets, demonstrating superior performance compared to existing methods primarily developed for English. Additionally, users can select optimal detection thresholds based on F1 score, accuracy, or TPR@0.05FPR requirements. The results are presented through the web interface, allowing users to easily review and verify suspicious texts or download them as a PDF report. The tool is publicly available at this https URL
60. 【2608.25449】MathAdv: What Theorem Provers Know, Reason, Formalize, and Generalize
链接：https://arxiv.org/abs/2608.25449
作者：Jiaxin Yuan,Connor Martinez Lockhart,Xiaoyu Liu,Jiaqi Wang,Chenghao Deng,Xiayimei Han,Vlasios Mastrantonis,Dmitrii Gudin,Shaopeng Zhu,Abdirisak Abdullahi Mohamed,Bilal Hamdi Aytekin,Jiewen Lang,Zezheng Song,Furong Huang
类目：Computation and Language (cs.CL); Artificial Intelligence (cs.AI); Logic in Computer Science (cs.LO)
关键词：Formal theorem proving, proving enables machine-verifiable, provide limited evidence, emphasize aggregate proof, enables machine-verifiable evaluation
备注：
点击查看摘要
Abstract:Formal theorem proving enables machine-verifiable evaluation of mathematical reasoning, yet existing benchmarks often emphasize aggregate proof accuracy, concentrate on a narrow range of mathematics, and provide limited evidence of robustness to equivalent reformulations. We introduce MathAdv, a diagnostic benchmark spanning 13 domains across undergraduate- and graduate-level mathematics. Alongside Lean 4 theorem proving, MathAdv provides up to three auxiliary tasks: multiple-choice questions that probe mathematical knowledge, fill-in-the-blank problems that isolate informal reasoning, and expert-crafted transformations that test robustness to problem presentation. Our evaluation of contemporary theorem provers yields four findings: formalization remains a major bottleneck; performance varies substantially across mathematical domains; natural-language guidance helps general-purpose LLMs but can hinder proof-specialized models; and mathematically equivalent reformulations expose substantial robustness limitations. Together, these results show how component-wise evaluation can reveal model capabilities and failure modes that aggregate theorem-proving accuracy obscures. The dataset and evaluation scripts are available at this https URL.
61. 【2608.25428】DCGC: Draft-Conditioned Global Correction for Complex Reasoning with Masked Diffusion Models
链接：https://arxiv.org/abs/2608.25428
作者：Minhae Oh,Nakyung Lee,Jungwoo Lee
类目：Computation and Language (cs.CL); Artificial Intelligence (cs.AI)
关键词：Large Language Models, Large Language, propagate early mistakes, Language Models, Masked Diffusion Model
备注： 21 pages, 3 figures, 12 tables
点击查看摘要
Abstract:Correcting flawed reasoning traces remains a significant challenge for Large Language Models (LLMs), whose autoregressive generation can propagate early mistakes into subsequent reasoning. We introduce DCGC, a Masked Diffusion Model (MDM) framework for global correction that uses an imperfect solution draft from an upstream solver as auxiliary context. DCGC combines task-specific Supervised Fine-Tuning (SFT) with a novel inference-time mechanism called Dynamic Dual-CFG. This mechanism separates problem-only and joint problem-draft branches and scales the draft-conditioned residual using a relative confidence gap. Across math, code, and knowledge reasoning benchmarks, DCGC outperforms standard sampling and simpler CFG variants, with additional results suggesting transfer to different diffusion backbones. In test-time setting where ground-truth failure labels are unavailable, DCGC improves full test set accuracy by correcting low-consensus upstream outputs, highlighting its utility as a verifier-free global correction module for difficult reasoning instances.
62. 【2608.25398】OmniPhys: A Unified Multimodal Benchmark for Physics Understanding and Generation from Chinese Educational Corpora
链接：https://arxiv.org/abs/2608.25398
作者：Hao Chen,Yumin Lin,Nadila Yushanjiang,Xin Lin,Min Zhang
类目：Computation and Language (cs.CL)
关键词：Large Language Models, Multimodal Large Language, Language Models, Large Language, demonstrated strong abilities
备注： Accepted to Findings of EMNLP 2026
点击查看摘要
Abstract:Multimodal Large Language Models (MLLMs) have demonstrated strong abilities in solving diverse visual and textual reasoning tasks. However, their development in the physics domain is significantly hindered by the lack of a comprehensive benchmark. To fill this gap, we introduce OmniPhys, a large-scale benchmark for multimodal physics understanding and reasoning, covering middle school through university-level problems from Chinese Educational Corpora. OmniPhys consists of 15,246 questions and 19,850 images, accompanied by detailed annotations that support fine-grained analysis of reasoning processes and knowledge usage. Beyond conventional evaluation, OmniPhys is a benchmark that systematically evaluates multimodal outputs in the physics domain, including models' ability to generate structured physics diagrams, which constitute a fundamental component of authentic physics problem solving. Extensive evaluations reveal critical gaps in the capabilities of current MLLMs, especially in complex reasoning and visual generation. To address this, we release OmniPhys to serve as a foundational resource for advancing multimodal intelligence in physics and scientific domains. Codes and data are available at this https URL.
63. 【2608.25379】Adaptive Triggering for Bias Correction in LLM Reasoning
链接：https://arxiv.org/abs/2608.25379
作者：Nayoung Kim,Mickey Mancenido,Huan Liu
类目：Computation and Language (cs.CL); Artificial Intelligence (cs.AI)
关键词：LLM intermediate reasoning, amplify demographic stereotypes, prompting can expose, expose and amplify, amplify demographic
备注： 10 pages, 6 figures, Under review
点击查看摘要
Abstract:Chain-of-thought prompting can expose and amplify demographic stereotypes within an LLM's intermediate reasoning and create a failure mode that final-answer debiasing alone cannot address. Mitigating such bias during generation presents a fundamental timing problem: intervening too late allows biased reasoning to propagate, while unnecessarily intervening can disrupt otherwise correct reasoning. Existing approaches largely avoid this decision by either evaluating completed reasoning chains post hoc or intervening at predetermined steps, leaving open when a developing reasoning trajectory provides sufficient evidence to warrant correction. We formulate this decision as an online change-point detection problem. A per-step bias signal updates a CUSUM statistic and a targeted correction is injected only when accumulated evidence crosses a detector-specific threshold calibrated on held-out data. We instantiate the framework with a white-box signal derived from next-token probabilities and a black-box signal obtained from an LLM judge, enabling deployment with both open-weight and hosted models. On gpt-4o-mini adaptive black-box triggering recovers most of the disambiguated-context accuracy lost under fixed-interval intervention while requiring substantially fewer interventions. That result holds even with an independent judge. Across six open-weight models, the white-box signal improves ambiguous-item accuracy on all six but reduces disambiguated-item accuracy on five because it cannot distinguish unsupported stereotype reliance from correct, stereotype-congruent evidence.
64. 【2608.25375】GGSS: Geodesic-Gated Spherical Steering for Inference-Time Debiasing of Generative Vision-Language Models
链接：https://arxiv.org/abs/2608.25375
作者：Yiqun Sun,Junyu Chen,Pengfei Wei,Lawrence B. Hsieh
类目：Computers and Society (cs.CY); Computation and Language (cs.CL); Computer Vision and Pattern Recognition (cs.CV)
关键词：produce demographically biased, demographically biased outputs, human-centered settings, race or gender, Generative vision-language models
备注： Accepted to EMNLP 2026
点击查看摘要
Abstract:Generative vision-language models (VLMs) are increasingly used in human-centered settings, yet they can produce demographically biased outputs even when images differ only in controlled attributes such as perceived race or gender. However, existing inference-time debiasers were largely designed for static embeddings or CLIP-like models rather than generative VLMs. We propose GGSS---Geodesic-Gated Spherical Steering---a norm-preserving intervention that discovers a counterfactual bias subspace on the unit hypersphere, steers visual tokens along geodesic arcs, and uses an adaptive gate to focus correction on tokens that carry stronger demographic signal. We evaluate four generative VLMs against ten adapted inference-time debiasing baselines and prompt-based mitigation under a single operating-point protocol across categorical, pairwise, and occupation-gender bias tests, while also measuring general visual-language capability. GGSS achieves the lowest average bias on all four models, significant on three of four backbones under paired permutation tests, while preserving MMStar accuracy within +/- 0.6 p.p. of the unsteered baseline. Code is available at this https URL.
65. 【2608.25359】Leveraging Speech Acts for Low-Data and Cross-Domain Conversation Derailment Forecasting
链接：https://arxiv.org/abs/2608.25359
作者：Angela Yifei Yuan,Christine De Kock,Christopher Leckie
类目：Computation and Language (cs.CL)
关键词：Conversational derailment forecasting, enabling proactive moderation, derailment forecasting aims, Conversational derailment, escalate into hostility
备注：
点击查看摘要
Abstract:Conversational derailment forecasting aims to predict when online discussions will escalate into hostility, enabling proactive moderation. Existing approaches often struggle in low-data settings and to generalize across domains. This poses a challenge for new platforms and smaller communities where annotated data is limited. We propose modeling pragmatic representations of conversations to reduce lexical noise and improve generalizability. Specifically, speech act information is used as an auxiliary learning signal alongside textual semantics. Experimental results show improved performance across three datasets, particularly in low-data and cross-domain settings.
66. 【2608.25354】Escaping Low-Dimensional Overlap: Multi-Task Model Merging via High-Dimensional Sparse Disentanglement
链接：https://arxiv.org/abs/2608.25354
作者：Yihang Zhang,Shengke Sun,Junjie Wen,Feng Zeng
类目：Machine Learning (cs.LG); Artificial Intelligence (cs.AI); Computation and Language (cs.CL)
关键词：construct multi-task generalist, multi-task generalist models, severe task interference, additional training, construct multi-task
备注：
点击查看摘要
Abstract:Model merging provides an efficient way to construct multi-task generalist models without additional training, but its performance often degrades under severe task interference. Task interference in model merging primarily stems from \textit{superposition}, where task-specific features become entangled within the parameter space. This entanglement renders conventional decomposition methods insufficient for effectively isolating useful task directions from interfering components. In this paper, we propose a sparse-representation-based merging framework that uses Sparse Autoencoders (SAEs) to project task vectors into a high-dimensional sparse feature space, enabling feature-level disentanglement before fusion. To reduce computational overhead, we further introduce a lightweight Group-Ranked Zeroth-Order Optimizer (GR-ZOO) to identify task-critical layers for selective merging. Experiments on both Qwen2.5-1.5B and Qwen2.5-7B demonstrate that our method consistently outperforms representative baselines, including Task Arithmetic, TIES-Merge, DARE, Fisher-Merge,and several recent training-free merging methods, across mathematical reasoning, code generation, instruction following, and general knowledge tasks. In a highly conflicting four-task setting on Qwen2.5-1.5B, our method further achieves a 2.78\% improvement over the strongest baseline.
67. 【2608.25347】Short Horizons and Sparse Concepts: a Mathematical View of the Readout in the J-lens
链接：https://arxiv.org/abs/2608.25347
作者：Shi-Qi Yan,Kai-Xuan Ding,Chao-Hong Tan,Qian Chen,Wen Wang,Xiangang Li,Zhen-Hua Ling
类目：Computation and Language (cs.CL)
关键词：read verbalizable representations, language models, verbalizable representations, representations from language, J-lens
备注：
点击查看摘要
Abstract:The Jacobian lens (J-lens) has been proposed as a way to read verbalizable representations from language models. However, its principle and meaning lack a detailed and theoretical discussion. We provide a mathematical view of this interpretation and of its assumed causal structure. Besides treating the J-lens as a heuristic probe, we further regard it as a first-order causal transfer operator from intermediate activations to expected future readouts. We study the Jacobian matrix as the optimal local linear approximation of the downstream mapping, analyze its global approximation behavior and bias, and identify its mathematical meaning as an expectation over anticipated future readouts. Further analysis of the Jacobian energy distribution reveals that its causal geometry is highly sparse. The energy decays with depth, concentrates in an extremely small proportion, and decomposes into diagonal pathways and specific critical positions. This decomposition further resolves the expectation of the J-lens over future outputs into short-horizon and sparse concept predictions, providing a more intuitive attribution and explanation for the ability of the J-lens to visualize concepts during the thinking process. Based on the theory, we propose a simple but effective improvement strategy and decoupling method for the J-lens, which significantly enhances the ability of the J-lens to read out correct intermediate concepts.
68. 【2608.25343】GUIDE: Generative Unsupervised Chinese Query Correction via Phonetic and Visual Shared-ID Encoding
链接：https://arxiv.org/abs/2608.25343
作者：Lei Yang,Binbin Huang,Jiwei Tan,Xuhui Sui,Chang Tu,Yi Wang,Han Li
类目：Computation and Language (cs.CL)
关键词：supervised methods rely, large annotated correction, annotated correction pairs, Chinese query correction, content platforms
备注： Accepted to EMNLP 2026 Industry Track; 7 pages, 3 figures, 8 tables
点击查看摘要
Abstract:Chinese query correction (CQC) is important for search and query recommendation on content platforms, but supervised methods rely on large annotated correction pairs that are costly to maintain as query vocabularies evolve. Unsupervised correction with language models is attractive, yet in the short-query setting, unconstrained generation often over-corrects ambiguous inputs toward high-frequency phrases, causing intent drift. We propose \textsc{GUIDE}, a generative unsupervised framework for CQC based on a confuse-then-clarify paradigm. \textsc{GUIDE} encodes phonetically or visually confusable characters with shared-IDs and reconstructs the original query with an encoder--decoder architecture, which constrains correction to plausible confusion neighborhoods while learning from unlabeled query streams. A time-decayed, query-frequency-weighted objective further supports adaptation to rapidly changing query vocabularies. Experiments on \textit{QSpell 250K} and a large-scale real-world dataset (\textit{KwaiSearch}) show that \textsc{GUIDE} consistently outperforms strong baselines, while online A/B testing further confirms gains in correction quality and downstream engagement.
69. 【2608.25336】Provenance Before Prose: Claim-Locked Reporting
链接：https://arxiv.org/abs/2608.25336
作者：Xiao Fan,Jingyuan Li,Hongbin Guo,Yubo Han,Yi Zhang
类目：Computation and Language (cs.CL)
关键词：Large language models, restate thresholded contrasts, invert effect directions, fluently verbalize statistical, invert effect
备注：
点击查看摘要
Abstract:Large language models (LLMs) can fluently verbalize statistical evidence, yet statistical reports can still drift numerical values, invert effect directions, or restate thresholded contrasts as categorical effects. We frame these failures as a control problem: the evidence-bearing content of a scientific report should be fixed by structured statistical results rather than sampled during prose generation. We therefore use cross-run reproducibility to stress-test whether report-visible numbers and claims are bound before prose generation. Existing controls operate at the text or slot level; a deterministic hybrid template reproduces only 61.1% of report-visible numerical content across seeds because the LLM still selects which findings and numbers the template renders. We propose claim-locked reporting, a provenance-before-prose protocol that fixes the evidence source, numbers, direction, and allowed language strength of each reportable claim before the LLM writes only connective prose. Across fMRI functional-connectivity reporting and randomized controlled trial reporting on Evidence Inference 2.0, claim-locked reporting improves reproducibility over the hybrid template by 37.4 and 20.5 points, respectively. Blinded human audits support the observed direction-preservation and governance trends. In an fMRI cost analysis with DeepSeek, claim-locked reporting also yields the lowest observed token use and median generation latency.
70. 【2608.25329】Learning What to Share and What to Personalize: Hierarchical Strategy Co-Evolution for Agent Memory
链接：https://arxiv.org/abs/2608.25329
作者：Yupeng Han,Shuochen Liu,Kai Zhang,Ze Liu,Zhihong Pan,Xianquan Wang
类目：Artificial Intelligence (cs.AI); Computation and Language (cs.CL)
关键词：entire dialogue history, agents maintain compact, maintain compact user, textbf, compact user profiles
备注： EMNLP'2026 Main Conference
点击查看摘要
Abstract:Memory-augmented agents maintain compact user profiles throughout extended conversations, enabling personalized and consistent responses without the need to process the entire dialogue history. The quality of these user profiles relies on the underlying memory management strategy: at each step, the agent must determine what to retain, compress, or discard. However, existing methods typically employ a static, one-size-fits-all strategy established before training. In practice, the optimal memory decision is inherently user-specific and dynamically evolves alongside policy optimization. To address this, we propose \textbf{HiPS} (\textbf{Hi}erarchical \textbf{P}ersonalized \textbf{S}trategy), a framework that decouples memory management into a globally shared foundation and a user-specific adaptive tier. Specifically, HiPS employs \textbf{Universal Strategy} to extract shared principles from cross-persona trajectories, alongside \textbf{Persona Delta Distillation} to generate tailored rules for users whose behaviors diverge from general patterns. \textbf{Cross-Level Rule Flow} dynamically calibrates their boundary by promoting broadly validated personal rules and demoting contradicted global ones. The architecture establishes a co-evolution loop where a mechanism guarantees that all strategy refinements are anchored to task outcomes. Extensive experiments demonstrate consistent improvements over memory-augmented baselines.
71. 【2608.25325】FinRiskAtlas: Decision-Aligned Evaluation of Large Language Models for Financial Risk Review
链接：https://arxiv.org/abs/2608.25325
作者：Suyang Zhong,Jingzhe Zhu,Qi Xu,Liyao Sun,Yin Wang,Qingqing Sun,Shuai Chen,Tianyi Zhang
类目：Artificial Intelligence (cs.AI); Computation and Language (cs.CL)
关键词：Deploying large language, Deploying large, general financial competence, measuring general financial, large language models
备注：
点击查看摘要
Abstract:Deploying large language models for professional financial review requires more than measuring general financial competence: models must perform the specific review operation required by a workflow and determine whether available evidence is sufficient for a defensible decision. Existing financial benchmarks cover knowledge, reasoning, compliance, and professional tasks, but their evaluation units are often organized around datasets or task formulations rather than the decisions that deployed systems support. We introduce FinRiskAtlas, a Chinese-language benchmark that evaluates financial LLMs along two complementary dimensions: operation execution under fixed evidence states and evidence-state control under evolving review conditions. The static benchmark contains 9,742 instances across 53 task families, including 42 Domain Knowledge families and eleven downstream review operations defined by explicit evaluation contracts. FinRisk-Ask extends this framework through offline replay of 680 pre-action states from 104 de-identified professional trajectories, withholding future evidence during inference and using it only to construct expert-verified evidence targets. Across 33 model configurations, operation-level evaluation yields non-redundant rankings (mean pairwise Spearman correlation 0.42 across downstream operations), and knowledge-based shortlisting can incur up to 18.01 points of regret on individual operations. FinRisk-Ask further shows that entering the Ask branch more frequently does not necessarily improve request targeting or end-to-end evidence acquisition. These results show that broad financial capability scores do not fully capture where models are reliable in professional workflows, motivating evaluation units aligned with the decisions and evidence states that deployed systems must support.
72. 【2608.25277】Routed Graph Handoff: Adaptive Format Selection for Multi-Agent LLM Delegation
链接：https://arxiv.org/abs/2608.25277
作者：Pratyay Banerjee,Ankit Chadha
类目：Computation and Language (cs.CL); Artificial Intelligence (cs.AI)
关键词：Multi-agent LLM systems, Multi-agent LLM, LLM systems coordinate, messages that consume, coordinate through natural-language
备注： Accepted in EMNLP 2026
点击查看摘要
Abstract:Multi-agent LLM systems coordinate through natural-language messages that consume 40--60\% of their token budget. Replacing these with structured graphs reduces cost but fails on tasks requiring adaptive reasoning. We propose \textbf{Routed Graph Handoff}, where a lightweight LLM router (155 tokens, 0.15\% overhead) selects between a typed dependency graph and natural language for each delegation. On four benchmarks (1,050+ trajectories), the routed system matches or exceeds NL-only on every task: \textbf{+12.7\,pp} on $\tau$-retail at 3.2$\times$ compression ($p{}0.01$), \textbf{+8.7\,pp} on BrowseComp at 2.2$\times$ compression ($p{}0.05$), and parity on BFCL and AppWorld. Without the router, graph-only delegation regresses 14.6\,pp on AppWorld; the router eliminates this at near-zero cost. A graph-aware executor prompt is required: the same schema without interpretation guidance yields no gain. An oracle analysis reveals 8.6\,pp of additional headroom, motivating execution-time adaptive routing as future work.
73. 【2608.25276】Groundhog Bit-Flip Attack: Seeding Infinite Generation Loops in Mixture-of-Experts LLMs through Bit Flips
链接：https://arxiv.org/abs/2608.25276
作者：Huakang Lin,Tiancheng Zheng,Mingxuan Sun,Tianhong Xu,Fan Zhang,Yunsi Fei,Ruyi Ding
类目：Computation and Language (cs.CL)
关键词：efficient large language, large language models, activating expert sub-networks, selectively activating expert, architectures enable scalable
备注： 9 pages, 3 figures; Accepted at EMNLP 2026
点击查看摘要
Abstract:Mixture-of-Experts (MoE) architectures enable scalable and efficient large language models (LLMs) by selectively activating expert sub-networks through a routing mechanism. However, this adaptive design introduces a new attack surface: specific experts become disproportionately correlated with certain tokens (e.g., end-of-sequence), allowing adversaries to manipulate model behavior via lightweight perturbations. In this work, we present \textbf{Groundhog Bit-Flip Attack (GBFA)}, the first bit-flip-based \textit{ Denial-of-Wallet availability attack} against MoE-based LLMs. By identifying and flipping routing-layer bits associated with related expert activations, we demonstrate that GBFA substantially extends the decoding token usage across three different LLM modes: conversational, reasoning, and agentic tasks, while largely preserving semantic fidelity. Across four main real-world MoE-based LLMs, manually deactivating on average fewer than \textbf{4 experts} drives average output inflation to $\mathbf{5912\%}$, with the majority of test samples reaching max tokens. These results reveal a robustness vulnerability of MoE architectures to bit flip, and highlight the potential of GBFA as an availability attack against LLMs.
74. 【2608.25245】he "Curse of Knowledge" in LLM Query Simulation: Concept Provenance for Tracing Answer-Side Intrusion
链接：https://arxiv.org/abs/2608.25245
作者：Chenglong Ma,Xinye Wanyan,Danula Hettiachchi,Ziqi Xu,Jeffrey Chan
类目：Information Retrieval (cs.IR); Computation and Language (cs.CL)
关键词：presuppose answer-side document, LLM-generated search queries, LLM-generated search, violating the information-access, pre-search users
备注： 12 pages, 4 figures, and 2 tables. To appear in the Proceedings of the 35th ACM International Conference on Information and Knowledge Management (CIKM '26)
点击查看摘要
Abstract:LLM-generated search queries are widely used to augment IR evaluation, yet they may contain concepts that presuppose answer-side document knowledge, violating the information-access boundary of pre-search users. Existing validation metrics, including overlap, diversity, and effectiveness, cannot distinguish rare human-tail variation from candidate answer-side intrusion. We introduce concept provenance, a framework that assigns query concepts to backstory-supported, human-central, human-tail, and candidate answer-side zones, operationalizing a boundary that retrieval metrics alone cannot detect. Applying concept provenance to 77,004 queries across 100 UQV100 topics, 8 LLMs, and 5 prompt conditions with two extraction pipelines, we obtain a cross-pipeline token-HCIR Spearman rho of 1.0 over five condition means. Candidate answer-side concepts constitute 7.40 percent of non-generic concepts and appear in 97 of 100 topics, with topic explaining approximately 67 percent of variance. Human validation yields 68.2 percent relaxed precision, revealing two mechanisms: knowledge intrusion at 45.5 percent and deployment intrusion at 45.0 percent. Diagnostic probes show disproportionate localized retrieval effects, with deletion effect size d = -0.47 compared with d = -0.34 for random deletion, but these concepts explain less than 2 percent of aggregate evaluation variance. Concept provenance therefore serves as a boundary-compliance diagnostic rather than an evaluation-shift predictor. Under the tested conditions, no prompt condition eliminates intrusion; post-generation concept-provenance selection achieves 99 percent elimination.
75. 【2608.25243】From Memorization to Absorption: Mixed-Policy RL for Continual Knowledge Injection
链接：https://arxiv.org/abs/2608.25243
作者：Zhibo Hou,Fan Zhao,Zhiyu An,Wan Du
类目：Computation and Language (cs.CL); Machine Learning (cs.LG)
关键词：large language models, keeping large language, Continual knowledge injection, language models, fast-evolving world
备注：
点击查看摘要
Abstract:Continual knowledge injection is essential for keeping large language models up-to-date in a fast-evolving world. Existing methods rely on supervised fine-tuning (SFT), which memorizes injected facts in their training format but fails to generalize across paraphrasing, document combinations, and reasoning. To address this, we propose Golden-GRPO Injection (GRIN), a three-stage self-learning framework for continual knowledge injection. Golden-GRPO is a mixed-policy reinforcement learning algorithm designed specifically for knowledge injection, which injects a golden answer to provide learning signal even when on-policy rollouts fail on novel facts. We further introduce Blank and Counter, two document-level benchmarks targeting novel acquisition and counterfactual overwrite respectively, each evaluating single-fact recall, multi-source retrieval, and inferential reasoning. Our experiments establish a clear empirical claim: mixed-policy reinforcement learning enables knowledge absorption beyond what supervised fine-tuning can achieve. GRIN substantially outperforms SFT and mixed-policy RL baselines on the harder question types while matching them on basic fact recall.
76. 【2608.25236】Rare Diseases, Common Dilemmas: LLMs Prioritize Equal Resource Distribution over Patient Benefit in Decision-Making
链接：https://arxiv.org/abs/2608.25236
作者：Minda Zhao,Xu Han,Rishabh Goel,Maya Dagan,Noa Dagan,Adithya Madduri,Payal Chandak,Shilpa Nadimpalli Kobren,Isaac S. Kohane
类目：Computers and Society (cs.CY); Artificial Intelligence (cs.AI); Computation and Language (cs.CL)
关键词：involves prioritizing ethical, involves prioritizing, Abstract, non-maleficence, value-laden clinical judgments
备注：
点击查看摘要
Abstract:Clinical decision-making often involves prioritizing ethical values, such as beneficence, non-maleficence, respecting a patient's autonomy, and justice. Recent work has begun to assess how large language models (LLMs) make such subjective, value-laden clinical judgments. However, evaluations of LLM decision-making in rare disease care contexts, where ethical tensions are ubiquitous and where scarce prior information likely impacts LLM behavior, are still lacking. Here, we present a benchmark of 208 clinically grounded rare disease vignettes, each of which presents genuine, high-stakes conflicts. When prompting 11 state-of-the-art LLMs to choose between clinically defensible yet ethically conflicting next steps embedded within these vignettes, we found that all evaluated models consistently prioritized justice over other core bioethical principles. Specifically, models overwhelmingly favor equal resource allocation over need-based considerations, indicating LLMs' limited responsiveness to differences in clinical severity or situational context. We also identify a strong authority-framing effect: models favor justice in committee-based contexts and shift toward beneficence and autonomy only when final decisions are framed as being made by clinicians or patients respectively. Our work suggests that institutional pressures surrounding rare disease resource utilization may be silently reflected in LLM-based decision support systems, with finer ethical considerations disregarded.
77. 【2608.25231】Output Dilution: Redundant but Fragile Representations in MoE Models
链接：https://arxiv.org/abs/2608.25231
作者：Orion Reblitz-Richardson
类目：Machine Learning (cs.LG); Artificial Intelligence (cs.AI); Computation and Language (cs.CL)
关键词：encode moral content, content as robustly, encode moral, moral content, cs.LG
备注： 18 pages, 4 figures. Code and outputs at [this https URL](https://github.com/deepsteer/deepsteer)
点击查看摘要
Abstract:Mixture-of-Experts (MoE) models appear to encode moral content as robustly as dense models, yet prove far more fragile in their encoding. In OLMoE-1B-7B, linear probes recover moral valence from nearly every expert-layer combination, with mean peak-layer accuracy above 90%. But these representations collapse under levels of activation noise that a dense model of matched size easily tolerates, with a 4.2-fold difference in robustness. We trace this to output dilution. Because the MoE block averages across active experts before contributing to the residual stream, the feedforward signal reaching downstream layers is nearly two orders of magnitude smaller than in a dense MLP. Moral information, our interest, survives aggregation intact but at a scale trivially overwhelmed by perturbation. Routing itself remains stable under noise while the vulnerability originates entirely in the diluted aggregate. Checkpoint trajectories confirm this is architectural, not learned. Experts never specialize and accuracy saturates within the first few thousand steps. In sparse architectures, redundant encoding does not imply robust encoding.
Comments:
18 pages, 4 figures. Code and outputs at this https URL
Subjects:
Machine Learning (cs.LG); Artificial Intelligence (cs.AI); Computation and Language (cs.CL)
ACMclasses:
I.2.6; I.2.7
Cite as:
arXiv:2608.25231 [cs.LG]
(or
arXiv:2608.25231v1 [cs.LG] for this version)
https://doi.org/10.48550/arXiv.2608.25231
Focus to learn more
arXiv-issued DOI via DataCite (pending registration)</p>
78. 【2608.25230】rust the Mass: Forced Weights in KV-Cache Eviction
链接：https://arxiv.org/abs/2608.25230
作者：Jack Shi,Jerry Gu
类目：Machine Learning (cs.LG); Computation and Language (cs.CL)
关键词：discards the rest, deployed sparse-attention, attention weights, subset closes, subset
备注：
点击查看摘要
Abstract:Every deployed sparse-attention or KV-cache-eviction rule keeps a subset of the keys, discards the rest, and renormalizes the attention weights over the kept set. Enumerating the exact best subset under that constraint on $168{,}192$ attention rows from five models shows that keeping the largest weights is already near-optimal, since the best subset closes only a median $2$ to $5\%$ of the remaining gap to full attention. If selection closes this little, published margins between eviction methods must come from elsewhere, so we measure the bytes each method holds. In the shared evaluation pipeline, the strongest query-agnostic methods hold the full cache because their per-head selections are stored as masks, and only ragged per-head storage frees that memory. Enforcing a nominal budget on one fixed selection costs $14$ to $62$ benchmark points. We trace an $87.6$-point retrieval margin to rankings computed while the question is visible. ContourKV, a training-free allocator built from the dropped-mass statistic, wins $93$ of $160$ paired comparisons against that state of the art and loses $22$ at the byte count of the budget-enforcing baselines, and it ties the strongest of them.
79. 【2608.25204】LibriBrain100: One Hundred Hours of Broad and Deep MEG Data for Neural Speech Decoding at Scale
链接：https://arxiv.org/abs/2608.25204
作者：Francesco Mantegna,Dulhan Jayalath,Gereon Elvers,Tasha Kim,Benjamin Ballyk,Alex Fung,SungJun Cho,Teyun Kwon,Luisa Kurth,Miran Özdogan,Gilad Landau,Pratik Somaiya,Natalie Voets,Mark Woolrich,Oiwi Parker Jones
类目：Machine Learning (cs.LG); Computation and Language (cs.CL)
关键词：large-scale MEG dataset, large-scale MEG, speech decoding designed, high-quality MEG acquired, MEG dataset
备注： 10 content pages + 22 pages for references and appendices (in total: 32 pages, 16 figures, 6 tables)
点击查看摘要
Abstract:We introduce LibriBrain100, a large-scale MEG dataset for speech decoding designed from the ground up for reproducible, standardised evaluation. LibriBrain100 more than doubles the size of the original LibriBrain release, resulting in over 100 hours of high-quality MEG acquired while subjects listened to naturalistic continuous speech. With $\sim$80 hours from a single subject, LibriBrain100 sets a new record for deep, within-subject neural data (8$\times$ more than the next comparable dataset and roughly 80$\times$ more than other datasets). To demonstrate the payoff of this depth-first design, we evaluate on a word-classification benchmark---an increasingly well-established stepping stone towards the open challenge of noninvasive brain-to-text decoding. Using an existing decoding model, we achieve state-of-the-art performance---validating both the quality of the recordings and the value of within-subject data at scale. Because collecting 80 hours of data per user is impractical for real-world applications, we also collected $\sim$40 minutes of additional data from each of 32 subjects. Using the same word-classification benchmark, we demonstrate the value of broad multi-subject data: supervised finetuning of a pre-trained model can substantially compensate for limited per-subject data. We provide standard train, validation, and test splits, all reproducible through an open-sourced Python library that supports easy downloading, optional preprocessing, and data loading for common deep learning frameworks. In addition, the dataset and evaluation infrastructure are being released alongside an open machine-learning competition with a public leaderboard for standardised benchmarking. Ultimately, our hope is that LibriBrain100 will accelerate progress towards practical non-invasive brain-computer interfaces, capable of restoring communication to people living with severe paralysis.
80. 【2608.25200】Learning Mixtures of Plackett-Luce Models for Multi-Objective Alignment
链接：https://arxiv.org/abs/2608.25200
作者：Dongyue Li,Ziniu Zhang,Lu Wang,Hongyang R. Zhang
类目：Machine Learning (cs.LG); Artificial Intelligence (cs.AI); Computation and Language (cs.CL)
关键词：represent heterogeneous underlying, models, responses from annotators, heterogeneous underlying preferences, underlying preferences
备注： 19 pages; To appear in EMNLP 2026
点击查看摘要
Abstract:We consider the problem of learning a mixture of $k$ Plackett-Luce models given multi-way ranking responses from annotators that may represent heterogeneous underlying preferences. This problem has many applications in AI alignment and preference optimization. Prior work has studied mixtures of Bradley-Terry models from pairwise comparisons. However, uncovering mixture models is theoretically unidentifiable when $k$ exceeds $m/2$, where $m$ is the length of a ranking. We propose an efficient implementation to address this limitation, which involves first augmenting the rankings to a larger size by generating new responses from a base language model, followed by a gradient-based estimation to reduce inference cost in the input embedding space. Based on this procedure, we then design an expectation-maximization algorithm with these two steps to fit a mixture of Plackett-Luce models, called MoPLEx. Extensive experiments are conducted to verify this approach. First, we show that the gradient-based approximation estimates true probabilities with less than 5% error on models with up to 34 billion parameters. Second, we show that MoPLEx improves clustering and ranking accuracy by an average of 43.7% and 15.2% over baselines using single ranking and mixtures of Bradley-Terry models, on preference optimization datasets. These results demonstrate the effectiveness of MoPLEx for tackling multi-way rankings from heterogeneous preferences through measuring alignment between gradients.
81. 【2608.25190】BanglaMamba: Exploring State Space Models for Bangla Fake News Detection
链接：https://arxiv.org/abs/2608.25190
作者：M. K. Khalidi Siam
类目：Computation and Language (cs.CL); Machine Learning (cs.LG)
关键词：important Natural Language, Natural Language Processing, Natural Language, important Natural, Language Processing
备注：
点击查看摘要
Abstract:Fake news detection has become an important Natural Language Processing (NLP) task due to the rapid spread of misinformation through online news platforms and social media. While transformer-based models such as BanglaBERT achieve strong performance for Bangla text classification, their quadratic computational complexity makes them less suitable for long-document processing in resource-constrained environments. This paper investigates Mamba-based State Space Models (SSMs) as an efficient alternative for Bangla fake news detection. We propose BanglaMamba and compare it with pre-trained BanglaBERT and a similarly configured BERT model trained from scratch. Experimental results show that BanglaBERT achieves the highest Macro-F1 score (0.9260), while BanglaMamba (0.9029) achieves performance comparable to the from-scratch CustomBERT (0.9057) despite using a different architecture. Meanwhile, BanglaMamba achieves approximately $2.2\times$ higher inference throughput and 49% lower inference peak GPU memory usage than the BERT-based models. Cross-dataset evaluation shows that BanglaBERT generalizes better to an external dataset, highlighting the importance of large-scale pretraining. These findings demonstrate that Mamba-based SSMs can provide a competitive and computationally efficient alternative to Transformer-based architectures for Bangla fake news detection, particularly in resource-constrained settings.
82. 【2608.25166】he Changing Geometry of Grammar: Dimensionality and Neighborhood Reorganization across Transformer Layers
链接：https://arxiv.org/abs/2608.25166
作者：Samuele Vallisa,Federico Ravenda,Claudio Palominos,Rui He,Andrea Raballo,Antonietta Mira,Philipp Homan,Wolfram Hinzen
类目：Computation and Language (cs.CL)
关键词：Transformer representations describe, high-dimensional vector spaces, representations describe trajectories, incorporate relational context, tokens incorporate relational
备注：
点击查看摘要
Abstract:Transformer representations describe trajectories through high-dimensional vector spaces, which are shaped dynamically as tokens incorporate relational context across layers. Such data tend to concentrate on lower-dimensional sub-manifolds, a form of compression quantified by the Intrinsic Dimensionality (ID), the minimum number of independent variables needed to represent them without significant information loss. In this work, we ask whether the grammatical role of tokens, as marked by their part-of-speech (PoS) tag, shapes the local geometry of this manifold. To this end: (1) We investigate the layer-wise evolution of ID, finding that closed-class items expand earlier and collapse sooner than open-class ones; (2) We show its expansion and contraction to be explained by changes in the neighborhood structure, and hence in the relations between words within a sentence; (3) We compare encoders (ModernBERT, bigbird-roberta-large) and decoders (gemma-2-2B, Llama-3.2-3B), finding that the two families evolve differently across layers, consistently with how each integrates context;(4) We show that geometric features alone recover a token's grammatical role, and use them to interpret how the semantic content of each PoS evolves across layers in a downstream classification task.
83. 【2608.25152】Belief Cascades Drive Persuasion in LLM Agent Networks
链接：https://arxiv.org/abs/2608.25152
作者：Haoyi Qiu,Genglin Liu,Pranav Narayanan Venkit,Kung-Hsiang Huang,Saadia Gabriel,Chien-Sheng Wu,Nanyun Peng
类目：Computation and Language (cs.CL); Artificial Intelligence (cs.AI)
关键词：increasingly debate answers, mediate information flows, systems increasingly debate, LLM systems increasingly, Multi-agent LLM systems
备注：
点击查看摘要
Abstract:Multi-agent LLM systems increasingly debate answers, coordinate research, simulate users, and mediate information flows, making agent-to-agent persuasion a basic but undermeasured capability. We introduce a controlled testbed for studying how goal-directed persuaders shift elicited stances in networks of LLM agents grounded in real-world ego-network topologies. Across four LLM backbones, five graphs, and 55 policy statements, we find that persuasion dynamics depend on the interaction between topology, competition, topic, and model prior. Additionally, we show that direct exposure reliably predicts next-round stance change in competing runs, and peer relays carry smaller but measurable influence, showing that agents not assigned to persuade can still transmit persuasive force. Finally, analyzing post text alone misses important movement: planned strategies are only partly realized in executed messages, action choices can diverge from message content, and persuadees rarely state the stance shifts detected by probes. These results argue for evaluating multi-agent persuasion as a trajectory- and exposure-level process, using belief probes, exposure provenance, and action logs to identify who influenced whom and whether visible language reflects underlying stance movement.
84. 【2608.25140】RefLAM: A Reference-Grounded Line Annotation Pipeline for Historical Arabic Manuscripts
链接：https://arxiv.org/abs/2608.25140
作者：Mohamed Guechaoui,Mohamed Diaa Zellagui,Souleyman Chaib,Sahraoui Dhelim
类目：Computer Vision and Pattern Recognition (cs.CV); Computation and Language (cs.CL)
关键词：Existing approaches, building line-level Arabic, provable correctness guarantee, extended to multi-script, Reference-grounded Line Annotation
备注： 11 pages, 6 figures, 3 tables, 2 algorithms
点击查看摘要
Abstract:Existing approaches to building line-level Arabic handwritten-text-recognition (HTR) training data either rely on fully manual annotation, which does not scale, or on automatic OCR-to-reference alignment methods not yet extended to multi-script, two-zone (main-plus-margin) manuscript layouts with a provable correctness guarantee. We present RefLAM (Reference-grounded Line Annotation for Manuscripts), a pipeline converting manuscript page images and clean transcriptions into validated, line-level ground truth without sacrificing human oversight. RefLAM couples a deep-learning page-segmentation model with a multimodal large language model (MLLM) for structured OCR and a diacritic-agnostic fuzzy alignment engine that grounds each OCR line in a contiguous span of the reference text, with a character-level confidence score in $[0,100]$. A perfect score is provably equivalent to character-for-character identity of the normalised strings (the Confidence-100 rule), verified with no counterexample across the released corpus. A reviewer can thus trust a perfect score, confirming most lines at a glance rather than retyping them, so annotation becomes triaged, with attention concentrated on uncertain alignments. Across 7 fully page-validated books we measured a 75$\times$ throughput gain over manual annotation (3,000 vs. 40 lines/hr); applying the same guarantee to 7 further books, we retained 16,533 confidence-100 main-text lines within one week, excluding sub-100 lines rather than manually correcting them. Using RefLAM, we release AraMS-28k: 14 historical Arabic manuscript books, 3,043 pages, and 27,971 main-text and 629 margin-line annotations with bounding boxes, layout labels, and insertion anchors for 191 margin entries (30.4%). We also finetune Muharaf-pretrained baselines (including HATFormer) on AraMS-28k and report CER results confirming its practical utility for downstream HTR training.
85. 【2608.25123】SelfGraphRAG: Bridging the Supervision Gap in Graph-Based RAG with Synthetic QA Generation
链接：https://arxiv.org/abs/2608.25123
作者：Ben Lagnese,Manas Gaur
类目：Computation and Language (cs.CL); Artificial Intelligence (cs.AI)
关键词：large language models, Retrieval-augmented generation, incorporating external knowledge, improves large language, large language
备注： 16 pages, 2 figures. Based on M.S. thesis work. Thesis available at [this https URL](https://www.proquest.com/docview/3350071346) . Under review. Code available upon request from manas@umbc.edu
点击查看摘要
Abstract:Retrieval-augmented generation (RAG) improves large language models by incorporating external knowledge without retraining, but existing methods often underuse the relational structure encoded in knowledge graphs. Graph-based RAG can capture entity relationships, yet supervised graph retrieval typically requires labeled question-answer data that may not be available for newly constructed graphs. We address this limitation with SelfGraphRAG, a framework that generates question-answer pairs directly from knowledge graph structure and uses them to train a query-conditioned graph retriever. The generated questions capture multi-hop paths and local neighborhoods, providing relational supervision without manual annotation. Experiments on multi-hop question answering and classification benchmarks show that SelfGraphRAG improves retrieval precision and downstream reasoning performance over embedding-based baselines. These results suggest that knowledge graph structure can provide useful supervision for training graph retrievers when labeled data are unavailable.
86. 【2608.25115】Less can be More: Relieving RAG Bottlenecks via Evidence Frontloading and Pressure-Adaptive Budgeting
链接：https://arxiv.org/abs/2608.25115
作者：Weibin Cai,Reza Zafarani
类目：Computation and Language (cs.CL); Information Retrieval (cs.IR)
关键词：improving Retrieval-Augmented Generation, Existing methods, optimize downstream LLM, downstream LLM generation, PACE
备注：
点击查看摘要
Abstract:Existing methods for improving Retrieval-Augmented Generation (RAG) efficiency mainly optimize downstream LLM generation, such as context compression or serving optimization. However, RAG is an end-to-end system, and its bottleneck can shift between upstream reranking and downstream generation under different serving loads and reranking this http URL this paper, we first empirically characterize this shifting-bottleneck behavior and show that upstream reranking can become the dominant bottleneck under high query rates or large reranking budgets. Reducing the reranking budget can relieve this bottleneck, but it may also drop supporting evidence and degrade recall. To address this problem, we propose \textbf{\textsf{PACE}} (\textbf{P}rioritized \textbf{A}daptive \textbf{C}overage of \textbf{E}vidence), a training-free framework that combines \textit{evidence frontloading} with \textit{pressure-adaptive budgeting}. \textsf{PACE} first reorders candidates by marginal evidence coverage, prioritizing documents that are query-relevant, complementary, and useful for forming multi-hop evidence chains. We show that this objective is monotone submodular, giving greedy selection a $(1-1/e)$ approximation guarantee. \textsf{PACE} then dynamically adjusts the reranking budget according to the relative pressure of the reranker and the LLM. Experiments on three multi-hop QA datasets and online serving simulations show that \textsf{PACE} improves evidence recall, reduces p95 latency under ranking-heavy workloads. More importantly, the two components together reveal that \textit{less can be more}: an evidence-dense top-ranked candidates enable higher final recall with fewer reranked documents.
87. 【2608.25089】Apples to Apples? Towards Comparable Crosslingual Language Model Evaluation
链接：https://arxiv.org/abs/2608.25089
作者：Xiulin Yang,Ethan Gotlieb Wilcox,Catherine Arnett
类目：Computation and Language (cs.CL)
关键词：multilingual NLP, enables fair comparisons, fair comparisons remains, enables fair, remains a fundamental
备注： EMNLP 2026
点击查看摘要
Abstract:Crosslingual evaluation of language models that enables fair comparisons remains a fundamental challenge in multilingual NLP. Existing studies adopt a variety of downstream tasks and intrinsic metrics with different theoretical justifications, yet there has been little empirical investigation into whether these approaches yield meaningful crosslingual conclusions. We systematically examine crosslingual evaluation approaches using controlled monolingual language models trained on parallel data with varying tokenizer vocabulary sizes and model sizes, and further validate our findings on multilingual LLMs. We further discuss challenges in achieving comparable downstream evaluation across languages. Our results show that several widely used normalized metrics introduce crosslinguistic biases rooted in tokenization, encoding, and orthographic differences. In contrast, sentence-level negative log-likelihood computed over semantically equivalent sequences provides more meaningful and consistent crosslingual comparisons.
88. 【2608.25085】MTDiag: A Multi-Turn Diagnostic Dataset Towards Clinically Meaningful LLM Evaluation
链接：https://arxiv.org/abs/2608.25085
作者：Pia Chouayfati,Alexander M. Fichtl,Miriam Anschütz,George Doumat,Georg Groh
类目：Computation and Language (cs.CL)
关键词：Large Language Models, medicine remains static, evaluating Large Language, Language Models, Large Language
备注： 20 pages, published in the SIGDIAL 2026 conference proceedings, see [this https URL](https://aclanthology.org/2026.sigdial-1.58/)
点击查看摘要
Abstract:Clinical diagnosis is fundamentally interactive and incremental, yet the dominant paradigm for evaluating Large Language Models (LLMs) in medicine remains static QA benchmarks or template-based dialogues. These benchmarks say little about whether a model can serve as a diagnostic agent in a dynamic clinical encounter, with LLMs showing significant accuracy and reliability degradation in multi-turn settings. To address this issue, we present MTDiag, a large multi-turn diagnostic dialogue dataset constructed from three heterogeneous sources: DDXPlus, MIMIC-IV, and published case reports (AJCR), covering common ED presentations as well as long-tail rare and atypical conditions. All cases are normalized into a canonical schema anchored in the most comprehensive and widely-adopted medical knowledge bases (UMLS concept identifiers, with ICD-10 diagnosis codes). We release the schema, a UserLM-8B-based utterance-generation pipeline, and the physician-validated dataset that converts structured clinical evidence into natural-language utterances. Importantly, we introduce and motivate clinical knowledge-grounded metrics for evaluating LLMs as diagnostic agents, beyond diagnostic accuracy, for the task of multi-turn differential diagnosis.
89. 【2608.25071】HealthBench-Psych: A Mental Health Subset of OpenAI's HealthBench
链接：https://arxiv.org/abs/2608.25071
作者：Matthew Flathers,Phuong Anh Nguyen,Jill Noorily,Julian Herpertz,Meiting Chen,Jasreen Multani,Samuel Powell,Mason Granof,Mark Kalinch,John Torous
类目：Computation and Language (cs.CL); Artificial Intelligence (cs.AI); Computers and Society (cs.CY)
关键词：making domain-specific performance, domain-specific performance hard, increasingly anchor claims, General-purpose health benchmarks, LLM medical performance
备注： 21 pages (6-page main text plus appendices), 2 figures. Code: [this https URL](https://github.com/mindbench-ai/healthbench-psych) Data: [this https URL](https://huggingface.co/datasets/mindbench-ai/healthbench-psych)
点击查看摘要
Abstract:General-purpose health benchmarks increasingly anchor claims about LLM medical performance, but they are not always resolved by clinical specialty, making domain-specific performance hard to isolate. Mental health is of acute public-health concern as millions of people turn to LLMs for psychological support, and most existing evaluations are bespoke academic benchmarks that are difficult to integrate into developer workflows. We introduce HealthBench-Psych and HealthBench-Psych-Hard. We screened HealthBench's 5,000 physician-rubric conversations for mental-health relevance with a transparent LLM-applied rubric, then validated the subset through two rounds of blinded clinician review with concealed known-exclude controls, yielding 610 conversations (12.2% of the corpus). Evaluating 20 frontier and open models under a cross-vendor panel of three LLM judges, we find a statistically tied frontier cluster, measurable refusal behavior in two models, and near-identical rankings across judges ($\tau \ge 0.92$). We release the subset, pipeline, model responses, grades, and analysis code as a reusable resource.
90. 【2608.25061】DataKernelBench: Can LLMs Optimize Database Queries on GPUs?
链接：https://arxiv.org/abs/2608.25061
作者：Gokul Karthik Kumar,Yotam Perlitz,Corey Lammie,Andrea Giovannini,Katja Hose
类目：Computation and Language (cs.CL); Artificial Intelligence (cs.AI); Databases (cs.DB); Machine Learning (cs.LG); Programming Languages (cs.PL)
关键词：accelerate database systems, increasingly accelerate database, query-specific peak performance, GPUs increasingly accelerate, database systems
备注： Accepted at EMNLP 2026. Homepage: [this https URL](https://kerneldf.github.io/datakernelbench)
点击查看摘要
Abstract:GPUs increasingly accelerate database systems, but query-specific peak performance still often relies on hand-written kernels. Existing LLM kernel benchmarks focus on machine learning operators, leaving irregular, heterogeneous, data-movement-heavy database-style operators untested. We introduce DataKernelBench, which translates SQL into validated PyTorch TorchPlan programs and evaluates LLMs that optimize either the core tensor-bounded snippet or the full query in CUDA or Triton through execution-guided repair. Across ten proprietary and open-weight models on TPC-H SF10 with an H100 GPU, the strongest full-query CUDA configuration achieves $2.11\times$ speedup over this http URL at full pass rate. We find that higher-performing implementations commonly use kernel fusion and execution-strategy changes, stronger models benefit most from full-query specialization, and workload context matters more than hardware context. To handle data larger than GPU memory, we extend TorchPlan with Dask-cuDF for on-demand partition loading on TPC-H SF100 with four H100 GPUs, achieving $2.54\times$ speedup
91. 【2608.25038】Padamitra: Grounded Glossary Generation for Classical Sanskrit
链接：https://arxiv.org/abs/2608.25038
作者：Manoj Balaji Jagadeeshan,Sai Pragnaan Marala,Pawan Goyal
类目：Computation and Language (cs.CL)
关键词：evaluable NLP objective, grounded glossary generation, introduce grounded glossary, structured task requiring, task requiring models
备注： Accepted in the Findings of EMNLP 2026
点击查看摘要
Abstract:We introduce grounded glossary generation, a structured task requiring models to recover semantically meaningful Sanskrit phrases and produce translation-grounded meanings from a sloka-translation pair, formalizing the traditional patha commentary practice as an evaluable NLP objective. We construct a benchmark of 31,316 sloka-translation-glossary triples from the Valmiki Ramayana and Srimad Bhagavatam, paired with two metrics: Jaccard for phrase recovery and Meaning Faithfulness for semantic consistency. Across zero-shot, few-shot, and instruction fine-tuned variants of Gemma-3n-E4B, Gemma-3-12B, Phi-4, and Qwen3.5-9B, instruction fine-tuning substantially outperforms prompting, while explicit segmentation yields gains. Error analysis identifies over-segmentation of sandhi and samasa compounds as the dominant failure mode, pointing to morphological modeling as the key bottleneck for faithful Sanskrit lexical decomposition.
92. 【2608.25037】Retrieve, Match, Escalate: Accurate and Scalable Product Linking with VLM-Distilled Cross-Encoders and Agentic VLMs
链接：https://arxiv.org/abs/2608.25037
作者：Jian Wang,Steven Xu,Sanjyot Thete,Maryam Barouti,Tom Tang,Elaine Wu,Charu Sareen,Kyle MacDonald
类目：Artificial Intelligence (cs.AI); Computation and Language (cs.CL); Databases (cs.DB); Information Retrieval (cs.IR)
关键词：consolidates fragmented listings, mapping merchant product, canonical catalog products, merchant product records, consolidates fragmented
备注： 9 pages, 3 figures, 5 tables
点击查看摘要
Abstract:Product linking, the entity-resolution task of mapping merchant product records to canonical catalog products, consolidates fragmented listings so downstream search, recommendation, and advertising see one clean entry per product. At marketplace scale, billions of noisy, multi-category records must be resolved against tens of millions of canonical products, where scoring every candidate with a single model is either too weak for the hard cases or too costly for the easy ones. We present a production retrieve-then-match cascade that spends computation in proportion to difficulty: retrieval surfaces plausible matches, a lightweight text cross-encoder auto-resolves the high-confidence majority, and an agentic multimodal vision-language model settles the ambiguous remainder by inspecting product images and issuing web searches for evidence that is in neither record. The cross-encoder is distilled from millions of dual-VLM-consensus labels, retiring human annotation from the training set, and is calibrated to auto-accept links at a 98% precision bar validated against a smaller operator-certified audit. The agent is a self-hosted open-weight model that reaches a closed frontier VLM's precision at a four-point recall cost (88% versus 92%) for roughly one-seventh the per-pair cost, with no fine-tuning. Per-pair cost spans nearly five orders of magnitude from the cheap cross-encoder to the frontier VLM, so escalating only the hard tail to the agent raises end-to-end link coverage from the cheap stage's 68% to 77%.
93. 【2608.25028】Behind the [MASK]: Disentangling Representation and Faithfulness in DAPF-Based Dementia Detection
链接：https://arxiv.org/abs/2608.25028
作者：Pardis Ranjbar-Noiey,Natalie Parde
类目：Computation and Language (cs.CL); Machine Learning (cs.LG)
关键词：remain internally opaque, non-invasive dementia screening, prompt-based domain-adaptive models, models remain internally, Spoken-language analysis
备注： 16 pages, 1 figure, 19 tables. Under review at ACL Rolling Review
点击查看摘要
Abstract:Spoken-language analysis via prompt-based domain-adaptive models is a promising direction for low-resource, non-invasive dementia screening, but such models remain internally opaque. We study the interpretability of the Domain-Adapted models via Prompt-based Fine-tuning (DAPF) framework, which casts dementia detection as diagnosis-related masked-token prediction. We interpret DAPF and strong baselines using a variety of probing and analysis techniques, finding that DAPF achieved the best overall performance (accuracy=0.83 and macro-F1=0.83) with diagnosis most recoverable from its [MASK] representation. However, this representational advantage did not extend to token-level explanation faithfulness. DAPF attributions primarily reflected language task vocabulary, discourse markers, and transcription artifacts, with perturbation tests showing weak or negative effects. This suggests that its masked-token interface determines diagnosis information without producing faithful token-level explanations.
94. 【2608.25022】A Primer on Computational Semantics for Artificial Intelligence Systems
链接：https://arxiv.org/abs/2608.25022
作者：Casey Kennington
类目：Computation and Language (cs.CL); Artificial Intelligence (cs.AI)
关键词：ChatGPT and Gemini, people adopt transformer-based, number of use-cases, adopt transformer-based language, people adopt
备注： 23 pages
点击查看摘要
Abstract:As people adopt transformer-based language models (e.g., ChatGPT and Gemini) for an increasing number of use-cases, it is important to know how such models learn and represent the meaning of the language, and to be more informed about what language is. This document is an attempt to help the reader understand how linguistic meaning (i.e., semantics) is approached from different fields of scientific and philosophical examination. I also explain three primary semantic theories: formal semantics, grounded semantics, and distributional semantics then compare how transformer-based language models differ from how humans learn language.
95. 【2608.25005】he Imperfective Paradox Is Not Necessarily in Large Language Models: A Benchmark Failure Before a Model Failure
链接：https://arxiv.org/abs/2608.25005
作者：Kaiqiao Han,Yizhou Sun
类目：Computation and Language (cs.CL); Machine Learning (cs.LG)
关键词：imperfective paradox, Teleological Bias, Calibration Crisis, Matched Minimal Pairs, Group
备注：
点击查看摘要
Abstract:The imperfective paradox provides a useful test of compositional semantic analysis. Recent work constructs an NLI benchmark and reports that models frequently infer completed telic events from progressive descriptions, attributing this behavior to a Teleological Bias. It further argues that prompting interventions cause a Calibration Crisis. We reexamine the benchmark and conclusions and show that it is substantially affected by conceptual and evaluation mis-specifications. We identify three conceptual mis-specifications. In particular, Aspectual Reduction affects the benchmark construction, analysis, experiments, and conclusions. Under a strict NLI standard, 76% of Group A instances do not explicitly rule out culmination. In our native-speaker annotation, 38% of Group A examples and 29% of the Group C examples were judged to permit an alternative interpretation. To control these issues and lexical variation, we construct Lexically Matched Minimal Pairs. At the evaluation level, we formulate event-semantic NLI as a Multi-step Reasoning Problem and assess both intermediate semantic decisions and final predictions. Our results show that models often do not affirm culmination but nevertheless accept the corresponding simple-past hypothesis, a pattern we characterize as Sufficiency Bias. We further show that prompting interventions produce a Decision Shift among labels without reliably improving the underlying semantic understanding and reasoning. Intermediate and oracle-guided analyses identify two additional failure modes: errors in compositional aspectual classification and Surface-form Attraction toward surface-associated answers. Our experiments on Qwen-7B with suitable prompts, GPT-5.4, and Qwen-72B provide initial evidence for the context sensitivity of aspectual classification and suggest that these models can achieve performance comparable to that of human annotators.
96. 【2608.24988】Does Fine-Tuning Undo Activation Steering? Behavioural Recovery Without Weight-Edit Reversal
链接：https://arxiv.org/abs/2608.24988
作者：Philipp E. Glass,Allan Tucker,Yongmin Li,Alina Miron
类目：Computation and Language (cs.CL); Artificial Intelligence (cs.AI); Machine Learning (cs.LG)
关键词：encode alignment prior, Activation steering, prior to release, encode alignment, alignment prior
备注： Accepted at EMNLP 2026 Main. Earlier version at ICLR 2026 Re^4-Align Workshop
点击查看摘要
Abstract:Activation steering can be embedded directly into a language model's weights, shaping behaviour without inference-time intervention and offering a way to encode alignment prior to release. However, models are routinely fine-tuned after deployment, and it is unknown whether embedded interventions survive this. We study the stability of embedded steering for refusal suppression and brevity induction across five instruction-tuned models (3B-14B) under non-adversarial SFT and RLHF. Behaviourally, preservation tracks the training data: steering degrades when optimisation pressure contradicts the targeted behaviour and persists otherwise, with refusal ablation losing 64% of its effect on average under SFT. Mechanistically, however, the weight edit survives almost untouched even where behaviour reverts: mean vector recovery is $\rho = 0.004$, and the fine-tuning update along the steering direction is near-orthogonal to its pre-edit weight pattern (mean $\cos\theta = 0.074$). When steered behaviour degrades, fine-tuning does not achieve it by dismantling or reversing the steering mechanism itself. Embedded steering is therefore mechanistically durable but functionally vulnerable, and requires behavioural re-validation after downstream training.
97. 【2608.24982】Unsupervised Post-Training of Foundation Models: A Survey
链接：https://arxiv.org/abs/2608.24982
作者：Yijie Xu,Qianyi Cai,Huizai Yao,Yili Wang,Tianfu Wang,Cehao Yang,Xingbo Yao,Zhiyu Guo,Aiwei Liu,Xuming Hu,Weiyu Guo,Hui Xiong
类目：Computation and Language (cs.CL); Artificial Intelligence (cs.AI); Computer Vision and Pattern Recognition (cs.CV); Machine Learning (cs.LG); Multimedia (cs.MM)
关键词：preference data, stronger teachers, Foundation-model post-training, human labels, executable verifiers
备注： Accepted to Findings of EMNLP 2026. 20 pages, 3 figures, 8 tables
点击查看摘要
Abstract:Foundation-model post-training usually relies on human labels, preference data, stronger teachers, or executable verifiers. We study Unsupervised Post-Training (UPT): update-bearing adaptation on unlabeled inputs whose learning signal is derived from same-lineage model artifacts rather than an external oracle. We catalog 80 strict UPT methods and organize them by the object that supplies the update signal: a prediction statistic, a sample relation, a self-generated target, or an internal evaluator. Beyond inventory, we show how the choice of internal signal and task structure determines whether post-training improves the model or recursively amplifies error. An orthogonal Input Visibility $\times$ Update Persistence view maps deployment regimes and defines a unified framework for UPT selection and evaluation.
98. 【2608.24979】FrontierChallenge: Evaluating Scientific Workflow Completion
链接：https://arxiv.org/abs/2608.24979
作者：Liangcai Su,Zhaopeng Feng,Zhuo Chen,Zhen Zhang,Xiang Lin,Ruilin Li,Handuo Zhang,Ning Wang,Kailong Wen,Yueqi Guo,Feng Xing,Yiling Guo,Chenxiong Qian,Simon Shaolei Du,Lidong Bing,Xinyu Wang
类目：Artificial Intelligence (cs.AI); Computation and Language (cs.CL); Software Engineering (cs.SE)
关键词：increasingly analyze data, produce research artifacts, emphasize final answers, benchmarks emphasize final, agents increasingly analyze
备注： Project Website: [this https URL](https://apodexai.github.io/FrontierAgent/benchmarks/FrontierChallenge/)
点击查看摘要
Abstract:Scientific agents increasingly analyze data, execute code, and produce research artifacts, yet most benchmarks emphasize final answers, isolated programs, or a single domain. We introduce FrontierChallenge, a cross-domain benchmark comprising 300 end-to-end scientific workflows. In this paper, we release and evaluate 97 of these tasks, spanning quantum chemistry, molecular dynamics, materials characterization, analytical chemistry, life science, and electrochemistry/environment. Each task provides fixed inputs and specifies a bundle of required scientific deliverables. We evaluate twelve frontier models with three agent scaffolds. Pass Rate measures the fraction of tasks satisfying the full-completion criterion, while Avg. Score captures partial progress. Each of the best-performing configurations completed only 20 of the 97 released tasks, yielding a Pass Rate of 20.6%. Partial progress translated especially poorly into complete delivery in analytical chemistry and electrochemistry/environment: Avg. Scores reached 87.6 and 94.9, but the highest Pass Rates were only 4% and 0%. Among non-passing Claude Code trajectories, 75.5% still ended with language claiming completion. These findings show that neither high partial scores nor confident claims of completion reliably indicate that a scientific task has been fully delivered, highlighting the need to evaluate end-to-end workflow execution and the completeness of scientific deliverables together.
99. 【2608.24977】Retrieved But Not Reliable: A Survey on Attacks, and Defenses in Retrieval-Augmented Generation
链接：https://arxiv.org/abs/2608.24977
作者：Minh Tran,Cuong Dang,Tuc Nguyen,Khanh-Tung Tran,Minh Huynh Nguyen,Trinh Chau,Kien Le,Do Xuan Long,Jiahao Zhang,Hoang D. Nguyen,Thanh Le,Suhang Wang
类目：Cryptography and Security (cs.CR); Computation and Language (cs.CL); Machine Learning (cs.LG)
关键词：enhances large language, large language models, enhances large, external knowledge, improving factuality
备注： 24 pages, 6 figures. Accepted to Findings of the Association for Computational Linguistics: EMNLP 2026. Peer-reviewed through ACL Rolling Review (ARR)
点击查看摘要
Abstract:Retrieval-Augmented Generation (RAG) enhances large language models by grounding outputs in external knowledge, improving factuality and reducing hallucinations. At the same time, the retrieval-augmented pipeline introduces new robustness and security risks, including corpus poisoning, backdoor attacks, privacy leakage, and fairness violations. Despite rapid progress in this area, existing surveys remain limited in their treatment of attacker objectives, threat models, and stage-specific defenses across the full RAG pipeline. This survey presents a unified and pipeline-aware overview of RAG robustness. We formalize threat models over the corpus, retriever, and generator, and organize attacks into three main objectives: accuracy, privacy, and fairness. We further review defenses from a pipeline-aware perspective, covering the retrieval, rerank, generation, and traceback stages. In addition, we summarize robustness benchmarks and explainability methods for more deeply evaluating and explaining RAG robustness.
100. 【2608.24958】Can We Read the Mind of an Audio LLM? A Verbalizable, Multilingual Middle-Layer Workspace
链接：https://arxiv.org/abs/2608.24958
作者：Jiajun Fan,Jingyuan Li,Prashanth Gurunath Shivakumar,Qi Luo,Jia-Hong Huang,M. Maruf,Roger Ren,Yile Gu,Rahul Pandey,Ge Liu,Ivan Bulyko
类目：ound (cs.SD); Artificial Intelligence (cs.AI); Computation and Language (cs.CL)
关键词：audio language model, black box, writes its reasoning, language model, model writes
备注：
点击查看摘要
Abstract:An audio language model is a black box in a specific way: we see what it says, never what it works out on the way there, and chain-of-thought monitoring helps only if the model writes its reasoning down. Reading a base Qwen3-Omni with a logit lens at the audio-token positions, we find that the answer to a spoken question becomes legible - in words - in the model's middle layers, before it emits any token. Five findings follow. (1) The readout carries concepts in neither the question, the options, nor the model's own transcription: on a clip whose verbatim transcription is empty garbling, it reconstructs Watergate and scandal, passes through the role president, and resolves to Nixon - a hidden multi-hop chain, read with no chain-of-thought. (2) The content is language-agnostic: one audio-inferred concept surfaces in several scripts at once, and 38% of top-1 readouts are Chinese on English inputs. (3) It is paralinguistic: given the same clip as audio and as the model's own emotion-free caption, the audio mind forms the sound source, speaker role, or affect that the caption discards, and answers correctly more often. (4) The audio-driven signal is absent at the input, turns on about a tenth of the way into the network, separates most cleanly from the text prior in the middle band (35-80% of depth), and activation patching shows it is causally used and committed before the last fifth of the layers. (5) Deleting single layers maps the pipeline: reading the sound in is localized to the entry layers and answer delivery to the output layer, while retrieval is distributed across the interior. Throughout, a waveform-swap control - identical text, only the sound changed - isolates the audio-driven signal from a prior over the printed options. This is a qualitative account of what an audio model works out before it speaks: the quantities are controls, not benchmark scores.
101. 【2608.24952】he Dialect Tax: Dialectal Biases Persist throughout the Language Modeling Pipeline
链接：https://arxiv.org/abs/2608.24952
作者：Elle
类目：Computation and Language (cs.CL); Artificial Intelligence (cs.AI); Machine Learning (cs.LG)
关键词：pipeline remains unclear, Systematic dialectal performance, Standard American English, Systematic dialectal, remains unclear
备注： To be published at EMNLP 2026 under the full author name "Elle"
点击查看摘要
Abstract:Systematic dialectal performance gaps in language models (LMs) are well documented, but the source of these disparities within the modern language modeling pipeline remains unclear. Our study traces this "dialect tax" across the natural language processing pipeline. Using parallel English dialect corpora that hold meaning fixed while varying surface form, we first confirm that LMs recognize matched Standard American English (SAE) and dialectal texts as semantically equivalent. However, we discover further representational gaps corresponding to downstream performance gaps. Across model families and generations, modern LMs still encode dialectal texts unequally during tokenization, pre-training, post-training, and inference. Strikingly, bypassing traditional subword segmentation via a character-level counterfactual tokenizer removes neither input and output asymmetries nor dialectal accuracy gaps. During pre-training, dialect pairs induce more divergent gradient updates than pairs of entirely unrelated SAE documents, indicating that models find semantically equivalent dialectal content harder to learn from than unrelated SAE documents. During post-training, reward models show contextual, unstable dialect preferences, assigning higher values to isolated AAVE-exclusive tokens than to SAE-exclusive tokens, while full reasoning contexts receive task- and model-dependent dialect penalties. Overall, our findings suggest that the dialect tax is encoded and accumulated not by any one step in isolation, but at every step of the language modeling process.
102. 【2608.24949】Demystifying Reinforcement Learning Post-Training of Language Models
链接：https://arxiv.org/abs/2608.24949
作者：Donovan Clay,Saket Gollapudi,Sankar Harilal,Min Jang,Jacob Morrison,Sewoong Oh,Natasha Jaques
类目：Machine Learning (cs.LG); Artificial Intelligence (cs.AI); Computation and Language (cs.CL)
关键词：enabling impressive reasoning, large language models, Reinforcement learning, coding capabilities, enabling impressive
备注： Link to website: [this https URL](https://minjang10.github.io/demystifying-rl-finetuning-web) Link to code: [this https URL](https://github.com/sankarh-1/demystifying-rl-finetuning)
点击查看摘要
Abstract:Reinforcement learning (RL) post-training has emerged as a powerful framework for enhancing the capabilities of large language models (LLMs), enabling impressive reasoning, math, and coding capabilities. Yet for many researchers and practitioners, the principles behind classical RL remain a "black box". In this work, we deconstruct the RL post-training algorithm, investigating each step to clarify what is actually happening beneath the surface. By isolating the mechanics of RL with Verifiable Rewards in a controlled and simplified environment, we examine how RL outcomes are shaped by the base model's prior distribution, the granularity of the reward signal, the diversity of the prompt distribution, and model scale. We use the entropy of the policy's output distribution as a lens to compare the distributions learned through pretraining, SFT, and RL post-training, revealing how each stage shapes model certainty. Our investigation sheds light on how these choices interact to affect post-training success. For example, we show that the effect of so-called 'spurious rewards' depends on the prompt distribution used for post-training. We also provide insight into why the success of RL post-training depends on whether the base model already places sufficient probability mass on the desired behavior, linking it to the classical concept of exploration in RL. Ultimately, we provide this primer as a resource to those in the NLP community wishing to incorporate RL as a tool in their toolbox.
103. 【2608.24936】GreenLeaf Law Embed Tiny: A Compact Embedding Model for Legal Domain Retrieval
链接：https://arxiv.org/abs/2608.24936
作者：Surya Saka
类目：Machine Learning (cs.LG); Artificial Intelligence (cs.AI); Computation and Language (cs.CL)
关键词：Law Embed Tiny, GreenLeaf Law Embed, Embed Tiny, present GreenLeaf Law, Law Embed
备注： 7 pages, 2 figures, IEEE dual-column format. Submitted to arXiv for preprint distribution
点击查看摘要
Abstract:We present GreenLeaf Law Embed Tiny, a 0.6B parameter embedding model for legal domain retrieval. GreenLeaf-Tiny achieves 75.11% on the Massive Legal Embedding Benchmark (MLEB) and 64.38% on MTEB(Law, v1),demonstrating competitive performance among models under 1B parameters. Our approach combines a two-stage training pipeline that first distills knowledge from a larger teacher model into a compact student architecture, then applies domain-specific fine-tuning with hard negative mining; a carefully curated dataset of 3.4 million query-passage pairs, including 150,000 human-curated samples across diverse legal jurisdictions; and an efficient inference architecture supporting multiple quantization levels (BF16, INT8, binary) enabling deployment in resource-constrained environments. We provide detailed analysis of our training methodology, architectural choices, and comprehensive evaluation across legal retrieval tasks. Our results demonstrate that domain-specific training with high-quality data can improve performance for specialized domain applications
104. 【2608.24920】Semantic Variability of Replies Across LLMs: Implications for Designing Conversation-Based Assessment
链接：https://arxiv.org/abs/2608.24920
作者：Jiangang Hao
类目：Computation and Language (cs.CL); Artificial Intelligence (cs.AI); Human-Computer Interaction (cs.HC)
关键词：remain semantically consistent, LLM-generated replies remain, replies remain semantically, study examines, examines whether LLM-generated
备注： 9 pages, 4 figures, two tables. Accepted to the AI in Measurement and Education Conference 2026
点击查看摘要
Abstract:This study examines whether LLM-generated replies remain semantically consistent when the underlying LLM changes. Using messages from real collaborative conversations, we compared the semantic similarity of generated replies across LLMs under two conditions: with and without preceding chat history. Results show that model choice and conversational context both affect response similarity and alignment with human replies. These findings indicate that prompting and conversational context alone may not be sufficient to preserve response consistency across LLMs, highlighting the need for infrastructure and design strategies that can maintain stable and comparable responses amid the rapid and continuous evolution of LLMs.
105. 【2608.24907】PA-CoT: Profile-Adaptive Chain-of-Thought for Personalized Nutritional Consulting
链接：https://arxiv.org/abs/2608.24907
作者：Evgenii Garmashov,Nikita Kulin,Artur Khairullin,Viktor Zhuravlev,Daniil Sukhorukov,Mikhail Mozikov,Ilya Makarov,Sergey Muravyov
类目：Human-Computer Interaction (cs.HC); Artificial Intelligence (cs.AI); Computation and Language (cs.CL); Machine Learning (cs.LG)
关键词：critical structural gap, dedicated analysis step, prompting methods pass, structural gap, health and nutrition
备注： Accepted at the ICML 2026 Workshop on Structured Data for Health (SD4H)
点击查看摘要
Abstract:In health and nutrition consulting, widely used prompting methods pass the user profile as an unstructured block without a dedicated analysis step, leaving personalization as a critical structural gap. We introduce PA-CoT (Profile-Adaptive Chain-of-Thought), a multi-stage prompting method that treats profile interpretation as an explicit, standalone reasoning step prior to response generation. To enable systematic evaluation, we introduce the QPA (Question--Profile--Answer) benchmark -- 200 nutritional consulting samples with structured user profiles scored on four criteria. In a comparative study against 11 comparison methods (CoT, Few-Shot, Role Prompting, DSPy, TextGrad, Self-Refine, and others, plus a Zero-Shot Baseline; 12 total including PA-CoT), PA-CoT achieves the best average score (4.21 on the G-Eval 1--5 scale) and leads on both Personalization (4.71 vs. 4.39) and Safety (4.68 vs. 4.52) with non-overlapping 95\% confidence intervals over the nearest competitor -- the only method to simultaneously top both criteria. The results confirm that an explicit profile-analysis step is the key driver of personalization gains over widely used prompting approaches.
106. 【2608.24901】Detection != Reliable Control: Decodable Empathy Directions Yield at Most Partial Shifts in Automated Empathy Scores
链接：https://arxiv.org/abs/2608.24901
作者：Haoran Jisun
类目：Computation and Language (cs.CL); Human-Computer Interaction (cs.HC); Machine Learning (cs.LG)
关键词：conflating decodability, causal lever, routinely read, automated-metric control, discriminative EPITOME classifier
备注： Under review at BlackboxNLP 2026 (EMNLP). 8 pages body, 10 figures/tables, plus appendix
点击查看摘要
Abstract:A decodable "empathy" direction is routinely read as a causal lever, conflating decodability, automated-metric control, and human-perceived change. We test this for two EPITOME-derived facets -- Recognition (cognitive) and Resonance (affective) -- in three instruction-tuned LLMs, scoring every intervention with two LLM judges and a discriminative EPITOME classifier, each gated by an emotional-vs-neutral positive control. The control passes for the affective facet across all automated instruments, but cognitive range is inconsistent across them. Both facets remain decodable after residualizing against a sentence-embedding-derived surface score, and steering can substantially rewrite the text. Yet adding the Resonance direction raises the affective score only partially -- in Qwen by +0.29 (approximately 26% of the natural gap). A direct between-direction contrast confirms the shift is facet-specific in Qwen and Llama (not Gemma); we do not, however, establish a matching human-perceived change. Additive cognitive steering produces no measurable change, but a within-domain control shows the cognitive instrument is too coarse to resolve the differences such steering would produce -- unmeasurable, not a clean null. By contrast, Gemma Recognition ablation lowers the classifier's cognitive score even after adjusting for response length. Detection does not imply reliable control under global interventions, and cognitive-empathy claims warrant an explicit measurement-sensitivity check.
107. 【2608.24892】Natural Language Input, Semantic Track Representation, and LLM Inference: Making the Maritime Information Exchange Model Tractable
链接：https://arxiv.org/abs/2608.24892
作者：Frederick Roth
类目：Artificial Intelligence (cs.AI); Computation and Language (cs.CL)
关键词：Information Exchange Model, Maritime Information Exchange, Rich Semantic Track, broader Rich Semantic, Information Exchange
备注： 11 pages, under review
点击查看摘要
Abstract:We describe a practical architecture for making the Maritime Information Exchange Model (MIEM) and the broader Rich Semantic Track model tractable using current large language model (LLM) technology. The barrier to adoption of semantic track models in defense and law enforcement has been the requirement that operators learn formal ontology languages and manually encode observations as typed logical assertions. We propose eliminating this barrier entirely: operators contribute observations in natural language; an LLM translates these into typed Semantic Assertion Records (SARs), which are named case frames that capture n-ary relations in a single compact structure; a knowledge graph accumulates the SARs; and a second LLM pass performs inference, anomaly detection, and hypothesis ranking over the graph. We work through two detailed examples (a 9/11-era pre-attack indicator scenario and a maritime cargo inspection scenario) showing the full pipeline from natural language input to SAR representation to inference output. We argue that this architecture makes the Track Model and MIEM immediately deployable with current technology, establishes prior art against proprietary enclosure of the approach, and grounds the method in a theoretical framework connecting semantic track representations to neural manifold geometry.
108. 【2608.25218】urnBench: A Multi-Domain Benchmark for Turn-Taking Dynamics in Spoken Dialogue
链接：https://arxiv.org/abs/2608.25218
作者：Freeman Jiang,Ramon Sanabria,Soham Deshmukh,Bandhav Veluri,Simon Michael Vuch Williams,Elliott K. Suen,Garreth Lee,Kevin Yoonho Choi,Takuya Umeki,Riku Kubo,Sathvik Udupa,Chien-yu Huang,Shih-Yun Shan Kuan,Zhuoyan Tao,Satyapriya Krishna,Sefik Emre Eskimez,Yu Tsao,Hung-yi Lee,Shinji Watanabe
类目：Audio and Speech Processing (eess.AS); Computation and Language (cs.CL)
关键词：Speakers in natural, deciding in real, real time, Speakers, natural conversation
备注： 8 pages, 2 figures. Submitted to IEEE SLT 2026
点击查看摘要
Abstract:Speakers in natural conversation take turns speaking and listening, deciding in real time when to take, hold, or yield the floor. However, turn-taking evaluation remains limited due to the lack of a consistent, linguistically grounded evaluation protocol and hand-annotated data covering diverse conversation types. To address this, we present TurnBench, a multi-domain benchmark that pairs a 30-hour, hand-labeled corpus of dyadic human conversation with a standardized evaluation protocol for end-of-turn and interruption detection. We set conversation type as a controllable experimental variable, covering six distinct interaction styles, and triple-annotate each conversation. Benchmarking 14 heterogeneous turn-taking systems, we find end-of-turn recall stable across types, while interruption false positives are strongly type-dependent and concentrated in backchannel-dense interaction styles. Although in smooth floor transfers human listeners begin speaking a median 151 ms before the current turn ends, no current system performs equivalently without incurring excessive false positives. We release our corpus, a 104-hour training set, and a public leaderboard with an interactive dataset viewer at this https URL
信息检索
1. 【2608.26091】PlanSightRAG: A Visual-First Multimodal RAG for Automating Question Answering and Compliance Checking for Civil Standard Plans
链接：https://arxiv.org/abs/2608.26091
作者：Nabaraj Subedi,Shuvo Dip Datta,Ahmed Abdelaty,Shivanand Venkanna Sheshappanavar
类目：Information Retrieval (cs.IR); Computation and Language (cs.CL); Computer Vision and Pattern Recognition (cs.CV)
关键词：OCR-based automation strips, manually reading legacy, engineers manually reading, Civil infrastructure compliance, Civil infrastructure
备注： 32 pages, 9 figures, 25 tables. Preprint submitted to Automation in Construction
点击查看摘要
Abstract:Civil infrastructure compliance checking has long relied on engineers manually reading legacy 2D plans; however, OCR-based automation strips away the geometry and layout essential for interpreting these plans. We present a Visual-First Multimodal Retrieval-Augmented Generation (RAG) framework called PlanSightRAG. It indexes and reasons directly over plan imagery, integrates a ColNomic-3B multi-vector retrieval, an agentic Planner-Retriever-Auditor-Synthesizer, and MaxSim heatmaps as an evidence trail. We introduce a 4,056-pair benchmark from five state Departments of Transportation (DOT) standard plans (1,898 pages). PlanSightRAG achieves 91.47% Recall@5 on zero-shot retrieval, while on a held-out Michigan DOT corpus, it achieves 91.40%. On synthetic, parametrically-generated compliance drawings, our Qwen2.5-VL-72B pipeline reaches 100% verdict accuracy only when supplied a pre-resolved rule threshold, a controlled ceiling that a non-VLM OCR baseline already reaches at 76.4%. Finally, we demonstrate autonomous visual rule-grounding by extracting numeric limits directly from a specification corpus without any human-supplied rules.
2. 【2608.25922】Query-Side Attacks on GNN-Based KGQA: Tracing Failures from Entity Linking to Answer Generation
链接：https://arxiv.org/abs/2608.25922
作者：Pankaj Kumar,Subhankar Mishra
类目：Computation and Language (cs.CL); Artificial Intelligence (cs.AI); Information Retrieval (cs.IR)
关键词：Graph Question Answering, GNN-based Knowledge Graph, pipelines process queries, Question Answering, Knowledge Graph Question
备注：
点击查看摘要
Abstract:GNN-based Knowledge Graph Question Answering (KGQA) pipelines process queries through four discrete stages: entity linking, subgraph retrieval, GNN reasoning, and answer generation. Standard robustness evaluations conflate stage-level failures into a single end-to-end metric, obscuring both the source of brittleness and the appropriate mitigation target. We ask which stage fails, and why, when the pipeline is subjected to adversarial perturbations on the input question. We introduce a stage-isolation protocol with two answer-preserving adversarial perturbations verified against the knowledge graph: Compositional Restructuring (CR) and Relation Synonym Swap (RS) target distinct stages while leaving entity seeds intact. Evaluated across ComplexWebQuestions and WebQSP, the results run counter to prevailing assumptions: the GNN reasoning stage retains near-baseline accuracy when the subgraph is intact, while subgraph construction accounts for over 99\% of the end-to-end collapse under CR, occurring even when the gold answer is present in 74\% of retrieved subgraphs. This exposes a fundamental distinction between answer presence and answer reachability that end-to-end metrics cannot detect, and places the mitigation target firmly at the subgraph construction stage rather than the reasoning model. Perturbed datasets and evaluation infrastructure are released at this https URL .
3. 【2608.25780】PUMA: Post-Hoc Sparsification of Universal Multimodal Embeddings for Efficient Retrieval
链接：https://arxiv.org/abs/2608.25780
作者：Matteo Attimonelli,Alessandro De Bellis,Franco Maria Nardini,Claudio Pomo,Cosimo Rulli,Rossano Venturini,Tommaso Di Noia
类目：Information Retrieval (cs.IR)
关键词：representations incur high, incur high memory, multimodal embedders enable, Universal multimodal embedders, embedders enable retrieval
备注：
点击查看摘要
Abstract:Universal multimodal embedders enable retrieval across text, image, and combined queries, but their dense representations incur high memory and inference costs. Post-hoc sparsification could reduce these costs but remains underexplored for multimodal retrieval. We introduce PUMA, a sparse autoencoder recipe that maps universal multimodal embeddings to compact sparse codes without retraining the backbone: a pretraining stage preserves dense dot-product geometry, after which the sparse encoder is fine-tuned for retrieval. We evaluate on five benchmarks covering text-to-image and composed image retrieval. On Qwen3-VL-Embedding-2B, PUMA is statistically indistinguishable from or improves over dense retrieval on four of five datasets. We further identify two failure modes of post-hoc sparsification: insufficient pre-TopK support and retrieval-misaligned active support. PUMA reduces vector storage by 8-16x (FP32) and is up to 25x faster than exact dense scoring on larger candidate pools, enabling efficient multimodal retrieval.
4. 【2608.25755】Hamiltonian Spectral-Temporal Dissipative Dynamics for Sequential Recommendation
链接：https://arxiv.org/abs/2608.25755
作者：Shuiying Liao,P. Y. Mok
类目：Information Retrieval (cs.IR)
关键词：current latent representation, recommendation requires understanding, state depends solely, user preferences evolve, Sequential recommendation requires
备注： 10 pages
点击查看摘要
Abstract:Sequential recommendation requires understanding how user preferences evolve over time, yet most existing models treat such evolution as a first order process where the next state depends solely on the current latent representation. Nevertheless, real user behavior often exhibits richer dynamics, including inertia, periodicity, and sudden shifts that cannot be fully captured by these first order assumptions. Motivated by these behavioral characteristics, we reconceptualize sequential recommendation through the lens of second order dynamical systems and introduce the Hamiltonian Spectral Recommender (HSR), which recasts preference evolution as a dissipative Hamiltonian system in a latent phase space of position (stable preference) and momentum (short-term tendency). The linear time-invariant structure of the governing equation admits a closed-form solution in the frequency domain. A learnable dissipation mechanism further captures natural interest decay, while a short local impulse refinement module models abrupt behavioral fluctuations commonly observed in sparse interaction logs. This design jointly accounts for global periodic patterns, inertial evolution, and localized shocks, where three phenomena that are underrepresented in existing sequential models. Extensive experiments on three benchmark datasets demonstrate that HSR consistently outperforms state-of-the-art Transformer-based and state space model (SSM)-based recommenders.
5. 【2608.25737】D3ER: Supporting Multi-Modal Recommendation via Disentangle and Distillation-based Dynamic Ensemble
链接：https://arxiv.org/abs/2608.25737
作者：Bingnan Wang,Yi Li,Xiongxin Tang,Fanjiang Xu,Jiangmeng Li
类目：Information Retrieval (cs.IR); Multimedia (cs.MM)
关键词：Incorporating items' information, canonical unimodal recommendation, demonstrated documented success, Incorporating items', items' information shared
备注： Accepted by ACMMM 2026
点击查看摘要
Abstract:Incorporating items' information shared among multiple modalities into a fused representation, multi-modal recommendation (MR) has demonstrated documented success than canonical unimodal recommendation. Although several attempts have been made to extract the discriminative information unique in each modality, existing methods suffer from a core limitation: the joint learning of modal-homogeneity discriminative information (HOI) and modal-heterogeneity discriminative information (HEI) tends to weaken their individual effectiveness. To remedy this deficiency, we propose a novel method, dubbed Disentangle and Distillation-based Dynamic Ensemble for multi-modal Recommendation (D3ER). We introduce gradient boosting into MR for the first time to formalize the optimization objective for alternately learning HOI and HEI. This design enables models dedicated to each type of information to focus on their proficient samples, thereby promoting specialized optimization. Furthermore, to mitigate the inherent high storage cost and risk of local optima in gradient boosting, we enhance our framework with knowledge distillation and a global correction regularization. Experiments on prevalent real-world datasets confirm the superiority of our proposed method on MR.
6. 【2608.25735】Pointing the Way, Hiding the Destination: Practical Private Dense Retrieval at Scale
链接：https://arxiv.org/abs/2608.25735
作者：Peichun Hua,Danyang Chen,Junan Zhang,Haifeng Sun,Jingyu Wang,Diwen Xue,Mingyu Li,Yunming Xiao
类目：Cryptography and Security (cs.CR); Artificial Intelligence (cs.AI); Information Retrieval (cs.IR); Machine Learning (cs.LG)
关键词：Hosted retrieval-augmented generation, query valuable provider-held, Hosted retrieval-augmented, valuable provider-held corpora, retrieval-augmented generation
备注： 30 pages, 9 figures, 16 tables
点击查看摘要
Abstract:Hosted retrieval-augmented generation (RAG) and semantic search allow users to query valuable provider-held corpora, raising two competing demands: to hide each query and chosen result, yet reveal only the documents that the user is authorized to receive. Existing cryptographic approaches either make this costly by processing the entire corpus for every query, or sacrifice quality for efficiency by scanning a few clusters. We repurpose learned deep hashing as a private filter: a randomized binary code points the provider to a short candidate list, while encrypted reranking and oblivious key transfer protect the precise query and final selection. This shortlist short-circuits full-corpus cryptographic search without sacrificing retrieval quality: with 200-500 candidates, it closely matches full-corpus retrieval across five zero-shot corpora spanning 25K to 5.4M documents. On the full 2.68M-passage NQ corpus over a 10-Gbps link, our protocol only adds 0.73 seconds, or 10 percent, to a 128-token Qwen3-32B RAG pipeline. The released code satisfies directional metric differential privacy (DP) and substantially reduces embedding-inversion and property-inference leakage, demonstrating that a carefully learned shortlist can make private dense retrieval both accurate and practical.
7. 【2608.25663】Data Citation for Large Language Models: A Challenge
链接：https://arxiv.org/abs/2608.25663
作者：Gianmaria Silvello
类目：Information Retrieval (cs.IR); Artificial Intelligence (cs.AI); Databases (cs.DB); Digital Libraries (cs.DL)
关键词：increasingly mediate access, models increasingly mediate, Large language models, language models increasingly, increasingly mediate
备注： 7 pages, journal paper
点击查看摘要
Abstract:Large language models increasingly mediate access to information, and a growing body of work asks whether they cite the sources behind their outputs. That work treats citation as a verification device and applies it to textual documents. Scholarly citation serves two further functions, credit and provenance, and it applies to data as much as to text. This paper argues that data citation for large language models is an open challenge, distinct from document-level citation grounding and harder to solve. We ask how such models should cite data so that outputs stay verifiable, provenance stays traceable, and credit reaches data creators and curators. We set out three research directions. Training data attribution has to turn influence estimates into references for corpora absorbed into model parameters. Data citation at inference time has to identify datasets, subsets, and query results at the right granularity and fixity. Citing knowledge graph facts has to define what a reference to a single triple denotes and how credit propagates along provenance. Progress on all three depends on joint work across the database, information retrieval, knowledge representation, and artificial intelligence communities.
8. 【2608.25635】DCEO: Direct Causal Effect Optimization for Long-Term User Value Modeling in E-commerce Search
链接：https://arxiv.org/abs/2608.25635
作者：Junzhao Zhang,Tao Zhang,Liren Yu,Feiyi Dong,Zhixuan Zhang,Dan Ou,Haihong Tang
类目：Machine Learning (cs.LG); Information Retrieval (cs.IR)
关键词：n-day cumulative purchases, ultimate objective, systems ultimately aim, item-level proxy scores, Industrial e-commerce search
备注：
点击查看摘要
Abstract:Industrial e-commerce search systems ultimately aim to optimize the user-level long-term objective, such as n-day cumulative purchases or gross merchandise value (GMV) per user. However, such objectives are defined at the user level, whereas search ranking is based on item-level scores within each request. Existing methods typically bridge this granularity gap through manually designed multi-objective fusion, where predictions of multiple item-level objectives, such as clicks, carts, purchases, and transaction value, are combined into a ranking score that serves as a proxy for the ultimate objective. Such hand-crafted fusion schemes rely on a small set of manually tuned weights, limiting fine-grained personalization and leading to suboptimal alignment with the ultimate objective. In this paper, we propose DCEO (Direct Causal Effect Optimization), a data-driven framework for learning item-level proxy scores that are better aligned with the ultimate objective. We first aggregate the item-level proxy scores into a user-level proxy metric and quantify its alignment with the ultimate objective using a relative causal effect. We then develop an actor-critic framework, where the critic estimates the ultimate objective for a given user-level proxy metric, and the actor dynamically generates context-dependent fusion weights over multiple objectives to construct the item-level proxy scores and is trained to directly optimize the relative causal effect. Extensive offline experiments and analyses demonstrate the effectiveness and interpretability of DCEO. In addition, DCEO has been deployed in a large-scale industrial e-commerce search system, outperforming the conventional GMV proxy by 0.36% in GMV in a 41-day online A/B test.
9. 【2608.25625】RetrievalRouter: Joint Modality and Architecture Selection for Document Retrieval
链接：https://arxiv.org/abs/2608.25625
作者：Emre Kuru,Mehmet Onur Keskin,Reza Farahbakhsh,Noel Crespi
类目：Information Retrieval (cs.IR)
关键词：increasingly supports high-stakes, supports high-stakes information, high-stakes information access, retrieval increasingly supports, Document retrieval increasingly
备注： Accepted at EMNLP 2026
点击查看摘要
Abstract:Document retrieval increasingly supports high-stakes information access in finance, healthcare, and law. Modern retrieval pipelines vary both in modality (text or multimodal) and in retrieval architecture (dense or late-interaction). These choices impose a hard compromise: the most effective pipelines are too slow and expensive to run at scale, while the fastest fail to retrieve evidence from complex documents. Practitioners must therefore choose between missed evidence and unusable latency, with no principled basis for adapting that choice at the query level. We show that this compromise is unnecessary. Not every query requires the same pipeline. Across benchmarks spanning financial and scientific corpora, no static pipeline dominates. We introduce RetrievalRouter, a lightweight query-aware router that learns, from the query text alone, which retrieval pipeline best fits each query. A single tunable parameter exposes the full accuracy-latency frontier, and for every static baseline, RetrievalRouter offers an operating point that is simultaneously more accurate and faster. Against the best static baseline, RetrievalRouter is 2.5% more accurate and 12.4 times faster. Furthermore, compared with prior adaptive strategy selection methods, RetrievalRouter achieves significantly higher nDCG@5 across accuracy-oriented settings, while matching or numerically outperforming them on both nDCG@5 and latency in latency-oriented settings. Our code and data are available at this https URL.
10. 【2608.25553】When Stale Constraints Go Unchecked: Budgeted Verification Failures in Inherited Agent Memory
链接：https://arxiv.org/abs/2608.25553
作者：Kazuki Nakayashiki
类目：Information Retrieval (cs.IR); Artificial Intelligence (cs.AI); Computation and Language (cs.CL)
关键词：newer authoritative record, inherits a consolidated, true when written, newer authoritative, authoritative record
备注： 18 pages, 3 figures, 7 tables. Manuscript, LaTeX source, all 5,400 episode files, frozen specifications, timestamp proofs, analysis and generator scripts are archived at Zenodo: doi: [https://doi.org/10.5281/zenodo.22108558](https://doi.org/10.5281/zenodo.22108558)
点击查看摘要
Abstract:An agent that inherits a consolidated memory may inherit a constraint that was true when written and has since been withdrawn by a newer authoritative record. Under a scarce verification budget, does the agent recover the withdrawal, and if not, is the error avoidable without spending more? We model supersession explicitly -- historical provenance is immutable; what changes is which record is current -- and assign by design the memory's form, the world's state (source current or superseded), and the verification policy at a fixed budget of two records: the agent's own allocation, or the same budget with one slot re-assigned to the critical provenance path or to a random record. With a constraint stated, agents inspected its provenance path in about one episode in five; when that constraint had been superseded, native allocation produced stale-consistent decisions in 77.3%, 74.7% and 74.7% of episodes across a primary run, a fresh-wording replication and a held-out domain. Re-assigning one slot to the critical path raised current-record-consistent decisions by +74.0, +72.7 and +61.3 points, positive in six of six models in each of those runs, and changed nothing when the record agreed with the memory. The held-out scenario was later found to contain a temporal inconsistency; a robustness replication with one sentence corrected, deposited externally before execution, gave +73.3 points and is reported alongside the original. The intervention uses knowledge of the critical path and is not a scheduler; it identifies that the share of stale-memory error attributable to verification allocation is close to its structural ceiling. Memory systems may need freshness or supersession signals separate from relevance.
11. 【2608.25546】An Event is Worth One Token: Event Tokenization for Industrial-scale LLM Recommendation
链接：https://arxiv.org/abs/2608.25546
作者：Fan Xia,Zhaoheng Zheng,Iman Setayesh,Ruogu Lin,Yiqin Pan,Samarth Mittal,Wentao Bao,Vinti Pandey,Sachin Patil,Jianpeng Cheng,Jun Xiao,Zhuang Wang,Xiangjun Fan,Sri Reddy,Minghai Chen
类目：Information Retrieval (cs.IR)
关键词：discarding rich user, semantic IDs, encodes only text, discarding rich, rich user
备注： 11 pages, 10 figures, 7 tables
点击查看摘要
Abstract:LLM-based recommendation has scaled along model capacity and sequence length, yet each position encodes only text, semantic IDs, or a few categorical features, discarding rich user, item, context, and outcome signals available at each event. Under autoregressive modeling, this yields weak queries at each position and, since each position becomes context for the next, the degradation compounds across the sequence. We propose an event-centric paradigm that represents each interaction by its full temporal snapshot, and identify a new scaling dimension we term snapshot resolution: the amount of information encoded per event. To efficiently scale snapshot resolution, we introduce AMBER (Autoregressive Modeling via Bottlenecked Event Representation), which compresses each temporal snapshot into a compact Event Token, a new LLM input modality. The representation is learned end-to-end, while Event Tokens are pre-computed and cached for serving, decoupling snapshot resolution from real-time serving compute. On industrial-scale ranking and retrieval benchmarks, AMBER advances the compute-quality Pareto frontier relative to alternative recommendation paradigms. At sufficient capacity, a single unified tokenizer even outperforms dedicated per-entity tokenizers, demonstrating positive transfer across structurally different entity types. AMBER's Event Tokens also transfer across model architectures: when integrated into a heavily optimized non-LLM ranker as serving-time historical features, they yield statistically significant improvements. Further scaling Event Tokenizer capacity provides additional improvements.
12. 【2608.25528】ransRetrieval: Scaling Up Transformer-Based Retrieval for Industrial Recommendation
链接：https://arxiv.org/abs/2608.25528
作者：Zhifei Zheng,Yunfei Liu,Bin Liu,Qiren Zhu,Hanbing Liu,Ziru Xu,Han Zhu,Jian Xu,Qi Qi,Bo Zheng
类目：Information Retrieval (cs.IR)
关键词：severe token-norm divergence, naively stacking Transformer, heterogeneous fields produce, fields produce severe, produce severe token-norm
备注： Accepted at the 35th ACM International Conference on Information and Knowledge Management (CIKM 2026)
点击查看摘要
Abstract:Applying scaling laws to recommendation retrieval is hindered by feature heterogeneity: naively stacking Transformer layers yields diminishing returns because heterogeneous fields produce severe token-norm divergence. We present TransRetrieval, a Transformer-based retrieval framework that scales with both computational budget and cross-domain data. The key enabler is (1) weighted average aggregation, which restores the homogeneous-token assumption Transformers rely on. Building on this, we introduce (2) target token compression that cuts per-candidate FLOPs by 85% while preserving cross-attention expressiveness, and (3) position-style domain embeddings that unify multiple domains at negligible additional cost, turning cross-domain data into a scaling asset. On a 40-billion-interaction industrial dataset and the public KuaiRand benchmark, scaling compute from 0.1 to 2 MFLOPs per target yields +19.3/+22.2 pt Recall@2000, confirming robust log-linear scaling. In online A/B tests, TransRetrieval lifts platform revenue by 2.53% under the same end-to-end latency constraint as the production baseline.
13. 【2608.25521】Query Expansion Is More Than Generation: Improving Dense Retrieval through Better Integration
链接：https://arxiv.org/abs/2608.25521
作者：Siyuan Sun,Mihai Surdeanu
类目：Information Retrieval (cs.IR)
关键词：Large language models, Large language, dense retriever worse, frozen dense retriever, language models
备注：
点击查看摘要
Abstract:Large language models (LLMs) can generate query expansions without task-specific training, yet the same expansions often make a frozen dense retriever worse. We identify an underexplored factor: prior work has often focused on what text is generated, while how generated text is incorporated into dense retrievers has received less systematic attention. By holding generated expansions fixed, we show that performance degradation can often be attributed to the integration method itself. We introduce AnchorQE, a training-free method that separately encodes the original query and its expansion before interpolating them. The interpolation factor is estimated using an unsupervised online strategy that operates over a small part of the unlabeled test stream. Intuitively, our strategy assigns high expansion trust only when expansions are both retrieval-strong and consistent with the original query's retrieved evidence. We show that AnchorQE improves retrieval effectiveness by up to 12.89% when compared to widely-used expansion-only or text-level concatenation baselines across TREC-DL, LoTTE, and BEIR. Further, we show that our online strategy to estimate the interpolation factor outperforms a fixed weight tuned on a development partition by up to 3.81%.
14. 【2608.25489】A Storage-Retrieval Gap in Parametric Knowledge Graph Memory
链接：https://arxiv.org/abs/2608.25489
作者：Martino M. L. Pulici,Cuong Xuan Chu,Evgeny Kharlamov,Volker Tresp
类目：Machine Learning (cs.LG); Computation and Language (cs.CL); Information Retrieval (cs.IR)
关键词：exposing source data, recurring token cost, generation places retrieved, model context window, places retrieved subgraphs
备注： 12 pages, 2 figures, 7 tables, accepted at SKGi 2026
点击查看摘要
Abstract:Graph retrieval-augmented generation places retrieved subgraphs into the model's context window at query time, paying a recurring token cost and exposing source data on every call. We study an alternative: compiling a knowledge graph offline into a bank of LoRA adapters, one per entity, that serve as a parametric knowledge layer queried by injecting weights rather than text, at zero query-time context cost. On the MetaQA dataset, we find that subgraph-trained adapters encode context-free factual knowledge that generalizes to unseen questions: on single-valued relations the adapter gains $+0.243$ exact-match score over a base model that is nearly blind closed-book ($0.007$), and only the correct adapter recovers this knowledge (an oracle gap of $+0.283$ over the base model). However, the stored knowledge is not recoverable by similarity: given a query with no subgraph, embedding-based and weight-space geometry retrieval both perform at chance, because a semantically neighbouring entity's adapter does not contain the answer - knowledge is stored locally and does not transfer. Weight geometry correlates with subgraph semantics ($\rho = +0.329$) but not with functional retrievability. We quantify the byte and context-token costs against graph retrieval-augmented generation and discuss deployment implications. Our results establish that parametric knowledge graph memory is feasible for storing knowledge, and identify selecting and composing the right adapters by a mechanism other than semantic similarity as the central open problem - motivating a learned, query-conditioned composition mechanism.
15. 【2608.25487】ReliableRAG: Combating Misinformation in Retrieval-Augmented Generation via Reliability-Guided Reasoning Chains
链接：https://arxiv.org/abs/2608.25487
作者：Jinpu Jiang,Xuan Wu,Wenhao Song,Bo Yang,You Zhou,Hongwei Ge,Heow Pueh Lee,Yanchun Liang,Chunguo Wu
类目：Computation and Language (cs.CL); Information Retrieval (cs.IR)
关键词：Large Language Models, Language Models, Large Language, Retrieval-Augmented Generation, Question Answering
备注：
点击查看摘要
Abstract:Retrieval-Augmented Generation (RAG) has emerged as a powerful architecture for Question Answering (QA) by integrating external information into Large Language Models (LLMs). However, false, inaccurate, and misleading information in news and social media poses a serious challenge to real-world RAG systems, especially in multi-hop QA, where complex multi-step reasoning can be misled by even a single deceptive misinformation segment in the retrieved documents. Existing approaches mainly rely on implicit alignment or explicit regulation, but their limited ability to assess fine-grained information reliability makes them vulnerable to deceptive misinformation that is semantically relevant to the question yet factually incorrect, leading to erroneous answers. To address this limitation, we propose ReliableRAG, which, to the best of our knowledge, is the first reliability-driven framework that mitigates deceptive misinformation in multi-hop QA through fine-grained evaluation of individual triples. ReliableRAG first extracts information segments from source documents and represents them as structured triples. It then quantifies triple reliability by combining query-triple semantic relevance with triple credibility, retaining only the top-$K$ reliable and non-redundant triples. Based on these refined triples, ReliableRAG autoregressively constructs robust reasoning chains to consolidate trustworthy evidence and filter deceptive misinformation, producing accurate answers faithful to reliable information. Experiments on three multi-hop QA datasets show that ReliableRAG outperforms existing methods, substantially improving the factual reliability and robustness of RAG systems under deceptive misinformation injection.
16. 【2608.25434】DocPC: Document-Level Visual Retrieval via Representative Page Composition
链接：https://arxiv.org/abs/2608.25434
作者：Chengsong You,Junwei Zhou,Nan Du
类目：Information Retrieval (cs.IR)
关键词：bypassing OCR pipelines, bypassing OCR, OCR pipelines, encoding page screenshots, vision-language models
备注： 15 pages, 5 figures, 8 tables
点击查看摘要
Abstract:Visual document retrieval has advanced by encoding page screenshots with vision-language models, bypassing OCR pipelines. However, existing methods remain page-centric, misaligned with real-world scenarios requiring complete document retrieval. A naive page-then-document aggregation suffers from linear indexing cost and degraded retrieval when relevance spans multiple pages. We propose DocPC, a document-level visual retrieval framework based on Representative Page Composition: selecting representative pages and composing them into a single grid image for document-level indexing, reducing indexed images, vectors, and storage by 10.1x and end-to-end indexing time by roughly 7.7x. To handle multi-positive supervision prevalent at the document level, we combine multi-positive contrastive learning with sparsely scheduled listwise optimization. We also introduce DocViRe, a benchmark with multi-positive relevance annotations. DocPC-ColQwen achieves NDCG@5 of 44.09 on DocViRe, outperforming the strongest page-level baseline at 38.91 while reducing storage by 10.1x. Code is available at this https URL. Data is available at this https URL.
17. 【2608.25382】QA or Document-Based? The Effects of Interface Type on How Screen Reader Users Access Interconnected Documents
链接：https://arxiv.org/abs/2608.25382
作者：Colleen F. Cipriano,Yichun Zhao,Miguel A. Nacenta,Kotaro Hara,Jaylee Soh
类目：Human-Computer Interaction (cs.HC); Artificial Intelligence (cs.AI); Information Retrieval (cs.IR)
关键词：Blind and low-vision, build interconnected knowledge, large language model, increasingly engaging, engaging with large
备注： 17 pages, 12 figures, accepted at ASSETS 2026
点击查看摘要
Abstract:Blind and low-vision (BLV) users are increasingly engaging with large language model (LLM) interfaces to access documents, but it is unclear how such systems support or hinder their ability to build interconnected knowledge. To examine this gap, we compared a Question-Answer Interface (QAI) that supports open-ended conversational inquiry, with a Document Interface (DI) based mostly on traditional structured text document navigation. We recruited 16 BLV screen reader users where they used both interfaces to explore two fictional worlds. Data from interaction logs, concept maps, decision-based tasks, and semi-structured interviews provide comparative insights into how interface design supports knowledge construction. Findings show that participants visited more distinct documents with the DI and formed larger and more correct mental models with the DI than with the QAI. They were also more able to apply knowledge they had gained. Simultaneously, many still preferred the QAI and often estimated that they had explored more, formed better mental models and applied their models better when acquiring the information with the QAI, despite this not being the case. Our analysis suggests possible interface design reasons for these differences and highlights some of the risks introduced by using question-answer interfaces to access information spaces.
18. 【2608.25381】MOTIF: Motivation-guided Topology Inference for Cold-start Multimodal Recommendation
链接：https://arxiv.org/abs/2608.25381
作者：Yurui Shi,Yuchen Miao,Ximing Hu,Zijun Wang,Chang Han
类目：Information Retrieval (cs.IR)
关键词：sparse interactions obscure, obscure user intent, remain topologically isolated, cold items remain, interactions obscure user
备注： 15 pages, 3 figures, 7 tables. Accepted at WISE 2026
点击查看摘要
Abstract:Cold-start multimodal recommendation faces three coupled challenges: (i) sparse interactions obscure user intent, (ii) cold items remain topologically isolated, and (iii) similarity-based item graphs may cause semantic drift. To address these issues, we propose MOTIF, a Motivation-guided Topology Inference framework for cold-start multimodal recommendation. MOTIF integrates Semantic Motivation Reasoning, Knowledge-enhanced Graph Reconstruction, Weighted Graph Contrastive Learning, and Semantic-Structural Alignment. It uses offline LLM reasoning to infer motivation semantics, reconstructs transferable item-item topology, and learns robust graph embeddings without injecting generated text into prediction. Experiments on three multimodal benchmarks show consistent gains over graph-based, multimodal, cold-start, and LLM-enhanced baselines, with up to 6.07% relative improvement over the strongest recent baseline.
19. 【2608.25370】CRAMER: Control via Request-Aware Masking for Editing Recommenders
链接：https://arxiv.org/abs/2608.25370
作者：Zhiyuan Julian Su,Naihe Feng,Zhen Luther Qin,Ga Wu
类目：Information Retrieval (cs.IR); Artificial Intelligence (cs.AI); Machine Learning (cs.LG)
关键词：user timely interests, making it difficult, timely interests, limited flexibility, flexibility in responding
备注： Accepted by ICML 2026
点击查看摘要
Abstract:Sequential recommendation models, while powerful, have limited flexibility in responding to immediate user requests, making it difficult to adapt their recommendations to the user's timely interests. Unfortunately, existing user request adaptation methods often incur high computational overhead due to either 1) retraining the entire backbone network or 2) leveraging the inference ability of large language models (a.k.a. prompt engineering), limiting their applicability in large-scale recommendation services. This paper presents Control via Request-Aware Masking for Editing Recommenders (CRAMER), a framework that takes users' natural-language requests to immediately change sequential recommendation models' behavior. Specifically, inspired by the model control theory, CRAMER treats user requests as control signals to modulate frozen backbone parameters through masking, achieving instant adaptation to diverse requests while avoiding costly retraining. Experiments on multiple large-scale benchmark datasets show that CRAMER outperforms four state-of-the-art request-aware baselines across multiple recommendation metrics while achieving minimal overhead. Moreover, the proposed framework exhibits enhanced controllability and cross-domain adaptability, establishing a new paradigm for request-aware sequential recommendation.
20. 【2608.25318】Rank-Deviation Quality: A Distance-Aware Metric for Multi-Answer Retrieval and Ranking Evaluation
链接：https://arxiv.org/abs/2608.25318
作者：Xiaokun Zhou,Alessandro Moschitti,Danielle Class
类目：Information Retrieval (cs.IR)
关键词：introduce Rank-Deviation Quality, single correct answer, Rank-Deviation Quality, varying numbers, single correct
备注：
点击查看摘要
Abstract:We introduce Rank-Deviation Quality (RDQ), an evaluation metric for retrieval and ranking systems that adapts to queries with varying numbers of reference items, from a single correct answer to many valid results. RDQ scores a candidate ranking against an ordered reference list (ORL): each retrieved reference item contributes its output-position weight multiplied by a rank-deviation penalty, and items outside the ORL receive zero credit. Application-specific parameters control tolerance to misordering. Larger values emphasize retrieving valid reference items, whereas smaller values place more weight on matching their reference order. The output-position weights can reflect visibility in the application's interface, such as a vertical list or a carousel. Unlike metrics that require absolute relevance grades, RDQ operates on ordinal rankings, which annotators can produce through pairwise or listwise judgments. Unlike rank-correlation measures such as Kendall's tau, RDQ accounts for both which items are returned and how they are ordered. On a 5,000-query point-of-interest (POI) dataset with 12 systems, RDQ has the highest median empirical power@100 among the 13 evaluated metric configurations. It reaches mean tau = 0.8 agreement with its own full-query ordering at 200 queries; RBP(0.9), the strongest tested non-RDQ configuration, reaches the same threshold at 250. On TREC Deep Learning benchmarks, where NDCG uses native graded labels and RDQ uses ordinal tiers derived from them, RDQ reaches comparable median power at n=25, while NDCG is higher at n=100.
21. 【2608.25245】he "Curse of Knowledge" in LLM Query Simulation: Concept Provenance for Tracing Answer-Side Intrusion
链接：https://arxiv.org/abs/2608.25245
作者：Chenglong Ma,Xinye Wanyan,Danula Hettiachchi,Ziqi Xu,Jeffrey Chan
类目：Information Retrieval (cs.IR); Computation and Language (cs.CL)
关键词：presuppose answer-side document, LLM-generated search queries, LLM-generated search, violating the information-access, pre-search users
备注： 12 pages, 4 figures, and 2 tables. To appear in the Proceedings of the 35th ACM International Conference on Information and Knowledge Management (CIKM '26)
点击查看摘要
Abstract:LLM-generated search queries are widely used to augment IR evaluation, yet they may contain concepts that presuppose answer-side document knowledge, violating the information-access boundary of pre-search users. Existing validation metrics, including overlap, diversity, and effectiveness, cannot distinguish rare human-tail variation from candidate answer-side intrusion. We introduce concept provenance, a framework that assigns query concepts to backstory-supported, human-central, human-tail, and candidate answer-side zones, operationalizing a boundary that retrieval metrics alone cannot detect. Applying concept provenance to 77,004 queries across 100 UQV100 topics, 8 LLMs, and 5 prompt conditions with two extraction pipelines, we obtain a cross-pipeline token-HCIR Spearman rho of 1.0 over five condition means. Candidate answer-side concepts constitute 7.40 percent of non-generic concepts and appear in 97 of 100 topics, with topic explaining approximately 67 percent of variance. Human validation yields 68.2 percent relaxed precision, revealing two mechanisms: knowledge intrusion at 45.5 percent and deployment intrusion at 45.0 percent. Diagnostic probes show disproportionate localized retrieval effects, with deletion effect size d = -0.47 compared with d = -0.34 for random deletion, but these concepts explain less than 2 percent of aggregate evaluation variance. Concept provenance therefore serves as a boundary-compliance diagnostic rather than an evaluation-shift predictor. Under the tested conditions, no prompt condition eliminates intrusion; post-generation concept-provenance selection achieves 99 percent elimination.
22. 【2608.25115】Less can be More: Relieving RAG Bottlenecks via Evidence Frontloading and Pressure-Adaptive Budgeting
链接：https://arxiv.org/abs/2608.25115
作者：Weibin Cai,Reza Zafarani
类目：Computation and Language (cs.CL); Information Retrieval (cs.IR)
关键词：improving Retrieval-Augmented Generation, Existing methods, optimize downstream LLM, downstream LLM generation, PACE
备注：
点击查看摘要
Abstract:Existing methods for improving Retrieval-Augmented Generation (RAG) efficiency mainly optimize downstream LLM generation, such as context compression or serving optimization. However, RAG is an end-to-end system, and its bottleneck can shift between upstream reranking and downstream generation under different serving loads and reranking this http URL this paper, we first empirically characterize this shifting-bottleneck behavior and show that upstream reranking can become the dominant bottleneck under high query rates or large reranking budgets. Reducing the reranking budget can relieve this bottleneck, but it may also drop supporting evidence and degrade recall. To address this problem, we propose \textbf{\textsf{PACE}} (\textbf{P}rioritized \textbf{A}daptive \textbf{C}overage of \textbf{E}vidence), a training-free framework that combines \textit{evidence frontloading} with \textit{pressure-adaptive budgeting}. \textsf{PACE} first reorders candidates by marginal evidence coverage, prioritizing documents that are query-relevant, complementary, and useful for forming multi-hop evidence chains. We show that this objective is monotone submodular, giving greedy selection a $(1-1/e)$ approximation guarantee. \textsf{PACE} then dynamically adjusts the reranking budget according to the relative pressure of the reranker and the LLM. Experiments on three multi-hop QA datasets and online serving simulations show that \textsf{PACE} improves evidence recall, reduces p95 latency under ranking-heavy workloads. More importantly, the two components together reveal that \textit{less can be more}: an evidence-dense top-ranked candidates enable higher final recall with fewer reranked documents.
23. 【2608.25104】SWIM: Step-Wise Integrated Measure for Session-supervised List Evaluation in Generative Re-ranking
链接：https://arxiv.org/abs/2608.25104
作者：Yuanhao Pu,Chenghao Zhang,Chao Feng,Xunyong Yang,Xiang Li,Yongqi Liu,Defu Lian,Kaiqiao Zhan,Kun Gai
类目：Information Retrieval (cs.IR)
关键词：Modern industrial recommender, industrial recommender systems, adopted the Generator-Evaluator, re-ranking stage, Modern industrial
备注： 12 pages, 2 figures
点击查看摘要
Abstract:Modern industrial recommender systems have increasingly adopted the Generator-Evaluator (G-E) framework for the re-ranking stage. Within this paradigm, the generator produces candidate item lists from a pool filtered by upstream retrieval and ranking modules, while the evaluator scores these lists and selects the highest-scoring one for final exposure per request. However, on sequential platforms (e.g., short-video apps), users consume items continuously, ignoring artificial list boundaries. Conventional evaluators score lists by aggregating point-wise values, implicitly assuming exposure independence. This fails to capture critical session-level dynamics, such as contextual dependencies, user continuation, and diminishing marginal utility from repetitive content. To bridge this gap, we propose SWIM (Step-Wise Integrated Measure), a list-level evaluator that models user behaviors as a finite-horizon prefix session-level survival process. SWIM estimates the prefix-conditioned contribution of the current list to the session-level objective by factorizing it into a recursive survival distribution and reached-position conditional rewards. Leveraging a causally-masked Transformer, SWIM efficiently estimates continuation probabilities and utilities in parallel, satisfying strict industrial latency constraints. Extensive experiments demonstrate that SWIM significantly outperforms baselines in listwise reranking tasks, yielding substantial improvements in overall recommendation engagement.
Comments:
12 pages, 2 figures
Subjects:
Information Retrieval (cs.IR)
Cite as:
arXiv:2608.25104 [cs.IR]
(or
arXiv:2608.25104v1 [cs.IR] for this version)
https://doi.org/10.48550/arXiv.2608.25104
Focus to learn more
arXiv-issued DOI via DataCite (pending registration)
Related DOI:
https://doi.org/10.1145/3799682.3840732
Focus to learn more
DOI(s) linking to related resources</p>
24. 【2608.25037】Retrieve, Match, Escalate: Accurate and Scalable Product Linking with VLM-Distilled Cross-Encoders and Agentic VLMs
链接：https://arxiv.org/abs/2608.25037
作者：Jian Wang,Steven Xu,Sanjyot Thete,Maryam Barouti,Tom Tang,Elaine Wu,Charu Sareen,Kyle MacDonald
类目：Artificial Intelligence (cs.AI); Computation and Language (cs.CL); Databases (cs.DB); Information Retrieval (cs.IR)
关键词：consolidates fragmented listings, mapping merchant product, canonical catalog products, merchant product records, consolidates fragmented
备注： 9 pages, 3 figures, 5 tables
点击查看摘要
Abstract:Product linking, the entity-resolution task of mapping merchant product records to canonical catalog products, consolidates fragmented listings so downstream search, recommendation, and advertising see one clean entry per product. At marketplace scale, billions of noisy, multi-category records must be resolved against tens of millions of canonical products, where scoring every candidate with a single model is either too weak for the hard cases or too costly for the easy ones. We present a production retrieve-then-match cascade that spends computation in proportion to difficulty: retrieval surfaces plausible matches, a lightweight text cross-encoder auto-resolves the high-confidence majority, and an agentic multimodal vision-language model settles the ambiguous remainder by inspecting product images and issuing web searches for evidence that is in neither record. The cross-encoder is distilled from millions of dual-VLM-consensus labels, retiring human annotation from the training set, and is calibrated to auto-accept links at a 98% precision bar validated against a smaller operator-certified audit. The agent is a self-hosted open-weight model that reaches a closed frontier VLM's precision at a four-point recall cost (88% versus 92%) for roughly one-seventh the per-pair cost, with no fine-tuning. Per-pair cost spans nearly five orders of magnitude from the cheap cross-encoder to the frontier VLM, so escalating only the hard tail to the agent raises end-to-end link coverage from the cheap stage's 68% to 77%.
25. 【2608.24926】A Pathway for Assessing Grey Literature: Leveraging AI to Extract Conference Metadata and Organiser Information from Calls for Papers
链接：https://arxiv.org/abs/2608.24926
作者：Angelo Salatino,Francesco Osborne,Alexis Vizcaino,Aliaksandr Birukou,Enrico Motta
类目：Digital Libraries (cs.DL); Computers and Society (cs.CY); Information Retrieval (cs.IR)
关键词：highly heterogeneous format, remains largely overlooked, Metascience and Scientometric, Scientometric analysis due, traditional tools struggle
备注： Paper accepted at STI-ENID 2026 [this https URL](https://www.uantwerpen.be/en/conferences/30th-annual-international-conference-on-science-and-technology-indicators/)
点击查看摘要
Abstract:Despite its importance, grey literature, including Calls for Papers (CfPs), remains largely overlooked in Metascience and Scientometric analysis due to its unstructured, highly heterogeneous format, which traditional tools struggle to process at scale. However, Large Language Models now offer a pivotal opportunity to devise innovative tools for systematically harvesting and processing such data. In this paper, we introduce COCI, an AI-based framework that automates the extraction of granular, structured metadata from raw CfP text. COCI employs a multi-stage pipeline for entity extraction, followed by author disambiguation against OpenAlex and semantic mapping of topics and conference series. This process identifies key data points, including conference editions, geographic locations, and comprehensive lists of organisers, along with their specific roles and affiliations. By structuring this previously inaccessible information, COCI establishes a foundation for the systematic analysis of grey literature, enabling new research opportunities and shifting the scholarly focus towards non-publisher-based events.
26. 【2608.17889】VisDocAgentBench: Benchmarking Agents for Visually Rich Document Retrieval
链接：https://arxiv.org/abs/2608.17889
作者：Lexiang Hu,Yanzhao Zhang,Mingxin Li,Dingkun Long,Yikang Li,Fuwei Zhang,Yisen Wang,Zhouchen Lin
类目：Information Retrieval (cs.IR); Artificial Intelligence (cs.AI); Computer Vision and Pattern Recognition (cs.CV)
关键词：Visually rich documents, Visually rich, rich documents encode, structured visual elements, documents encode relevance
备注：
点击查看摘要
Abstract:Visually rich documents encode relevance through language, layout, structured visual elements, and corpus context, yet retrieval is typically evaluated by one-shot query--page matching. Agentic-search benchmarks usually score downstream question answering or report generation, leaving document ranking under iterative evidence acquisition underexplored. We introduce VisDocAgentBench, a closed-corpus benchmark comparing static and agentic retrieval under a shared ranked-output contract. It contains 2,375 pages from 100 documents and 120 unique-target queries balanced across direct, one-bridge, and two-bridge evidence structures. Relation-preserving construction yields semantic, relational, and visual queries, followed by full-document review and hard-negative validation. A strong late-interaction visual retriever reaches 97.50% Recall@1 on direct items but 2.50% on two-bridge items, exposing the limits of query--target matching when relevance depends on corpus context. Agents recover much of this loss, but planner choice and retrieval representation remain decisive. Every planner performs better with visual retrieval, whose best R@1 reaches 67.50% versus 37.50% for OCR-text. Ablations identify iterative search and page inspection as consequential capabilities, and providing the complete support context improves ranking on both routes. Trace analysis localizes the remaining losses to target discovery, candidate examination, and evidence-role integration. These findings motivate retrieval agents that combine modality-preserving discovery with evidence-directed verification.
27. 【2608.26005】VoiceMem: Streaming Dual-Brain Memory for Real-Time Interaction
链接：https://arxiv.org/abs/2608.26005
作者：Zhifei Xie,Jiaqi Lang,Ze An,Yifan Zhao,Dongchao Yang,Kai Li,Ziyang Ma,Mingbao Lin,Chunyan Miao,Shuicheng Yan
类目：Audio and Speech Processing (eess.AS); Artificial Intelligence (cs.AI); Information Retrieval (cs.IR); Multimedia (cs.MM); Sound (cs.SD)
关键词：speech language models, duplex speech language, language models, empathetic memory system, empathetic memory
备注： 18 pages, 9 figures, 6 tables
点击查看摘要
Abstract:Conversational systems, such as duplex speech language models (SLMs), still lack a streaming, accurate, and empathetic memory system as their soul. We introduce VoiceMem, a simple memory architecture with a parallel informational left brain, an emotional right brain, and streaming memory I/O mechanisms. We further build a complete pipeline for memory-aware SLM training, long-horizon evaluation, and decoupled deployment with interchangeable memory backends. Experiments and real-world deployment show three advantages: i) Accuracy: under top-5 retrieval, the left brain outperforms classical systems such as Mem0 at top-200 by nearly 30 points; ii) Emotional Personal: the right brain, with short- and long-horizon affective attribution and dual-node persona modeling, achieves state-of-the-art performance across three persona benchmarks and improves the aggregate score by 4.29 points over the previous best system; and iii) Real-Time Cheap: VoiceMem completes retrieval in 134 ms, well within standard VAD latency, adding no extra conversational delay while maintaining high accuracy and low cost. These results show that VoiceMem provides a practical memory foundation for real-time, personalized, and emotionally aware speech interaction.
计算机视觉
1. 【2608.26105】VBVR-Pro: A Scalable and Verifiable Suite for Native Visual Reasoning
链接：https://arxiv.org/abs/2608.26105
作者：Junxiang Xu,Ruisi Wang,Fanyi Pu,Maijunxian Wang,Ran Ji,Tongxi Zhou,Chenyang Gu,Jing Zuo,Hongcan Xiao,Yimeng Geng,Wanqi Yin,Wei Chen,Oscar Qian,Zhengan Yan,Ziqi Huang,Haiwen Diao,Liang Pan,Bo Li,Xiangyu Fan,Dezhi Luo,Fengyuan Yu,Zehong Zhao,Qingying Gao,Tinghui Zhu,Yilan Zhang,Jingqi Tong,Pinyuan Feng,Zhengze Jiang,Letian Wang,Ziyu Guo,Renrui Zhang,Jieneng Chen,Sonia Joseph,Constantin Venhoff,Saman Motamed,Mengyue Yang,Chandra Sripada,Alan Yuille,Philip Torr,Lvmin Zhang,Vikash Kumar,Daniel Khashabi,Nikolaus Kriegeskorte,Raphaël Millière,Vincent C. Müller,Anyi Rao,Quan Wang,Ziwei Liu,Dahua Lin,Lei Yang,Hokin Deng,Zhongang Cai
类目：Computer Vision and Pattern Recognition (cs.CV); Artificial Intelligence (cs.AI); Machine Learning (cs.LG); Multimedia (cs.MM); Robotics (cs.RO)
关键词：Native visual reasoning, solving beyond language, visual reasoning, understood or outputs, problem solving
备注： Homepage: [this https URL](https://video-reason.com/)
点击查看摘要
Abstract:Native visual reasoning treats visual generation as the medium of reasoning itself: visual states (i.e. images and videos) are not merely inputs to be understood or outputs to be rendered, but first-class substrates for problem solving beyond language. Yet progress remains bottlenecked by the lack of scalable training tasks, reliable feedback, and controlled comparisons across generative substrates. In this work, we introduce VBVR-Pro, a closed-loop testbed that makes native visual reasoning through generation trainable, verifiable, optimizable, and experimentally controllable. 1) Task scaling. VBVR-Pro turns visual reasoning into a controlled task space of 300 procedurally generated tasks. Models trained on VBVR-Pro show strong transfer beyond the proposed suite across seven external visual reasoning benchmarks such as RISE-Video, MME-CoF-Pro, and BabyVision. 2) Verifiable rewards. VBVR-Pro provides verifiable reward scorers for task-grounded evaluation. Through a systematic study of leading MLLMs as judges, we identify recurring failure modes of the prevalent VLM-as-a-judge paradigm. In contrast, the proposed scorers are grounded in deterministic, task-specific rules, achieve fine-grained alignment with human judgments. Importantly, they serve as reliable reward signals for large-scale multi-task reinforcement learning and demonstrate stronger post-RL performance across visual reasoning tasks. 3) Mechanism study. VBVR-Pro enables controlled modality studies across more than 30 image, video, and interleaved generators. Our analysis shows that video generation remains strongest for tasks requiring persistent spatiotemporal state tracking, while interleaved generation provides a compute-efficient alternative. Critically, ablations and probing suggest the presence of vision-native trajectories that are crucial to visual reasoning. We release all data, models, scorers, and code.
2. 【2608.26103】Zero-WAM: In-Context World-Action Modeling from Human Videos for Open-Ended Task Generalization
链接：https://arxiv.org/abs/2608.26103
作者：Jiaming Zhou,Qihang Zhang,Gangwei Xu,Cunxin Fan,Yujie Zhao,Ruilin Wang,Yiming Luo,Shuai Yang,Xing Zhu,Yujun Shen,Junwei Liang,Yinghao Xu
类目：Robotics (cs.RO); Computer Vision and Pattern Recognition (cs.CV)
关键词：Zero-shot cross-task generalization, Zero-shot cross-task, remains a central, central challenge, Zero-shot
备注： [this https URL](https://robbyant-research.github.io/Zero-WAM/)
点击查看摘要
Abstract:Zero-shot cross-task generalization, where a policy must execute manipulation tasks never seen during training, remains a central challenge in robot learning. In large language models, a novel task can be performed simply by specifying it in the context, without any parameter update. This form of in-context learning (ICL) turns generalization into a problem of task specification. To achieve cross-task generalization, we bring this paradigm to robotic manipulation, and argue that the natural task specification for manipulation is a human video: unlike language, it provides rich visual cues about the intended task evolution. We present Zero-WAM, a causal video-action model that executes unseen tasks by following in-context human video guidance. To address the scarcity of task-rich paired human-robot data, we propose an automatic pipeline that converts task-sampled robot trajectories into semantically matched human videos, yielding HumanGen, a dataset of 74.2K human-robot ICL pairs across 8.6K tasks. For model training, we further introduce an in-context future chunk prediction (IFP) objective that suppresses shortcuts learned from seen tasks and forces the policy to draw task information from the video prompt. On seven unseen tasks in RoboTwin 2.0 simulation, Zero-WAM achieves a 47.0% average success rate, an absolute improvement of 29.5 percentage points over the strongest video-action baseline. In real-world evaluations, it follows human video guidance to generalize to unseen task configurations involving multi-object scenes, long-horizon manipulation, and fine-grained insertion.
3. 【2608.26101】RefVideo-6M: A Reliable Reference-Based Dataset for Instructional Video Editing
链接：https://arxiv.org/abs/2608.26101
作者：Bojia Zi,Xiaoyan Yang,Yu Zhou,Ruijie Sun,Lihan Zhang,Bin Liang,Kam-Fai Wong,Haibin Huang,Chi Zhang,Xuelong Li
类目：Computer Vision and Pattern Recognition (cs.CV)
关键词：Recent advances, editing, largely driven, Recent, large-scale instruction-based datasets
备注：
点击查看摘要
Abstract:Recent advances in video editing have been largely driven by large-scale instruction-based datasets. However, existing datasets still suffer from two critical limitations. First, target videos are commonly produced by automatic editing models, which may introduce visible artifacts and unreliable supervision signals. Second, most public datasets rely primarily on textual instructions, while lacking visual references that are crucial for precise, identity-preserving, and controllable editing. To address these limitations, we introduce RefVideo-6M, a large-scale reference-guided editing dataset containing 5 million video editing samples and 1 million image editing samples. To ensure reliable supervision, our dataset uses a construction pipeline that treats artifact-free real videos as editing targets and generates quality-filtered input conditions with multiple editing experts. In addition, it provides approximately 6 million visual references, covering diverse reference types and editing scenarios, thereby enabling models to learn fine-grained visual correspondence beyond text-only instructions. Based on RefVideo-6M, we further train a reference-guided video editing model, Ref-MoT, to evaluate the effectiveness and scalability of the proposed dataset. Extensive experiments demonstrate that RefVideo-6M provides substantially more reliable supervision than existing datasets and enables the training of powerful editing models with improved visual quality, controllability, and reference consistency. The open-source dataset is available at this https URL.
4. 【2608.26095】A Visual Dependence-Aware Framework for Multimodal Unsupervised Continual Post-Training
链接：https://arxiv.org/abs/2608.26095
作者：Kaichen Li,Zhilin Zhu,Jianhao Huang,Zhengqin Lai,Baochen Xiong,Zibo Shao,Yaguang Song,Linhui Xiao,Xiaoshan Yang,Changsheng Xu
类目：Computer Vision and Pattern Recognition (cs.CV); Artificial Intelligence (cs.AI)
关键词：Multimodal Unsupervised Continual, Unsupervised Continual Post-Training, streaming unlabeled data, enabling deployed MLLMs, task of Multimodal
备注：
点击查看摘要
Abstract:In this paper, we explore a novel task of Multimodal Unsupervised Continual Post-Training (MU-CPT), enabling deployed MLLMs to continually evolve from streaming unlabeled data. Existing unsupervised post-training methods for MLLMs typically optimize target tokens uniformly, overlooking their heterogeneous visual dependence (VD). However, we reveal that token-level VD is crucial for MU-CPT. Specifically, its structural distortion serves as an indicator of cross-modal catastrophic forgetting, and its inherent heterogeneity acts as a compass to guide new-task learning. Leveraging this property, we propose a Visual Dependence-Aware (VDA) framework with two main components. First, Visually Constrained Optimal Transport (VC-OT) formulates the VD structural distortion of old-task VD during new-task learning as an optimal transport problem to mitigate cross-modal forgetting. By designing a region-aware ground cost and a dependence-stratified transport penalty, it prevents global shifts in visual focus while strictly prohibiting visual reliance from degenerating into language bias. Second, Visually Modulated Adaptation (VMA) exploits VD heterogeneity to emphasize visually grounded new-task learning, promoting new-task plasticity. Together, our method simultaneously maintains old-task stability and new-task plasticity during challenging MU-CPT. Extensive experiments under our MU-CPT setting validate the effectiveness of VDA.
5. 【2608.26094】MyoMechanix: Biomechanically-Grounded Compositional Skilled Activity Understanding and Coaching
链接：https://arxiv.org/abs/2608.26094
作者：Hao Yin,Paritosh Parmar,Lijun Gu,Lin Xu,Tianxiao Guo,Xiujin Liu,Tianyou Zheng,Yang Zhang,Weiwei Fu
类目：Computer Vision and Pattern Recognition (cs.CV); Artificial Intelligence (cs.AI); Emerging Technologies (cs.ET); Human-Computer Interaction (cs.HC); Machine Learning (cs.LG)
关键词：Existing action quality, methods rely primarily, overlooking physiological dynamics, Existing action, datasets and methods
备注：
点击查看摘要
Abstract:Existing action quality assessment (AQA) datasets and methods rely primarily on visual inputs such as RGB and pose, overlooking physiological dynamics such as muscle mechanics and often modeling actions as monolithic patterns. These limitations hinder fine-grained, biomechanically grounded feedback. We introduce MyoMechanix, a multimodal ecosystem for weight-loaded actions that aligns motion with muscle activity. Expert-annotated, it contains 7,500+ samples of 20 actions from 38 subjects, with synchronized multiview RGB video, 3D pose, sEMG, and additional physiological signals, forming the largest multimodal AQA benchmark to date. We further construct the Fitness Knowledge Graph (FKG), which organizes expert annotations into structured relationships among actions, phases, key steps, errors, and corrective feedback, enabling compositional scoring and interpretable assessment. Building on these representations, we develop CUBIST (Compositional Ontological Reasoning Engine), which performs decomposition-analysis-recomposition for fine-grained error attribution and feedback generation. We also establish MyoMechanix-AQA, MyoMechanix-VideoQA, and a novel MyoMechanix-Video2EMG task. Experiments show that multimodal sensing and structured representations improve performance, interpretability, and error attribution, with CUBIST achieving state-of-the-art results; VideoQA enhances language-grounded action understanding; and Video2EMG suggests video-based alternatives to costly EMG sensing. MyoMechanix advances skilled activity understanding toward biomechanically grounded, multimodal, and compositional reasoning for Physical AI applications in fitness, rehabilitation, healthcare, and machine learning. Project page: this https URL
6. 【2608.26091】PlanSightRAG: A Visual-First Multimodal RAG for Automating Question Answering and Compliance Checking for Civil Standard Plans
链接：https://arxiv.org/abs/2608.26091
作者：Nabaraj Subedi,Shuvo Dip Datta,Ahmed Abdelaty,Shivanand Venkanna Sheshappanavar
类目：Information Retrieval (cs.IR); Computation and Language (cs.CL); Computer Vision and Pattern Recognition (cs.CV)
关键词：OCR-based automation strips, manually reading legacy, engineers manually reading, Civil infrastructure compliance, Civil infrastructure
备注： 32 pages, 9 figures, 25 tables. Preprint submitted to Automation in Construction
点击查看摘要
Abstract:Civil infrastructure compliance checking has long relied on engineers manually reading legacy 2D plans; however, OCR-based automation strips away the geometry and layout essential for interpreting these plans. We present a Visual-First Multimodal Retrieval-Augmented Generation (RAG) framework called PlanSightRAG. It indexes and reasons directly over plan imagery, integrates a ColNomic-3B multi-vector retrieval, an agentic Planner-Retriever-Auditor-Synthesizer, and MaxSim heatmaps as an evidence trail. We introduce a 4,056-pair benchmark from five state Departments of Transportation (DOT) standard plans (1,898 pages). PlanSightRAG achieves 91.47% Recall@5 on zero-shot retrieval, while on a held-out Michigan DOT corpus, it achieves 91.40%. On synthetic, parametrically-generated compliance drawings, our Qwen2.5-VL-72B pipeline reaches 100% verdict accuracy only when supplied a pre-resolved rule threshold, a controlled ceiling that a non-VLM OCR baseline already reaches at 76.4%. Finally, we demonstrate autonomous visual rule-grounding by extracting numeric limits directly from a specification corpus without any human-supplied rules.
7. 【2608.26083】ICON Decomposition: Multivariate Concept-Level Explanations of Deep Representations for Model Auditing
链接：https://arxiv.org/abs/2608.26083
作者：Roshan Prakash Rane,Marco Simnacher,Manuel Pfeuffer,Marc-Andre Schulz,Nys Tjade Siegel,Maximilian Dreyer,Frederik Pahde,Wojciech Samek,Sonja Greven,Kerstin Ritter
类目：Machine Learning (cs.LG); Artificial Intelligence (cs.AI); Computer Vision and Pattern Recognition (cs.CV); Machine Learning (stat.ML)
关键词：Deep neural networks, exploit spurious associations, Deep neural, exploit spurious, spurious associations
备注： 44 pages, 12 figures, 3 tables. Includes Extended Data (7 figures, 2 tables). Code: [this https URL](https://github.com/RoshanRane/ICON_decomposition)
点击查看摘要
Abstract:Deep neural networks often exploit spurious associations in their training data, a failure known as shortcut learning. Concept-based explainability methods screen for shortcuts by testing whether concepts such as a patient's sex or scanner settings can be decoded from a network layer. Because each concept is evaluated in isolation, these methods can mistake correlations between concepts as evidence that the model uses them. We introduce ICON decomposition, which instead quantifies how much of a layer's variance each concept explains after accounting for all other concepts and the outcome. On synthetic data with known ground truth, ICON recovers concept importance more accurately than seven alternative baseline methods. On skin-lesion and brain-imaging models, it isolates the concepts on which a model genuinely relies, quantifies the representation unexplained by any of the supplied concepts, and yields sparse explanations that we validate by retraining and out-of-distribution testing.
8. 【2608.26067】StreamPI: Streaming Multimodal Temporal Modeling for Vision-Language-Action Models
链接：https://arxiv.org/abs/2608.26067
作者：Zhe Liu,Jinghua Hou,Yuxiang Lu,Zhenya Yang,Xianzhe Fan,Junwei Luo,Junyi Li,Ruihua Han,Zhi Hou,Hengshuang Zhao
类目：Computer Vision and Pattern Recognition (cs.CV)
关键词：retain past observations, develop precise spatial, robot manipulation, limiting their ability, demonstrated effectiveness
备注：
点击查看摘要
Abstract:Vision-Language-Action (VLA) models have demonstrated effectiveness in robot manipulation, yet state-of-the-art models such as pi0.5 operate under a single-frame paradigm, limiting their ability to retain past observations and develop precise spatial perception. In this paper, we propose StreamPI, a streaming multimodal temporal modeling framework that equips single-frame VLA with temporal reasoning capability without introducing any additional parameters. One core design is instruction-anchored temporal modeling. It treats each (visual observation, language instruction) pair as an atomic temporal unit: bidirectional attention within each pair enables cross-modal fusion, while causal attention across pairs preserves autoregressive streaming inference. This ensures the language instruction serves as a persistent semantic anchor throughout task execution. To bridge the gap between synchronous training and asynchronous real-robot deployment, we introduce a andom-interval streaming training strategy: a proper inter-frame interval (e.g., every 3 frames) enables faster and smoother action execution. Beyond this, randomizing the interval further improves robustness to frame-timing perturbations, supporting asynchronous deployment in practice. Furthermore, by leveraging the length extrapolation capability of the LLM backbone, StreamPI seamlessly inherits pretrained single-frame weights and supports flexible single-frame and multi-frame inference. Experiments on real-robot tasks spanning memory-dependent and precise perception scenarios, as well as the simulation benchmark LIBERO, demonstrate that StreamPI outperforms pi0.5 across diverse tasks.
9. 【2608.26033】UltraPIPS: Improving model perception in B-mode ultrasound with foundation models
链接：https://arxiv.org/abs/2608.26033
作者：Tal Grutman,Tali Ilovitsh
类目：Computer Vision and Pattern Recognition (cs.CV); Image and Video Processing (eess.IV)
关键词：feature space, LPIPS, semantically in feature, image patch similarity, learned perceptual image
备注： MICCAI ASMUS 2026
点击查看摘要
Abstract:In medical imaging, it is common to use learned perceptual image patch similarity (LPIPS) to compare images semantically in feature space. Although backbones pretrained on natural images are widely used for LPIPS computation, B-mode ultrasound images possess distinct speckle patterns and acoustic-specific image statistics that are fundamentally different from natural images and even from other images in radiology. Consequently, we propose that domain-specific models are needed to measure perceptual similarity in ultrasound data, a finding which is not necessarily the case for other imaging modalities. We compare LPIPS metrics across downstream tasks like classification, segmentation and reconstruction using natural image, medical generalist and ultrasound backbone models and show that selection of LPIPS backbone is a non-trivial design choice. In particular, the ultrasound backbone models were more correlated with downstream performance of supervised models than classical and natural image models, and optimization of the LPIPS loss with an ultrasound backbone achieved a strong balance between reconstruction quality and realism. Our code is available at this https URL and introduces the UltraPIPS library, a set of LPIPS metrics based on the open-source foundation models analyzed in this paper.
10. 【2608.25998】Uncertainty-Guided Latent Diffusion Models for Faithful Super Resolution
链接：https://arxiv.org/abs/2608.25998
作者：Ren Wang,Yung-Yu Chuang
类目：Computer Vision and Pattern Recognition (cs.CV)
关键词：perception-distortion trade-off poses, single-image super-resolution, trade-off poses, poses a fundamental, fundamental challenge
备注： ICIP 2026
点击查看摘要
Abstract:The perception-distortion trade-off poses a fundamental challenge in single-image super-resolution (SR). Although diffusion-based SR methods excel at generating perceptually realistic images, achieving high fidelity remains a key limitation. Recent advances in diffusion-based SR have shown promise in improving fidelity, but these methods often compromise perceptual quality due to their high reliance on a high-fidelity image. To address this, we introduce UGDiff, a novel diffusion guidance paradigm designed to further improve the perception-distortion balance. In particular, we first estimate the reconstruction uncertainty of the latent features corresponding to a high-fidelity image. This uncertainty is then used to guide the diffusion process to selectively restore high-frequency details in high-uncertainty regions, while preserving fidelity elsewhere. Furthermore, our guidance method adaptively identifies the high-uncertainty regions by considering not only the estimated uncertainty but also the posterior variance of the diffusion sampler at each timestep. This relaxes the reliance on the high-fidelity image in the later stages of sampling, thereby achieving a better perception-distortion balance. Extensive experimental results demonstrate that our method performs favorably against state-of-the-art diffusion-based SR methods.
11. 【2608.25981】FRAME: separating sampling variation from representational cause in medical imaging fairness
链接：https://arxiv.org/abs/2608.25981
作者：Mahshad Lotfinia,Daniel Truhn,Andreas Maier,Soroosh Tayebi Arasteh
类目：Computer Vision and Pattern Recognition (cs.CV); Artificial Intelligence (cs.AI); Machine Learning (cs.LG)
关键词：usual response removes, model encodes, Fair-model Reference, Mechanism Evaluation, standard evidence
备注：
点击查看摘要
Abstract:Subgroup performance differences are the standard evidence for fairness bias in medical imaging, and the usual response removes the demographic information that a model encodes. Here we introduce Fair-model Reference And Mechanism Evaluation (FRAME), a two-step framework for auditing such a claim. The first step derives a fair-model reference, the distribution of the difference under exact fairness at the observed subgroup sizes. In the second step, we test the remainder with two operators in representation space. One operator cannot change a within-group ranking by construction. Across 702,206 images and 36 encoders, the reference accounts for a median 41% of the reported race difference and 22% of the age difference. Injecting demographic decodability leaves the remainder unchanged, while entangling the group with the disease direction raises the race difference from 0.077 to 0.118. No intervention we tested changes the remainder more than a change of random seed does. Those interventions reduce a difference at the operating point and leave the within-group ranking difference at a median of 0.000. Applied to 89 differences in 9 published studies across 6 medical imaging modalities, the reference accounts for a median 25% of a rate difference and 70% of a difference in the area under the receiver operating characteristic curve. Image-text pretraining instead raises worst-group performance by about 0.05. Applying FRAME before choosing an intervention could distinguish differences that need a mechanistic explanation from differences compatible with sampling variation at the current cohort sizes.
12. 【2608.25970】PANDA - Prototype-Anchored Alignment for Partially Unpaired Multimodal Learning, with Applications to Alzheimers MRI and TCGA Pathology
链接：https://arxiv.org/abs/2608.25970
作者：Sheethal Bhat,Mahfuzur Rahman Chowdhury,Paula Andrea Perez-Toro,Stephan Wunderlich,Rose Dawn Bharat,Siming Bayer,Andreas Maier
类目：Computer Vision and Pattern Recognition (cs.CV); Artificial Intelligence (cs.AI)
关键词：Multimodal medical prediction, Multimodal medical, assumed at deployment, Anchored Data Alignment, Prototype Anchored Data
备注：
点击查看摘要
Abstract:Multimodal medical prediction often faces incomplete pairing: auxiliary modalities with complementary signal are available for only a subset of subjects (or none) and cannot be assumed at deployment. We introduce PANDA (Prototype Anchored Data Alignment), a two-stage framework that transfers auxiliary information to a primary-modality model without auxiliary inputs at inference. Stage 1 learns a shared embedding from the paired subset and estimates class prototypes from auxiliary modalities; Stage 2 trains the primary encoder on all subjects using cross-entropy plus alignment to the frozen prototypes. Because supervision is defined at the class-prototype level, PANDA accommodates arbitrary pairing rates, including zero subject overlap. We evaluate PANDA on two applications. On a 1,021-subject multi-scanner ADNI cohort, we perform AD/CN classification with three auxiliary modalities at distinct pairing rates: tabular scores (44.8%), FDG-PET (18.7%), and external handwriting kinematics (0% overlap). Relative to the same-backbone MRI-only baseline, PANDA attains AUC 0.868 +-0.020 (+7.9pp) and reduces 1.5T CN false positives by 24.3pp; on a fully trainable Conv5-FC3 backbone it reaches AUC 0.893 (best overall). A pairing-rate ablation shows that the joint anchor remains within seed noise from 75% to 5% pairing. On TCGA-Lung survival prediction from whole-slide images with RNA-seq as auxiliary data, PANDA improves over WSI-only on 2-year OS (AUC +3.5pp) and Cox PH (C-index +9.0pts) and outperforms full-fusion training, which underperforms WSI-only, while requiring no RNA at inference; wide confidence intervals on this smaller cohort keep the gains below conventional significance. Overall, PANDA provides a deployment-oriented mechanism for leveraging incomplete auxiliary modalities to improve primary-modality prediction.
13. 【2608.25965】Less Contouring, More Accuracy: Lesion-Guided ROI Deep Learning for Ovarian Ultrasound Classification
链接：https://arxiv.org/abs/2608.25965
作者：Mehran Ahmad,Ali Abbasian Ardakani,Afshin Mohammadi,Alisa Mohebbi,Gernot Kronreif,Sepideh Hatamikia
类目：Computer Vision and Pattern Recognition (cs.CV)
关键词：remains challenging due, overlapping imaging characteristics, transvaginal ultrasound remains, ultrasound remains challenging, ovarian ultrasound
备注： Submitted as a research article. The manuscript contains figures, tables, and supplementary material
点击查看摘要
Abstract:Ovarian lesion classification using transvaginal ultrasound remains challenging due to overlapping imaging characteristics and the dependence on expert interpretation. This study investigates whether lesion-guided region-of-interest (ROI) deep learning can achieve competitive diagnostic performance while reducing the annotation burden associated with pixel-level lesion segmentation. Two publicly available ovarian ultrasound datasets were evaluated: the Multi-Modality Ovarian Tumor Ultrasound (MMOTU) dataset for eight-class classification and the Ovarian Ultrasound Dataset (OUD) for binary classification. Four strategies were compared under a unified framework: global image-based deep learning, lesion-guided ROI-based deep learning, lesion contour-based deep learning, and contour-based radiomics with machine learning classifiers. Four deep learning architectures, MaxViT-Tiny, Swin Transformer, EfficientNet-B7, and ResNet18, were evaluated. Radiomics models were developed using support vector machine, k-nearest neighbors, and artificial neural network classifiers, with ANOVA-based feature selection applied for the lower-sample OUD dataset. The lesion-guided ROI strategy achieved the strongest overall performance, with MaxViT-Tiny obtaining 93.10% accuracy and an AUC of 0.99 on MMOTU and 97.56% accuracy and an AUC of 0.99 on OUD. The contour-based approach achieved comparable accuracy but required substantially higher annotation effort. These findings demonstrate that lesion-guided ROI deep learning provides an effective balance between diagnostic performance and annotation efficiency, offering a practical approach for scalable AI-assisted ovarian ultrasound analysis
14. 【2608.25956】4DGS-WAM: Bridging Past and Future with an Object-Centric World Action Model based on 4D Gaussian Splatting
链接：https://arxiv.org/abs/2608.25956
作者：Yueen Ma,Zenglin Xu,Irwin King
类目：Computer Vision and Pattern Recognition (cs.CV)
关键词：Current world action, Current world, typically operate, Current, visual data
备注： This is a work in progress
点击查看摘要
Abstract:Current world action models (WAMs) typically operate on 2D visual data. These models can achieve exceptional visual quality, but they lack explicit spatial structure for individual objects and repeatedly process redundant background content. Although point clouds can represent the world in 3D space, they can be difficult to align and accumulate across viewpoints. In this paper, we leverage an explicit 4D Gaussian Splatting (4DGS) representation that separately models dynamic objects and the static background of a scene. For dynamic objects, we use a policy model to predict future actor actions and a world model to predict transformations of their observed Gaussian splats. The static background need not be regenerated for future states, as much of it has already been observed in past frames. This forms an object-centric world action model, which we name 4DGS-WAM. It lifts 2D observations into a persistent 4D representation so that previously observed static content can be reused during future prediction. Future-state extrapolation can then focus on modeling the evolution of dynamic objects. Experiments on KITTI-MOT evaluate short-horizon prediction and past reconstruction.
15. 【2608.25948】Auditable CT Phenotyping Through Report-derived Radiological Observations
链接：https://arxiv.org/abs/2608.25948
作者：Riga Wu,Walter Witschey,Yicheng Li,Felix Barajas Ordonez,Keno K. Bressem,Lisa C. Adams,Gary E. Weissman,Li Shen,Christos Davatzikos,Eduardo Barbosa,Daniel Truhn,Tianyu Han
类目：Computer Vision and Pattern Recognition (cs.CV)
关键词：Medical image foundation, image foundation models, strong performance leaves, performance leaves open, read disease-specific findings
备注：
点击查看摘要
Abstract:Medical image foundation models can predict clinical phenotypes from computed tomography (CT), but strong performance leaves open whether they read disease-specific findings or shortcuts that correlate with the diagnosis. We tested this in 221 electronic-health-record (EHR) phenotypes using Auditable CT phenotyping (ACT), built on report-derived radiological observations. We trained ACT on 38,317 patients, mined 376,194 observations and evaluated it in 25,183 held-out patients. ACT exceeded five vision-language baselines on zero-shot annotation, and CT-CLIP across 221 phenotypes from unseen CT pulmonary angiography, both under zero-shot scoring (0.651 versus 0.572) and under linear probing (0.709 versus 0.662). Reading each probe exposes what accuracy conceals: only 97 observations occupy the 221 rank-1 positions, and one phrase describing aortic and coronary calcification ranks first for 20 phenotypes, including osteoporosis, urinary tract infection and major depressive disorder. Restricting the bank to clinician-specified evidence redirects those probes onto phenotype-related observations in 86 phenotypes at no accuracy cost (0.751 versus 0.741). Accurate CT-based EHR phenotyping can therefore rest on observations that are not valid evidence for the coded phenotype and that ACT can identify and intervene on.
16. 【2608.25935】AU-Agent: An Agentic Retrieval-Augmented Framework for Traffic Anomaly Understanding
链接：https://arxiv.org/abs/2608.25935
作者：Yuqiang Lin,Yan Shi,Sam Lockyer,Harish Tayyar Madabushi,Adrian Evans,Wenbin Li,Yinhai Wang,Nic Zhang
类目：Computer Vision and Pattern Recognition (cs.CV); Artificial Intelligence (cs.AI)
关键词：Traffic Anomaly Understanding, explain anomalous events, Anomaly Understanding, Traffic Anomaly, systems to detect
备注：
点击查看摘要
Abstract:Traffic Anomaly Understanding (TAU) requires models and systems to detect, reason about, and explain anomalous events in transportation videos. To address this challenge, we propose TAU-Agent, an agentic retrieval-augmented framework for traffic anomaly understanding. Given a task query, a central retrieval agent orchestrates two visual perception tools, namely a Video Captioning Tool and an Open-Vocabulary Tracking Tool, to retrieve and select query-relevant evidence, including captions, temporal intervals, and object trajectories. The selected evidence, together with sampled video frames and the input query, is provided to a supervised fine-tuned vision-language model for final reasoning and answer generation. We evaluate TAU-Agent on both the in-domain and the out-of-domain benchmarks from the AI City Challenge 2026. TAU-Agent achieves scores of 0.6779 on Track 3, 0.3998 on Track 7, and 67.9275 on Track 8, ranking second, twelfth, and fifth, respectively. Code is available at: this https URL.
17. 【2608.25933】When Composition Doesn't Add Up: Humans Identifying Defects in AI-Generated Images
链接：https://arxiv.org/abs/2608.25933
作者：Ruoqi Hu,Chulin Zhao,Jiashuo Chang,Ramon Ruiz-Dolz,Hanhe Lin
类目：Computer Vision and Pattern Recognition (cs.CV); Artificial Intelligence (cs.AI)
关键词：Chulin Zhao, Zhao and Ruoqi, Ruoqi Hu contributed, contributed equally, Chulin
备注： 6 pages, accepted at IEEE MMSP 2026
点击查看摘要
Abstract:*Chulin Zhao and Ruoqi Hu contributed equally to this work. State-of-the-art text-to-image (T2I) models exhibit pronounced and systematic defects when prompts involve intricate compositional factors such as multiple entities and multiple attributes. In this paper, we investigate how humans identify such defects. Specifically, we manually select 651 reference images from the four categories of people, hand, object, and scene that exhibit complex compositional characteristics, from which prompts emphasizing compositional factors are derived by manually editing ChatGPT-generated prompts. We then feed the prompts into three selected T2I models to generate AI images and conduct a comprehensive subjective study to identify their defects. For each image, 29 participants provide multi-label assessments specifying defect types and locations. The study yields the compositional AI-generated image defect (CO-AID) dataset, including reference images, prompts, AI-generated images, and information on defect locations and types. Experimental results show that training a deep model on CO-AID can both predict defects in AI-generated images and optimize AI image generation, demonstrating its usability and effectiveness. The database and supplementary materials are available at: this https URL .
Comments:
6 pages, accepted at IEEE MMSP 2026
Subjects:
Computer Vision and Pattern Recognition (cs.CV); Artificial Intelligence (cs.AI)
Cite as:
arXiv:2608.25933 [cs.CV]
(or
arXiv:2608.25933v1 [cs.CV] for this version)
https://doi.org/10.48550/arXiv.2608.25933
Focus to learn more
arXiv-issued DOI via DataCite (pending registration)</p>
18. 【2608.25927】Code World Model: Coding Agent as World Brain
链接：https://arxiv.org/abs/2608.25927
作者：Yiwen Chen,Guosheng Lin,Chi Zhang
类目：Computer Vision and Pattern Recognition (cs.CV); Artificial Intelligence (cs.AI); Computation and Language (cs.CL)
关键词：complex environments evolve, existing video-based world, mechanisms governing world, models primarily learn, primarily learn dynamics
备注： Project Page: [this https URL](https://buaacyw.github.io/cwm/)
点击查看摘要
Abstract:World models aim to simulate how complex environments evolve under actions and events, yet existing video-based world models primarily learn dynamics from visual observations, which reveal outcomes rather than the underlying knowledge, rules, and mechanisms governing world evolution. This makes it difficult to maintain persistent consequences and support coherent, open-ended evolution. We introduce Code World Model, a framework that separates world evolution from visual realization by combining the reasoning and coding capabilities of language models with the generative priors of video models. A coding agent serves as the world brain, reasoning about events and their consequences and generating executable code to maintain persistent world state and perform rule-consistent evolution. To connect executable state with visual generation, we introduce a proxy representation that encodes frame-wise spatiotemporal constraints and is compiled into a proxy video, which conditions a video model to render high-fidelity visual observations. We further develop data pipelines for constructing aligned proxy-observation pairs from gameplay and real-world videos. After fine-tuning on paired gameplay data, MiniMax-H3 follows proxy-based spatiotemporal specifications from simple interactive worlds built by the coding agent while preserving rich visual details and dynamics. These results demonstrate the potential of combining code for persistent world evolution with video models for flexible visual realization, providing a new path toward open-ended world models.
19. 【2608.25924】Visual General Intelligence: A White Paper
链接：https://arxiv.org/abs/2608.25924
作者：Hirokatsu Kataoka,Yoshihiro Fukuhara,Yonglong Tian,Shangzhe Wu,Oishi Deb,Ryousuke Yamada,Christian Rupprecht,Jianyuan Wang,Kohsuke Ide,Koichi Namekata,Xianzheng Ma,Yiming Chen,Robert Geirhos,Aditi Raghunathan,Yuki M. Asano,Deva Ramanan,David Fouhey,Andrew J. Davison,Yilun Du,Jiajun Wu,Zhuang Liu
类目：Computer Vision and Pattern Recognition (cs.CV)
关键词：paper reconsiders intelligence, vision-centered perspective, perspective and examines, paper reconsiders, provide a pathway
备注：
点击查看摘要
Abstract:This paper reconsiders intelligence from a vision-centered perspective and examines whether intelligence emerging from visual experience and learning may provide a pathway toward AGI. In the language domain, beginning with the introduction of the Transformer architecture, the GPT series has demonstrated transfer to unseen tasks through autoregressive language modeling on web-scale text combined with aggressive scaling. This raises a natural question, namely, what capabilities and forms of intelligence can emerge from visual modalities such as images, videos, and geometry? In this paper, we discuss whether visual intelligence can serve as a pathway toward AGI, referred to in this paper as visual general intelligence (VGI), by bringing together contributors from diverse standpoints and affiliations. Our aim is not to offer a single definition of visual intelligence, but to clarify the principles that computer vision should pursue in the AGI era, the visual input modalities, the benchmarks, the learning paradigms, and the relationship between vision, when taken as the core, and other modalities such as language.
20. 【2608.25888】Embedding NDRE Trajectories into Contrastive Learning for Label-Free, Physiology-Aware Crop-Stress Staging and DSS Outputs
链接：https://arxiv.org/abs/2608.25888
作者：Shafqaat Ahmad
类目：Computer Vision and Pattern Recognition (cs.CV)
关键词：conventional vegetation index, increasing drought frequency, Timely detection, capture stress progression, vegetation index thresholds
备注：
点击查看摘要
Abstract:Timely detection of crop stress is critical for sustaining yields under increasing drought frequency, yet conventional vegetation index thresholds or image-based clustering often fail to capture stress progression, limiting their value for farm decision-making. To address this gap, we present EigenCL, a physiology-guided contrastive learning framework that stages crop stress from Sentinel-2 NDRE trajectories, with the goal of providing interpretable and transferable stress diagnostics for decision support systems (DSS). EigenCL was trained on 10,000 maize NDRE patches from drought-affected Iowa fields in 2020 and tested on Nebraska fields in 2023 without retraining, with validation incorporating soil-moisture records, U.S. Drought Monitor maps, and county-level yield statistics. The model produced four physiologically coherent stress clusters (Healthy, Mild, Moderate, Severe), significantly outperforming baselines including K-Means, SimCLR, ProtoCLR, and an ablation model (Silhouette = 0.748, DBI = 0.35, CHI = 49,624). Clusters aligned with maize growth stages, with severe stress peaking around tasseling-silking (VT-R1), a stage known to drive yield loss; moreover, EigenCL clusters correlated with soil moisture at 0-14-day lags (rho up to 0.72) and matched yield anomalies in drought-affected counties. By embedding NDRE trajectory dynamics into contrastive learning, EigenCL enables early stress alerts and interpretable DSS outputs (e.g., heatmaps, scouting priorities, regional risk indices), extending beyond single-date NDRE thresholds and supporting scalable monitoring for climate-smart agronomy.
21. 【2608.25876】Do Vision-Language Models Agree on the Affective Qualities of Shape? A Cross-Model Audit for Generative Design Interfaces
链接：https://arxiv.org/abs/2608.25876
作者：Luca Bux,Thiago Rios,Ingo Scholtes,Stefan Menzel
类目：Human-Computer Interaction (cs.HC); Computer Vision and Pattern Recognition (cs.CV)
关键词：Generative design interfaces, design interfaces increasingly, users steer output, Generative design, interfaces increasingly expose
备注： 13 pages, 5 figures, 7 tables
点击查看摘要
Abstract:Generative design interfaces increasingly expose semantic controls that let users steer output with concepts such as "more elegant" or "more minimalist," typically encoded by a vision-language model (VLM). A practical question is whether state-of-the-art VLMs represent objects consistently in terms of the same concept. We audit 6 VLMs by ranking untextured 3D objects along Kansei adjective pairs, where Kansei describes affective impressions of product form, with each axis defined as the difference between the text representations of its two poles. Geometric pairs serve as positive controls, and pairs of unrelated adjectives establish an empirical null. Across 10 categories of ShapeNet database, affective axes converge above the null (mean pairwise rank correlation 0.36 vs. 0.14) but below the geometric ceiling (0.44). The agreement between models is partial and highly uneven: on the three axes shared by all categories, mean convergence ranges from 0.21 for bookshelves to 0.51 for jars. Convergence depends primarily on whether a category's representational variation aligns with the semantic direction being evaluated, rather than simply on how much the objects vary in shape overall. Cross-model convergence does not imply agreement with human judgments. Based on our findings, we implement a UI prototype that shows how the audit can inform which Kansei descriptors to expose as controls for a given object class and which to withhold.
22. 【2608.25866】LUTSeg: A Longitudinal Multi-Expert Dataset for Ulcer Tissue Segmentation
链接：https://arxiv.org/abs/2608.25866
作者：Karen Sanchez,Carlos Hinojosa,Albert A. Ávila,Andrea C. Riano-Rojas,Diego H. Romero,Jenny C. Páez,Martina Llinás,Bernard Ghanem
类目：Computer Vision and Pattern Recognition (cs.CV)
关键词：guiding treatment decisions, Quantifying wound tissue, Quantifying wound, treatment decisions, monitoring chronic ulcer
备注： Published at ISIC in MICCAI 2026
点击查看摘要
Abstract:Quantifying wound tissue composition is essential for monitoring chronic ulcer progression and guiding treatment decisions. However, pixel-level annotations are costly, and multi-tissue wound datasets remain scarce, particularly for neglected diseases such as leprosy. We introduce LUTSeg, a longitudinal chronic ulcer dataset comprising 141 images from 39 patients with wound masks and five tissue categories annotated by five expert clinicians, including a multi-expert gold-standard subset for inter-rater agreement analysis. To establish an initial benchmark for LUTSeg, we further propose TiSage, a semi-supervised tissue segmentation framework that integrates multi-scale semantic priors from a frozen medical vision-language model within a teacher-student architecture. We evaluate TiSage on LUTSeg and DFUTissue, showing improvements over supervised and semi-supervised baselines in most low-label settings. Code data: this https URL
23. 【2608.25862】Learning Late, Guiding Early: Timestep-Decoupled Semantic Guidance for Fair Face Generation
链接：https://arxiv.org/abs/2608.25862
作者：Subir Kumar Parida,Rajbabu Velmurugan,Ketan Kotwal,R.S. Sengar,Swati Hiremath
类目：Computer Vision and Pattern Recognition (cs.CV)
关键词：face recognition systems, downstream face recognition, synthetic face generation, recognition systems, synthetic face
备注：
点击查看摘要
Abstract:Demographic imbalance in synthetic face generation can propagate to downstream face recognition systems, making fairness an important consideration when diffusion models are used for data generation. Existing fairness-aware generation approaches often require model retraining, architectural modifications, or repeated guidance throughout the reverse diffusion process. In this work, we introduce Semantic Boundary Predictor (SBP), an inference-time framework that performs demographic guidance through a one-shot intervention during reverse denoising. Our approach is motivated by the observation that latent representations at different diffusion timesteps play distinct semantic roles: late-stage latents provide stronger demographic separability, whereas early-stage latents offer greater flexibility for semantic intervention. SBP leverages this timestep decoupling by learning linear semantic boundaries from late-stage latent representations while applying them only once at the initial noisy latent, allowing the remainder of the reverse denoising process to proceed unchanged. The method requires neither retraining nor fine-tuning of the underlying Latent Diffusion Model and operates without external balanced datasets. Experiments on CelebA-HQ demonstrate substantial improvements in demographic fairness, reducing fairness disparity by 98% for gender, 95% for binary race, and 15% for four-class race, while maintaining perceptual image quality across demographic groups. Owing to its one-shot inference strategy and model-agnostic design, SBP introduces only a small computational overhead and can be readily integrated with existing pre-trained latent diffusion models.
24. 【2608.25858】Precipitation Downscaling Using Foundation Model-Conditioned Diffusion
链接：https://arxiv.org/abs/2608.25858
作者：Victor Nascimento Ribeiro,Jorge Guevara,Jorge Sebastian Moraga,Chris Lucas,Natalie Lord,Andrew Taylor,Edward Lockhart,Will Trojak,Johannes Schmude,Anne Jones
类目：Computer Vision and Pattern Recognition (cs.CV); Machine Learning (cs.LG); Atmospheric and Oceanic Physics (physics.ao-ph)
关键词：hydrological impact assessment, High-resolution precipitation fields, High-resolution precipitation, climate model outputs, global climate model
备注：
点击查看摘要
Abstract:High-resolution precipitation fields are essential for hydrological impact assessment, yet global climate model outputs are too coarse and biased for direct use. AI-based statistical downscaling with diffusion models offers a promising approach, but the mechanism by which large-scale atmospheric predictors condition generation remains largely unexplored. We investigate three conditioning strategies for a denoising diffusion probabilistic model applied to daily precipitation downscaling: channel concatenation of upsampled coarse predictors, cross-attention conditioning with a learned convolutional encoder, and cross-attention conditioning with the frozen encoder of the pretrained Prithvi WxC weather foundation model. All strategies are evaluated against an unconditioned baseline under identical conditions using probabilistic, distributional, spectral, and extreme-event metrics for the Colorado River Basin. Concatenation conditioning achieves the lowest point-wise CRPS and MSE, but tends to produce over-smoothed fields that suppress high-intensity events. In contrast, cross-attention conditioning provides substantially better distributional realism and modest improvements in spectral fidelity. Improvements are greatest for extremes: the Prithvi-WxC conditioned model retains over half of 100mm/day events, although estimates are uncertain due to limited samples. When trained on the full dataset, the learned convolutional model performs similarly to the foundation model-conditioned approach while requiring lower computational resources. However, the Prithvi-WxC-conditioned model achieves comparable performance with only five years of training data. These results indicate that cross-attention conditioning offers advantages over simple concatenation for probabilistic precipitation downscaling, and that pre-trained foundation model representations may offer benefits in data-limited settings.
25. 【2608.25851】DEFUSE: Generalizable Backdoor Defense for Self-Supervised Encoders with Generative Priors
链接：https://arxiv.org/abs/2608.25851
作者：Tuo Chen,Jie Gui,Minjing Dong,Lanting Fang,Ju Jia,Benlei Cui,Jian Liu
类目：Computer Vision and Pattern Recognition (cs.CV)
关键词：Self-supervised learning, posing threats, visual SSL, SSL, SSL encoders
备注： Accepted at ACM Multimedia 2026
点击查看摘要
Abstract:Self-supervised learning (SSL) encoders are vulnerable to backdoor attacks, posing threats to both visual SSL encoders and vision-language encoders. Existing defenses are typically designed for only one of these paradigms and rely on restrictive assumptions such as access to uninfected in-distribution data or precomputed pseudo-labels, which are difficult to satisfy in practice. To address these limitations, we propose DEFUSE, a generalizable backdoor detection framework for SSL encoders. Inspired by Bayesian posterior inference, we reformulate backdoor detection as a representation-conditioned image likelihood estimation problem parameterized by a conditional diffusion generative model. Uninfected representations tend to yield semantically consistent reconstructions, whereas backdoored ones are more likely to be mapped to the attacker's target class or semantically meaningless images, deviating from the original semantics and thereby exposing the backdoor. However, we find that the exact likelihood is intractable, because highly abstracted representations discard the low-level information necessary for pixel-faithful reconstruction. We therefore relax the objective to semantic reconstruction and evaluate it in a well-separated representation space provided by a reference encoder. Rather than training from scratch, we fine-tune a pretrained diffusion model, leveraging its generative prior to map data onto the natural image manifold while preserving semantic content. Extensive experiments demonstrate that DEFUSE substantially outperforms existing detectors across diverse attack settings, generalizing to both visual SSL and vision-language encoders. Notably, our method greatly reduces the reliance on prior knowledge about the victim encoder or the attack strategy. The source code is available at this https URL .
26. 【2608.25845】HA-Flow Generative Model: Prosthesis Geometry Prediction from Preoperative CT
链接：https://arxiv.org/abs/2608.25845
作者：Yiping Wang,Jie Li,Jingyu Shen,Liao Wang
类目：Computer Vision and Pattern Recognition (cs.CV)
关键词：total hip arthroplasty, single prosthesis configuration, patient osseous anatomy, commonly framed, framed as selecting
备注： 17 pages, 7 figures, 2 tables
点击查看摘要
Abstract:Preoperative planning for total hip arthroplasty (THA) is commonly framed as selecting a single prosthesis configuration and placement for a patient's osseous anatomy. In practice, however, the same anatomy may admit several clinically reasonable solutions, making planning inherently a one-to-many problem that is better represented by a conditional probability distribution. We present THA-Flow, a conditional flow-matching model that generates three-dimensional prosthesis geometry directly from preoperative CT. Separate AutoencoderKL models compress preoperative bone anatomy and prosthesis geometry, while a three-dimensional UNet learns a rectified flow from Gaussian noise to the prosthesis latent space under spatial bone conditioning and optional structured prosthesis parameters. The retrospective cohort comprised 1,355 hips from 1,149 patients undergoing primary THA. Following rigid registration of postoperative CT to preoperative CT, the actual postoperative prostheses were transformed independently according to the pelvic and femoral registrations and represented as a dual-channel truncated signed distance field. The prosthesis autoencoder achieved a peak signal-to-noise ratio of 47.11 dB and a structural similarity index of 0.9964 on the validation set. Complete acetabular and femoral geometries were generated across seven major stem models representing 93.4% of the cohort. Repeated bone-conditioned sampling preserved component position, alignment, and the principal bone-prosthesis interfaces while allowing limited local geometric variation. To our knowledge, THA-Flow represents the first application of generative AI to three-dimensional surgical planning for THA.
27. 【2608.25836】Socialized Detector Learning: Trajectory-Guided and Reciprocal Distillation for Heterogeneous Object Detectors
链接：https://arxiv.org/abs/2608.25836
作者：Weihao Li,Yunqi Zhu,Zhihe Fan,Ruipu Zhao,Boan Tao,Xinjie Yao,Yan Fan,Pengfei Zhu
类目：Computer Vision and Pattern Recognition (cs.CV)
关键词：Object detection knowledge, independently trained, fragmented across independently, socialized learning, Socialized Detector Learning
备注： 12 pages; supplementary material included
点击查看摘要
Abstract:Object detection knowledge is fragmented across independently trained, heterogeneous detectors with complementary category supports. In socialized learning, this knowledge resides in a society, and learning aims to evolve the society collectively through exchange. However, aggregation-based socialization does not explicitly plan transfer order, whereas progressive multi-teacher distillation considers order but remains a one-way student enhancement in a shared category space. Building on Socialized Learning, we formulate Socialized Detector Learning (SDL) for heterogeneous, category-specialized object detectors and propose Trajectory-Guided and Reciprocal Distillation (TGRD).TGRD estimates directed operational Inter-Detector Transfer Difficulty (IDTD) from held-out feature-alignment residuals, precomputes a fixed score table, and greedily constructs a carrier trajectory. Along the trajectory, knowledge is progressively consolidated into a union-category carrier and then returned to experts through reciprocal transfer. A conditional proxy-certificate analysis shows that, under stated assumptions, the progressive certificate is no larger than an aggregated-target counterpart. On MS COCO with four heterogeneous experts and two carrier initializations, final carriers outperform epoch-matched simultaneous aggregation controls by 2.6 AP in both settings. Reciprocal detectors attain 20.8--28.4 AP on previously unsupported categories while remaining within 1.3 AP of original expert-specific performance. These results support order-aware progressive consolidation followed by reciprocal transfer as a viable mechanism for detector-society evolution.
28. 【2608.25828】FlowMoDL: Model-Based Deep Learning with Conjugate-Gradient Data Consistency for Highly Accelerated 4D Flow MRI Reconstruction
链接：https://arxiv.org/abs/2608.25828
作者：Tristan Gottwald,Michelle Bruch,Mubashir-Ul Hassan,Fatma Alickovic,Milan Kloiber,Daniel Tenbrinck,Torsten Panholzer,Melanie Schaller,Jana Hutter
类目：Computer Vision and Pattern Recognition (cs.CV); Machine Learning (cs.LG)
关键词：flow MRI reconstruction, flow MRI, MRI reconstruction, unrolled neural network, highly accelerated
备注：
点击查看摘要
Abstract:We present FlowMoDL, an unrolled neural network for highly accelerated 4D flow MRI reconstruction that directly optimizes for both anatomical magnitude and phase-derived velocity accuracy. Building on the MoDL framework, FlowMoDL alternates a learned (3+1)D spatiotemporal denoiser with conjugate-gradient data-consistency updates based on the SENSE forward model. A novel dual-pathway conditioning scheme adapts the denoiser features and data-consistency weighting, enabling a single model to handle varying acceleration factors ($10\times$ to $50\times$). To ensure physiological accuracy, the network is trained using a deep-supervision composite loss that explicitly penalizes velocity magnitude and angular errors, stabilized by a curriculum schedule. We evaluate FlowMoDL on the multi-center CMRx4DFlow dataset against classical and deep-learning baselines (CG-SENSE, MoDL, FlowVN, and FlowMRI-Net). A key advantage of FlowMoDL is its superior gradient step efficiency. When evaluated under an equivalent, limited budget of gradient steps, competing flow-specific networks degrade significantly. In contrast, FlowMoDL robustly converges and strictly outperforms all competitors across all acceleration factors in magnitude SSIM, nRMSE, relative velocity error, and angular error, successfully recovering sharp structural details and temporally coherent velocity fields.
29. 【2608.25819】Steer the Sampling, Not the Kernel Grid: Geometry-Guided Sampling Operator for Volumetric Segmentation
链接：https://arxiv.org/abs/2608.25819
作者：Sizhe Wang,Himashi Peiris,Zhaolin Chen
类目：Computer Vision and Pattern Recognition (cs.CV)
关键词：quantitative lesion assessment, segmentation is central, planning and follow-up, central to quantitative, quantitative lesion
备注： Accepted at MICCAI 2026
点击查看摘要
Abstract:Accurate 3D segmentation is central to quantitative lesion assessment and anatomy mapping for clinical planning and follow-up. Thin, elongated, and fine anatomical/pathological structures (e.g., vessels) are a particularly challenging case: a one-voxel boundary error can disconnect a branch and change clinically relevant topology. In encoder-decoder networks (e.g., U-Net), repeated downsampling and fixed-grid convolution blur or alias fine structures and weaken orientation cues, so early mistakes propagate across scales. We propose a geometry-guided local operator that steers where features are sampled, rather than deforming convolutional kernels, under a single formulation for both feature refinement (stride 1) and resolution reduction (stride 1). At each voxel, it predicts a local orientation and bounded step sizes, samples symmetrically along these directions, and transforms paired samples into compact geometric and boundary cues with lightweight mixing; a cross-scale consensus aligns encoder and decoder features at skip connections to reduce geometric mismatch. Replacing all stride 1 and stride 2 operators in a 3D U-Net yields consistent improvements on BraTS, MSD Hepatic Vessel, and TDSC-ABUS, with notably better boundary metrics (e.g., BraTS Dice 86.1 to 88.9, HD95 7.1 to 6.2; TDSC-ABUS HD95 39.1 to 27.8) while reducing parameters from 2.3M to 0.8M. We further demonstrate that the operator can be integrated into other backbones (e.g., nnU-Net, Swin-UNETR, and MedNeXt) without changing their macro-architectures while providing consistent performance gains.
30. 【2608.25810】Label-Free Foundational Model Selection for Medical Image Classification under Distribution Shift via Pseudo Label Discrepancy
链接：https://arxiv.org/abs/2608.25810
作者：Juan Iñaki Larrea,Lucas Mansilla,Enzo Ferrante
类目：Computer Vision and Pattern Recognition (cs.CV)
关键词：medical image analysis, Foundation models, image analysis, increasingly deployed, deployed for medical
备注： Accepted at MIRASOL Workshop, MICCAI 2026. 10 pages, 3 figures
点击查看摘要
Abstract:Foundation models are increasingly deployed for medical image analysis. However, under the inter-institutional distribution shift typical of deployment, their performance varies widely and cannot be known without target-domain labels, which are rarely available. This leaves a practical question unresolved: given several candidate foundational models and labeled-data from a source domain, which one to deploy in an unlabeled target domain? We propose a label-free selection criterion built on SUDO, a framework for evaluating clinical AI systems without ground-truth annotations. SUDO partitions the unlabeled target data by predicted probability and, for each region, measures a pseudo-label discrepancy reflecting class contamination; aggregated across regions, this yields a score (AURCC) requiring neither target annotation nor fine-tuning. We show that AURCC can be used to rank a variety of vision-language models (BioMedCLIP, CXR-CLIP, CheXzero, MedCLIP, MedImageInsight, CLIP) on chest X-ray classification across three inter-hospital shift scenarios, under zero-shot and MLP-probe regimes. The AURCC ranking recovers the ground-truth ranking with Spearman rho up to 0.943 (p0.05). Against the natural baseline of ranking by held-out source accuracy, AURCC is competitive when the labeled source is large and yields a more accurate ranking once it is small; the regime of interest in resource-constrained settings.
31. 【2608.25808】DFNet: Tri-projection Deformable Fusion Network for Panoramic Salient Object Detection
链接：https://arxiv.org/abs/2608.25808
作者：Qiangqiang Zhou,Jiacong Yu,Jiawei Xu,Yong Chen,Xin Huang,Ping Li
类目：Computer Vision and Pattern Recognition (cs.CV)
关键词：Recent years, virtual reality, panoramic salient object, salient object detection, robotic vision
备注：
点击查看摘要
Abstract:Recent years have witnessed the growing potential of panoramic salient object detection in robotic vision, virtual reality, and related applications. However, projecting spherical scenes onto 2D planes inevitably introduces geometric distortions, which fundamentally limit the effectiveness of existing projection-based methods. Specifically, Equirectangular Projection (ERP) suffers from severe polar stretching distortions, while cube map projection introduces discontinuities across cube-face boundaries, resulting in degraded feature discriminability and compromised geometric consistency. To address these limitations, we propose TDFNet, the first Tri-projection Deformable Fusion Network for panoramic salient object detection, exploiting complementary projection representations to alleviate geometric distortions and improve detection this http URL, we design a cross-projection deformable attention (CDA) module that leverages spatial correspondences between different projections to construct geometry-aware sampling locations, guiding deformable attention for cross-projection contextual aggregation and enhancing robustness against projection-induced deformations. Furthermore, we introduce a latitude-guided fusion module, which utilizes spherical latitude priors to construct geometric confidence weights for adaptively balancing ERP and CMP features. Meanwhile, LGF incorporates distortion-reduced semantic references from Tangent Projection to achieve cross-projection feature refinement and spatial this http URL constructing a three-branch encoding architecture based on ERP, CMP, and Tangent Projection, TDFNet simultaneously preserves global spatial continuity, local geometric details, and fine-grained boundary information.
32. 【2608.25759】Learning from waste: Machine Learning for health risk prediction and computer vision-based sorting in Ghana
链接：https://arxiv.org/abs/2608.25759
作者：Hilda Adwubi Osei,Catherine Tenewaa Osei,Desdemona Yaa Asobayire
类目：Machine Learning (cs.LG); Computer Vision and Pattern Recognition (cs.CV); Computers and Society (cs.CY)
关键词：environmental concern worldwide, solid waste remains, concern worldwide, remains a significant, environmental concern
备注： 16 pages, 3 figures
点击查看摘要
Abstract:The inappropriate disposal of solid waste remains a significant public health and environmental concern worldwide, including in Ghana. Poor sanitation and improper waste management practices contribute to substantial economic costs and avoidable deaths annually. In 2022, a field study in Atonsu, Kumasi, Ghana, reported a community-perceived relationship between household waste disposal and illness patterns, but only through descriptive analysis without quantitative validation. This study extends that investigation using two data-driven approaches. First, a Random Forest classifier was developed to predict illness categories using waste disposal practices and demographic survey data. On a held-out group of respondents who reported illness (N=69), the model obtained a macro F1 score of 0.63, with disposal method emerging as the most important substantive predictor of illness type. Second, a MobileNetV2 image classification model enabled automated waste sorting via visual recognition, achieving 88.2% accuracy and a macro F1 score of 0.87 on the test set (N=415). The vision-based approach offers an affordable, camera-driven alternative to complex multi-sensor systems, making it highly suitable for resource-constrained settings. Taken together, the findings provide quantitative evidence for a community health relationship previously documented only qualitatively. They demonstrate the potential for automated waste-sorting in low-resource environments. Importantly, the results illustrate that technological performance alone does not guarantee public health improvements; effective institutional support and implementation are equally necessary.
33. 【2608.25736】Moving Beyond More Views: Redundancy-Aware Ego-Exo Fusion for Proficiency Estimation
链接：https://arxiv.org/abs/2608.25736
作者：Xu Dong,Wanqing Li,Anthony Adeyemi-Ejeye,Andrew Gilbert
类目：Computer Vision and Pattern Recognition (cs.CV)
关键词：proficiency estimation aims, assess action quality, integrating fine-grained motion, EgoExo proficiency estimation, fine-grained motion cues
备注： Accepted by ECCV 2026
点击查看摘要
Abstract:EgoExo proficiency estimation aims to assess action quality by integrating fine-grained motion cues from egocentric (1st-person) views with spatial context from multiple exocentric (3rd-person) views. Simply adding more exocentric views degrades EgoExo performance, as redundant or noisy perspectives dilute useful motion cues. Our analysis identifies two key causes: (1) Multiview redundancy - From the data perspective, certain views provide limited or noisy information, diluting discriminative cues; (2) Overfitting - From the feature perspective, conventional fusion increases representational complexity, causing the model to memorise view-specific patterns rather than learn generalisable representations. To address these issues, we propose two complementary modules: AdaMVS, which adaptively identifies and fuses the most informative view tokens under weak supervision from the data perspective, and VIB-GB, which combines Gradient Blending and Variational Information Bottleneck regularisation from the feature perspective to compress redundant signals and suppress overfitting during training. Experiments on EgoExo-4D and EgoExo-Fitness demonstrate that our method learns both which view to look at and how to fuse them, achieving new state-of-the-art results. Our source code is available at this https URL
34. 【2608.25734】InteractGesture: Progressive Chunk Guidance for Continuous Streaming Co-Speech Gesture Control
链接：https://arxiv.org/abs/2608.25734
作者：Ekkasit Pinyoanuntapong,Ajinkya Deogade,Paul Streli,Wenjing Zhang,Joanna Materzynska,Pu Wang,Vittorio Ferrari,Jie Shen
类目：Computer Vision and Pattern Recognition (cs.CV)
关键词：made significant progress, existing models lack, models lack fine-grained, realistic full-body motion, lack fine-grained spatial
备注： ECCV 2026 Workshop - Interactive Social Avatars
点击查看摘要
Abstract:Co-speech gesture generation has made significant progress toward realistic full-body motion from speaker audio, yet existing models lack fine-grained spatial controllability of individual joints. To address this, we introduce \emph{InteractGesture}, a model-agnostic, inference-time method for spatially controllable gesture generation. \emph{InteractGesture} guides target latent estimates of a diffusion sampler through a differentiable RVQ-VAE decoder, backpropagating spatial control gradients to adjust motion latents during sampling. A primary challenge in streaming co-speech generation is chunk-wise dependency: standard sequential inference freezes prior chunks, preventing spatial constraints in future chunks from adjusting preceding trajectories and causing boundary inconsistencies. To overcome this limitation, we propose \emph{Progressive Chunk Guidance}, a chunk-window strategy that maintains an active set of editable chunk latents with staggered delays, enabling spatial constraints to propagate gradients backward across chunk boundaries during streaming generation. Experiments on the BEAT2 dataset show that \emph{InteractGesture} improves multi-joint spatial control while preserving overall gesture quality. Furthermore, our approach supports diverse applications, including sparse joint positioning, dense joint trajectory control, and directional pointing. Our project page is available at this https URL .
35. 【2608.25733】MIMONet: Multi-scale Input and Multi-scale Output Network for Salient Object Detection
链接：https://arxiv.org/abs/2608.25733
作者：Zhaojian Yao,Wei Gao,Tiesong Zhao,Hui Yuan,Sam Kwong
类目：Computer Vision and Pattern Recognition (cs.CV)
关键词：detection task focus, strengths of high, multi-level features, task focus, respective strengths
备注：
点击查看摘要
Abstract:The existing methods for saliency detection task focus on the application of multi-level features, aiming to take advantage of the respective strengths of high- and low-level features. However, because the inputs of these models are single-size images, their multi-level features have difficulty in learning the knowledge of size variations of salient objects. Object-scale variation learning has great potential for detecting multi-scale objects, which has not been fully explored by existing methods. To improve the recognition ability of a model for objects with different sizes, we are inspired by the image pyramid to propose a Multi-scale Input and Multi-scale Output Network (MIMONet). In MIMONet, we extract multi-level features for three images with different resolutions to form three encoder branches, and information will be exchanged between the branches. The advantage of this approach is that the features of one branch can learn the knowledge of target size variation from the features of the other two branches. In addition, we design a Multi-scale Perception (MSP) module, in which the input feature layer is divided into several sub-layers with different resolutions. Capturing the multi-level structure information of the objects in these sub-layers can make the objects more fully perceived. For network training, we propose a Joint Saliency Loss (JSL), which can constrain multiple saliency maps output by the network to identify the same foreground objects, and induce their boundaries to be preserved clearly. Experimental results show that MIMONet has stronger detection capabilities and harvests better evaluation scores on multiple datasets compared to existing models. The code of our model will be released.
36. 【2608.25729】LongVU-TTT: Causal Test-Time Training for Visual Resampling in Long Video Understanding
链接：https://arxiv.org/abs/2608.25729
作者：Mahmoud Ahmed,Sameh Abdulah,Olatunji Ruwase,Sam Ade Jacobs,Mathis Bode,Mohamed Elhoseiny
类目：Computer Vision and Pattern Recognition (cs.CV)
关键词：limited visual-token budget, visual-token budget removes, Long-video MLLMs, model temporal change, MLLMs must model
备注：
点击查看摘要
Abstract:Long-video MLLMs must model temporal change before a limited visual-token budget removes most frame evidence. We introduce LongVU-TTT, which inserts a convolutional Test-Time Training (TTT) resampler with causal fast-weight updates between the vision encoder and the LLM. Its grouped 2D fast weights adapt to each video and contextualize frame features before compression, while a hybrid uniform-and-change-aware selector retains explicit visual evidence for downstream reasoning. Under controlled conditions, TTT-Conv improves over TTT-MLP by up to +2.12 and bidirectional Mamba2 by up to +3.04 on MLVU, and it is stronger than attention- and fixed-state recurrent resamplers across three benchmarks. Analysis shows that the fast weights behave as a temporal aggregation state rather than a reliable long-horizon episodic memory: their benefit attenuates as evidence becomes more distant, motivating explicit frame retention. LongVU-TTT processes up to 512 frames before reducing them to 128 LLM frames and achieves competitive performance across five video understanding benchmarks.
37. 【2608.25710】Difficulty-Aware Sample Allocation for Adaptive Data Augmentation in Semantic Segmentation
链接：https://arxiv.org/abs/2608.25710
作者：Olasimbo Ayodeji Arigbabu,Abimbola Ismail Arigbabu
类目：Computer Vision and Pattern Recognition (cs.CV); Artificial Intelligence (cs.AI)
关键词：techniques allocate transformations, allocate transformations uniformly, semantic segmentation pipelines, modern semantic segmentation, single difficulty signal
备注： 18
点击查看摘要
Abstract:Data augmentation is a standard component of modern semantic segmentation pipelines, but most augmentation techniques allocate transformations uniformly across training samples or adapt to a single difficulty signal such as loss. This ignores the fact that segmentation difficulty is multi-factorial, since ambiguous predictions, persistent optimization errors, rare classes, and complex object boundaries can each make a sample informative in different ways. This paper introduces Difficulty-Aware Sample Allocation (DASA), an architecture-agnostic framework that assigns stronger augmentation to samples estimated to be more difficult. DASA combines prediction ambiguity, training loss, class rarity, and boundary complexity into a normalized difficulty score, then maps that score to sample-specific augmentation strength during iterative training. Experiments on Oxford-IIIT Pet and binary Pascal VOC segmentation with U-Net, DeepLabV3, and SegFormer-B0 show that DASA improves over standard training and is competitive with or stronger than single-signal adaptive baselines. On Oxford-IIIT Pet, DASA improves DeepLabV3 from 0.633 to 0.740 mIoU. On binary Pascal VOC, DASA obtains the best foreground IoU for all three evaluated architectures. These results attest to the value of multi-factor difficulty estimation as a practical mechanism for directing augmentation where it is most useful.
38. 【2608.25701】Skeleton-based Zero-Shot Spatio-Temporal Action Localization via Weakly-Supervised Pretraining
链接：https://arxiv.org/abs/2608.25701
作者：Koshiro Nagano,Fumiaki Sato,Ryo Hachiuma,Kazuki Tsutsukawa,Taiki Sekii
类目：Computer Vision and Pattern Recognition (cs.CV)
关键词：skeleton-based zero-shot spatio-temporal, overcoming high annotation, high annotation costs, large-scale action scenery, estimate unseen actions
备注： 13 pages, 6 figures
点击查看摘要
Abstract:We propose a novel pretraining strategy for skeleton-based zero-shot spatio-temporal action localization to estimate unseen actions for person instances while overcoming high annotation costs for training via new target actions and pretraining using large-scale action scenery datasets. Specifically, our approach, termed Skeleton-Language feature Pooling Switching, introduces a weakly-supervised vision-language pretraining mechanism. This mechanism transitions pooling kernels from pretraining, which aggregates skeleton features at the video level and aligns them with each video's known action text embeddings, to the inference phase that computes instance-level features without training via target actions. Furthermore, we propose Scene-Mixed Discriminative Contrastive Learning to distinguish actions at the instance level within the combined scene through the MIL framework. Our experiments on four public spatio-temporal action localization and classification datasets demonstrate that the proposed method effectively addresses annotation limitations.
39. 【2608.25693】Unsupervised Anatomical Feature Learning via Diffusion Models: Enhanced Medical Image Segmentation with Denoising Diffusion Probabilistic Models
链接：https://arxiv.org/abs/2608.25693
作者：Akshat G,Divyansh Gupta,Shaleen Bhatnagar,Shilpa Ankalaki,Tusar Kanti Mishra
类目：Computer Vision and Pattern Recognition (cs.CV); Artificial Intelligence (cs.AI); Machine Learning (cs.LG)
关键词：Acquiring pixel-level annotations, Acquiring pixel-level, severe bottleneck, pixel-level annotations, annotations for medical
备注： 15 pages, 5 figures. Submitted for peer review
点击查看摘要
Abstract:Acquiring pixel-level annotations for medical image segmentation is a severe bottleneck. Traditional U-Net architectures, while effective, learn local texture patterns and lack awareness of global anatomical structures, leading to boundary delineation failures in low-data regimes. This research paper proposes utilizing unsupervised Denoising Diffusion Probabilistic Models (DDPMs) to extract anatomical features. We train a DDPM on 21 unlabeled abdominal CT scans to learn structural representations, transferring the encoder weights to a downstream segmentation task evaluated on the BTCV multi-organ dataset. Diffusion pretraining significantly improved liver segmentation: Dice increased from $0.75\pm0.36$ to $0.93\pm0.16$ ($p 5.33\times10^{-26}$, 0.529 Cohen's d), Average Surface Distance (ASD) decreased by 66%, and 95th-percentile Hausdorff Distance (HD95) reduced by 45%. For kidney segmentation, Dice improved from $0.90\pm0.19$ to $0.95\pm0.10$ ($p 4.01\times10^{-11}$). Multi-organ pooled performance showed a 68% variance reduction and a 74% improvement in boundary precision (Dice $0.95\pm0.07$). Crucially, frozen encoder models retained 80% of fine-tuned performance without exposure to segmentation labels, proving the existence of learned anatomical priors. In low-data scenarios, diffusion-pretrained models maintained robust performance with only 50% (Dice: 0.92 liver, 0.94 kidney), 25%, and even 10% (Dice: 0.89 liver, 0.71 kidney) of labeled data. Using unlabeled images for diffusion-based pretraining successfully embeds robust anatomical features prior to human supervision, transforming U-Nets into anatomy-aware systems.
40. 【2608.25692】CloSeR: Unified Relational Distillation from Closed-Set Teachers for Category Discovery
链接：https://arxiv.org/abs/2608.25692
作者：Yuanpei Liu,Zhenqi He,Jialu Tang,Kai Han
类目：Computer Vision and Pattern Recognition (cs.CV)
关键词：Generalized Category Discovery, Generalized Category, garnered increasing attention, intriguing open-world problem, Category Discovery
备注： Accepted as a conference paper at ECCV 2026
点击查看摘要
Abstract:Generalized Category Discovery (GCD) is an intriguing open-world problem that has garnered increasing attention: given partially labelled data, the goal is to correctly recognize known classes while discovering coherent novel categories from unlabelled samples. Recent GCD methods typically adapt foundation models by jointly optimizing supervised classification and unsupervised discovery objectives on mixed labelled and unlabelled data. While effective, this coupled training can entangle closed-set recognition and open-set discovery, leading to objective conflict and biased predictions, and may disturb the semantic geometry of pretrained representations under limited labels and noisy pseudo-labels. We propose CloSeR, a simple plug-and-play framework that injects Closed-Set Relational knowledge into GCD training. CloSeR first builds a domain-adapted closed-set teacher by tuning lightweight block-wise adapters on labelled known-class data while keeping the foundation model backbone frozen, thereby preserving pretrained priors at low training cost. It then transfers the teacher's knowledge to downstream GCD via Unified Relational Distillation (URD), which distills complementary global sample-to-prototype relations to anchor known-class semantics and local sample-to-sample relations to preserve neighborhood structure, using separate feature pathways to reduce optimization interference. CloSeR is head-agnostic and readily integrates with both parametric and non-parametric GCD methods. Extensive experiments with DINO and DINOv2 backbones on six benchmarks (CIFAR-10/100, ImageNet-100, CUB, Stanford-Cars, and FGVC-Aircraft) show consistent gains over GCD baselines, achieving state-of-the-art performance. Project page: this https URL
41. 【2608.25675】Deep Learning Segmentation of Diffusion-Weighted MRI Acute Ischaemic Stroke: A Pragmatic Evaluation Across Three Datasets
链接：https://arxiv.org/abs/2608.25675
作者：Atle Bjørnerud,Till Schellhorn,Thor H. Skattør,Terje Nome,Jon André Ottesen,Anne Hege Aamodt,Bradley J MacIntosh
类目：Computer Vision and Pattern Recognition (cs.CV)
关键词：Diffusion-weighted MRI, accurate AIS lesion, AIS lesion segmentation, AIS lesion, segment AIS lesions
备注： 19 pages, 8 figure and 4 tables
点击查看摘要
Abstract:Objective: Diffusion-weighted MRI (DWI-MRI) is the gold standard for visualizing and quantifying acute ischaemic stroke (AIS). Although deep learning methods can accurately segment AIS lesions, the optimal image inputs and model architecture remain uncertain. We evaluated whether accurate AIS lesion segmentation can be achieved using a pragmatic deep learning approach with minimal preprocessing and clinically feasible inference times. Materials and Methods: Self-configured nnU-Net models were trained on 1,744 DWI cases from local, national, and open-access datasets and tested on 436 cases. Four experimental conditions were evaluated using five-fold cross-validation: with or without brain extraction and using either DWI alone or DWI plus apparent diffusion coefficient (ADC) images as inputs. Two architectures were compared: the baseline nnU-Net (base) and a residual encoder nnU-Net (ResEnc). Performance was benchmarked against the DeepISLES ensemble model from the 2022 ISLES challenge. Results: In the test set (n=436), the base model achieved a median (IQR) Dice similarity coefficient (DSC) of 0.84 (0.19). For the base model, only two of six pairwise comparisons between input configurations showed significant differences. ResEnc produced small but significant improvements in DSC compared with the base model for DWI, DWI+brain extraction, and DWI+ADC inputs (all p0.02), but not for DWI+ADC+brain extraction (p0.50). The base model significantly outperformed DeepISLES, particularly in patients with smaller infarct volumes (signed-rank test, p0.01). Conclusions: A baseline nnU-Net trained on DWI alone, without preprocessing, enabled fast and accurate AIS lesion segmentation. This streamlined approach may facilitate clinical research and support acute stroke imaging workflows
Comments:
19 pages, 8 figure and 4 tables
Subjects:
Computer Vision and Pattern Recognition (cs.CV)
Cite as:
arXiv:2608.25675 [cs.CV]
(or
arXiv:2608.25675v1 [cs.CV] for this version)
https://doi.org/10.48550/arXiv.2608.25675
Focus to learn more
arXiv-issued DOI via DataCite (pending registration)
Submission history From: Atle Bjørnerud PhD [view email] [v1]
Wed, 26 Aug 2026 11:53:40 UTC (909 KB)
42. 【2608.25653】owards Purified Multi-Label Test-Time Adaptation of Vision-Language Models
链接：https://arxiv.org/abs/2608.25653
作者：Yiwen Liang,Hui Chen,Yizhe Xiong,Mengyao Lyu,Yuhan Cao,Zijia Lin,Shuaicheng Niu,Sicheng Zhao,Jungong Han,Guiguang Ding
类目：Computer Vision and Pattern Recognition (cs.CV)
关键词：effectively mitigating distribution, multi-label test-time adaptation, Test-time adaptation, mitigating distribution shifts, effectively mitigating
备注： Accepted by ECCV 2026
点击查看摘要
Abstract:Test-time adaptation (TTA) has been widely explored in single-label recognition, effectively mitigating distribution shifts, especially when combined with vision-language models. However, real-world images often contain multiple objects, while the more practical multi-label test-time adaptation (MLTTA) has received little attention so far. Recent cache-based TTA methods have shown promising efficiency and effectiveness, yet directly extending them to multi-label scenarios suffers from a one-to-many mapping problem: a shared global representation entangling co-occurring objects is stored as class-wise cache prototypes, inducing dominant-label bias and compromised cache calibration. While introducing region-level cues helps isolate class-specific evidence, such regional evidence can also be unreliable under distribution shifts, making its identification and utilization non-trivial. To address these issues, we introduce PuRF, a novel PuRiFication-driven cache-based method for multi-label test-time adaptation of vision-language models. Specifically, PuRF first performs region purification to identify reliable regions, providing comprehensive regional cues for multi-label recognition and enabling fine-grained alignment. Based on these purified regions, PuRF conducts cache purification to enhance cache representation and adaptability, where episodic purification builds a discriminative region-based cache, and temporal refreshing further promotes long-term cache adaptability. Experiments demonstrate that PuRF consistently outperforms state-of-the-art methods, achieving a notable 4.05% mAP improvement on ViT-B/32 across five datasets.
43. 【2608.25652】Diffusion Transformers for Roof Graph Synthesis and Reconstruction
链接：https://arxiv.org/abs/2608.25652
作者：Daniel Panangian,Ksenia Bittner
类目：Computer Vision and Pattern Recognition (cs.CV)
关键词：present RoofDiT, roof, reconstruction, direct reconstruction objectives, Abstract
备注： Pre-review manuscript. Accepted at the ICPR 2026 Workshop on Pattern Recognition in Remote Sensing (PRRS)
点击查看摘要
Abstract:We present RoofDiT, a generative framework for 2D roof graph synthesis and reconstruction. Roofs are compactly described as planar graphs of junctions and structural edges, but existing methods often rely on fixed geometric rules or direct reconstruction objectives. RoofDiT instead models roof structures directly as vertex-edge graphs and learns a conditional generative prior over their geometry and connectivity. Our framework follows a two-stage design: a diffusion transformer generates roof vertices, and an edge prediction module infers the corresponding graph topology. To improve geometric fidelity, RoofDiT combines relative geometry-aware attention with footprint and aerial-image conditioning, while using an alignment regularizer to encourage common horizontal, vertical, and diagonal roof patterns. The same model supports unconditional generation, footprint-conditioned synthesis, and image-guided reconstruction by changing the conditioning signal. Experiments show improved graph generation quality over a diffusion baseline, favorable performance against a straight-skeleton prior in the footprint-conditioned setting, and the highest edge F1 among compared methods for image-guided reconstruction.
44. 【2608.25648】MAMA-FLUX.2: Image-to-Image Synthesis of Post-Contrast Breast DCE-MRI for the MAMA-SYNTH Challenge
链接：https://arxiv.org/abs/2608.25648
作者：Kamil Kwarciak,Marek Wodzinski
类目：Computer Vision and Pattern Recognition (cs.CV)
关键词：gadolinium-based contrast agents, Dynamic contrast-enhanced breast, requires gadolinium-based contrast, contrast breast MRI, breast MRI
备注： 9 pages, 3 figures, 1 table
点击查看摘要
Abstract:Dynamic contrast-enhanced breast MRI is central to cancer diagnosis and monitoring, but requires gadolinium-based contrast agents. In this work, we address pre-to-post contrast breast MRI synthesis for the MAMA-SYNTH challenge. We propose MAMA-FLUX.2, a conditional latent flow-matching approach based on FLUX.2-Klein-4B. The pre-contrast image is encoded as spatial conditioning, while the model predicts the flow field associated with the post-contrast target latent. To adapt the pretrained model efficiently, we use LoRA fine-tuning and introduce a regional training objective combining global flow matching, tumor-region supervision, and stable foreground regularization. We further investigate LoRA rank, intensity windowing, and regional loss weights on axial slices, prioritizing clinically relevant tumor-focused metrics. Our ablation study shows that moderate tumor and stable-foreground weighting improves the trade-off between image fidelity and tumor-region accuracy. The final model achieves the best overall balance with LoRA rank/$\alpha=64/64$, $\mathrm{MHA}_{\max}=25$, $\lambda_{\mathrm{tumor}}=0.25$, and $\lambda_{\mathrm{stable}}=0.1$. These results demonstrate that compact pretrained rectified-flow transformers can be adapted for contrast-enhanced MRI synthesis using parameter-efficient fine-tuning and task-aware regional losses.
45. 【2608.25630】SeVeR: Selective Visual Exposure and Retrieval for 3D Medical Image Question Answering
链接：https://arxiv.org/abs/2608.25630
作者：Yaojun Hu,Danyang Tu,Yang Liu,Jiajin Zhang,Wei Fang,Zhiqiang Liu,Chunlai Dong,Yingda Xia,Haochao Ying,Jian Wu,Ling Zhang
类目：Computer Vision and Pattern Recognition (cs.CV)
关键词：Volumetric medical VQA, medical VQA requires, VQA requires reasoning, repeated anatomical regions, modalities provide diverse
备注：
点击查看摘要
Abstract:Volumetric medical VQA requires reasoning over long and redundant 3D visual token sequences, especially in multi-sequence MRI where complementary modalities provide diverse diagnostic cues but expose the decoder to many repeated anatomical regions. To investigate reasoning under multi-sequence visual redundancy, we first introduce BreMRIs-VQA, a clinically curated breast MRI benchmark with 1.19M QA pairs from 71.0K sequences and 12.9K patients, covering both free-text and multiple-choice questions. We further propose SeVeR, a selective visual exposure framework that compresses dense volumes into modality-wise prototypes and retrieves complementary multi-level evidence with change-aware gated attention during decoding, trained with a marginal-utility self-consistency objective that suppresses unhelpful retrieval. Experiments on BreMRIs-VQA and public benchmarks show that SeVeR improves both discriminative and generative performance while exposing substantially fewer visual tokens.
46. 【2608.25622】Plans You Can Check: Verifier-Grounded Learning of an Open-Weight Planner for Executable Video-Editing
链接：https://arxiv.org/abs/2608.25622
作者：Haoyu Wang,Cheng Feng,Liuyang Bian,Ruiyang Huang,Lei Wei,Yafei Wen,Xiaoxin Chen,Xiaoying Tang
类目：Computer Vision and Pattern Recognition (cs.CV); Computation and Language (cs.CL)
关键词：Practical video editing, Practical video, pixel generation, editor must turn, hard constraints
备注： Accepted to the Main Conference of EMNLP '26
点击查看摘要
Abstract:Practical video editing is not only pixel generation: an editor must turn a brief, a clip pool, music metadata, and hard constraints into an executable timeline. We study this decision layer as \emph{executable video-editing planning} and introduce RefineCut, which, unlike workflow systems that wrap a prompted frontier model, trains a compact open-weight planner for it. The planner edits a typed timeline through structured patches covering clip selection, trimming, ordering, transitions, and duration and music alignment; a deterministic verifier applies each patch and checks it against an explicit constraint ledger. Because editing has no single ground-truth repair, we do not imitate teachers directly: RefineCut replays every multi-teacher branch through the verifier and keeps verifier-best repairs as supervision. A second stage, RefineCut-Evo, lets the student score its own repairs with the verifier and a task rubric and trains on high-margin preference pairs, so the final $8$B planner runs in a closed verifier loop with no teacher calls at inference. On RefineCut-Bench ($3{,}578$ tasks, $7{,}971$ captioned clips, $499$ music tracks, explicit ledgers), verifier-replayed distillation lifts the planner from $0.620$ to $0.858$ on the protocol-specific Video-Editing Score and RefineCut-Evo reaches $0.924$; the gain transfers to Llama-3.1-8B and GLM-4-9B, and in the same closed loop the $8$B planner matches or exceeds its frontier teachers. Code and RefineCut-Bench are publicly released; see the Data Availability statement.
47. 【2608.25609】On the Separation of Human and AI-Generated Images in CLIP Embedding Space
链接：https://arxiv.org/abs/2608.25609
作者：Andrea Asperti
类目：Computer Vision and Pattern Recognition (cs.CV)
关键词：AI-generated paintings spontaneously, paintings spontaneously separate, joint embedding distribution, previously unreported phenomenon, supervised objective designed
备注：
点击查看摘要
Abstract:We identify a previously unreported phenomenon in CLIP representations: human and AI-generated paintings spontaneously separate along the dominant principal directions of their joint embedding distribution, without any supervised objective designed to distinguish the two classes. Rather than exploiting this phenomenon for detection, our objective is to interpret it: we seek to identify the visual information underlying the separation and to trace it back from the embedding space to the image domain. We pursue this objective through a progressive investigation combining interpretable image representations with gradient-based inversion, used systematically as an experimental probe of the relationships identified in feature space. Robustness experiments and increasingly expressive statistical descriptors progressively rule out several intuitive explanations based on global image properties and simple local statistics, and point instead to distributed multiscale image structure. Multiscale scattering provides the most informative interpretable representation considered, but offers only a partial account of the phenomenon. Direct inversion provides a complementary and striking observation: substantial displacements along the dominant CLIP directions can be induced by image perturbations that remain nearly imperceptible to human observers, showing that the directions involved in the separation are highly sensitive to image variations with very low perceptual salience for humans. Taken together, these results reveal a significant difference between the visual evidence reflected in CLIP representations and that readily accessible to human perception, raising broader questions about the relationship between artificial and human vision and, ultimately, between artificial and human aesthetic judgment.
Subjects:
Computer Vision and Pattern Recognition (cs.CV)
MSC classes:
68T45 (Primary) 68T07, 62H25 (Secondary)
ACMclasses:
I.4.10; I.4.7
Cite as:
arXiv:2608.25609 [cs.CV]
(or
arXiv:2608.25609v1 [cs.CV] for this version)
https://doi.org/10.48550/arXiv.2608.25609
Focus to learn more
arXiv-issued DOI via DataCite (pending registration)</p>
48. 【2608.25608】When Should a Network Emit Geometry, and When Should It Detect It? Readout, Reconciliation, and Representation in Floorplan Vectorization
链接：https://arxiv.org/abs/2608.25608
作者：He Zhang
类目：Computer Vision and Pattern Recognition (cs.CV)
关键词：autoregressive coordinate sequence, assembling a graph, rasterized floorplan, floorplan can produce, emitting the geometry
备注： 21 pages, 4 figures. Code and benchmark: [this https URL](https://github.com/Cyprinus12138/fpvec-lab)
点击查看摘要
Abstract:A network trained to recover the walls, openings, and rooms of a rasterized floorplan can produce its output in two ways: by emitting the geometry as an autoregressive coordinate sequence, or by detecting it on dense junction and centerline heatmaps and assembling a graph. We compare the two readouts on the same trained network. On real scans (CubiCasa5K) detection is better on every wall measure (+2.7 wall F1 at tolerance 0.05, +5.1 at 0.015; paired bootstrap intervals exclude zero), and reading an opening heatmap the decoder never used raises opening F1 by 2.6x without retraining. Within real scans the readout's advantage grows with plan size and reverses on small plans; on clean vector renders sequence decoding is better by 5 to 8 points where its training covered the render style, while under full domain shift the readout, given calibrated thresholds, stays ahead; neither ink density nor plan size explains the reversal. With matched data and recipe, a room-centric system with a reconciliation step and a wall-first sequence model reach comparable wall quality, so the output representation matters less than is usually assumed. A prior from the other family helps at the output but not at the input: deterministic fusion of the two outputs raises wall F1 by 7 points, whereas conditioning one model on the other's output gives no gain in three forms, including two ground-truth-content controls. We also provide an edit-cost metric that scores a draft by the human work needed to correct it, corrected CubiCasa5K annotations, and ResPlan-FP, a CC BY 4.0 benchmark of 16,998 plans with frozen splits and three baseline tracks. Code, the benchmark, and the corrected annotations are available at this https URL
49. 【2608.25601】A Dual-Transformer for Multi-Camera View Recommendation
链接：https://arxiv.org/abs/2608.25601
作者：Josep Cabacas-Maso,Carles Ventura,Ismael Benito-Altamirano
类目：Computer Vision and Pattern Recognition (cs.CV); Artificial Intelligence (cs.AI)
关键词：modern media production, Multi-camera systems, multi-camera editing, media production, systems are foundational
备注：
点击查看摘要
Abstract:Multi-camera systems are foundational to modern media production, and multi-camera editing is a critical task. This involves the proper selection of the appropriate camera view at each moment. In this paper, we propose a novel Dual-Transformer architecture with Cross-Attention that heavily outperformed the current SOTA models over the TVMCE dataset (TV Shows Multicamera Editing dataset). Our model decouples these tasks: (1) a dedicated temporal encoder first processes the sequence of past frames to build a rich memory of the recent history, and (2) the candidate camera views then act as queries to this memory via a cross-attention module, allowing each candidate to independently interrogate the historical context and find the most relevant information for its own evaluation. Our approach achieved 56.60% Precision@0.5, representing a substantial improvement over the prior best result of 37.16%. We further conducted an ablation study exploring the use of lightweight backbone architectures, where the SwinV2 backbone yielded the best performance, achieving 69.65% Precision@0.5. Using this best-performing configuration, we then investigated the feasibility of adapting the model to replicate the editing style of a specific human editor. To this end, we fine-tuned the model using varying proportions of the initial segment of a target video. Our results demonstrate that even with only 20% of the video used for fine-tuning, the model exhibited measurable improvements in Precision@0.5, indicating strong potential for data-efficient personalization of editing style adapted to each individual TV show or producer.
Subjects:
Computer Vision and Pattern Recognition (cs.CV); Artificial Intelligence (cs.AI)
Cite as:
arXiv:2608.25601 [cs.CV]
(or
arXiv:2608.25601v1 [cs.CV] for this version)
https://doi.org/10.48550/arXiv.2608.25601
Focus to learn more
arXiv-issued DOI via DataCite (pending registration)</p>
50. 【2608.25580】V-Rubrics: Visual Faithfulness via Rubric-Based Reinforcement Learning
链接：https://arxiv.org/abs/2608.25580
作者：Shulin Tian,Minglun Li,Yuhao Dong,Hao Ding,Jiarui Yao,Haiwen Diao,Jingkang Yang,Hongyuan Zhu,Ziwei Liu
类目：Computer Vision and Pattern Recognition (cs.CV); Artificial Intelligence (cs.AI)
关键词：single unsupported object, produce fluent answers, unsupported object, Vision-language models, produce fluent
备注： Proj page: [this https URL](https://shulin16.github.io/v-rubrics/)
点击查看摘要
Abstract:Vision-language models can produce fluent answers that are insufficiently grounded in the visual evidence: a single unsupported object, chart value, or intermediate inference can undermine an otherwise plausible response. We argue that this is a credit-assignment failure in multimodal post-training. Scalar outcome rewards indicate whether an answer is acceptable, but do not identify which visual facts are grounded, which reasoning steps are valid, or which instruction constraints are missed. We introduce Visual Rubrics-Based Reinforcement Learning, which decomposes reference responses into atomic propositions and scores generated answers along Visual Faithfulness (VF), Reasoning Consistency (RC), and Instruction Following (IF). The resulting rubric items provide structured partial credit and localize rubric credit when supporting evidence spans are available. We first obtain an SFT checkpoint by fine-tuning Qwen3-VL-8B-Instruct on the public OpenMMReasoner-SFT-874K corpus, adapting OpenMMReasoner's cold-start data recipe. We construct V-Rubrics 50K, a 50,248-example training set from 17 visually grounded sources, by applying rule-based filters before deriving example difficulty from rejection-sampling scores and then annotating every example with Gemini-3-Pro under the same structured prompt and protocol. We train our model based on the same SFT checkpoint using component-wise, prefix-localized rubric credit. Experiments show that our rubricbased GRPO improves over both the shared SFT baseline and answer-only GRPO, with the largest gains on knowledge-oriented and visually grounded reasoning benchmarks. The results show rubrics as a useful reward abstraction for visual post-training.
51. 【2608.25575】MLLMCLIP: Feature-Level Distillation of MLLM for Robust Vision-Language Representations
链接：https://arxiv.org/abs/2608.25575
作者：Jongsuk Kim,Qiyu Wu,Zhuoyuan Mao,Hiromi Wakaki,Junmo Kim,Yuki Mitsufuji
类目：Computer Vision and Pattern Recognition (cs.CV); Artificial Intelligence (cs.AI)
关键词：Pretrained vision-language models, large language models, Large Language Model, fail at compositionality, relational structures
备注： EMNLP 2026 Main Conference
点击查看摘要
Abstract:Pretrained vision-language models such as CLIP excel at zero-shot recognition but often fail at compositionality, particularly attribute-object and relational structures. Recent studies mitigate this issue by augmenting training with synthetic hard negatives generated by a cascade of large language models and text-to-image models, which incurs substantial pipeline overhead. We instead propose MLLMCLIP, a heterogeneous distillation framework that transfers multimodal knowledge directly from a generative Multimodal Large Language Model (MLLM) teacher into a discriminative CLIP student, bypassing synthetic data entirely. To bridge the architectural mismatch between the two paradigms, we introduce an attention-based per-layer token selection and a CKA-based distillation loss. Compared to prior CLIP-enhancement methods, MLLMCLIP achieves state-of-the-art compositional accuracy while delivering consistent gains on standard zero-shot classification and image-text retrieval, showing that feature-level distillation strengthens both compositional and general vision-language representation capability.
52. 【2608.25568】CrossMambaTuning: Synergistic Spatial and Cross-Layer Adaptation for Machine Vision Compression
链接：https://arxiv.org/abs/2608.25568
作者：Haobo Xiong,Shaobo Liu,Kai Liu,Chongyang Ding
类目：Computer Vision and Pattern Recognition (cs.CV); Artificial Intelligence (cs.AI)
关键词：learned image compression, attracted growing attention, adapting pretrained learned, pretrained learned image, reduce deployment cost
备注： Accepted by ACM MM26
点击查看摘要
Abstract:To reduce deployment cost and retraining overhead, adapting pretrained learned image compression (LIC) models to downstream machine vision tasks has attracted growing attention. However, existing methods typically insert fine-tuning modules independently into frozen backbones, lacking explicit mechanisms for cross-layer coordination. To address this limitation, we propose a novel framework named CrossMambaTuning, which integrates State Space Models with cross-layer interaction mechanisms for parameter-efficient fine-tuning. Specifically, we design an efficient Mamba adapter equipped with task-specific prompts and multi-scale branching to precisely capture both local features and global dependencies. Furthermore, we introduce a Scale-Invariant Cross-Layer Adapter (SICA) utilizing a parameter-sharing strategy to fuse task information across different scales and reduce redundancy. Extensive experiments demonstrate that CrossMambaTuning achieves state-of-the-art (SOTA) performance on multiple machine vision tasks, reducing parameter overhead by 72\% compared to SOTA methods. Code is available at this https URL.
53. 【2608.25559】AdaVDR: Adaptive Tool Use and Reflection for Video Deep Research
链接：https://arxiv.org/abs/2608.25559
作者：Xintong Zhang,Xiaomeng Fan,Shilin Yan,Ekko He,Zicheng Liu,Zijian Zou,Guannan Zhang,Yuwei Wu,Zhi Gao,Hongwei Xue
类目：Computer Vision and Pattern Recognition (cs.CV); Artificial Intelligence (cs.AI)
关键词：open Web, research answers complex, Video deep research, answers complex questions, deep research answers
备注：
点击查看摘要
Abstract:Video deep research answers complex questions by jointly understanding video content and retrieving external knowledge from the open Web. However, diverse questions and videos require different tool-use strategies, and inappropriate tool calls can produce incorrect results. Uncertain grounding and retrieval also make unnecessary interactions costly and error-prone, increasing latency and reasoning errors. To address these challenges, we propose AdaVDR, an adaptive video deep research agent with adaptive tool invocation and reflection. AdaVDR selects tools according to the task and its capabilities, and backtracks only when unreliable intermediate results require correction. To enable these capabilities, we develop a video deep research data construction pipeline. We first discover retrieval-relevant events and entities in diverse videos and acquire detailed information through grounding and external retrieval to construct high-quality QA pairs. For each QA, task-specific prompts organize the information acquisition process into a tool-use trajectory, allowing different question and video types to follow different grounding and retrieval strategies. We further introduce model-conditioned tool necessity filtering, which evaluates tool calls against the target model's video understanding and internal knowledge, removing tools or tool chains the model can bypass. This yields trajectories tailored to the target model's video understanding capability and knowledge. Using this pipeline, we construct training data and VDR-EE, a benchmark covering entity-centric and event-centric questions. We perform supervised fine-tuning followed by reinforcement learning with a redundancy-aware reward to strengthen adaptive tool invocation and reflection. Experiments show that our method performs best among the evaluated open-source models on VDR-EE and substantially improves over its base models on VideoDR.
54. 【2608.25539】CropCop: An Auditable 120-Class Plant-Health Model from Benchmark Reconstruction to a Quantised Runtime Artifact
链接：https://arxiv.org/abs/2608.25539
作者：Rana Muhammad Ahmed,Sabahat Abbas
类目：Computer Vision and Pattern Recognition (cs.CV); Machine Learning (cs.LG)
关键词：long-tailed label space, duplicated image families, label space, precise while resting, resting on duplicated
备注： 25 pages, 6 figures, and 15 tables. Includes benchmark-audit, model-retention, runtime-fidelity, and reproducibility appendices
点击查看摘要
Abstract:A plant-health score can appear precise while resting on duplicated image families, a long-tailed label space, or a runtime file that was never evaluated. We present CropCop, a closed-set recognition system spanning 120 operational plant-health classes and an evidence chain from corpus reconstruction to direct execution of the final quantised artifact. Starting from 117,546 audited images, we rejected the inherited partition after confirming 3,233 duplicate relationships across split boundaries and froze a 109,107-image benchmark with zero crossings among the audited trusted leakage groups and a 151.7 largest-to-smallest class ratio. A fully fine-tuned DINOv3 ConvNeXt-Tiny reference achieved 98.51% accuracy and 96.87% macro-F1 on the locked internal test. A compact MobileNetV4 Conv-Medium derivative achieved 98.46% accuracy and 96.27% macro-F1 without being presented as evidence for a new distillation method. Validation-only post-training quantisation selected dynamic activations with per-channel weights, and the final 22.60 MiB ExecuTorch/XNNPACK PTE achieved 98.46% accuracy and 96.23% macro-F1 when executed directly. Only six of 16,363 top-1 decisions changed between the converted INT8 graph and the PTE, while paired analysis showed a modest class-balanced loss; an exploratory post hoc fruit-label slice localized a larger recall decline than aggregate accuracy revealed. CropCop establishes strong leakage-controlled internal recognition and software-runtime fidelity; it does not establish performance on unseen farms, camera pipelines, or physical Android hardware.
55. 【2608.25529】Video-IFBench: Evaluating Instruction Following of Multimodal LLMs in Video Understanding Scenarios
链接：https://arxiv.org/abs/2608.25529
作者：Hongbo Liu,Peixian Chen,Sihan Liu,Peiyuan Zhang,Kai Zou,Dian Zheng,Xiaoxing Hu,Yuhao Dong,Mengdan Zhang,Yunhang Shen,Haoyu Cao,Wei Liu,Weibo Gu,Xing Sun,Shengjie Zhao
类目：Computer Vision and Pattern Recognition (cs.CV)
关键词：Multimodal Large Language, Large Language Models, Multimodal Large, Large Language, shown strong performance
备注：
点击查看摘要
Abstract:Multimodal Large Language Models (MLLMs) have shown strong performance in video understanding. However, their ability to follow instructions in this domain remains under-explored. Real-world video understanding requires models not only to interpret video content correctly, but also to satisfy diverse user-specified constraints. Existing benchmarks focus primarily on task accuracy rather than instruction adherence, leaving this capability insufficiently evaluated. To address this gap, we introduce Video-IFBench, a comprehensive benchmark for evaluating instruction following in video understanding, where models must satisfy diverse user-specified constraints, including those grounded in visual and audio content. We develop an instruction taxonomy with four templates, including single-task, multi-task, selection, and nested instructions, covering 32 task types and 39 manually designed constraint categories spanning both semantic and format requirements. To reduce annotation cost, we build a semi-automatic data construction pipeline that combines MLLMs, programmatic processing, and human verification, resulting in 1.5K samples. We conduct a large-scale evaluation of more than 20 recent MLLMs and show that video instruction following remains challenging for current models, especially for instructions with many constraints, semantic constraints, or complex conditional structures that require selecting the correct branch or path based on video content. We hope our work will facilitate future research on instruction following in video understanding scenarios.
56. 【2608.25520】Asymmetric Cross-Modal Fine-Grained Visual Categorization: ACF-Net and the BirdPro Benchmark
链接：https://arxiv.org/abs/2608.25520
作者：Bohan Deng,Shuo Ye,Zitong Yu
类目：Computer Vision and Pattern Recognition (cs.CV)
关键词：Fine-Grained Visual Categorization, jointly leveraging visual, identify fine-grained categories, asymmetric cross-modal FGVC, Visual Categorization
备注： Accepted by the 9th Chinese Conference on Pattern Recognition and Computer Vision (PRCV 2026). 15 pages, 5 figures
点击查看摘要
Abstract:Audio-visual cross-modal Fine-Grained Visual Categorization (FGVC) aims to identify fine-grained categories by jointly leveraging visual and auditory information. However, FGVC under asymmetric cross-modal scenarios has received limited attention, where paired video and audio are not strictly synchronized and may not even correspond to the same individual or moment. Such weak and ambiguous cross-modal correspondence poses substantial challenges to effective representation learning and modality alignment. To address these issues, we propose ACF-Net, a novel optical flow-guided framework for asymmetric audio-visual fine-grained learning. ACF-Net consists of two key modules: Optical Flow-Guided Motion (OFGM) and Asymmetric CrossModal Adaptive Fusion (ACAF). OFGM captures motion-sensitive visual cues and suppresses irrelevant background interference, thereby enhancing discriminative dynamic representations in videos. ACAF estimates modality reliability under weakly matched audio-video pairs and performs uncertainty-aware adaptive fusion to improve category-level recognition robustness. To support research on asymmetric cross-modal FGVC, we further construct BirdPro, a new bird-oriented audio-visual benchmark, since existing datasets often lack large-scale category-level audio-video associations under non-strict temporal and instance correspondence. BirdPro contains 1,919 audio recordings and 11,965 videos covering 194 bird species. Extensive experiments show that ACF-Net achieves the best results compared with representative baseline methods, outperforming the strongest baselines by 2.97% and 1.92% in the fused and mismatched settings, respectively.
57. 【2608.25515】OpenVeinNet: Robust Open-Set Finger Vein Verification with Dynamic Snake Convolution and Graph Learning
链接：https://arxiv.org/abs/2608.25515
作者：Sushrut Patwardhan,Raghavendra Ramachandra
类目：Computer Vision and Pattern Recognition (cs.CV)
关键词：promising biometric modality, Dynamic Snake Convolution, Finger vein verification, Finger vein, patterns are internal
备注： preprint: Accepted for publication in IEEE Transactions on Biometrics, Behavior, and Identity Science (T-BIOM)
点击查看摘要
Abstract:Finger vein verification is a promising biometric modality for secure authentication because vascular patterns are internal, difficult to observe externally, and relatively resistant to presentation attacks. However, reliable verification remains challenging in open-set settings, where test identities are unseen during training and non-enrolled probes must be rejected at inference. This paper presents OpenVeinNet, a finger vein verification framework designed for cross-dataset and open-set evaluation. The proposed model combines Dynamic Snake Convolution with graph-based feature modelling. Dynamic Snake Convolution extracts local curvilinear and tubular vein structures using adaptive sampling, while the graph convolutional backbone models long-range topological relationships between vein regions. To improve the discriminative quality of the embedding space, we introduce a Centroid Angular Hybrid Loss, which jointly encourages intra-class compactness and inter-class angular separation for cosinesimilaritybased verification. Experiments are conducted on five public finger vein datasets: FV-300, MMCBNU, FV-USM, PolyU, and VERA. The method is evaluated using leaveonedatasetout training under both enrolmentbased unknownrejection and fullsubject verification protocols, and is compared with handcrafted and recent deep learning-based baselines. The results show that OpenVeinNet achieves strong cross-dataset generalisation, consistently low equal error rates, and competitive true accept rates at fixed false accept rate operating points. Ablation studies further confirm the individual and combined contributions of adaptive tubular feature extraction, graph-based relational modelling, and the proposed loss function. These findings indicate that explicitly modelling local vein geometry, global vascular relationships, and angularly compact embeddings is effective for openset finger vein verification.
58. 【2608.25495】Pose-Anchored Optical Flow for Low-Latency Human Action Anticipation in Human-Robot Teaming
链接：https://arxiv.org/abs/2608.25495
作者：Lewis de Zoete Grundy,Chris McCarthy,Christopher Fluke
类目：Computer Vision and Pattern Recognition (cs.CV); Artificial Intelligence (cs.AI)
关键词：respond safely, execution in order, order to respond, HRI, human
备注： Presented at the 35th IEEE International Conference on Robot and Human Interactive Communication (RO-MAN 2026) in Kitakyushu, Japan
点击查看摘要
Abstract:Human-robot interaction (HRI) requires robots to interpret human actions early in their execution in order to respond safely, efficiently, and naturally. However, many existing approaches to human action recognition rely either on sparse skeletal representations, which lack fine-grained motion cues, or dense optical flow, which can be computationally expensive for low-latency perception pipelines. In this paper, we propose PoseOFF, a pose-anchored optical flow representation that captures local motion information around human joints to support earlier human intent understanding. By conditioning motion feature extraction on human pose, PoseOFF encodes localised motion dynamics at semantically meaningful body locations, forming a structured motion representation that is explicitly aligned with human kinematics. We evaluate PoseOFF across multiple benchmark datasets and backbone architectures for action anticipation, demonstrating consistent improvements in recognition accuracy, particularly at early observation ratios. Our results show that PoseOFF enables models to achieve comparable or improved performance while observing less of the action sequence, highlighting its effectiveness for early prediction. Importantly, these gains are achieved without requiring full-frame motion processing, making the approach practical for real-time and resource-constrained settings. These findings suggest that pose-centred motion representations such as PoseOFF can enhance the ability of interactive robot systems to infer human actions earlier, supporting more responsive and anticipatory behaviour in human-robot interaction scenarios.
59. 【2608.25493】SMART: MLLM-guided Temporal Alignment for Unifying Sign Language Recognition and Spotting
链接：https://arxiv.org/abs/2608.25493
作者：Eunjee Choi,JungHoon Sung,Seongwhan Cho,Chu Xin,Younggeun Choi
类目：Computer Vision and Pattern Recognition (cs.CV)
关键词：Continuous sign language, recognize gloss sequences, Continuous sign, aims to recognize, sequences from unsegmented
备注：
点击查看摘要
Abstract:Continuous sign language recognition (CSLR) aims to recognize gloss sequences from unsegmented sign videos under weak sequence-level supervision. However, existing methods rely on sentence-level gloss annotations, providing limited temporal and semantic guidance for fine-grained representation learning. Conventional video-text alignment also requires large batch sizes, making it inefficient for memory-intensive sign language video training. In this work, we propose SMART, an MLLM-guided temporal alignment framework for joint sign recognition and spotting. SMART uses MLLMgenerated motion descriptions as auxiliary semantic cues and performs stable videotext alignment under small-batch training. To improve temporal representation learning, we introduce a Multi-Scale Temporal Adapter that models temporal interactions during transformer encoding. For dense temporal localization, SMART incorporates CSFormer, a CSLR-guided spotting module that injects recognition-derived gloss evidence into a boundary-aware spotting network. This unified framework enables CSLR features to benefit spotting, while spotting supervision complements weak CTC-based recognition. Experiments on four sign language benchmarks, including PHOENIX14-T, CSL-Daily, Large-scale KSL, and Disaster and Safety KSL datasets, demonstrate the effectiveness of SMART across both recognition and spotting tasks.
60. 【2608.25485】Semi-Supervised Adaptation of Vision-Language Models for Image Classification
链接：https://arxiv.org/abs/2608.25485
作者：Mohamed L. Mekhalfi,Mohamad M. Al Rahhal,Yakoub Bazi,Salah E. Khenfer,Mingdeng Shi,Hua Zou,Mansour Zuair
类目：Computer Vision and Pattern Recognition (cs.CV)
关键词：handling natural images, shown sig, nificant potential, natural images, satellite imagery
备注：
点击查看摘要
Abstract:Vision-language models like CLIP have shown sig- nificant potential in handling natural images, yet their perfor- mance is often limited by the distinct characteristics of satellite imagery. While parameter-efficient adaptation techniques exist, their efficacy is frequently limited by the scarcity of annotated samples. In this letter, we propose Self-Evolutionary CLIP (SE- CLIP), a semi-supervised framework designed for recursive label mining in scene classification. The approach follows a dual-phase pipeline, where an initial warm-up on a few annotated seeds is followed by a recursive discovery phase that iteratively identifies high-confidence samples from unlabeled pools. To maintain the integrity of the evolving support set, we employ a class-balanced selection strategy that prevents the model from being dominated by easily learned categories. Results on the UCM and NWPU benchmarks indicate that SE-CLIP significantly outperforms existing semi-supervised approaches. The framework provides a viable solution for adapting VLMs to the remote sensing domain with minimal human intervention.
61. 【2608.25483】Gaussian Splatting Underwater: A Controlled Cross-Regime Study
链接：https://arxiv.org/abs/2608.25483
作者：Olaya Álvarez-Tuñón,Stella Graßhof
类目：Computer Vision and Pattern Recognition (cs.CV)
关键词：rarely uniform, environment is challenging, particles suspended, scatter and diffuse, Gaussian splatting
备注：
点击查看摘要
Abstract:The underwater environment is challenging for 3D reconstruction, because particles suspended in the water scatter and diffuse light, turbidity varies, absorption depends on wavelength, and illumination is rarely uniform. Methods based on Gaussian splatting have generally been developed for conditions that allow good image quality, and have primarily been tested on relatively shallow water. This paper examines how well Gaussian splatting performs across publicly available underwater datasets representing different degrees of turbidity, loss of illumination, and colour attenuation, together with an industrial survey. Five systems with public code are run under one protocol, with shared poses, initialisation, budget, and evaluator, to establish their relative advantages, disadvantages, and limitations. What these methods can do turns out to depend more on the setup than on the architecture. Water clarity binds upstream of rendering, since structure-from-motion registers 99.5 \% of frames in clear water and 0.0 \% at 12 NTU. Illumination geometry decides whether a medium model helps at all: under an artificial light that moves with the camera, medium-blind splatting beats both medium-aware systems. On the survey the benchmark's photometric leader comes last, beaten on geometry by a restoration pre-pass in front of vanilla 3DGS---and none of it is visible in the scores the field reports. Scene builds, per-run configurations, and evaluation code are released at this https URL
62. 【2608.25480】DeCO: Discriminative Evidence Composition for Fine-Grained Dataset Distillation
链接：https://arxiv.org/abs/2608.25480
作者：Chuixuan Fan,Guang Li,Shijie Wang,Dongzhan Zhou,Baoli Sun,Takahiro Ogawa,Miki Haseyama,Zhihui Wang
类目：Computer Vision and Pattern Recognition (cs.CV); Artificial Intelligence (cs.AI)
关键词：large training set, compact synthetic set, Dataset distillation compresses, training set, synthetic set
备注：
点击查看摘要
Abstract:Dataset distillation compresses a large training set into a compact synthetic set while preserving its downstream utility. However, existing methods primarily preserve global image statistics and may overlook the localized evidence essential for fine-grained visual classification (FGVC), such as object parts, subtle textures, and region-specific structures. We formulate fine-grained dataset distillation as budgeted discriminative-evidence preservation and propose Discriminative Evidence Composition (DeCO). DeCO uses attention rollout from a pretrained TransFG teacher to identify informative patches, applies spatial diversification to reduce redundant coverage, and organizes the resulting regions into class-wise evidence banks. Multiple same-class regions are then packed into compact grid-composed images. The teacher is used only for dataset construction, whereas downstream students are trained with standard hard-label supervision without teacher logits. Experiments on CUB-200-2011, FGVC-Aircraft, and Stanford Cars show that DeCO consistently outperforms representative coreset and dataset-distillation baselines under different IPC budgets.
63. 【2608.25479】4DStreamCtrl: Interactive Video Generation with Online 4D Control
链接：https://arxiv.org/abs/2608.25479
作者：Shiqian Li,Chenguo Lin,Zhiguang Liu,Yu Tang,Jiarong Ou,Rui Chen,Yixin Zhu
类目：Computer Vision and Pattern Recognition (cs.CV); Artificial Intelligence (cs.AI)
关键词：Generative video models, Generative video, indistinguishable from reality, synthesize footage, footage nearly indistinguishable
备注： 23 pages
点击查看摘要
Abstract:Generative video models now synthesize footage nearly indistinguishable from reality. Their promise as interactive tools hinges on fine-grained control of how objects and the camera move over time, yet each existing approach captures only part of this: camera-parameter methods steer the viewpoint but cannot move objects, 2D-trajectory methods act in the image plane and ignore depth and occlusion, and recent 3D methods add geometry but run only offline at a fixed length. In particular, none combines 3D-consistent control of both camera and objects with real-time, streaming generation. Here we show that camera motion, object trajectories, and depth can be unified into a single 3D point-track representation, from which one model performs joint camera and object control, depth editing, and motion transfer in a single forward pass. To learn this interface at scale, we mine in-the-wild video for 3D motion supervision, yielding OpenVidHD-Motion3D, and encode it with a lightweight Geometric Motion Head that plugs into a pretrained video diffusion model. Because this encoder is temporally separable, we distill the model into a causal streaming student that generates arbitrarily long video in four denoising steps at memory independent of length. This unified design surpasses prior camera-only, 2D, and offline-3D methods in motion-control precision while covering modalities they address only in isolation. 4DStreamCtrl runs at 20 FPS on a single high-end GPU for 480p video and stays temporally coherent over hundreds of frames, enabling, to our knowledge, interactive 4D-controllable streaming generation for the first time. More broadly, grounding generation in explicit 3D geometry with efficient causal inference points toward interactive world models with closed-loop spatiotemporal control, from controllable simulators to real-time visual imagination for embodied agents.
64. 【2608.25472】PAGS: Autofocusing Photoacoustic Tomography via Speed-of-Sound-Adaptive Gaussian Splatting
链接：https://arxiv.org/abs/2608.25472
作者：Jiarui Ge,Jintao Ma,Bangxu Fan,Jinyan Zhang,Xiaokang Yang,Shuai Na,Xiaoyun Yuan
类目：Computer Vision and Pattern Recognition (cs.CV); Medical Physics (physics.med-ph)
关键词：combines optical absorption, high-resolution deep-tissue imaging, optical absorption contrast, Photoacoustic computed tomography, computed tomography
备注： 13 pages, 6 figures
点击查看摘要
Abstract:Photoacoustic computed tomography (PACT) combines optical absorption contrast with acoustic detection for high-resolution deep-tissue imaging. A persistent challenge is that unknown speed-of-sound (SoS) heterogeneity changes acoustic time-of-flight, causing defocusing artifacts when reconstruction assumes a uniform SoS. Existing SoS-adaptive methods either rely on calibrated acoustic priors or optimize dense physical medium models, which becomes expensive and difficult to scale in 3D. We propose PAGS, a differentiable framework for blind autofocusing PACT via speed-of-sound-adaptive Gaussian splatting. PAGS represents the initial pressure field with sparse Gaussian photoacoustic (PA) sources and replaces explicit medium recovery with a compact anisotropic path-averaged SoS (ASoS) field parameterized by spherical harmonic probes. This latent propagation field directly controls source-to-transducer arrival-time alignment, while an analytic Gaussian acoustic projection maps the source representation to transducer signals efficiently. The resulting closed-loop signal-domain optimization jointly updates the Gaussian PA source parameters and the ASoS field from measured data, without calibrated SoS priors. Experiments on simulated and physical phantom data demonstrate improved reconstruction sharpness under heterogeneous acoustic media, robustness to sparse-view sampling, and computational benefits from the analytic Gaussian projection.
65. 【2608.25465】Automatic weld seam segmentation for industrial quality control: a comparison of RGB and polarimetric imaging with CNN and transformer architectures
链接：https://arxiv.org/abs/2608.25465
作者：Simone Garbin,Leonardo Venturoso,Marco Todescato
类目：Computer Vision and Pattern Recognition (cs.CV); Artificial Intelligence (cs.AI)
关键词：industrial production processes, special-purpose machinery cabins, welded assemblies remains, Visual inspection, production processes
备注：
点击查看摘要
Abstract:Visual inspection of welded assemblies remains one of the least automated stages in many industrial production processes, still depending largely on the experience of human operators and thus subject to inter-operator variability; the manufacturing of special-purpose machinery cabins, the setting of this study, is one representative case. This work evaluates the feasibility of automatic weld seam segmentation from RGB and polarimetric imagery, comparing controlled laboratory acquisitions with images captured under real, uncontrolled conditions. Convolutional neural network (CNN) architectures and transformer-based architectures are benchmarked under a unified, threshold-independent protocol, training each CNN with three random seeds to separate genuine effects from seed noise. In controlled RGB conditions, CNN models reach a mean mask mAP50 of up to 0.87, but drop to 0.22-0.48 under uncontrolled acquisition, showing that the acquisition setup is a first-order component of the inspection system. Polarimetric imaging with alignment-preserving geometric augmentation localizes previously unseen welds with a mean mask mAP50 up to 0.93: on par with, rather than ahead of, the best controlled-RGB result, but reaching that accuracy on uncontrolled RGB without requiring acquisition control. The clearest architectural finding concerns viewpoint robustness. In-distribution, transformers and CNNs are broadly comparable; but under a test-time viewpoint shift, the transformer models, and RF-DETR in particular, retain high accuracy while every CNN collapses. The gap holds across three seeds and a resolution-matched control, pointing to architecture rather than training resolution. Within the CNN family, capacity brings no reliable in-distribution gain once seed variance is accounted for: small CNNs suffice for fixed viewpoints, transformers for variable ones.
66. 【2608.25452】VGA-BenchV2: An Expanded Unified Benchmark and Multi-Model Framework for Evaluating Video Aesthetics and Generation Quality
链接：https://arxiv.org/abs/2608.25452
作者：Longteng Jiang,DanDan Zheng,Qianqian Qiao,Heng Huang,Huaye Wang,Yihang Bo,Bao Peng,Jingdong Chen,Jun Zhou,Xin Jin
类目：Computer Vision and Pattern Recognition (cs.CV); Artificial Intelligence (cs.AI)
关键词：extended human-aligned benchmark, improving video generation, extended human-aligned, framework for jointly, jointly evaluating
备注： IJCAI 2026
点击查看摘要
Abstract:We introduce VGA-BenchV2, an extended human-aligned benchmark and optimization framework for jointly evaluating and improving video generation quality and aesthetic value. Built upon VGA-Bench, VGA-BenchV2 preserves the original fine-grained taxonomy with two primary dimensions-Aesthetic and Generation-and 52 sub-dimensions. Guided by this taxonomy, we curate 1,016 diverse prompts and collect over 60,000 videos generated by 12 mainstream video generation models. More importantly, VGA-BenchV2 substantially expands human-labeled supervision by adding 36,000 task-level annotations, including 16,200 for aesthetic quality, 13,200 for aesthetic tagging, and 6,600 for generation quality, corresponding to 13.46x, 11.15x, and 1.55x scale-ups over VGA-Bench, respectively. Leveraging this enlarged annotation corpus, we develop a hybrid evaluator architecture consisting of VAQA-Net for continuous aesthetic scoring and two Qwen-based Large Vision-Language Model evaluators, VTag-Net and VGQA-Net, for aesthetic tagging and generation quality assessment. Extensive experiments demonstrate strong alignment with human judgments across diverse generation models. Beyond evaluation, VGA-BenchV2 further introduces an evaluation-to-optimization pipeline, where the learned aesthetic evaluator serves as a reward model for reinforcement learning-based generator fine-tuning. This closes the loop from benchmark construction and human supervision to automated evaluation and model optimization, enabling video generators to improve not only in realism but also in aesthetic quality and human preference alignment. Resources are available at this https URL.
67. 【2608.25435】Saliency-Depth Conditioning for Zero-Shot Segmentation of Communication-Tower Components in Cluttered UAV Imagery
链接：https://arxiv.org/abs/2608.25435
作者：Ali Lesani,Chul Min Yeum,Su-Min Kang
类目：Computer Vision and Pattern Recognition (cs.CV); Artificial Intelligence (cs.AI); Robotics (cs.RO)
关键词：limited instance-level annotations, Fine-grained segmentation, automated inspection, instance-level annotations, imagery is essential
备注：
点击查看摘要
Abstract:Fine-grained segmentation of communication-tower components in UAV imagery is essential for automated inspection, yet task-specific models are hard to develop due to limited instance-level annotations. Zero-shot segmentation models offer a promising alternative, but in cluttered scenes, visually similar background structures interfere with component localization, causing missed instances and false positives. We propose a model-agnostic saliency-depth foreground-conditioning strategy combining appearance-based saliency with monocular relative depth to construct a coarse tower prior and suppress irrelevant content. We integrate this module with Grounded-SAM and SAM 3, yielding SD-Grounded-SAM and SD-SAM 3. SD-Grounded-SAM further applies geometric and depth-aware box refinement before mask generation, while SD-SAM 3 relies on SAM 3's internal setup. On TOW-300, a dataset of 340 communication-tower UAV images, our strategy improves both baselines: SD-SAM 3 achieves the strongest instance-segmentation performance, while SD-Grounded-SAM produces fewer false positives. Ablations confirm complementary gains from saliency, depth, and box refinement, improving robustness in cluttered scenes.
68. 【2608.25418】Bootstrapping a 4D LiDAR Annotation Tool from Video Foundation Models
链接：https://arxiv.org/abs/2608.25418
作者：Jihun Kim,Hyun-Kurl Jang,Hyemin Yang,Jinnyeong Yang,Hyeokjun Kweon,Kuk-Jin Yoon
类目：Computer Vision and Pattern Recognition (cs.CV)
关键词：Progress, LiDAR, labels, video, annotation
备注： ECCV 2026 Workshop
点击查看摘要
Abstract:Progress in 4D LiDAR segmentation is bottlenecked by data. Assigning temporally consistent labels across sparse point cloud sequences is costly and hard to scale, and every new task or domain tends to demand fresh dense annotation. This motivates a simple question of whether high-quality LiDAR training data can be produced automatically, without any human labeling. To this end, we introduce LiDAR-SAM2, a framework that turns a 2D video foundation model, SAM2, into a scalable source of supervision for the 4D LiDAR domain. On the data side, it automatically generates temporally coherent LiDAR-level labels from SAM2 video masks through multi-view projection and spatio-temporal aggregation. On the modeling side, a tailored modality interface and a two-stage learning objective adapt SAM2's video segmentation kernel to spatio-temporal LiDAR structure, so that a single click per object yields a consistent mask track across the sequence. Trained with no human LiDAR annotation, LiDAR-SAM2 produces semantic and panoptic labels on SemanticKITTI that approach the quality of full human annotation from only a few points, and models trained on these labels approach the performance of full ground-truth supervision. This positions LiDAR-SAM2 as a scalable labeling tool that substantially reduces the annotation burden for 3D and 4D scene understanding.
69. 【2608.25412】AdaptiveEmbed: Sample-Adaptive Multi-Vector Representation for Multimodal Retrieval
链接：https://arxiv.org/abs/2608.25412
作者：Xinze Liu,Lei Yang,Dayan Wu,Hengjie Zhu,Zihao Zhang,Hanqi Wu,Tianzhu Hu,Peng Fu,Zheng Lin,Weiping Wang
类目：Computer Vision and Pattern Recognition (cs.CV)
关键词：fine-grained cross-modal information, capture fine-grained cross-modal, emph, multiple complementary embeddings, Multi-vector representations
备注：
点击查看摘要
Abstract:Multi-vector representations have emerged as an effective paradigm for multimodal retrieval, representing each sample with multiple complementary embeddings to capture fine-grained cross-modal information. However, existing approaches typically employ a fixed representation capacity, assigning the same number of vectors to all samples regardless of their individual retrieval demands. Such a fixed-capacity formulation overlooks the fact that different samples may require different amounts of representation capacity for effective retrieval. In this work, we introduce \emph{Sample-Adaptive Multi-Vector Representation} (SAMVR), a new problem setting for multimodal retrieval that studies how multi-vector representation capacity can be allocated at the sample level. Under SAMVR, each sample is represented by a \emph{content-adaptive embedding set} (CAES), whose capacity is determined according to the sample-specific retrieval utility of additional representation vectors. To instantiate SAMVR, we propose \emph{AdaptiveEmbed}, a unified framework for learning sample-adaptive multi-vector representations. AdaptiveEmbed learns structured multi-vector representations through \emph{Multi-Group Contrastive Learning} (MGCL) with the symmetric \emph{set-to-set similarity} (SetSim), and further employs \emph{Utility Policy Optimization} (UPO) to determine sample-specific representation capacity via \emph{Marginal Utility Allocation} (MUA). Experiments across multimodal retrieval benchmarks involving image, text, video, and audio show that sample-adaptive capacity allocation achieves overall better retrieval performance than fixed-capacity multi-vector representations, validating the effectiveness of SAMVR for multimodal retrieval. These results establish SAMVR as a viable formulation for adaptive capacity allocation in multi-vector multimodal retrieval.
70. 【2608.25401】PIVOT: A Multi-Trajectory Dataset and Testbed for Pose, Intrinsics, and Novel Viewpoint Evaluation in Real-World 3D Reconstruction
链接：https://arxiv.org/abs/2608.25401
作者：Mary Raymond
类目：Computer Vision and Pattern Recognition (cs.CV); Artificial Intelligence (cs.AI)
关键词：Neural radiance fields, Gaussian Splatting, reconstruction conditions cleaner, Neural radiance, related novel-view synthesis
备注：
点击查看摘要
Abstract:Neural radiance fields (NeRFs), 3D Gaussian Splatting (3DGS), and related novel-view synthesis methods are commonly evaluated under capture and reconstruction conditions cleaner than those encountered by robots, drones, and autonomous systems. Benchmarks often rely on reconstruction-friendly trajectories, optimized camera poses and intrinsics, and held-out views sampled from trajectories represented during training. These assumptions can obscure performance with measured poses, reusable camera calibration, and structurally different camera paths. We introduce PIVOT (Pose, Intrinsics and Viewpoint Oriented Testbed), a multi-trajectory dataset, processing pipeline, and evaluation framework for independently studying these factors. PIVOT captures each scene using diverse camera trajectories and retains, where available, both sensor-derived measured poses and COLMAP-optimized poses, together with calibrated and optimized camera intrinsics. It defines three benchmark families: (1) seen versus unseen trajectory novel-view generalization, (2) measured versus optimized pose sensitivity, and (3) calibrated versus optimized intrinsics sensitivity. We also introduce a directed pose-space Chamfer distance to quantify how well training poses cover an evaluation trajectory. PIVOT v1 contains five real-world scenes captured with a DJI Mini 4 Pro and provides an open processing and Nerfstudio-based evaluation toolchain. Benchmark results show a consistent quality gap between held-out views on represented trajectories and unseen trajectories, as well as substantial sensitivity to pose source and camera intrinsics.
Subjects:
Computer Vision and Pattern Recognition (cs.CV); Artificial Intelligence (cs.AI)
Cite as:
arXiv:2608.25401 [cs.CV]
(or
arXiv:2608.25401v1 [cs.CV] for this version)
https://doi.org/10.48550/arXiv.2608.25401
Focus to learn more
arXiv-issued DOI via DataCite (pending registration)</p>
71. 【2608.25386】Efficient Training with Foresight: Multi-Token Auxiliary Supervision for Autoregressive Image Generation
链接：https://arxiv.org/abs/2608.25386
作者：Guo Niu,Xiongfei Yao,Teng Wang,Nannan Zhu
类目：Computer Vision and Pattern Recognition (cs.CV)
关键词：shown strong potential, scalable high-fidelity synthesis, discrete token sequences, shown strong, strong potential
备注： Accepted by ACM Multimedia 2026 (ACM MM 2026)
点击查看摘要
Abstract:Autoregressive (AR) image generation has shown strong potential for scalable high-fidelity synthesis by modeling images as discrete token sequences. However, traditional next token prediction (NTP) continues to suffer from sparse and myopic supervision, insufficiently discriminative representations, and high training cost caused by dense computation over the full token sequence. To address these issues, we propose multi-token autoregressive (MTAR), a unified training framework that improves autoregressive image generation from three aspects: prediction objectives, representation regularization, and training efficiency. Specifically, MTAR introduces multi-token prediction (MTP) to alleviate the sparsity and myopia of traditional NTP by imposing joint supervision on multiple future tokens; employs token-level contrastive regularization (TCR) to explicitly enhance the separability of sampled token representations and thereby improve representation discriminability; and incorporates semantic dropping (SD) as a semantics-aware training acceleration strategy to reduce redundant computation on low-information tokens while preserving informative learning signals. All three components are applied only during training and introduce no additional overhead during autoregressive inference. On ImageNet, MTAR achieves a better balance between generation quality and training efficiency. Compared with LlamaGen, MTAR achieves up to 0.95 lower FID and 39\% faster training. Moreover, even with only 1/3 of the training iterations, it still attains performance comparable to or better than the baseline, substantially reducing training time.
72. 【2608.25375】GGSS: Geodesic-Gated Spherical Steering for Inference-Time Debiasing of Generative Vision-Language Models
链接：https://arxiv.org/abs/2608.25375
作者：Yiqun Sun,Junyu Chen,Pengfei Wei,Lawrence B. Hsieh
类目：Computers and Society (cs.CY); Computation and Language (cs.CL); Computer Vision and Pattern Recognition (cs.CV)
关键词：produce demographically biased, demographically biased outputs, human-centered settings, race or gender, Generative vision-language models
备注： Accepted to EMNLP 2026
点击查看摘要
Abstract:Generative vision-language models (VLMs) are increasingly used in human-centered settings, yet they can produce demographically biased outputs even when images differ only in controlled attributes such as perceived race or gender. However, existing inference-time debiasers were largely designed for static embeddings or CLIP-like models rather than generative VLMs. We propose GGSS---Geodesic-Gated Spherical Steering---a norm-preserving intervention that discovers a counterfactual bias subspace on the unit hypersphere, steers visual tokens along geodesic arcs, and uses an adaptive gate to focus correction on tokens that carry stronger demographic signal. We evaluate four generative VLMs against ten adapted inference-time debiasing baselines and prompt-based mitigation under a single operating-point protocol across categorical, pairwise, and occupation-gender bias tests, while also measuring general visual-language capability. GGSS achieves the lowest average bias on all four models, significant on three of four backbones under paired permutation tests, while preserving MMStar accuracy within +/- 0.6 p.p. of the unsteered baseline. Code is available at this https URL.
73. 【2608.25371】Capacity Overflow: A Blind Spot for Backdoor Attacks in Vision MoE
链接：https://arxiv.org/abs/2608.25371
作者：Xiaocheng Zou,Tiancheng Zheng,Xiaolin Xu,Ruyi Ding
类目：Computer Vision and Pattern Recognition (cs.CV); Cryptography and Security (cs.CR)
关键词：Vision Transformers efficiently, scaling Vision Transformers, Transformers efficiently, Vision Transformers, Vision MoE architectures
备注： 17 pages, 3 figures, ECCV2026
点击查看摘要
Abstract:Mixture-of-Experts (MoE) has become a prevalent paradigm for scaling Vision Transformers efficiently. To ensure computational scalability and prevent expert overload, Vision MoE architectures employ a capacity-bounded token dispatch mechanism, where each expert's processing budget depends on the inference batch size. This work identifies this batch-dependent behavior as an overlooked attack surface, and proposes a stealthy supply-chain backdoor attack that exploits this property through a three-phase framework. First, we inject a backdoor into an early MoE layer. Second, we train a neutralizer in a deeper MoE layer that suppresses the backdoor under normal capacity. Third, we configure a batch-adaptive capacity factor that preserves high capacity for small batches while reducing it for large batches, naturally disabling the neutralizer via token overflow at deployment-scale batch sizes. The attack remains in dormant mode during small-batch security audits and enters activation mode during large-batch deployment. Experiments on V-MoE and Swin-MoE across ImageNet-100 and GTSRB demonstrate activation-mode attack success rates of 76-87% with dormant-mode ASR below 9%, while evading Neural Cleanse, STRIP, Fine-Pruning, and Activation Clustering. Our findings reveal a fundamental security risk arising from batch-dependent execution in scalable Vision MoE architectures.
74. 【2608.25367】RSFusionDet: Underwater RGB-Sonar Multimodal Object Detection
链接：https://arxiv.org/abs/2608.25367
作者：Zhuoyan Liu,Yihan Wang,Bo Wang,Bing Wang,Ye Li
类目：Computer Vision and Pattern Recognition (cs.CV)
关键词：sonar images limited, optical images limited, images limited, Underwater unimodal object, object structural information
备注： 18 pages, 13 figures, 18 tables
点击查看摘要
Abstract:Underwater unimodal object detection faces many challenges in sensor imaging, such as optical images limited by underwater noise and visible distance, and sonar images limited by less object structural information. While, optical images have rich object structural information, and sonar images are less affected by underwater noise and have a longer visible distance. Optical (RGB modality) and sonar (Sonar modality) images have complementary information underwater. In this paper, we create an RGB-Sonar multimodal object detection dataset, \textbf{R}GB-\textbf{S}onar \textbf{Fusion} (RSFusion) and propose evaluation metrics for the benchmark. And we propose the \textbf{R}GB-\textbf{S}onar \textbf{Fusion} \textbf{Det}ector (RSFusionDet) with a new RGB-Sonar multimodal object detection result expression for RGB-Sonar multimodal object detection. We analyze the features of RGB and Sonar modal information, and design a Cross-Attention Fusion (CAFusion) module to fuse RGB-Sonar spatial misalignment features and Object Matching Head (OMHead) with Loss (OMLoss) to match identical objects in RGB-Sonar modalities. Our RSFusionDet achieves 76.4/48.6 AP (RGB/Sonar) for object detection and 83.4 \(\text{F1-Score}_{match}\) for object matching, on RSFusion, which outperforms other object detection models. Compared with the DINO baseline, our method improves by 0.7/1.4 AP (RGB/Sonar) while simultaneously providing reliable cross-modal object matching. The code and datasets are publicly available at this https URL.
75. 【2608.25360】FlashNormal: Detailed Surface Normal Estimation from Flash and No-Flash Images
链接：https://arxiv.org/abs/2608.25360
作者：Ruiyang Chen,Feiran Li,Heng Guo,Zhanyu Ma
类目：Computer Vision and Pattern Recognition (cs.CV)
关键词：High-quality surface normal, detailed surface shape, High-quality surface, preferred for detailed, surface normal estimation
备注： (c) 2026 IEEE. Personal use of this material is permitted. Permission from IEEE must be obtained for all other uses, in any current or future media, including reprinting/republishing this material for advertising or promotional purposes, creating new collective works, for resale or redistribution to servers or lists, or reuse of any copyrighted component of this work in other works
点击查看摘要
Abstract:High-quality surface normal estimation is preferred for detailed surface shape recovery and image editing. Existing single image-based methods, though being a practical setup, often struggle to recover fine surface details and are sensitive to inherent shape-reflectance ambiguity. While photometric stereo achieves high-fidelity surface normal estimation from images under varying lights, its applicability is strictly limited by requiring a multi-illumination capture setup. To this end, we propose FlashNormal, a diffusion-based surface normal estimator from flash/no-flash image pairs. While retaining high practicability on modern smartphones, our proposal takes advantage of flash-induced shading variations, and leverages curvature-guided detail enhancement strategy, improving surface detail recovery and mitigating shape-reflectance ambiguity effectively. To evaluate our proposed method, we further present EvalFlash, the first real-world flash/no-flash evaluation dataset containing 20 objects aligned with ground-truth surface normals for quantitative benchmarking. Extensive experiments demonstrate the effectiveness of FlashNormal over state-of-the-art single image-based methods and show a significant out-performance over flash/no-flash-based normal estimation method on EvalFlash.
76. 【2608.25356】Where to Look Matters: On-Policy Self-Distillation for Long-Video Understanding
链接：https://arxiv.org/abs/2608.25356
作者：Kaishen Wang,Dongdi Zhao,Yijun Liang,Dingqiang Ye,Ruibo Chen,Heng Huang,Di Fu
类目：Computer Vision and Pattern Recognition (cs.CV)
关键词：made substantial progress, typically answering questions, models typically answering, Vision-language models, made substantial
备注： 15 pages, 8 figures, 5 tables
点击查看摘要
Abstract:Vision-language models (VLMs) have made substantial progress in long-video understanding, with standard backbone models typically answering questions from frames sampled across the full video. However, as videos become longer, the full-video context inevitably contains more question-irrelevant temporal content, which can distract the model from the evidence needed to answer a specific question. We empirically find that focusing the visual input on short annotated clue intervals containing question-relevant evidence consistently improves prediction accuracy across model scales compared with using the corresponding full videos, while requiring fewer input frames. Based on this finding, we introduce Clue-OPSD, a clue-privileged on-policy self-distillation framework for long-video understanding. During training, a full-video student learns from a self-teacher conditioned on the corresponding clue interval by aligning their next-token distributions along student-generated trajectories. Clue-OPSD thus uses clue intervals as privileged supervision without relying on ground-truth answer labels, while requiring no clue annotations or additional modules at inference time. Extensive experiments across multiple long-video understanding benchmarks and Qwen3.5 model scales demonstrate consistent improvements over the corresponding backbone models and strong performance against supervised post-training baselines.
77. 【2608.25344】CoRE: Weakly Supervised Coarse-to-Fine Risk Evidence Learning in Driving Videos
链接：https://arxiv.org/abs/2608.25344
作者：Kaiser Hamid,Can Cui,Nade Liang
类目：Computer Vision and Pattern Recognition (cs.CV); Artificial Intelligence (cs.AI)
关键词：specific scene entities, Perceived risk, driving evolves, evolves over time, supported by specific
备注：
点击查看摘要
Abstract:Perceived risk in driving evolves over time and may be supported by specific scene entities, yet supervision is typically limited to coarse video-level judgments. Learning \emph{when} supporting evidence emerges and \emph{which entities} support a risk predictor would ordinarily require costly temporal- and entity-level annotations. We introduce \textbf{CoRE}, a weakly supervised coarse-to-fine framework that learns fine-grained prediction support from coarse video supervision. CoRE first trains a video-level predictor and then freezes it. Structured interventions over candidate temporal regions or entity tracks measure how each candidate changes the coarse prediction, producing graded prediction-effect targets. These targets are distilled into a student that directly predicts temporal and entity support from the original video, without requiring interventions at inference. We evaluate this learning principle across three complementary settings: RISEE tests perceived-risk support from subjective clip-level judgments without temporal or entity-level risk annotations; DoTA provides independent temporal event annotations for evaluating weakly supervised traffic-anomaly localization; and UCF-Crime tests whether the same coarse-to-fine mechanism extends to a standard non-driving anomaly-detection benchmark. Across these settings, CoRE learns informative fine-grained support from coarse supervision, with strong temporal localization on DoTA and competitive performance on UCF-Crime. These results show that coarse video predictions can provide useful supervision for recovering the fine-grained evidence supporting them, without requiring corresponding fine-grained labels.
78. 【2608.25334】GraftSR: Grafting Authentic Textures for Real-World Image Super-Resolution via Identical-Instance Guidance
链接：https://arxiv.org/abs/2608.25334
作者：Qifan Yu,Haoran Bai,Zongyao He,Weijie He,Sibin Deng,Honggang Qi,Ying Chen
类目：Computer Vision and Pattern Recognition (cs.CV)
关键词：Diffusion-based real-world image, impressive perceptual quality, real-world image super-resolution, Diffusion-based real-world, achieves impressive perceptual
备注： 15 pages, 12 figures
点击查看摘要
Abstract:Diffusion-based real-world image super-resolution (SR) achieves impressive perceptual quality but inherently suffers from severe texture hallucination. To overcome this limitation, we propose GraftSR, a texture-reference-guided generative SR framework that leverages reference images of the identical instance to anchor the restoration of authentic textures. However, severe spatial misalignment between low-quality inputs and their references poses significant challenges, often leading to ambiguous transfer targets and background feature leakage. To address these issues, GraftSR employs a novel dual-mask reference guidance mechanism that systematically decouples the cross-view texture injection process. By explicitly isolating what authentic textures to extract from the reference and precisely localizing where to apply them within the target, GraftSR achieves robust texture transfer without relying on brittle spatial alignment. Furthermore, to bridge the critical gap in appropriate training data, we construct TexRefSR-141K, the first large-scale dataset providing high-quality reference tuples equipped with complementary spatial masks. Extensive experiments on our newly established benchmark, TexRefSR-Eval, demonstrate that GraftSR sets a new state-of-the-art. Notably, it reduces LPIPS by 20.2\% over top-performing baselines, achieving superior reference-faithful restoration.
79. 【2608.25332】Not All Attention Heads Contribute to Critical Visual Token Selection: Head-Aware Pruning Matters More
链接：https://arxiv.org/abs/2608.25332
作者：Chaofang Ma,Lin Jiang,Carol Jingyi Li,Xingyu Liu,Zeyu Li,Jiang Xu,Wei Zhang
类目：Computer Vision and Pattern Recognition (cs.CV)
关键词：Large Language Model, diverse visual scenarios, exhibited impressive performance, Vision-Language Models, Language Model
备注：
点击查看摘要
Abstract:Vision-Language Models (VLMs) have exhibited impressive performance across diverse visual scenarios. However, this success comes at the cost of explosive growth in visual tokens, which imposes substantial memory and computational overhead during inference, ultimately increasing latency. To improve VLM inference efficiency, a typical class of visual token pruning methods estimates token importance by aggregating attention scores across all heads in the pruning layer of the Large Language Model (LLM) backbone and prunes tokens based on aggregated scores. However, in this paper, we reveal a compelling phenomenon: the capability to pinpoint critical visual tokens is concentrated within a small fraction of heads. Aggregation exclusively on these heads can improve task performance. Inspired by this observation, we propose ProViP, a training-free progressive visual token pruning framework. ProViP first removes redundant visual tokens based on the embedding similarity of input tokens before reasoning of the LLM backbone, and then further prunes tokens during reasoning via head-aware pruning. Experiments demonstrate that ProViP delivers outstanding task performance and inference efficiency. For instance, when applied to LLaVA-1.5-7B, ProViP retains 95.9% of the original performance and achieves 1.62x inference speedup under an 88.9% pruning ratio.
80. 【2608.25308】V-Link: Recovering Lost Visual Representations in Action DiT for Vision-Language-Action Models
链接：https://arxiv.org/abs/2608.25308
作者：Yehao Lu,Jiarui Yang,Yuning Su,Yufeng Xie,Yu Zhong,Yazhou Zhang,Haiyu Lan,Kaixiang Lu,Peiwen Lin,Chuang Wang,Zequn Qin,Enyu Li,Xi Li
类目：Computer Vision and Pattern Recognition (cs.CV)
关键词：integrating visual perception, continuous action control, generalist robotic manipulation, language understanding, scalable path
备注：
点击查看摘要
Abstract:Vision-language-action (VLA) models provide a scalable path toward generalist robotic manipulation by integrating visual perception, language understanding, and continuous action control. However, we reveal a critical limitation of VLA architectures: the action expert has limited access to the 3D geometric and 2D semantic information available in VLM features. This accessibility gap weakens perceptual grounding and limits performance on fine-grained robotic manipulation. To address this issue, we propose V-Link, which explicitly recovers visual representations during the vision-language (VL) to action (A) feature transfer. Specifically, V-Link learns complementary Spatial and Semantic Query representations within the VLM and injects them into Action DiT through asymmetric pathways. Semantic Queries complement the original VLM image tokens, whereas Spatial Queries provide dedicated geometric conditioning for spatially grounded action generation. Across LIBERO, LIBERO-Plus, and RoboTwin 2.0, our V-Link improves the average success rate over base model GR00T N1.6 by +1.9%, +31.2%, and +18.8%, respectively. On the AGIBOT A3 Ultra, V-Link further achieves gains of +20% and +24% on two real-world humanoid tasks.
81. 【2608.25305】MulVec: Fine-Grained Role-Aware Matching for Training-Free Zero-Shot Composed Image Retrieval
链接：https://arxiv.org/abs/2608.25305
作者：Zihao Zhang,Dayan Wu,Xinze Liu,Hengjie Zhu,Yiliang Zhu,Ding Wang,Peng Fu,Zheng Lin,Weiping Wang
类目：Computer Vision and Pattern Recognition (cs.CV)
关键词：Training-free zero-shot composed, task-specific image triplets, zero-shot composed image, Training-free zero-shot, composed image retrieval
备注：
点击查看摘要
Abstract:Training-free zero-shot composed image retrieval finds a target image in a gallery from a reference image and a text edit without learning from task-specific image triplets. Existing methods typically describe the target as a whole and match this description with a global image representation. This global matching can mix different semantic cues and lose fine- grained details. We propose MULVEC, a role-aware method whose compiler produces a structured query record that is mapped to four retrieval roles: Global describes the full target, Desired states what should appear, Preserve states what should remain, and Forbidden states what should disappear. Frozen encoders map the query to one target description vector and role-specific probe vectors, while each candidate is represented by one global visual vector and a bank of local visual vectors. The retrieval roles then use this shared evidence for their respective purposes, and a fixed weighted sum of their scores ranks the entire gallery in a single retrieval pass. Across CIRCO, CIRR, and FashionIQ and three backbone scales, MULVEC improves CIRCO mAP@5 by up to 23.0% over the strongest compared method and gives the best CIRR and FashionIQ results in our comparison.
82. 【2608.25302】WAVE: Reversing the Guidance Hierarchy for Coarse-to-Fine Guided Depth Super-Resolution
链接：https://arxiv.org/abs/2608.25302
作者：Tayyab Nasir,Daochang Liu,Ajmal Mian
类目：Computer Vision and Pattern Recognition (cs.CV); Machine Learning (cs.LG)
关键词：Guided depth super-resolution, typically extracts RGB, extracts RGB guidance, RGB guidance features, typically extracts
备注：
点击查看摘要
Abstract:Guided depth super-resolution (GDSR) typically extracts RGB guidance features through convolutional hierarchies, inheriting their fine-to-coarse bias. Thus, low-level spatial cues surface in early layers, leaving the deeper layers to suppress those that do not correspond to true depth boundaries, which risks artifacts and blurred edges. The same fine-to-coarse bias persists in semantics-based methods that consume low-level tokens early and global tokens late. We present WAVE, which introduces a multi-level discrete wavelet transform (ML-DWT) as an explicit and interpretable feature-control mechanism, enabling a coarse-to-fine reconstruction by consuming sub-bands and semantic tokens in reverse of their generation order. WAVE further exploits these sub-bands to treat high- and low-frequency content separately, filtering at its source the misleading RGB color and texture cues that often lead to blurred boundaries and artifacts, offering an intuitive alternative to the suppression learned implicitly by an opaque network. WAVE separates structure and detail reconstruction into dedicated modules that: i) model interactions within and across wavelet sub-bands, depth features, and semantic priors, ii) apply semantic gating to the high-frequency bands, and iii) fuse modalities through an invertible coupling mechanism that prevents collapse onto a single modality. Extensive experiments across multiple benchmarks demonstrate that WAVE matches or outperforms existing methods, with the largest gains at high upsampling factors, where low-resolution depth contains the least structure.
83. 【2608.25299】PointRL: Learning Point-Level Vision-Language Grounding from Verifiable Annotation Evidence
链接：https://arxiv.org/abs/2608.25299
作者：Jingyang Su,Pu Cao,Xiuze Jin,Longyue Zhang,Qing Song,Lu Yang
类目：Computer Vision and Pattern Recognition (cs.CV)
关键词：interactive visual systems, Vision-language models, GUI interaction, visual systems, robotic manipulation
备注：
点击查看摘要
Abstract:Vision-language models (VLMs) increasingly rely on point coordinates as a compact and executable interface for visual grounding in GUI interaction, robotic manipulation, and interactive visual systems. However, learning reliable pointing behavior remains difficult because the supervision space is inherently non-unique: many coordinates may be valid within the same target region, while multi-instance instructions require target coverage, count consistency, and duplicate suppression. This work presents PointRL, a verifiable reinforcement learning framework that learns point-level grounding from existing heterogeneous annotation evidence. PointRL converts bounding boxes, masks, and instance labels into pointing instructions, while retaining their target supports, instance membership, and set constraints as hidden verifier evidence, i.e., annotations kept outside the prompt and used by a deterministic checker to score predictions. The proposed reward evaluates parseability, point validity, instance coverage, cardinality consistency, and redundant or missing predictions. On PointArena, PointRL improves the overall accuracy of Qwen3.5-4B from 56.11% to 65.58%. Further evaluations on RoboSpatial, BLINK, and Ref-Adv show same-backbone gains on the evaluated external benchmarks, suggesting that verifiable point-level feedback may benefit spatial grounding in these settings.
84. 【2608.25274】OpenCVL: An Open, Diverse, and Large-Scale Dataset for Fine-Grained Cross-View Localization
链接：https://arxiv.org/abs/2608.25274
作者：Zimin Xia,Mubariz Zaffar,Junsheng Fu,Alexandre Alahi,Julian F. P. Kooij
类目：Computer Vision and Pattern Recognition (cs.CV)
关键词：Navigation Satellite Systems, Global Navigation Satellite, Fine-grained Cross-View Localization, Satellite Systems, Global Navigation
备注：
点击查看摘要
Abstract:Fine-grained Cross-View Localization (CVL) estimates the precise position and orientation of a ground-level image by aligning it with geo-referenced aerial imagery, offering a scalable alternative to Global Navigation Satellite Systems (GNSS) in challenging urban environments. Existing datasets rely on data collected with high-end sensor suites, which inherently limit image diversity and scalability. While in-the-wild images are abundant, their noisy geo-tags make them unsuitable for reliable evaluation. To bridge this gap, we introduce OpenCVL, a large-scale, diverse, and open dataset containing 617,388 ground-aerial image pairs spanning 41 cities across four European countries. All images are sourced from permissive platforms, ensuring long-term accessibility and supporting open and reproducible research. The training set combines images captured with high-end sensors with diverse in-the-wild imagery. We further develop a data curation framework that filters and corrects pose annotations to construct reliable in-the-wild evaluation data. In addition, OpenCVL includes dedicated cross-area and snowy test sets to assess generalization and robustness. Experiments with a state-of-the-art CVL model on OpenCVL show that incorporating noisy in-the-wild data consistently improves performance on clean test sets, suggesting a promising direction for scaling CVL with diverse real-world imagery.
85. 【2608.25261】Hierarchical MoE for Multi-Modal ILD Diagnosis
链接：https://arxiv.org/abs/2608.25261
作者：Alec K. Peltekian,Gorkem Durak,Halil Ertugrul Aktas,Carrie Lynn Richardson,Mary Carns,Kathleen Aren,GR Scott Budinger,Anthony J. Esposito,Alexander Misharin,Alok Nidhi Choudhary,Ankit Agrawal,Ulas Bagci
类目：Artificial Intelligence (cs.AI); Computer Vision and Pattern Recognition (cs.CV)
关键词：combine specialized predictors, models combine specialized, offering a principled, medical data, combine specialized
备注： 11 pages, 2 figures
点击查看摘要
Abstract:Mixture-of-experts (MoE) models combine specialized predictors under learned routing, offering a principled mechanism for leveraging heterogeneity in medical data. We present a hierarchical multimodal MoE for interstitial lung disease (ILD) classification that integrates a frozen, pre-trained imaging expert with structured electronic health records (EHR) via two-stage gating. A modality-level gate assigns patient-specific weights to imaging and EHR predictions, while a sub-gating module decomposes the EHR branch into clinically defined feature groups with learned, group-specific contributions. This design preserves stable imaging representations while enabling input-dependent clinical weighting and explicit EHR specialization. Under strict patient-level cross-validation, the model achieved the highest mean AUC among the evaluated methods (0.8750 +- 0.0443), compared with 0.8646 for imaging-only REN and 0.7685 for SwinUNETR. The framework extends interpretability across anatomical regions, imaging--EHR utilization, and clinically defined EHR feature groups.
86. 【2608.25251】What Do Medical Vision-Language Models Learn in Radiology? Transfer, Alignment, and Source-Proxy Leakage Under Distribution Shift
链接：https://arxiv.org/abs/2608.25251
作者：Ayoub Louaye Bouaziz,Lokmane Chebouba,Yassine Himeur
类目：Computer Vision and Pattern Recognition (cs.CV); Artificial Intelligence (cs.AI)
关键词：Medical vision-language models, paired supervision, vision-language models, acquisition domain, reliable in-domain
备注：
点击查看摘要
Abstract:Medical vision-language models (VLMs) can appear reliable in-domain while failing when acquisition domain, paired supervision, or evaluation protocol changes. We study this failure mode as a representation-level blind spot relevant to epistemic intelligence, without claiming a formal estimator of epistemic uncertainty. Using NIH ChestXray14 and CheXpert, we first isolate source-only cross-dataset visual transfer from unsupervised domain-adaptation diagnostics. Using PadChest and OpenI, we then evaluate multimodal alignment under strict pair-index retrieval and quantify metadata-derived source-proxy information retained in frozen embeddings. Self-supervised visual initialization improves NIH-to-CheXpert transfer over supervised ImageNet initialization in matched ResNet-18 comparisons, whereas adversarial adaptation is useful only in a narrow regime and becomes unstable as adversarial pressure increases. Multimodal exact-pair retrieval remains low under external OpenI stress testing, and source-proxy information remains recoverable from learned representations. Qualitative nearest-neighbor and Grad-CAM analyses show clinically plausible cross-dataset structure and thoracic attention patterns in many cases, while device-heavy and false-positive cases remain ambiguous. Auxiliary architecture checks are task-dependent and do not support a universal backbone ranking. Overall, the study shows that apparent competence under a single protocol can conceal transfer, alignment, and shortcut-related failure modes, motivating stress-tested evaluation of medical VLMs under distribution shift.
87. 【2608.25178】Lightweight Machine Learning-Driven Monocular Sidewalk Path Extraction for Embedded Micromobility Navigation
链接：https://arxiv.org/abs/2608.25178
作者：Lkhanaajav Mijiddorj,Yang Yan,Tyler Beringer,Bilguunzaya Mijiddorj,Alex N. Ho,Bin Xu,Binbin Weng
类目：Computer Vision and Pattern Recognition (cs.CV); Artificial Intelligence (cs.AI)
关键词：Sidewalk-scale path extraction, extraction demands perception, path extraction demands, Sidewalk-scale path, path extraction
备注：
点击查看摘要
Abstract:Sidewalk-scale path extraction demands perception and planning that run reliably on compact, low-power hardware in cluttered, map-sparse environments. We present a monocular vision pipeline for sidewalk path extraction in micromobility systems that progresses through three design iterations, from a skeleton-graph baseline through distance-transform corridor planning to a lightweight image-space architecture, and provides a systematic comparison of five path-planning methods across both bird's-eye-view (BEV) and image-space domains. A compact SegFormer-B0 student model, trained with a semi-supervised teacher-student framework using OneFormer Swin-L pseudo-labels, achieves a hand-annotated IoU of 0.946 at 11.7 ms per frame, improving over the baseline checkpoint (IoU 0.758, 18.9 ms). In a controlled planner comparison on 32 hand-labeled frames, image-space midpoint planning achieves the lowest lateral center error (14.3 px) at 2.2 ms, a 421x speedup over BEV distance-transform planning (926.8 ms, 65.0 px center error), while maintaining comparable mask-path alignment (98.5% versus 98.6%). A full-video replay across six campus sequences (22,679 frames) confirms that the improved segmentation reduces temporal instability from 1.46% to 0.33% and increases template-path availability from 73.7% to 79.3%. We further show that BEV-only path extraction is fragile in monocular settings: in one profiled run, 99.3% of frames produced no valid BEV path. The final recommended architecture, image-space midpoint primary, image-space distance-transform fallback, and BEV reserved for visualization, runs the full perception-to-path stack in under 50 ms per frame on CPU, making it suitable for embedded pedestrian-speed micromobility systems.
88. 【2608.25176】Lowering the Barrier to AI-Driven Inspection: A No-Code Workflow for Automated Structural Defect Detection
链接：https://arxiv.org/abs/2608.25176
作者：Michael Holm,Tanner McElroy,Xinghang Zhang,Guang Lin
类目：Computer Vision and Pattern Recognition (cs.CV); Machine Learning (cs.LG); Image and Video Processing (eess.IV)
关键词：lifecycle assessment, Structural health monitoring, SHM, providing data, Structural health
备注： 11 pages, 7 figures. Accepted to ASME SMASIS 2026 (paper SMASIS2026-190654). Software available at [this https URL](https://github.com/michaelholm6/YOLOEZ)
点击查看摘要
Abstract:Structural health monitoring (SHM) is essential in modern engineering, providing data for condition-based maintenance, lifecycle assessment, and predictive decision-making. Traditionally, SHM relied on visual inspection to detect defects such as cracks and deformations. Early computer vision (CV) methods, including thresholding, edge detection, and handcrafted features, aimed to automate this process but were highly sensitive to noise, imaging variations, and multiscale defects, limiting their reliability. Recent advances in machine learning, particularly convolutional neural networks (CNNs) and You Only Look Once (YOLO), have improved defect detection accuracy and enabled real-time analysis. However, adoption in SHM remains limited due to technical barriers such as data labeling, model training, and deployment, which typically require programming expertise. To address this gap, we introduce YOLOEZ, an open-source, GUI-based tool for end-to-end YOLO model application. YOLOEZ integrates data labeling, training, and inference into a single interface, enabling high-performance model development without code while supporting reproducible workflows. Evaluation against existing software and classical image processing demonstrates that YOLOEZ not only outperforms traditional methods across most detection metrics, but also lowers adoption barriers present in other modern CV tools. By combining accuracy with accessibility, YOLOEZ facilitates wider use of AI-driven monitoring for predictive maintenance, digital twins, and intelligent structural systems.
Comments:
11 pages, 7 figures. Accepted to ASME SMASIS 2026 (paper SMASIS2026-190654). Software available at this https URL
Subjects:
Computer Vision and Pattern Recognition (cs.CV); Machine Learning (cs.LG); Image and Video Processing (eess.IV)
Cite as:
arXiv:2608.25176 [cs.CV]
(or
arXiv:2608.25176v1 [cs.CV] for this version)
https://doi.org/10.48550/arXiv.2608.25176
Focus to learn more
arXiv-issued DOI via DataCite (pending registration)
Submission history From: Michael Holm [view email] [v1]
Tue, 25 Aug 2026 21:46:03 UTC (1,865 KB)
89. 【2608.25168】See More, Detect Less? Taming Information Leakage in Multi-View Anomaly Detection
链接：https://arxiv.org/abs/2608.25168
作者：Shang-Fu Chen,Kuan-Chuan Peng,Jhih-Ciang Wu,Wen-Huang Cheng,Kai-Lung Hua
类目：Computer Vision and Pattern Recognition (cs.CV); Multimedia (cs.MM)
关键词：multi-view anomaly detection, multi-view anomaly, Multi-view Merging Attention, anomaly detection, multi-view
备注：
点击查看摘要
Abstract:In multi-view anomaly detection, more cross-view information can actually hurt. When multiple inspection views are naively fused in a reconstruction-based pipeline, normal cues from intact views propagate to the decoder, which faithfully reconstructs anomalous regions, collapsing the reconstruction gap the detector depends on. We call this failure mode \emph{cross-view information leakage} and show that effective multi-view fusion must explicitly restrict the information reaching the decoder. Building on this insight, we present GLAD(Global-Local Attention Driven framework), the first framework combining vision foundation model features with local and global cross-view fusion for multi-view anomaly detection. The Multi-view Merging Attention (MMA) module performs local cross-view fusion at linear complexity with learnable view importance weighting and token-wise gating, letting each view selectively incorporate fine-grained evidence from other views at $\mathcal{O}(N)$ cost. The Object-Guided Attention (OGA) module captures global context by aggregating class tokens from all views into a single object-level representation and broadcasting it back to patch tokens via temperature-scaled sigmoid gating, replacing the original patch representations rather than adding a residual to preserve the reconstruction gap. Experiments on Real-IAD and MANTA-Tiny show that GLAD outperforms state-of-the-art methods across sample-, image-, and pixel-level metrics, confirming that principled information restriction is key to multi-view anomaly reasoning.
90. 【2608.25157】What Do Audio-Visual Synchronization Metrics Actually Measure?
链接：https://arxiv.org/abs/2608.25157
作者：Jai Kumar Sharma,Peeyush Tapadiya
类目：Computer Vision and Pattern Recognition (cs.CV); Multimedia (cs.MM); Sound (cs.SD)
关键词：train audio-visual generators, Automatic AV-sync metrics, audio-visual generators, measurement instruments, Automatic AV-sync
备注： Accepted at the ECCV 2026 Workshop on Generative AI for Audio-Visual Content Creation (Gen4AVC), poster presentation; non-archival workshop. 7 pages (4-page main text + references + 2-page appendix), 3 figures, 8 tables. Project page: [this https URL](https://jaishrm07.github.io/avsync-reliability-card/)
点击查看摘要
Abstract:Automatic AV-sync metrics are widely used to rank and train audio-visual generators, but they are rarely audited as measurement instruments. We jointly audit AV-Align, ImageBind AV-relevance, JavisScore, and Synchformer/DeSync under a common reliability protocol: controlled-distortion monotonicity, preprocessing sensitivity, rank uncertainty, cross-metric agreement, PEAVS-proxy agreement, and learned fusion. The result is an axis split, not a single winner: Synchformer/DeSync is the strongest temporal-offset tracker ($\tau=0.84$), ImageBind/JavisScore better match the PEAVS human-aligned proxy ($\tau=0.20$) and content-disruption families, and AV-Align is the weakest standalone metric. The metrics mutually disagree (Krippendorff $\alpha=0.066$), and neither linear nor simple $k$-NN fusion improves PEAVS agreement over the best individual metric. We recommend reporting AV-sync as a Reliability Card (metric-family breakdowns with confidence intervals) rather than a single bare synchronization score.
91. 【2608.25148】Can You Trust Frozen Hematology Foundation Models under Acquisition Shift?
链接：https://arxiv.org/abs/2608.25148
作者：Jai Kumar Sharma,Peeyush Tapadiya
类目：Computer Vision and Pattern Recognition (cs.CV); Artificial Intelligence (cs.AI); Quantitative Methods (q-bio.QM)
关键词：embeddings reach near-saturated, clinical deployment demands, deployment demands reliability, Frozen hematology foundation-model, reach near-saturated in-domain
备注： Accepted at the HemaRAI 2026 workshop (MICCAI 2026 satellite event), oral presentation; to appear in MICCAI 2026 Satellite Events, LNCS, Springer. 25 pages (10 main incl. references + 15 supplementary), 4 figures. Project page: [this https URL](https://jaishrm07.github.io/hematology-fm-robustness/)
点击查看摘要
Abstract:Frozen hematology foundation-model (FM) embeddings reach near-saturated in-domain white-blood-cell (WBC) accuracy, but clinical deployment demands reliability across scanners, sites, stains and preparation pipelines. We audit 15 frozen encoders (hematology, pathology, and general vision) across four public single-cell acquisition domains along two axes: accuracy robustness and calibration. In-domain linear-probe macro-F1 is saturated (0.98-0.997), yet cross-dataset macro-F1 drops 34-72% and rankings re-order: DinoBloom-L, the in-domain best, falls to 10th of 15 on the most-shifted target (MLL23) at the benchmark's shared 224-px input, behind RedDino and several general and pathology encoders. Rank transfer is probe-dependent: 1-NN retrieval is more stable on average than a source-fitted linear head (median $\rho$ 0.65 vs 0.45), but neither probe universally predicts target robustness. Calibration also collapses: source-trained probes are nearly calibrated in-domain (expected calibration error, ECE, 0.004) but confidently wrong off-domain (ECE 0.35), and source-fitted temperature scaling transfers poorly. We further audit pretraining exposure and identify MLL23 as DinoBloom's internal cohort; because DinoBloom's only held-out dataset is also our source domain, this benchmark cannot isolate exposure from scanner-associated shift. Label-free adaptation and marginal-entropy-based model selection appear safe under balanced evaluation but fail under realistic WBC class-prior shift. Class-Balanced Re-standardization (CBR), a training-free pseudo-label-balanced feature normalization, improves all evaluated target-prior scenario means and partially improves calibration, although encoder-level exceptions and residual miscalibration remain. Hematology FM benchmarks must therefore jointly audit accuracy, calibration, exposure, and class-prior robustness.
92. 【2608.25140】RefLAM: A Reference-Grounded Line Annotation Pipeline for Historical Arabic Manuscripts
链接：https://arxiv.org/abs/2608.25140
作者：Mohamed Guechaoui,Mohamed Diaa Zellagui,Souleyman Chaib,Sahraoui Dhelim
类目：Computer Vision and Pattern Recognition (cs.CV); Computation and Language (cs.CL)
关键词：Existing approaches, building line-level Arabic, provable correctness guarantee, extended to multi-script, Reference-grounded Line Annotation
备注： 11 pages, 6 figures, 3 tables, 2 algorithms
点击查看摘要
Abstract:Existing approaches to building line-level Arabic handwritten-text-recognition (HTR) training data either rely on fully manual annotation, which does not scale, or on automatic OCR-to-reference alignment methods not yet extended to multi-script, two-zone (main-plus-margin) manuscript layouts with a provable correctness guarantee. We present RefLAM (Reference-grounded Line Annotation for Manuscripts), a pipeline converting manuscript page images and clean transcriptions into validated, line-level ground truth without sacrificing human oversight. RefLAM couples a deep-learning page-segmentation model with a multimodal large language model (MLLM) for structured OCR and a diacritic-agnostic fuzzy alignment engine that grounds each OCR line in a contiguous span of the reference text, with a character-level confidence score in $[0,100]$. A perfect score is provably equivalent to character-for-character identity of the normalised strings (the Confidence-100 rule), verified with no counterexample across the released corpus. A reviewer can thus trust a perfect score, confirming most lines at a glance rather than retyping them, so annotation becomes triaged, with attention concentrated on uncertain alignments. Across 7 fully page-validated books we measured a 75$\times$ throughput gain over manual annotation (3,000 vs. 40 lines/hr); applying the same guarantee to 7 further books, we retained 16,533 confidence-100 main-text lines within one week, excluding sub-100 lines rather than manually correcting them. Using RefLAM, we release AraMS-28k: 14 historical Arabic manuscript books, 3,043 pages, and 27,971 main-text and 629 margin-line annotations with bounding boxes, layout labels, and insertion anchors for 191 margin entries (30.4%). We also finetune Muharaf-pretrained baselines (including HATFormer) on AraMS-28k and report CER results confirming its practical utility for downstream HTR training.
93. 【2608.25068】SHIFT-LLM: Distribution Shift Correction in Depth-Pruned LLMs
链接：https://arxiv.org/abs/2608.25068
作者：Ali Bahri,Hang Li,Hongliang Li,Zhitang Chen
类目：Computer Vision and Pattern Recognition (cs.CV)
关键词：removes entire Transformer, entire Transformer blocks, hidden-state distributions expected, pruning removes entire, entire Transformer
备注：
点击查看摘要
Abstract:Depth pruning removes entire Transformer blocks to reduce the inference cost of large language models, but disrupts the hidden-state distributions expected by downstream layers, leading to significant accuracy loss. We introduce SHIFT-LLM, a training-free post-pruning correction framework that inserts a Linear Residual Adapter (LRA) at each pruning site. Each LRA preserves the identity pathway of the original residual block and adds a lightweight affine residual correction. This correction is calibrated via closed-form least-squares regression on a small held-out set, without gradient computation, to approximate the missing residual update produced by the pruned block. Together with the preserved identity pathway, the resulting LRA output approximates the hidden state produced by the original block, thereby mitigating the distributional mismatch introduced by layer removal while avoiding the expensive attention and feed-forward computations of the removed blocks. The resulting LRAs support low-rank factorization and exact merging across consecutive pruned layers for additional compression, and combine naturally with parameter-efficient fine-tuning for further recovery beyond fine-tuning the pruned model alone. Experiments on five model families, six layer-selection criteria, and seven zero-shot benchmarks show that SHIFT-LLM consistently recovers accuracy lost to depth pruning across most configurations, achieving gains up to +15.7 points on Llama-3.1-8B-Instruct while requiring only a few hundred calibration samples and no gradient computation.
94. 【2608.25023】CVE-SAI: Counterfactual Visual Evidence-Guided Selective Attribute Indexing for Risk-Controlled E-commerce Search
链接：https://arxiv.org/abs/2608.25023
作者：Xiaolong Sun,Qichao Wang,Hangyu Li,Liang Chen
类目：Artificial Intelligence (cs.AI); Computer Vision and Pattern Recognition (cs.CV)
关键词：Multimodal product models, conflate transient prediction, complete missing e-commerce, optimize attribute-answer accuracy, verifying visual support
备注：
点击查看摘要
Abstract:Multimodal product models can complete missing e-commerce attributes, yet current methods still optimize attribute-answer accuracy without verifying visual support, conflate transient prediction with persistent index admission, and lack explicit risk control over factually incorrect or visually unsupported values. We address these gaps with Counterfactual Visual Evidence-Guided Selective Attribute Indexing (CVE-SAI), which first infers and freezes an ontology-constrained candidate from the primary image and attribute question without catalog text, and then decides whether that candidate should enter the index. Focus-Zone Distortion (FZD) constructs an attribute-specific visual-dependence proxy through a controlled counterfactual intervention, and Evidence-Guided Attention Redistribution (EGAR) uses the proxy to refine ontology-constrained scoring. The canonical candidate is frozen before evidence necessity, evidence retention, nuisance-transformation stability, and candidate-specific catalog-text conflict audits; catalog text can only tighten admission and cannot revise the candidate. Independent family-level calibration selects one policy with a simultaneous one-sided finite-sample bound under a 5% unsafe-admission budget. Experiments on five visual attributes derived from Amazon Berkeley Objects show that CVE-SAI improves attribute inference and evidence localization, achieves the highest certified admission coverage under the shared risk protocol, and yields the strongest controlled retrieval performance with the lowest unsafe auto-induced exposure among automatic-admission systems. Separating inference from admission therefore enables visually supported attribute completion to improve retrieval while limiting persistent index contamination.
95. 【2608.24982】Unsupervised Post-Training of Foundation Models: A Survey
链接：https://arxiv.org/abs/2608.24982
作者：Yijie Xu,Qianyi Cai,Huizai Yao,Yili Wang,Tianfu Wang,Cehao Yang,Xingbo Yao,Zhiyu Guo,Aiwei Liu,Xuming Hu,Weiyu Guo,Hui Xiong
类目：Computation and Language (cs.CL); Artificial Intelligence (cs.AI); Computer Vision and Pattern Recognition (cs.CV); Machine Learning (cs.LG); Multimedia (cs.MM)
关键词：preference data, stronger teachers, Foundation-model post-training, human labels, executable verifiers
备注： Accepted to Findings of EMNLP 2026. 20 pages, 3 figures, 8 tables
点击查看摘要
Abstract:Foundation-model post-training usually relies on human labels, preference data, stronger teachers, or executable verifiers. We study Unsupervised Post-Training (UPT): update-bearing adaptation on unlabeled inputs whose learning signal is derived from same-lineage model artifacts rather than an external oracle. We catalog 80 strict UPT methods and organize them by the object that supplies the update signal: a prediction statistic, a sample relation, a self-generated target, or an internal evaluator. Beyond inventory, we show how the choice of internal signal and task structure determines whether post-training improves the model or recursively amplifies error. An orthogonal Input Visibility $\times$ Update Persistence view maps deployment regimes and defines a unified framework for UPT selection and evaluation.
96. 【2608.24966】argeting the Attention Heads Behind Object Hallucination in LLaVA
链接：https://arxiv.org/abs/2608.24966
作者：Armaan Sandhu,Abhilasha Senapati,Hima Kammachi
类目：Computer Vision and Pattern Recognition (cs.CV); Artificial Intelligence (cs.AI)
关键词：Vision-language models, hallucinate objects absent, hallucinated object, Vision-language, object
备注： 10 pages, 5 figures, 3 tables. Accepted at the Actionable Interpretability Workshop, COLM 2026
点击查看摘要
Abstract:Vision-language models such as LLaVA-1.5-7B often hallucinate objects absent from the image when generating captions. We ask whether an interpretability diagnosis of this failure can guide a targeted fix, and we measure what that fix actually changes. We rank attention heads by how much their image attention drops around hallucinated object words, then screen the shortlist by ablating candidate heads and measuring the change in hallucination-token log probability, yielding a 32-head set. We restrict two interventions to these heads: a head-sliced LoRA adapter and an inference-time grounding controller. On 400 held-out COCO images, the combined method lowers CHAIRs (the fraction of captions with a hallucinated object) from 0.370 to 0.230 and CHAIRi (the fraction of hallucinated object mentions) from 0.156 to 0.096 (p 0.001, paired sign-flip tests). Two controls sharpen attribution. A random-head LoRA control, matched layer-for-layer and trained identically, performs no better than the matched baseline on a separate 200-image control split, supporting the role of head selection rather than LoRA capacity. Under fixed decoding budgets, the CHAIR reduction persists and grows with budget (23% at 64 tokens to 58% at 128), arguing against a pure max-token or truncation artifact, although the method remains shorter and more conservative. The resulting behavior reduces unsupported object mentions while also lowering object recall (0.78 to 0.70). We present a diagnosis-to-intervention pipeline for object hallucination, and, more importantly, a controlled account of what acting on the diagnostic signal actually does: it localizes intervention sites with real, non-random leverage, reported as a behavioral profile rather than a single score.
97. 【2608.24959】GaussVLA: Geometry-Aware Spatial Reasoning for Vision-Language-Action Model
链接：https://arxiv.org/abs/2608.24959
作者：Md Selim Sarowar,Md Tanvir Islam,Sungho Kim,Sangtae Ahn
类目：Robotics (cs.RO); Computer Vision and Pattern Recognition (cs.CV)
关键词：models encode visual, encode visual observations, injects per-pixel scalar, intrinsic geometric structure, dense monocular depth
备注： Accepted to BMVC 2026
点击查看摘要
Abstract:Vision-Language-Action (VLA) models encode visual observations as flat 2D patch tokens that carry no intrinsic geometric structure, and augmenting them with dense monocular depth injects per-pixel scalar values that encode neither surface orientation nor geometric confidence. This leaves the policy with limited structured spatial reasoning for action prediction. We propose GaussVLA, a Mamba-based VLA that incorporates two custom modules: Gaussian Spatial Tokenizer (GST) to lift frozen semantic and depth features into compact 3D Gaussian tokens, pools geometrically salient regions with learned queries, and \emph{Depth-Aware Chain-of-Thought (DA-CoT)} that performs structured, non-autoregressive geometric reasoning under language and flow-time conditioning. Across both simulation and real-world evaluations, GaussVLA demonstrates strong spatial-manipulation performance while remaining parameter-efficient. On LIBERO, it achieves 93.5% average success and 100.0% success on the Spatial suite with only 200M parameters, improving over SpatialVLA by 19.7% relative average success while remaining significantly more parameter-efficient.
98. 【2608.24956】Synergising Local Geo-Environmental Characteristics with Spatial Context for Enhancing Landslide Susceptibility Mapping
链接：https://arxiv.org/abs/2608.24956
作者：Yusen Cheng,Lei Fan,Qinfeng Zhu,Cheng Zhang,Yangyang Li,Ron Mahabir
类目：Computer Vision and Pattern Recognition (cs.CV)
关键词：Data-driven methods, methods are widely, complex relationships, spatial context, Existing data-driven approaches
备注：
点击查看摘要
Abstract:Data-driven methods are widely used in landslide susceptibility mapping (LSM) because they can effectively model the complex relationships between landslides and geo-environmental conditions. Existing data-driven approaches generally follow two types of data representations. Pixel-based models focus solely on the geo-environmental characteristics of a specific landslide but neglect the influence of its surrounding environment. Patch-based models incorporate surrounding spatial context but may include pixels with weak or no spatial relevance to the target landslide location. To address this limitation, this study proposes a Local-Geo and Spatial Context Fusion (LGSCF) strategy, which synergises the geo-environmental characteristics of landslide points with their corresponding spatial context through a feature-wise modulation mechanism. We tested the LGSCF strategy by integrating it into several representative convolutional neural network (CNN) architectures, creating nine different LGSCF-based models. The study area covers approximately 2644 km2 across Jenai and Sinyi Townships in Nantou County, Taiwan, and the dataset comprises 5332 landslide samples and an equal number of non-landslide samples. The results show that LGSCF-based models consistently outperform their original versions, achieving F1-scores up to 87.09% and AUC values up to 0.9472. Furthermore, the susceptibility maps produced by LGSCF-based models show that known landslides are more accurately concentrated in "very high" susceptibility zones with fewer misclassifications. These findings demonstrate that our fusion strategy can significantly improve the accuracy of landslide susceptibility mapping.
99. 【2608.24935】A Lightweight Multimodal Vision-Language Framework for Early-Stage Anatomical Green Fruit Classification in Commercial Orchards
链接：https://arxiv.org/abs/2608.24935
作者：Ranjan Sapkota,William Bu,Chen Chen,Yunjun Xu,Manoj Karkee
类目：Computer Vision and Pattern Recognition (cs.CV); Artificial Intelligence (cs.AI)
关键词：precision orchard operations, Accurate identification, crop-load management, identification of early-stage, early-stage apple fruitlet
备注：
点击查看摘要
Abstract:Accurate identification of early-stage apple fruitlet anatomical structures, including the calyx, fruitlet body, and peduncle, is essential for robotic thinning, crop-load management, and other precision orchard operations. This study presents a lightweight multimodal vision-language framework that adapts TinyCLIP for fine-grained fruitlet anatomy classification in complex orchard environments. A dataset of 600 high-resolution RGB images collected from Scilate and Scifresh apple orchards was converted into 224 x 224 image patches and annotated for three anatomical classes. Domain-specific language prompts, such as ``a photo of a class,'' were used to guide multimodal alignment between orchard imagery and horticultural structures. A sliding-window inference strategy with a stride of 112 pixels aggregates patch-level predictions into spatial heatmaps, enabling interpretable whole-image localization of fruitlet components relevant to robotic thinning. Patch-level evaluation on an NVIDIA T4 GPU achieved F1-scores of 0.95 for calyx, 0.98 for fruitlet, and 0.85 for peduncle, with a macro-F1 score of 0.93. Deployment-oriented optimization using ONNX and TensorRT enabled efficient inference on NVIDIA Jetson hardware, preserved accuracy under INT8 quantization, and supported model sizes of approximately 127-137 MB with millisecond-level patch inference. These results demonstrate that lightweight vision-language models can provide interpretable and edge-deployable perception for automated fruitlet analysis and future robotic thinning systems. The source code and implementation details are publicly available at this https URL.
100. 【2608.24934】Fusing Perceptual Vision Experts with Multimodal Large Language Models for Explainable Plant Disease Diagnosis: From Benchmark Imagery to Real-World Robotic Field Validation
链接：https://arxiv.org/abs/2608.24934
作者：Ranjan Sapkota,Konstantinos I. Roumeliotis,Pengyao Xie,Nikolaos D. Tselikas,Lirong Xiang,Manoj Karkee
类目：Computer Vision and Pattern Recognition (cs.CV); Artificial Intelligence (cs.AI)
关键词：Accurate field plant, plant disease diagnosis, disease diagnosis requires, diagnosis requires reliable, requires reliable fusion
备注：
点击查看摘要
Abstract:Accurate field plant disease diagnosis requires reliable fusion of uncertain and conflicting perceptual evidence. We present the Hybrid Hierarchical Multi-Agent Framework (H$^{2}$MAF), combining decision-level fusion of EfficientNet-B3 and ConvNeXt-Tiny with semantic arbitration by open-weight multimodal large language models (MLLMs), Gemma 4 E4B and Qwen3.5 4B, using structured JSON evidence to generate explainable diagnoses, risk levels, treatment urgency, and financial exposure. (H$^{2}$MAF) is evaluated on 14,364 images (1,370 test images) across PlantDoc (2,922 images, 27 classes) and two non-public, continuously captured Cornell robot-acquired field datasets: Stage 2 (20 GB; 4,215 images) and Stage 4 (40 GB; 7,227 images), covering Early Blight, Late Blight, and Septoria Leaf Spot under uncontrolled field conditions. On PlantDoc, Gemma improves accuracy from 63.9% to 68.5%, achieving +7.6 points on the 41.7% CNN-conflict subset. Cornell accuracies reach 99.3% and 98.9%, with only 1.7-4.1% disagreement, demonstrating conflict-dependent MLLM utility. The critical-risk error of gemma is 0.14-0.5 points, whereas Qwen overflags by 3.5-14.4 points. These results establish MLLM arbitration as a promising, yet calibration-dependent, approach for explainable agricultural AI and robotic field decision support. Github Link: this https URL
101. 【2608.24909】Super Star: Towards Streaming Real-time Interactive Agents for Digital Humans
链接：https://arxiv.org/abs/2608.24909
作者：Wentao Jiang,Youchen Xie,Haidi Fan,Yajing Chen,Xin Wang,Ye Shi,Jingya Wang
类目：Human-Computer Interaction (cs.HC); Computer Vision and Pattern Recognition (cs.CV); Sound (cs.SD)
关键词：complete speech segments, predominantly studied, synthesized from complete, interactive digital humans, methods are predominantly
备注： Accepted by ACM Multimedia 2026. Project Page: \url{ [this https URL](https://super-star-2026.github.io/) }
点击查看摘要
Abstract:Existing co-speech gesture generation methods are predominantly studied in offline settings, where gestures are synthesized from complete speech segments. However, interactive digital humans in real-world scenarios are required to generate speech-synchronous gestures online, using only currently available response audio under strict latency constraints. As a result, prior methods are unsuitable for real-time interaction, as they either rely on future speech information or incur substantial inference delay. In this paper, we formulate online co-speech gesture generation for interactive digital humans and propose a real-time interactive framework that couples a streaming speech response module with an online gesture generation module. Specifically, the gesture generator is designed as a causal multimodal autoregressive model that predicts body motion from streaming response speech and motion history, enabling low-latency and speech-aligned gesture synthesis without access to future speech. To support this setting, we further propose an offline data synthesis pipeline tailored to virtual companion scenarios, which leverages topic- and emotion-aware subject corpora to construct diverse human-agent dialogues and then generates co-speech gestures conditioned on the agent responses. Moreover, to bridge the gap between offline data construction and online deployment, we establish a self-evolving training loop by incorporating user feedback collected during online interaction into the data generation process, enabling continual adaptation to user preferences. Extensive experiments demonstrate that our framework achieves superior better latency-quality trade-off, stronger speech-motion synchronization, and higher user preference than competitive existing baselines. Project Page: this https URL
102. 【2608.17889】VisDocAgentBench: Benchmarking Agents for Visually Rich Document Retrieval
链接：https://arxiv.org/abs/2608.17889
作者：Lexiang Hu,Yanzhao Zhang,Mingxin Li,Dingkun Long,Yikang Li,Fuwei Zhang,Yisen Wang,Zhouchen Lin
类目：Information Retrieval (cs.IR); Artificial Intelligence (cs.AI); Computer Vision and Pattern Recognition (cs.CV)
关键词：Visually rich documents, Visually rich, rich documents encode, structured visual elements, documents encode relevance
备注：
点击查看摘要
Abstract:Visually rich documents encode relevance through language, layout, structured visual elements, and corpus context, yet retrieval is typically evaluated by one-shot query--page matching. Agentic-search benchmarks usually score downstream question answering or report generation, leaving document ranking under iterative evidence acquisition underexplored. We introduce VisDocAgentBench, a closed-corpus benchmark comparing static and agentic retrieval under a shared ranked-output contract. It contains 2,375 pages from 100 documents and 120 unique-target queries balanced across direct, one-bridge, and two-bridge evidence structures. Relation-preserving construction yields semantic, relational, and visual queries, followed by full-document review and hard-negative validation. A strong late-interaction visual retriever reaches 97.50% Recall@1 on direct items but 2.50% on two-bridge items, exposing the limits of query--target matching when relevance depends on corpus context. Agents recover much of this loss, but planner choice and retrieval representation remain decisive. Every planner performs better with visual retrieval, whose best R@1 reaches 67.50% versus 37.50% for OCR-text. Ablations identify iterative search and page inspection as consequential capabilities, and providing the complete support context improves ranking on both routes. Trace analysis localizes the remaining losses to target discovery, candidate examination, and evidence-role integration. These findings motivate retrieval agents that combine modality-preserving discovery with evidence-directed verification.
103. 【2608.25930】Controlling for Omitted Variable Bias in Deep Neural Networks
链接：https://arxiv.org/abs/2608.25930
作者：Manuel Pfeuffer,Roshan Prakash Rane,Kerstin Ritter,Sonja Greven
类目：Methodology (stat.ME); Computer Vision and Pattern Recognition (cs.CV); Machine Learning (cs.LG)
关键词：omitted variable bias, omitted variable, deep learning, variable bias, deep learning models
备注： 28 pages, 14 figures. Code at [this https URL](https://github.com/mpff/cocodeel)
点击查看摘要
Abstract:Control variables are widely used in statistical modelling to account for omitted variable bias of known confounders. However, they have largely been underexplored in deep learning. This is surprising, given that deep learning models encode image-inferable covariates, such as demographic variables, into their predictions when these covariates are correlated with the outcome---a form of omitted variable bias referred to as 'shortcut learning'. While many existing confound-control or fairness methods try to restrict the correlation of such covariates with model predictions, we show that this fails to correct for omitted variable bias. We therefore propose a control variable approach for deep learning models, based on generalised additive modelling of the effects of model inputs and covariates. As flexible additive models can suffer from concurvity, we introduce an estimation procedure that refits the final layer of a pre-trained network to include covariate effects, using cross-fitting with ridge penalisation. We show how these effects can be orthogonalised with respect to covariates to exclude their mediated effects and that model predictions can be marginalised over the covariate distribution to control for their effect. This yields unbiased, interpretable predictions and offers flexibility to model the desired effects depending on the scientific or fairness objective. We verify our approach using simulated images, and demonstrate consistent estimation of true effects. Existing methods either require more data or fail to recover the true effects. We apply our method to real neuroimaging data with experimentally induced confounding, where it recovers prediction performance to near the level of a model trained on unconfounded data. Code is available at this https URL.
104. 【2608.25410】oken-Oriented Semantic Communication with Pretrained Vision Transformers
链接：https://arxiv.org/abs/2608.25410
作者：Jiwoong Im,Minwoo Kim,Jaeho Lee,Yo-Seb Jeon,Yongjune Kim
类目：ignal Processing (eess.SP); Artificial Intelligence (cs.AI); Computer Vision and Pattern Recognition (cs.CV); Machine Learning (cs.LG)
关键词：resource-constrained edge systems, server collaborative inference, providing a promising, direction for client, edge systems
备注：
点击查看摘要
Abstract:Token communications realize the semantic communication principle at the granularity of transformer tokens, providing a promising direction for client--server collaborative inference in resource-constrained edge systems. However, directly transmitting token embeddings presents two practical challenges: substantial communication cost and limited interoperability across model-specific token embedding spaces. To address these challenges, we propose a \emph{token-oriented} semantic communication framework. In this framework, token-level task relevance determines which compressed image latents are transmitted, enabling token-granular transmission without directly transmitting token embeddings. The framework is modular, coordinating three pretrained components---a lightweight client-side vision transformer (ViT), a learned image compression (LIC) model, and a large server-side ViT---without end-to-end training. The key enabler is the one-to-one spatial alignment between ViT patch tokens and the LIC latent vectors, which allows token-level task relevance to directly determine which latent vectors are transmitted. Building on this alignment, token-aligned LIC selectively transmits task-relevant latents, layer-selective attention rollout estimates token relevance from a selected range of attention layers in a single forward pass, and surrogate token substitution adapts the frozen server model by optimizing a single learnable token. Experiments on ImageNet show that the proposed framework achieves a more favorable rate--accuracy trade-off than recent semantic communication schemes, hand-crafted codecs, and task-agnostic LIC models.
105. 【2608.25127】Learning spatially varying regularisation parameters of low regularity for image reconstruction
链接：https://arxiv.org/abs/2608.25127
作者：Kostas Papafitsoros,Luca Calatroni,Andreas Kofler
类目：Image and Video Processing (eess.IV); Computer Vision and Pattern Recognition (cs.CV); Optimization and Control (math.OC)
关键词：Total Generalised Variation, spatially adaptive regularisation, Total Variation, Generalised Variation, Total Generalised
备注：
点击查看摘要
Abstract:In this chapter, we review and discuss the regularity properties of spatially adaptive regularisation weight functions used in variational image reconstruction. Incorporating such weights into classical model-based regularisers, such as Total Variation (TV) and Total Generalised Variation (TGV), allows the regularisation strength to vary across the image and adapt to local image content. When appropriately estimated, these weights can thus significantly improve edge and detail preservation in the reconstructions. We review the existing theoretical literature on this topic for different regularity classes, including constant, continuous, and piecewise constant functions. Our discussion is motivated by recent work on hybrid image reconstruction methods that combine model-based regularisation with deep neural networks to learn highly adaptive regularisation weights. In particular, we discuss how the structural properties of these weights influence the reconstruction from both theoretical and practical perspectives. Through representative examples in image denoising and magnetic resonance imaging (MRI) reconstruction, we demonstrate that the learned weights are often of low regularity and can adapt not only to the image structure but also to the specific noise realisation. We conclude by highlighting several directions for future research on this topic.
106. 【2608.25109】Improving Cross-Site Whole-Heart Segmentation
链接：https://arxiv.org/abs/2608.25109
作者：Tanish Mudaliar,Justin Li,Daniel Lin,Julianna Vo,Kaitao Liao,Xin Wang,Shu Hu
类目：Image and Video Processing (eess.IV); Computer Vision and Pattern Recognition (cs.CV)
关键词：multi-modality distribution shift, CARE whole-heart segmentation, Whole-heart segmentation, essential for quantitative, remains challenging
备注： 12 pages, 2 figures. Accepted to the MICCAI 2026 for the CARE Whole Heart Segmentation Challenge proceedings
点击查看摘要
Abstract:Whole-heart segmentation from CT and MRI is essential for quantitative cardiac image analysis, but remains challenging under multi-center and multi-modality distribution shift. In the CARE whole-heart segmentation task, models must generalize from limited labeled sites to unseen acquisition distributions, where variation in spacing, intensity, reconstruction texture, and anatomy can degrade out-of-distribution performance. We propose a modality-routed 3D cardiac segmentation pipeline that combines TotalSegmentator-initialized nnU-Netv2 models with site-characterized, label-preserving appearance augmentation. We first characterize the available sites using measurable image properties and use this analysis to motivate candidate data-space generalization routes. The final retained recipe applies Bias Field + Bezier appearance augmentation, combining smooth spatial intensity perturbation with nonlinear intensity remapping, followed by lightweight class-wise largest-connected-component cleanup. On the primary held-out-site validation splits, the final configuration improves CT mean Dice from 0.8350 to 0.9135 and MRI mean Dice from 0.7695 to 0.7830, while also reducing HD95. These results suggest that site-motivated appearance augmentation is a practical strategy for improving cross-site robustness in limited-data whole-heart segmentation. Our code can be found in this https URL
107. 【2608.24931】Modality Contribution Score - A Per-Patient Framework for Quantifying the Relative Diagnostic Contribution of Structural MRI and Amyloid PET in Alzheimer's Disease
链接：https://arxiv.org/abs/2608.24931
作者：Dawa Chyophel Lepcha,Aaliya Ali,Sophie A. Martin,Deepika Koundal,Pierrick Coupe,Shabbir Syed-Abdul
类目：Image and Video Processing (eess.IV); Artificial Intelligence (cs.AI); Computer Vision and Pattern Recognition (cs.CV); Machine Learning (cs.LG)
关键词：Multimodal neuroimaging combining, positron emission tomography, complementary structure-function relationships, existing artificial intelligence, artificial intelligence systems
备注： 15 pages, 7 figures, Under review
点击查看摘要
Abstract:Multimodal neuroimaging combining structural MRI and positron emission tomography (PET) captures complementary structure-function relationships across the Alzheimer's disease (AD) continuum, yet existing artificial intelligence systems produce a single diagnostic label without quantifying which imaging modality drove that decision for a specific patient. We introduce the Modality Contribution Network (MCNet) and the Modality Contribution Score (MCS), the first per-patient attribution framework quantifying the shift in modality dominance from structural atrophy to amyloid and metabolic dysfunction across the cognitively normal to MCI to AD continuum. MCS is normalised to unity per subject via modality ablation (MCS_MRI_i + MCS_PET_i = 1.0 for every subject i), providing an interpretable, clinically actionable score that fluid biomarkers cannot supply. Applied to 327 ADNI-3 participants balanced across cognitively normal, mild cognitive impairment, and AD groups, MCNet achieved competitive three-class staging performance (AUC=0.881). The MCS revealed a statistically significant monotonic gradient (Kruskal-Wallis p0.0001), with increasing PET dominance from cognitively normal (MCS_PET 0.412+/-0.229) through MCI (0.489+/-0.289) to AD (0.671+/-0.426), validated against amyloid SUVR (r=0.172, p=0.006) and FDG metabolic biomarkers (r=-0.287, p=0.0005) from separate imaging pipelines. External replication in 1,073 independent OASIS-3 subjects confirmed cross-cohort generalisability (H=166.99, p0.0001, eta^2=0.156). A mechanistic comparison with SHAP demonstrated that ablation-based MCS captures clinically meaningful modality dependence that deviation-based methods cannot. These findings position MCNet as a foundation for personalised imaging decisions, clinical trial stratification, and trustworthy AI in dementia care.
