本篇博文主要展示每日从Arxiv论文网站获取的最新论文列表，以自然语言处理、信息检索、计算机视觉等类目进行划分。
统计
今日共更新615篇论文，其中：
- 自然语言处理107篇
- 信息检索13篇
- 计算机视觉131篇
自然语言处理
1. 【2608.05148】Reasoning Core: Designing Broad Procedural Data for Completion-Supervised Reasoning Training
链接：https://arxiv.org/abs/2608.05148
作者：Damien Sileo,Valentin Lacombe,Dimitri Kachler
类目：Computation and Language (cs.CL)
关键词：verifiable reasoning problems, Reasoning Core, Procedural generators produce, introduce Reasoning Core, compare Reasoning Core
备注： 20 pages, 3 figures. Code: [this https URL](https://github.com/sileod/reasoning-core) Data: [this https URL](https://hf.co/collections/reasoning-core/datasets)
点击查看摘要
Abstract:Procedural generators produce useful verifiable reasoning problems at scale, but have received less attention as data for completion-supervised fine-tuning. We introduce Reasoning Core, a collection of 50 generators spanning mathematics, logic, planning, state tracking, formal languages, structured data, games, causality, and code, with semantic scorers, difficulty controls, and task evaluators. Under a matched completion-supervised protocol, we compare Reasoning Core with Procedural Warmup, Reasoning Gym, and SynLogic across four base-model settings and multiple training durations. In the primary 3B comparison, Reasoning Core achieves the highest mean scores on DROP, LogiQA, and ARC-Challenge, exceeding both the baseline without procedural data and all three alternative procedural collections. Task-level analyses show that semantic validity alone does not ensure training utility, highlighting compact targets and calibrated difficulty as important design factors. We ran audits combining model-assisted review, human adjudication, and regression testing. Applied throughout Reasoning Core development and to the other collections, they reveal subtle mismatches among generation, rendering, targets, and scoring, a reminder that procedural generation alone does not guarantee correctness. The library, generated datasets, and audit material are publicly available.
2. 【2608.05139】oward Skill-Native LLMs: Skill Entropy for Benchmarking and Training Long-Horizon Reasoning
链接：https://arxiv.org/abs/2608.05139
作者：Yinghui He,Ling Yang,Jiarui Liu,Yongjin Yang,Lechen Zhang,Yingcheng Wu,Zhenfei Yin,Mengdi Wang,Sanjeev Arora
类目：Computation and Language (cs.CL); Machine Learning (cs.LG)
关键词：recent LLMs demands, distinct skills inside, Skill, cross-skill long-horizon tasks, math derivation
备注： [this https URL](https://github.com/Gen-Verse/Skill-Entropy-RL)
点击查看摘要
Abstract:Long-horizon reasoning in recent LLMs demands that the model switch between distinct skills inside a reasoning chain, such as first doing a math derivation, then using the result to plan a schedule. We call such problems cross-skill long-horizon tasks: multi-step tasks whose steps require different reasoning skills and depend on earlier outputs. Existing benchmarks often evaluate individual skills, lacking a principled way to measure how well a model switches between skills. We address this gap from both the evaluation and training sides. We introduce Skill Entropy, a measure of the difficulty of switching from one skill to another. We then propose Skill^2-Bench, a benchmark of cross-skill long-horizon tasks built over 558 skills across 9 verifiable and open-ended domains. Each task is assigned a task-level skill-entropy score and grouped into three difficulty levels. Evaluating 8 frontier and 4 open-source models on Skill^2-Bench reveals a skill-switching gap: accuracy decreases on higher-entropy tasks. We then turn skill entropy from a benchmark scale into a training signal. We propose Skill-Entropy RL, an RL framework where the model predicts not only the answer at each step but also the skill used to produce it. The reward combines step-level correctness with a skill-entropy reward that measures the alignment between the model-predicted skill sequence and the gold skill sequence. On Qwen3-4B-Instruct and Qwen3-1.7B, Skill-Entropy RL improves the Skill^2-Bench score from 34.4% to 68.4% and from 14.6% to 40.1%, respectively, outperforming competitive baselines. The same pipeline can be applied to off-the-shelf training data such as OpenR1-Math, indicating that skill entropy is a reusable training signal. Code available at: this https URL
3. 【2608.05126】Spoken Function Calling: A New Perspective on Spoken Language Understanding for Large Audio Language Models
链接：https://arxiv.org/abs/2608.05126
作者：Yuezhang Peng,Yuxin Liu,Changfeng Gao,Zhifu Gao,Xiangang Li,Xie Chen
类目：Computation and Language (cs.CL); Multimedia (cs.MM)
关键词：seamless human-agent interaction, achieving seamless human-agent, Audio Language Models, Large Language Models, task-oriented dialogue systems
备注： ACM Multimedia 2026
点击查看摘要
Abstract:Spoken Language Understanding (SLU) is the core component of task-oriented dialogue systems and a pivotal link in achieving seamless human-agent interaction. While traditional SLU can effectively extract user semantics for closed-set tasks after in-domain supervised fine-tuning, it faces significant challenges in leveraging in-context learning for open-domain tasks due to its ambiguous rule definitions. This work proposes Spoken Function Calling (SFC), a novel semantic understanding perspective that optimizes semantic understanding with structured rule definitions, to evolve beyond traditional closed-set SLU. Specifically, we curate and extend a suite of spoken functions based on traditional SLU datasets, construct a multi-agent system to synthesize the SFC-Bench dataset, evaluate the performance of Large Language Models (LLMs) and Large Audio Language Models (LALMs), and enhance the SFC capabilities of LALMs through post-training. Experiments demonstrate that SFC outperforms traditional SLU, substantially enhancing the semantic extraction accuracy for LLMs and LALMs.
4. 【2608.05124】Chained Recursive Language Models for Multi-Iteration Reasoning
链接：https://arxiv.org/abs/2608.05124
作者：Purbesh Mitra,Sennur Ulukus
类目：Computation and Language (cs.CL); Artificial Intelligence (cs.AI); Information Theory (cs.IT); Machine Learning (cs.LG); Signal Processing (eess.SP)
关键词：Long context reasoning, store intermediate state, single inference trajectory, verify evidence, Chained Recursive Language
备注：
点击查看摘要
Abstract:Long context reasoning in large language models (LLMs) is usually constrained by the fact that a single inference trajectory has to simultaneously explore the context, store intermediate state, verify evidence, and produce the final answer. This becomes particularly difficult in tasks that require extraction, counting, ordering, or multi-hop reasoning, where an early mistake can propagate until the final response. In this work, we propose Chained Recursive Language Models (Chained RLM), an inference-time architecture, in which the same underlying model is called repeatedly as a sequence of fresh reasoning roots. Each root receives the original problem and context, but does not inherit the full conversational history. Instead, it receives a compact plain-text summary, a plain-text blackboard, and some durable task-specific artifacts written by predecessor roots. The motivation is to manage the context by chopping into partial tasks rather than one large inference response; in each staged computation, intermediate artifacts can be inspected, corrected, and extended by a later fresh inference by the same model. We describe the system model, handoff mechanism, artifact workspace, and evaluation protocol for this system. We study when fresh-context artifact continuation gives a measurable gain in accuracy over direct LLM answering even with recursive tool-calling.
5. 【2608.05097】Same Formulas, Different Semantics: Do Language Models Follow Modal Logic Specifications?
链接：https://arxiv.org/abs/2608.05097
作者：Réemi Andrieu,Damien Sileo
类目：Computation and Language (cs.CL)
关键词：necessity and possibility, assumptions about accessibility, accessibility between worlds, objects exist, Abstract
备注： 9 pages. Code: [this https URL](https://github.com/sileod/modal-semantics-reasoning) . Data and artifacts: [this https URL](https://huggingface.co/datasets/sileod/modal-semantics-reasoning)
点击查看摘要
Abstract:Reasoning about necessity and possibility depends on assumptions about accessibility between worlds and about which objects exist at each one. The same inference may therefore hold under one modal system and fail under another. Evaluating language models on such problems requires testing whether their judgments follow the stated semantics rather than a familiar logic. We construct paired modal problems with identical premises and conjecture but different frame or domain conditions; automated reasoning verifies opposite labels. A balanced core prevents the semantic condition alone from revealing the answer. On this core, four of five recent models perform below the condition-only baseline under direct prompting. Yet enabling reasoning mode raises DeepSeek V4 Flash from 4.4% to 88.1% on unchanged prompts. Following stipulated modal semantics thus depends strongly on inference mode as well as model identity. When frame conditions are omitted, models often agree but fit different familiar logics best. We release the formulas, oracle artifacts, countermodels, and responses.
6. 【2608.05086】Item Response Theory for AI Safety
链接：https://arxiv.org/abs/2608.05086
作者：Joshua Fonseca Rivera(1),Neil Shah(1),David Demitri Africa(2),Konstantinos Voudouris(2) ((1) Independent, (2) UK AI Security Institute)
类目：Artificial Intelligence (cs.AI); Computation and Language (cs.CL)
关键词：Language models differ, Item Response Theory, safely they behave, differences are measured, Language models
备注： 15 pages, 9 figures, 6 tables
点击查看摘要
Abstract:Language models differ in how safely they behave and these differences are measured by safety benchmarks. But aggregated benchmark scores are hard to trust and interpret, because benchmarks duplicate one another, correlate heavily, and models may sandbag when they detect evaluation. To address these issues, we draw on Item Response Theory (IRT), a statistical toolkit for measuring these latents from performance on items with inferred psychometric properties. We fit IRT models to eight safety benchmarks across 192 language models, the largest psychometric analysis of LLM safety evaluations to date, and contribute three results. First, we find that three interpretable factors of refusal strictness, truthfulness, and contextual harm explain most of the variance between models across benchmarks. Second, psychometrically selected items recover full benchmark scores with lower error than random subsets of the same size, and roughly ten adaptively chosen items suffice for several individual benchmarks, cutting evaluation cost by 97-99%. Third, IRT supports audits of individual models, showing that it can be used to detect naive sandbagging and changes of model behind APIs. Overall, we show IRT is a ready-made toolkit for reading, reducing, and auditing safety benchmarks, which we recommend frontier labs and evaluators adopt.
7. 【2608.05080】Optimizing What Policies Learn From: Recoverability-aware Rollout Intervention Learning
链接：https://arxiv.org/abs/2608.05080
作者：Zheyuan Zhang,Manqing Mao,Hong Wang,Zhuoer Wang,Samson Koelle,Jie Yuan,Yanjun Lin,James Feng,Nikki Lijing Kuang,Yanfang Ye,Wei Niu
类目：Machine Learning (cs.LG); Computation and Language (cs.CL)
关键词：Critic-free group-based reinforcement, Critic-free group-based, group-based reinforcement learning, large language models, post-training large language
备注：
点击查看摘要
Abstract:Critic-free group-based reinforcement learning has become a scalable approach for post-training large language models. However, most existing methods allocate the same number of rollouts to every task and trajectory state, even though some rollouts provide much more useful learning signals than others. Recent work has started to treat rollout generation as an adaptive decision, but two important limitations remain. First, intervention strategies are often based on fixed heuristics and therefore cannot adjust as the policy changes during training. Second, these methods usually decide only how many rollouts to generate, without explicitly controlling where and how to intervene. To address these limitations, we propose Recoverability-Aware Intervention Learning (RAIL), a training-time framework that learns how to generate rollouts based on the improvement produced by each intervention. RAIL models intervention selection as an online contextual-bandit problem and trains a recoverability controller using intervention traces collected through a shadow-to-live procedure. This allows the controller to keep learning while the underlying policy evolves. We evaluate RAIL in terms of effectiveness, adaptivity, expressiveness, and efficiency. Across multiple settings, RAIL consistently improves performance under limited rollout budgets. These results show that recoverability-aware intervention provides a principled way to generate more informative and less redundant rollouts, leading to stronger learning signals during post-training.
8. 【2608.05075】German parties shifted towards intuition-based rhetoric after the far right's parliamentary breakthrough
链接：https://arxiv.org/abs/2608.05075
作者：Peer Saleth,Segun T. Aroyehun,Fabio Carrella,Christoph M. Abels,Stephan Lewandowsky,David Garcia
类目：Computation and Language (cs.CL)
关键词：remains poorly understood, elites' rhetorical commitments, political elites' rhetorical, populist actors remains, actors remains poorly
备注： 34 pages, 6 figures; includes 49 pages of Supplementary Information. Code available at [this https URL](https://github.com/peersal/German-EMI) , data at [this https URL](https://osf.io/x3zpc/)
点击查看摘要
Abstract:The spread of misinformation is widely perceived as a threat to democratic deliberation, yet how political elites' rhetorical commitments to truth shift alongside the rise of populist actors remains poorly understood. Analysing 4.5 million tweets and 59,170 parliamentary speeches by German political elites between 2015 and 2025, we measure evidence-based and intuition-based rhetoric using a validated distributed dictionary representation. Across both arenas, intuition-based language has become more prominent, and right-leaning actors consistently exhibit the lowest Evidence Minus Intuition (EMI) scores. The parliamentary entry of the extreme-right Alternative for Germany (AfD) in 2017 coincides with sharp downward shifts in EMI across the broader chamber, while a more gradual decline is observed on Twitter. These findings document an association between far-right visibility and a changing approach to truth in elite discourse in a multiparty European democracy.
9. 【2608.05064】Provable Limits and Certified Deferral for Verbalized Uncertainty in Small Language Models
链接：https://arxiv.org/abs/2608.05064
作者：Jianru Shen
类目：Computation and Language (cs.CL); Artificial Intelligence (cs.AI); Machine Learning (cs.LG)
关键词：key deployment question, Small open-weight language, open-weight language models, language models increasingly, models increasingly run
备注： Accepted at MIWAI 2026 (The 19th International Conference on Multi-disciplinary Trends in Artificial Intelligence), to appear in Springer LNAI
点击查看摘要
Abstract:Small open-weight language models increasingly run in private, offline, and cost-sensitive settings, where the key deployment question is not only what a model answers but when it should defer to a human. We study whether verbalized confidence can support risk-controlled deferral, evaluating eleven instruction-tuned models from three families, 0.5B to 14B parameters, on ARC-Challenge and TruthfulQA with 25,168 local predictions. Three theoretical results delimit what calibration can provide: strictly monotone calibration preserves the risk-coverage frontier and error-detection AUROC; temperature scaling cannot calibrate models whose confidence stays above one half while accuracy falls below it; and a Clopper-Pearson procedure converts a 200-question calibration set into a finite-sample risk certificate under an i.i.d. deployment assumption. Empirically, eight of 22 model-task pairs hit the temperature-scaling infeasibility floor within one percentage point of the predicted bound. Platt scaling reduces ECE to as low as 0.02, yet certified autonomy at a 20% risk budget is granted to only three model-task pairs and to none at 10%. We also identify and repair an answer-ordering artifact in the multiple-choice form of TruthfulQA. Calibration gives confidence semantics; certified deferral determines when small models are safe to use.
10. 【2608.05050】he Effect of Perceived Race and Gender on Police Language Use: Experimental Evidence from VR Simulations
链接：https://arxiv.org/abs/2608.05050
作者：Sandra C. Sandoval,Navita Goyal,Rashawn Ray,Long Doan,Rachel Rudinger,Hal Daumé III
类目：Computers and Society (cs.CY); Artificial Intelligence (cs.AI); Computation and Language (cs.CL)
关键词：virtual characters depicted, Black adult males, Black man character, Black man characters, police officers speak
备注：
点击查看摘要
Abstract:Against the backdrop of violence in police interactions with the U.S. public, we explore how deferentially police officers speak to virtual characters depicted as Black adult males in vir- tual reality (VR) simulations. We evaluate the effect of seeing and communicating with these characters through a causal in- ference lens, where the assignment of the Black man character to a police officer and simulation is the treatment variable. Our (marginal) average treatment effect AT E measures the social impact of the character on the deference of officer statements with each turn of the conversation. Soberingly, we find that most officers speak less deferentially to Black man characters, except for White, biracial, and multiracial female officers, es- pecially in settings where the VR character was known to be a suspect. Across a full conversation of a typical VR scene, these marginal AT Es can result in notable changes in def- erence of tone (two to several points difference on a scale of 0-10), above and beyond that due to the initial effect of per- ceiving a Black male character. Even more disconcerting is that this can contribute to conversation breakdowns that po- tentially result in violence or danger to both the public and the police. We also explored the capabilities of large language models (LLMs) for ATE estimation. From our methods com- parison analysis, including model validation against synthetic data, we provide unique scientific insights on LLM-assisted methodologies for ATE estimation. As such, for ATE esti- mation with multilevel data with text, we recommend mixed effects models with the inverse propensity treatment weighted (iptw) approach, which utilized an LLM for text feature cre- ation. While we also tested LLMs for finetuning prediction models ultimately for ATE estimation, we conclude they are an area for further development and refinement.
11. 【2608.05045】Gradient Immunity: Null-Space Resistance to Malicious Fine-Tuning
链接：https://arxiv.org/abs/2608.05045
作者：Yuxuan Huang,Xingyu Zeng,Tianhang Zheng,Chaochao Lu
类目：Cryptography and Security (cs.CR); Artificial Intelligence (cs.AI); Computation and Language (cs.CL)
关键词：Released aligned large, aligned large language, Released aligned, large language models, models remain vulnerable
备注：
点击查看摘要
Abstract:Released aligned large language models remain vulnerable to malicious downstream finetuning. Existing defenses are largely designed for the fine-tuning-as-a-service (FTaaS) paradigm or rely on downstream users to follow additional safety procedures, and therefore do not directly address the setting we study: a provider controlled partially protected open-weight (PPOW) release setting in which most weights remain trainable while a small safety-critical component is preserved at release. We propose a Unidirectional Safety Gate (USG), instantiated as a Null Space Cubic Layer together with an Inverse Adapter inserted after the final Transformer layer. During downstream fine-tuning, the cubic layer suppresses or blocks gradients from harmful samples whose hidden states fall in a calibrated protected region, while the Inverse Adapter restores the base model's forward behavior. In practice, we calibrate a threshold using defender-held harmful data, allowing protection to generalize to nearby in-distribution harmful samples. Across six evaluated model-dataset settings, USG keeps post-finetuning attack success rate close to the pre-release level under a fixed release threshold, while maintaining high safe-pass rates on easier settings and exhibiting a clearer safety-utility trade-off on unsafe samples from BeaverTails. These results suggest that release-time representation-space blocking can raise the cost of malicious downstream adaptation without requiring downstream cooperation. The code is available at this https URL.
12. 【2608.05028】Language Models Generalize to Human-like Word Order Preferences
链接：https://arxiv.org/abs/2608.05028
作者：Amanda Popadich,Shane Steinert-Threlkeld
类目：Computation and Language (cs.CL)
关键词：central question, emerge from general, operating over underdetermined, Artificial Language Learning, learning mechanisms operating
备注：
点击查看摘要
Abstract:A central question in language acquisition is whether linguistic biases can emerge from general learning mechanisms operating over underdetermined input. Artificial Language Learning (ALL) studies have shown that human learners reliably generalize beyond the evidence provided, including by preferring scope-homomorphic noun phrase modifier orders. In this work, we investigate whether language models exhibit the same bias under similar conditions. We create a controlled learning environment in which models are trained on a corpus where all noun phrases containing multiple modifiers have been removed, eliminating direct evidence about modifier ordering, and are then evaluated on multiple modifier sentences. Across three model sizes, we find that they consistently prefer scope-homomorphic orders despite never observing them during training. These preferences vary in strength by modifier type. To investigate the source of these preferences, we examine noun-modifier association strength using pointwise mutual information (PMI). While PMI reflects known modifier-ordering patterns, it does not explain the models' ordering preferences. These findings demonstrate that LMs can recover human-like linguistic generalizations from impoverished input and provide a controlled framework for investigating the mechanisms underlying such biases.
13. 【2608.05013】OneDayAgent: Towards a Long-Horizon Harness for Autonomous Agents
链接：https://arxiv.org/abs/2608.05013
作者：Jingsheng Zheng,Xinyuan Fang,Jintian Zhang,Zhengke Gui,Huajun Chen,Ningyu Zhang
类目：Computation and Language (cs.CL); Artificial Intelligence (cs.AI); Human-Computer Interaction (cs.HC); Machine Learning (cs.LG); Multiagent Systems (cs.MA)
关键词：increasingly applied, open-ended everyday requests, span work, open-ended everyday, everyday requests
备注： Ongoing work
点击查看摘要
Abstract:LLM agents are increasingly applied to open-ended everyday requests that span work, study, and life. These tasks are long-horizon, cross-environment, and multimodal, forcing the agent to preserve goals and constraints across many steps while navigating heterogeneous tools and attachments. While prior work has addressed individual failure modes such as goals drift, states loss, and context overflow, whether a single harness can manage them jointly and remain effective across backends has received less study. We present OneDayAgent, a long-horizon harness for autonomous agents. OneDayAgent turns an open-ended request into a managed execution process that decomposes tasks into bounded subtasks, maintains execution memory under context pressure, and verifies and repairs the final deliverable. We evaluate OneDayAgent on AgentIF-OneDay across 104 tasks. With the GLM-5.2 backend, OneDayAgent sets a new state of the art with an overall score of 0.821. The same harness runs across five backend LLMs from three model families, indicating the harness generalizes across backends without tuning, even as different models induce distinct execution styles under the same workflow.
14. 【2608.05004】DelusionEval: Measuring Delusion-Linked Behaviors in AI Chatbots
链接：https://arxiv.org/abs/2608.05004
作者：Jared Moore,Andrea Mock,Yifan Mai,Jacy Reese Anthis,Ryan Louie,William Agnew,Ashish Mehta,Kevin Klyman,Percy Liang,Nick Haber,Eric Lin,Desmond C. Ong
类目：Computation and Language (cs.CL)
关键词：Mental health professionals, Mental health, large language models, psychological harm, delusional spirals
备注：
点击查看摘要
Abstract:Mental health professionals have raised concerns about risks of psychological harm from interaction with large language models (LLMs), including "delusional spirals" in which concerning human and LLM behaviors reinforce each other over time. With growing public use of LLM-powered chatbots, there is an urgent need to build evaluations grounded in real-world episodes of psychological harm experienced by users. We developed DelusionEval, an evaluation protocol that tests a model's tendencies to exhibit behaviors linked to promoting user delusions. We prompt each model with 589 unique conversation histories from 18 participants, comprising 12,591 messages from users who experienced delusions and psychological harm. We find that the tendency of an evaluated LLM to exhibit delusion-linked behavior does not reliably correlate with model size, release date, or the presence of test-time reasoning. However, extending the context of prior messages substantially increases rates of delusion-linked behaviors, providing evidence for the importance of context in LLM safety evaluation. For example, the rate of failing to discourage self-harm when the user expresses suicidal ideation increases from 30.0% to 41.1% when an additional 350 messages are prepended to the conversation history. All model families (e.g., GPT, Claude) exhibit substantial rates of delusion-linked behaviors. Within families, later, larger, or higher-reasoning models are not uniformly better across all behavior categories. Our results raise concerns regarding the potential psychological impact of LLMs and the need for more rigorous studies of real-world human-AI interaction.
15. 【2608.04980】Protoreasoning in Tiny Transformers
链接：https://arxiv.org/abs/2608.04980
作者：Eduardo Valle,Fergal Reid
类目：Computation and Language (cs.CL); Artificial Intelligence (cs.AI); Machine Learning (cs.LG)
关键词：Chain of Thought, form of Chain, show that tiny, tiny transformers, transformers can profitably
备注：
点击查看摘要
Abstract:We show that tiny transformers can profitably employ a simple form of Chain of Thought, which we call protoreasoning, allowing us to study step-by-step reasoning on ~1M-parameter models and opening up opportunities for much more detailed experimentation and analysis than is feasible for larger models. Current Large Language Models exhibit impressive step-by-step reasoning, but we have yet to understand its generality, i.e., when and how LLMs learn genuinely general algorithms rather than "bags of heuristics." Such questions are hard to settle on compute-intensive frontier models trained on opaque data. To work at model scales far below the threshold for natural-language competence, we define reasoning-friendly tasks on Dyck languages (sentences of correctly nested brackets). We find that protoreasoning traces substantially close the out-of-distribution generalization gap, and ablations confirm that the trace's content, not merely its extra tokens, drives the gain.
16. 【2608.04962】SpecRoll: Fast-Slow Verifier-Feedback Adaptation for Speculative Reinforcement Learning Rollouts
链接：https://arxiv.org/abs/2608.04962
作者：Nhat Minh Pham,Duy Tung Doan,Thi Duyen Ngo,Vinh Van Nguyen,Khac-Hoai Nam Bui
类目：Machine Learning (cs.LG); Computation and Language (cs.CL)
关键词：major efficiency bottleneck, Reinforcement learning, large language models, post-training improves, efficiency bottleneck
备注：
点击查看摘要
Abstract:Reinforcement learning (RL) post-training improves the reasoning capabilities of large language models, but autoregressive rollout generation remains a major efficiency bottleneck. Speculative decoding can accelerate generation, yet applying it during RL is difficult because the target policy continually evolves: static proposers become stale, while frequent drafter updates add substantial overhead. We introduce SpecRoll, a speculative rollout engine that preserves the target model's sampling distribution while adapting at two timescales. Lightweight future-token heads generate parallel proposals, while our proposed Reflex module uses delayed verifier feedback to perform bounded, trajectory-local hidden-state corrections without backpropagation. A complementary slow path updates the head parameters only when sustained degradation is detected. SpecRoll combines these mechanisms with concurrency-aware sparse-tree verification and exact target verification, leaving the target rollout distribution and GRPO objective unchanged. Across five models ranging from 1.5B to 14B and three mathematical reasoning datasets, SpecRoll achieves 1.26-2.15x generation speedup and 1.21-2.04x end-to-end speedup over vanilla GRPO. It also outperforms FastGRPO in both generation and end-to-end time across all 15 matched settings, with an average pairwise end-to-end gain of 1.18x. Controlled ablations show that the fast and slow adaptation paths provide complementary benefits. Our source code is available at this https URL.
17. 【2608.04949】UG-UMRE: Uncertainty-Guided Modality Augmentation and Distributional Calibration for Unified Multimodal Relation Extraction
链接：https://arxiv.org/abs/2608.04949
作者：Bo Kong,Liruiz Jia,Yi Liang,Chao Liu,Dongfang Han,Tianwei Yan,Yuan Liu,Shengquan Liu
类目：Computer Vision and Pattern Recognition (cs.CV); Computation and Language (cs.CL); Information Theory (cs.IT); Multimedia (cs.MM)
关键词：Unified Multimodal Relation, Multimodal Relation Extraction, Unified Multimodal, Relation Extraction, Multimodal Relation
备注： Accepted at ACM MM2026
点击查看摘要
Abstract:Unified Multimodal Relation Extraction (UMRE) aims to identify intra-modal and cross-modal relations between textual entities and visual objects. However, existing UMRE studies still encounter two critical issues: ignoring inherent aleatoric uncertainty causes noise propagation, and deep-seated heterogeneity between distinct modal distributions hinders alignment. To address these issues, we propose the Uncertainty-Guided UMRE Network (UG-UMRE). Specifically, we design an Uncertainty-Driven Unimodal Augmentation (UDUA) module, which models features as Gaussian distributions based on the Variational Information Bottleneck. By incorporating an uncertainty-aware self-supervised contrastive learning mechanism, UDUA effectively filters out noise while maintaining semantic consistency. Furthermore, we introduce the Joint Aleatoric Uncertainty Alignment (JAUA) module as a global semantic pre-calibration mechanism. JAUA leverages probabilistic distribution consistency to construct a shared latent space, eliminating the distributional gap by synchronizing cross-modal statistical properties, thereby laying a robust foundation for fine-grained interaction. Experiments on three benchmark datasets (UMRE, MORE, and MNRE) demonstrate that UG-UMRE achieves state-of-the-art performance. Further analysis validates the pluggable and effective performance of the proposed UDUA and JAUA modules.
18. 【2608.04939】Reading Between the Frames: Interpreting Implicit and Non-literal Meaning in Social Media Videos
链接：https://arxiv.org/abs/2608.04939
作者：Yang Wang,Yanan Ma,Yiqi Liu,Zi Yan Chang,Chi-Li Chen,Chia-Yi Hsiao,Tyler Loakman,Aline Villavicencio,Chenghao Xiao,Chenghua Lin
类目：Computation and Language (cs.CL)
关键词：visible actions, Social media, Social media videos, communicate meanings, models
备注：
点击查看摘要
Abstract:Social media videos often communicate meanings that go beyond their visible actions, captions, or speech. A mundane clip may become humorous, ironic, or satire only through the interaction of multimodal cues and cultural context, making such content a difficult test case for video-language models. In this paper, we introduce \textit{DrivelHub+}, a benchmark for evaluating whether models can infer the implicit, non-linear, and rhetorically layered meanings of social media videos that appear nonsensical on the surface but convey deliberate pragmatic meanings. DrivelHub+ consists of 1,000 videos collected from social media, each annotated with a human-written implicit narrative explanation. Unlike conventional video understanding tasks focused on recognition or description, we present a benchmark that targets contextual multimodal reasoning. We evaluate current video-language models from two perspectives: explanation, where models must explain the pragmatic comprehension of a video in natural language; and representation, where we adapt reasoning-as-retrieval to test whether model representations align videos with their corresponding implicit narratives in both video-to-text and text-to-video retrieval. Our benchmark provides a diagnostic setting for measuring the gap between multimodal perception and pragmatic comprehension, asking whether current models can move beyond describing what is shown to inferring what is meant.
19. 【2608.04934】State2State: Environment-Derived Mid-Training for LLM Agents
链接：https://arxiv.org/abs/2608.04934
作者：Xuanyu Lei,Yiqi Zhu,Chenliang Li,Kaiming Liu,Peng Li,Ming Yan,Jieping Ye,Ya-Qin Zhang,Yang Liu
类目：Computation and Language (cs.CL); Machine Learning (cs.LG)
关键词：LLM agents commonly, Training LLM agents, online reinforcement learning, agents commonly relies, Training LLM
备注： Work in progress
点击查看摘要
Abstract:Training LLM agents commonly relies on supervised fine-tuning from expert trajectories or online reinforcement learning over human-specified tasks with handcrafted verifiers. Though effective, both remain bottlenecked by externally specified tasks and supervision signals, limiting the scalability and diversity of agent training. We study an environment learning paradigm in which agents acquire interaction and manipulation capabilities solely through environment interaction, without externally specified tasks. We propose State2State, an environment-derived mid-training method that converts explored environment states into training objectives, challenging agents to reach a specified target state. By deriving tasks from environment exploration and verifying success through rule-based state matching, State2State provides scalable and verifiable training objectives without expert supervision or manual task design. Experiments on ALFWorld and ScienceWorld show that State2State improves agent performance as a standalone environment-learning stage in most settings. As initialization for downstream RL, it further improves final performance and learning efficiency, with promising evidence of cross-environment generalization.
20. 【2608.04928】Does Out-of-Sight Equal Out-of-Mind in CoT Monitorability?
链接：https://arxiv.org/abs/2608.04928
作者：Pedro Ferreira,Wilker Aziz,Ivan Titov
类目：Computation and Language (cs.CL)
关键词：large language models, motivating work, offers a window, decision-making of large, large language
备注： 23 pages
点击查看摘要
Abstract:Chain-of-thought (CoT) reasoning offers a window into the decision-making of large language models (LLMs), which can be monitored for target behaviors by reading the reasoning trace, motivating work on CoT monitorability. Latent CoT approaches, however, replace the explicit tokens with a small number of continuous states, lowering inference costs but removing the readable trace this monitoring relies on. Monitoring then requires alternative access to the model, such as probing its activations or verbalizing the latent states back into text, but how much monitorability these alternatives preserve is unclear. We study this question with a hint-based intervention setup, a proxy for behaviors where models exploit biasing input cues, e.g., an inadvertently leaked answer or a belief stated by the user, without acknowledging them. Taking hint-reliance as the monitorability target, we compare monitors across reasoning modes, from explicit CoT to weakly- and strongly-supervised latent CoT, on math reasoning and question answering. We find that, in this setup, monitorability depends more on properties of the task (such as whether the correct answer constrains the supporting reasoning) and the level of access to model internals than on the reasoning mode.
21. 【2608.04926】Consistency-Driven Co-Evolution for Self-Supervised Cross-Representation Learning
链接：https://arxiv.org/abs/2608.04926
作者：Xuehang Guo,Pengyuan Li,Tom Hope,Tirthankar Ghosal,Manling Li,Qingyun Wang
类目：Machine Learning (cs.LG); Artificial Intelligence (cs.AI); Computation and Language (cs.CL)
关键词：play increasingly important, increasingly important roles, modalities poses fundamental, poses fundamental challenges, visualization code play
备注：
点击查看摘要
Abstract:As chart images, tabular data, and visualization code play increasingly important roles across diverse domains, cross-representation understanding across these modalities poses fundamental challenges for AI systems: the relationships across representations are inherently \textit{one-to-many}, supervision is ambiguous and costly, and model optimization lacks a principled signal that is both direction-adaptive and representation-generalizable beyond task-specific objectives. We introduce CoCoEvolve to improve consistency across chart, table, and code representations. Instead of treating cross-representation mapping as a one-to-many problem, we define explicit one-to-one correspondences and optimize models using agreement between representations, without additional annotations. During training, CoCoEvolve@Train performs co-evolution across the chart-table-code cycle, while CoCoEvolve@Test applies the same consistency objective at inference time for test-time co-optimization. We also present CoCoEvolve@Eval, an evaluation suite covering all six cross-representation tasks. Across four benchmarks, CoCoEvolve improves performance in both training-time and test-time settings. Our project page: this https URL.
22. 【2608.04904】Strengthening Target-Language Features: SAE-Based Steering for Multilingual Inference
链接：https://arxiv.org/abs/2608.04904
作者：Hongsheng Wang,Phlipp Koehn
类目：Computation and Language (cs.CL)
关键词：exhibit substantial performance, substantial performance differences, require parameter updates, existing adaptation methods, models exhibit substantial
备注：
点击查看摘要
Abstract:Multilingual large language models exhibit substantial performance differences across languages, while existing adaptation methods often require parameter updates and considerable multilingual training data. We propose an inference-time multilingual steering method that uses pretrained sparse autoencoders to identify and strengthen target-language-related features. Using multilingual parallel sentences, we compare SAE activations across languages and select a small number of layer-specific features associated with each target language. These features are decoded into steering signals and injected into the model's hidden states without additional training. Experiments with Gemma-3-12B-it show average accuracy improvements of 10.9 percentage points on XCOPA, 5.3 points on XNLI, and 1.9 points on MGSM.
23. 【2608.04899】Evaluation Pitfalls and Sparsity Limitations in LLM-based Confidence Estimates for Classification
链接：https://arxiv.org/abs/2608.04899
作者：Elena Merdjanovska,Omar Zaidan,Andreas Rücklé
类目：Computation and Language (cs.CL)
关键词：indicating when predictions, estimation is essential, Confidence estimation, Abstract, extremely sparse outputs
备注： Published at Findings of ACL 2026
点击查看摘要
Abstract:Confidence estimation is essential when LLMs are used for classification, indicating when predictions can be trusted. However, common approaches such as verbalization produce extremely sparse outputs. For instance, Qwen3-32B verbalizes only eight unique confidence values on SST-2, with over half being exactly 95%, a pattern we observe consistently across four datasets and two LLMs. Besides limiting practical utility, we show that this sparsity critically affects evaluation: the choice of interpolation in area under the accuracy-rejection curve (AUARC) dramatically alters rankings, with consistency sampling dropping from best to worst under stepwise versus linear interpolation. We advocate for standardizing stepwise interpolation for a fairer comparison. Under such a fair evaluation, we find that weighting verbalized digits by token probabilities, a method we term verbalization logprobs, addresses sparsity and achieves the best AUARC (+2.3 points over vanilla verbalization) without incurring additional inference cost.
24. 【2608.04885】Evaluating the Diagnostic Robustness of Vision-Language Models Under Visual and Textual Perturbations
链接：https://arxiv.org/abs/2608.04885
作者：Ali Khoramfar,Mohammad Javad Dousti,Alireza Mohamadian,Heshaam Faili
类目：Computer Vision and Pattern Recognition (cs.CV); Computation and Language (cs.CL)
关键词：Standard accuracy metrics, sensitive domains, Standard accuracy, failures in sensitive, mask significant reliability
备注：
点击查看摘要
Abstract:Standard accuracy metrics for VLMs often mask significant reliability failures in sensitive domains. In this work, we utilize a histopathology-validated brain MRI dataset to systematically assess the diagnostic robustness of four VLM families under evidence-preserving perturbations. By reordering anatomical slices and swapping target label positions, we evaluate whether models maintain consistent predictions when clinical evidence remains invariant. Our results reveal significant vulnerabilities in presentation-order stability, with models exhibiting prediction flips in up to 48.9% of cases under simple sequence reversals. We further identify a textual selection bias, where label reordering triggers inconsistent diagnoses in up to 67.8% of cases despite identical visual inputs. Negative-control tests further reveal diagnostic overcommitment: models generate categorical diagnoses in up to 76.1% of cases after expert-annotated lesion slices are removed. These results demonstrate that high accuracy can overestimate clinical reliability, masking sensitivity to sequential presentation and textual framing that is not captured by aggregate accuracy. Our findings highlight the necessity of stability-based metrics for the deployment of VLMs in safety-critical clinical applications. Our evaluation data and code will be made public upon acceptance.
25. 【2608.04872】A-SR: Self-Evolving Agentic LLMs for Symbolic Regression via Hierarchical Coordination
链接：https://arxiv.org/abs/2608.04872
作者：Wenxiao Zhao,Dong Liu,Kaiyi Xu,Feng Liu,Zhen Zhao,Fei Ben,Shu Wang,Wenhao Li,Yingnian Wu,Fenghua Ling,Haobo Li,Lei Bai
类目：Computation and Language (cs.CL); Artificial Intelligence (cs.AI); Machine Learning (cs.LG)
关键词：Symbolic regression aims, discover closed-form equations, existing LLM-guided methods, Symbolic regression, unified proposal loop
备注： 18 pages, 8 figures, including appendix
点击查看摘要
Abstract:Symbolic regression aims to discover closed-form equations from data, but existing LLM-guided methods often rely on a unified proposal loop that compresses heterogeneous search failures into a scalar score and a single prompt. We propose A-SR, a self-evolving agentic framework that shifts the control unit from expression edits to role-conditioned evidence views. A-SR coordinates formula discovery through routing among coordination protocols, an online evaluator-reward role policy, and state-routed process memory. During search, evaluator feedback characterizes reliability and productivity, updates role-level utilities, and routes elite motifs, failure traces, and validity diagnostics to different agents. The framework self-evolves at two timescales: within a run, it adapts the search process without updating LLM parameters; across runs, recorded trajectories can be distilled into open-source LLMs as role-conditioned proposal priors. Averaged over the four LSR-Synth scientific domains in LLM-SRBench, A-SR improves Acc@0.01 over baselines from 25.79% to 48.30% with Llama3.1-8B, while A-SR-LoRA improves the corresponding Qwen3-4B result from 24.58% to 38.29%. On four real-world scientific discovery tasks, A-SR obtains the best in-distribution or out-of-distribution normalized mean squared error on 7 of 8 reported metrics.
26. 【2608.04869】Preverbal Uninflected and Underived Roots in Mapudungun. Wuno and Its Implications
链接：https://arxiv.org/abs/2608.04869
作者：Andres Chandia
类目：Computation and Language (cs.CL)
关键词：roots in Mapudungun, uninflected and underived, underived roots, preverbal uninflected, Mapudungun
备注： 54 pages, 4 tables, 2 graphics, 23 examples
点击查看摘要
Abstract:This study examines the grammatical status of preverbal uninflected and underived roots in Mapudungun, with particular focus on wuno 'return/re-'. Through a critical review of scholarly classifications--auxiliaries (Smeets, 2008), modal prefixes (Longkon, 2011), and preverbal particles/complex verb stems (Zúñiga, 2006)--we demonstrate the limitations of existing frameworks. A diachronic corpus analysis spanning four centuries (1606-present) reveals that these elements exhibit three distinct profiles: stable V1 compounds (kim, shinge), volatile V1 rates reflecting orthographic shift (pepi, wuno), and a true particle (kalli). The discovery of V2 attestations for kim and kupa confirms their status as full lexical verbs. We propose a prosodic-orthographic hypothesis: apparent "variable binding" results from the fossilization of prosodic pauses transcribed by early missionaries as spaces, a convention later reanalysed by speakers as syntactic boundaries. The evidence supports Zúñiga's radical concatenation as the correct grammatical model, with implications for the study of languages with no pre-contact written tradition.
27. 【2608.04847】Do Language Models Know Their Slang? Queer Slang Understanding in User-Generated Content
链接：https://arxiv.org/abs/2608.04847
作者：Arianna Denitto,Beatrice Savoldi
类目：Computation and Language (cs.CL)
关键词：Natural Language Processing, Language Processing research, slang remains underrepresented, Processing research, underrepresented in Natural
备注：
点击查看摘要
Abstract:Despite its cultural relevance and diffusion, queer slang remains underrepresented in Natural Language Processing research. Towards addressing this gap, we introduce Slang-Q, a manually curated dataset of naturally user-generated English sentences paired with queer slang terms and reference definitions, built upon a newly constructed taxonomy of 118 queer terms. We use this resource to conduct a first exploratory evaluation of language models on their ability to understand and define queer slang under varying prompting conditions. Slang-Q is intended as a basis for studying how current models handle sensitive, community-specific language and whether they can provide accurate and reliable information about such forms of identity and linguistic expression.
28. 【2608.04828】Skill-Use: Can LLMs Actually Use Skills in Agentic Harnesses?
链接：https://arxiv.org/abs/2608.04828
作者：Jinyi Han,Yuanjian Xu,Ying Liao,Xinyi Wang,Zishang Jiang,Zixiang Di,Fanyang Lu,Zhichao Hu,Yanghua Xiao
类目：Computation and Language (cs.CL)
关键词：Large language model, Large language, agents increasingly rely, structured documents, tools are allowed
备注：
点击查看摘要
Abstract:Large language model (LLM) agents increasingly rely on skills, structured documents that specify when to act, which procedure to follow, and which tools are allowed. Existing evaluations mostly judge the quality of a skill or its contribution to task success, leaving unexamined whether an agent can recognize a relevant skill and apply it on its own. We introduce Skill-Use, a benchmark that evaluates skill use under progressive disclosure, where an agent sees only a skill's name and short description and must retrieve the full procedure before following it. Skill-Use separates three facets of skill use. Trigger measures whether the agent invokes the relevant skill, Compliance measures how faithfully it follows the prescribed procedure, and Boundary measures whether it avoids forbidden operations. A Skill-Use (SU) score combines the three and credits execution only after the skill is triggered. Skill-Use pairs 79 real skills with 177 executable tasks across nine domains, each grounded in real files, run in an isolated Docker sandbox, and scored by a trajectory-based rubric. Evaluating eight LLMs under two agent harnesses, we find that reliable skill use remains out of reach, as the strongest configuration reaches an SU of only 0.613. Triggering and procedural compliance fail as independent bottlenecks, and both scores and model rankings shift with the harness, so skill use behaves as a capability conditioned on the harness rather than a fixed property of the model.
29. 【2608.04808】A Modular Part-of-Speech Tagger for Scottish Gaelic using spaCy
链接：https://arxiv.org/abs/2608.04808
作者：Peter Stefan,Peter J Barclay,Alistair Lawson
类目：Computation and Language (cs.CL)
关键词：remains challenging due, limited annotated data, Scottish Gaelic, languages remains challenging, Annotated Reference Corpus
备注： A revised version of this paper has been accepted for presentation at UKCI 2026 ( [this https URL](https://ukci2026.coventry.ac.uk/home/) ) and will be published by Springer
点击查看摘要
Abstract:Part-of-speech tagging for low-resource languages remains challenging due to limited annotated data, especially for linguistically complex languages. Gaidhlig (Scottish Gaelic) is a morphologically rich and endangered language with limited digital resources, making it suitable for examining a lightweight language processing approach. This paper describes using the modular spaCy Natural Language Processing framework to build part-of-speech taggers for Gaidhlig using the Annotated Reference Corpus of Scottish Gaelic. We train two models with minimal pre-processing and configuration: one using a fine-grained tagset and another using a reduced coarse-grained tagset. Both models are trained without external embeddings or pre-trained language models, using only supervised learning from the available corpus. The fine-grained model achieves 88.6% tagging accuracy, while the coarse-grained model achieves 93.7%. The results are comparable to those of the two previously published Gaidhlig taggers, indicating that simple, off-the-shelf language processing pipelines can demonstrate good performance in low-resource and morphologically complex linguistic settings.
30. 【2608.04788】Agentic Reinforcement Learning with Observation-Calibrated Self-Distillation
链接：https://arxiv.org/abs/2608.04788
作者：Yi Yang,Cong Qin,Xiaodan Liu,Chishui Chen,Qing Dong,Yan Zhang,Cao Liu,Zhao Yang,Lu Pan,Jiaye Lin,Yi Feng
类目：Machine Learning (cs.LG); Artificial Intelligence (cs.AI); Computation and Language (cs.CL)
关键词：Large language model, offer limited guidance, Large language, strongly individual tokens, sparse trajectory-level rewards
备注：
点击查看摘要
Abstract:Large language model agents are commonly trained through reinforcement learning with sparse trajectory-level rewards, which offer limited guidance on how strongly individual tokens should be updated. On-Policy Self-Distillation (OPSD) addresses this by re-scoring generated tokens under a privileged replay view to obtain dense, token-level supervision. However, we identify a confounding issue: the resulting support may reflect both the privileged information contained in the replay view and score shifts induced by the replay scaffold, making it difficult to attribute the support specifically to that information. This issue is especially pronounced when future environment observations serve as privileged information, since replaying them requires reconstructing an extended scaffold that itself perturbs token scores. To resolve this confounding, we propose Observation-Calibrated Self-Distillation (OCSD), which contrasts two structurally matched replay views, Full and Observation-Ablated, differing only in whether the actual future observation is present, to derive an observation residual that discounts score changes shared by the replay scaffold. OCSD then applies this residual to modulate token-level GRPO updates at high-uncertainty steps, while preserving the trajectory-level update direction. Experiments on ALFWorld, WebShop, and Search-QA across three Qwen3 model scales show that OCSD consistently outperforms strong baselines. Diagnostic analyses further confirm that the calibrated residual aligns better with local environment feedback. Our code is publicly available at this https URL.
31. 【2608.04786】Reachability in 3-VAS
链接：https://arxiv.org/abs/2608.04786
作者：Łukasz Kamiński,Sławomir Lasota
类目：Computation and Language (cs.CL)
关键词：fixed low dimension, vector addition systems, fixed low, VAS, vector addition
备注：
点击查看摘要
Abstract:We settle the exact complexity of the reachability problem in (stateless) vector addition systems (VAS) in fixed low dimension. In dimensions 2-4 it has only been known to be sandwiched between NP and PSPACE. We prove PSPACE-hardness of the reachability problem for symmetric vector addition systems in dimension 3 (3-VAS), a restricted fragment of general 3-VAS. Combined with previously established PSPACE upper bounds, our result settles the complexity of the problem to be PSPACE-complete in 3-VAS and 4-VAS, as well as in their symmetric fragments.
32. 【2608.04772】Guideline-as-Oracle: Zero-Annotation Training of an Ophthalmic Telephone Triage Agent
链接：https://arxiv.org/abs/2608.04772
作者：Chenyu Wang,Yi Liu,Baoqing Li,Min Tu,Diping Song
类目：Computation and Language (cs.CL); Artificial Intelligence (cs.AI)
关键词：multi-turn medical agents, expert dialogue annotation, Scaling supervision, compiles American Academy, conversations are privacy-restricted
备注：
点击查看摘要
Abstract:Scaling supervision for multi-turn medical agents is difficult because expert dialogue annotation is costly and clinical conversations are privacy-restricted. We introduce Guideline-as-Oracle (GAO), which compiles American Academy of Ophthalmology guidance into a 70-row operational rule table and uses it as the sole source of instance-level supervision for 3,000 training dialogues, reserving human labeling for evaluation. Because converting rules into dialogues is itself a design problem, we catalog eight construction strategies, including cited-row tier assignment, one-fact boundary pairs, metadata-only repair, and label repair, and characterize the evidential status of each: labeling mechanism, null, confounded, or evaluated only as a package. Fine-tuning a 9B backbone on this corpus yields GAO-Triage, improving agreement with a 201-case operational reference from 61.7% to 74.1% (exact McNemar p=0.0046) and emergent-case recall from 9.5% to 69.0%; the gains persist across a second seed and patient simulator. None of the seven general-purpose systems we test dominates GAO-Triage on both metrics, and GAO-Triage requires no frontier model at inference time. Permuting label-dialogue assignments collapses the model to a constant-routine predictor, indicating that the signal lies in guideline-derived assignment rather than dialogue surface form. Label repair coincides with the disappearance of a late-training safety degradation.
33. 【2608.04761】InsightEmb: Learning Action-Intent Embeddings for Agentic Insight Retrieval
链接：https://arxiv.org/abs/2608.04761
作者：Tsz Ting Chung,Jiangnan Li,Jie Zhou,Mo Yu
类目：Computation and Language (cs.CL); Artificial Intelligence (cs.AI)
关键词：Self-improving agents accumulate, turning accumulated experience, making retrieval increasingly, accumulate reusable insights, agents accumulate reusable
备注：
点击查看摘要
Abstract:Self-improving agents accumulate reusable insights from prior trajectories, making retrieval increasingly important for turning accumulated experience into actionable guidance. At each decision step, retrieving the right insight can help the agent progress toward its goal, a setting we refer to as agentic insight retrieval. However, existing retrieval methods primarily model semantic similarity, while overlooking whether a retrieved insight resolves the agent's current decision bottleneck. We propose InsightEmb, a contrastive embedding framework that learns transferable progress-oriented retrieval geometry using only mathematical reasoning data. InsightEmb jointly learns to align concrete situations with abstract heuristic rules and to cluster reasoning trajectories with similar progress structures. We evaluate InsightEmb on dynamic agent tasks and a static skill-retrieval benchmark. Without any environment-specific training, InsightEmb improves over all these evaluations, surpassing the performance of existing reasoning embedding models. These results suggest that the geometry of state-insight matching can transfer across domains, enabling effective training from publicly available reasoning data without expensive environment-specific supervision.
34. 【2608.04759】race, Verify, and Correct: A Training-Free Framework for Spatial Reasoning in Multimodal LLMs
链接：https://arxiv.org/abs/2608.04759
作者：Yang Yang,Jiawei Chen,Tairan Chen,Zhaoxia Yin
类目：Computer Vision and Pattern Recognition (cs.CV); Computation and Language (cs.CL)
关键词：Multimodal Large Language, Large Language Models, Multimodal Large, Large Language, made substantial progress
备注： 19 pages, 7 figures
点击查看摘要
Abstract:Although Multimodal Large Language Models (MLLMs) have made substantial progress, their spatial reasoning may still produce intermediate judgments inconsistent with the input image, allowing errors to propagate through the reasoning chain and affect the final answer. Existing methods mainly improve spatial reasoning through training or additional spatial information, without considering whether the reasoning process itself is faithful to the model input. Our study shows that unfaithful reasoning chains significantly reduce final-answer accuracy. To address this issue, we propose a modular and training-free framework for spatial reasoning verification and correction. The framework constructs a Spatial Evidence Graph (SEG), which associates atomic spatial evidence extracted from Chain-of-Thought reasoning with visual entities, spatial relations, source steps, and visual evidence. Spatial Evidence Reliability Assessment (SERA) evaluates the reliability of visual evidence based on object existence, localization, and geometric measurements. The framework then identifies the earliest spatial evidence unit contradicted by reliable visual evidence and guides the original MLLM to revise the subsequent reasoning and final answer. Across 15 model-dataset settings, our method achieves an average accuracy of 68.94%, outperforming the compared baselines by 8.55 percentage points on average. Our code will be open-sourced.
35. 【2608.04750】Simile Understanding in Text-to-Image Models: An Evaluation Framework
链接：https://arxiv.org/abs/2608.04750
作者：Luecheng Wang,Shintaro Ozaki,Hidetaka Kamigaito,Katsuhiko Hayashi,Jingun Kwon,Manabu Okumura,Taro Watanabe
类目：Computer Vision and Pattern Recognition (cs.CV); Computation and Language (cs.CL); Multimedia (cs.MM)
关键词：describe visual characteristics, provide a compact, compact and expressive, models, describe visual
备注： Accepted as a full paper at ACM Multimedia 2026
点击查看摘要
Abstract:Similes provide a compact and expressive way to describe visual characteristics in text prompts. Recent text-to-image models (t2i models) can produce visually compelling outputs from simile prompts, yet even frontier models frequently misinterpret the metaphorical vehicle and confuse it with the object. These systematic failures reveal a gap between figurative language and object-level visual grounding in t2i models. To investigate this issue, we propose a scalable evaluation framework for simile understanding. Our framework includes (1) a controlled simile dataset in which metaphorical vehicles are drawn from a predefined set of object-detectable categories and combined with diverse templates, (2) automatic grounding metrics based on YOLO (You Only Look Once) detection, and (3) text encoder layer analysis using Diffusion Lens to track how metaphorical vehicles emerge during generation. Experiments across architecturally diverse t2i models reveal consistent literalization failure patterns. We further discuss potential mitigation strategies for improving simile grounding in t2i models.
36. 【2608.04746】Caching for the Future: Scrub Jay Episodic Memory Principles for Agent Memory Systems
链接：https://arxiv.org/abs/2608.04746
作者：Kartikey Singh Bhandari,Aarya Wadhwani,Dhruv Kumar,Pratik Narang
类目：Computation and Language (cs.CL); Information Retrieval (cs.IR)
关键词：accumulate stored memories, sessions accumulate stored, validity varies enormously, systematically contaminate retrieved, contaminate retrieved context
备注：
点击查看摘要
Abstract:LLM agents that persist across sessions accumulate stored memories whose validity varies enormously by content type, yet existing memory architectures treat all memories as equally persistent and systematically contaminate retrieved context with outdated facts. We show that per-memory, type-conditioned temporal decay, a property of western scrub jay episodic memory, can be operationalized as an auto-classified coefficient $\pi_i$ in an external LLM-agent memory store, yielding ScrubJay-MEM: each memory is encoded as a jointly-bound What--Where--When tuple with an estimated perishability $\pi_i$ and utility horizon $\tau_i$, retrieved by query-adaptive scoring, and revised retroactively at $O(1)$ LLM calls per update. We introduce the Temporal Generalization Test (TGT), a benchmark with held-out retention intervals and a Generalization Gap (GenGap) metric. On TGT, ScrubJay-MEM is the only retrieval-based system with substantially positive GenGap ($+0.108$); on MemoryAgentBench EventQA-64k it improves F1 by $+2.66$ over Mem0 and $+3.09$ over Qwen3-Embedding-4B under a llm backbone. A decay ablation collapses GenGap by $5.7\times$, establishing type-conditioned decay as necessary for the result. Gains narrow under stronger backbones and reverse on fact-consolidation tasks, scoping the contribution to temporal reasoning over perishable facts.
37. 【2608.04709】EmpaAva: An Open-source Agentic 3D-Avatar Empathetic Live Chatbot
链接：https://arxiv.org/abs/2608.04709
作者：Jie Yang,Wenhao Xu,Shuhui Lin,Hao Fei
类目：Computation and Language (cs.CL)
关键词：empathetic response generation, paper presents EmpaAva, carries empathetic response, paper presents, empathetic response planning
备注： ProjectDemo: [this https URL](https://empaava.top/)
点击查看摘要
Abstract:This paper presents EmpaAva, to our knowledge the first open-source, agentic 3D-avatar empathetic chatbot, which carries empathetic response generation (ERG) from text-only exchanges into live, face-to-face interaction. Through a video-call-like interface, a user speaks to a 3D digital human that reads their affect from speech and optional vision, and replies with emotional speech, lip-synced facial motion, and photorealistic 3D Gaussian rendering. At its core, an LLM coordinates a Tri-Agent Architecture, in which perception, empathetic response planning, and embodied rendering form a closed loop, paired with a Response Planning layer that compiles each reply into an executable multimodal plan, keeping voice, expression, and rendering on one empathetic intent. Building on strong open-source modules, EmpaAva supplies the intelligence that binds them into one controllable, inspectable experience. In automatic and human evaluations, EmpaAva surpasses text-only, 2D talking-face, and multimodal avatar baselines in emotion understanding, response quality, and audio-visual consistency. We open-source EmpaAva with an online live demo.
38. 【2608.04703】IslamicTurathBench: A Multi-Task, Multi-Discipline Benchmark for Evaluating Large Language Models on the Islamic Scholarly Tradition (turath)
链接：https://arxiv.org/abs/2608.04703
作者：Shahd Gaben,Heba Sbahi,Samer Rashwani,Abdessalam Bouchekif,Mutaz Al-Khatib,Emad Mohamed,Somaya Eltanbouly,Mohammed Ghaly
类目：Computation and Language (cs.CL)
关键词：Large language models, Large language, specialised source traditions, Islamic Studies, including in religious
备注： Includes supplementary materials. Submitted to the Journal of Scientific Data. Data and code are publicly available
点击查看摘要
Abstract:Large language models (LLMs) are increasingly used for question answering, education, and research, including in religious and cultural domains where answers depend on specialised source traditions. Yet in Islamic Studies, key concepts, methods, and debates preserved in the authoritative scholarly tradition, known as turath, lack high-quality annotated resources. We introduce IslamicTurathBench (ISTB), a multi-task, multi-discipline dataset for evaluating LLMs on classical Islamic scholarship. Developed and reviewed by domain experts, ISTB contains 3,465 question-answer items drawn from 35 recognised source works spanning more than 12 centuries of scholarship across seven key fields of Islamic Studies. To enable comprehensive profiling of model capabilities, ISTB is structured along two axes: scholarly demand (Beginner, Intermediate, and Advanced) and task format (multiple-choice questions, passage-based comprehension, and open-ended knowledge questions). ISTB includes aggregated scores from a scholarly human reference panel and zero-shot baselines from ten systems. The dataset supports reproducible evaluation of language-model behaviour across source works, disciplines, scholarly-demand levels, and question formats in a historically layered scholarly domain.
39. 【2608.04678】Kathleen Writes: Autoregressive Generation and Data Scaling Without Attention
链接：https://arxiv.org/abs/2608.04678
作者：George Fountzoulas
类目：Computation and Language (cs.CL); Machine Learning (cs.LG)
关键词：Kathleen series showed, multi-scale reverberant state, match strong baselines, Kathleen series, attention-free architecture built
备注： Paper 3 of the Kathleen series. 11 pages, 3 figures. All experiments reproducible on a free Kaggle T4
点击查看摘要
Abstract:Papers 1-2 of the Kathleen series showed that a byte-level, attention-free architecture built from a wavetable encoder and multi-scale reverberant state can match strong baselines on classification at ~450-700K parameters, without pretraining. We ask whether the same ingredients can generate. (1) Scaling: on byte-level language modeling (WikiText-103, raw UTF-8, no tokenizer), the reverberant model beats a parameter-matched transformer at every dataset scale measured (2-512 MB), e.g. 1.84 vs 2.04 bits/byte at 512 MB with ~0.5M parameters; the transformer needs more than 512 MB to match what the attention-free model learns from 32 MB. (2) Measurement: we introduce FORM DISTANCE, a non-parametric, gaming-resistant instrument for "reads like text": nine statistical axes of human text define a reference cloud, and five constructed fakes are all rejected. (3) Generation: decoding policy dominates architecture -- widening the sampler halves the same model's distance (3.17 to 1.52), and a retrieval-augmented decoding scheme takes the frozen model further (1.52 to 1.14) with no training step involved; the ablation attributes the gain to the sparse phrase dose itself, not the selection gate. The gain has a sharp boundary condition: the phrases must come from the model's own training corpus -- a 40x larger foreign library helps not at all, an effect the attention twin shares, consistent with in-context integration being a capability of scale. We also report four architectural additions that did not help, and a computed lexicon reaching 94% of a learned table's top-1 accuracy at one fifth of the parameters. Everything runs offline; all experiments are reproducible on a free Kaggle T4.
40. 【2608.04670】Easy to Complete, Hard to Choose: Investigating LLM Performance on the ProverbIT Benchmark
链接：https://arxiv.org/abs/2608.04670
作者：Enrico Mensa,Lorenzo Zane,Calogero Jerik Scozzaro,Matteo Delsanto,Tommaso Milani,Daniele Paolo Radicioni
类目：Computation and Language (cs.CL); Artificial Intelligence (cs.AI)
关键词：transformed computational linguistics, significant gaps persist, embedded linguistic expressions, numerous natural language, natural language processing
备注：
点击查看摘要
Abstract:Large Language Models (LLMs) have transformed computational linguistics and achieved remarkable performance across numerous natural language processing tasks, yet significant gaps persist in understanding how these systems process culturally embedded linguistic expressions. This paper introduces ProverbIT, a novel Italian benchmark comprising 100 multiple-choice questions designed to evaluate LLMs' ability to complete Italian proverbs. We assess 13 frontier models, including Large Reasoning Models (LRMs) and traditional LLMs, across three tasks: proverb completion, multiple-choice selection with correct answers, and multiple-choice selection without correct answers. Our evaluation reveals surprising results: while nearly all models demonstrate knowledge of the proverbs through successful completion tasks, performance drops dramatically when transitioning to multiple-choice formats without correct answers, with even state-of-the-art reasoning models showing substantial degradation. Through detailed Chain-of-Thought analysis of two LRMs, we uncover that models exhibit a strong bias toward selecting literal synonyms and frequently mention correct proverb endings during reasoning without successfully identifying their absence from the given options. These findings suggest that current LLMs rely heavily on memorized patterns rather than deeper semantic understanding of culturally grounded expressions, highlighting important limitations in their reasoning capabilities for figurative language comprehension.
41. 【2608.04646】Evaluating Theory of Mind in Reasoning Models: Robustness over Reasoning
链接：https://arxiv.org/abs/2608.04646
作者：Ian B. de Haan,Peter van der Putten,Max van Duijn
类目：Computation and Language (cs.CL)
关键词：Theory of Mind, Large language models, recently shown strong, shown strong performance, Large language
备注： Accepted for 29th International Conference on Discovery Science, October 5-9, 2026, Mainz, Germany
点击查看摘要
Abstract:Large language models (LLMs) have recently shown strong performance on Theory of Mind (ToM) tests, prompting debate about the nature and validity of the underlying capabilities. At the same time, reasoning-oriented LLMs trained via reinforcement learning with verifiable rewards have demonstrated notable improvements across a range of benchmarks. In this work, we examine the behavior of such reasoning models in ToM tasks using novel adaptations of machine psychological experiments together with results from established benchmarks. We observe that reasoning models consistently exhibit increased robustness to prompt variations and task perturbations. Our analysis suggests these gains come at least partly from models being more robust at reaching the correct answer under prompt and task variation. We read this as evidence for a robustness-based account rather than for a new ToM-specific ability.
42. 【2608.04641】AI Literacy for Legal Translation: Developing Digital Resilience
链接：https://arxiv.org/abs/2608.04641
作者：Łucja Biel
类目：Artificial Intelligence (cs.AI); Computation and Language (cs.CL); Computers and Society (cs.CY); Human-Computer Interaction (cs.HC)
关键词：opportunities alongside linguistic, introducing opportunities alongside, transforming legal translation, legal translation, alongside linguistic
备注： 19 pages, 2 tables, 2 figures
点击查看摘要
Abstract:Generative AI is transforming legal translation by introducing opportunities alongside linguistic, technical, legal, ethical and cognitive risks. This chapter examines the implications of AI for professional legal translation and proposes an AI literacy framework tailored to the profession. It argues that AI does not change the fundamental objectives of legal translation but requires an extension of professional competence through AI literacy. The proposed framework comprises four mutually reinforcing dimensions, foundational, procedural, critical and strategic, and conceptualises AI literacy as a transversal component of legal translation competence that fosters digital resilience. It further discusses the pedagogical implications of this framework by proposing classroom activities designed to develop AI literacy in legal translator education, enabling future translators to integrate AI critically, responsibly and in accordance with professional standards.
43. 【2608.04591】When Absence Is Evidence: Evaluating Completeness-Sensitive Negative Reasoning in Large Language Models
链接：https://arxiv.org/abs/2608.04591
作者：Byoungjae Min,Kennedy Edemacu,Sae-Hong Cho,Yoonhyuk Choi,Beakcheol Jang,Jong Wook Kim
类目：Computation and Language (cs.CL); Artificial Intelligence (cs.AI)
关键词：Large language models, Large language, retrieved context, Large, negative answer
备注： 19 pages, 2 figures, 20 tables
点击查看摘要
Abstract:Large language models (LLMs) are often asked whether something is absent from a record, list, or retrieved context. Yet non-observation licenses a negative answer only when evidence completely covers the query scope; otherwise, the answer should remain unknown. We call this completeness-sensitive negative reasoning. We introduce CROWN-QA, comprising CROWN-Synth, a controlled paired core that fixes the question and observed facts while varying only query-relative coverage, and CROWN-Real, a real-document contrast-set evaluation with controlled coverage variants. Across three LLM families, models show unstable closure judgments and substantial over-closure, failing to reliably distinguish a justified negative answer (Certified-Negative) from insufficient evidence (Unknown). The dominant CROWN-Synth failure is asymmetric: models often recognize implicitly complete evidence yet treat implicitly partial evidence as query-covering. Prompting redistributes errors between over- and under-closure rather than consistently resolving them. Structured certificate elicitation traces many errors to evidence-coverage mischaracterization. CROWN-Real shows that the core partial-coverage asymmetry persists on real-document content, while its strength and the balance between over- and under-closure vary by model, prompt, and source.
44. 【2608.04588】EASy: Towards Efficient LLM-Based Agentic System
链接：https://arxiv.org/abs/2608.04588
作者：Junnan Liu,Linhao Luo,Thuy-Trang Vu,Gholamreza Haffari
类目：Computation and Language (cs.CL); Artificial Intelligence (cs.AI)
关键词：specialized LLM-based agents, coordinating specialized LLM-based, promising paradigm, paradigm for solving, coordinating specialized
备注： Preprint
点击查看摘要
Abstract:Agentic systems have emerged as a promising paradigm for solving complex tasks by coordinating specialized LLM-based agents. However, most existing systems primarily optimize task success while giving limited consideration to execution efficiency under practical constraints such as executor capability and computational cost. Existing router-based methods have limited ability to reason over rich, evolving task contexts, multi-step dependencies, and intermediate execution feedback, and often generalize poorly to unseen executors. We propose EASy, a trainable agentic framework that jointly optimizes task performance and computational efficiency through reinforcement learning. EASy equips an LLM-based orchestrator with explicit knowledge of the capability and cost profiles of heterogeneous executors, enabling context-sensitive coordination beyond performance-only routing. It further introduces a milestone-plan-act workflow that decomposes complex tasks into manageable milestones, constructs dependency-aware execution graphs, assigns suitable executors, and parallelizes independent steps while adapting subsequent decisions to intermediate outcomes. To train the orchestrator, we develop a tree-structured rollout procedure that explores alternative milestone decompositions and execution plans, together with multi-component rewards that capture task correctness, execution efficiency, and trajectory completeness. Extensive experiments on mathematical reasoning, embodied decision-making, and deep research benchmarks show that EASy consistently achieves stronger performance-efficiency trade-offs than strong agentic baselines.
45. 【2608.04586】Breaking the Curse ofMultilinguality inMany-to-Many Speech-to-Text Translation via a Resource-AwareMixture of Speech Encoders
链接：https://arxiv.org/abs/2608.04586
作者：Yexing Du,Kaiyuan Liu,Youcheng Pan,Bo Yang,Chengpeng Fu,Yu Wang,Ming Liu
类目：Computation and Language (cs.CL); Artificial Intelligence (cs.AI)
关键词：Multimodal large language, achieved significant success, Multimodal large, achieved significant, significant success
备注：
点击查看摘要
Abstract:Multimodal large language models (MLLMs) have achieved significant success in speech-to-text translation (S2TT). However, when processing multilingual speech inputs, a single speech encoder shared across all languages suffers from the curse of multilinguality: languages at different resource levels compete for limited representation capacity, leading to strong high-resource performance but substantial degradation on low-resource speech. To address this problem and improve multilingual consistency, we propose MSRT, a novel framework built around a resource-aware Mixture of Speech Encoders (MoSE). MoSE uses an explicit language router to assign each utterance to an appropriate expert encoder. A frozen expert preserves high-resource language capabilities, while a trainable expert adapts to and specializes in medium- and low-resource languages. We further introduce a five-stage curriculum learning strategy that substantially reduces data dependence, requiring only 10 hours of paired S2TT data per language for effective alignment. We conduct extensive experiments on 45 languages, systematically evaluating all $45 \times 44$ translation directions. Our 4B-parameter model achieves state-of-the-art performance, outperforming substantially larger baselines. Empirical analyses show that MoSE improves high-, medium-, and low-resource languages simultaneously, with the largest gains on low-resource speech, thereby breaking the curse of multilinguality without compromising high-resource performance. To support future multilingual S2TT research, we release our code and models.
46. 【2608.04576】Causal Evidence Extraction and Triangulation in Crisis Reports using Large Language Models: A ReliefWeb-based Study
链接：https://arxiv.org/abs/2608.04576
作者：Yuanjun Zhang,Mourad Oussalah
类目：Computation and Language (cs.CL)
关键词：consolidate decision-relevant causal, Large Language Model, decision-relevant causal evidence, two-stage Large Language, making it difficult
备注：
点击查看摘要
Abstract:Humanitarian reports are long, noisy, and multi-topic, making it difficult to consolidate decision-relevant causal evidence. We present a ReliefWeb study (2000-2024) and a two-stage Large Language Model (LLM) pipeline that extracts structured intervention-outcome records with direction and strength attributes. Query-conditioned extraction restricts output to a specified intervention class, reducing retrieval-induced over-extraction, while snippet grounding links each relation to supporting text for auditability and classification. In an expert-annotated dataset of 100 reports, the best closed-source LLM achieved a weighted F1 score of 90.73% with strong cost-efficiency, while Llama-3.1-8B with supervised fine-tuning reached 94.15% weighted F1 score. We further propose context-preserving triangulation that aggregates strength-weighted evidence within disaster$\times$source cells, applies Laplace smoothing and equally weights cells to quantify cross-context convergence via a Level-of-Evidence score. Applied to cash assistance, food-related outcomes show strong positive convergence (LoE=0.865) and stable long-horizon trajectories.
47. 【2608.04574】When Memory Lies: An Empirical Study of Spatial Memory Staleness in VLM Agents
链接：https://arxiv.org/abs/2608.04574
作者：Yushi Sun,Yanjie Zhang
类目：Computation and Language (cs.CL)
关键词：persistent spatial knowledge, VLM agents act, spatial knowledge, knowledge silently, act on persistent
备注：
点击查看摘要
Abstract:Memory-augmented VLM agents act on persistent spatial knowledge, yet that knowledge silently goes stale as the environment changes. We ask what happens when an agent must reconcile a confident memory claim with a contradicting observation, and whether current models can catch the conflict before it becomes a safety-relevant mistake. Using a dynamic FrozenLake testbed, we pair a staleness-detection task with a downstream navigation task across three closed-source models and three open-weight VLMs under both text and image inputs (1,800 detection runs, and 12,000 text-mode navigation episodes over four LLM navigators at a shared 50-seed scale). Three findings emerge. First, text solvability does not imply visual grounding: models that flag stale entries reliably from text nonetheless span vision F1 from 0.887 down to 0.067 on the identical grids, and the weakest keeps making fluent, confident decisions that ignore the image. Second, consuming stale memory without an audit is a safety liability: in our primary GPT-4o setting, an agent that trusts raw memory dies more than twice as often as the same agent given no memory at all. Third, auditing helps but does not close the gap: a transparent read-time filter removes much of the safety cost in text mode, yet even oracle stale labels bring no further significant gain on the current grid size, and when visual auditing is unreliable, filtering yields no consistent benefit. Together these results frame spatial-memory staleness as a safety failure mode and isolate reliable visual grounding and action selection under memory--observation conflict as the central open challenges for memory-augmented agents.
48. 【2608.04570】he Personalization Mirage: How LLMs Fabricate User Profiles, and Why Self-Monitoring Misleads
链接：https://arxiv.org/abs/2608.04570
作者：Yushi Sun,Yanjie Zhang,Rui Sheng
类目：Computation and Language (cs.CL)
关键词：Personalized LLMs, models remains unexamined, increasingly deployed, remains unexamined, user models remains
备注：
点击查看摘要
Abstract:Personalized LLMs with persistent memory are increasingly deployed, yet the faithfulness of their user models remains unexamined. We study over-inference (OI): the phenomenon where LLMs fabricate user attributes beyond what evidence supports. We introduce MirageBench, comprising 150 personas balanced across stereotypical, counter-stereotypical, and neutral profiles, 6 personalization tasks spanning an ``imagination gradient'', a four-way faithfulness taxonomy operationalized by an independent judge (validated against a blind human annotator on 400 claims: Cohen's kappa = 0.863 four-class, kappa = 0.900 binary), and a leaderboard of 12 models across 7 families on 143616 judged claims. We find that over-inference is pervasive: every one of the 12 models over-infers 35%--49% of its claims (cross-model mean 41.6%; claim-weighted 41.8%), with no model in this evaluation escaping it. Most strikingly, we surface a Self-Monitoring Inversion: at the model-selection level, models' self-assessed OI is negatively rank-correlated with their judge-measured OI (rho = -0.60, p = 0.044; exploratory, wide bootstrap CI [-0.90, +0.06], n = 12). The models that report the least over-inference tend to be flagged as fabricating the most, so self-reported confidence is a misleading signal for comparing models, even though within a single model self-audit still ranks that model's own claims moderately well (AUROC 0.58--0.83). We further show that OI is task-dependent (27%--59%) and that, in a multi-turn pilot, inferred attributes accumulate approximately linearly with little revision. MirageBench positions external verification, rather than model self-report, as a more reliable foundation for trustworthy personalization.
49. 【2608.04569】Relevant but Incomplete: Referential Dangling as a Paradigm-Level Failure Mode in Hard Prompt Compression
链接：https://arxiv.org/abs/2608.04569
作者：Zhengpei Hu,Kai Li,Dapeng Fu,Xuechao Zou,Yuanhao Tang,Yue Li,Tengfei Cao,Jianqiang Huang
类目：Computation and Language (cs.CL); Machine Learning (cs.LG)
关键词：prompt compression reduces, compression reduces long-context, independently scoring tokens, reduces long-context inference, long-context inference cost
备注： Code: [this https URL](https://cslikai.cn/Referential-Dangling)
点击查看摘要
Abstract:Hard prompt compression reduces long-context inference cost by independently scoring tokens, sentences, or chunks and retaining the highest-scoring units under a budget. We identify a structural failure in this procedure: independent selection can split dependent evidence pairs, retaining one member while deleting the other. When retained text contains an answer but deleted text defines the entity needed to interpret it, we call the result referential dangling. At a compression ratio of 0.30, Beaver, which ranks coherent chunks using Qwen3-0.6B embeddings, leaves the answer path incomplete in 34-54% of bridge examples across three multi-hop question answering datasets. On a shared HotpotQA bridge set, all six hard compressors we test exhibit dangling at rates up to 60%, and every document in LongBench-v2 Single-Document QA contains at least one dangling reference. On dangling examples evaluated with Qwen3-8B, reinserting the missing supporting paragraph while removing nonsupporting paragraphs to maintain the token budget improves accuracy by 29-34 percentage points (p 0.0001), recovering at least 88% of the gap to contexts retaining both supporting paragraphs. Stronger answer models do not absorb the loss: on MuSiQue, GPT-5.5 is 8.8 points less accurate on compressed contexts than on contexts retaining both supporting paragraphs. Finally, we train a compact classifier to rank omitted sentences by whether they are needed to interpret retained text and reinsert the top-ranked candidates without support annotations at inference. On HotpotQA with Qwen3-8B, this automatic restoration improves accuracy by 4.7 points while changing the compression ratio only from 0.30 to 0.31. Hard compressors should optimize both relevance and referential completeness.
50. 【2608.04567】STRIVE: Probing Reasoning Limits in Graded Plausibility Generation and Evaluation
链接：https://arxiv.org/abs/2608.04567
作者：Bhiman Kumar Baghel,Anna Chrabaszcz,Tessa Warren,Michael Walsh Dickey,Haley C. Dresang,Xiang Lorraine Li
类目：Computation and Language (cs.CL)
关键词：Event knowledge concerns, Event, knowledge concerns, STRIVE, controlled event sets
备注： Under Review
点击查看摘要
Abstract:Event knowledge concerns who does what to whom. Psycholinguists use event-plausibility judgments to examine how this knowledge supports human language processing. To isolate plausibility effects, these studies require controlled event sets in which one event slot varies across plausibility levels while all other event features remain fixed. Constructing such sets manually is labor-intensive. We therefore introduce STRIVE, an LLM-based framework for jointly generating and evaluating controlled event sets crossing plausibility class (plausible vs. implausible) with intended classification difficulty (easy vs. hard). Given a verb, STRIVE constructs a shared event frame, then produces one event per condition by varying one slot while holding all others fixed. In experiments with six models across 60 verbs, GPT-5.1 produced high-quality sets only 16.7% of the time using the baseline generation prompt. Adding a global reasoning scratchpad and evaluator-guided refinement raised this rate to 75.0%. Greater reasoning effort also improved evaluator--human agreement. Nevertheless, events near the plausibility boundary remain most difficult. They elicit the greatest human disagreement, and the best evaluator reaches only 57% accuracy on the implausible-hard condition, indicating a need for human input. Overall, STRIVE offers a scalable approach to reducing manual effort by automating initial event-set generation and evaluation for psycholinguistic studies.
51. 【2608.04565】Breadcrumbing Search Agents
链接：https://arxiv.org/abs/2608.04565
作者：Xuebin Li,Hanqing Zhao,Siyuan Liang,Kejiang Chen,Weiming Zhang,Dacheng Tao,Nenghai Yu
类目：Cryptography and Security (cs.CR); Artificial Intelligence (cs.AI); Computation and Language (cs.CL)
关键词：web content retrieved, external tool returns, critical security risk, tool returns introduces, LLM-based search agents
备注： 38 pages, 7 figures
点击查看摘要
Abstract:LLM-based search agents are widely used for information-seeking tasks, but their reliance on external tool returns introduces a critical security risk: web content retrieved during execution is untrusted, exposing agents to prompt injection and goal hijacking. Prior work on search-agent safety primarily focuses on static web-content injection, but modern agents issue follow-up queries and cross-check competing sources, so a single injected page is often diluted or rejected. We show that the channel delivering search and page observations is a fragile security boundary: beyond exposing the agent to a single poisoned page, a mediated search interface can repeatedly steer how the agent gathers evidence and forms its final answer. Under a constrained tool-intermediary threat model, appending only one controlled result per query can substantially increase attack success when the evidence is coordinated across the agent's trajectory. We study this setting with a strategy-driven long-horizon attack system and introduce Authority-Chain Hijack (ACH), an expert-refined strategy that turns isolated search-result and page-content manipulations into a coherent evidence chain across seemingly corroborating sources. ACH achieves the highest Overall ASR among all baselines, reaching 55.9% / 83.3% ASR / MaxN ASR on the full SafeSearch test split. We further introduce Trace-Guided Strategy Evolution (TGSE), which automatically improves attacker strategies from execution traces, replacing manual redesign with trace-driven refinement; its strongest single setting reaches 71.4% / 95.0% in held-out evaluation.
52. 【2608.04554】Representing Visual Evidence for Item Difficulty Prediction: Visual Textualization and Image-Native Modeling
链接：https://arxiv.org/abs/2608.04554
作者：Han Chen,Ming Li,Hong Jiao,Tianyi Zhou
类目：Computation and Language (cs.CL); Computer Vision and Pattern Recognition (cs.CV)
关键词：Predicting item difficulty, newly developed questions, Predicting item, content can provide, provide an initial
备注：
点击查看摘要
Abstract:Predicting item difficulty from content can provide an initial estimate for newly developed questions before sufficient student responses are available. Existing approaches typically represent the question stem and answer choices as text. When mathematics items contain visual components, a common pipeline first textualizes that evidence and then applies a text predictor. We ask: how should visual evidence be represented for item difficulty prediction? We compare question text alone, visual textualization, which expresses visual evidence in language, and image-native modeling, which retains the original image. Using Eedi items with difficulty calibrated from student responses, we train large language models (LLMs) and vision-language models (VLMs) directly for difficulty regression. Both visual interfaces achieve the lowest point estimates, although the leading systems cannot be reliably ordered. Open-VLM textualization yields lower RMSE point estimates for all evaluated LLMs, while broader adaptation does so for all image-native VLMs. Test-time interventions show dependence on the paired full-item image, but do not isolate the additional visual component. The two visual interfaces also make partially complementary item-level errors and differ substantially in computational workflow. Thus, textualization should not be treated as the only practical interface: image-native modeling is a competitive alternative whose effectiveness depends on how the VLM is adapted.
53. 【2608.04552】Relational Response Fields: A General Theory of Black-Box LLM Response Consistency and Recovery
链接：https://arxiv.org/abs/2608.04552
作者：Song Zichen
类目：Computation and Language (cs.CL); Machine Learning (cs.LG)
关键词：revising individual answers, iteratively revising individual, Black-box language-model reliability, pursued by sampling, individual answers
备注：
点击查看摘要
Abstract:Black-box language-model reliability is commonly pursued by sampling, prompting, voting, verifying, or iteratively revising individual answers. We ask a prior question: \emph{what determines whether a collection of black-box responses is recoverable at all?} We represent responses to typed transformations of a query as a \emph{relational response field} (RRF). Edge transports encode how valid responses must change under paraphrase, scaling, decomposition, refactoring, or other task symmetries; anchors encode independently trusted evidence such as execution or a verifier. For relation operator $D$, anchor operator $A$, and at most $k$ corrupted response nodes, we identify $\gamma_k(D,A)$ as the intrinsic difficulty of black-box response recovery. It is positive exactly when every $k$-node corruption is identifiable; it gives a deterministic stability bound proportional to $1/\gamma_k$; and a matching two-point minimax lower bound shows that no estimator can improve this dependence. Thus consistency is not truth: relation-only methods are blind to null directions, including shared hallucinations. We derive sparse field-repair algorithms while separating information-theoretic identifiability from the stronger null-space conditions required by convex optimization. Controlled theorem tests and black-box mathematics/code experiments evaluate four theory-fixed consequences: consistency--truth separation, anchor phase transitions, redundancy saturation, and cross-model, cross-task prediction of repair difficulty. The results support $\gamma_k(D,A)$ as a measurable property of a response-recovery instance, rather than a score attached to one repair heuristic.
54. 【2608.04549】EuroExec: Frontier Language Models Fall Short of Expert Judgment on European Executive Decision Tasks
链接：https://arxiv.org/abs/2608.04549
作者：Pau Arnal,Khaled Denfir,Danylo Smahliuk,Amrut Avhad,Marcus A. Castro
类目：Computation and Language (cs.CL); Artificial Intelligence (cs.AI); Machine Learning (cs.LG)
关键词：open-ended complex questions, Frontier LLMs, increasingly put, open-ended long-form European, complex questions
备注： 16 pages, 9 figures, 12 tables, submitted to EACL 2027
点击查看摘要
Abstract:Frontier LLMs are increasingly put to use on open-ended complex questions, different in nature from the ones they are typically evaluated on. We dedicate more than 4,000 human expert hours to evaluate a selection of six frontier LLMs on a member of this class of problems: EuroExec, our introduced human expert-based benchmark composed of 413 open-ended long-form European executive tasks authored by 47 vetted domain experts, each question drawn from experience in a real case. Every response is manually evaluated through a multi-attribute rubric, an item-specific checklist of requirements, and a preference rank ordering, extracting an aggregate metric "Solve Rate". The strongest model solves only 56.9% of tasks, while expert-written reference answers judged blindly are solved at near-ceiling levels and are preferred over every model response in 74% of direct rankings, placing frontier generative systems well below the professional standard of work they are already used for. We see that the best way to extract this kind of conclusion is by employing human evaluators, carefully checking their consistency through rigorous statistical analysis, and observe that automatic measurements also fall short when evaluating on this case of real-world open-ended problems with a subjective ground truth.
55. 【2608.04524】ODRA: Synthesizing Cognitive Behavioral Therapy Sessions with Structured Chain-Of-Thought and Dynamic Patient Resistance
链接：https://arxiv.org/abs/2608.04524
作者：Javier Rodriguez-Juan,Hiba Arnaout,Jose Garcia-Rodriguez,David Tomás,Iryna Gurevych
类目：Computation and Language (cs.CL); Machine Learning (cs.LG)
关键词：Cognitive Behavioral Therapy, generation of Cognitive, Cognitive Behavioral, competing demands, adhering to strict
备注： 39 pages, 23 figures, 12 tables
点击查看摘要
Abstract:Synthetic generation of Cognitive Behavioral Therapy (CBT) sessions is challenged by two competing demands: adhering to strict therapeutic structure while modeling the resistant, unpredictable behavior of real patients. Existing script-based methods fail to capture dynamic therapeutic interactions, while multi-agent approaches struggle to adhere to CBT's sequential structure; both suffer from sycophancy, producing overly compliant patients that misrepresent real clinical settings. In this work we introduce ODRA, a novel framework for synthesizing therapy dialogues through a Chain-of-Thought (CoT) strategy grounded in foundational CBT guidelines (Beck, 2020). ODRA further incorporates a resistance orchestrator to solve patient sycophancy, which employs steering techniques to elicit behaviors aligned with their resistance level. Automated and expert evaluations show that ODRA significantly outperforms existing methods across therapeutic skills, CBT alignment, and patient behavioral fidelity, with licensed psychologists preferring ODRA sessions across 12 of 13 clinical metrics. Furthermore, models fine-tuned on our dataset demonstrate superior therapeutic performance against both cooperative and resistant patients, validating that explicit resistance modeling in synthetic training data directly translates to downstream clinical robustness.
56. 【2608.04519】Leak-Resistant Unlearning: A New Benchmark for Evaluating Multi-Hop Reasoning Consistency and Recovery Robustness
链接：https://arxiv.org/abs/2608.04519
作者：Haoting Qian,Qingjie Zhang,Zhicong Huang,Cheng Hong,Han Qiu
类目：Artificial Intelligence (cs.AI); Computation and Language (cs.CL)
关键词：Benchmarking machine unlearning, Benchmarking machine, large language models, removed from large, large language
备注： 19 pages, 7 figures
点击查看摘要
Abstract:Benchmarking machine unlearning methods is critical to understand whether sensitive knowledge is removed from large language models (LLMs) or not. Current unlearning benchmarks include mainly single-hop questions and a narrow set of multi-hop questions. Although effective, they still face two challenges. (1) Knowledge is not isolated, whereby diverse multi-hop reasoning paths can potentially induce knowledge leakage than normal queries. (2) Unlearning may be fragile: unlearned knowledge can be partially recovered through recovery attacks such as lightweight post-unlearning adaptation, making static evaluation insufficient. Therefore, in this paper, we introduce \unlearning as a novel benchmark to understand robust LLM knowledge removal across diverse reasoning paths and recovery attacks. We experiment with this benchmark on 3 models, 6 unlearning methods, and 2 carefully curated datasets. Results show that existing methods are vulnerable to multi-hop reasoning paths and recovery attacks. We further explore the trade-off among forget quality, robustness, and model utility for LLM unlearning.
57. 【2608.04515】CARVE: Cross-Slice Anisotropic Reallocation of Visual Evidence for Efficient 3D Medical Volume Understanding
链接：https://arxiv.org/abs/2608.04515
作者：Zhenyu Yi,Qiang Hu,Zhenhao Li,Jiaxuan Zhao,Yusong Sun,Lichi Zhang
类目：Computer Vision and Pattern Recognition (cs.CV); Artificial Intelligence (cs.AI); Computation and Language (cs.CL)
关键词：Slice-based MLLMs leverage, MLLMs leverage mature, Slice-based MLLMs, encoders by representing, leverage mature
备注：
点击查看摘要
Abstract:Slice-based MLLMs leverage mature 2D encoders by representing 3D volumes as sequences of 2D slices. However, this slice-wise formulation produces thousands of visual tokens that burden the LLM backbone, many of which capture overlapping visual evidence across adjacent slices. To understand how effectively a growing visual token budget improves performance, we perform scaling analyses on two 3D medical VQA benchmarks and find diminishing returns: cost keeps rising while accuracy saturates, and improving in-plane resolution is more effective than adding slices at comparable budgets. The budget should therefore be allocated more selectively rather than simply enlarged, yet most token compression methods are designed for 2D images or videos, where redundancy arises from spatial layout or temporal motion rather than from near-duplicate content along the depth axis. We present CARVE, a training-free framework that compresses visual tokens prior to LLM inference and casts token reduction as budget-constrained 2.5D allocation. CARVE partitions the depth axis into coherent windows and allocates tokens non-uniformly according to normalized cross-slice evidence. Under a shared budget, CARVE builds spatial anchors on representative slices and retrieves locally varying evidence from the full volume, then merges remaining eligible tokens into nearby anchors within each window. Removing roughly 80% of the visual tokens on Hulu-Med-7B, CARVE leads all compression baselines on every AMOS-MM report-generation metric, with 6.2 points higher retention of full-token quality than the strongest baseline, and preserves 98.1% of full-token performance across three VQA benchmarks.
58. 【2608.04514】RESPClinBench: Benchmarking Multimodal Clinical Decision-Making and Longitudinal Disease Management in Respiratory Specialty Care
链接：https://arxiv.org/abs/2608.04514
作者：Mouxiao Bian,Zhi Chen,Ruiyao Chen,Lu Lu,Hengrui Liang,Chaoyi Huang,Yiluo Lin,Jingru Ding,Yun Zhong,Yuming Su,Jie Xu
类目：Computation and Language (cs.CL)
关键词：specialty care requires, Respiratory specialty care, requires multimodal interpretation, care requires multimodal, examination-oriented medical benchmarks
备注：
点击查看摘要
Abstract:Background: Respiratory specialty care requires multimodal interpretation, longitudinal risk assessment, guideline-concordant intervention, and whole-course management, which are poorly represented by examination-oriented medical benchmarks. Objective: To develop RESPClinBench, a real-world scenario-based benchmark for respiratory clinical decision-making, and evaluate seven contemporary large language models across AECOPD-PIM and PNBIM. Methods: RESPClinBench cases were adapted from de-identified respiratory clinical data. Three attending-level respiratory physicians revised cases, reference answers, and atomic clinical-action points, while one senior respiratory specialist performed cross-review and final adjudication. AECOPD-PIM comprised 427 open-ended COPD cases, and PNBIM comprised 196 multimodal pulmonary nodule cases combining chest CT with structured clinical information. Seven models generated 4,361 responses through standardized API inference with temperature 0 and a maximum output length of 8192 tokens. An automated framework calculated the final score as the arithmetic mean of atomic-action recall and rubric-based LLM-as-a-Judge assessment. Results: Across 623 cases, the mean final score was 68.58. Qwen3.6-27B ranked first overall at 71.22, Qwen3.5-397B-A17B led PNBIM at 72.48, and Qwen3.6-27B led AECOPD-PIM at 71.11. Imaging hallucination and serious medical risk occurred in 31.85% and 8.16% of PNBIM responses; medication-safety risk and serious medical risk occurred in 26.93% and 1.44% of AECOPD-PIM responses. Conclusions: RESPClinBench identifies task-specific limitations in multimodal pulmonary nodule assessment and longitudinal COPD management. Combining explicit clinical-action coverage, holistic evaluation, and independent safety flags provides a clinically grounded basis for model selection and prospective validation.
59. 【2608.04505】K-EXAONE 2.0 Technical Report
链接：https://arxiv.org/abs/2608.04505
作者：Eunbi Choi,Kibong Choi,Sehyun Chun,Seokhee Hong,Junwon Hwang,Hyojin Jeon,Ahra Jo,Hyunjik Jo,Yeonsik Jo,Minhyeok Jung,Doyoung Kim,Heegyu Kim,Joonkee Kim,Seonghwan Kim,Soyeon Kim,Sunkyoung Kim,Yireun Kim,Yongil Kim,Byungoh Ko,Changhun Lee,Dohaeng Lee,Haeju Lee,Jinsik Lee,Kyungmin Lee,Minwoo Lee,Wonkee Lee,Sangha Park,Sungjune Park,Kwangrok Ryoo,Kijung Seo,Minju Seo,Yongwoo Song,Sejong Yang,Heuiyeen Yeen,Stanley Jungkyu Choi,Yemuk Choi,Yongchan Chun,Jiwon Ham,Dasol Hong,Sujeong Im,Kijeong Jeon,Gerrard Jeongwon Jo,Hyeongjun Jo,Yujin Jo,Jiyeon Jung,Naeun Kang,Daeseong Kim,Euisoon Kim,Hayeon Kim,Hyosang Kim,Myoungshin Kim,Unsol Kim,Youchul Kim,Chaeeun Lee,ChaeYoon Lee,Edward Hwayoung Lee,Honglak Lee,Hwansoo Lee,Minkyung Lee,Sangeun Lee,Solji Lim,Woohyung Lim,Chanwoo Moon,Jueun Mun,Jimin Park,Seojeong Park,Yongmin Park,Hyerin Seo,Donghyeon Shin,Donghyun Son,Eunyong Son,Kaehyun Um,Sihoon Yang,Chang En Yea,Sihyuk Yi,Kyungjae Yoo,Chansik Yoon
类目：Computation and Language (cs.CL)
关键词：technical report presents, report presents K-EXAONE, foundation model developed, frontier-scale foundation models, global frontier-scale foundation
备注：
点击查看摘要
Abstract:This technical report presents K-EXAONE 2.0, an open-weight multilingual foundation model developed by LG AI Research as a step in our effort toward global frontier-scale foundation models. Rather than training from scratch, we upcycle K-EXAONE and expand its architecture, yielding a Mixture-of-Experts (MoE) model with 750B total parameters and approximately 37B activated per token---more than three times the capacity of its predecessor. K-EXAONE 2.0 supports context lengths of up to 256K tokens and expands multilingual coverage from six to ten languages. Its training pipeline combines continual pre-training, difficulty-focused mid-training, and post-training to strengthen reasoning, agentic coding, multilingual capability, and safety grounded in Korean sociocultural contexts. Across nine evaluation categories selected to reflect the conditions of practical use, K-EXAONE 2.0 improves over K-EXAONE and remains competitive with open-weight models, showing its largest gains in agentic coding and long-context understanding and its clearest strengths in long-context retrieval and safety. Released under the Apache 2.0 license, K-EXAONE 2.0 enables the wider AI ecosystem to evaluate, deploy, adapt, and build upon it, while marking the beginning---rather than the endpoint---of our challenge toward the global frontier.
60. 【2608.04488】Energy- and Memory-Efficient PEFT Methods for Personalized On-Device SLMs on Consumer GPUs
链接：https://arxiv.org/abs/2608.04488
作者：Kuanysh Akhmetzhanov,Jurn-Gyu Park
类目：Computation and Language (cs.CL)
关键词：large language models, Small Language Models, resource-constrained devices remains, devices remains impractical, remains impractical due
备注：
点击查看摘要
Abstract:Despite rapid advances in large language models (LLMs), deploying and personalizing them on resource-constrained devices remains impractical due to high VRAM, time, and energy costs. Parameter-Efficient Fine-Tuning (PEFT) of Small Language Models (SLMs) offers a promising alternative, yet few studies compare PEFT methods across architectures using both general and personalization benchmarks while accounting for energy consumption. We compare five fine-tuning approaches (Full Fine-Tuning, LoRA, LoRA+, QLoRA, and BitFit) on four SLMs from two families (Transformer-based: TinyLlama-1.1B, Qwen3-1.7B; SSM-based: Mamba-1.4B, Mamba-2-1.3B) across three GLUE tasks (SST-2, QNLI, STS-B) and three LaMP personalization tasks (LaMP-1, LaMP-2, LaMP-3). Each configuration is evaluated with the energy-focused NetScore-E and the memory-focused NetScore-M, the two variants that reflect the constraints binding on-device deployment. Methods are selected with a strict energy-first rule (highest NetScore-E, ties broken by NetScore#). LoRA+ achieves the highest NetScore-E in 19 of 24 configurations and the highest NetScore-M in 13 of 24, and is the selected method in 18 of 24. QLoRA, available only for the Transformer models, cuts peak finetuning VRAM by up to 3.9x relative to LoRA and therefore takes the best NetScore-M in 5 of the 12 Transformer configurations, although its de-quantization overhead leaves it selected in only one of them once energy decides. BitFit and full fine-tuning are almost never competitive on either variant, and TinyLlama-1.1B leads the energy-focused NetScore-E on five of the six benchmarks and the memory-focused NetScore-M on four. These results show that compact SLMs paired with PEFT provide a practical, energy-aware path to personalized on-device deployment, with the optimal method set by the dominant constraint: LoRA+ for energy and QLoRA for memory.
61. 【2608.04477】DeepInvert: Semi-Supervised Embedding Inversion Against Obfuscated Language Models
链接：https://arxiv.org/abs/2608.04477
作者：Zhicong Huang,Cheng Hong,Tao Wei
类目：Cryptography and Security (cs.CR); Computation and Language (cs.CL); Machine Learning (cs.LG)
关键词：services routinely process, routinely process prompts, Cloud-based language model, Cloud-based language, model services routinely
备注： 20 pages
点击查看摘要
Abstract:Cloud-based language model services routinely process prompts containing sensitive information. Obfuscation-based defenses---including ObfusLM, SentinelLMs, TextObfuscator, and DPNR---mitigate this risk by transforming prompt representations before transmission, offering a lightweight alternative to cryptographic solutions. We show these defenses provide far less protection than previously believed. We present DeepInvert, a semi-supervised embedding inversion attack that recovers original tokens from obfuscated representations with higher accuracy than prior methods. The key insight is that unlabeled obfuscated embeddings retain exploitable semantic structure despite perturbation. DeepInvert combines supervised training on labeled shadow data with a novel unsupervised consistency objective over unlabeled target embeddings, alternating between the two via a mixed training pipeline. Defense-aware adaptations further extend the attack to diverse obfuscation mechanisms across encoder-based and autoregressive architectures. Experiments on nine defenses, five tasks, and four model architectures show that DeepInvert outperforms prior attacks on most defenses. Against ObfusLM, DeepInvert achieves 73.5\% top-1 token recovery versus 26.2\% for the previous best. Our results reveal a task-dependent tension: obfuscation schemes preserving enough signal for utility also retain sufficient structure for inversion, while schemes resisting inversion collapse utility. On simpler classification tasks, some DP-based defenses can maintain both. We call for a re-evaluation of this defense class.
Comments:
20 pages
Subjects:
Cryptography and Security (cs.CR); Computation and Language (cs.CL); Machine Learning (cs.LG)
Cite as:
arXiv:2608.04477 [cs.CR]
(or
arXiv:2608.04477v1 [cs.CR] for this version)
https://doi.org/10.48550/arXiv.2608.04477
Focus to learn more
arXiv-issued DOI via DataCite (pending registration)</p>
62. 【2608.04472】EndoVLM: An Endoscopy Vision-Language Pre-training Model via Anatomy-Guided Sparsity and Progressive Alignment
链接：https://arxiv.org/abs/2608.04472
作者：Zhenyu Yi,Jianwei Xu,Yue Hu,Zhongwei Qiu,Sijing Li,Liang Huang,Bin Lv,Ling Zhang,Yingda Xia
类目：Computer Vision and Pattern Recognition (cs.CV); Artificial Intelligence (cs.AI); Computation and Language (cs.CL)
关键词：crucial for advancing, endoscopic image analysis, advancing endoscopic image, image analysis, advancing endoscopic
备注：
点击查看摘要
Abstract:The development of foundation models (FMs) is crucial for advancing endoscopic image analysis. However, existing endoscopy FMs mainly rely on self-supervised learning from uni-modal images or videos, overlooking the rich semantic knowledge contained in clinical reports. Furthermore, effectively leveraging these records is hindered by a fundamental modality gap: structured anatomical descriptions are not naturally mapped to specific frames within the high-redundancy, uncurated visual streams. In this paper, we present EndoVLM, a novel vision-language FM pre-trained on over 348K endoscopic examinations, each pairing a clinical report with its corresponding image collection. An Anatomy-Guided Sparse Pooling mechanism utilizes textual descriptions as queries to drive sparse attention, efficiently aggregating semantically salient frames into anatomy-specific visual representations across redundant image-sets. Next, a Progressive Semantic-Aware Alignment strategy models clinical taxonomy (anatomy and pathological status) via structured soft targets, bridging the gap from global patient-level matching to fine-grained localized alignment. Finally, a Semantic-Concentrated Masked Autoencoder is applied exclusively to these semantic-rich frames, integrating low-level visual precision with robust high-level semantic representation. Extensive experiments across various downstream tasks demonstrate that EndoVLM outperforms existing foundation models and remains competitive with task-specific methods. Remarkably, EndoVLM also exhibits robust zero-shot generalization capabilities, highlighting its potential for broader clinical application.
63. 【2608.04463】he Evaluator Is Part of the Experiment: Measuring Open-Ended LLM Conformity
链接：https://arxiv.org/abs/2608.04463
作者：Alicia Guerra,Yibo Hu
类目：Computation and Language (cs.CL)
关键词：LLM conformity largely, Prior work, work on LLM, largely measures discrete, verifiable labels
备注：
点击查看摘要
Abstract:Prior work on LLM conformity largely measures discrete answer flips under verifiable labels. Open-ended revisions require a different measurement strategy because answer quality is graded, latent, and judged imperfectly. We introduce an experimental protocol implemented across a pooled main peer-condition corpus and separately constructed decomposition corpora, allowing us to separate ordinary re-answering, candidate-content exposure, a bundled peer-presentation residual, and directional judge sensitivity to visible peer context. Across four open-weight generators and three benchmarks, all-wrong peer input produces the lowest-quality revisions in every generator-dataset cell. Blind and informed ratings of identical answers also differ by evaluator: one judge shifts toward the peer-endorsed position, two shift away, one is approximately neutral, and GPT-4o and GPT-5.4-mini audits are likewise non-neutral. Finally, an anchor audit shows that terse correct anchors can be misread often enough to destabilize the latent scale unless calibration is checked explicitly. These results support four conclusions: flip rates are insufficient as a complete measure of open-ended conformity, wrong peers harm open-ended revision, evaluators are not neutral, and anchor calibration is necessary.
64. 【2608.04452】Q-CueGraph: Query-Conditioned Visual Evidence Graphs for Multimodal Reasoning
链接：https://arxiv.org/abs/2608.04452
作者：Pengcheng Pan,Xinfang Zhang
类目：Computer Vision and Pattern Recognition (cs.CV); Artificial Intelligence (cs.AI); Computation and Language (cs.CL)
关键词：zoom tools give, tools give multimodal, give multimodal large, multimodal large language, large language models
备注：
点击查看摘要
Abstract:High-resolution pixels and crop or zoom tools give multimodal large language models the ability to inspect an image, but they do not provide a reliable task-conditioned policy for deciding where to inspect. Q-CueGraph makes this decision explicit. It maps a question and an image representation to budgeted, coordinate-level observations for a frozen reader. Text-rich images use a reusable OCR/layout graph; natural-image search instantiates query-conditioned visual nodes behind the same selection, composition, and budgeting interface. Optional utility refinement learns which candidate crops the frozen reader can use from training-answer correctness, without region-box supervision. With a frozen Qwen2.5-VL-7B reader, Q-CueGraph reaches 0.833 accuracy on V*Bench versus 0.696 for full-image inference from a 19% image-area budget, and reaches 92% of full-image ANLS on InfographicVQA from about half the image area. Across six benchmarks, explicit observation is most valuable when evidence is localizable, the question discriminates its location, and resolution limits full-image reading.
65. 【2608.04444】D$^2$F-ReAG: Dynamic Decomposition and Filtering for Multi-Hop Reasoning-Augmented Generation
链接：https://arxiv.org/abs/2608.04444
作者：Jiaoyang Li,Junhao Ruan,Shengwei Tang,Kaiyan Chang,Zhengtao Yu,Tong Xiao,Jingbo Zhu
类目：Computation and Language (cs.CL); Artificial Intelligence (cs.AI)
关键词：Large language models, Large language, static internal knowledge, inaccurate answers due, reliance on static
备注：
点击查看摘要
Abstract:Large language models (LLMs) often generate inaccurate answers due to their reliance on static internal knowledge. Retrieval-augmented generation (RAG) addresses this limitation by integrating external knowledge and excelling at single-hop queries. However, it struggles with multi-hop questions that require cross-document reasoning. Existing methods, such as graph structured RAG or question decomposition, often lack dynamic decomposition and effective filtering, which leads to lower efficiency and accuracy. To overcome these limitations, we propose Dynamic Decomposition and Filtering for Multi-Hop Reasoning-Augmented Generation (D2F-ReAG), a novel paradigm that adaptively controls reasoning depth by judging the reliability of the root-level reasoning. If the root reasoning is reliable, the model directly generates the answer. Otherwise, the question is logically decomposed into sub-questions, and the verified reasoning derived from these sub-questions is used to refine the root reasoning. Experiments on three multi-hop benchmarks demonstrate the effectiveness of our method in handling complex multi-hop questions.
66. 【2608.04433】MERaLiON-GR: Speech Gender Recognition Model for English and SEA Languages
链接：https://arxiv.org/abs/2608.04433
作者：Qiongqiong Wang,Ai Ti Aw,Nancy F. Chen,Ying Lay Chiu,Yang Ding,Yingxu He,Ridong Jiang,Zhuohan Liu,Yanfeng Lu,Yi Ma,Muhammad Huzaifah,Nabilah Binte Md Johan,Nattadaporn Lertcheva,Pham Minh Duc,Sailor Hardik Bhupendra,Siti Umairah Binte Mohammad Salleh,Shuo Sun,Tarun Kumar Vangani,Jeremy H. M. Wong,Jinyang Wu,Longyin Zhang
类目：Computation and Language (cs.CL); Artificial Intelligence (cs.AI)
关键词：performs binary classification, Southeast Asian languages, gender recognition system, Southeast Asian, speech gender recognition
备注：
点击查看摘要
Abstract:We present MERaLiON-GR, a speech gender recognition system that performs binary classification (female / male) on English and Southeast Asian (SEA) languages. The model finetunes MERaLiON-SpeechEncoder-2, a large conformer based transformer pre-trained on a broad speech corpus, and applies parameter efficient fine-tuning via Low-Rank Adaptation (LoRA) to adapt the encoder to the gender recognition task, and appends a multi-scale ECAPA-TDNN down stream network with attention pooling and a lightweight linear classifier. Extensive evaluations across multilingual Singaporean and Southeast Asian languages (English, Chinese, Malay, Tamil, Thai, Vietnamese, Indonesian, and Khmer) show that MERaLiON-GR consistently surpasses the state-of-the-art gender recognition model Vox-Profile and a large Audio-LLM, in both full-utterance and segment level evaluation modes. The results underscore the value of dedicated speech models in achieving accurate paralinguistic understanding and strong cross-lingual generalization.
67. 【2608.04426】Predict, Then Retrieve: Cross-Instance Future-State Retrieval from Video Prefixes
链接：https://arxiv.org/abs/2608.04426
作者：Quynh Vo,Thong Nguyen,Vinh-Hien Do,Cong-Duy Nguyen,Anh-Tuan Luu
类目：Computer Vision and Pattern Recognition (cs.CV); Computation and Language (cs.CL)
关键词：introduce Predictive State, Predictive State Retrieval, introduce Predictive, short video prefix, PSR combines anticipation
备注： Work in progress
点击查看摘要
Abstract:We introduce Predictive State Retrieval (PSR), a task in which a model observes a short video prefix and a temporal question about an object's future state, then retrieves instances from other videos or images that depict that state. Unlike action anticipation, which predicts a label, moment retrieval, which localizes an observed event within a video, or video generation, which synthesizes pixels, PSR combines anticipation with cross-instance retrieval across multiple temporal horizons. We construct a benchmark from four datasets with graded, human-validated ground truth, difficulty tiers, and an oracle ceiling. We also propose LFTR, a lightweight retriever with frozen encoders that predicts a question- and horizon-conditioned future latent and matches it in complementary semantic and visual spaces. A ceiling decomposition reveals a clear bottleneck: the true future state is highly retrievable once specified, whereas every predictor we evaluate, including a large multimodal language model with access to the prefix frames, remains far below the oracle. Thus, forecasting rather than perception is the central learnable challenge. LFTR narrows this gap at substantially lower inference cost, and ablations attribute its gains to cross-space fusion and hard-negative training rather than latent rollout. We release the benchmark, code, and evaluation scripts.
68. 【2608.04415】Social Pressure Breaks Majority Voting in LLM Safety Panels
链接：https://arxiv.org/abs/2608.04415
作者：Yibo Hu,Jiaming Qu
类目：Computation and Language (cs.CL)
关键词：Large language models, Large language, detect unsafe content, Large, language models
备注：
点击查看摘要
Abstract:Large language models (LLMs) are increasingly used to detect unsafe content. A common approach is to combine judgments from a panel of models to correct individual mistakes, but this benefit may disappear when every model sees the same misleading context before voting. We study this risk in a controlled two-round experiment. Each model first judges an item alone, then judges it again after six simulated peers either assert the wrong label or abstain. We combine the final judgments by majority vote. Across six open-weight LLMs and six datasets, we find that the wrong-label peer message raises the average reviewer false-alarm rate from 56.5% under silent peers to 87.5%, and majority voting raises the panel false-alarm rate to 100%. Without an asserted label, the same panel outperforms its average member. The effect is strongly asymmetric: reviewers follow pushes toward "unsafe" far more than pushes toward "safe" (about 75% versus 17%), so the panel's false-alarm rate rises sharply while its harmful-miss rate changes little. The proprietary-model probe shows substantial variation across models. These results identify susceptibility to shared social cues as a failure mode of safety panels and provide a simple pre-deployment diagnostic.
69. 【2608.04407】MESH: Memory-Efficient Sinkhorn Optimization for Mixture-of-Experts Training
链接：https://arxiv.org/abs/2608.04407
作者：Masato Fujitake
类目：Machine Learning (cs.LG); Artificial Intelligence (cs.AI); Computation and Language (cs.CL)
关键词：dense Transformer matrices, dense Transformer, Memory-efficient matrix optimizers, gradient descent remove, Sinkhorn gradient descent
备注： 10 pages
点击查看摘要
Abstract:Memory-efficient matrix optimizers such as Sinkhorn gradient descent remove most AdamW optimizer state for dense Transformer matrices, but direct application to Mixture-of-Experts (MoE) training is unreliable. We study this failure in a controlled 110M-parameter nanowhale DeepSeek-style MoE pretraining setting. A SAGE/Sinkhorn hybrid reduces optimizer state from 0.883GB to 0.331GB but degrades evaluation loss to 3.8265, far above the AdamW baselines observed in the same setup (3.58--3.64 across the seeds we study). We show that routed MoE expert matrices are the dominant failure point: their gradients are conditional, temporally varying, and poorly served by stateless Sinkhorn normalization. We propose MESH, a hidden-momentum Sinkhorn update for MoE experts. MESH restores a temporal first-moment signal through the gradient-buffer lifecycle, without storing the expert first moment as optimizer state. MESH is an optional block-preconditioned variant that adds a coarse neuron/block inverse-RMS multiplier. Across ablations, temporal smoothing before matrix normalization is the primary causal ingredient; block/neuron preconditioning can improve the memory-quality frontier, but is not established as universally necessary. In two additional seeds, MESH and MESH-B reduce optimizer-state memory by 62.5\% and peak PyTorch CUDA allocation by about 12.6\% relative to AdamW, with a modest evaluation-loss gap. Full-state diagnostic variants recover AdamW-like performance in ablations, supporting the conclusion that MoE experts need temporal smoothing, but not necessarily full coordinate-wise AdamW state.
70. 【2608.04405】raining-Free Hashing-Based Attention via Binary Principal Components
链接：https://arxiv.org/abs/2608.04405
作者：Daohai Yu,Zhanpeng Zeng,Keyu Chen,Wenhao Li,Zhifeng Shen,Luxi Lin,Ruizhi Qiao,Xing Sun,Rongrong Ji
类目：Machine Learning (cs.LG); Artificial Intelligence (cs.AI); Computation and Language (cs.CL)
关键词：major efficiency bottleneck, processing ever-growing key-value, repeatedly processing ever-growing, Long-context large language, large language models
备注： ICML 2026
点击查看摘要
Abstract:Long-context large language models (LLMs) are increasingly deployed in real-world applications, yet self-attention remains a major efficiency bottleneck -- especially during decoding -- due to the necessity of repeatedly processing ever-growing key-value (KV) caches. Existing sparse attention reduce computation by attending to fewer KV pairs, but often suffer from substantial accuracy degradation, require additional training, or rely on expensive hashing. In this work, we present BinaryPC, a training-free, data-aware hashing-based sparse attention for long-context LLMs. BinaryPC constructs compact binary hash codes and corresponding hash function by computing binary principal components of data. Unlike Locality-Sensitive Hashing (LSH) with data-independent random projections or learned non-linear hashing methods, BinaryPC constructs binary codes that explicitly preserve the structural information of data without requiring gradient-based training. Comprehensive experiments across multiple model families and long-context benchmarks show that BinaryPC preserves accuracy relative to full attention while achieving superior performance among sparse and hashing-based baselines. On modern GPUs, BinaryPC improves end-to-end decoding throughput by 3.56$\times$ over the FlashAttention kernel. Our code is available at this https URL.
71. 【2608.04397】NOLLI: A Difficulty-Calibrated Puzzle Benchmark for Diagnosing the English-Korean Performance Gap
链接：https://arxiv.org/abs/2608.04397
作者：Dasol Choi,Joonyong Park,Daegon Yu,Soo Yong Kim,Youngsook Song,Seunghyeok Hong
类目：Computation and Language (cs.CL)
关键词：introduce NOLLI, puzzle benchmark designed, performance gaps arise, procedurally generated English-Korean, generated English-Korean puzzle
备注：
点击查看摘要
Abstract:We introduce NOLLI, a procedurally generated English-Korean puzzle benchmark designed to diagnose where Korean performance gaps arise. It comprises 15 puzzle types (25 tasks; 7,500 items), with every instance seed-regenerable, verified to have a unique solution, and scored deterministically. Rather than equating harder with bigger, we calibrate difficulty behaviorally, tuning each generator until a fixed reference model lands in target accuracy bands. Its three-level design combines matched direct translations, script adaptations over Hangul jamo (sub-syllabic letters), and Korean-only tasks grounded in Korean culture or orthography. We evaluate 15 frontier, open-weight, and Korean-developed models; among the 12 above a 3% overall-accuracy floor, matched English-Korean accuracy is statistically equivalent within a +/- 10 pp margin (TOST), suggesting little cost from presentation language alone. Writing-system-intensive tasks show sharper gaps: Korean Cipher falls behind English by up to 68.7 pp, whereas Cryptarithmetic over the same jamo shows no systematic penalty, and Jamo Composition accuracy predicts Korean Cipher accuracy. These contrasts are diagnostic rather than causal, consistent with difficulty in multi-step sub-syllabic execution. Korean-only tasks separate rule-application deficits, which vary in sign, from a Kinship deficit positive in all 12. Finally, a salient size measure fails to grow from Easy to Hard in 7 of 15 types, making structural size an unreliable proxy for empirical difficulty.
72. 【2608.04390】EdgeLM: Edge Demonstrations for Language Models' Table Understanding
链接：https://arxiv.org/abs/2608.04390
作者：Soroush Omidvartehrani,Mohammadamin Habibollah,Mohammadreza Daviran,Davood Rafiei
类目：Computation and Language (cs.CL); Databases (cs.DB)
关键词：Large language models, perform table-centric prediction, Large language, making demonstration selection, demonstration selection critical
备注：
点击查看摘要
Abstract:Large language models (LLMs) perform table-centric prediction through in-context learning, making demonstration selection critical to performance. Existing retrieval methods prioritize similarity to the query, but similar demonstrations often reinforce the model's likely prediction rather than reveal the distinctions needed for difficult decisions. We propose EdgeLM, a retrieval framework that instead selects edge evidence, demonstrations that are both relevant to the query and informative about the decision boundary. EdgeLM retrieves two complementary forms of edge evidence by selecting data edges, nearby examples with different ground-truth labels, and model edges, similar examples previously misclassified by the deployed model. EdgeLM requires neither model retraining nor task-specific engineering. Across five data wrangling tasks, fifteen datasets, and five open-weight and proprietary LLMs, EdgeLM consistently achieves the best or near-best performance in every setting, while ablations show that the two forms of edge evidence provide complementary benefits. Our code and datasets are publicly available at this https URL.
73. 【2608.04374】FinReportBench: Measuring and Improving Institution-Grade Financial Report Generation
链接：https://arxiv.org/abs/2608.04374
作者：Yinghao Tang,Tan Zhenwei,Yiyao Wang,Wanli Gu,Xiaolu Zhang,Jun Zhou,Wei Chen
类目：Computation and Language (cs.CL); Artificial Intelligence (cs.AI)
关键词：Large language models, fluent financial analysis, produce fluent financial, Large language, produce fluent
备注： 9 pages, 9 figures
点击查看摘要
Abstract:Large language models can produce fluent financial analysis, but fluency alone does not establish whether a report is suitable for institutional delivery. We introduce FinReportBench, an expert-grounded benchmark for measuring and improving institution-grade financial report generation. Expert review reveals recurring gaps in report identity, institutional components, source discipline, and visual delivery. We derive a 35-item rubric through expert partial orders, multimodal evidence, and audits of decision boundaries, covering deliverability, report identity, and institutional completeness. Starting from 10,000 balanced Chinese and English financial-research source records, we curate 244 bilingual tasks across three research objects and two input tiers. Each task separates the public query, reconstructed research trajectory, and hidden source packet. Three independent judge families reproduce the expert partial order at near-ceiling rates, showing that bounded, observable criteria support reliable evaluation. Across nine model families, basic deliverability is nearly saturated, while report identity and institutional completeness remain the primary bottlenecks. The largest cross-model gaps concern generation-trace control, information density, and data discipline rather than basic report framing. We then use benchmark-guided skill distillation to turn recurrent failures into reusable generation and self-review constraints. Across five model families, the evolved skill improves mean G1 by 33.85 points and mean G2 by 13.83 points over paired no-skill runs while preserving G0 for every pair. Code and benchmark artifacts are available at this https URL.
74. 【2608.04355】he Calibration Floor: Format Repair Can Masquerade as Self-Correction at Small-to-Mid Scale
链接：https://arxiv.org/abs/2608.04355
作者：Mingguang Chen,Bo Qu,Licheng Wang
类目：Computation and Language (cs.CL)
关键词：language-model self-revision, cells, content margin, content, margin
备注： 36 pages, 5 figures
点击查看摘要
Abstract:Accuracy changes after language-model self-revision are usually interpreted as changes in reasoning. We show this can fail at the answer-extraction boundary, and test the failure causally rather than only observationally. Across Qwen3.5 (0.8B-9B), Gemma-4-12B, and two frontier models via API (Tencent Hy3, Nvidia Nemotron-3-Ultra-550B) in 29 primary cells plus a frontier arm, we decompose the always-revise accuracy shift into a content margin (both answers parseable) and format-recovery/loss margins (parseability changes). On 12 cells with meaningful unparseable-answer rates, format effects exceed content effects (Wilcoxon p=1.7e-3). To test this causally, we force already-generated reasoning through grammar-constrained decoding so every answer is parseable by construction: across 14 cells this closes a median 71% of the gap between the naive total effect and the content-margin estimate, with two cells converging exactly and a residual on the two largest-effect cells reported rather than dismissed. A clustered model confirms floor-scale (0.8B/2B) models have far higher odds of content-level change and harm than capable-scale models (p1e-7). Replicating a cited confidence-gating protocol verbatim on Qwen3.5 does not reproduce its reported gain and shows the same near-zero content margin. A frontier check on much larger models shows format-dominance intensifying with scale: content margin is exactly zero in all 5 cells despite total effects up to +0.275, though this arm is lower-powered. The calibration-floor criterion on the content margin reveals a squeeze: floor-scale cells have headroom but insufficient signal, capable-scale cells have signal but little headroom; only one cell is marginally viable, with negligible sealed-holdout gain. Content is a minority share of what the field has measured as self-correction. We release the instrument, code, and derived results.
75. 【2608.04339】Equitable System-Prompt Selection via Constrained Mixed-Strategy GroupDRO
链接：https://arxiv.org/abs/2608.04339
作者：Mengyu Xu,Qiaoxin Yang,Zhihan Liu,Ruiyao Xu,Zachary Liu,Kezhen Chen,Chongyang Gao
类目：Computation and Language (cs.CL); Artificial Intelligence (cs.AI); Computational Engineering, Finance, and Science (cs.CE); Machine Learning (stat.ML)
关键词：Large language models, semantically equivalent questions, equivalent questions phrased, Large language, information seeking
备注：
点击查看摘要
Abstract:Large language models are increasingly used for information seeking, yet semantically equivalent questions phrased in different ways can receive answers of considerably different quality. System prompts are widely employed to steer response behavior, but they are typically optimized for average-case quality, so some question phrasings may still receive incomplete or low-quality answers. To address this, we formulate a constrained mixed-strategy GroupDRO framework for system-prompt selection. Instead of optimizing the system-prompt text, the framework assigns weights to system prompts in an existing pool to minimize the worst-case information-quality loss across evaluation metrics and groups, while constraining the mean loss to stay close to that of average-based selection. Because pool generation and selection are decoupled, the method applies to any system-prompt pool and can leverage an ensemble of complementary system prompts rather than a single one. Across five LLMs on two bilingual medical and consumer-finance benchmarks, the constrained method reduces the Overall Mean, Worst 25% Mean, and Worst by 13.1%, 13.2%, and 13.7% on average relative to no mitigation while keeping overall quality close to Average selection. Its multi-prompt weights reveal complementarity across metric-group pairs. Code and data are available at this https URL.
76. 【2608.04330】Right Reset: Chunking by Prefix Removal
链接：https://arxiv.org/abs/2608.04330
作者：Mike Vegeto
类目：Computation and Language (cs.CL); Machine Learning (cs.LG)
关键词：Removing the left, causal language model, language model reveals, right-hand tokens, right-hand hidden-state trajectory
备注： 12 pages, 2 figures, 4 tables. Code, data, and reproduction materials: [this https URL](https://github.com/ZECTBynmo/right-reset-paper)
点击查看摘要
Abstract:Removing the left context from a causal language model reveals a useful kind of boundary: an edge where the model processes the same right-hand tokens with little change. We turn this observation into prefix-removal probing and introduce Right Reset (RR), which measures preservation of the right-hand hidden-state trajectory. A dynamic program converts RR edge scores into variable-length chunks. On flattened text formed by concatenating topically similar records after deleting their separators and layout, RR recovers 47.7% of the original records as clean units, versus 25.9% for a BGE embedding boundary, the strongest tested conventional baseline without task-specific model training. The gain persists after rendering and OCR. Passive scores from the same Qwen3-4B layer and direct prompting of a same-scale instruction model perform substantially worse on flattened records. Across six language models, RR-selected cuts also undergo consistently less local output disruption than unselected candidate edges. An observed-token likelihood-ratio readout is competitive in some architectures, indicating that the central contribution is the intervention: context dependence itself can provide a boundary signal when surface structure is weak.
77. 【2608.04322】DataRx: Missingness-Aware Sampling for Safer Large Language Model Task-Specific Fine-Tuning
链接：https://arxiv.org/abs/2608.04322
作者：Junbo Zhang,Qianli Zhou,Xinyang Deng,Wen Jiang
类目：Computation and Language (cs.CL)
关键词：Task-specific fine-tuning, large language models, safety, improve the performance, performance of large
备注：
点击查看摘要
Abstract:Task-specific fine-tuning can improve the performance of large language models (LLMs) on downstream tasks. However, our study reveals that task-specific fine-tuning can also weaken the safety guardrails of aligned LLMs. A widely adopted strategy for preserving safety during fine-tuning is to incorporate safety data. Although previous studies have shown that randomly mixing safety data can alleviate safety degradation, the underlying principle determining why some safety examples are more effective than others still remains unclear. In this paper, we propose DataRx, a missingness-aware sampling method for selecting safety-critical examples. DataRx is based on the hypothesis that a safety sample is more effective when the selected examples provide safety signals that fill the missing parts of LLMs' safety capabilities. DataRx's key insight is leveraging high-dimensional hidden representations rather than discrete tokens to quantify the safety signal gap between the target model's native response and the safety reference response. The results show that, with only 1% additional safety samples from BeaverTails, DataRx reduces the average attack success rate of Llama3-8B-Instruct across seven downstream tasks from 59.23% under random sampling to 13.70%. In addition, DataRx can be combined with the existing safety data synthesis method to further enhance safety defenses during fine-tuning. We hope that DataRx will inspire more data-centric defense research.
78. 【2608.04311】Pun Intended: Multi-Agent Translation of Wordplay with Contrastive Learning and Phonetic-Semantic Embeddings
链接：https://arxiv.org/abs/2608.04311
作者：Russell Taylor,Benjamin Herbert,Michael Sana
类目：Computation and Language (cs.CL)
关键词：long challenged, challenged both professional, professional translators, translators and machine, large language models
备注：
点击查看摘要
Abstract:Translating wordplay across languages has long challenged both professional translators and machine translation systems. We investigate three approaches to translating puns from English to French by combining large language models with linguistic constraints for wordplay generation. Our baseline uses a large language model with feedback from a discriminator prompted with positive and negative French examples. Our guided reasoning pipeline uses combined phonetic-semantic embeddings to retrieve lexical candidates for wordplay generation. Finally, our multi-agent framework iteratively evaluates and regenerates candidate translations using specialized feedback. Moving beyond literal translation, our objective is to preserve the linguistic creativity, ambiguity, and humor of the source-text wordplay rather than simply reproduce its vocabulary. The multi-agent and guided chain-of-thought systems ranked first and second, respectively, in the CLEF JOKER 2025 Task 2 competition under expert human evaluation, despite only modest improvements in BLEU and BERTScore. These findings suggest that both explicit phonetic-semantic guidance and iterative multi-agent evaluation can improve LLM-based wordplay translation relative to direct discriminator-guided generation, particularly when balancing semantic fidelity, phonetic similarity, and natural target-language expression
79. 【2608.04307】MIDAS: Multi-LLM Iterative Data-Adaptive Summarization
链接：https://arxiv.org/abs/2608.04307
作者：Karen Lee,Dhanashree Balaram,Seojun Shon,Umair Rasheed
类目：Computation and Language (cs.CL); Artificial Intelligence (cs.AI); Multiagent Systems (cs.MA)
关键词：deceptively difficult, Text summarization, Large Language Model, Text, summarization
备注： Accepted at the 20th International Conference on Document Analysis and Recognition (ICDAR 2026). 17 pages, 2 figures
点击查看摘要
Abstract:Text summarization is deceptively difficult. While condensing information seems straightforward, real-world enterprise summarization of support tickets, legal documents, incident reports, and more, demands strict adherence to domain-specific guidelines, output formats, and organizational conventions. Crafting prompts that reliably satisfy these constraints is labor-intensive, requiring significant human expertise and continuous maintenance as requirements evolve. Existing automated prompt optimization methods reduce this burden through Large Language Model (LLM) critique-driven refinement, yet remain limited by static prompts that cannot adapt to the diversity of summary applications. We propose Multi-LLM Iterative Data-Adaptive Summarization (MIDAS), a multi-LLM framework that extends this paradigm with data-driven pattern learning and use-case-specific personalization, enabling automatic adaptation to different summarization requirements without manual prompt engineering. Applied to enterprise customer ticket summarization across five output formats, MIDAS achieves the strongest overall performance against state-of-the-art critique-driven optimization frameworks such as CriSPO and ZERA, improving ROUGE-1 by up to 11.0%, ROUGE-2 by up to 18.2%, and ROUGE-L by up to 8.0%, while consistently improving BERTScore F1 across all formats and output types. We additionally demonstrate cross-model and cross-domain generalization through multi-LLM configurations and finance-domain summarization benchmarks.
80. 【2608.04299】Searching for Sound-Meaning Collisions: Graph-Based Affordance Retrieval and Multi-Evaluator Ranking for Pun Translation at CLEF 2026 JOKER Task 2
链接：https://arxiv.org/abs/2608.04299
作者：Russell Taylor,Adam Brikman,Prateek Awate
类目：Computation and Language (cs.CL)
关键词：Fifteen years ago, Fifteen years, years ago, translators should stop, stop searching
备注： CLEF 2026 Working Notes, 21-24 September 2026, Jena, Germany
点击查看摘要
Abstract:Fifteen years ago, Low proposed that pun translators should stop searching for equivalent words and instead search for new points of contact between sound and meaning. In this paper, we investigate that idea computationally. We model pun translation as a process of discovery, exploration, and selection. A retrieval system searches semantic and phonological neighborhoods for target-language affordances: sound-meaning bridges that may support new wordplay. Multiple language models then explore these opportunities by generating competing translations, while a multi-perspective generate-and-rank architecture selects among them. Beyond system development, our primary contribution is an analysis of how retrieved affordances propagate through the translation process. We find that generators actively exploit retrieved opportunities, evaluators progressively concentrate around stronger sound-meaning bridges, and exact phonological collisions are selected at disproportionately high rates when available. At the same time, many puns still yield no usable affordances, suggesting that retrieval remains the central bottleneck in computational pun translation. The resulting picture is remarkably close to the process envisioned by Low. Successful pun translation emerges not from preserving source-language words, but from discovering new places in the target language where sound and meaning collide.
81. 【2608.04289】SafeCommit: Certifying When Memory-Grounded Agents May Safely Act
链接：https://arxiv.org/abs/2608.04289
作者：Mayur Akewar,Ravi Ranjan
类目：Artificial Intelligence (cs.AI); Computation and Language (cs.CL)
关键词：Long-horizon agents increasingly, Long-horizon agents, external side effects, increasingly use persistent, Long-horizon
备注： 14 pages, 6 tables, and 1 figure, target NeurIPS
点击查看摘要
Abstract:Long-horizon agents increasingly use persistent memory and tools to take actions with external side effects. A central failure mode is premature commitment: an agent acts before resolving whether its memory grounding is stale, conflicting, incomplete, or corrupted. We formalize this problem as safe commitment under memory uncertainty and introduce SafeCommit, a risk controlled layer between agent reasoning and external execution. The layer constructs a calibrated set of plausible latent worlds from memory, observations, tool outputs, provenance, and policy constraints. It permits a side effectful action only when a conformal action certificate shows that the action is safe in every retained world. Otherwise, it selects a low-side-effect probe that targets the worlds blocking certification, or returns a conservative fallback. Under calibrated world coverage, the probability of an unsafe certified commit is at most the target level {\alpha}; with imperfect world proposal, the bound separates calibration and representation error. A dependency-free controlled simulator illustrates the safety-utility tradeoff and reproduces all reported results with one command. The goal is to offer a concrete approach for deciding not only what an agent should do, but when the available evidence is sufficient to safely do it.
82. 【2608.04286】Eliciting Intrinsic Hallucinations in LLMs via Semantically Equivalent Adversarial Attacks
链接：https://arxiv.org/abs/2608.04286
作者：Atri Vivek Sharma,Brian Formento,Alessio Lomuscio
类目：Computation and Language (cs.CL); Machine Learning (cs.LG)
关键词：Large language models, external knowledge sources, Large language, Retrieval-Augmented Generation, conjunction with external
备注： To be presented at COLM 2026
点击查看摘要
Abstract:Large language models (LLMs) are often used in conjunction with external knowledge sources to improve their factual accuracy and decrease hallucinations, through methods such as Retrieval-Augmented Generation (RAG). However, these systems remain susceptible to intrinsic hallucinations, where the model generates unfaithful or fabricated information that is not supported by the retrieved evidence. We propose a novel framework to assess model robustness against this phenomenon by stress-testing using natural, semantically equivalent variations of a user query found via adversarial optimization methods. We apply our framework, which enforces strict semantic equivalence constraints and an intrinsic hallucination objective, to a range of adversarial attack techniques across white-box, gray-box, and black-box adversarial settings. Evaluating these attacks on 5 open-source and 5 closed-source generator models across 3 datasets, we demonstrate that even state-of-the-art models are highly susceptible to meaning-preserving perturbations, which significantly degrade contextual faithfulness (by up to 50% for GPT-5-mini). Our findings indicate that faithful use of in-context evidence remains fragile even in state-of-the-art LLMs, motivating architectures and training objectives that enforce robust grounding independent of surface query form. Code is available at: this https URL
83. 【2608.04271】LLM-based Vulnerability Discovery in Business Process Documentation
链接：https://arxiv.org/abs/2608.04271
作者：Ben Falchuk,Himanshu Garg,Euthimios Panagos,Sioan Zohar
类目：oftware Engineering (cs.SE); Computation and Language (cs.CL)
关键词：product quality issues, software and hardware, increased costs, processes are susceptible, lead to product
备注：
点击查看摘要
Abstract:Just like software and hardware, business processes are susceptible to vulnerabilities that can lead to product quality issues, delays, and increased costs. Business process vulnerabilities can arise from a variety of sources, including conflicting requirements, ambiguous documentation, invalid measurement spec-ifications, omission of quality checks, or implementations that differ from speci-fications. MIRABELLE is a system that identifies and characterizes business logic (BL) vulnerabilities from available business process representations, in-cluding ISO 9000/9001 documentation, user guides, work instructions, and pro-cess execution logs. MIRABELLE leverages recent advances in AI/ML to pro-cess available business process documentation and generate attributed graph rep-resentations of the business logic that can be processed using both graph and for-mal logic approaches for identifying potential vulnerabilities. However, extract-ing the business logic (e.g., operation execution sequences, decisions, input/out-put resources) from mostly natural language artifacts is challenging due to the required domain expertise, inherent process complexity, and the sometimes very large volumes of information. This paper focuses on our experimentation with Large Language Models (LLMs) and their role within MIRABELLE. We report on the performance of several LLMs across vital stages of vulnerability detection, from grammatical and technical error-flagging in short phrasings, to complete process structure recovery and extraction.
84. 【2608.04268】he Fairness Collapse Phenomenon: Bias Amplification in Language Models Trained on Synthetic Data
链接：https://arxiv.org/abs/2608.04268
作者：Irina Proskurina,Antoine Gourru,Julien Velcin
类目：Computation and Language (cs.CL)
关键词：Generative models trained, artificially generated data, Generative models, significant performance degradation, resulting in significant
备注：
点击查看摘要
Abstract:Generative models trained on artificially generated data have been shown to exhibit model collapse, resulting in significant performance degradation. As synthetic content increasingly contaminates the training corpora of language models, this raises critical concerns about the use of open data in continued pretraining. Although previous work has demonstrated model collapse in language models, it remains unclear whether exposure to synthetic data amplifies or attenuates the social biases already present in pretrained models. Because language models are known to reproduce and amplify demographic stereotypes, recursive training on self-generated data may create a self-reinforcing feedback loop in which biased associations become progressively stronger across generations. We call this hypothesized phenomenon fairness collapse. In this work, we construct controlled training regimes in which models are repeatedly trained on synthetic data using the Bias in Bios dataset. Across experiments, we observe a consistent and concerning pattern: fairness degradation emerges before substantial degradation is reflected by standard language-modeling metrics. This result highlights a critical risk associated with synthetic data contamination in language model training: bias can increase silently before strong indicators of model collapse become apparent.
85. 【2608.04260】owards End-to-End Multilingual Metaphor Processing: Integrating Detection, Translation, and Evaluation
链接：https://arxiv.org/abs/2608.04260
作者：Jiahui Liang,Lifeng Han
类目：Computation and Language (cs.CL)
关键词：literal lexical meaning, Metaphorical language remains, translation require reasoning, lexical meaning, Metaphorical language
备注： Scientific report on PhD thesis plans and milestones achieved (current progress)
点击查看摘要
Abstract:Metaphorical language remains a major challenge for multilingual natural language processing because successful interpretation and translation require reasoning beyond literal lexical meaning. Existing research has largely investigated metaphor detection, machine translation, and translation evaluation as separate tasks, while little work has explored how these components can be integrated into a unified computational framework. This PhD proposal aims to develop an end-to-end framework for multilingual metaphor processing consisting of three complementary research directions: (1) robust metaphor detection across languages, (2) metaphor-oriented translation evaluation for both human assessment and automatic quality estimation, and (3) joint modelling that connects metaphor detection with translation evaluation. The proposed research will combine linguistic theory with recent advances in large language models to develop new datasets, annotation methodologies, evaluation benchmarks, and automatic evaluation approaches for metaphor-aware machine translation. The expected outcome is a unified framework that improves both the development and evaluation of multilingual NLP systems when processing figurative language.
86. 【2608.04244】SIGNPOST-Bench: Benchmarking Text-Vision Conflict Resolution in Multimodal Large Language Models
链接：https://arxiv.org/abs/2608.04244
作者：Sirun Li,Minghao Liu,Ling Dai,Yong Li,Haoxin Lyu,Junting Zhou,Fan Zhang
类目：Computer Vision and Pattern Recognition (cs.CV); Computation and Language (cs.CL)
关键词：existing benchmarks rarely, benchmarks rarely reveal, Multimodal large language, make grounded predictions, make grounded
备注： 27 pages, 25 figures
点击查看摘要
Abstract:Multimodal large language models (MLLMs) make grounded predictions in real-world scenes by combining visual and textual cues, yet existing benchmarks rarely reveal how they arbitrate between these evidence sources when they conflict. We introduce SIGNPOST-Bench, a controlled counterfactual benchmark for evaluating text-vision conflict resolution. Each source image is transformed into a counterfactual quintuplet of Original, Blank, Similar, Random, and Adversarial variants. Synthetic, localized scene-text interventions are designed to preserve non-textual content, enabling paired measurements of changes in localization performance and directed shifts toward geographic targets introduced by conflicting text. SIGNPOST-Bench contains 5,111 counterfactual groups and 25,555 image variants from four datasets. We evaluate 20 MLLMs from seven providers. Compared with Original images, Adversarial variants raise median localization error from 282 km to 1,347 km, a 4.8-fold increase. Among geocodable adversarial samples, 6.5-20.1% of predictions lie less than 50 km from the injected target across models, and every evaluated model exhibits a positive mean paired reduction in target distance from Blank to Adversarial. Compatible, unrelated, and conflicting text replacements produce distinct effects on model predictions, while clean-input localization performance does not fully predict robustness to conflicting text. These results establish visual geolocation as a continuous diagnostic of scene-text arbitration and provide a controlled framework for evaluating how MLLMs resolve conflicting multimodal evidence.
87. 【2608.04240】Hallucinations on the Board: Tool-Augmented Evaluation of LLM Chess Commentary
链接：https://arxiv.org/abs/2608.04240
作者：S. Ashwin Hebbar,Peiyao Sheng,Sewoong Oh,Pramod Viswanath
类目：Computation and Language (cs.CL); Artificial Intelligence (cs.AI)
关键词：Superhuman game engines, evaluations easily accessible, Superhuman game, made expert-level evaluations, expert-level evaluations easily
备注： 23 pages, 6 figures
点击查看摘要
Abstract:Superhuman game engines in domains like chess have made expert-level evaluations easily accessible, yet they communicate what is true without the natural-language explanations that make such expertise educationally useful to experts and non-experts alike. Large language models could, in principle, bridge this gap, but they frequently hallucinate due to limited domain-specific knowledge, and standard reference-based or LLM-as-a-judge frameworks cannot reliably detect these errors. In this work, we present ACT-Eval, an evaluation framework that decomposes chess commentary into atomic claims and routes them to engine-supported tools and expert-annotated gold references to assess factual correctness, conceptual coverage, and move-quality judgment. We release a benchmark of 325 position--move pairs spanning pedagogical, tournament, and critical positions, including 125 positions with expert-verified gold atoms and a five-class error taxonomy. Evaluating leading proprietary and open-weight models, we find that factual hallucinations remain pervasive in chess commentary: GPT-5.4 without tools produces incorrect sub-claims 22.0% of the time, while smaller open-weight models exceed 40%. Although tool augmentation substantially improves factual correctness and move-quality assessment, coverage of expert strategic and tactical ideas remains limited across all models. Human calibration shows that ACT-Eval's factual judgments fall within the observed range of inter-human agreement, while its coverage scores correlate strongly with human assessments of strategic completeness.
88. 【2608.04193】Patients-like-me: A Variational LM--GNN Framework for Explainable Clinical Prediction
链接：https://arxiv.org/abs/2608.04193
作者：Xinyu Wang,Yixuan Li,Hanwei Wu,Qincheng Lu,Chi-Kuang Yeh,Xiao-Wen Chang,Ziyang Song
类目：Computation and Language (cs.CL); Artificial Intelligence (cs.AI)
关键词：offer strong textual, electronic health records, strong textual representations, provide limited explainability, encode patient sequences
备注：
点击查看摘要
Abstract:Language models (LMs) offer strong textual representations for electronic health records (EHRs), but they encode patient sequences in isolation and provide limited explainability. Graph neural networks (GNNs) complement LMs by incorporating inter-patient relationships and enabling reference-patient attribution, yet they rely on high-quality patient representations. We propose Patients-like-me (PLM), a unified LM--GNN framework that integrates local patient semantics with global cohort structure. To train PLM efficiently, we introduce a Variational Expectation-Maximization algorithm that alternates LM and GNN updates under a supervised variational objective. Extensive experiments on MIMIC-III and MIMIC-IV show that PLM consistently outperforms state-of-the-art methods, with improvements generalizing across encoder-only and decoder-only LM backbones. These gains are achieved with only modest additional computational overhead. PLM also provides reference-patient explanations by retrieving influential similar patients, while edge-masking experiments confirm that the highest-ranked references have the greatest impact on model predictions.
89. 【2608.04192】Behavioral Skill Reconstruction: Reconstructing Hidden Functionality from LLM Agent Skills
链接：https://arxiv.org/abs/2608.04192
作者：Peichun Hua,Haoxuan Xu,Mengyuan Li
类目：Cryptography and Security (cs.CR); Artificial Intelligence (cs.AI); Computation and Language (cs.CL)
关键词：Closed source agent, encode proprietary instructions, Closed source, source agent skills, proprietary instructions
备注： 20 pages, 5 figures, 20 tables
点击查看摘要
Abstract:Closed source agent skills may encode proprietary instructions, scripts, constants, and data. Providers may offer their capabilities as services while keeping the underlying packages hidden. Prior work focuses on prompt injection attacks that directly disclose these artifacts, and existing defenses accordingly aim to prevent such leakage. However, preventing file disclosure does not prevent users from recovering the functionality those files implement. This raises a fundamental question: can a user reconstruct a skill's functionality through ordinary use while its files remain hidden? We study behavioral skill reconstruction (BSR), in which an attacker uses valid task requests and observed responses to build a functional clone of a hidden skill. We introduce SkillClone, a black-box attack that clones a target skill by forming an interface hypothesis from its public advertisement, issuing structured benign probes, synthesizing an executable replica, and iteratively repairing it through differential validation against the victim skill. Across 30 skills spanning rules, tables, procedures, and algorithms, SkillClone achieves exact or partial recovery on held-out inputs for several targets. Iterative requerying closes gaps missed by single-round reconstruction. Because SkillClone uses only legitimate interactions, disclosure-focused defenses provide limited coverage, and less detailed skill descriptions offer limited protection. These results show that file secrecy alone does not ensure functional secrecy. Defenses must also limit cumulative information leakage from ordinary use.
Comments:
20 pages, 5 figures, 20 tables
Subjects:
Cryptography and Security (cs.CR); Artificial Intelligence (cs.AI); Computation and Language (cs.CL)
Cite as:
arXiv:2608.04192 [cs.CR]
(or
arXiv:2608.04192v1 [cs.CR] for this version)
https://doi.org/10.48550/arXiv.2608.04192
Focus to learn more
arXiv-issued DOI via DataCite (pending registration)</p>
90. 【2608.04186】Large Language Models for Low-Resource Languages: A Conceptual Framework for an Electronic Explanatory Dictionary of the Tajik Language
链接：https://arxiv.org/abs/2608.04186
作者：Mullosharaf K. Arabov
类目：Computation and Language (cs.CL)
关键词：large language models, Tajik, natural language processing, language models, language
备注： Preprint
点击查看摘要
Abstract:This paper presents a conceptual framework for developing an electronic explanatory dictionary of the Tajik language using large language models (LLMs). The relevance of the work stems from the absence of a comprehensive digital lexicographic resource for Tajik that is comparable in functionality to dictionaries for high-resource languages, and from the limited adaptation of modern natural language processing technologies to low-resource language systems. Based on a systematic survey of existing linguistic, statistical, and corpus resources, we propose a dictionary architecture that integrates modules for morphological analysis, lemmatization, semantic clustering, and dictionary entry generation using LLMs. The choice of subword tokenization is justified by the agglutinative nature of Tajik morphology and its high morphological variability, along with a parameter-efficient fine-tuning (PEFT) strategy suitable for limited annotated data. The novelty of the work lies in proposing the first holistic conceptual architecture of an explanatory dictionary for Tajik that unifies classical lexicographic methods, language statistics, and generative capabilities of LLMs into a single system. The practical significance of the study is the formation of a methodological foundation for developing a full-featured electronic dictionary that can serve both as a lexicographic tool and as a core resource for machine translation, automatic summarization, sentiment analysis, and other applied NLP tasks. The paper is intended for specialists in computational linguistics, lexicography, and developers of natural language processing systems working with low-resource languages.
91. 【2608.04183】st, then Route: How Language Models Execute In-Context Conditional Rules Across Models and Languages
链接：https://arxiv.org/abs/2608.04183
作者：Luxshan Thavarasa,Sivasuthan Sukumar
类目：Computation and Language (cs.CL)
关键词：in-context conditional rule, in-context conditional, conditional rule, assemble a runtime, runtime circuit
备注： 19 pages, 16 figures. Code and data: [this https URL](https://github.com/Luxshan2000/icl-conditional-circuits)
点击查看摘要
Abstract:When a language model follows an in-context conditional rule such as "if P(x) then A else B," does it assemble a runtime circuit with one module that tests the predicate and another that routes the answer? We probe this with activation patching under a four-donor design whose two swapped-rule donors make the condition and the answer word disagree, so each layer reveals which of the two it carries. Across three open models from two families and six languages sharing one fixed item bank, a mid-stack residual band carries the predicate's truth value: patching it reroutes the answer with predicate-outcome flip near 1.0 and mapping flip near 0.0, meeting a strict pre-specified isolation criterion in 17 of 18 cells, and the same localization holds across five predicate families. The router shows the opposite profile. A learned subspace flips A and B near-perfectly within the trained pair yet transfers to a new pair at approximately 0 in every model, while in Gemma-3-4B (the only model probed cross-lingually) it transfers at approximately 0.98 to the same pair in other languages. Under every probe we ran, the router direction is token-bound and non-transferable (largely answer-readout in Gemma, pair-specific in Qwen) rather than an abstract routing module. Test is modular; under these probes, route is not.
92. 【2608.04170】Visualizing Graph-to-Answer Mechanism Recovery in Materials-Science Hypothesis Generation
链接：https://arxiv.org/abs/2608.04170
作者：Shashwat Sourav,Subhadeep Pal,Markus J. Buehler,Sanjay Das,Fiona Y. Wang,Dominik Soos,Tirthankar Ghosal
类目：Computation and Language (cs.CL); Artificial Intelligence (cs.AI)
关键词：fluent materials-science hypotheses, generate fluent materials-science, scientifically meaningful mechanism, co-scientists can generate, generate fluent
备注：
点击查看摘要
Abstract:AI co-scientists can generate fluent materials-science hypotheses, but fluency does not show that an answer preserves a scientifically meaningful mechanism. We present a graph-to-answer mechanism-tracing case study for Graph-PRefLexOR-8B, a Qwen3-8B model adapted to expose distinct stages for brainstorming, graph construction, pattern extraction, and synthesis. We organize semantic backtracking, graph corruption, activation-based recovery measurements, and layer-by-token-region grids into a visual diagnostic workflow for inspecting this pathway. Across 100 open-ended materials-science questions, final answers remain closest to the model's own structured stages, especially synthesis. Under graph corruption, a full sweep over 37 residual-stream checkpoints, the embedding output and 36 transformer blocks, shows little mechanism recovery in the earlier transition region at layers 7--10, recovery instead concentrates in late synthesis and answer-start regions around layers 30 and 36. The workflow is intended to help scientists and model developers identify where a generated hypothesis loses or regains mechanism support before it is passed to downstream experimental planning.
93. 【2608.04160】Mind the Cap: Output-Budget Regimes Change the Measured Multilingual Reasoning Gap
链接：https://arxiv.org/abs/2608.04160
作者：Ankit Goyal,Jaideep Ray
类目：Computation and Language (cs.CL); Machine Learning (cs.LG)
关键词：Multilingual evaluations report, Multilingual evaluations, hidden experimental variable, hidden experimental, points
备注： 15 pages, 2 figures, 11 tables. Under review
点击查看摘要
Abstract:Multilingual evaluations report accuracy at a single output-token cap, but languages need different numbers of tokens to express the same content, so the cap is a hidden experimental variable. We test whether the native-vs-translate gap on MGSM (German, Thai, Swahili) is a token-budget artifact for Qwen3-8B and Llama-3.1-8B-Instruct under four prompting strategies. The measured gap swings by up to 57 points across budgets, length normalization moves it by up to 38.9 points where the cap binds, and at tight caps normalization can reverse which strategy scores higher. We prospectively froze the sweep's three Qwen peaks and its near-zero value at 1024 and evaluated them on 540,000 independently hard-capped decodes: a second frozen family of six Holm-corrected tests rejects every null. The frozen test at $B^*=1024$ still fails to reject because native accuracy has already saturated there; above saturation, the residual difference is a strategy-performance gap, not an identified reasoning deficit. The same truncation channel prices a cost-ordered adaptation ladder: a cross-fitted Thai vocabulary extension closes 0.0 points of the gap at the frozen budget and 4.9 points where 19% of traces still truncate. A third frozen family varies only the announced budget at a fixed enforced cap; announcing 128 rather than 2048 tokens moves Thai native accuracy by 5.1 points, so accuracy is not a function of the enforced cap alone. A correct-emission timing identity computed from one long-cap run matches the three pre-specified MGSM peaks to 0.65 points and, in an exploratory Qwen-only analysis of three further benchmarks, tracks held-out items to 0.92 points, locating the peak exactly in five of seven cells. Treat the output cap as an independent variable and report accuracy across the budget regime, not at a single budget.
94. 【2608.04144】Neighborhood-Aware Dual Biomedical Entity Linking
链接：https://arxiv.org/abs/2608.04144
作者：Yicheng Tao,Jie Liu
类目：Information Retrieval (cs.IR); Computation and Language (cs.CL); Machine Learning (cs.LG)
关键词：curated knowledge base, Biomedical entity linking, supports downstream applications, literature-scale information extraction, linking grounds mentions
备注：
点击查看摘要
Abstract:Biomedical entity linking grounds mentions in clinical and scientific text to entities in a curated knowledge base (KB) with ontological structure, which supports downstream applications such as literature-scale information extraction and patient-record normalization. The task has several challenges at once: the KB contains large numbers of entities, mentions are often ambiguous, and gold labels follow annotation conventions specific to each corpus. To address these challenges, we propose PILOT, a three-stage framework made up of neighborhood-aware retrieval, dual reranking, and score fusion. The retriever injects ontological structure from both the query and KB side, by reformulating mentions and pooling entity embeddings. The retrieved pool is then scored from two complementary views, one over surface forms and one over context, and fused together. PILOT achieves the state of the art on average across five widely-used benchmarks and remains efficient at inference.
95. 【2608.04111】GEB-Bench: Abstract Structures Told in Many Voices
链接：https://arxiv.org/abs/2608.04111
作者：Tong Zhang,Zhiyuan Shi,Yun Peng,Tao Xie
类目：Computer Vision and Pattern Recognition (cs.CV); Computation and Language (cs.CL); Logic in Computer Science (cs.LO)
关键词：river delta, lightning bolt, abstract structural motif, Mobius twist, spirit of Godel
备注：
点击查看摘要
Abstract:Can a model look at a river delta and a lightning bolt and see that they share a structure? We introduce GEB-Bench, a benchmark whose unit is an abstract structural motif--self-reference, a strange loop, a Mobius twist--in the spirit of Godel, Escher, Bach. Each motif is told in several voices: a natural scene whose composition is the structure, a folk story whose telling enacts it through a mechanically checkable form device, a mathematical theorem, and a programmatic skeleton; surface parameters are declared nuisance variables and never scored. Motifs, voices, and the structural changes between them form a small cross-modal category, and GEB-Bench's tasks are its questions. Evaluating twelve open and proprietary models, we find that abstraction failure is lawful. The central finding is a gap between recognition and cross-voice mapping: models identify a structure within one voice far better than they carry it across voices; every model pays this tax, and mapping strong enough to narrow it appears only at the frontier tier. Two patterns support it. Errors align more strongly with the designed formal geometry than with measured perceptual geometries, and frontier models from different vendors converge on the same wrong answers; and surface complexity taxes every model that reads structure, with capacity buying headroom rather than immunity. GEB-Bench is fully generative and released with its pipeline.
96. 【2608.04095】FinPerMA: A Theory-Informed, Event-Grounded Personalized-Memory Benchmark for LLM Agents
链接：https://arxiv.org/abs/2608.04095
作者：Ben Wang,Kang Zhou,Lifan Guo,Feng Chen,Chi Zhang
类目：Artificial Intelligence (cs.AI); Computation and Language (cs.CL)
关键词：Large language model, Large language, financial advising, long horizons, assistants in high-stakes
备注：
点击查看摘要
Abstract:Large language model (LLM) agents are increasingly used as personalized assistants in high-stakes domains such as financial advising, yet it remains unclear whether they can maintain and update an individualized user model over long horizons. Existing personalized-memory benchmarks primarily test factual retention or rely on weakly constrained model-generated trajectories, leaving event-driven preference adaptation underexplored. We introduce FinPerMA, an event-grounded benchmark that evaluates personalized memory against frozen longitudinal investor trajectories. Its generation pipeline combines deterministic, theory-informed impact rules, controlled LLM narration, and automated quality screening; a Post-Shock checkpoint isolates whether an agent has integrated a material event into its persistent user model. On 2,994 questions from 276 personas, seven frontier LLMs and up to seven memory configurations remain far from saturated: no full-context configuration exceeds approximately 0.47 overall accuracy or approximately 39% on multiple-choice questions. Attribution analysis shows that summary-based memory often preserves factual details while losing the preference signals needed for personalization; simple retrieval can therefore outperform purpose-built memory systems, with the gap widening after shocks.
97. 【2608.04084】SpecDrop: Parameter-Free Category-Conditioned Routing for Modular Specialization
链接：https://arxiv.org/abs/2608.04084
作者：Boyao Wang,Zhihan Lei
类目：Machine Learning (cs.LG); Computation and Language (cs.CL); Computer Vision and Pattern Recognition (cs.CV)
关键词：networks pursue specialization, budgets learned routers, total-parameter budgets learned, underperform equal-weight No-Routing, learned routers
备注： 35 pages, 6 figures. Code: [this https URL](https://github.com/Beryex/SpecDrop)
点击查看摘要
Abstract:Mixture-of-experts (MoE) networks pursue specialization through learned routers, gates, and load-balancing losses, yet at matched total-parameter budgets learned routers can underperform equal-weight No-Routing baselines. Is the bottleneck the routing algorithm, or the alignment between training-signal granularity and the target categories? We probe the question with SpecDrop, a fixed parameter-free routing scheme: each of $K$ branches receives weight $p_a$ for its assigned category and a small leakage $p_i 0$ otherwise, merged through a category-independent fixed denominator, with no learned routing parameters and no auxiliary losses; the category label is required at inference. On vision tasks where each image has one superclass label (CIFAR-100 on ResNet-110; ImageNet-1K on ViT-S/16), SpecDrop reaches 79.23% on CIFAR-100 and 79.89% on ImageNet-1K, exceeding parameter-matched baselines that do not use the label (+4.75 over dense on CIFAR-100; +6.53 over the No-Routing+SE control on ImageNet-1K). These gains quantify what category supervision buys when deployed through routing -- not an advantage over label-aware deployments of the baselines: given the same label, masking a dense model's outputs is stronger for accuracy alone (85.2 / 83.7). SpecDrop's contribution is converting the label into trained-in modular structure: 58%/100% branch-category alignment, and masking gains of 0.00 (CIFAR) / +1.06 (ImageNet) -- the output-space restriction is largely internalized during training. On fuzzy partitions, where training units span multiple categories (SlimPajama-6B language modeling with a 30M Transformer; SuperNI instruction tuning over Llama-3.2-1B with LoRA), the routing mechanism reduces to the matched No-Routing controls within seed noise, the null our thesis predicts. Granularity alignment, not algorithm choice, localizes when routing helps. Code: this https URL
98. 【2608.04077】FinProBench: Evaluating Financial AI Agents with Role-Grounded Rubrics Derived from Professional Deliverables
链接：https://arxiv.org/abs/2608.04077
作者：Ben Wang,Kang Zhou,Lifan Guo,Feng Chen,Chi Zhang
类目：Artificial Intelligence (cs.AI); Computation and Language (cs.CL)
关键词：agents requires criteria, requires criteria aligned, real professional work, Evaluating financial, agents requires
备注：
点击查看摘要
Abstract:Evaluating financial AI agents requires criteria aligned with real professional work. Existing rubric methods typically derive criteria from task prompts or model outputs, overlooking tacit standards visible only in practitioner deliverables. We introduce FinProBench, a benchmark for professional financial tasks, and Role-Grounded Rubric Construction (RGRC), a reusable pipeline that derives rubrics from deliverables produced by practitioners in the same role. RGRC comprises four stages: Deliverable Collection, Competency Extraction, Rubric Synthesis, and Validation. Its rubrics capture tacit standards, distinguish quality levels, and transfer across tasks within a role. Before analysis, we classified 57 occupations by deliverable genre into 30 prior-rich conventional roles and 27 prior-sparse role-specialized roles. Across all roles, Prompt-only nearly matches RGRC for conventional roles (89.2% vs. 90.7%), but RGRC substantially outperforms it for role-specialized roles (99.1% vs. 78.0%). This split indicates that prompt engineering can approximate rubrics when conventions are well represented in model priors, while professional grounding is essential for standards beyond those priors. FinProBench is built from 1,723 curated deliverables spanning 57 occupations, 8 financial sub-industries, and 161 deliverable types, and releases an initial evaluation set of 20 complete tasks covering 20 roles in 7 sub-industries. With heterogeneous LLM judges and role-level rubrics, human deliverables rank first on average (73.7 vs. 70.3, 70.2, and 69.6 out of 100), while all four systems show overlapping 95% confidence intervals and complementary strengths. Reusing rubrics at the role level reduces estimated per-task construction effort by 6.7 times relative to authoring each rubric from scratch.
99. 【2608.04056】Learning Sexism Detection Using Multi-Agent Perspectivist Preference Optimization
链接：https://arxiv.org/abs/2608.04056
作者：Hadi Mohammadi,Tina Shahedi,Robert A. Bagheri,Mehdi Dastani,Masoume M. Raeissi
类目：Computation and Language (cs.CL); Computers and Society (cs.CY); Machine Learning (cs.LG)
关键词：perceive sexism differently, genuinely perceive sexism, sexism differently, people label text, perceive sexism
备注： 17 pages, 12 figures, 14 tables. Preprint; under review at EACL 2027 (ACL Rolling Review, August 2026 cycle). Code and data: [this https URL](https://github.com/mohammadi-hadi/MAP-PO)
点击查看摘要
Abstract:When people label text for sexism, they often disagree, and not because some of them are wrong: they genuinely perceive sexism differently. Most NLP systems discard this disagreement by collapsing it into a majority vote. We propose the Multi-Agent Perspectivist Preference Optimization (MAP-PO) framework to keep these different perspectives. On the EXIST 2024 dataset of labeled English and Spanish tweets, we first cluster annotators by their labeling behavior rather than their demographic attributes. We then fine-tune one Large Language Model agent per cluster to reproduce that cluster's annotation behavior, and coordinate the agents with preference optimization that combines individual and team-level rewards. We evaluate MAP-PO in four settings defined by two languages and two backbone language models, asking whether each agent reproduces the annotations of its own cluster and whether the agents together reproduce the majority label. Two findings hold in all four settings. First, without fine-tuning the agents behave almost identically, so cluster-specific training is necessary. Second, we show that training each agent only on the labels of its own cluster pushes the agents far beyond the clusters they should represent, while adding a shared team-level training signal consistently keeps each agent calibrated to its cluster.
100. 【2608.04054】Modality Agreement- and Conflict-Aware Prototype Hypergraph Learning for Multimodal Intent Understanding
链接：https://arxiv.org/abs/2608.04054
作者：Mohnish Raj,Suraj Kumar,Soumi Chattopadhayay,Chandranath Adak,Ayan Dutta
类目：Multimedia (cs.MM); Artificial Intelligence (cs.AI); Computation and Language (cs.CL); Machine Learning (cs.LG)
关键词：visual signals share, intent recognition requires, recognition requires understanding, Multimodal intent recognition, signals share
备注：
点击查看摘要
Abstract:Multimodal intent recognition requires understanding not only what textual, acoustic, and visual signals share, but also how they disagree. Such disagreement is frequently class-informative; for example, lexical positivity accompanied by incongruent vocal or facial behavior may indicate sarcasm or taunting, yet most fusion methods either encourage modality alignment or treat inconsistency as uncertainty to be suppressed. We propose MACH (Modality Agreement- and Conflict-aware prototype Hypergraph), a hierarchical prototype-hypergraph framework that represents multimodal agreement and conflict as distinct, recurring relational structures. MACH progressively composes unimodal representations into bimodal and trimodal abstractions. At each applicable level, modality-composition anchors activate sparse agreement prototype hypergraphs that capture reusable consensus patterns, while a separate conflict pathway maps cross-modal discrepancies to dedicated conflict prototype hypergraphs. The two pathways are combined through a feature-wise, sample-adaptive arbitration mechanism, enabling the model to preserve informative disagreement while suppressing incidental modality noise. A progressive optimization strategy stabilizes the interdependent hierarchy before joint agreement-conflict learning. Experiments on benchmark datasets demonstrate the effectiveness of the proposed formulation, while component and robustness analyses validate the distinct roles of hierarchical composition, prototype-mediated semantic refinement, and agreement-conflict arbitration.
101. 【2608.04037】Reconstructing Persistent Worlds from Narratives for Narrative-Grounded Interactive Experiences
链接：https://arxiv.org/abs/2608.04037
作者：Yi-Chun Chen
类目：Computation and Language (cs.CL); Artificial Intelligence (cs.AI); Graphics (cs.GR); Human-Computer Interaction (cs.HC)
关键词：Designing narrative-grounded interactive, underlying world implied, experiences remains labor-intensive, Designing narrative-grounded, interactive experiences remains
备注：
点击查看摘要
Abstract:Designing narrative-grounded interactive experiences remains labor-intensive because interactive content must align with the underlying world implied by the narrative. Existing approaches formulate problems such as narrative planning, scene generation, and gameplay generation, each constructing computational representations tailored to specific downstream tasks rather than explicitly reconstructing and maintaining the persistent world that grounds them. We investigate reconstructing explicit persistent worlds from narrative descriptions as the central computational objective for narrative-grounded interactive realization. Rather than treating the world as an implicit by-product of downstream generation, our approach reconstructs and maintains persistent entities, locations, semantic relationships, and evolving world states while inferring only the contextual information required to support coherent interactive experiences. To investigate this perspective, we develop a reference prototype that reconstructs structured persistent world representations from narrative descriptions and subsequently instantiates playable tile-based environments. Through three representative case studies spanning a procedural scenario, an original fantasy narrative, and an adapted public-domain story, we demonstrate the feasibility of reconstructing persistent worlds and show how a shared world representation supports coherent gameplay while remaining grounded in the source narrative. By explicitly reconstructing persistent worlds prior to interactive realization, this work bridges computational narrative understanding and interactive content generation, providing a semantic foundation for AI-assisted game authoring, mixed-initiative design, educational simulations, and narrative-grounded interactive experiences.
Subjects:
Computation and Language (cs.CL); Artificial Intelligence (cs.AI); Graphics (cs.GR); Human-Computer Interaction (cs.HC)
Cite as:
arXiv:2608.04037 [cs.CL]
(or
arXiv:2608.04037v1 [cs.CL] for this version)
https://doi.org/10.48550/arXiv.2608.04037
Focus to learn more
arXiv-issued DOI via DataCite
Submission history From: Yi-Chun Chen [view email] [v1]
Mon, 3 Aug 2026 20:06:13 UTC (2,391 KB)
102. 【2608.04021】When More Becomes Less: Position-Dependent Repetition Effects in Language Models
链接：https://arxiv.org/abs/2608.04021
作者：Han-yu Wang
类目：Computation and Language (cs.CL); Machine Learning (cs.LG)
关键词：target affect prediction, readout slot sits, Cloze-style probes, token appears implicitly, implicitly assume
备注：
点击查看摘要
Abstract:Cloze-style probes that vary how often a target token appears implicitly assume that more copies of a target affect prediction the same way regardless of where the readout slot sits. We show this assumption fails. Our two-probe design holds a repeated-target prefix fixed and varies only the readout position: the adjacent probe places the slot immediately after the repeated block; the displaced probe places it inside a fresh sentence frame. Adjacent repetition behaves as priming intuition predicts: $P(\text{target})$ climbs with $N$ and plateaus. Displaced repetition produces an inverted-U: $P(\text{target})$ rises to an early peak and then declines as more copies are added. The displaced inverted-U shows a per-word drop with bootstrap CI excluding zero in all 13 open-access encoder and decoder models we test, and replicates across Spanish, Chinese, German, and French in 42 of 42 multilingual cells. A six-condition causal ablation isolates the effect to exact lexical repetition rather than length, generic redundancy, or semantic-neighbour exposure. A frame-pragmatics control rules out an artefact of the readout frame. Internally, per-target-token attention falls with $N$ while the total budget assigned to the repeated block grows in causal LMs but not in the masked LM we probe. Probes that vary repetition count cannot treat the readout position as orthogonal to what they measure.
103. 【2608.04015】ransfer Learning for Named Entity Recognition of Classical Latin through LLM Prompting
链接：https://arxiv.org/abs/2608.04015
作者：Callum Chan
类目：Computation and Language (cs.CL)
关键词：Large Language Models, Classical Latin texts, Named Entity Recognition, resources of Classical, breakthroughs of Large
备注：
点击查看摘要
Abstract:With the increase in digitized resources of Classical Latin texts and modern breakthroughs of Large Language Models (LLMs), I contribute to ancient language research by participating in EvaLatin 2026. This paper describes Team uOttawa's system description and results for the Named Entity Recognition (NER) shared task. The task is divided into two subtasks: coarse-grained NER with 11 classes and fine-grained NER with 28 classes, each evaluated under strict and fuzzy regimes. Through prompt engineering of commercial LLMs gemini-2.5-pro and claude-sonnet-4-5, I show that the underrepresented ancient Latin language can take advantage of cross-lingual transfer learning by using advancements made by the wider LLM development community. Overall, the methods discussed in this report demonstrate very strong results, placing first in both NER subtasks and achieving the best scores across all evaluation metrics and regimes among all submissions.
104. 【2608.02829】Wiring Beats Blending: What Transfers Between Transformer Sizes -- and What Doesn't
链接：https://arxiv.org/abs/2608.02829
作者：Ravi Satya Durga Prasad Yenugula
类目：Machine Learning (cs.LG); Artificial Intelligence (cs.AI); Computation and Language (cs.CL)
关键词：Model families train, families train, Model families, pretrained large model, large model
备注： 15 pages, 5 figures. Independent research preprint
点击查看摘要
Abstract:Model families train every size from scratch. Can a pretrained large model be converted into a smaller sibling? We characterize the 1.4B-410M conversion in the Pythia family end-to-end: (i) representations align strongly across sizes (ridge R^2=0.84) while parameters align weakly; (ii) dense weight projection is functionally destructive -- provably not an assembly artifact -- because basis mixing breaks rotary, per-head, GELU, and LayerNorm structure; (iii) after the best-fit linear operator, weight residuals are statistically indistinguishable from noise under shuffle controls; (iv) conversion value therefore lives in initialization. In matched-budget continued pre-training we decompose conversion into two independent levers -- least-squares compensation (function: best zero-shot) and variance-preserving rescale (dynamics: best endpoints). Compensation is a token-efficient, low-budget win rather than a universal one: at 30M tokens it beats the strongest subcloning variant on both a width-reduced pair (84.0 +/- 1.8 vs. 89.7 +/- 3.7, 3/3 seeds) and a held-out depth-reduced pair (109.3 vs. 117.9, 3/3 seeds), reaching a given quality with fewer tokens; at a 33x larger budget the two converge to parity (40.0 vs. 40.0), both far ahead of from-scratch, which transfer initialization always beats -- by up to 18x at low budget, the margin narrowing at convergence and at the largest scale. We further map the method's boundary: at ~5x the donor scale (6.9B-1.4B) stacking both levers over-corrects, which we trace to ill-conditioning of the compensation solve at large width, pointing to dimension-aware regularization as the fix. Code, checkpoints, and the frozen evaluation corpus are released.
105. 【2607.22699】Similarity All The Way Up: Multilingual Generalization in LLMs Relies on Language-Level Similarity Structures
链接：https://arxiv.org/abs/2607.22699
作者：Supantho Rakshit,Adele Goldberg,Henry Conklin
类目：Artificial Intelligence (cs.AI); Computation and Language (cs.CL)
关键词：Large Language Models, generalize remains difficult, grow more capable, diverse tasks, limited domains
备注： 8 pages, 3 figures, submitted to CogSci 2026
点击查看摘要
Abstract:As Large Language Models (LLMs) grow more capable across diverse tasks, their (in)ability to generalize remains difficult to quantify and poorly understood beyond limited domains. In particular, LLMs are known to struggle generalizing multilingually, to languages outside of English, and that are poorly attested in their training data. To understand why this may be, and what enables some models to perform better than others, we turn to a long history of work across the cognitive sciences, arguing that successful generalization derives from appropriate representations in similarity space. We look at how well LLMs' representations capture the hierarchical similarity structure between distinct languages. Strikingly, we show LLMs' latent representations largely recover the hierarchical structure of the Indo-European language family tree -- grouping languages that are members of the same subfamily closely together in representation space. Furthermore, we show that the degree to which models reflect the similarity structure of languages correlates with their performance on XNLI, a multilingual natural language inference benchmark. This extends classic work on similarity-driven generalization at scale, showing how models that represent similar languages similarly generalize better from one language to another.
106. 【2608.05138】aching Nemotron Greek: Mining a Corpus, Adapting Retrieval, and Grounding Generation for Modern Greek across Specialist Domains
链接：https://arxiv.org/abs/2608.05138
作者：Ayoub Kirouane,Christos Petrocheilos
类目：Audio and Speech Processing (eess.AS); Artificial Intelligence (cs.AI); Computation and Language (cs.CL)
关键词：NVIDIA Nemotron retrieval, NVIDIA Nemotron, absent from NVIDIA, Modern Greek, Nemotron retrieval models
备注： 15 pages, 10 figures, 7 tables. Includes release of the HERA benchmark and Sophea Nemo RAG models
点击查看摘要
Abstract:Modern Greek is absent from NVIDIA's Nemotron retrieval models and from major multilingual retrieval benchmarks, despite being important for retrieval-augmented generation (RAG) in legal, energy, financial, and medical applications. We present an end-to-end adaptation of the Nemotron retrieval stack for Modern Greek, including corpus mining, synthetic supervision, retrieval model training, reranker adaptation, reader fine-tuning, and a new benchmark called HERA. Our study shows that a parameter-free BM25 baseline outperforms several off-the-shelf multilingual dense retrieval models on specialist Greek corpora. After fine-tuning on 65,773 Greek retrieval pairs, a Nemotron 1B embedder improves nDCG@10 from 0.362 to 0.835 and substantially outperforms its unadapted counterpart. The learned language competence transfers to general-domain Greek, although the advantage over BM25 remains domain-dependent. We further adapt a cross-encoder reranker and demonstrate consistent improvements across specialist domains. Finally, we LoRA-tune a Nemotron 30B-A3B mixture-of-experts reader for grounded generation, increasing judged answer correctness from 29.4% to 66.9% while significantly improving faithfulness and citation quality. We also introduce HERA, the first large-scale Greek benchmark for retrieval-augmented generation, and release our adapted models and benchmark to support future research on Greek-language RAG systems.
107. 【2305.06177】Leveraging Machine Learning to Gain Insights on Quantum Thermodynamic Entropy
链接：https://arxiv.org/abs/2305.06177
作者：Srinivasa Rao. P
类目：Quantum Physics (quant-ph); Statistical Mechanics (cond-mat.stat-mech); Computation and Language (cs.CL); Machine Learning (cs.LG); Computational Physics (physics.comp-ph)
关键词：Szilard classical single-particle, classically-chaotic Szilard Map, single quantum particle, Szilard Map, quantum
备注： 9 pages, 7 figures
点击查看摘要
Abstract:We present a thermodynamic analysis of a quantum engine that uses a single quantum particle as its working fluid, inspired by Szilard's classical single-particle engine. Our design is modeled after the classically-chaotic Szilard Map and involves a thermodynamic cycle of measurement, thermal-energy extraction, and memory reset. Our focus is on investigating the thermodynamic costs associated with observing and controlling the particle and comparing these costs in the quantum and classical limits. Through our study, we aim to shed light on the thermodynamic trade-offs that arise from Lindauer's Principle for information-processing-induced thermodynamic dissipation in both the quantum and classical regimes. Using machine learning methods, we demonstrate that energy analysis can be performed and the quantum engine can be simulated according to the Szilard engine based Second Law of Thermodynamics in its working condition. However, we note that the quantum engine operates using significantly different mechanisms than its classical counterpart, where the cost of inserting partitions plays a critical role in the quantum implementation.
信息检索
1. 【2608.04843】MemoryCPT: An End-to-End Agent Memory Framework for Cost-Performance Trade-off
链接：https://arxiv.org/abs/2608.04843
作者：Songxin Lei,Kun Ouyang,Weilin Ruan,Yuqian Wu,Zhijiang Guo,Yushi Sun,Fugee Tsung
类目：Information Retrieval (cs.IR)
关键词：Long-horizon LLM agents, large interaction histories, Long-horizon LLM, LLM agents require, require memory systems
备注：
点击查看摘要
Abstract:Long-horizon LLM agents require memory systems that recover useful evidence from large interaction histories without passing excessive context to downstream models. Existing memory pipelines often rely on hand-crafted heuristics and repeated LLM calls, which can introduce redundant context and high inference cost. We propose MemoryCPT, an end-to-end trainable agent memory pipeline that spans offline memory construction and online query-conditioned context generation. MemoryCPT consists of two stages: Query-agnostic Distillation (QAD), which distills a modular memory-construction pipeline into a compact model using explicit reasoning traces; and Query-aware Retrieval and Summarization (QAR), which combines reciprocal rank fusion (RRF) with a LoRA-based summarizer trained via Group Relative Policy Optimization (GRPO) under a cost-aware reward. We further introduce Quality per Cost (QPC) to quantify answer quality per unit inference cost. Experiments on LoCoMo and LongMemEval show that MemoryCPT improves the cost-performance trade-off over the evaluated baselines, while ablation and sensitivity analyses characterize the contributions of its components and the effects of key design choices.
2. 【2608.04809】DEGR: Dual Exploration-Driven Generative Re-Ranking for Adaptive Cross-Request Context Bridging
链接：https://arxiv.org/abs/2608.04809
作者：Binglei Zhao,Xuanhua Yang,Xiwei Zhao,Sulong Xu
类目：Information Retrieval (cs.IR)
关键词：stage balances business, balances business objectives, modeling contextual information, business objectives, re-ranking stage balances
备注： Accepted by KDD2026 ADS Track, 11 pages
点击查看摘要
Abstract:In industrial recommendation systems, the re-ranking stage balances business objectives and diversity for sequence-level optimization while modeling contextual information. However, constrained by fixed upstream supply, existing methods fail to deliver further effectiveness gains, especially under low-quality supply. To overcome this, re-ranking can actively balance immediate and exploratory value, for instance, by prioritizing exploratory exposure under low-quality supply to preserve browsing potential and facilitate serendipitous conversions. Therefore, we propose a Dual Exploration-Driven Generative Re-Ranking (DEGR) method. DEGR adopts a hybrid supervised-reinforcement exploration and optimization paradigm, guided by an exploratory reward model that adaptively balances immediate and exploratory value. The hybrid optimization paradigm integrates three key components: supervised learning, exploration diversity constraint, and adaptive reward-weighted ORPO for preference optimization. Through this dual exploration, the generator ultimately acts as an adaptive cross-request contextual bridge. Offline and online experiments indicate that DEGR outperforms SOTA methods, achieving improvements of up to 1.22% UCTR and 0.20% PV in the JD E-commerce recommendation system.
3. 【2608.04807】WatchLens: A Configurable Platform for Online Video Recommendation Experiments
链接：https://arxiv.org/abs/2608.04807
作者：Deogyong Kim,Dongha Lee
类目：Information Retrieval (cs.IR)
关键词：requires online experiments, experiments that link, conditions that produced, user behavior requires, link playback behavior
备注： 6 pages, 3 figures. Accepted to RecSys 2026
点击查看摘要
Abstract:Studying how video recommender systems shape user behavior requires online experiments that link playback behavior with the recommendation conditions that produced it. Existing user-study infrastructure provides one or the other, but not both within a single experimentation workflow. We present WatchLens, an open-source platform that fills this gap. WatchLens adopts a modular architecture in which user interfaces, content sources, and recommendation policies are independently configurable, with policies assignable separately to the feed and the watch page, while a standardized logging layer attaches the recommendation policy and ranking position to every event at recording time. This design enables researchers to analyze how recommendation policies and ranking positions shape downstream playback behavior, session continuation, and navigation between the feed and the watch page, with the linkage between policy and outcome available in each event rather than reconstructed afterwards. We demonstrate WatchLens with a short-form video case study that holds the interface, feed policy, and content pool constant while varying only the watch-page policy, showing how the platform supports session-level comparison of recommendation effects on real viewing behavior. WatchLens is released as a publicly available, single-server deployable system for reproducible online video recommendation research.
4. 【2608.04746】Caching for the Future: Scrub Jay Episodic Memory Principles for Agent Memory Systems
链接：https://arxiv.org/abs/2608.04746
作者：Kartikey Singh Bhandari,Aarya Wadhwani,Dhruv Kumar,Pratik Narang
类目：Computation and Language (cs.CL); Information Retrieval (cs.IR)
关键词：accumulate stored memories, sessions accumulate stored, validity varies enormously, systematically contaminate retrieved, contaminate retrieved context
备注：
点击查看摘要
Abstract:LLM agents that persist across sessions accumulate stored memories whose validity varies enormously by content type, yet existing memory architectures treat all memories as equally persistent and systematically contaminate retrieved context with outdated facts. We show that per-memory, type-conditioned temporal decay, a property of western scrub jay episodic memory, can be operationalized as an auto-classified coefficient $\pi_i$ in an external LLM-agent memory store, yielding ScrubJay-MEM: each memory is encoded as a jointly-bound What--Where--When tuple with an estimated perishability $\pi_i$ and utility horizon $\tau_i$, retrieved by query-adaptive scoring, and revised retroactively at $O(1)$ LLM calls per update. We introduce the Temporal Generalization Test (TGT), a benchmark with held-out retention intervals and a Generalization Gap (GenGap) metric. On TGT, ScrubJay-MEM is the only retrieval-based system with substantially positive GenGap ($+0.108$); on MemoryAgentBench EventQA-64k it improves F1 by $+2.66$ over Mem0 and $+3.09$ over Qwen3-Embedding-4B under a llm backbone. A decay ablation collapses GenGap by $5.7\times$, establishing type-conditioned decay as necessary for the result. Gains narrow under stronger backbones and reverse on fact-consolidation tasks, scoping the contribution to temporal reasoning over perishable facts.
5. 【2608.04609】Characterizing the Evolving Landscape of Modern Information Seeking
链接：https://arxiv.org/abs/2608.04609
作者：Shuoqi Sun
类目：Information Retrieval (cs.IR)
关键词：human IS process, modern, modern IS process, rise of Generative, Abstract
备注： Best Paper Award at FDIA 2026; 2 Pages (Excluding References)
点击查看摘要
Abstract:Information seeking (IS) evolves, as does the human IS process. Since the rise of Generative AI (GenAI), modern IS has shifted by introducing more interfaces, more complex interactions, and expanded system capabilities. We argue that these changes in modern IS should be systematically examined. This PhD research characterizes the changes in the modern IS process. We use mechanisms, including online crowdsourcing survey experiments, theoretical IS frameworks, and in-lab experiments with neurophysiological signals, to characterize the shifts in modern IS, especially those driven by GenAI. We offer insights into the current landscape of search interface preferences and the cognitive efforts involved in seeking information. We believe this PhD research will contribute to and inform future designs of personalized, cognition-aware IS systems.
6. 【2608.04543】owards Robust Version Identification in the Wild: A Dataset, Benchmark, and Fine-Tuning Study
链接：https://arxiv.org/abs/2608.04543
作者：Simon Hachmeier,R. Oguz Araz,Dmitry Bogdanov,Robert Jäschke,Xavier Serra
类目：ound (cs.SD); Information Retrieval (cs.IR)
关键词：professionally recorded tracks, SecondHandSongs and Discogs, musical version identification, curated metadata sources, recorded tracks
备注： Accepted to the Proceedings of the 27th International Society for Music Information Retrieval Conference (ISMIR 2026)
点击查看摘要
Abstract:Existing datasets for musical version identification (VI) are primarily derived from curated metadata sources such as SecondHandSongs and Discogs, and are therefore dominated by professionally recorded tracks. This leads to a domain mismatch with real-world scenarios, where amateur and user-generated content is prevalent. To address this limitation, we introduce DiVers, a large-scale VI dataset comprising over 1.1 million musical versions, with train-validation-test splits compatible with established datasets such as Discogs-VI-YT, SHS100K, and Da-TACOS. In addition to standard version-level annotations, DiVers provides automatically assigned tags (e.g., instrumental, live) and segment-level predictions indicating the presence or absence of music. We evaluate the proposed dataset by training state-of-the-art VI systems. Our results show that models trained on DiVers achieve substantially improved robustness to acoustically diverse and noisy inputs, while maintaining a stable performance on cleaner, studio-quality benchmarks. We release the dataset metadata, code for its construction, and all experimental pipelines to support reproducibility.
7. 【2608.04482】Skills Know Their Neighbors: Cluster-Contrastive Capability Pages for Skill Retrieval
链接：https://arxiv.org/abs/2608.04482
作者：Zifei Wang,Wei Wen,Qiang Ji,Ruizhi Qiao
类目：Information Retrieval (cs.IR)
关键词：skill libraries grow, retrieve reusable skills, large language model, language model agents, libraries grow
备注： 16 pages, 4 figures
点击查看摘要
Abstract:As skill libraries grow, large language model agents must retrieve reusable skills from candidates that often share the same topic and vocabulary but implement different capabilities. Retrieval is limited not only by the scorer but also by the text being scored: a document may describe what a skill does without stating which similar requests should be routed elsewhere. We formalize a skill's capability as its \emph{executable region}, the set of queries it can solve, and view its document as a lossy observation of that region. This view exposes a document-imposed component of retrieval error that cannot be removed by improving the retriever alone. We therefore propose \emph{Capability Pages}, cluster-contrastive skill representations containing a positive trigger $\Tpos$, a negative boundary $\Tneg$, and a discriminative body $B$. An offline compiler compares neighboring skills to write these fields. At inference time, the index uses $\Tpos$ and $B$ for candidate recall, while the router uses $\Tneg$ to reject confusable alternatives. On SRA-Bench, which contains 26{,}262 skills and 5{,}400 questions from six datasets, Capability Pages improve Recall@10 for all five tested retrievers, with a mean gain of $2.94$ points. Adding $\Tneg$ to candidate cards improves end-to-end task success by $3.62$ points on average across four executors and six datasets. A transfer evaluation on Chinese SSL-SkillDiscovery reaches $73.07\%$ MRR@50 using the same encoder across conditions. Capability Pages require no modification to the online models; they improve routing by rewriting the offline skill library.
8. 【2608.04455】Multi-Objective Ranking for Live-Streaming: Balancing Fresh and Delayed Signals with Segment-Aware Targeting
链接：https://arxiv.org/abs/2608.04455
作者：Xiaoyi Gu,Julia Tavares,Eder Santana,Carlos Mendoza-Cardenas,Nikita Mishra,Saad Ali
类目：Information Retrieval (cs.IR); Machine Learning (cs.LG)
关键词：challenging problems entertainment, data exhibits bias, problems entertainment live-streaming, entertainment live-streaming services, live-streaming services face
备注： 9 pages, 3 figures. Accepted to the Industry Track of the 20th ACM Conference on Recommender Systems (RecSys 2026)
点击查看摘要
Abstract:One of the most challenging problems entertainment live-streaming services face in recommendation systems is that user behaviors are sparse and delayed, and interaction data exhibits bias for different user segments. Unlike e-commerce applications where user actions follow linear sequences, live-streaming viewers engage in multiple concurrent behaviors of watching, chatting, following, and spending, each occurring with varying delays. We address these challenges through three key contributions: 1) a delayed window approach that extends feedback collection beyond immediate responses, 2) a multi-model architecture that combines fresh and delayed signals, and a segment-aware targeting module that optimizes ranking scores differently across user lifecycle stages, and 3) Multi-gate Mixture-of-Experts (MMoE) integration that jointly models correlated targets while reducing model parameters by 41.9% compared to independent models. Online A/B testing demonstrates significant improvements, including a +0.09% increase in Daily Active Viewers (DAV), generating millions more annual active viewer days, and +0.56% increase in highly engaged viewers' capped Average Revenue Per User (ARPU). Viewer-segment targeting achieved an additional +0.15% DAV improvement for newer and less engaged viewers, while MMoE enhancement added +0.08% overall DAV and +0.27% new follows. The proposed system processes ranking requests with low latency, providing a scalable approach for balancing multiple business objectives across diverse user populations. In addition, we tested the multi-model architecture on the Twitch mobile live feed and achieved a +1.12% increase in positive user-channel interactions (clicks, follows, and likes), demonstrating applicability beyond the primary use case.
9. 【2608.04432】he Price of Isolation: Estimating the Ecosystem Cost of Symmetric Two-Sided A/B Testing
链接：https://arxiv.org/abs/2608.04432
作者：Yuanyuan Shen,Yiren Yan,Wenjie Li,Chunhui Zhu
类目：Information Retrieval (cs.IR); Machine Learning (cs.LG); Social and Information Networks (cs.SI); Methodology (stat.ME)
关键词：cross-arm marketplace interference, removes cross-arm marketplace, symmetric two-sided isolation, assigning matched fractions, two-sided content platforms
备注：
点击查看摘要
Abstract:On two-sided content platforms, symmetric two-sided isolation (assigning matched fractions of creators and viewers to isolated treatment and control submarkets) is widely used for creator-side and cold-start experiments because it removes cross-arm marketplace interference. Isolation, however, thins each viewer's candidate catalog, and intuition suggests the resulting engagement cost should fade as the platform grows: a small fraction of a vast catalog is still vast. We show that, in an order-statistics model of engagement, whether this intuition holds depends on the upper tail of match quality. Extreme-value theory yields tail-class loss laws with a sharp dichotomy: for light or bounded tails the loss vanishes as the candidate pool grows, whereas under heavy tails it converges to a size-independent constant, so expanding the candidate pool, even by orders of magnitude, does not asymptotically eliminate the cost. Evidence from two production experiments on a platform with millions of active creators is consistent with this picture: a pure A/A traffic sweep reveals a measurable, depth-graded engagement cost; a one-sided catalog ablation independently shows that per-viewer thinning contributes to the loss; and a tail index calibrated on the small exploration pool predicts an effect consistent with the one observed in the far larger full-catalog ablation. Isolation thus carries a price that experimenters should budget for, like any other cost. We give practitioners a preflight procedure that estimates it before launch, sizes traffic accordingly, and recommends a fallback design when the predicted cost exceeds a chosen tolerance.
10. 【2608.04302】CLIP-CC-Bench: Evaluating Paragraph-Level Video Descriptions in Video-Language Models
链接：https://arxiv.org/abs/2608.04302
作者：Mukhtiar Ali,Harsh Dubey,Sugam Mishra,Chulwoo Pack
类目：Computer Vision and Pattern Recognition (cs.CV); Information Retrieval (cs.IR); Multimedia (cs.MM)
关键词：generate accurate long-form, Benchmarking video-language models, Benchmarking video-language, long-form video description, single-sentence metrics
备注： Accepted and presented at EvalMG 2026, the Second Workshop on Evaluation for Multimodal Generation, co-located with ACM SIGIR 2026
点击查看摘要
Abstract:Benchmarking video-language models has largely focused on short clips and single-sentence metrics, leaving open whether current systems can generate accurate long-form, paragraph-level descriptions. We introduce CLIP-CC-Bench, an evaluation suite for long-form video description built from 5 hours of movie content segmented into 90-second clips, each paired with an expert-written paragraph-style reference. The evaluation suite employs an ensemble of five state-of-the-art LLM-based embedding models to increase reliability and mitigate single-model bias, and applies two complementary methodologies: (i) coarse-grained semantic matching and (ii) fine-grained semantic matching to compare model-generated descriptions against CLIP-CC-Bench references. Using this framework, we evaluate 17 state-of-the-art video-language models and report both their Borda-aggregated rankings and their average scores on CLIP-CC-Bench. We further quantify the protocol's internal reliability through inter-judge agreement and bootstrap ranking stability. We release standardized evaluation scripts, model outputs, and aggregation tools at this https URL to support reproducibility. CLIP-CC-Bench provides a practical evaluation framework for long-form video description, filling a gap left by existing short-clip and QA-only benchmarks.
11. 【2608.04144】Neighborhood-Aware Dual Biomedical Entity Linking
链接：https://arxiv.org/abs/2608.04144
作者：Yicheng Tao,Jie Liu
类目：Information Retrieval (cs.IR); Computation and Language (cs.CL); Machine Learning (cs.LG)
关键词：curated knowledge base, Biomedical entity linking, supports downstream applications, literature-scale information extraction, linking grounds mentions
备注：
点击查看摘要
Abstract:Biomedical entity linking grounds mentions in clinical and scientific text to entities in a curated knowledge base (KB) with ontological structure, which supports downstream applications such as literature-scale information extraction and patient-record normalization. The task has several challenges at once: the KB contains large numbers of entities, mentions are often ambiguous, and gold labels follow annotation conventions specific to each corpus. To address these challenges, we propose PILOT, a three-stage framework made up of neighborhood-aware retrieval, dual reranking, and score fusion. The retriever injects ontological structure from both the query and KB side, by reformulating mentions and pooling entity embeddings. The retrieved pool is then scored from two complementary views, one over surface forms and one over context, and fused together. PILOT achieves the state of the art on average across five widely-used benchmarks and remains efficient at inference.
12. 【2608.03487】RAG-Stack: Co-Optimizing RAG Serving Performance and Quality
链接：https://arxiv.org/abs/2608.03487
作者：Haiqiang Zhang,Yuanqing Lei,Wanting Li,Tao Zhang,Wenqi Jiang
类目：Databases (cs.DB); Artificial Intelligence (cs.AI); Information Retrieval (cs.IR)
关键词：augments large language, Retrieval-augmented generation, large language model, retrieved from databases, augments large
备注：
点击查看摘要
Abstract:Retrieval-augmented generation (RAG), which augments large language model (LLM) generation with information retrieved from databases, has become a widely used approach for knowledge-intensive applications. Modern RAG systems, however, expose many configuration choices, such as retrieval indexes, model selections, and how models invoke retrieval. Each configuration yields a different trade-off between answer quality and serving performance, making it challenging to choose the optimal setting for a specific application deployment. We present RAG-Stack, a framework for efficiently discovering quality-performance Pareto frontiers across diverse RAG applications and serving systems. RAG-Stack consists of RAG-PE, an iterative design-space exploration algorithm that selects the next RAG configuration to evaluate; RAG-IR, a workload abstraction for diverse RAG algorithms; and RAG-CM, a performance model that predicts the optimal deployment and serving performance on the given hardware. Together, these components allow RAG-Stack to search the joint algorithm-system configuration space without deploying every candidate and to transfer an existing Pareto frontier to a new serving system. Given the same number of optimization iterations across diverse datasets, the Pareto frontiers found by RAG-Stack cover 52.5% to 153.2% more of the normalized quality-performance space than those found by state-of-the-art configuration-search methods evaluated over the same RAG design space.
13. 【1706.02795】A Deep Causal Inference Approach to Measuring the Effects of Forming Group Loans in Online Non-profit Microfinance Platform
链接：https://arxiv.org/abs/1706.02795
作者：Thai T. Pham,Yuanyuan Shen
类目：Machine Learning (stat.ML); Information Retrieval (cs.IR); General Finance (q-fin.GN)
关键词：non-profit crowdsouring microfinance, crowdsouring microfinance platform, online non-profit crowdsouring, non-profit crowdsouring, crowdsouring microfinance
备注：
点击查看摘要
Abstract:Kiva is an online non-profit crowdsouring microfinance platform that raises funds for the poor in the third world. The borrowers on Kiva are small business owners and individuals in urgent need of money. To raise funds as fast as possible, they have the option to form groups and post loan requests in the name of their groups. While it is generally believed that group loans pose less risk for investors than individual loans do, we study whether this is the case in a philanthropic online marketplace. In particular, we measure the effect of group loans on funding time while controlling for the loan sizes and other factors. Because loan descriptions (in the form of texts) play an important role in lenders' decision process on Kiva, we make use of this information through deep learning in natural language processing. In this aspect, this is the first paper that uses one of the most advanced deep learning techniques to deal with unstructured data in a way that can take advantage of its superior prediction power to answer causal questions. We find that on average, forming group loans speeds up the funding time by about 3.3 days.
计算机视觉
1. 【2608.05149】CoCo-IR: Contextual Composed Image Retrieval
链接：https://arxiv.org/abs/2608.05149
作者：Shengcao Cao,Tanmaya Shekhar Dabral,Zhongli Ding,Madhuri Shanbhogue,Kaifeng Chen,Zhe Li,Mojtaba Seyedhosseini,Yu-Xiong Wang,Liang-Yan Gui
类目：Computer Vision and Pattern Recognition (cs.CV)
关键词：Current instruction-based image, real-world visual searches, instruction-based image retrieval, image retrieval systems, Composed Image Retrieval
备注： ECCV 2026
点击查看摘要
Abstract:Current instruction-based image retrieval systems are powerful but limited to single-turn interactions, failing to capture the iterative nature of complex, real-world visual searches. To overcome this limitation, we introduce Contextual Composed Image Retrieval (CoCo-IR), a novel task that enables users to progressively refine search results through interactions. We address this new task by proposing a new model based on a Large Multimodal Model (LMM) that functions as a context-aware reasoner for CoCo-IR. Our model interprets the entire interaction history to generate Transformable Image Embeddings (TIE) that evolve across turns. To fuel the model training without expensive human annotations, we develop a fully autonomous, scalable data engine that leverages LMMs to generate high-quality contextual retrieval data, and uses model-guided verification to mine challenging hard negatives. Extensive experiments demonstrate that our approach establishes new state-of-the-art performance: We achieve 39.4 mAP@5 on the challenging single-turn benchmark CIRCO; furthermore, on our new CoCo-IR benchmark, our model maintains robust performance with 44.1 R@1 on 4-turn dialogues, dramatically outperforming existing methods (28.2 4-turn R@1) that fail to handle multi-turn context. Project page: this https URL.
2. 【2608.05145】Objects as Audio-Visual Modal Sound Fields
链接：https://arxiv.org/abs/2608.05145
作者：Zisen Shao,Zihao Wei,Derong Jin,Ruohan Gao
类目：Computer Vision and Pattern Recognition (cs.CV)
关键词：rich acoustic cues, acoustic cues revealed, modeling object geometry, geometry and appearance, physical interaction
备注： ECCV 2026, Project page: $\href{ [this https URL](https://zisenshao.github.io/AV-MSF/) }{\text{this https URL}}$
点击查看摘要
Abstract:While modern 3D reconstruction excels at modeling object geometry and appearance, it largely ignores the rich acoustic cues revealed through physical interaction. Object impact sounds convey material, stiffness, and structural properties that complement vision, yet existing impact sound modeling approaches either rely on expensive physics-based simulation or require large datasets to generalize in a purely data-driven manner. We introduce Audio-Visual Modal Sound Field (AV-MSF), a novel object-level acoustic representation reconstructed from multi-view images and only a few impact sound recordings. AV-MSF builds on 3D Gaussian Splatting integrated with dense 3D visual feature to provide a strong geometry-aware prior, and represents the impact sound field using compact, physically meaningful modal parameters, enabling robust few-shot reconstruction. Experiments on two real-world datasets show that AV-MSF achieves state-of-the-art impact sound rendering, outperforming both physics-based and data-driven baselines. Furthermore, we demonstrate downstream applications enabled by our representation, including contact localization and object sound editing.
3. 【2608.05137】SmartMage: Dynamic Modality Orchestration for 3D Scene Understanding
链接：https://arxiv.org/abs/2608.05137
作者：Yue Zhang,Yingzhao Jian,Yunqiu Xu,Xiaoxiao Sun,Hehe Fan
类目：Computer Vision and Pattern Recognition (cs.CV)
关键词：requiring joint reasoning, Large Language Models, embodied intelligence, requiring joint, including visual
备注：
点击查看摘要
Abstract:Understanding 3D scenes is fundamental to embodied intelligence, requiring joint reasoning over heterogeneous information from multiple modalities, including visual and geometric cues. However, the relevance of these modalities often varies across queries. Existing Multimodal Large Language Models (MLLMs) typically rely on fixed modality combinations, overlooking query-dependent modality needs. Such a rigid design can introduce semantic noise from irrelevant modalities while underutilizing more informative ones, leading to wasted computation and diluted reasoning. To address these challenges, this paper proposes SmartMage, a unified MLLM that dynamically orchestrates heterogeneous modalities for semantic-aware 3D scene understanding. Specifically, SmartMage incorporates: (1) a Semantic-guided Modality Adaptive RouTng (SMART) module that selects task-relevant modalities using semantic priors, text-modality alignment, and modality quality; and (2) a Modality-Aware Gating Expert (MAGE) module that leverages modality priors to guide expert activation, fostering adaptive specialization in multimodal reasoning. Empirically, SmartMage achieves state-of-the-art performance across five 3D scene understanding benchmarks, and attains competitive results on RGB-only video understanding benchmarks. In our diagnostic benchmark ScanFacet, tasks are divided into fine-grained semantic categories, enabling analysis of modality combinations preferred by each semantic type. The observed modality-semantic patterns provide further evidence of SmartMage's effectiveness. Project page: this https URL.
4. 【2608.05132】Predicting Brain Morphometry with MT-GNN: Mesh Evolution in Continuous Time with Graph-Based Metric Tensor Embeddings
链接：https://arxiv.org/abs/2608.05132
作者：Hao Ding,Daniel Semchin,Paul M. Thompson,Boris Gutman
类目：Computer Vision and Pattern Recognition (cs.CV); Machine Learning (cs.LG)
关键词：clinical-trial enrichment, prior scans, scans could support, support prognosis, prognosis and clinical-trial
备注：
点击查看摘要
Abstract:Predicting how a subcortical structure's shape will evolve from a few prior scans could support prognosis and clinical-trial enrichment. Existing longitudinal mesh predictors either extrapolate shape trajectories via high-dimensional embeddings or regress vertex deformations directly. We instead predict the surface's intrinsic geometry in continuous time: a single per-structure graph network predicts the future per-vertex first fundamental form (metric tensor) for an arbitrary causal multiple-visit history and an arbitrary prediction horizon, conditioned on a Fourier encoding of the lead time. The predicted metric is decoded into a surface by a differentiable As-Rigid-As-Possible solver, and the model is trained end-to-end on the rigid-aligned vertex error. Training through the reconstruction keeps the decoded prediction a valid surface and consistently improves it. On 14 subcortical structures from the ADNI dataset, the proposed mesh evolution model (MT-GNN) predicts best among the evaluated methods at every horizon ($-2.29\%$ mean vertex error vs. the temporal mean, $p{=}6.1{\times}10^{-5}$, beating it on 14/14 structures), ahead of geodesic shape regression (DCM, $-0.19\%$) and a mesh transformer (TransforMesh, $-0.45\%$; $p{=}1.2{\times}10^{-4}$), with the lead widening as the horizon grows.
5. 【2608.05131】OPD-V: Visual On-Policy Self-Distillation with Modality Balance
链接：https://arxiv.org/abs/2608.05131
作者：Aniri,Jinhe Bi,Peng Liao,Zengjie Jin,Volker Tresp,Fei Shen,Yunpu Ma,Tat-Seng Chua
类目：Computer Vision and Pattern Recognition (cs.CV); Artificial Intelligence (cs.AI)
关键词：multimodal large language, large language models, standard post-training approach, Modality Imbalance, approach for improving
备注：
点击查看摘要
Abstract:On-Policy Self-Distillation (OPSD) has become a standard post-training approach for improving visual reasoning in multimodal large language models (MLLMs). Existing methods draw privileged information from diverse input sources to guide self-distillation. Yet these designs overlook Modality Imbalance, a challenge inherent to MLLM reasoning. When textual information dominates generation, the model cannot fully integrate its multimodal input. Consequently, carefully designed privileged information remains underused, limiting the effectiveness of OPSD. To examine this limitation, we construct a Positive Teacher with the Zoom-In Image and a Negative Teacher with the Mask Image, which exhibit different degrees of Modality Imbalance. Changes in their reasoning correctness and token logits reveal that Modality Balance can itself serve as privileged information. Motivated by this finding, we introduce OPD-V, a visual OPSD paradigm that instantiates such information through the Positive Teacher and Negative Teacher. Positive Modality-Balance Logits Margins define a Modality-Balance Trust Region that selects the on-policy tokens used for self-distillation. Experiments across 6 benchmarks, 4 MLLM backbones, and 5 post-training methods show that OPD-V consistently improves reasoning performance while reducing training cost.
6. 【2608.05122】IRIS: A Visual Cortex-Inspired Framework for Analyzing Orientation Selectivity in Vision Transformers
链接：https://arxiv.org/abs/2608.05122
作者：Vaishnavi B Mohan,Vijayakrishna Naganoor,Yashas Annadani,Shashank Hegde
类目：Computer Vision and Pattern Recognition (cs.CV)
关键词：Vision transformers, facto standard, standard for image, Vision, features
备注：
点击查看摘要
Abstract:Vision transformers (ViTs) have become the de facto standard for image encoding across many perception tasks. Despite their empirical success, it remains mechanistically unclear how they encode low-level features, given their lack of inductive biases: ViTs process information globally rather than relying on local structure. Biological visual systems, in contrast, build low-level features, such as orientation selectivity in the primary visual cortex, by combining information from small, localized regions of the visual field. These features are general-purpose representations, shared and required across multiple specialized neural pathways, unlike higher-level, task-specific semantic features. This raises the question if such biologically-grounded features arise in ViTs. In this work, we systematically study how orientation selectivity emerges in ViTs by introducing a suite of neuroscience-inspired metrics: representational similarity score (RSS), orientation recruitment score (ORS), and orientation tuning bandwidth to quantify how orientation is encoded in representational geometry and as a function of model depth. Through extensive analysis, we find that: (1) the training paradigm is the strongest determinant of orientation selectivity, with models sharing an objective, peaking at comparable relative depths regardless of scale (2) many units are orientation-selective early in training, with early-to-middle layers recruiting more such units over time, while deeper layers lose selectivity and broaden their tuning toward semantic encoding and (3) our metrics offer a mechanistic heuristic for how many layers to unfreeze for best downstream generalization. Our framework presents a way to track biologically-grounded features during ViT training, probes how desired properties are encoded in transformer representations, and builds a systematic understanding of how ViTs generalize across tasks.
7. 【2608.05115】Robust and Efficient Motion Reasoning for Privacy-Aware Classroom Incident Recognition
链接：https://arxiv.org/abs/2608.05115
作者：Paritosh Parmar,Landy Lan,Hong Yang,Chen Yi,Chiat Pin Tay
类目：Computer Vision and Pattern Recognition (cs.CV); Artificial Intelligence (cs.AI); Emerging Technologies (cs.ET); Human-Computer Interaction (cs.HC); Machine Learning (cs.LG)
关键词：make classrooms safer, computer vision, vision help make, classrooms safer, make classrooms
备注：
点击查看摘要
Abstract:Can computer vision help make classrooms safer? In this pilot study, we investigate privacy-aware and computationally efficient classroom incident recognition from CCTV-style observations. This setting remains underexplored, with limited benchmarks and few methods designed for the privacy, efficiency, and generalization demands of real-world deployment. We introduce a novel hybrid benchmark combining generative CCTV-style videos with real-world classroom pose data, and propose a lightweight, but robust motion-reasoning framework motivated by the observation that many incidents differ more in motion direction, speed, acceleration, and intensity than in pose alone. To that end, our method first constructs hierarchical kinematic representations of human actions. Our method then distills hierarchical, multi-order kinematic reasoning from a large teacher into a much smaller single-order student, enabling efficient per-person inference while preserving expressive motion understanding. Experiments show that our model outperforms substantially larger baselines at less than one-tenth of their computational cost, while also demonstrating stronger out-of-domain motion reasoning and zero-shot synthetic-to-real generalization. We will publicly release the benchmark, codebase, and supporting tools to facilitate further research in privacy-aware classroom safety.
8. 【2608.05101】HexMIL: Hierarchical Attention MIL for Ante-Hoc Explainable Detection of AI-Manipulated CT Volumes
链接：https://arxiv.org/abs/2608.05101
作者：Orazio Pontorno,Luca Guarnera,Zahid Akhtar,Sebastiano Battiato
类目：Computer Vision and Pattern Recognition (cs.CV); Multimedia (cs.MM)
关键词：deep generative models, Multiple Instance Learning, poses a significant, clinical workflows, medical images manipulated
备注： Accepted at ACM Multimedia 2026 (MM '26)
点击查看摘要
Abstract:The emergence of medical deepfakes, i.e., medical images manipulated by deep generative models, poses a significant threat to clinical workflows. However, existing detectors suffer from two critical limitations: poor generalization to unseen generative architectures for manipulation detection and lack of interpretability. In this context, we present HexMIL (Hierarchical EXplainable Multiple Instance Learning), a mask-free medical deepfake detector that simultaneously addresses both limitations using only binary volume-level supervision. HexMIL decomposes each CT volume into a two-level hierarchy of patches and slices, aggregated via independent Gated Attention modules whose weights are directly combined into a full-resolution 3D attention volume that localizes the manipulated sub-region without any pixel-level annotation. Unlike post-hoc methods such as Grad-CAM, HexMIL's attention weights constitute the exact forward computation driving the classification decision, providing ante-hoc and structurally faithful spatial attribution. We evaluate HexMIL on M3DSynth and CT-GAN datasets under a rigorous cross-generator generalization protocol, training on a single generative architecture and testing on unseen ones. HexMIL outperforms all baselines by $+9.1$ AUC and $+9.4$ F1 in out-of-domain classification, and achieves the best average IoU and Pointing Game score in localization. Project page: this http URL.
9. 【2608.05100】Lesion Detection in CT with Frozen Self-Distilled Features: SALT, a Spatially Adaptive Label-Guided Temperature
链接：https://arxiv.org/abs/2608.05100
作者：Mahmut S. Gokmen,Evan W. Damron,Mitchell A. Klusty,Caroline N. Leach,Emily B. Collier,V. K. Cody Bumgardner
类目：Computer Vision and Pattern Recognition (cs.CV)
关键词：patches wide contributes, Self-supervised pretraining objectives, per-patch loss weight, Spatially Adaptive Label-guided, Self-supervised pretraining
备注：
点击查看摘要
Abstract:Self-supervised pretraining objectives are spatially uniform: the teacher temperature and the per-patch loss weight are identical everywhere in the image, so a lesion a few patches wide contributes no more to the training signal than the surrounding parenchyma. Prior work biases the views toward annotated regions, which changes what the model sees but adds no pressure on the objective. We instead condition the targets of self-distillation, a method we call SALT (Spatially Adaptive Label-guided Temperature). Weak, box-derived labels, available only during pretraining, define a compact region on the encoder's patch grid, inside which the teacher's softmax temperature is sharpened and the masked-patch loss is up-weighted. The objectives, the masking policy and the centering statistics are otherwise unchanged, and at every downstream use the encoder is a plain feature extractor with no labels and no conditioning. We evaluate by freezing the encoder and training only a lightweight multi-depth CenterNet-style head, detecting lesions in 3D on four CT cohorts, and we isolate the mechanism against a backbone identical in architecture, pretraining data, schedule and label-guided cropping but with no target conditioning. We report patch-level separability, 3D detection stratified by cohort and by lesion size, box quality, and a detector-free probe in which a single frozen patch embedding re-identifies a lesion in a follow-up scan without registration, masks or fine-tuning. Because the conditioning is expressed through a spatial indicator rather than through label semantics, the formulation admits any weak spatial annotation; we instantiate and validate it for lesions.
10. 【2608.05074】Bag-of-Visual-Words for Spatial Mapping of Lung Adenocarcinoma Growth Patterns
链接：https://arxiv.org/abs/2608.05074
作者：Darya Ardan,Valentin Oreiller,Henning Müller
类目：Computer Vision and Pattern Recognition (cs.CV)
关键词：requires resolving architectural, resolving architectural context, generic morphological clusters, individual tile level, produce generic morphological
备注： 10 pages, 2 figures. Accepted at the 7th International Conference on Medical Imaging and Computer-Aided Diagnosis (MICAD 2026)
点击查看摘要
Abstract:Spatial mapping of lung adenocarcinoma (LUAD) growth patterns across whole slide images (WSIs) requires resolving architectural context at the region level, yet existing methods operate at the individual tile level and produce generic morphological clusters rather than clinically defined pattern maps. We propose a weakly supervised Bag-of-Visual-Words (BoVW) pipeline that learns a visual vocabulary from frozen foundation model embeddings extracted from a small set of annotated regions of interest (ROIs). Pattern prototypes are constructed as mean BoVW histograms of same-label ROIs and used for nearest-prototype classification of sliding-window regions under Jensen--Shannon divergence. The resulting predictions are projected onto the WSI tile grid to produce interpretable spatial pattern maps. We evaluate the method on 87 CPTAC-LUAD patients using three foundation model encoders and multiple vocabulary sizes on two clinically motivated tasks. For tumour/healthy classification, the best configuration achieves a balanced accuracy of $0.974$ with H-Optimus-1, approaching the $0.987$ obtained by a supervised SVM trained on mean-pooled WSI embeddings. For binary histologic grade classification, the BoVW pipeline achieves higher balanced accuracy than the supervised baseline for all encoders, suggesting that ROI-level pattern decomposition preserves grade-relevant heterogeneity that is attenuated by global mean pooling.
11. 【2608.05070】HelloWorld: Enabling Socially Interactive Characters in Video World Models
链接：https://arxiv.org/abs/2608.05070
作者：Liangyang Ouyang,Ruicong Liu,Xuangeng Chu,Kaipeng Zhang,Yoichi Sato
类目：Computer Vision and Pattern Recognition (cs.CV)
关键词：worlds remains unsupported, remarkable recent progress, video world models, video world model, video world
备注： Project page: [this https URL](https://github.com/AlayaLab/HelloWorld)
点击查看摘要
Abstract:Despite the remarkable recent progress of video world models, social interaction between users and the characters within these worlds remains unsupported. To fill this gap, we present HelloWorld, a video world model that enables social interaction with in-world characters. With a single button press, users can prompt the on-screen character to respond toward the camera, e.g., turning to the viewer, waving, nodding, or speaking a short greeting. To make these interactions natural, we propose a self-distillation pipeline that finetunes the video generation model on data synthesized by itself. Each synthesized clip contains both social interactions and camera motion, allowing the model to learn camera-pose conditioning without degrading interaction quality. At inference, we further introduce a training-free module that determines when the interaction occurs. Upon a button press, it modulates the cross-attention masks of the DiT so that the interaction-related text prompt attends only to the frames within the press window, temporally localizing the character's response. We further build HelloWorldBench, a 400-sample benchmark with three social interaction metrics alongside three conventional metrics, for evaluation. Experiments demonstrate that HelloWorld surpasses a variety of baselines in interaction quality, while maintaining state-of-the-art picture aesthetics and camera-pose following. Project page: this https URL
12. 【2608.05069】VQ-VAD: Vector-quantized Motion Representation Learning for Human-centric Video Anomaly Detection
链接：https://arxiv.org/abs/2608.05069
作者：Narges Rashvand,Ghazal Alinezhad Noghre,Shanle Yao,Gabriel Maldonado,Hamed Tabkhi
类目：Computer Vision and Pattern Recognition (cs.CV); Artificial Intelligence (cs.AI)
关键词：inherently challenging due, Video Anomaly Detection, large visual variability, Anomaly Detection, Video Anomaly
备注：
点击查看摘要
Abstract:Video Anomaly Detection (VAD) is inherently challenging due to the scarcity of anomalies and the large visual variability in surveillance footage, including changes in lighting, viewpoint, and human appearance. To mitigate visual noise and address privacy concerns, recent work has shifted to pose-based VAD, which focuses on motion dynamics rather than raw video data. However, existing pose-based approaches model human behavior in continuous latent spaces, limiting their ability to learn compact motion patterns necessary for robust behavior analysis. We address this by proposing Vector-Quantized Video Anomaly Detection (VQ-VAD), a novel human-centric anomaly detection framework that learns discrete motion representations. VQ-VAD adapts Vector-Quantized GAN (VQ-GAN), originally developed for image generation, to operate on keypoint sequences and construct a motion codebook of normal behavior. Trained exclusively on normal motion sequences, VQ-VAD detects anomalies by identifying high reconstruction errors when an observed motion sequence cannot be mapped to the learned codebook. We conduct extensive experiments across three complementary evaluation settings, including in-domain, cross-domain, and cross-dataset generalization, on four anomaly detection benchmarks. VQ-VAD achieves strong in-domain accuracy (81.83% on HR-SHT [15]), effective cross-domain transfer from CMU Panoptic [14] (76.69% on HR-SHT [15] without retraining), and competitive cross-dataset robustness. The code base for this work is available at this https URL.
13. 【2608.05066】Beyond Reprojection Error: Camera Calibration with 3D Targets
链接：https://arxiv.org/abs/2608.05066
作者：Dennis Ruppel,Hasan Kutlu,Kai A. Neumann,Martin Knuth,Pedro Santos,Andreas Weinmann,Arjan Kuijper
类目：Computer Vision and Pattern Recognition (cs.CV)
关键词：achieving high fidelity, reconstructed geometry, essential element, element for achieving, achieving high
备注： 16 pages, 7 figures, 2 tables. To appear in the proceedings of Computer Graphics International (CGI 2026)
点击查看摘要
Abstract:In 3D reconstruction, camera calibration is an essential element for achieving high fidelity and accuracy of the reconstructed geometry. While existing approaches rely upon 2D planar calibration, this work proposes a framework tailored for 3D reconstruction that is based on predicting scene rays, which adds flexibility to the reconstruction pipeline and enables the use of recent advances in camera models. Novel metrics, reconstruction and intersection error, derived from predicted scene rays are employed in combination with a bootstrapping procedure that statistically evaluates different calibration objects and calibration pipelines for both intrinsic and extrinsic camera parameters. The results show that the generalized distortion model more faithfully captures physical camera effects and yields an improvement in calibration accuracy. Reprojection error is shown to be a potentially misleading indicator of 3D accuracy, and the proposed ray-based metrics provide a more holistic assessment. An icosahedron calibration target is designed to enrich calibration information for 3D reconstruction together with a ring-feature-based detector. The icosahedral target yields approximately 40% lower mean intersection and more stable calibration across bootstrap trials on synthetic data, while real-data performance demands very tight fabrication tolerances.
14. 【2608.05049】OmniEdit-Bench: A Comprehensive Benchmark for Instruction-based Video Editing
链接：https://arxiv.org/abs/2608.05049
作者：Chenxuan Miao,Yutong Feng,Yi Lu,Yunfeng Yan,Donglian Qi,Shiwei Zhang,Yu Liu,Xi Chen,Hengshuang Zhao
类目：Computer Vision and Pattern Recognition (cs.CV)
关键词：Instruction-based video editing, broad applications, emerging field, field with broad, Instruction-based video
备注：
点击查看摘要
Abstract:Instruction-based video editing (IVE) is an emerging field with broad applications, yet evaluating editing models remains challenging. Existing benchmarks suffer from two major limitations: limited task coverage inherited from image editing, which overlooks video-specific dimensions, and inadequate metrics that fail to measure instruction fidelity, allowing incorrect edits to receive high scores due to strong visual priors from the original video. To address these issues, we introduce a comprehensive and structured benchmark for IVE. Our benchmark decomposes editing tasks into multiple video-specific dimensions, including spatial, temporal, audio, and reference-based editing, extending beyond conventional frame-level evaluation. It also distinguishes explicit and implicit instructions and incorporates reasoning-based scenarios to better reflect real-world requirements. Furthermore, we propose an evaluation framework that assesses editing quality from four complementary dimensions: accuracy, preservation, realism, and consistency, using both human judgments and state-of-the-art vision-language models. To emphasize instruction fidelity, we introduce an accuracy-aware penalty mechanism that conditions other scores on accuracy, preventing visually plausible but incorrect edits from receiving inflated evaluations. Extensive experiments on representative open-source and commercial models show that current IVE models remain far from satisfactory. OmniEdit-Bench provides a comprehensive and reliable testbed for evaluating instruction-based video editing and offers insights into future research directions.
15. 【2608.05000】owards Physics of Multimodal Pretraining: Knowledge Flow, Modality Synergy, Early Unification, and Recipes
链接：https://arxiv.org/abs/2608.05000
作者：Junlin Han,Shengbang Tong,David Fan,Minghao Chen,Philip Torr,Filippos Kokkinos,Mike Lewis
类目：Computer Vision and Pattern Recognition (cs.CV); Machine Learning (cs.LG); Multimedia (cs.MM)
关键词：natively unified multimodal, driving a shift, multimodal pretraining, offers a critical, critical axis
备注： Project page: [this https URL](https://junlinhan.github.io/projects/physics_of_mm_pretrain/)
点击查看摘要
Abstract:Vision offers a critical axis for advancing foundation models, driving a shift towards natively unified multimodal pretraining. Despite this momentum, the design space and the fundamental mechanisms of how modalities interact during unified training remain underexplored. We provide empirical clarity through a systematic exploration of multimodal pretraining. Our controlled experiments on both synthetic and large-scale real-world datasets yield four key insights into the physics of multimodal pretraining: (i) Knowledge Flow: We disentangle how language, visual understanding, and visual generation transfer knowledge across modalities, revealing distinct patterns of influence and asymmetry; (ii) Synergy vs. Competition: We show that data "complexity" largely determines whether modalities are synergistic, identify architectural choices that promote synergy: such as shared attention and normalization with modality-specific feed-forward layers, and find that these behaviors generalize across different visual tokenizer designs; (iii) Early Unification: Unifying modalities from the very early stages and training them jointly is shown to be more effective than late alignment or sequential training. This process uncovers a vision laziness phenomenon, where delayed integration leads models to rely on language priors; (iv) Recipes: We derive efficient pretraining recipes that achieve strong generative performance using only 5% of the compute budget. These core findings are subsequently validated at scale by training multiple 13.5B MoE models on 2T tokens. We hope this study provides a principled foundation for understanding and scaling multimodal pretraining.
16. 【2608.04995】Promptable Animal Pose Tracking Across Species
链接：https://arxiv.org/abs/2608.04995
作者：Le Li,Daniela Ivanova,Nicolas Pugeault
类目：Computer Vision and Pattern Recognition (cs.CV)
关键词：limited expert time, labelling automated approaches, Animal pose estimation, Animal pose, pose estimation
备注： Accepted for presentation at the ECCV 2026 Workshop on CV4Ecology
点击查看摘要
Abstract:Animal pose estimation and tracking is important for wildlife monitoring and conservation research, and with limited expert time for labelling automated approaches are imperative. While human pose estimation and tracking has seen rapid progress thanks to large annotated datasets, animal pose remain challenging, due to large morphological and behavioural differences between species and limited annotated data. Existing approaches either optimise generic keypoint localisation from annotated datasets (such as APTv2) with poor generalisation, or track custom keypoints using visual tracking, at the cost of performance. In this paper, we demonstrate that vision foundation models trained on large datasets can be used effectively to track animal pose with limited labelled data. We propose two models, one unsupervised and the other supervised, to track user-selected keypoints in videos. The supervised approach delivers superior tracking accuracy by employing a keypoint prompt encoder to explicitly inject structural priors from a reference frame into feature matching. In parallel, the unsupervised route provides strong cross-species robustness by leveraging diverse foundation-model features for training-free correspondence matching. Extensive evaluation on challenging animal video benchmarks APTv2 and TigDog demonstrates that our framework achieves strong performance while maintaining an effective balance between accuracy and generalisation, offering a practical solution for real-world animal behaviour analysis and conservation applications.
17. 【2608.04956】ContextMaster: Interactive Multi-Shot Video Creation via Fixed-Budget Sparse Context Routing
链接：https://arxiv.org/abs/2608.04956
作者：Xu Guo,Zhengxuan Wei,Xinghui Li,Hanzhuo Huang,Xinyu Liu,Xiangyang Luo,Min Wei,Yiran Zhu,Qiulin Wang,Yulong Xu,Xintao Wang,Pengfei Wan,Qi Fan,Xiangwang Hou
类目：Computer Vision and Pattern Recognition (cs.CV)
关键词：increasingly support generation, Recent video models, Recent video, models increasingly support, support generation
备注： Project page: [this https URL](https://guoxu1233.github.io/ContextMaster/)
点击查看摘要
Abstract:Recent video models increasingly support generation, reference conditioning, and editing within a single model, yet typically expose them as separate operations over fixed inputs. Practical creation unfolds across multiple shots, requiring one model to generate from text, follow a reference, or edit source footage while maintaining shared history. We formalize this setting as interactive multi-shot video creation (IMVC) and introduce ContextMaster, a unified model with a role-aware context representation for these operations. An interactive model must retain access to an expanding history without allowing the context read cost at each denoising step to grow. ContextMaster combines reusable clean context states with fixed budget sparse context routing and uses ConstraintSink to keep task constraints visible. To address the dual challenges of sparse context access and inference with few denoising steps, we propose a two-stage privileged context distillation framework, which transfers full context behavior from a dense teacher through consistency distillation and then refines deployment rollouts with distribution matching. Experiments on the three primitive tasks demonstrate improved task fulfillment and consistency across shots over specialized baselines. User studies further validate flexibly composed workflows, while the model reaches 16 FPS on a single GPU.
18. 【2608.04955】owards Valid B-Rep Generation: Training-Free Wireframe Anomaly Detection and Repair
链接：https://arxiv.org/abs/2608.04955
作者：Jingyu Wu,Youcheng Cai,Tengyu Luo,Ligang Liu
类目：Computer Vision and Pattern Recognition (cs.CV)
关键词：Multi-stage boundary representation, Multi-stage boundary, leverages intermediate wireframes, boundary representation, synthesize CAD models
备注： AAAI 2027 submission; 9-page main paper plus supplementary material
点击查看摘要
Abstract:Multi-stage boundary representation (B-Rep) generation leverages intermediate wireframes to synthesize CAD models. However, geometric and topological risks in these wireframes -- such as self-intersections, edge collapses, and disconnected vertices -- can propagate to invalid final B-Reps. Mitigating such failures by retraining large generative models is computationally prohibitive. We propose Wireframe Detection and Repair (WDR), a training-free framework that intervenes at the intermediate wireframe stage to improve downstream B-Rep validity. WDR features a Geometric-Topology Anomaly Detector (GTAD) that combines parallel VLM-based coarse screening with geometric and topological detectors to predict downstream invalidity risk and route generation to dedicated branches. An Energy-Guided Geometric-Topology Repair (EGGTR) module then performs detector-triggered guided regeneration through geometry and topology branches. By scaling test-time computation via Energy-Guided Resampling and training-free guidance for diffusion models, WDR can be integrated into autoregressive and diffusion pipelines without retraining. Extensive experiments demonstrate consistent improvements in kernel-checked validity while largely retaining the measured diversity and distributional quality of synthesized CAD models. The code will be made publicly available upon acceptance.
19. 【2608.04949】UG-UMRE: Uncertainty-Guided Modality Augmentation and Distributional Calibration for Unified Multimodal Relation Extraction
链接：https://arxiv.org/abs/2608.04949
作者：Bo Kong,Liruiz Jia,Yi Liang,Chao Liu,Dongfang Han,Tianwei Yan,Yuan Liu,Shengquan Liu
类目：Computer Vision and Pattern Recognition (cs.CV); Computation and Language (cs.CL); Information Theory (cs.IT); Multimedia (cs.MM)
关键词：Unified Multimodal Relation, Multimodal Relation Extraction, Unified Multimodal, Relation Extraction, Multimodal Relation
备注： Accepted at ACM MM2026
点击查看摘要
Abstract:Unified Multimodal Relation Extraction (UMRE) aims to identify intra-modal and cross-modal relations between textual entities and visual objects. However, existing UMRE studies still encounter two critical issues: ignoring inherent aleatoric uncertainty causes noise propagation, and deep-seated heterogeneity between distinct modal distributions hinders alignment. To address these issues, we propose the Uncertainty-Guided UMRE Network (UG-UMRE). Specifically, we design an Uncertainty-Driven Unimodal Augmentation (UDUA) module, which models features as Gaussian distributions based on the Variational Information Bottleneck. By incorporating an uncertainty-aware self-supervised contrastive learning mechanism, UDUA effectively filters out noise while maintaining semantic consistency. Furthermore, we introduce the Joint Aleatoric Uncertainty Alignment (JAUA) module as a global semantic pre-calibration mechanism. JAUA leverages probabilistic distribution consistency to construct a shared latent space, eliminating the distributional gap by synchronizing cross-modal statistical properties, thereby laying a robust foundation for fine-grained interaction. Experiments on three benchmark datasets (UMRE, MORE, and MNRE) demonstrate that UG-UMRE achieves state-of-the-art performance. Further analysis validates the pluggable and effective performance of the proposed UDUA and JAUA modules.
20. 【2608.04935】Unleashing the Potential of Vision-Language Models for Generalizable AI-Generated Image Detection
链接：https://arxiv.org/abs/2608.04935
作者：Weihan Cai,Hao Tan,Zichang Tan,Jun Wan,Xinping Gao
类目：Computer Vision and Pattern Recognition (cs.CV)
关键词：substantially outperforming specialized, modern vision foundation, Recent work, AIGI detection performance, vision foundation models
备注：
点击查看摘要
Abstract:Recent work has shown that a simple linear probe on frozen representations from modern vision foundation models (VFMs) can achieve state-of-the-art AIGI detection performance, substantially outperforming specialized detectors in challenging in-the-wild scenarios. This finding has established DINOv3 as the dominant foundation-model baseline for subsequent improvements. However, we find that the vision-language model Perception Encoder (PE) holds greater potential for AIGI detection, because its language-aligned representation preserves high-level provenance semantics. Specifically, PE exhibits stronger local provenance organization than DINOv3 in its frozen feature space. However, semantic-agnostic linear probing fails to exploit this structure, as PE-Linear still underperforms DINOv3-Linear by 4.1% on In-the-Wild. Based on this observation, we propose Semantic Prototype Calibration (SPC), which constructs category prototypes from forensic semantic information and calibrates them with supervised data. We apply SPC to PE and refer to the resulting detector as PE-SPC. Our analysis shows that this simple design achieves stronger generalization. Across cross-generator, post-processing, and in-the-wild benchmarks, PE-SPC surpasses the previous DINOv3 baseline and achieves new state-of-the-art results.
21. 【2608.04917】An active-learning framework for real-time depth perception from monocular vision streams
链接：https://arxiv.org/abs/2608.04917
作者：Xiaorong Zeng,Weiqiang Chen,Peng Shi,Liang Su,Zirui Wang,Xuewu Ji,Shuiwen Shen
类目：Computer Vision and Pattern Recognition (cs.CV)
关键词：monocular vision flow, continuously integrating temporal, integrating temporal visual, temporal visual cues, Biological visual systems
备注：
点击查看摘要
Abstract:Biological visual systems can perceive depth from monocular vision flow, continuously integrating temporal visual cues while maintaining a balance between stability and plasticity in dynamic environments. In contrast, artificial perception models deployed on resource-constrained edge devices are typically trained in a static offline manner and remain frozen after deployment, often suffering severe performance degradation under domain shifts. While large-scale models may encode broad knowledge through massive parameter redundancy, lightweight networks face a static optimization dilemma: forcing compact models to learn universal geometric representations is computationally inefficient and often leads to performance saturation. To resolve this issue, an Online Active Learning (OAL) mechanism is introduced to endow compact neural networks with the capability to adapt continuously during operation. A closed-loop Predict-Evaluate-Correct learning paradigm is established to actively select high-confidence, information-rich signals from streaming visual input. Crucially, Elastic Weight Consolidation (EWC) is employed not merely to prevent catastrophic forgetting, but to enforce Selective Plasticity, preserving parameters that encode globally relevant structural knowledge while allowing local alignment to newly observed environments. Built upon a MobileNetV3-Small backbone, the proposed system achieves approximately a 75% reduction in computational cost while maintaining competitive depth estimation accuracy. Experimental results demonstrate that adaptability is not solely determined by model size, but rather by how effectively parameter plasticity is regulated in dynamic environments.
22. 【2608.04906】Enhancing Low Back Pain Assessment with Diffusion Models for Lumbar Spine MRI Segmentation
链接：https://arxiv.org/abs/2608.04906
作者：Maria Monzon,Thomas Iff,Ender Konukoglu,Catherine R. Jutzeler
类目：Computer Vision and Pattern Recognition (cs.CV)
关键词：low back pain, accurate semantic segmentation, spine MRI scans, lumbar spine MRI, back pain
备注： Maria Monzon and Thomas Iff contributed equally to this work. Published in Proceedings of The 8th International Conference on Medical Imaging with Deep Learning (MIDL 2025), PMLR volume 301, pages 1145-1163, 2026
点击查看摘要
Abstract:This study introduces a diffusion-based framework for robust and accurate semantic segmentation of lumbar spine MRI scans from patients with low back pain (LBP), regardless of whether the scans are T1- or T2-weighted. We compared with advanced models for segmenting vertebrae, intervertebral discs (IVDs), and spinal canal using the SPIDER dataset. The results showed that SpineSegDiff achieved a segmentation performance comparable to that of the state-of-the-art non-diffusion nnUnet, particularly in improving the identification of degenerated IVDs. In addition, the uncertainty maps generated by our model provide valuable insights for clinical review, enhancing the robustness and reliability of the segmentation results. The potential of diffusion models to enhance the diagnosis and management of LBP through more precise analysis of pathological spine MRI is underscored by our findings.
23. 【2608.04902】Visual Representation Matters: Exploiting Temporal Differences in Video-to-Audio Generation
链接：https://arxiv.org/abs/2608.04902
作者：Zehua Chen,Junyou Wang,Yuxuan Jiang,Zhenying Fang,Yusheng Dai,Jianfei Chen,Ziwei Liu,Jun Zhu
类目：Computer Vision and Pattern Recognition (cs.CV); Machine Learning (cs.LG); Sound (cs.SD)
关键词：provide essential temporal, essential temporal cues, introducing consecutive frames, audio synthesis, introducing consecutive
备注：
点击查看摘要
Abstract:Video-to-audio (V2A) generation extends image-to-audio generation (I2A) by introducing consecutive frames that provide essential temporal cues for audio synthesis. However, existing conditional diffusion-based V2A methods typically enhance visual conditioning with additional audio-visual supervision, acoustic structure prediction, or reasoning from large multimodal models, requiring extra networks or strong inductive biases. Inspired by recent advances in visual representation learning, we introduce TD-V2A, which leverages temporal differences (TD) as the key representation that distinguishes V2A from I2A, enriching visual conditioning with minimal architectural modification. We first investigate TD at both the frame and feature levels to identify the most effective representation level at which TD complements visual representations. Based on these findings, we develop a hierarchically continual learning strategy and an annealed temporal differences guidance method to progressively learn and exploit TD information during diffusion training and sampling process, respectively. Extensive experiments on benchmark datasets demonstrate that effectively exploiting TD through our proposed framework significantly improves end-to-end V2A generation quality, even outperforming dedicated V2A representations such as contrastive audio-visual pretraining.
24. 【2608.04896】When Shared Rollouts Fail in Defensive Driving Evaluation: A NAVSIM Score Basis Audit
链接：https://arxiv.org/abs/2608.04896
作者：Ziang Wei,Minjun Yu,Zheyuan Lai,Mingjie Pang,Wei Li
类目：Artificial Intelligence (cs.AI); Computer Vision and Pattern Recognition (cs.CV)
关键词：observe surrounding actors, preserve distinctions, distinctions between policies, policies that observe, observe surrounding
备注： 17 pages, 1 figure
点击查看摘要
Abstract:Defensive driving scores are useful only when they preserve distinctions between policies that observe surrounding actors and those that do not. Re-simulation benchmarks may use reference-conditioned forgiveness, under which an agent receives credit when the logged human reference fails a compliance channel. When agent and reference share an unstable rollout transformation, this rule can propagate shared reference failures into broad compliance credit. We audit this risk in NAVSIM v2.2 original scene single-stage scoring. Under the affected documented-stack condition on the audited numerical backend, the route-blind Ignore-All probe and a route-aware actor-blind probe outrank human replay and PDM-Closed over the complete 12,146-token navtest split. A fresh installation following the public specification reproduces rollout divergence on a fixed 32-token diagnostic set. A same-source dependency stack control and an exact-input diagnostic isolate dependency-sensitive numerical behavior in the shared velocity refit. On a 450-token control pool, replacing only the solver eliminates rollout divergence and restores blind-last ordering while keeping forgiveness enabled. Thus, the numerical instability is the direct trigger. Reference-conditioned forgiveness propagates the resulting shared reference failures into compliance credit. We contribute an audit protocol requiring score basis and stack disclosure, blind probes, overwrite reporting, and rollout stability tests before using such scores for defensive driving claims.
Comments:
17 pages, 1 figure
Subjects:
Artificial Intelligence (cs.AI); Computer Vision and Pattern Recognition (cs.CV)
Cite as:
arXiv:2608.04896 [cs.AI]
(or
arXiv:2608.04896v1 [cs.AI] for this version)
https://doi.org/10.48550/arXiv.2608.04896
Focus to learn more
arXiv-issued DOI via DataCite (pending registration)</p>
25. 【2608.04887】STEP-OPD: Rethinking Output Targets and Internal Dynamics in On-Policy Distillation for Diffusion Models
链接：https://arxiv.org/abs/2608.04887
作者：Qingyan Wei,Guangzhao Li,Xiaobing Tu,Yinggui Wang,Xiantao Zhang,Jinkui Ren,Xiaohong Liu,Linfeng Zhang
类目：Computer Vision and Pattern Recognition (cs.CV)
关键词：consolidating multiple task-specialized, multiple task-specialized image, Standard OPD methods, OPD methods, task-specialized image generation
备注： 9 pages, 5 figures
点击查看摘要
Abstract:On-policy distillation (OPD) has become an effective approach for consolidating multiple task-specialized image generation models into a single student. However, existing OPD methods optimize the student mainly to match the teacher's output velocity, making the teacher the upper limit of the optimization objective. While output-level supervision alone leaves the student's blockwise representation evolution underconstrained, which weakens the transfer of capabilities that must be progressively developed across layers. We propose STEP-OPD, an on-policy distillation framework for image generation that extends the student's learning target beyond the teacher and introduces explicit constraints on its internal representation evolution. Instead of treating the teacher as the final target, we use the velocity difference between each task-specific teacher and the shared base model as a direction for further learning and add a scaled version of this difference to the teacher velocity. In addition, we align the direction and magnitude of representation changes between the student and teacher, enabling the student to learn how representations are progressively transformed across network blocks. Experiments on compositional alignment, text rendering, and human preference show that our method consistently improves Standard OPD methods. In particular, it increases the GenEval score of DiffusionOPD from 0.927 to 0.961, while also improving OCR and all preference-based metrics. The resulting unified student surpasses the corresponding single-task teachers across all three capability groups, showing that output extrapolation enables beyond-teacher learning. And representation change alignment provides complementary guidance for the student's internal transformations.
26. 【2608.04885】Evaluating the Diagnostic Robustness of Vision-Language Models Under Visual and Textual Perturbations
链接：https://arxiv.org/abs/2608.04885
作者：Ali Khoramfar,Mohammad Javad Dousti,Alireza Mohamadian,Heshaam Faili
类目：Computer Vision and Pattern Recognition (cs.CV); Computation and Language (cs.CL)
关键词：Standard accuracy metrics, sensitive domains, Standard accuracy, failures in sensitive, mask significant reliability
备注：
点击查看摘要
Abstract:Standard accuracy metrics for VLMs often mask significant reliability failures in sensitive domains. In this work, we utilize a histopathology-validated brain MRI dataset to systematically assess the diagnostic robustness of four VLM families under evidence-preserving perturbations. By reordering anatomical slices and swapping target label positions, we evaluate whether models maintain consistent predictions when clinical evidence remains invariant. Our results reveal significant vulnerabilities in presentation-order stability, with models exhibiting prediction flips in up to 48.9% of cases under simple sequence reversals. We further identify a textual selection bias, where label reordering triggers inconsistent diagnoses in up to 67.8% of cases despite identical visual inputs. Negative-control tests further reveal diagnostic overcommitment: models generate categorical diagnoses in up to 76.1% of cases after expert-annotated lesion slices are removed. These results demonstrate that high accuracy can overestimate clinical reliability, masking sensitivity to sequential presentation and textual framing that is not captured by aggregate accuracy. Our findings highlight the necessity of stability-based metrics for the deployment of VLMs in safety-critical clinical applications. Our evaluation data and code will be made public upon acceptance.
27. 【2608.04879】raining Crossroads for Recurrent Vision Transformers: Recurrence, Neural ODEs, and Deep Supervision
链接：https://arxiv.org/abs/2608.04879
作者：Grzegorz Gruszczynski,Pawel Olszowiec,Michal Byra,Grzegorz Stefanski,Alberto Presta
类目：Machine Learning (cs.LG); Computer Vision and Pattern Recognition (cs.CV)
关键词：Vision Transformers, achieve strong image-recognition, count grows linearly, parameter count grows, strong image-recognition performance
备注：
点击查看摘要
Abstract:Vision Transformers (ViTs) achieve strong image-recognition performance, but their parameter count grows linearly with depth when each block is independently parameterized. Single-block recurrent ViTs (bViT) remove this growth by repeatedly applying one shared block. Rather than proposing a new architecture, we fix a bViT and provide a controlled empirical characterization of three training and inference regimes under a common CIFAR-100 protocol, asking: (i)~when does recurrence beat independently parameterized depth---at matched FLOPs or at matched parameter memory? (ii)~when a residual recurrent block is trained through an ODE solver, does solver order act as numerical refinement or as an architectural bias? and (iii)~what does robustness beyond the training horizon cost in nominal accuracy? We find that standard ViTs remain preferable when FLOPs are the primary constraint, whereas recurrent ViTs offer a better accuracy--parameter trade-off under memory constraints. Consistent with the standard view of residual networks as Euler discretizations of ODEs, the continuous-time analogue of a residual recurrent block is the state-subtracted vector field $\dot{z}=F_\theta(z)-z$; although known in principle, this distinction is easy to violate when the block is wrapped as a black-box vector field, and we qualify the cost at few accuracy points. Because the vector field is learned jointly with the solver, higher-order solvers act as a solver-induced architectural bias rather than a numerical-accuracy improvement, and their gains are not uniform. Finally, stage-wise deep supervision traces an accuracy--robustness frontier: it does not improve nominal accuracy, but degrades gracefully far beyond the training horizon, where naive recurrence collapses to near-random performance.
28. 【2608.04866】Persistent Object Narratives for Token-Efficient Video Language Models
链接：https://arxiv.org/abs/2608.04866
作者：Junzhe Chen,Siyuan Meng,Xiaojie Guo
类目：Computer Vision and Pattern Recognition (cs.CV)
关键词：large language models, open-ended video understanding, Video large language, made strong progress, language models
备注：
点击查看摘要
Abstract:Video large language models (Video-LLMs) have made strong progress in open-ended video understanding. However, their visual interfaces remain token-intensive and provide limited explicit structure for linking recurring object evidence across time. We introduce SlotNarrative, a slot-based interface that organizes a video into persistent object narratives represented by compact object-state tokens. Rather than compressing frame-wise features before establishing temporal correspondence, SlotNarrative first groups visual features into object-like slots and then associates recurring observations with clip-level object entries through a lightweight, parameter-free memory that integrates multiple complementary matching cues. Each retained entry is serialized into two token types: an identity token that summarizes persistent object appearance and a set of state tokens that encode segment-level appearance, geometry, visibility, and trajectory information. This design yields an interface of only 144 allocated visual-token positions for a frozen Video-LLM, independent of the number of sampled frames. Across multiple datasets, SlotNarrative achieves a favorable trade-off between accuracy and visual-token count compared with prior compact Video-LLM interfaces. Experimental results establish persistent object narratives as a compact, structured, and temporally organized visual interface for Video-LLMs. Our code will be made publicly available.
29. 【2608.04865】Cooking beyond Frames: A Stereo Event Camera Dataset in the Kitchen
链接：https://arxiv.org/abs/2608.04865
作者：Chengming Feng,Hesam Araghi,Liming Zheng,Julien Dupeyroux,Xucong Zhang,Jan van Gemert,Nergis Tömen
类目：Computer Vision and Pattern Recognition (cs.CV)
关键词：high temporal resolution, high dynamic range, low power consumption, gained significant attention, recent years due
备注： Accepted at ECCV 2026
点击查看摘要
Abstract:Event cameras, also known as neuromorphic cameras, have gained significant attention in recent years due to their high temporal resolution, high dynamic range, and low power consumption. While many studies and datasets in neuromorphic vision have focused on automotive and drone applications, human-centric daily-life scenarios remain largely underrepresented, despite their importance for developing and benchmarking event-based perception systems. Moreover, the few existing event-based human activity datasets are typically recorded with scripted human actions, limiting their ability to capture natural human behaviors. In this paper, we introduce EventKitchen, a large-scale stereo event camera benchmark dataset of human cooking activities in the kitchen. EventKitchen is egocentrically collected from 10 participants in 13 diverse kitchens, where the participants wear a helmet with multiple sensors and naturally perform cooking activities, without any scripted actions. EventKitchen comprises 5.5 hours of stereo event recordings with synchronized RGB, depth, and IMU data. We provide human annotations for 10,762 action segments and 13,482 bounding boxes. We train baseline models on EventKitchen to perform multiple event-based tasks, including action recognition, object detection, and stereo depth estimation. By capturing natural, real-world human activities, EventKitchen establishes a challenging benchmark for neuromorphic vision beyond autonomous driving.
30. 【2608.04840】owards a satellite image manipulation and deepfake localization benchmark dataset
链接：https://arxiv.org/abs/2608.04840
作者：Jacob Arndt,Debvrat Varshney,Philipe Dias,Nivedita Nukavarapu
类目：Computer Vision and Pattern Recognition (cs.CV); Artificial Intelligence (cs.AI)
关键词：generative artificial intelligence, Verifying the authenticity, artificial intelligence, increasingly critical, critical given advances
备注： Accepted at IEEE IGARSS 2026
点击查看摘要
Abstract:Verifying the authenticity of satellite imagery has become increasingly critical given advances in generative artificial intelligence. Highly realistic synthetic imagery produced for malicious purposes (deepfakes) can have major consequences in the remote sensing domain, where this data is a fundamental source of information for science applications, planning, logistics, and monitoring. The remote sensing community lacks high-quality, fine-grained manipulation datasets suitable for training and evaluating detection and image forensics algorithms. Existing datasets are lacking and those that do exist either provide no ground truth masks for evaluating manipulation localization, or consist of entire images generated by GANs or diffusion models, which are inadequate for measuring localization performance. To address this gap, we describe a preliminary dataset construction process and prototype benchmark dataset for satellite image manipulation detection and localization. The dataset contains 60 images total, with 30 images carefully manipulated using three manipulation types including copy-paste splicing and diffusion model inpainting, and 30 authentic images. Each image is accompanied by a ground-truth mask and acquisition metadata, enabling both pixel-level localization metrics, image metadata studies, and analyses of how manipulation detection performance relates to image collection parameters. We describe the dataset construction process and present this initial release to support further research in image forensics and geospatial deepfake detection. The prototype dataset can be downloaded at this https URL.
31. 【2608.04833】RegisterBridgeMM: A Register-Centric Framework for RGB-Infrared Object Detection
链接：https://arxiv.org/abs/2608.04833
作者：Zian Wang,Hangchuan Liang,Yuehua Chen,Changchun Li,Chaoyi Guo,Mingzhe Liu,Fangming Gu
类目：Computer Vision and Pattern Recognition (cs.CV)
关键词：object detection benefits, effective fusion remains, fusion remains challenging, weather variation, object detection
备注：
点击查看摘要
Abstract:RGB-infrared (RGB-IR) object detection benefits from complementary visible and thermal cues, but effective fusion remains challenging under illumination changes, weather variation, and cluttered scenes. Existing RGB-IR fusion methods often trade expressive patch-level interaction for lighter but more constrained adaptation mechanisms. We empirically observe that pretrained register tokens contain both modality-shared and modality-specific information on paired RGB-IR inputs, suggesting that they can serve as a compact substrate for cross-modal communication. Building on this observation, we propose RegisterBridgeMM, a register-mediated fusion framework organized as a three-stage register lifecycle. Aggregate preserves per-modality register summarization inherited from pretraining; Bridge performs bidirectional register-to-patch reading with consensus-residual regulation; and Project translates the resulting register summary into spatially adaptive calibration of patch features. This register pathway avoids dense patch-to-patch cross-modal interaction while preserving the pretrained patch representation. With both backbone streams frozen, RegisterBridgeMM achieves the highest mAP50-95 among the evaluated methods on all four benchmarks: LLVIP, M3FD, DroneVehicle, and FLIR-Aligned.
32. 【2608.04821】Global Attention-Fused Image Cropping with Attention-Guided and Global-Aligned Crop Evaluator
链接：https://arxiv.org/abs/2608.04821
作者：Haotian Yang,Zhile Yang,Kin-Man Lam,Patrick Le Callet,Xin Sun
类目：Computer Vision and Pattern Recognition (cs.CV)
关键词：improve image aesthetics, appropriately composed region, Image cropping aims, preserving important content, aims to improve
备注： The source code is available at [this https URL](https://github.com/AIVRC/GAFIC.git)
点击查看摘要
Abstract:Image cropping aims to improve image aesthetics by preserving important content within an appropriately composed region. However, most existing methods focus primarily on salient regions and therefore have limited sensitivity to the global relationships among the main image components. To address this limitation, we propose Global Attention-Fused Image Cropping (GAFIC), which consists of an Attention-Guided Feature Fusion (AGFF) and a Global-Aligned Crop Evaluator (GACE). AGFF aggregates the importance of local regions to construct a global representation that captures both image structure and local details. GACE aligns candidate crop features with this global representation, enabling crop evaluation to remain sensitive to boundary changes. We further combine three ranking losses across multiple scales to obtain accurate and stable crop scores. Extensive experiments on the GAIC and CPC datasets demonstrate that GAFIC outperforms existing image-cropping methods, particularly in terms of accuracy and stability. Unlike pixel-level retargeting methods such as seam carving, inpainting, and diffusion-based synthesis, GAFIC does not synthesize or modify the retained pixels; instead, it selects an aesthetically preferred crop from the source image, making it suitable for scenarios where pixel integrity and efficient batch processing are important. The source code is available at this https URL.
33. 【2608.04820】When Diffusion Models Forget Who You Are: Identity Preservation in Face Inpainting under Large Occlusions
链接：https://arxiv.org/abs/2608.04820
作者：Feng Ding,Shuhuai Xie,Yue Zhou,Yulan Zhang,Guopu Zhu,Mengyao Xiao
类目：Computer Vision and Pattern Recognition (cs.CV)
关键词：recently achieved impressive, achieved impressive visual, conflicting text guidance, text guidance remains, Face inpainting
备注：
点击查看摘要
Abstract:Face inpainting with diffusion models has recently achieved impressive visual quality, yet preserving identity fidelity under significant occlusion and conflicting text guidance remains a major challenge. To address this issue, we present Reference Semantic Inpainting for Face (ReSem-Face), a cascaded diffusion framework that introduces an explicit identity-conditioned semantic prior for multi-reference face inpainting. Our approach distills representative identity features from multiple references to reconstruct missing semantic regions, which then guide the diffusion process through a multi-stream conditioning architecture. This design provides strong semantic constraints when pixels are absent and stabilizes identity reconstruction while remaining compatible with prompt-driven edits. Experiments on CelebAHQ-IDI-5 and VGGFace2 demonstrate that ReSem-Face yields more reliable identity-preserving completion under severe semantic masks and improves text-controlled editing quality compared with representative baselines.
34. 【2608.04818】Rethinking Pixel Mean Flows via Interval Denoiser
链接：https://arxiv.org/abs/2608.04818
作者：Alexander Zaytsev,Dmitry Baranchuk,Alexander Korotin,Aibek Alanov
类目：Computer Vision and Pattern Recognition (cs.CV)
关键词：Modern diffusion, external autoencoders, diffusion and flow-based, increasingly moving, bypass the computational
备注：
点击查看摘要
Abstract:Modern diffusion and flow-based models are increasingly moving toward few-step, latent-free generation to bypass the computational overhead of multi-step sampling and the reconstruction bottlenecks of external autoencoders. We propose the Interval Denoiser, a theoretically rigorous framework for latent-free generation. Derived directly from the flow matching ODE, it establishes an exact analytical mapping for intermediate trajectory states. Unlike prior formulations, our prediction is shown to reside on a low-dimensional manifold across any time interval, making the regression tractable for a network operating directly on pixels. Furthermore, by avoiding empirical algebraic substitutions, our formulation correctly isolates the pure time derivative to prevent biased gradient evaluations and ensure exact first-order optimization. By analyzing this objective, we equip our framework with residual clipping and a time-sampling curriculum, enabling effective long-interval training and improving few-step performance. Trained from scratch on ImageNet 256x256, our model achieves an FID of 4.55 in one step (1-NFE) and 3.98 in two steps (2-NFE) without perceptual losses.
35. 【2608.04811】StaticSegFormer: An Efficient High-Performance Semantic Segmentation Based on Static Structured Pruning
链接：https://arxiv.org/abs/2608.04811
作者：Timo Bartels,Danish Nazir,Jan Piewek,Thorsten Bagdonat,Tim Fingscheidt
类目：Computer Vision and Pattern Recognition (cs.CV)
关键词：deep neural networks, Structured pruning enhances, parameters during inference, Structured pruning, enhances the efficiency
备注：
点击查看摘要
Abstract:Structured pruning enhances the efficiency of deep neural networks (DNNs) by eliminating groups of parameters during inference. Previous methods mostly reduce computational complexity (FLOPs), while semantic segmentation performance (mIoU) slightly drops. Accordingly, recent dynamic structured pruning methods aim at reducing the performance drop, while lowering the FLOPs even more. However, on the ADE20K and Cityscapes benchmarks, our study reveals that on a GPU platform such dynamic methods exhibit a surprisingly low frame rate far below a simple static approach, while having comparable results in mIoU and FLOPs. To address this issue, we propose a static structured pruning method for attention layers, that achieves both, a lower FLOPs and a high frame rate [fps] of the SegFormer network, the latter increased by up to 34% relative on the Cityscapes dataset, while having no mIoU performance drop at all. Our so-called StaticSegFormer method is strongest for small encoders and large images.
36. 【2608.04810】Segmentation Pre-training for Label-Efficient Lumbar Spine Degeneration Grading
链接：https://arxiv.org/abs/2608.04810
作者：Monzon Maria,Zisserman Andrew,Jutzeler Catherine R.,Jamaludin Amir
类目：Computer Vision and Pattern Recognition (cs.CV)
关键词：magnetic resonance imaging, expert-annotated radiological gradings, resonance imaging, requires access, assessment of degenerative
备注： The 2nd MICCAI Workshop on Efficient Medical AI (2026)
点击查看摘要
Abstract:Automated assessment of degenerative pathology in the lumbar spine on magnetic resonance imaging (MRI) requires access to large-scale datasets of expert-annotated radiological gradings. In contrast, segmentation pseudo-labels can be generated by automated tools at negligible radiologist cost. We examine whether pre-training on segmentation can effectively replace a fraction of the manual grading annotations required for downstream supervision. We pre-train a 3D ResNet encoder to segment the vertebrae, intervertebral discs (IVDs), and the spinal canal, then fine-tune lightweight task-specific grading heads using different proportions of the available training data, ranging from $10\%$ to $100\%$. On a multicentre dataset of ${\sim}2{,}000$ subjects across 11 pathologies, segmentation pre-training, achieving a Dice score of $0.94$ against pseudo-labels, improved the task-averaged (macro) one-vs-rest ROC-AUC at all proportions. With only 20\% of grading labels after pre-training, the method achieved near full-supervision performance, with the largest gains observed for either low-prevalence or spatially grounded pathologies.
37. 【2608.04791】On the Effectiveness of Adaptation Strategies for VLM-Based Federated Learning in Remote Sensing
链接：https://arxiv.org/abs/2608.04791
作者：Simon Lösche,Barış Büyüktaş,Mathis Adler,Angelos Zavras,Ioannis Papoutsis,Begüm Demir
类目：Computer Vision and Pattern Recognition (cs.CV)
关键词：requiring data centralization, enables collaborative training, decentralized image archives, enables collaborative, archives without requiring
备注： Accepted at the SPIE Artificial Intelligence and Image and Signal Processing for Remote Sensing, Edinburgh, Scotland, 2026
点击查看摘要
Abstract:Federated learning (FL) enables collaborative training of deep learning models across decentralized image archives without requiring data centralization. This paradigm is particularly relevant in remote sensing (RS), where legal regulations, privacy concerns, and bandwidth constraints restrict data sharing. However, the presence of training data heterogeneity across clients (known as non-IID data) can impede convergence and limit the generalization capability of the aggregated global model. To mitigate the adverse effects of training data heterogeneity, vision-language models (VLMs) can be leveraged in FL due to their transferable representations, which have demonstrated robustness under distribution shifts. However, their large parameter size may substantially increase communication overhead and local computational complexity in federated settings. Therefore, it is crucial to select an appropriate VLM adaptation strategy that balances the generalization ability with the communication and computational constraints. To address this issue, in this paper, we present the first comparative study of VLM adaptation strategies for FL in the context of RS image classification. We investigate full fine-tuning, encoder-specific fine-tuning, prompt learning, and low-rank adaptation (LoRA) tuning, and analyze them with respect to three criteria: 1) generalization capability under non-IID data, 2) communication overhead, and 3) local computational complexity. Experiments on BigEarthNet-S2, EuroSAT, RESISC45, and ImageNet reveal distinct trade-offs between task specialization, cross-domain generalization, and efficiency. Based on our findings, we derive a guideline for the selection of an appropriate VLM adaptation strategy in FL for RS image classification under different operational constraints. The code of this work is publicly available at this https URL.
38. 【2608.04768】Embedding Large Language Models into Flow Controls: An Agentic Framework for Adaptive and Trustworthy Automated Cooking
链接：https://arxiv.org/abs/2608.04768
作者：Zihan Song,Hongwei Huang,Yueshuo Sun,Yonglin Tian,Fei-Yue Wang,Bai Li
类目：Computer Vision and Pattern Recognition (cs.CV)
关键词：recent large-model approaches, large-model approaches support, approaches support natural, opaque decision making, natural language interaction
备注：
点击查看摘要
Abstract:Automated cooking robots have traditionally relied on predefined procedures and rule-based control, ensuring stable execution but offering limited personalization, whereas recent large-model approaches support natural language interaction but often suffer from opaque decision making and unreliable execution in real kitchens. To address this challenge, this paper proposes an agentic framework that systematically decomposes personalized cooking requirements into structured and verifiable control programs rather than directly mapping language to actions. Multiple AI agents collaboratively transform user intents into canonical recipes, workflow programs with explicit flow control, and executable Python code grounded in an atomic action library. The system consists of three tightly coupled stages: offline recipe-to-code generation through multiple agents, online closed-loop execution with supervisory intervention enabled by multimodal perception, and post-run adaptation that updates user preference models for long-term personalization. Real-world experiments on a physical cooking platform demonstrate that the proposed framework achieves reliable task completion, transparent execution logic, and effective anomaly handling across diverse personalized scenarios, validating its practicality for trustworthy automated cooking in real environments.
39. 【2608.04766】FUSEP: A Multi-Center Benchmark for Diverse Tasks in Early Pregnancy Fetal Ultrasound Screening
链接：https://arxiv.org/abs/2608.04766
作者：Bin Pu,Jiewen Yang,Liwen Wang,Ying Tan,Guannan He,Xingbo Dong,Qika Lin,Jiarong Guo,Lixian Yang,Zuozhu Liu,Shengli Li,Kenli Li
类目：Computer Vision and Pattern Recognition (cs.CV); Artificial Intelligence (cs.AI)
关键词：underdeveloped medical resources, fetal ultrasound screening, year globally, ultrasound, large number
备注：
点击查看摘要
Abstract:A large number of infants with congenital anomalies are born each year globally, especially in areas with underdeveloped medical resources. Currently, fetal ultrasound screening is the most common modality for early pregnancy anatomy detection. This modality can detect anomalies earlier and provide opportune treatment advice. However, the lack of an ultrasound dataset on early fetal gestation has slowed down the development of automated assisted diagnosis. In this work, we present a benchmark dataset for Fetal Ultrasound Screening in Early Pregnancy to facilitate intelligent ultrasound examination and assisted diagnosis called FUSEP. Our dataset consists of two ultrasound views recommended by the international guideline, i.e., Crown-rump Length (CRL) and Nuchal Translucency (NT) views in three hospitals, totaling 4,017 ultrasound images, with 45,820 box-level expert-level annotations. Our dataset and baseline present the following three contributions: 1) Our medical experts annotated a total of 14 key anatomical structures in two views using a box-level format; 2) Our data is collected extensively from different sonographers, devices, scanning angles, hospitals, etc; 3) We report the performance of the semi-supervised learning, fully supervised learning, unsupervised domain adaptation (UDA), and source-free UDA in ultrasound images multi-object detection. To the best of our knowledge, this is the first publicly available dataset and benchmark for fetal early pregnancy ultrasound screening. We believe that FUSEP and benchmark can contribute to the medical community in the development of multiple tasks such as standard plane recognition, quality control on ultrasound images, automated assisted diagnostics in early fetal pregnancy, medical multi-object detection, domain adaptation for object detection, etc.
40. 【2608.04765】Explicit Language Memory for Long-Horizon Planning in Vision-Language-Action Models
链接：https://arxiv.org/abs/2608.04765
作者：Houze Xu,Jizhong Li,Ziyi Ye
类目：Robotics (cs.RO); Artificial Intelligence (cs.AI); Computer Vision and Pattern Recognition (cs.CV)
关键词：connecting visual perception, existing VLA models, VLA models, VLA, high-level VLM
备注： 11 pages, 4 figures
点击查看摘要
Abstract:Vision-language-action (VLA) models provide a unified paradigm for connecting visual perception, language understanding, and robotic control. However, existing VLA models still face major challenges in long-horizon tasks: sparse expert demonstrations constrain cross-task compositional generalization; the non-Markovian nature of long-horizon tasks makes it difficult for policies conditioned only on current observations to maintain temporal consistency; limited closed-loop error correction allows execution errors to accumulate; and end-to-end action fine-tuning may weaken the high-level semantic representations of vision-language model (VLM) backbones. To address these issues, we propose a hierarchical long-horizon VLA architecture with an explicit language-memory module. The central idea is to convert discrete temporal observations into a coherent textual memory sequence with temporal logic. The system is decoupled into a high-level VLM and a low-level VLA: the high-level VLM performs semantic reasoning through a visual question answering training paradigm, while the low-level VLA executes precise continuous control conditioned on subtask instructions and visual observations. The high-level VLM recursively updates both language memory and subtask instructions using the previous memory as a contextual anchor, enabling persistent temporal tracking and dynamic correction during long-horizon execution. We evaluate the proposed method in multiple simulation environments and conduct sim-to-real experiments on a real robotic platform. The results demonstrate that explicit language memory improves the success rate and robustness of VLA models on complex long-horizon tasks while providing an interpretable semantic account of the decision process.
41. 【2608.04764】Splat-Based Metal Artifact Reduction in Cone-Beam CT via Compact Attenuation Modeling
链接：https://arxiv.org/abs/2608.04764
作者：Kiseok Choi,Jaemin Cho,Inchul Kim,Min H. Kim
类目：Computer Vision and Pattern Recognition (cs.CV); Graphics (cs.GR)
关键词：X-ray computed tomography, X-ray computed, computed tomography, suffers from severe, high-attenuation objects
备注：
点击查看摘要
Abstract:X-ray computed tomography (CT) suffers from severe metal artifacts when high-attenuation objects such as dental fillings or orthopedic implants are present. These artifacts originate from the polychromatic nature of X-rays, where attenuation varies strongly with photon energy and material composition, breaking the monochromatic assumption used by conventional reconstruction algorithms. Recent neural rendering approaches attempt to address this mismatch through differentiable polychromatic projection models, but they still struggle with smoothness bias, loss of fine structures, and prohibitive computation when extended to large-scale cone-beam CT. We introduce a splat-based metal artifact reduction framework that incorporates a physically grounded polychromatic forward model into a continuous Gaussian representation for cone-beam CT. Each Gaussian encodes the energy-dependent attenuation of the underlying material using a compact material parameterization, which enables efficient joint optimization of geometric and material properties without relying on a metal mask. This compact attenuation formulation captures the essential variation across biological tissues and metallic implants, allowing our model to explain metal-induced nonlinearity while preserving high-frequency structure. Experiments on simulated and real cone-beam CT scans show that our method converges significantly faster and suppresses metal artifacts more effectively than existing reconstruction and neural field-based approaches.
42. 【2608.04759】race, Verify, and Correct: A Training-Free Framework for Spatial Reasoning in Multimodal LLMs
链接：https://arxiv.org/abs/2608.04759
作者：Yang Yang,Jiawei Chen,Tairan Chen,Zhaoxia Yin
类目：Computer Vision and Pattern Recognition (cs.CV); Computation and Language (cs.CL)
关键词：Multimodal Large Language, Large Language Models, Multimodal Large, Large Language, made substantial progress
备注： 19 pages, 7 figures
点击查看摘要
Abstract:Although Multimodal Large Language Models (MLLMs) have made substantial progress, their spatial reasoning may still produce intermediate judgments inconsistent with the input image, allowing errors to propagate through the reasoning chain and affect the final answer. Existing methods mainly improve spatial reasoning through training or additional spatial information, without considering whether the reasoning process itself is faithful to the model input. Our study shows that unfaithful reasoning chains significantly reduce final-answer accuracy. To address this issue, we propose a modular and training-free framework for spatial reasoning verification and correction. The framework constructs a Spatial Evidence Graph (SEG), which associates atomic spatial evidence extracted from Chain-of-Thought reasoning with visual entities, spatial relations, source steps, and visual evidence. Spatial Evidence Reliability Assessment (SERA) evaluates the reliability of visual evidence based on object existence, localization, and geometric measurements. The framework then identifies the earliest spatial evidence unit contradicted by reliable visual evidence and guides the original MLLM to revise the subsequent reasoning and final answer. Across 15 model-dataset settings, our method achieves an average accuracy of 68.94%, outperforming the compared baselines by 8.55 percentage points on average. Our code will be open-sourced.
43. 【2608.04752】Revisiting Pose Sensitivity in Splat-based Computed Tomography under Sparse-view Reconstruction
链接：https://arxiv.org/abs/2608.04752
作者：Kiseok Choi,Hyeongjun Cho,Inchul Kim,Min H. Kim
类目：Computer Vision and Pattern Recognition (cs.CV); Graphics (cs.GR)
关键词：X-ray computed tomography, reconstructs volumetric representations, X-ray computed, transmitting X-rays, projection images obtained
备注：
点击查看摘要
Abstract:X-ray computed tomography (CT) reconstructs volumetric representations of objects from projection images obtained by transmitting X-rays through a target. Recent splat-based tomography, which represents a volume as a continuous distribution of 3D Gaussians, has demonstrated both high reconstruction quality and fast convergence in cone-beam sparse-view CT. However, when deployed in real CT systems with limited and non-uniform view distributions, we observe distinctive streak and strip artifacts that are far more pronounced than in conventional reconstruction methods. Through detailed analysis, we show that these artifacts primarily originate from pose inaccuracies in the acquisition geometry rather than from view sparsity itself. We revisit pose sensitivity in the splatting formulation and derive a stable gradient-based framework that jointly refines geometric parameters during reconstruction. Our study not only identifies how pose perturbations propagate through the differentiable projection operator but also reveals why splat-based CT is particularly vulnerable to geometric misalignment. The resulting formulation remains lightweight and easily integrable into existing pipelines while substantially improving reconstruction fidelity under real-world sparse-view conditions.
44. 【2608.04750】Simile Understanding in Text-to-Image Models: An Evaluation Framework
链接：https://arxiv.org/abs/2608.04750
作者：Luecheng Wang,Shintaro Ozaki,Hidetaka Kamigaito,Katsuhiko Hayashi,Jingun Kwon,Manabu Okumura,Taro Watanabe
类目：Computer Vision and Pattern Recognition (cs.CV); Computation and Language (cs.CL); Multimedia (cs.MM)
关键词：describe visual characteristics, provide a compact, compact and expressive, models, describe visual
备注： Accepted as a full paper at ACM Multimedia 2026
点击查看摘要
Abstract:Similes provide a compact and expressive way to describe visual characteristics in text prompts. Recent text-to-image models (t2i models) can produce visually compelling outputs from simile prompts, yet even frontier models frequently misinterpret the metaphorical vehicle and confuse it with the object. These systematic failures reveal a gap between figurative language and object-level visual grounding in t2i models. To investigate this issue, we propose a scalable evaluation framework for simile understanding. Our framework includes (1) a controlled simile dataset in which metaphorical vehicles are drawn from a predefined set of object-detectable categories and combined with diverse templates, (2) automatic grounding metrics based on YOLO (You Only Look Once) detection, and (3) text encoder layer analysis using Diffusion Lens to track how metaphorical vehicles emerge during generation. Experiments across architecturally diverse t2i models reveal consistent literalization failure patterns. We further discuss potential mitigation strategies for improving simile grounding in t2i models.
45. 【2608.04737】Dense Metric Depth Completion from Sparse Direct Time-of-Flight Sensors
链接：https://arxiv.org/abs/2608.04737
作者：Hakyeong Kim,Ruicheng Wang,Chengtang Yao,Jiaolong Yang,Min H. Kim
类目：Computer Vision and Pattern Recognition (cs.CV); Graphics (cs.GR)
关键词：dense metric depth, provide highly accurate, highly accurate metric, challenging real-world conditions, metric depth
备注：
点击查看摘要
Abstract:Direct Time-of-Flight (dToF) sensors provide highly accurate metric depth and are more robust than indirect ToF systems in challenging real-world conditions. However, their high manufacturing cost and limited photodiode array size produce depth maps that are extremely sparse, low-resolution, and noisy, making them unsuitable for VR/XR, robotics, and 3D perception tasks that require dense metric depth. Existing monocular and depth completion methods struggle to handle the unique sampling patterns and hardware artifacts of dToF devices, and their performance often deteriorates significantly under severe sparsity or noise. We present a generalizable framework for dense metric depth completion from sparse dToF measurements, capable of operating across diverse sensor types, sparsity levels, and noise conditions. Our model employs a depth-guided dual-branch Vision Transformer encoder that processes RGB images and sparse dToF measurements separately, while a masked joint attention module allows depth tokens to reliably guide image features without being overwritten by them. A lightweight decoder reconstructs dense metric depth efficiently, without diffusion-based or refinement-heavy post-processing. To address the scarcity of paired training data, we introduce a comprehensive dToF simulation pipeline that reproduces the characteristics of flash, sub-VGA flash, and rotating sensors, including hardware-induced degradation, irregular sparsity, and realistic noise distributions. Trained entirely on synthetic data, our model achieves strong zero-shot generalization across 6 datasets and 3 real dToF devices, outperforming state-of-the-art approaches in both accuracy and computational efficiency. This establishes a robust and practical solution for dense metric depth completion from sparse direct ToF sensors. Our code and models are open-sourced. See this https URL.
46. 【2608.04726】When Prompts Become Pixels: Prompt-Region Grounding for Multimodal Reasoning
链接：https://arxiv.org/abs/2608.04726
作者：Yongxin Wang,Ruizhe Zhou,Yueling Tang,Yingying Zhu,Xuemin Zhao,Xiaojun Chang,Xiaodan Liang
类目：Artificial Intelligence (cs.AI); Computer Vision and Pattern Recognition (cs.CV)
关键词：Multimodal large language, Multimodal large, language models increasingly, models increasingly reason, large language models
备注：
点击查看摘要
Abstract:Multimodal large language models increasingly reason over screenshots and documents where the task itself may be written in pixels. Yet benchmarks usually place questions in text, leaving it unclear whether models use the same instruction equally well across channels. We introduce Visualized Task Semantics (VTS), a controlled intervention that moves the question into the image while keeping the source problem and answer fixed. Across six MLLMs and four benchmarks, accuracy drops in all 24 model-task pairs, by 17.8 points on average. Models often transcribe the visual question correctly yet fail to use it, exposing a semantic channel gap beyond OCR. To reduce this gap, we present prompt-region grounding, whose core design aligns the question region with typed semantics and recovers its clean representation from a masked view. At matched training cost, our method raises four-benchmark VTS accuracy from 58.0 to 66.3 while preserving accuracy on the original interface, and requires no OCR or region metadata at inference. Reading task-bearing text and grounding it as an instruction for reasoning are distinct capabilities.
47. 【2608.04724】A GitOps-Driven Annotation Catalog for Fully Automatic Railway Operations
链接：https://arxiv.org/abs/2608.04724
作者：Martin Köppel,Tobias Cronauer,Zekiye Ilknur-Öz,Sebastian Dubiel,Patrick Naumann,Philipp Neumaier
类目：Robotics (cs.RO); Computer Vision and Pattern Recognition (cs.CV)
关键词：Automatic train operation, requires robust AI-based, robust AI-based perception, AI-based perception systems, perception systems capable
备注：
点击查看摘要
Abstract:Automatic train operation (ATO) at grade of automation 3 and above (GoA3-GoA4) requires robust AI-based perception systems capable of reliably detecting obstacles and railway-specific objects under real-world conditions. The effectiveness of these modern artificial intelligence approaches depends heavily on large-scale, high-quality, and highly dynamic annotated datasets. However, managing metadata, maintaining provenance, and tracking the iterative evolution of these annotations impose significant infrastructural and regulatory requirements. Existing monolithic data catalogs often suffer from massive operational overhead, poor integration into developer workflows, and severe documentation drift. This paper introduces an innovative, lightweight GitOps-based architecture for metadata management. By leveraging Data-as-Code principles, Continuous Integration/Continuous Deployment (CI/CD) pipelines, and Static Site Generation (SSG), the proposed approach establishes a seamless, developer-centric workflow. This ensures an traceability, enforces strict regulatory compliance, and automatically generates a highly performant dataset overview.
48. 【2608.04722】Multi-View Face and Gesture Animation with Dynamic Gaussians
链接：https://arxiv.org/abs/2608.04722
作者：Alireza Javanmardi,Vippin Kumar Jeetmal,Christen Millerdurai,Alain Pagani,Didier Stricker
类目：Computer Vision and Pattern Recognition (cs.CV); Graphics (cs.GR)
关键词：Creating photorealistic, motion remains challenging, remains challenging, hand, Creating
备注： Accepted at SCA 2026
点击查看摘要
Abstract:Creating photorealistic 3D human avatars with realistic upper-body motion remains challenging. Existing approaches either focus on the head and overlook hand gestures, or reconstruct the full body but fail to preserve fine-grained facial fidelity and hand pose accuracy. As a result, current methods struggle to capture the subtle dynamics of facial expressions and hand gestures that are crucial for natural human communication. While methods based on full-body parametric models enable avatar reconstruction from monocular or multi-view inputs, they often lack accurate facial animation and detailed hand articulation. To address these limitations, we propose MVFGA, a novel multi-view-consistent pipeline for generating realistic upper-body avatars. Our approach models the face and hands separately and fuses them with a parametric upper-body mesh model, enabling the capture of fine-grained facial expressions and hand poses for accurate upper-body avatar reconstruction. We then splat 3D Gaussians onto the obtained mesh, enabling high-quality rendering of dynamic avatars from novel viewpoints. Furthermore, we introduce MVFGA-MoCap, a multi-view upper-body motion capture dataset featuring controlled facial expression sequences, diverse hand gestures, and free-form communication. Experiments show that MVFGA generates visually realistic avatars with high-fidelity facial expressions and hand motions, outperforming baselines for upper-body avatar animation. Project page: this https URL
49. 【2608.04720】YOLOv14:Unified Cross-Domain Real-Time Object Detectionwith Adaptive Multi-View Representation
链接：https://arxiv.org/abs/2608.04720
作者：Jinling Jia,Jian Lu,Jone Yawl,Chenbin Zhang
类目：Computer Vision and Pattern Recognition (cs.CV)
关键词：Real-time object detectors, object detectors achieve, detectors achieve remarkable, achieve remarkable accuracy, Real-time object
备注：
点击查看摘要
Abstract:Real-time object detectors achieve remarkable accuracy under controlled conditions, yet degrade sharply on non-ideal inputs: fisheye distortion, game-rendered characters, aerial viewpoints, and 360° panoramas. We present YOLOv14, aunified detection framework addressing these challenges through four synergisticinnovations. (1) Deformable Area-Attention (D-AAttn) replaces rigid attentiongrids with learned 2D deformation fields, enabling adaptive sampling under geometric distortion. (2) Game2Real Domain Adaptation aligns rendered-game and photographic feature distributions via Adaptive Instance Normalization (AdaIN)and adversarial domain confusion, allowing game characters are detected as realhumans. (3) Multi-View Conditioning injects learned viewpoint embeddings intothe backbone with a cross-view contrastive loss that pulls same-class features fromdifferent perspectives closer. (4) An Adaptive Augmentation Policy automaticallyclassifies each input' scene type and routes to optimal augmentations, while a DynamicScaleRouter learns per-input feature pyramid weights. Together, YOLOv14achieves 49.1 mAP on COCO val2017 at 2.91 ms (T4 GPU), and delivers substantial gains on fisheye (+4.1 mAP), panorama (+6.6 mAP), drone (+6.4 mAP), andgame-character (+26.1 mAP) benchmarks
50. 【2608.04704】A Multi-Sensor Dataset for Monitoring the Operational Environment of Rail Vehicles
链接：https://arxiv.org/abs/2608.04704
作者：Claudio Diotallevi,Rodrigo Gudiño,Zaharia Pachalieva,Philipp Neumaier,Patrick Naumann,Erik Bochinski,Volker Eiselein,Martin Köppel
类目：Computer Vision and Pattern Recognition (cs.CV); Robotics (cs.RO)
关键词：fully automated operation, Grades of Automation, covering all Grades, Reliable environment monitoring, automated operation
备注：
点击查看摘要
Abstract:Reliable environment monitoring is essential for the safe and efficient operation of automated railway systems, covering all Grades of Automation (GoA), from partially automated (GoA2) to fully automated operation (GoA4). Artificial Intelligence (AI) plays a central role in enabling these systems to detect, classify, and react to potential hazards in real time. The development of such AI-based perception systems requires large volumes of accurately annotated data for training and validation. Within the Digitale Schiene Deutschland (DSD) program, DB InfraGO AG and understandAI GmbH have developed a comprehensive multi- sensor dataset tailored to the needs of railway environment perception. This dataset contains over 7 million high-quality annotations of both railway-specific and general perception objects, captured under varying operational scenarios. The finalized dataset can now be requested at the DB InfraGO AG and serve as a valuable resource for advancing AI-driven environment monitoring in the railway domain.
Subjects:
Computer Vision and Pattern Recognition (cs.CV); Robotics (cs.RO)
Cite as:
arXiv:2608.04704 [cs.CV]
(or
arXiv:2608.04704v1 [cs.CV] for this version)
https://doi.org/10.48550/arXiv.2608.04704
Focus to learn more
arXiv-issued DOI via DataCite (pending registration)
Journalreference:
9th International Conference on Intelligent Traffic and Transportation, Amsterdam, Netherlands, September, 2025
51. 【2608.04702】Design Choices That Matter: A Functional ANOVA Analysis for Remote Sensing Multi-Label Classification
链接：https://arxiv.org/abs/2608.04702
作者：Maryam Gholami Shiri,Eva Tuba,Sašo Džeroski,Tome Eftimov,Ana Nikolikj
类目：Machine Learning (cs.LG); Artificial Intelligence (cs.AI); Computer Vision and Pattern Recognition (cs.CV)
关键词：Benchmarking deep learning, remote sensing images, typically yields rankings, Benchmarking deep, multi-label classification
备注： To appear at Discovery Science 2026
点击查看摘要
Abstract:Benchmarking deep learning (DL) models for multi-label classification (MLC) of remote sensing images (RSI) typically yields rankings that do not generalize beyond the evaluated datasets. In this work, we move beyond rankings by employing functional analysis of variance (fANOVA) to systematically quantify the contributions of individual design choices and their interactions to performance variability. We conduct two empirical analyses covering 48 and 20 DL models, respectively, spanning design choices such as network architecture, fine-tuning strategy, learning strategy, and initialization. By applying fANOVA across seven MLC RSI datasets, we construct dataset meta-representations that capture design-choice sensitivity profiles. Hierarchical clustering of these meta-representations reveals that datasets naturally group according to how they respond to design decisions, with patterns strongly linked to intrinsic dataset properties such as scale, spatial resolution, and label space complexity. Our findings show that for large-scale datasets, fine-tuning strategy and architecture are dominant factors, while in data-limited regimes, initialization becomes decisive. For intermediate regimes, the interaction between architecture and learning strategy governs performance.
52. 【2608.04701】UniWorld-View: Large-Baseline View Synthesis via Video Diffusion Models
链接：https://arxiv.org/abs/2608.04701
作者：Haiyang Zhou,Wangbo Yu,Chaoran Feng,Xunyu Zhou,Yonghong Tian,Li Yuan
类目：Computer Vision and Pattern Recognition (cs.CV)
关键词：immersive content creation, enhance user experiences, greatly enhance user, casually captured monocular, content creation
备注： Project Homepage: [this https URL](https://zhouhyocean.github.io/uniworld-view/) Code: [this https URL](https://github.com/PKU-YuanGroup/UniWorld-View)
点击查看摘要
Abstract:The abundance of casually captured monocular videos and images on social media provides a valuable source for immersive content creation, where generating novel views from such sparse observations can greatly enhance user experiences. However, producing photorealistic and geometrically consistent views with precise camera control remains challenging when input coverage is extremely limited. Reconstruction-based approaches such as NeRF and 3D Gaussian Splatting (3DGS) deteriorate severely under sparse inputs and fail to explicitly handle occlusions. Generative methods ease data requirements but still struggle with large-baseline view synthesis due to inaccurate or implicit geometric guidance. To overcome these limitations, we introduce UniWorld-View, a unified framework for controllable large-baseline novel view synthesis from monocular inputs. UniWorld-View integrates explicit 3D guidance with generative diffusion modeling to enable precise camera control and geometrically consistent view generation. The geometric guidance is obtained through an occlusion-aware point cloud rendering strategy that resolves visibility ambiguities and provides accurate priors for diffusion-based synthesis. By coupling this rendering strategy with powerful video diffusion backbones, UniWorld-View achieves high-fidelity novel view generation even under extreme camera motions and wide-baseline changes, and can further provide multi-view videos for downstream dynamic 3DGS reconstruction. Experiments on the WorldScore benchmark and zero-shot NVS benchmarks demonstrate the effectiveness of UniWorld-View in controllability, geometric consistency, and visual fidelity.
53. 【2608.04698】aching MLLMs to Say No: Generalized Referring Expression Comprehension via Refusal Calibrated GRPO
链接：https://arxiv.org/abs/2608.04698
作者：Xuzheng Yang,Jun Ling,Tao Huang,Caiyan Qin,Peng Wang
类目：Computer Vision and Pattern Recognition (cs.CV); Artificial Intelligence (cs.AI)
关键词：Referring Expression Comprehension, Generalized Referring Expression, Generalized Referring, Expression Comprehension, Referring Expression
备注：
点击查看摘要
Abstract:We tackle the challenging yet underexplored task of Generalized Referring Expression Comprehension (GREC), which requires a model to localize the object described by a textual expression when it exists (positive sample) and to refuse output when it does not (negative sample). Although Multimodal Large Language Models (MLLMs) excel at localizing existing objects, they often fail to reject nonexistent ones due to the absence of negative samples during training, producing hallucinated bounding boxes. Existing post-training approaches such as supervised fine-tuning (SFT) and reinforcement learning (RL) enhance refusal behavior but usually degrade localization accuracy on positive samples, undermining the model's core competence. To address this, we propose Refusal-Calibrated Group Relative Policy Optimization (RC-GRPO), a calibrated RL strategy that strengthens the refusal ability of MLLMs while preserving localization performance. It enforces "None" outputs in rollouts for valid advantage estimation on negative samples and applies a penalty to prevent over-refusal on positives, achieving a balanced trade-off between accuracy and reliability. A second-stage reasoning reinforcement further consolidates causal understanding and interpretability. Experiments on three GREC benchmarks demonstrate that RC-GRPO attains superior localization accuracy while maintaining strong refusal capability.
54. 【2608.04680】MOAT: Model-Agnostic Randomized Transformations for preventing Efficiency Degradation Attacks on ViTs
链接：https://arxiv.org/abs/2608.04680
作者：Anadi Goyal,Nandish Chattopadhyay,Chandan Karfa,Anupam Chattopadhyay,Norrathep Rattanavipanon
类目：Cryptography and Security (cs.CR); Computer Vision and Pattern Recognition (cs.CV)
关键词：Vision Transformers, reduce computational cost, adopt the Vision, token pruning, resource-constrained environment
备注： This paper has been accepted for publication at IEEE ISVLSI 2026
点击查看摘要
Abstract:To adopt the Vision Transformers (ViTs) in resource-constrained environment, token pruning is widely used to reduce computational cost without impacting accuracy. However, adversaries have developed targeted attacks against said token pruning techniques to undermine such attempts to make ViTs efficient. In this paper, we propose MOAT, a model-agnostic pre-processing defense pipeline that applies a combination of input transformations to protect efficient ViT implementations against adversarial efficiency attacks. MOAT operates directly on the input without requiring modifications to the model architecture or token pruning mechanism. Experimental results demonstrate that, across all evaluated ViT models, MOAT limits GFLOPs degradation under adversarial attacks to within 3.4% of the original unattacked model.
55. 【2608.04676】SurgNarrator: A Generative Retrieval Framework for Surgical Video Understanding
链接：https://arxiv.org/abs/2608.04676
作者：Yuqing Feng,Jiawei Ma,Kevin Qinghong Lin,Kun Yuan,Nicolas Padoy,Daniel S. Elson,Anh Nguyen,Stamatia Giannarou,Baoru Huang
类目：Computer Vision and Pattern Recognition (cs.CV)
关键词：recurring clinical events, intraoperative surgical videos, intraoperative decision-making, unfold as structured, structured and recurring
备注： This work has been submitted to the IEEE for possible publication. Copyright may be transferred without notice, after which this version may no longer be accessible
点击查看摘要
Abstract:Surgical procedures unfold as structured and recurring clinical events, whose real-time understanding via intraoperative surgical videos is critical for intraoperative decision-making and support. However, existing video understanding methods force a trade-off: autoregressive video-language models support comprehensive reasoning but are not practical for time-sensitive clinical applications, whereas contrastive models offer low latency but struggle with complex scene understanding. Recently, generative retrieval has been explored for general-domain video understanding, but transferring it to surgery is not trivial because near-identical visual appearances may indicate semantically distinct events, and the terminology involved is highly surgery-specific. To this end, we propose SurgNarrator, a new generative retrieval framework tailored for surgical video understanding. We construct a well-curated surgery-centric vocabulary from surgical captions to define a clinically meaningful retrieval space. We then adapt the pre-trained Qwen3-VL-Embedding-8B to learn discriminative clinical representations with a temporally-aware contrastive objective. During inference, a hierarchical, procedure-aware retrieval strategy narrows the search space to the relevant procedure type, delivering fast and effective responses. Our method is comprehensively evaluated on twelve benchmarks in a zero-shot setting and achieves consistent performance gains over state-of-the-art baselines, while reducing output-stage latency by more than two orders of magnitude compared with the generative baseline.
56. 【2608.04673】Differential 6-DOF Pose Estimation with Provable First-Order Immunity to Camera Calibration Errors
链接：https://arxiv.org/abs/2608.04673
作者：Yueqiang Zhang,Liang Deng,Yi Zhang,Baoqiong Wang,Wenjun Chen,Shuixin Pan,Yulan Guo,Qifeng Yu
类目：Computer Vision and Pattern Recognition (cs.CV); Robotics (cs.RO)
关键词：structural displacement monitoring, robotic manipulation, essential for robotic, recover platform motion, recovers platform motion
备注： 16 pages, 15 figures
点击查看摘要
Abstract:Accurate six-degree-of-freedom (6-DOF) motion estimation is essential for robotic manipulation, autonomous systems, and structural displacement monitoring. Conventional 3D-2D methods estimate absolute camera poses independently at each time and recover platform motion through camera-to-platform extrinsics, making them sensitive to extrinsic calibration errors, especially for micromotion. We present a differential pose estimation method that directly recovers platform motion from inter-frame image displacements and known 3D control points. By differencing perspective projection equations, using a depth-invariance approximation, and modeling motion on SE(3), the method avoids independent absolute-pose estimation and supports both monocular and multi-camera systems. We prove that translational extrinsic errors cancel exactly, while rotational errors induce a bounded perturbation determined by calibration error, motion magnitude, and observation geometry. We also derive generic observability conditions, a Cramer-Rao lower bound, and a bias-eliminated consistent estimator, and characterize the validity limits of the approximations. Extensive synthetic and real-world experiments establish a new state of the art for 6-DOF platform micromotion estimation, outperforming representative PnP and generalized-PnP methods in accuracy, calibration robustness, and computational efficiency. With five control points and 0.5-pixel image noise, the monocular solver obtains a combined pitch-yaw rotation RMSE of 10.09 arcsec, a translation RMSE of 3.70 mm, and a runtime of 0.34 ms. The binocular solver achieves a rotation RMSE of 10.58 arcsec, a translation RMSE of 3.91 mm, and a runtime of 0.27 ms. Code will be released upon publication at this https URL.
57. 【2608.04657】MobileWAM: Bridging World Action Models to Mobile Manipulation with Chain-of-Foresight
链接：https://arxiv.org/abs/2608.04657
作者：Zehua Fan,Junjie He,Wenxuan Song,Xi Wang,Wenqi Lyu,Linge Zhao,Fuhao Li,Zihan You,Yifei Yang,Kaiming Xu,Qi Jiang,Yue Jiang,Haoang Li,Cheng Chi,Bailin Li,Yan Wang
类目：Computer Vision and Pattern Recognition (cs.CV)
关键词：World action models, World action, robot learning, rising recipe, recipe for robot
备注：
点击查看摘要
Abstract:World action models (WAMs) built on video generation backbones are a rising recipe for robot learning, yet remain confined to tabletop manipulation. Mobile manipulation demands simultaneous locomotion and whole-body manipulation amid scene-scale dynamics, yet is still dominated by dynamics-blind visual encoders with hand-crafted coordination. We bridge this gap with MobileWAM, a mixture-of-transformers architecture that fuses a pretrained video diffusion transformer with a lightweight action expert through layerwise joint attention, translating internet-scale motion priors into whole-body control. To reconcile the heterogeneous dynamics of moving and manipulating, each feed-forward layer of the action expert becomes a three-expert mixture of shared, locomotion, and manipulation experts, softly routed by the motion intent in the action tokens. To densify supervision, we further propose Chain-of-Foresight (CoF): intermediate representations sequentially predict a chain of future latent chunks, each step conditioned on its predecessor. CoF pairs naturally with our decoupled video--action denoising scheme. At deployment, the WAM serves as a pure current-frame encoder; foresight acts only through gradients, so at inference the foresight chain and video generation are discarded, leaving only policy-level cost. MobileWAM surpasses state-of-the-art mobile manipulation policies on ManiSkill-HAB and fine-tunes to a real ARX Lift2 mobile manipulator across diverse tasks with strong generalization. Code will be released soon.
58. 【2608.04655】CSGen: A Multi-Domain Curvilinear Structure Generation Model via Hierarchical Multimodal Diffusion
链接：https://arxiv.org/abs/2608.04655
作者：Zhe Shan,Ziming Yang,Lei Zhou,Wenwen Zhang,Cong Lin,Xia Xie
类目：Computer Vision and Pattern Recognition (cs.CV); Artificial Intelligence (cs.AI)
关键词：important and fundamental, fundamental task, Curvilinear, Curvilinear structure, curvilinear structures
备注： Accepted to ACM MM 2026
点击查看摘要
Abstract:Curvilinear structure analysis is an important and fundamental task in multimedia. However, the controllable generation of images with precise curvilinear structure objects remains an open challenge. To address this, we propose CSGen, a hierarchical multimodal diffusion model that synthesizes high-fidelity images precisely aligned with multiple control conditions. The CSGen is built upon three key innovations: 1) We construct a multi-domain and multimodal dataset, including over 24K samples from 5 domains and 7 different types of annotations, to train the unified generation model. 2) We propose a novel hierarchical progressive control strategy that decouples topology clues from visual context by a phased signal injection, mitigating semantic drift while ensuring the topological integrity of sparse structures. 3) We design a sparsity-aware loss re-weighting mechanism to address the extreme sparsity of curvilinear structures, significantly enhancing the attention on thin and fragile structures during optimization. Extensive experiments demonstrate that CSGen generates images with superior structure accuracy and visual realism, significantly improving downstream segmentation performance while maintaining robustness across diverse prompts. Our results confirm CSGen as a scalable, data-centric paradigm for the analysis of complex curvilinear structures in diverse multimedia applications. Code and dataset are available at this https URL.
59. 【2608.04653】Overcoming Statistical Bias in Action-Controllable World Models
链接：https://arxiv.org/abs/2608.04653
作者：Yuhong Shi,Zhenhao Chu,Jie Wei,Jun Hao,Jianyi Liu,Jingwen Fu
类目：Computer Vision and Pattern Recognition (cs.CV); Robotics (cs.RO)
关键词：visual environments evolve, world models aim, aim to predict, environments evolve, Action-conditioned world models
备注：
点击查看摘要
Abstract:Action-conditioned world models aim to predict how visual environments evolve under an agent's actions. Yet future frames are often highly predictable from visual inertia and recurring motion patterns alone. This creates a shortcut: models can fit the data by exploiting statistical biases without making their visible dynamics meaningfully depend on the action. As a result, different actions may produce similar futures, while motion may persist even under zero action. The key question is how to reduce reliance on statistical shortcuts from dominating action-conditioned prediction. We argue that action control requires more than injecting action features; it requires enforcing consistency under counterfactual changes to actions and observations. Based on this insight, we introduce CoCo, a Counterfactual Consistency framework to enhance action controllability through two complementary constraints. Multi-step counterfactual consistency constrains reference, inverse-action, and zero-action rollouts, while action-spatial counterfactual consistency enforces consistent predictions under mirrored scenes and transformed actions. Together, they reduce reliance on statistical shortcuts from substituting for action-dependent dynamics. We further introduce Action Response Consistency (ARC) and Drift Energy (DE) to assess action controllability, together with Mini-SSMB for same-state, multi-action counterfactual evaluation. On Mini-SSMB, our full model achieved ARC_inv of 0.412 and ARC_ref of 0.483, while reducing DE by 17.07% relative to the baseline. On VP2 visual planning, it achieves the highest average success rate among SOTA models, at 73.1%. Experiments on BAIR and RoboNet further show that these gains preserve video prediction quality and transfer across model settings.
60. 【2608.04652】DisMix: Order-Aware Mixup for Medical Imaging via Disentangling Ordinal and Non-Ordinal Features
链接：https://arxiv.org/abs/2608.04652
作者：Dileepa Pitawela,Gustavo Carneiro,Hsiang-Ting Chen
类目：Computer Vision and Pattern Recognition (cs.CV); Artificial Intelligence (cs.AI)
关键词：data augmentation strategy, widely adopted data, adopted data augmentation, augmentation strategy, ordinal classification tasks
备注：
点击查看摘要
Abstract:Image mixup is a widely adopted data augmentation strategy, yet it is ill-suited for ordinal classification tasks such as medical disease grading, where labels encode a progression of severity. By indiscriminately blending disease-severity cues (ordinal) with appearance-level variation (non-ordinal), standard mixup produces samples that distort the very ordinal structure that underpins clinical severity grading. We introduce DisMix, an order-aware mixup framework for ordinal classification. DisMix disentangles ordinal and non-ordinal features via a dual-codebook VQ-VAE, allowing each subspace to be mixed independently: ordinal codes are interpolated to produce meaningful intermediate ranks, while non-ordinal codes are varied to introduce appearance diversity without corrupting the ordinal signal. Across four medical imaging datasets, DisMix shows the best aggregate performance among six image mixup baselines paired with six ordinal classifiers and remains effective under data scarcity and clinical grading variability.
61. 【2608.04642】YOLO-PVC: 2D-to-3D Consolidation of Slice-wise Detections for Volumetric Liver Tumor Localization in MRI
链接：https://arxiv.org/abs/2608.04642
作者：Talha Waqas,Mounir Lahlouh,Kawther Taibouni,Mahnoor Waqas,Salar Ahmed,Sébastien Mulé,Yasmina Leroul-Chenoune
类目：Computer Vision and Pattern Recognition (cs.CV)
关键词：volumetric data due, object detectors, efficiency and scalability, detectors are increasingly, increasingly applied
备注： 14 pages, 2 figures, 4 tables. Accepted at AI4M3D Workshop, ECCV 2026 (Spotlight)
点击查看摘要
Abstract:Slice-wise 2D object detectors are increasingly applied to volumetric data due to their computational efficiency and scalability, yet they often yield fragmented and unstable predictions along the depth axis. We propose YOLO-PVC, a lightweight and model-agnostic framework for 2D-to-3D consolidation of slice-wise detections. The method enforces depth continuity, aggregates bounding box coordinates using robust percentile statistics, and further refines axial extent through a lightweight MLP-based calibration module. Unlike naïve stacking or averaging strategies, YOLO-PVC explicitly addresses missing detections and outlier slices along the depth dimension. Experiments on 3D liver MRI volumes across three tumor categories demonstrate consistent improvements over multiple aggregation baselines. The heuristic PVC achieves an overall $\mathrm{IoU}_{3D}$ of $0.665$, while the calibrated variant further improves performance to $0.710$, with high planar overlap ($\mathrm{BEV\ IoU} \approx 0.78$). These results demonstrate that structured geometric consolidation provides an effective and practical solution for volumetric liver tumor localization in clinical MRI.
62. 【2608.04623】Visual Anchoring in Diffusion: Multimodal Zero-Shot Skeleton Action Recognition
链接：https://arxiv.org/abs/2608.04623
作者：Zehao Bao,Shujun Guo,Bruce X.B. Yu
类目：Computer Vision and Pattern Recognition (cs.CV)
关键词：Skeleton Action Recognition, unseen actions share, actions share similar, Action Recognition, unseen actions
备注：
点击查看摘要
Abstract:Zero-shot Skeleton Action Recognition (ZSAR) remains ambiguous when unseen actions share similar skeleton joint dynamics but differ in objects or scene context. RGB provides these missing cues, yet existing multimodal methods typically maintain independent skeleton and RGB scoring branches and fuse their outputs. Without using unlabeled test data for adaptation or fusion calibration, a fixed fusion weight cannot capture class-pair-dependent modality reliability, while an adaptive rule lacks target-side feedback for deciding which branch should dominate. We bypass this weight-selection problem via the classify-by-generation paradigm, where each class is scored by how accurately a text-conditioned denoiser predicts the noise added to the skeleton feature. This formulation separates the progressively corrupted skeleton from fixed conditioning, allowing RGB and text to jointly condition a single class-scoring function rather than produce independent scores. We instantiate this idea as Multimodal Triplet Diffusion for Skeleton-Text Matching (TDSM-MM), augmenting a text-conditioned denoising Transformer with a non-diffused RGB condition token that serves as a stable visual anchor during skeleton data reconstruction. Our proposed TDSM-MM has been ablated via extensive experiments and achieved the best inductive accuracy on three of four NTU-60/120 splits and surpasses the transductive state-of-the-art on NTU-120 96/24 (i.e., 71.3% vs. 69.1%), without test-time adaptation, suggesting that diffusion-based methods can be a promising direction for zero-shot learning.
63. 【2608.04622】DAC-Pose: Dual-Agent Collaborative Framework for Pose-Guided Human Generation
链接：https://arxiv.org/abs/2608.04622
作者：Haotian Yang,Zhile Yang,Huiyu Zhou,Xin Sun
类目：Computer Vision and Pattern Recognition (cs.CV)
关键词：passive pixel-level mapping, perform complex semantic, pixel-level mapping, passive pixel-level, complex semantic reasoning
备注： Code is available at [this https URL](https://github.com/AIVRC/DAC-Pose)
点击查看摘要
Abstract:AI agents have emerged as a powerful new paradigm in generative image synthesis, enabling systems to perform complex semantic reasoning rather than passive pixel-level mapping. In pose-guided human generation, conventional methods inevitably produce severe visual artifacts under drastic viewpoint shifts, fundamentally because they lack the cognitive capacity to logically deduce unseen regions and model complex spatial deformations. To bridge this gap, we propose DAC-Pose, a novel agent-driven multimodal framework that reformulates single-view human generation as a collaborative dual-agent system. DAC-Pose integrates two complementary components, namely, the Prior Semantic Reasoning (PSR) agent and the Discrepancy-Aware Visual Encoding (DAVE) agent. Functioning as a cognitive engine, PSR utilizes collaborative reasoning to deduce the fine-grained attributes of unseen regions. Concurrently, acting as a specialized visual perception agent, DAVE quantifies and encodes viewpoint-induced spatial misalignments, continuously feeding robust spatial constraints back into the generative process. This autonomous feedback loop between semantic deduction and visual perception ensures high-fidelity detail synthesis. Extensive experiments on the DeepFashion and Market-1501 benchmarks validate the superiority of our agent-driven paradigm. Notably, DAC-Pose excels in preserving texture alignment and identity consistency under drastic viewpoint changes. The code is available at this https URL.
64. 【2608.04610】HiSC: Hierarchical Spatial Clustering Token Compression for Efficient 3D Scene Understanding
链接：https://arxiv.org/abs/2608.04610
作者：Jiuhe Qu,Yingping Liang,Ying Fu
类目：Computer Vision and Pattern Recognition (cs.CV)
关键词：large uninformative regions, high computational cost, computational cost, suffer from substantial, due to duplicated
备注： Accepted by ACM MM 2026
点击查看摘要
Abstract:3D vision-language models (3D VLMs) enable spatial reasoning over multi-view scenes but suffer from substantial token redundancy due to duplicated observations and large uninformative regions, leading to high computational cost. Although visual token compression has shown promise in accelerating 2D VLMs, it fails to capture the structured nature of 3D scenes and leads to incomplete spatial coverage and loss of fine-grained details. In this paper, we propose \textbf{HiSC}, a training-free framework for hierarchical spatial clustering token compression in 3D VLMs. HiSC lifts token compression from token-level selection to cluster-level processing by organizing tokens into spatially grounded clusters using joint geometric and semantic cues. Specifically, we first introduce a \textbf{spatial graph-based merging (SGraM) strategy} that models cross-view redundancy as spatial connectivity and consolidates physically consistent regions, effectively merging extremely similar redundant tokens prior to LLM inference. We then propose a \textbf{spatial clustering-based pruning (SCluP) paradigm} within LLM inference, which performs hierarchical compression across clusters and within clusters, preserving object instance completeness while retaining fine-grained details for important regions. Extensive experiments on diverse 3D reasoning benchmarks show validate the effectiveness of HiSC, particularly under high visual token pruning ratios. Besides, HiSC achieves over 90\% token reduction with minimal performance degradation. Code is accessible at this https URL.
65. 【2608.04606】RCoRSurg: Temporal-Relational Co-Reasoning for Surgical Video Triplet Recognition
链接：https://arxiv.org/abs/2608.04606
作者：Fang Li,Shihao Zou,Weixin Si,Yang Gao,Shuai Li,Aimin Hao
类目：Computer Vision and Pattern Recognition (cs.CV)
关键词：Understanding complex surgical, multiple interdependent entities, scenes requires recognizing, requires recognizing multiple, recognizing multiple interdependent
备注： code: [this https URL](https://github.com/Neesky/TRCoRSurg)
点击查看摘要
Abstract:Understanding complex surgical scenes requires recognizing multiple interdependent entities, such as instruments, actions, and targets, while maintaining their relational consistency across time. Existing surgical triplet recognition methods struggle to jointly model intra-frame label dependencies and inter-frame temporal semantics in a unified manner. To address these limitations, we propose a unified framework that integrates spatial, relational, and temporal cues for robust surgical triplet recognition. Specifically, class-specific spatial priors are first extracted through a multi-scale encoder. These priors are then refined by a Label Correlation Modeling module with multi-scale class activation map-guided relational extraction (MS-CAMRE), enabling the model to capture both static co-occurrence patterns and dynamic contextual dependencies among triplet components. Furthermore, a Bidirectional Temporal-Relational Fusion Attention (BTRFA) module harmonizes temporal and relational representations to achieve coherent temporal reasoning. We also introduce a new evaluation metric, the Triplet Consistency Error Rate (TCER), which quantitatively measures the model's ability to preserve causal and semantic consistency across triplets. Extensive experiments on the CholecT45 and ProstaTD datasets show that our method achieves state-of-the-art performance, improving AP_IVT by 5.1 percent and 7.8 percent, respectively. Moreover, according to TCER, our approach achieves relative reductions of more than 36 percent and 25 percent on the two datasets, respectively, demonstrating the effectiveness of our framework in temporal-relational co-reasoning.
66. 【2608.04604】COSMO: Consensus-Driven Shift Modulation for Source-Free Domain Adaptation
链接：https://arxiv.org/abs/2608.04604
作者：Bo Li,Junjie Peng,Xiaohua Xie,Jianhuang Lai
类目：Computer Vision and Pattern Recognition (cs.CV)
关键词：Source-free domain adaptation, Source-free domain, unlabeled target domain, adapts a source-trained, storage constraints
备注： 30 pages, 7 figures
点击查看摘要
Abstract:Source-free domain adaptation (SFDA) adapts a source-trained model to an unlabeled target domain without source data, a practical setting under privacy or storage constraints. Yet its self-generated supervision can reinforce source bias under substantial domain shifts. Pretrained vision-language models (VLMs) offer complementary semantic knowledge, but the relative reliability of the source model and VLM varies across target samples. Existing cross-model guidance does not explicitly account for this variation and may overwrite valid source-derived evidence under conflict, a failure we term source-derived evidence forgetting. We formulate VLM-guided SFDA as a sample-wise reliability-allocation problem and propose Consensus-Driven Shift Modulation (COSMO). COSMO replaces expert-to-expert guidance with co-adaptation through an anchored shared consensus. It first forms a sample-specific initial consensus that favors the more concentrated prediction. During adaptation, COSMO re-aggregates both branches' evolving evidence and regulates how far the resulting consensus moves from its initial anchor based on consensus uncertainty and training progress. This keeps the shared supervision anchored yet adaptive. Across four benchmarks, COSMO achieves state-of-the-art performance under matched VLM backbones. Further analyses indicate that it better balances the retention of valid source-derived evidence with the absorption of complementary VLM evidence.
67. 【2608.04589】he First EgoCross Challenge at EgoVis 2026: Cross-Domain Egocentric Video Question Answering
链接：https://arxiv.org/abs/2608.04589
作者：Yuqian Fu,Tianwen Qian,Yanjun Li,Yu Li,Kunyu Peng,Xu Zheng,Yongqin Xian,Alessio Tonioni,Yanwei Fu,Xiaoling Wang,Danda Paudel,Federico Tombari,Luc Van Gool,Leyi Wu,Yifan Zhao,Jinjie Zhang,Yinchuan Li,Yingcong Chen,Zixu Li,Zhiwei Chen,Zhiheng Fu,Wenbo Wang,Yupeng Hu,Weili Guan,Liqiang Nie,Takuya Murakawa,Toru Tamaki,Yi Wen,Zhenglin Du,Zhengyang Li,Lingling Li,Licheng Jiao,Wenping Ma
类目：Computer Vision and Pattern Recognition (cs.CV); Artificial Intelligence (cs.AI)
关键词：common daily-life scenarios, multimodal large language, large language models, answering benchmark designed, video question answering
备注： 1st EgoCross challenge @ EgoVis workshop, CVPR26
点击查看摘要
Abstract:EgoCross is a cross-domain egocentric video question answering benchmark designed to evaluate whether multimodal large language models can generalize beyond common daily-life scenarios. The first EgoCross Challenge was hosted at the Third EgoVis Workshop at CVPR 2026 and evaluated models on first-person videos from four target domains: surgery, industrial assembly, extreme sports, and animal perspectives. Each test example consists of an egocentric video clip, a question, and four candidate answers, from which the model must select the correct option. This technical report introduces the challenge task, benchmark resources, and two official Codabench tracks. The Source-Limited Track restricts participants to the official baseline model and a small support set, whereas the Open-Source Track permits broader choices of models and training data under rules that prohibit the manual construction of target-domain training data. In total, the challenge received more than 1,500 submissions from over 130 participants, with 19 teams participating in the Open-Source Track and 38 teams in the Source-Limited Track. We further present the official leaderboard results and summarize the winning solutions from both tracks. We hope that this report will serve as a useful technical reference for advancing cross-domain egocentric video understanding. All resources, including the challenge data, baseline implementation, and code released by the winning teams, are made publicly available.
68. 【2608.04587】MetaVideoAgent: Automated Video-Agent Evolution for Long-Form Video Understanding
链接：https://arxiv.org/abs/2608.04587
作者：Benlei Cui,Ruize Wang,Junjie Li,Jinhao Chen,Longtao Huang,Yinghao Chen,Yuwen Zhai,Jingqun Tang,Ruijian Jia,Weiwei Wu,Pengfei Sun,Haiwen Hong
类目：Computer Vision and Pattern Recognition (cs.CV)
关键词：Long-form video understanding, requires locating sparse, understanding requires locating, video understanding requires, Long-form video
备注： 16 pages, 7 figures. Code: [this https URL](https://github.com/Alibaba-VELLDEPTH/MetaVideoAgent)
点击查看摘要
Abstract:Long-form video understanding requires locating sparse, question-relevant evidence in long, multimodal videos. Real-world video distributions differ in modality-specific information density, content structure, and evidence patterns, causing fixed video-agent designs to incur redundant processing or fail when mismatched. Extending automated agent evolution from text to video is challenging because full long-video execution makes candidate validation expensive, failures propagate across coupled evidence-processing stages, and complex preprocessing, perception tools, and localization strategies make code-level updates difficult to implement reliably. We introduce MetaVideoAgent, a framework that automatically evolves a video agent for a target distribution. It profiles information density and evidence requirements from sparsely sampled frames and associated queries to guide initial design, then compresses localized failures into independently executable minimal validation tasks. It constructs evidence-grounded Gold Paths, audits Student trajectories, aggregates recurring failures across samples, and attributes them to responsible modules. A modular agent representation constrains each update to the primary responsible module and its necessary dependencies. We further introduce VA-EvoBench, covering eight video distributions with separate evolution and held-out splits. With four evolution iterations per distribution, MetaVideoAgent improves every initial agent and raises macro-average accuracy from 38.44% to 51.47%, at an average evolution cost of 3.54M tokens per distribution. The evolved agents outperform the strongest prior fixed-design video agent by 6.39 percentage points while using the fewest tokens and video frames per question among the compared video agents. We will release all code and data to support reproducible research.
Comments:
16 pages, 7 figures. Code: this https URL
Subjects:
Computer Vision and Pattern Recognition (cs.CV)
Cite as:
arXiv:2608.04587 [cs.CV]
(or
arXiv:2608.04587v1 [cs.CV] for this version)
https://doi.org/10.48550/arXiv.2608.04587
Focus to learn more
arXiv-issued DOI via DataCite (pending registration)</p>
69. 【2608.04581】ACA-GS: Adaptive-Capacity Anchored Gaussian Splatting for Compact Dynamic Radiance Fields
链接：https://arxiv.org/abs/2608.04581
作者：Seunghyeon Song,Joo Chan Lee,Chanung Park,Jun Young Jeong,Minseo Lee,Eunbyung Park,Jong Hwan Ko
类目：Computer Vision and Pattern Recognition (cs.CV)
关键词：Gaussian Splatting, Recent advances, real-time spatiotemporal rendering, Neural Gaussian, Neural Gaussian counts
备注： 9 pages, 8 figures. Accepted to ACM Multimedia 2026
点击查看摘要
Abstract:Recent advances in 4D Gaussian Splatting (4DGS) enable high-fidelity, real-time spatiotemporal rendering, but expose a fundamental trade-off between motion expressiveness and storage efficiency. While anchor-based designs achieve compactness through anchor-level parameter sharing, their rigid uniform parametrization enforces fixed Neural Gaussian counts and feature budgets per anchor. Consequently, insufficient fidelity is addressed by excessive anchor density, rather than lightweight, targeted increases in Neural Gaussian count or feature capacity, resulting in memory waste. To overcome this rigidity, we introduce an adaptive-capacity anchor-based framework that dynamically allocates the representational capacity based on local spatiotemporal demands. Adaptive Anchor Cardinality varies the number of Neural Gaussians per anchor, concentrating primitives in regions of high geometric or motion complexity while suppressing redundancy. In parallel, Adaptive Anchor Feature Masking modulates anchor-level feature channels, assigning rich features to complex regions and lightweight representations to simpler ones. Experiments on MPEG, Panoptic Sports, and N3DV datasets demonstrate substantial storage reduction without degrading visual quality. Notably, on challenging MPEG sequences with complex motion, our method achieves up to 1.5x higher compression than state-of-the-art anchor-based methods while preserving comparable quality.
70. 【2608.04575】PhysMind: From Video to Executable Worlds for Training-Free Physical Reasoning
链接：https://arxiv.org/abs/2608.04575
作者：Chen Yang,Shenxiang Zeng,Haoyang Zhao,Zhouyuan Xu,Youquan He,Haoyu Li,Mingyi Deng,Jiansheng Fan,Chen Wang
类目：Computer Vision and Pattern Recognition (cs.CV); Artificial Intelligence (cs.AI)
关键词：video requires understanding, Reliable physical reasoning, Reliable physical, respond to interventions, requires understanding
备注： 27 pages, 18 figures. Project page: [this https URL](https://physmind.github.io/)
点击查看摘要
Abstract:Reliable physical reasoning from video requires understanding how objects move, interact, and respond to interventions. Existing vision-language models (VLMs) often struggle to interpret these dynamics and reason reliably about future and counterfactual outcomes. We introduce PhysMind, a training-free agentic framework that constructs one reusable, question-agnostic executable world per video. PhysMind recovers a temporally consistent dynamic scene through object segmentation, mesh reconstruction, and 6D pose tracking, then fits analytic continuous-time dynamics and latent physical parameters without unrolling a time-stepped simulator. Given a question, it inspects, continues, or edits the world and answers from the resulting trajectories and interactions. Relative to direct chain-of-thought (CoT) reasoning with the same VLM, PhysMind improves accuracy by 38.23 points on CLEVRER and 8.08 points on Physion++. On counterfactual questions, it exceeds the strongest evaluated VLM baseline, GPT-5.5, by 19.25 points.
71. 【2608.04568】alk2Sensors: 3D Visual Grounding in Autonomous Driving via Sensor-Adaptive Physical Cue Matching
链接：https://arxiv.org/abs/2608.04568
作者：Runwei Guan,Di Tian,Ningwei Ouyang,Ruixiao Zhang,Shaofeng Liang,Haocheng Zhao,Lianqing Zheng,Xiaokai Bai,Guotao Wang,Daizong Liu,Henghui Ding,Hui Xiong
类目：Computer Vision and Pattern Recognition (cs.CV)
关键词：extensions largely rely, existing outdoor extensions, outdoor extensions largely, scenes with RGB-D, RGB-D or point-cloud
备注： 14 pages, 12 figures
点击查看摘要
Abstract:As a key capability for embodied intelligence, 3D visual grounding (3DVG) has been predominantly studied in indoor scenes with RGB-D or point-cloud inputs, while existing outdoor extensions largely rely on monocular images alone. Both settings fall short of real-world outdoor perception, where heterogeneous sensors capture complementary yet distinct physical properties, such as visual texture, 3D geometry, and object kinematics, that are indispensable for flexible and robust query-adaptive grounding but remain under-exploited. To bridge this gap, we introduce Talk2Sensors, the first multi-sensor 3D visual grounding dataset built upon camera, LiDAR, and 4D radar. It contains 8,682 language instructions and 20,558 referred objects, with diverse prompts explicitly aligned with sensor-specific physical cues. Furthermore, we propose TSFormer, a unified Transformer-based framework for language-guided 3D visual grounding in autonomous driving. TSFormer adopts a coarse-to-fine property-aware fusion strategy: the Language-Routed Property Sampler first performs coarse text-conditioned feature retrieval by modulating sensor sampling weights with query-level linguistic cues, while the subsequent Sparse-Preserving Modality Arbiter module conducts fine-grained modality arbitration and text-guided refinement to determine the precise referred spatial location. This design enables dynamic routing of appearance, geometry, and motion cues according to the semantic requirements of each prompt, preventing dense modalities from overwhelming sparse but critical sensor signals. Extensive experiments demonstrate that TSFormer achieves state-of-the-art performance across multiple benchmarks: it improves over the strongest baseline by 8.05 mAP on Talk2Sensors, and transfers to the monocular Mono3DRefer benchmark with 53.05\% Acc@0.5.
72. 【2608.04560】OutLangSplat: 3D Language Gaussian Splatting for UAV Outdoor Scenes
链接：https://arxiv.org/abs/2608.04560
作者：Xia Yan,He Wu,Yanghui Xu,Zizhao Wu,Jiazhou Chen
类目：Computer Vision and Pattern Recognition (cs.CV); Graphics (cs.GR)
关键词：Gaussian Splatting embeds, Language Gaussian Splatting, Gaussian Splatting, Splatting embeds open-vocabulary, Unmanned Aerial Vehicle
备注： 9 pages, 6 figures, 7 tables
点击查看摘要
Abstract:3D Language Gaussian Splatting embeds open-vocabulary language features into 3D Gaussian Splatting, providing an efficient explicit representation for text-driven 3D scene understanding. However, existing methods are limited to indoor or small-scale scenes, and tend to fail in Unmanned Aerial Vehicle (UAV) outdoor scenes, where severe occlusions and long distance viewpoints often lead to incorrect semantic activations and missing target responses. In this paper, we present OutLangSplat which adapts language Gaussian representations to UAV outdoor scenes by improving feature representation and aggregation reliability. For the feature representation, a 2D-3D dual-branch representation with region-based alignment and fusion is designed to improve spatial consistency, reducing incomplete target responses and background misactivations. For the feature aggregation, we introduce a training-free contribution and consistency-aware Gaussian feature aggregation strategy that leverages pixel contribution reliability and cross-view semantic consistency to suppress unreliable responses from noisy viewpoints. A new dataset is provided by manually annotating various objects on four real-world public UAV outdoor scene datasets. To the best of our knowledge, it is the first accessible dataset of open-vocabulary 3D scene understanding for UAV outdoor scenes. Quantitative evaluations and ablation studies demonstrate that OutLangSplat outperforms SOTA methods on both open-vocabulary semantic segmentation and instance localization tasks. The datasets and codes will be open-sourced.
73. 【2608.04559】ColorFD: A Finite-Difference Guided Black-Box Physical Adversarial Attack for Remote Sensing Object Detection
链接：https://arxiv.org/abs/2608.04559
作者：Tiannuo Guo,Guhang Qiu,Yuzhen Xie,Rui Feng,Ligang Li,Deliang Xiang
类目：Computer Vision and Pattern Recognition (cs.CV)
关键词：deep neural network-based, neural network-based remote, network-based remote sensing, remote sensing object, achieved strong performance
备注： 13pages,12figures
点击查看摘要
Abstract:Although deep neural network-based remote sensing object detectors have achieved strong performance, they remain vulnerable to adversarial perturbations. Existing studies mainly focus on digital or white-box settings, whereas black-box physical attacks remain underexplored. These attacks are often constrained by limited physical feasibility and inefficient optimization in high-dimensional search spaces. To address these challenges, this paper proposes ColorFD, a black-box physical attack based on multiple pure-color patches. The patch positions and color parameters are jointly optimized using Differential Evolution (DE). A target-wise fitness and selection mechanism evaluates the attack state of each target and preserves target-specific improvements during evolution. Two guidance strategies further constrain the patch search space. Key-region localization identifies sensitive regions through finite-difference color probing. Common-feature extraction provides category-level spatial priors and avoids repeated localization. Although evaluated on aircraft, the formulation is not inherently restricted to this category. Experiments on YOLOv3u, YOLOv5u, and Faster R-CNN show that ColorFD outperforms the tested black-box patch method across all evaluated detectors and remains competitive with strong white-box baselines. Physical-world experiments further demonstrate that the optimized pure-color patches can be transferred from the digital domain to real imaging conditions.
74. 【2608.04557】VoxStruct3D: Structure-Leading Flow Matching for Voxel-Space 3D MRI Synthesis
链接：https://arxiv.org/abs/2608.04557
作者：Fang Li,Yang Gao,Shihao Zou,Weixin Si,Hongyu Wu,Qing Xia,Shuai Li,Aimin Hao
类目：Computer Vision and Pattern Recognition (cs.CV)
关键词：MRI synthesis requires, fine-grained voxel-level detail, synthesis requires, requires both globally, fine-grained voxel-level
备注： Project page: [this https URL](https://neesky.github.io/VoxStruct3D/)
点击查看摘要
Abstract:High-fidelity 3D MRI synthesis requires both globally coherent anatomy and fine-grained voxel-level detail. Although latent diffusion makes volumetric generation tractable, its image autoencoder introduces a reconstruction bottleneck that can limit the fine detail recoverable in the final volume. We present VoxStruct3D, a voxel-space flow-matching framework that directly models full-resolution MRI volumes using a clean-data prediction objective. Its Volumetric Voxel Generator (VVG) combines factorized 3D patch embedding with overlapping upsampling, time-modulated residual refinement, and skip fusion, enabling neighboring tokens to jointly reconstruct shared voxel regions and suppress patch-boundary artifacts. To complement direct voxel-space modeling with an explicit anatomical prior, we further introduce a Structure-First, Image-Follows (SFIF) strategy. A frozen pretrained 3D medical encoder and a StructVAE extract compact structure tokens that preserve dominant anatomy, while a structure-leading schedule keeps their trajectory ahead of the image trajectory. Patch-Aligned RoPE spatially aligns the unequal token grids, and asymmetric attention enforces one-way guidance from structure to image. Experiments on pathological and healthy T1-weighted brain MRI datasets show that VoxStruct3D achieves the strongest overall performance across feature-distribution alignment, sample diversity, and perceptual quality, producing anatomically coherent and visually realistic volumes.
75. 【2608.04554】Representing Visual Evidence for Item Difficulty Prediction: Visual Textualization and Image-Native Modeling
链接：https://arxiv.org/abs/2608.04554
作者：Han Chen,Ming Li,Hong Jiao,Tianyi Zhou
类目：Computation and Language (cs.CL); Computer Vision and Pattern Recognition (cs.CV)
关键词：Predicting item difficulty, newly developed questions, Predicting item, content can provide, provide an initial
备注：
点击查看摘要
Abstract:Predicting item difficulty from content can provide an initial estimate for newly developed questions before sufficient student responses are available. Existing approaches typically represent the question stem and answer choices as text. When mathematics items contain visual components, a common pipeline first textualizes that evidence and then applies a text predictor. We ask: how should visual evidence be represented for item difficulty prediction? We compare question text alone, visual textualization, which expresses visual evidence in language, and image-native modeling, which retains the original image. Using Eedi items with difficulty calibrated from student responses, we train large language models (LLMs) and vision-language models (VLMs) directly for difficulty regression. Both visual interfaces achieve the lowest point estimates, although the leading systems cannot be reliably ordered. Open-VLM textualization yields lower RMSE point estimates for all evaluated LLMs, while broader adaptation does so for all image-native VLMs. Test-time interventions show dependence on the paired full-item image, but do not isolate the additional visual component. The two visual interfaces also make partially complementary item-level errors and differ substantially in computational workflow. Thus, textualization should not be treated as the only practical interface: image-native modeling is a competitive alternative whose effectiveness depends on how the VLM is adapted.
76. 【2608.04533】EgoAfford: Task-Oriented Affordance Grounding via Egocentric Referring Segmentation
链接：https://arxiv.org/abs/2608.04533
作者：Xinyuan Guan,Feifan Chen,Xinyu Zhan,Fu-Cheng Zhang,Cewu Lu,Lixin Yang
类目：Computer Vision and Pattern Recognition (cs.CV)
关键词：Part-level affordance grounding, Part-level affordance, advanced the localization, Part-level, elemental actions
备注：
点击查看摘要
Abstract:Part-level affordance grounding has advanced the localization of functional object regions associated with elemental actions. Extending this capability to complex tasks calls for connecting the semantic roles of participating objects with task-state-aligned visual observations and multi-step planning. We introduce EgoAfford, a benchmark designed to connect these three aspects. Given an egocentric observation and a high-level tabletop task, a model must generate the remaining plan and segment the functional regions of up to three components of the next action: the direct object, instrument, and destination. EgoAfford comprises approximately 15.5k human-verified images from 2,000 generated multi-step scenes, organized as semantically aligned, task-complete image series, together with EgoAfford-Real, 102 manually captured images spanning 26 tasks. We further present EgoLens, a 3B multimodal large language model with role-specific mask decoders, as an in-domain reference model for this joint task. Evaluations of recent referring-segmentation MLLMs, commercial-VLM--SAM2 pipelines, and EgoLens highlight the complementary challenges of next-step inference and action-role-conditioned part grounding. EgoLens establishes strong reference performance on both generated and manually captured observations. Together, EgoAfford and EgoLens provide a foundation for jointly studying perception and planning in multi-step tabletop tasks. Our project page is available at: this https URL
77. 【2608.04530】FocusMem: Factorizing Content, Readout, and Trust in Latent GUI Memory
链接：https://arxiv.org/abs/2608.04530
作者：Zhuoran Zhang,Bowen Li,Jingcheng Ju,Yang Shi,Qixun Wang,Haotian Wang,Wei Chen,Tengjiao Wang
类目：Computer Vision and Pattern Recognition (cs.CV)
关键词：memory, Latent memory, earlier tasks, unfinished progress, memory block
备注： 36 pages
点击查看摘要
Abstract:GUI agents must remember both useful experience from earlier tasks and unfinished progress in the current interaction. Latent memory offers a compact solution by compressing multimodal trajectories into a few continuous tokens. Existing methods, however, usually map each trajectory to one fixed memory block and train it mainly through next-action supervision. This creates three practical problems: important details may be lost during compression, the same memory block must serve different decision stages, and irrelevant retrieved trajectories may still mislead the agent. We introduce FocusMem, which separates these responsibilities within a compact latent-memory interface. A role-aware content basis encourages episodic memory to retain reusable experience and working memory to retain task progress. A state-conditioned readout generates a decision-specific view of the same stored evidence, while a lightweight trust gate can suppress memory blocks that appear irrelevant to the current step. All components are trained while the GUI policy remains frozen. Across five GUI-agent benchmarks, FocusMem consistently outperforms a fully matched action-only fixed-memory baseline and prior latent memory adaptations. Further analysis shows that semantic and functional supervision preserve complementary information, state-conditioned readout is more robust as surrounding trajectory context grows, and the trust gate reduces the harm caused by injected irrelevant episodic evidence. These results show that effective latent memory depends not only on compressing past interaction, but also on what is retained, what is exposed, and what is allowed.
78. 【2608.04525】Coupled Continuous-Discrete Generation for Scene Text Image Super-Resolution
链接：https://arxiv.org/abs/2608.04525
作者：Axi Niu,Knag Zhang,Qingsen Yan,Hao Jin,Jinqiu Sun,Yanning Zhang
类目：Computer Vision and Pattern Recognition (cs.CV)
关键词：recover visually plausible, visually plausible appearance, preserving character semantics, Scene text image, Scene text
备注：
点击查看摘要
Abstract:Scene text image super-resolution (STISR) aims to recover visually plausible appearance while preserving character semantics from degraded inputs. Existing STISR systems often rely on externally generated priors or separate image and text models, resulting in error propagation and costly multi-stage inference. We present DualTSR, a unified framework that formulates STISR as coupled continuous-discrete generation. Conditional flow matching restores continuous image latents, while absorbing-state discrete diffusion reconstructs text tokens. Both processes share a multimodal transformer backbone, allowing the evolving image and text states to interact throughout generation without an external OCR prior at inference. On CTR-TSR, DualTSR achieves the best FID, LPIPS, ACC, and NED among the compared methods at both X2 and X4. On an aligned RealCE subset, it obtains the best FID, ACC, and NED with competitive LPIPS. Compared with DiffTSR at X4, DualTSR improves ACC by 12.78 percentage points while reducing the parameter count from 1.23B to 203M and end-to-end latency from 13.3s to 132ms. These results establish DualTSR as an accurate and efficient method for STISR.
79. 【2608.04515】CARVE: Cross-Slice Anisotropic Reallocation of Visual Evidence for Efficient 3D Medical Volume Understanding
链接：https://arxiv.org/abs/2608.04515
作者：Zhenyu Yi,Qiang Hu,Zhenhao Li,Jiaxuan Zhao,Yusong Sun,Lichi Zhang
类目：Computer Vision and Pattern Recognition (cs.CV); Artificial Intelligence (cs.AI); Computation and Language (cs.CL)
关键词：Slice-based MLLMs leverage, MLLMs leverage mature, Slice-based MLLMs, encoders by representing, leverage mature
备注：
点击查看摘要
Abstract:Slice-based MLLMs leverage mature 2D encoders by representing 3D volumes as sequences of 2D slices. However, this slice-wise formulation produces thousands of visual tokens that burden the LLM backbone, many of which capture overlapping visual evidence across adjacent slices. To understand how effectively a growing visual token budget improves performance, we perform scaling analyses on two 3D medical VQA benchmarks and find diminishing returns: cost keeps rising while accuracy saturates, and improving in-plane resolution is more effective than adding slices at comparable budgets. The budget should therefore be allocated more selectively rather than simply enlarged, yet most token compression methods are designed for 2D images or videos, where redundancy arises from spatial layout or temporal motion rather than from near-duplicate content along the depth axis. We present CARVE, a training-free framework that compresses visual tokens prior to LLM inference and casts token reduction as budget-constrained 2.5D allocation. CARVE partitions the depth axis into coherent windows and allocates tokens non-uniformly according to normalized cross-slice evidence. Under a shared budget, CARVE builds spatial anchors on representative slices and retrieves locally varying evidence from the full volume, then merges remaining eligible tokens into nearby anchors within each window. Removing roughly 80% of the visual tokens on Hulu-Med-7B, CARVE leads all compression baselines on every AMOS-MM report-generation metric, with 6.2 points higher retention of full-token quality than the strongest baseline, and preserves 98.1% of full-token performance across three VQA benchmarks.
80. 【2608.04504】GeoReward: Mitigating Contextual Variable Overestimation in Vision-Language Models for Cross-Market Preference Prediction
链接：https://arxiv.org/abs/2608.04504
作者：Shuo Liu,Huixiang Cai,Weiru Zhang,Xiaoyi Zeng
类目：Computer Vision and Pattern Recognition (cs.CV); Artificial Intelligence (cs.AI)
关键词：impactful failure mode, Vision-language models excel, decision-critical contextual variables, Contextual Variable Overestimation, overestimate dominant visual-textual
备注：
点击查看摘要
Abstract:Vision-language models excel in many multimodal tasks but remain prone to a subtle yet impactful failure mode: they tend to overestimate dominant visual-textual cues while underestimating sparse but decision-critical contextual variables. This issue, which we term Contextual Variable Overestimation (CVE), becomes particularly evident in real-world applications such as predicting advertisement image preferences across diverse geographic markets. For instance, when a VLM is asked to choose between two product images tailored for different countries, it often defaults to a consistent output, ignoring ground-truth regional variations. This collapse occurs because pervasive high-volume signals, such as product attributes and dense image patches, overwhelm the few but critical tokens that encode market-specific context. To address CVE, we first collect a new multimodal dataset of real advertising creatives and their click-through performance across multiple countries. We then introduce GeoReward, a reward model designed to predict ad image preferences across diverse geographic markets. GeoReward integrates three purpose-built mechanisms: (1) Market-Aware Retrieval Augmentation, (2) Context-Guided Visual Modulation, (3) Selective Sensitivity Loss. Furthermore, we demonstrate how GeoReward can guide the fine-tuning of RL for a VLM to generate background designs for text-to-image models, producing market-aware advertising creatives. Experiments validate that our framework mitigates CVE and outperforms existing baselines. This work not only diagnoses a systematic bias in VLMs toward dominant perceptual features but also delivers a targeted solution for applications where sparse contextual variables govern decision-making.
81. 【2608.04501】Privacy-Preserving Action Recognition: Taxonomy, Methods, and Privacy-Utility Trade-offs
链接：https://arxiv.org/abs/2608.04501
作者：Sareer Ul Amin,Muhammad Ayaz,Muhammad Munsif,Sanghyun Seo
类目：Computer Vision and Pattern Recognition (cs.CV)
关键词：human monitoring routine, raising real risks, made continuous human, continuous human monitoring, public safety
备注：
点击查看摘要
Abstract:Video surveillance in public safety, healthcare, and smart environments has made continuous human monitoring routine, raising real risks to personal identity and appearance. Privacy-preserving action recognition (PPAR) tackles the tension between the utility of video understanding and this exposure, and has drawn fast-growing interest. However, existing surveys remain narrow. Most catalog a single mechanism family, predate recent adversarial and hybrid work, or barely address evaluation. The result is a fragmented literature with incompatible threat models, inconsistent metrics, and no shared evaluation standard. We address this with a PRISMA-guided review of 32 peer-reviewed papers (2018--2026) drawn from 885 screened records. Methods sort into five families, namely adversarial learning (52%), skeleton-based (20%), cryptographic (12%), differential privacy (8%), and hybrid (8%), each with distinct privacy, utility, and efficiency trade-offs. Evaluation is the weak point. Only 10% of papers adopt a formal privacy definition, 65% rely on ad-hoc metrics, and 40% report an inconsistently defined cMAP. The trade-offs are steep. Skeleton methods reach about 85% accuracy but drop appearance, adversarial methods hold near 80% utility at moderate privacy (cMAP 0.9 to 0.3--0.5), and differential privacy often falls below 70%. Harder conditions stay under-tested, with fewer than 15% of papers checking cross-dataset generalization, under 10% testing adaptive attackers, and real-time edge deployment nearly untouched. We contribute a two-dimensional privacy-space taxonomy, a formal threat model, a comparative trade-off analysis, the PPAR Unified Evaluation Protocol, and a roadmap centered on benchmark standardization. With this grounding, we argue PPAR can move from prototypes toward deployment, with lessons extending to face recognition and medical imaging.
82. 【2608.04496】DIVE: Dynamic Iterative Visual Evidence Construction for Efficient Vision-Language Models
链接：https://arxiv.org/abs/2608.04496
作者：Chen Zhong,Xiao An,Zijie Wang,Jiepan Li,Guangyi Yang,Wei He
类目：Computer Vision and Pattern Recognition (cs.CV); Machine Learning (cs.LG)
关键词：substantially longer token, longer token sequences, sequences than text, efficient inference, inputs in vision-language
备注：
点击查看摘要
Abstract:Visual inputs in vision-language models (VLMs) are often encoded into substantially longer token sequences than text, making visual tokens a major bottleneck for efficient inference. Abundant recent methods address this bottleneck by scoring token importance and pruning low-scoring tokens in a single pass. However, one-shot scoring is insufficient because a token's prompt-relevant usefulness depends on the evidence already retained. Motivated by this insight, we introduce DIVE (Dynamic Iterative Visual Evidence Construction), a training-free framework that recasts visual-token pruning as dynamic evidence construction. DIVE repeatedly selects the remaining token with the highest residual-conditioned score, updates the visual and prompt residuals to discount the evidence already explained, and re-evaluates the remaining tokens. This select-update-re-evaluate process builds a retained set of complementary, prompt-relevant evidence. Experiments across eight image-understanding benchmarks show that DIVE consistently preserves performance across token budgets. With an 88.9% reduction in visual tokens, DIVE retains 98.2% of the uncompressed model's average performance. Code is available at this https URL.
83. 【2608.04483】Not All Redundant Tokens Are Alike: Analyzing Visual Token Pruning through Token Roles
链接：https://arxiv.org/abs/2608.04483
作者：Hyeonyu Kim,Sehwan Lim,Youngwon Choi,Taeyoun Kwon,Jaejin Kim
类目：Computer Vision and Pattern Recognition (cs.CV)
关键词：substantial computational bottleneck, visual token pruning, Vision-language models, process an image, bottleneck during inference
备注： Accepted to ECCV 2026 workshop, UniWorld
点击查看摘要
Abstract:Vision-language models (VLMs) process an image as a sequence of visual tokens, which creates a substantial computational bottleneck during inference. Recent visual token pruning methods address this issue by removing seemingly redundant tokens, yet it remains unclear how these pruning decisions relate to the functional roles of visual tokens. In this work, we analyze visual token pruning through the lens of token roles identified by EmbedLens. We first show that representative pruning methods exhibit distinct token-role biases, but these biases do not directly correlate with downstream performance. To better understand this behavior, we refine the token-role assignment procedure and evaluate role-protected pruning variants. Our results show that preserving non-alive tokens can sometimes maintain or improve performance, suggesting that tokens with weak direct semantic alignment may still affect model behavior under pruning. Our code is publicly available at this https URL.
84. 【2608.04480】REZE: Recognition-Based Zero-Shot Extraction for Video Temporal Grounding
链接：https://arxiv.org/abs/2608.04480
作者：Boyang Li,Chenhui Gou,Jianfei Cai
类目：Computer Vision and Pattern Recognition (cs.CV)
关键词：Video temporal grounding, identifying the time, time interval, REcognition-based Zero-shot Extraction, REZE
备注： 18 pages, 7 figures, 13 tables. Appendices included
点击查看摘要
Abstract:Video temporal grounding (VTG) refers to the task of identifying the time interval in a video that corresponds to a given natural-language query. A common zero-shot strategy asks a large vision-language model (VLM) to generate the start and end timestamps directly, so the result depends heavily on the design and training of the model, and grounding accuracy differs widely from one VLM to another. We therefore propose REcognition-based Zero-shot Extraction (REZE), a simple training-free method that splits the video into short clips, asks the model for a clip-level confidence score for the query, and uses a deterministic algorithm to convert the resulting score curve into the output required by the task. Because temporal aggregation is performed outside the model, REZE adapts to different task outputs, from single- and multi-interval moment retrieval to highlight detection. On QVHighlights, REZE improves the best reported training-free moment-retrieval mAP from 38.23 to 40.32, while on highlight detection it reaches 44.18 mAP and 73.41 HIT@1, establishing a new state of the art among training-free methods. Its HIT@1 also outperforms all fully supervised SoTAs on the QVHighlights test split. We evaluate REZE on seven backbones from three model families. On Charades-STA and QVHighlights, it outperforms direct timestamp generation in every available comparison. We further observe that with REZE an earlier-generation model can approach the native performance of a newer model in its family.
85. 【2608.04472】EndoVLM: An Endoscopy Vision-Language Pre-training Model via Anatomy-Guided Sparsity and Progressive Alignment
链接：https://arxiv.org/abs/2608.04472
作者：Zhenyu Yi,Jianwei Xu,Yue Hu,Zhongwei Qiu,Sijing Li,Liang Huang,Bin Lv,Ling Zhang,Yingda Xia
类目：Computer Vision and Pattern Recognition (cs.CV); Artificial Intelligence (cs.AI); Computation and Language (cs.CL)
关键词：crucial for advancing, endoscopic image analysis, advancing endoscopic image, image analysis, advancing endoscopic
备注：
点击查看摘要
Abstract:The development of foundation models (FMs) is crucial for advancing endoscopic image analysis. However, existing endoscopy FMs mainly rely on self-supervised learning from uni-modal images or videos, overlooking the rich semantic knowledge contained in clinical reports. Furthermore, effectively leveraging these records is hindered by a fundamental modality gap: structured anatomical descriptions are not naturally mapped to specific frames within the high-redundancy, uncurated visual streams. In this paper, we present EndoVLM, a novel vision-language FM pre-trained on over 348K endoscopic examinations, each pairing a clinical report with its corresponding image collection. An Anatomy-Guided Sparse Pooling mechanism utilizes textual descriptions as queries to drive sparse attention, efficiently aggregating semantically salient frames into anatomy-specific visual representations across redundant image-sets. Next, a Progressive Semantic-Aware Alignment strategy models clinical taxonomy (anatomy and pathological status) via structured soft targets, bridging the gap from global patient-level matching to fine-grained localized alignment. Finally, a Semantic-Concentrated Masked Autoencoder is applied exclusively to these semantic-rich frames, integrating low-level visual precision with robust high-level semantic representation. Extensive experiments across various downstream tasks demonstrate that EndoVLM outperforms existing foundation models and remains competitive with task-specific methods. Remarkably, EndoVLM also exhibits robust zero-shot generalization capabilities, highlighting its potential for broader clinical application.
86. 【2608.04454】Beyond Global Routing Aggregation: Phase-Aware Expert Merging for MoE Vision-Language Models
链接：https://arxiv.org/abs/2608.04454
作者：Hongyu Zhang,Cheng Yan,Xiang Xia,Wuyang Zhang
类目：Computer Vision and Pattern Recognition (cs.CV); Machine Learning (cs.LG)
关键词：deployment requires storing, sparse expert activation, increase model capacity, expert, phase-conditioned expert roles
备注： 17 pages, 3 figures, 17 tables
点击查看摘要
Abstract:Mixture-of-experts vision-language models (MoE-VLMs) increase model capacity with sparse expert activation, yet deployment requires storing the full expert pool. Training-free expert merging reduces this burden, and many routing-based methods aggregate routing statistics across all tokens to determine merge compatibility. However, MoE-VLM inference is phase-structured: image-context tokens carry visual content, question tokens specify the query, and answer tokens produce the output, with different counts and routing distributions. Because image-context tokens are far more numerous, global aggregation can overemphasize image-context processing and obscure phase-conditioned expert roles, making experts serving different phases appear interchangeable and degrading model performance. We therefore argue that MoE-VLM expert merging should preserve phase-conditioned expert roles, judging compatibility by how experts serve different phases rather than globally aggregated routing statistics. Based on this view, we propose RoleMerge, a training-free method that constructs each expert's Routing Role Profile (RRP) from phase-normalized routing statistics, capturing its relative phase preference. Guided by expert-phase information loss, RoleMerge merges experts with compatible profiles and their corresponding router entries while preserving answer-decoding expert distinctions. Experiments on three models and multiple benchmarks show that RoleMerge preserves more of the full model's performance than alternative expert-merging methods at matched expert-retention ratios, with relative improvements of up to 9.6 percent in six-task macro-average performance. These results validate phase-conditioned expert roles as a more effective basis than global routing aggregation for MoE-VLM expert merging.
87. 【2608.04453】winIR: Coordinated Invisible Dual-Point Attacks on Online HD Map Construction
链接：https://arxiv.org/abs/2608.04453
作者：Haibo Hu,Jianghuai Deng,Chen Tang,Yang Lou,Qian Xu,Jianping Wang
类目：Computer Vision and Pattern Recognition (cs.CV); Artificial Intelligence (cs.AI)
关键词：online map construction, map construction, autonomous driving, online map, critical to prediction
备注：
点击查看摘要
Abstract:Online HD map construction is critical to prediction and planning in autonomous driving. We find that existing physical attacks against online map construction are limited by a cross-boundary compensation effect: after the target boundary is perturbed, another visible boundary may retain sufficient geometric cues for the model to recover the original road geometry. Based on this observation, we propose TwinIR, a new mechanism-guided physical attack methodology for online map construction. TwinIR jointly optimizes attack effectiveness and point sparsity, seeking the minimum number of attack points needed to suppress compensating geometric cues from surrounding boundaries. To reduce the perceptibility of multi-point attacks, TwinIR models camera responses to near-infrared illumination and maps optimized attack points to feasible physical placements, producing camera-visible interference with minimal visible-spectrum changes. Experiments on nuScenes across state-of-the-art online map construction models show that TwinIR reduces mAP by 8.18-8.96 percentage points under RSA and 2.84-5.62 points under ETA, while increasing the unreachable-goal rate by 25-28 points and the unsafe-planned-trajectory rate by 19-20 points over clean inputs. These attacks are also validated on a real-world testbed AV, where TwinIR successfully induces both road straightening and early-turn deformations while remaining inconspicuous in full-color views.
88. 【2608.04452】Q-CueGraph: Query-Conditioned Visual Evidence Graphs for Multimodal Reasoning
链接：https://arxiv.org/abs/2608.04452
作者：Pengcheng Pan,Xinfang Zhang
类目：Computer Vision and Pattern Recognition (cs.CV); Artificial Intelligence (cs.AI); Computation and Language (cs.CL)
关键词：zoom tools give, tools give multimodal, give multimodal large, multimodal large language, large language models
备注：
点击查看摘要
Abstract:High-resolution pixels and crop or zoom tools give multimodal large language models the ability to inspect an image, but they do not provide a reliable task-conditioned policy for deciding where to inspect. Q-CueGraph makes this decision explicit. It maps a question and an image representation to budgeted, coordinate-level observations for a frozen reader. Text-rich images use a reusable OCR/layout graph; natural-image search instantiates query-conditioned visual nodes behind the same selection, composition, and budgeting interface. Optional utility refinement learns which candidate crops the frozen reader can use from training-answer correctness, without region-box supervision. With a frozen Qwen2.5-VL-7B reader, Q-CueGraph reaches 0.833 accuracy on V*Bench versus 0.696 for full-image inference from a 19% image-area budget, and reaches 92% of full-image ANLS on InfographicVQA from about half the image area. Across six benchmarks, explicit observation is most valuable when evidence is localizable, the question discriminates its location, and resolution limits full-image reading.
89. 【2608.04448】When does training on downscaled images yield the same gradients?
链接：https://arxiv.org/abs/2608.04448
作者：Seunghyun Ji
类目：Computer Vision and Pattern Recognition (cs.CV); Artificial Intelligence (cs.AI)
关键词：Diffusion transformers deliver, strong image generation, transformers deliver strong, deliver strong image, cost grows superlinearly
备注：
点击查看摘要
Abstract:Diffusion transformers deliver strong image generation, but their training cost grows superlinearly with resolution. Recent work justifies training or sampling at reduced resolution on a spectral premise: at high noise, a downscaled latent preserves almost the full surviving signal. Whether a downscaled step also preserves the native training gradient signal, however, has remained unresolved. We reduce how that signal changes under downscaling to two terms: a noise-dependent term governed by the downscale ratio, which decays at high noise as the spectral premise predicts, and a \sigma-independent floor governed by the target grid's absolute token count, carried by the compute graph itself and removed by no noise level. The measured (route, \sigma) map corroborates the account and uncovers structure the spectral picture cannot express: on the 1024-768 route, a window (0.65 \sigma 0.95), predicted by no spectral criterion at any tolerance, where the downscaled gradient stays within a small margin of the native one. Training LoRA adapters with downscaled steps restricted to the routes and noise windows the map validates reduces training time by 14.6% at a fixed step budget while remaining near-native in weight space. Code is available at this https URL.
90. 【2608.04442】Robustness Emerges Early in Training Dynamics, but Is Not Preserved
链接：https://arxiv.org/abs/2608.04442
作者：Jiangang Yang,Wenhui Shi,Lu Hu,Jing Xing,Jian Liu
类目：Machine Learning (cs.LG); Computer Vision and Pattern Recognition (cs.CV)
关键词：deep neural networks, natural corruptions remains, neural networks, natural corruptions, corruptions remains
备注： Accepted by ECCV2026
点击查看摘要
Abstract:Robustness to natural corruptions remains a fundamental challenge for deep neural networks. In this paper, we identify a robustness fading phenomenon where shallow layers spontaneously develop robust representations and flat loss landscapes in early training, yet these properties are not preserved during standard convergence. To address this, we propose a framework that performs strategic interventions on training dynamics to stabilize the empirically identified early-emergent robust priors. Our approach includes two parameter-free strategies: Early-Phase Stabilization~(EPS) and Asymmetric Weight Reversion~(AWR), which stabilize or recover robust shallow configurations without modifying the model architecture or introducing learnable parameters. Extensive experiments demonstrate the efficacy of our framework across various benchmarks and architectures, yielding significant gains in downstream transfer, dynamic adaptation, and diverse computer vision applications.
91. 【2608.04441】Season: Spectrum-Aware Orthogonal Gradient Refinement for Transfer-Based Adversarial Attacks
链接：https://arxiv.org/abs/2608.04441
作者：Tianyi Wang,Zhenghao Gao,Shengjie Xu
类目：Computer Vision and Pattern Recognition (cs.CV); Cryptography and Security (cs.CR)
关键词：Transfer-based adversarial attacks, Vision Transformers, Transfer-based adversarial, favor local textures, CNNs favor local
备注： 6 pages
点击查看摘要
Abstract:Transfer-based adversarial attacks often transfer poorly across heterogeneous architectures because CNNs favor local textures while Vision Transformers (ViTs) rely on global shapes. We propose Season, a spectrum-aware orthogonal gradient refinement framework for L-infinity transfer attacks against black-box target models on ImageNet, using a white-box surrogate. Season decomposes each update into a low-frequency branch capturing structural cues and a high-frequency branch capturing textures. A low-saliency guidance scheme reallocates high-frequency energy to background regions, preserving foreground structures that ViTs depend on. An orthogonal projection then forces the textural update to lie in the orthogonal complement of the structural direction, mitigating feature interference. As a training-free plug-and-play wrapper, Season enhances eight gradient-stabilization and input-enhancement attacks without modifying their cores. Across eight CNN, ViT, and MLP targets, Season improves transfer success rate by 6.6 percentage points on average and up to 16.0 points over strong baselines under a unified protocol.
92. 【2608.04436】oolArtist: Tool-Using Unified Multimodal Models for Agentic Image Generation
链接：https://arxiv.org/abs/2608.04436
作者：Jiahao Zhao,Xiaomin Yu,Zhongxiang Sun,Fengwei Teng,Chengwei Qin,Xiaobin Hu,Jun Xu,Shuicheng Yan
类目：Computer Vision and Pattern Recognition (cs.CV)
关键词：complex semantic understanding, produce visually compelling, require complex semantic, visually compelling images, external world knowledge
备注：
点击查看摘要
Abstract:Text-to-image (T2I) models can produce visually compelling images, yet they remain limited on open-world tasks that require complex semantic understanding, multi-step reasoning, and the integration of external world knowledge. Existing efforts introduce agent capabilities into image generation, but they either prescribe a fixed workflow or place only a subset of the open-world image generation process under agent control. Consequently, reasoning, tool invocation, and image generation are not coordinated by a single policy. We propose ToolArtist, a fully agentic image generation model obtained by post-training a Unified Multimodal Model (UMM). ToolArtist dynamically orchestrates reasoning, external tool use, and native image generation within one unified policy. During Supervised Fine-Tuning (SFT), we equip a teacher agent with search tools alongside an image-generation tool. We then convert the collected trajectories into a UMM compatible format, where the image-generation tool is concealed while the resulting generated images are retained. During Reinforcement Learning (RL), we develop an agentic RL infrastructure for UMMs and introduce Reason-Act-Draw GRPO (RAD-GRPO), which uses complementary intent and quality rewards to jointly optimize the model. Experiments show that placing the entire open-world image-generation process under an agent policy consistently outperforms approaches with fixed pipelines or only partially agent-controlled components. We release the training data and the complete post-training infrastructure.
93. 【2608.04434】OmniRouting: A Semantic-Coupled Multimodal Benchmark for Constraint-Aware Spatial Reasoning in PCB Routing
链接：https://arxiv.org/abs/2608.04434
作者：Taiting Lu,Kaiyuan Lin,Ziwei Dong,Sisong Bei,Haolin Ye,Yuxin Tian,Runze Liu,Mingjia Wang,Jingying Zeng,Hongxing Pan,Kai Zhang,Haoyu Wang,Guoliang Shi,Ling Ma,Yifan Yang,Jiaying Lu,Qi He,Yi-Chao Chen,Sung-Liang Chen,Yincheng Jin,Mahanth Gowda
类目：Computer Vision and Pattern Recognition (cs.CV)
关键词：Recent large language, large language models, demonstrated remarkable progress, Recent large, language models
备注：
点击查看摘要
Abstract:Recent large language models (LLMs) have demonstrated remarkable progress in constraint-aware navigation, maze reasoning, and graph reasoning. However, their ability to reason about complex routing problems under strict geometric, topological, and electrical constraints remains largely unexplored, despite routing being one of the most challenging and critical stages of electronic design automation (EDA). To bridge this gap, we introduce OmniRouting, the first large-scale benchmark designed to evaluate LLMs on printed-circuit-board (PCB) routing reasoning under real-world industrial design-rule, manufacturability, and connectivity constraints. OmniRouting contains 1,681 industrial-grade schematic-coupled PCB designs, including board geometries, routable component placements by human engineers, footprints, pad locations, netlists, stackup information, and routing constraints. The benchmark comprises four tasks: (1) geometric routing reasoning, generating physically valid copper traces, vias, and layer assignments to connect circuit nets within constrained board regions; (2) design-rule-aware routing reasoning, producing routable layouts that satisfy clearance, trace-width, via, obstacle-avoidance, and board-boundary constraints; (3) electrical functionality reasoning, preserving schematic-specified connectivity while reasoning over net names and functional roles to produce electrically correct routing; and (4) tool-augmented agentic routing, leveraging external tools for tasks (1)-(3). Our results reveal substantial limitations of current LMMs in PCB routing, including weak path-planning capabilities, poor adherence to design-rule constraints, and inconsistent preservation of electrical functionality. We will open-source all benchmark data, evaluation code, and tool interfaces to facilitate future research.
94. 【2608.04429】UBLLIE: Unified Backlight and Low-Light Image Enhancement
链接：https://arxiv.org/abs/2608.04429
作者：Yasmin Yasin,Muhammad Usman,Ibrahim Radwan,Saeed Anwar
类目：Computer Vision and Pattern Recognition (cs.CV)
关键词：computer vision tasks, presenting significant challenges, severe exposure imbalance, downstream computer vision, global underexposure
备注：
点击查看摘要
Abstract:Backlit and low-light images often suffer from severe exposure imbalance or global underexposure, presenting significant challenges for both visual perception and downstream computer vision tasks. In this paper, we propose a unified, unsupervised enhancement framework that addresses both types of degradation without relying on paired ground-truth data. Our approach builds on CLIP-guided prompt learning to semantically supervise enhancement using learned positive and negative textual prompts. To improve the quality of our improvements over prior work, we design a symmetric residual U-Net backbone augmented with an Atrous Spatial Pyramid Pooling module. This architecture captures multi-scale contextual information, enabling adaptive correction under spatially heterogeneous illumination. During training, the enhancement network is guided by CLIP-based semantic similarity losses and refined via an iterative prompt optimization mechanism. Extensive experiments on both paired and unpaired datasets, including BAID, Backlit300, LOL, and VE-LOL-L, demonstrate that our framework consistently outperforms state-of-the-art supervised and unsupervised methods in terms of fidelity, perceptual quality, and generalization. Furthermore, our work emphasizes the need for stronger benchmarking protocols for backlit enhancement, a relatively underexplored area. The proposed framework provides a robust, scalable solution for real-world illumination enhancement across diverse lighting conditions.
95. 【2608.04426】Predict, Then Retrieve: Cross-Instance Future-State Retrieval from Video Prefixes
链接：https://arxiv.org/abs/2608.04426
作者：Quynh Vo,Thong Nguyen,Vinh-Hien Do,Cong-Duy Nguyen,Anh-Tuan Luu
类目：Computer Vision and Pattern Recognition (cs.CV); Computation and Language (cs.CL)
关键词：introduce Predictive State, Predictive State Retrieval, introduce Predictive, short video prefix, PSR combines anticipation
备注： Work in progress
点击查看摘要
Abstract:We introduce Predictive State Retrieval (PSR), a task in which a model observes a short video prefix and a temporal question about an object's future state, then retrieves instances from other videos or images that depict that state. Unlike action anticipation, which predicts a label, moment retrieval, which localizes an observed event within a video, or video generation, which synthesizes pixels, PSR combines anticipation with cross-instance retrieval across multiple temporal horizons. We construct a benchmark from four datasets with graded, human-validated ground truth, difficulty tiers, and an oracle ceiling. We also propose LFTR, a lightweight retriever with frozen encoders that predicts a question- and horizon-conditioned future latent and matches it in complementary semantic and visual spaces. A ceiling decomposition reveals a clear bottleneck: the true future state is highly retrievable once specified, whereas every predictor we evaluate, including a large multimodal language model with access to the prefix frames, remains far below the oracle. Thus, forecasting rather than perception is the central learnable challenge. LFTR narrows this gap at substantially lower inference cost, and ablations attribute its gains to cross-space fusion and hard-negative training rather than latent rollout. We release the benchmark, code, and evaluation scripts.
96. 【2608.04424】hinking with Anchors: Grounded and Efficient Document Reasoning
链接：https://arxiv.org/abs/2608.04424
作者：Sichen Zhu,Yuchen Zhu,Wenzhuo Xu,Jason Kuen,Wanrong Zhu,Jing Shi,Xuan Shen,Quanyi Wang,Yiwei Wang,Yujun Cai,Bing Shuai,Qin Zhang,Yongxin Chen,Shilong Liu,Molei Tao,Jiuxiang Gu
类目：Computer Vision and Pattern Recognition (cs.CV)
关键词：Existing document understanding, Existing document, spatial relations, ADOPD, largely focused
备注：
点击查看摘要
Abstract:Existing document understanding benchmarks have largely focused on locating page elements, yet real-world document intelligence requires models to reason jointly about region semantics, spatial relations, and visual structure. We present ADOPD 2026, a reasoning-oriented extension of ADOPD that turns page decomposition into spatially grounded document understanding. ADOPD 2026 enriches page anchors inherited from ADOPD 2024 dataset with human-cleaned captions, semantic tags, and generated chain-of-thought (CoT) traces grounded to document regions. Instead of treating boxes, masks, and tags as independent supervision signals, we cast text blocks, visual entities, semantic labels, bounding boxes, and polygon masks as a shared vocabulary of visual anchors. This representation supports three connected capabilities. First, region-level semantic tagging asks models to identify document element types from both page context and local appearance, revealing long-tail semantic failures that standard layout benchmarks often hide. Second, unified vision-language grounding generates text regions and visual entities together with coordinates or polygonal outlines, transforming detection and segmentation outputs into structured anchors that can be reused by downstream reasoning systems. Third, current state-of-the-art models still struggle with dense counting tasks evaluated on DocCount, a benchmark derived from ADOPD 2026, highlighting the need for the Thinking-with-Anchors pipeline in document semantic understanding. By connecting page decomposition to verifiable visual-anchor reasoning, ADOPD 2026 provides a task framework that moves document understanding beyond localization toward anchor-grounded document intelligence.
97. 【2608.04423】Foreseeing the Invisible: Amodal Reconstruction of Leaf Fossil Images
链接：https://arxiv.org/abs/2608.04423
作者：Liuxiang Yue,Ailin Zhang,Ziyue Zhao,Yikun Duan
类目：Computer Vision and Pattern Recognition (cs.CV)
关键词：sedimentary rock hides, sedimentary rock, rock hides, erodes the lamina, rarely preserved
备注： 12 pages, 10 figures
点击查看摘要
Abstract:Fossil leaves are rarely preserved whole -- sedimentary rock hides, breaks, and erodes the lamina, yet paleobotany depends on the complete shape and outline of the leaf. We cast the recovery of the missing tissue as amodal reconstruction and present AmodalDINO, a multi-head dense-prediction model that predicts four masks from a single RGB image: visible leaf, amodal complete leaf, amodal main vein, and fine veins. Unlike essentially all prior amodal work, AmodalDINO is given no visible mask. It predicts the visible and amodal regions jointly, so it needs no upstream instance segmenter at runtime. Two simple but effective changes adapt the model to the amodal segmentation task: fully fine-tune a DINOv3 ViT-L/16 at a small learning rate instead of freezing it, and attach auxiliary venation heads alongside the leaf heads. These two changes enable the model to learn the structural shape prior of leaves. Trained only on synthetic leaf fossil images, AmodalDINO reaches 95.0% Dice / 90.5% IoU on the validation set and transfers well to real fossil specimens. Stripped to two heads, the same recipe can run on two benchmark datasets, reaching 85.05 full mIoU / 66.65 occluded mIoU on KINS and 80.90 / 38.15 on COCOA-cls. The model is also practical: by quantizing to 4-bit weights, it runs entirely offline in a browser, matching the original model with an IoU of 0.910. We also add ruler-based calibration to estimate surface area, and a generative visualization of living leaves on local devices.
98. 【2608.04412】muSync-GS: Physics-Synchronized Driving Video Synthesis for Weather and Geometric Road Hazards
链接：https://arxiv.org/abs/2608.04412
作者：Yang Chen,Yicheng Zhu,Tao Li,Zilin Bian
类目：Computer Vision and Pattern Recognition (cs.CV)
关键词：High-quality driving data, generative world models, High-quality driving, data are essential, essential for autonomous-driving
备注： 42 pages, 14 figures; includes an appendix
点击查看摘要
Abstract:High-quality driving data are essential for autonomous-driving systems and generative world models. However, rare and safety-critical scenarios involving adverse weather, braking under low tire--road friction, and uneven road geometry are costly and risky to collect at scale. Existing video-generation and 3D Gaussian editing methods can modify weather appearance or road geometry, but typically do not couple these edits with tire--road interaction and vehicle dynamics. As a result, an edited video may retain its original trajectory even when the modified road condition should alter braking, wheel slip, load transfer, and ego-camera motion. We present muSync-GS, a physics-synchronized framework for driving video synthesis under adverse-weather and road-elevation hazards. A precipitation-derived road-surface condition jointly controls road appearance and tire friction, while a shared road-elevation profile drives both visible road-geometry editing and axle excitation. A calibrated vehicle model predicts speed, slip ratio, normal loads, and pitch for constructing the ego-camera trajectory and synchronized physical annotations. On 12 held-out CarSim cases spanning precipitation levels, brake inputs, and road-profile parameters, the model achieves mean case-wise RMSEs of 0.0273 m/s for speed, 0.0590 degrees for pitch, 0.0101 for slip ratio, and 26.61 N for per-wheel normal load. Together with the reconstructed-scene experiments, these results show that muSync-GS accurately reproduces vehicle responses under held-out controls while synchronizing them with controllable scene edits and ego-camera motion.
99. 【2608.04404】Faster-WAM: Efficient Inference-Time Future Conditioning for Robust World Action Models
链接：https://arxiv.org/abs/2608.04404
作者：Weiheng Zhao,Haoyi Jiang,Xin Shi,Liu Liu,Fan Huang,Zhizhong Su,Wei Sui,Xinggang Wang
类目：Computer Vision and Pattern Recognition (cs.CV)
关键词：World Action Models, World Action, Action Models, environment evolves, World
备注：
点击查看摘要
Abstract:World Action Models (WAMs) improve robot manipulation by learning how the environment evolves beyond the current observation. However, existing approaches face a fundamental dilemma: Joint-WAMs preserve future-aware representations during inference but incur prohibitive computation costs, while efficient alternatives remove future modeling at inference time and may lose the robustness benefits of temporal reasoning. In this work, we revisit the role of future representations in WAMs and show that inference-time future conditioning is critical for generalization under distribution shifts. This observation motivates Faster-WAM, an efficient future-conditioning WAM that preserves future representations while avoiding expensive video-action interaction. Faster-WAM introduces a sparse future-conditioning framework that computes future representations once and selectively reuses them throughout action denoising. Specifically, we propose SparseMoT to replace ubiquitous layer-wise fusion with selective video-action interaction at a compact subset of network stages, and Interval KV-Fusion to aggregate multi-depth future representations without increasing attention complexity. Experiments demonstrate that Faster-WAM achieves a substantially better performance-efficiency trade-off than existing WAMs. On the out-of-distribution LIBERO-Plus benchmark, Faster-WAM improves success rate from 49.14% to 73.57% compared with Fast-WAM, while running 2.21$\times$ faster than Joint-WAM. It further achieves state-of-the-art performance on LIBERO and RoboTwin 2.0, while demonstrating strong robustness in real-world manipulation.
100. 【2608.04396】CofactVLA: Deconfounding Vision-Language-Action Models via Counterfactual Intervention
链接：https://arxiv.org/abs/2608.04396
作者：Yan Zhang,Yinan Wu,Haoran Duan,Jungong Han
类目：Computer Vision and Pattern Recognition (cs.CV)
关键词：driven significant progress, robotic manipulation, vision-override phenomenon, VLAs frequently fall, significant progress
备注：
点击查看摘要
Abstract:Vision-Language-Action (VLA) models have driven significant progress in robotic manipulation, yet they fundamentally struggle with the vision-override phenomenon. Driven by the severe modality imbalance between dense visual streams and sparse linguistic instructions, VLAs frequently fall prey to causal confusion. Instead of treating language as the primary causal driver, the policy entirely bypasses the original instruction by overfitting to spurious visual confounders, such as prominent objects or familiar layouts. To systematically alleviate this bias, we formalize the process of action generation as a Dual-path Deconfounding Graph (DDG) and propose CofactVLA, a novel causal intervention framework. By dynamically constructing a language-masked counterfactual branch within a single forward pass, CofactVLA isolates and neutralizes visual confounders through two synergistic mechanisms. First, Action-Level Orthogonal Projection Guidance (OPG) geometrically projects the factual velocity field away from the counterfactual visual bias during continuous flow matching, extracting the pure semantic intent. Second, Feature-Level Counterfactual Covariance Reduction (CCR) mathematically deconfounds latent representations by penalizing the positive eigenspace of the covariance difference, explicitly suppressing dominant visual shortcuts while preserving the causal language intent. Extensive experiments demonstrate that CofactVLA establishes a new state-of-the-art across diverse simulation benchmarks. Beyond simulation, real-world robot experiments demonstrate the causal efficacy of our method in bridging the generalization gap, yielding a 52.3\% absolute success rate gain under out-of-distribution scenarios.
101. 【2608.04394】Free-Lunch Augmentation by Revisiting Diffusion-Based Data Generation for Cross-Domain Few-Shot Object Detection
链接：https://arxiv.org/abs/2608.04394
作者：Zijian Zhuang,Yixiong Zou,Yuhua Li,Ruixuan Li
类目：Computer Vision and Pattern Recognition (cs.CV)
关键词：Few-Shot Object Detection, Object Detection, data-rich upstream generic, upstream generic domains, Cross-Domain Few-Shot Object
备注：
点击查看摘要
Abstract:Cross-Domain Few-Shot Object Detection (CDFSOD) aims to transfer knowledge from data-rich upstream generic domains to downstream expert domains using scarce training data, where the significant domain gap and data scarcity make it an unsolved challenge. To address this problem, we revisit a natural yet underexplored approach in CDFSOD: data augmentation, by directly synthesizing data through diffusion models to supplement limited training samples. However, due to large domain gaps, we find that current diffusion methods cannot produce good results, leading to performance even lower than using the original images. To address these limitations, we divide the domain gaps into visual gaps and semantic gaps for separate analysis. For the visual gap, we find that the diffusion model cannot distinguish noise from useful information on expert domains, which can be mitigated by adding weakened noise. For the semantic gap, we find that the background semantics shows much smaller gaps between domains than foreground semantics, and we can bridge this gap by background inpainting. Based on the above analysis, we propose a method (Selective Inpainting with Tailored Noise, SITN) to dynamically take different strategies for downstream data synthesis based on their different gaps from the general domain, including a Generation Module for adding tailored noise and a Selection Module to dynamically select the inpainting regions. Extensive experiments on 6 datasets of CDFSOD and 4 datasets of cross-domain few-shot segmentation (CDFSS) validate that we can synthesize helpful data, achieving new state-of-the-art performance. Our codes is available at this https URL
102. 【2608.04385】ReGround: Restoring Visual Grounding in Multi-Step Reasoning through Self-Diagnosis and Visual Re-Examination
链接：https://arxiv.org/abs/2608.04385
作者：Lei Peng,Shuai Lv,Wei Hu
类目：Computer Vision and Pattern Recognition (cs.CV)
关键词：chains grow longer, steps rely increasingly, reasoning chains grow, grow longer, inference steps rely
备注： Accepted to ACM Multimedia 2026 (MM '26). 8 pages main text, 4 figures, plus appendix
点击查看摘要
Abstract:Vision-Language Models (VLMs) often lose visual grounding during multi-step reasoning: as reasoning chains grow longer, later inference steps rely increasingly on language priors rather than image evidence. We identify a consistent benchmark-level signature associated with this degradation: across 2,510 re-examined samples from four benchmarks, attention entropy over image tokens typically decreases during Round 1 and rises again after image re-injection. However, we find that effective visual re-examination requires two complementary ingredients: image re-injection and targeted self-diagnosis. Without targeted diagnosis, re-examination can even hurt performance, whereas accurate self-diagnosis yields substantial gains -- a swing of several points on key benchmarks, indicating that diagnostic quality is a key factor in whether re-examination helps or hurts in our setting. We present ReGround, a two-stage framework that teaches VLMs to self-diagnose grounding failures and selectively re-examine visual evidence, without architectural modifications or external tools. Through capability bootstrapping, a stronger variant from the same model family provides diagnostic scaffolding only during data construction, while the policy model learns to diagnose autonomously at inference time and retains most of the assisted gains. Experiments on eight benchmarks across two VLM backbones demonstrate consistent gains, especially on visually intensive multi-step reasoning tasks, while incurring only modest inference overhead relative to tool-augmented baselines. Project page: this https URL . Code: this https URL .
Comments:
Accepted to ACM Multimedia 2026 (MM '26). 8 pages main text, 4 figures, plus appendix
Subjects:
Computer Vision and Pattern Recognition (cs.CV)
ACMclasses:
I.2.10; I.2.7; I.4.8
Cite as:
arXiv:2608.04385 [cs.CV]
(or
arXiv:2608.04385v1 [cs.CV] for this version)
https://doi.org/10.48550/arXiv.2608.04385
Focus to learn more
arXiv-issued DOI via DataCite (pending registration)</p>
103. 【2608.04379】Image Classification Using CNN-QNN Hybrid Model with Optimized Correlated Features
链接：https://arxiv.org/abs/2608.04379
作者：Minseo Seong,Youngwook Kim
类目：Computer Vision and Pattern Recognition (cs.CV); Artificial Intelligence (cs.AI)
关键词：convolutional neural network, enhance image classification, QNN, neural network, quantum neural network
备注： Accepted by CVPR Findings 2026
点击查看摘要
Abstract:We propose a method to optimize the correlation among convolutional neural network (CNN) features that are used as inputs to quantum neural network (QNN) to enhance image classification accuracy. Unlike prior approaches that employ orthogonal decomposition as preprocessing, we intentionally introduce correlated features that are more physically compatible with QNN. This design leverages the QNN's inherent ability to exploit quantum entanglement for representing correlated states-an advantage unavailable to classical neural networks. We hypothesize that aligning feature correlations with the entanglement structure of QNN improves binary classification performance. Based on a mathematical derivation of QNN outputs, Monte Carlo simulations indicate that an average correlation between features of 0.5 yields optimal classification accuracy. To validate this finding, we evaluate a quantum-classical hybrid model on three tasks: CIFAR-10 (automobile vs. truck), Fashion-MNIST (shirt vs. coat), and radar micro-Doppler signatures (robotic dogs vs. non-robots). To regulate feature correlations, we introduce a correlation-regularization term on the outputs of the CNN, driving the off-diagonal entries of the feature correlation matrix toward a target constant. Across all datasets, inducing intermediate correlation consistently improved accuracy compared to low, high, or unregulated correlations, while also reducing classification accuracy variance. These results demonstrate that imposing moderate feature correlations-without modifying the quantum circuit-enhances classification accuracy and stability by aligning feature statistics with the QNN's entanglement structure. This study highlights the potential of QNN to surpass the performance of classical classifiers as more qubits become available.
104. 【2608.04349】Poly-OPD: Heterogeneous Multi-Teacher On-Policy Distillation for Capability-Selectable Flow Models
链接：https://arxiv.org/abs/2608.04349
作者：Siming Fu,Haojun Xu,Ruizhe He,Zheming Fu,Hualiang Wang,Jie Huang,Xiaoxiao Ma,Mingchen Zhong,Weihu Huang,Xiaoxuan He,Linjiang Huang,Si Liu
类目：Computer Vision and Pattern Recognition (cs.CV)
关键词：Leading open, instructions more faithfully, carry complementary strengths, lead on preference-aligned, preference-aligned aesthetics
备注：
点击查看摘要
Abstract:Leading open text-to-image models often carry complementary strengths: one may lead on preference-aligned aesthetics while another follows compositional instructions more faithfully. However, differences in their autoencoders and noise schedules make it difficult to transfer these strengths across models. In this paper, we present Poly-OPD, a framework that can consolidate complementary strengths of heterogeneous teachers into a single compact flow-matching student. To bridge the incompatible latent spaces of different teachers, Poly-OPD performs on-policy distillation through a pixel bridge. Each student-generated image is re-encoded by a selected teacher's encoder and refined from a noise level matched by magnitude under the teacher's noise schedule. The resulting target is further matched to the student in frozen DINOv2 space, enabling supervision across incompatible latent spaces. To retain complementary capabilities without cross-teacher interference, Poly-OPD uses a gradient compatibility diagnostic to organize its adapters: attention LoRA modules are shared across teachers, whereas feed-forward adapters remain teacher-specific. During distillation, a gap-aware curriculum devotes more training to compositional categories where the student still falls short of the teacher. As each gap narrows, training shifts toward categories with larger remaining gaps. By distilling FLUX.1-dev and Z-Image into a 2.5B SD3.5-Medium student, Poly-OPD improves GenEval from 67.3 to 73.3, surpassing both larger teachers, and raises DrawBench HPSv3 from 9.34 to 11.35, consolidating both strengths within a switchable model.
105. 【2608.04348】StructTab: Structured Feature Sequencing for Multimodal Learning of Image and Tabular Data
链接：https://arxiv.org/abs/2608.04348
作者：Al Zadid Sultan Bin Habib,Md Younus Ahamed,Prashnna Gyawali,Gianfranco Doretto,Donald A. Adjeroh
类目：Computer Vision and Pattern Recognition (cs.CV); Artificial Intelligence (cs.AI); Machine Learning (cs.LG); Machine Learning (stat.ML)
关键词：Column Permutation Problem, resulting in redundancy, generalization problems, ineffective representations, Permutation Problem
备注： This paper has been accepted for presentation at the 28th International Conference on Pattern Recognition (ICPR 2026) in Lyon, France Code: [this https URL](https://github.com/zadid6pretam/iStructTab) PyPI: pip install istructtab
点击查看摘要
Abstract:Multimodal learning of images and tabular data is often impaired by ineffective representations, resulting in redundancy, dispersion, and generalization problems. To tackle this challenge, we introduce Graph-Enhanced Descriptor Sequencing (GEDS), a structured feature sequencing algorithm grounded in principles from the Column Permutation Problem (CPP). GEDS refines statistical descriptors of the features through similarity graph-based computations, systematically determining an effective feature sequencing. We incorporate GEDS within an order-aware efficient transformer framework, utilizing order-aware memory tokens that explicitly adhere to the derived feature sequencing via a dedicated loss function. Experimental results across multimodal benchmarks demonstrate that iStructTab effectively minimizes feature dispersion, improving predictive performance and robustness, and highlighting the significance of structured feature sequencing in multimodal learning.
106. 【2608.04329】An Analysis and Implementation of Seam Carving for Content-Aware Image Resizing
链接：https://arxiv.org/abs/2608.04329
作者：Francesco Tosoni
类目：Computer Vision and Pattern Recognition (cs.CV); Graphics (cs.GR)
关键词：locally minimal importance, classical content-aware image, content-aware image resizing, image resizing operator, repeatedly removing
备注： one column, 10 pages, 8 figures
点击查看摘要
Abstract:Seam carving is a classical content-aware image resizing operator that modifies the width or height of an image by repeatedly removing (or inserting) seams, i.e., 8-connected monotonic paths of pixels of locally minimal importance. Because seams bend around salient content rather than uniformly scaling or cropping it, the operator preserves vital image structures while discarding (or duplicating) low-energy regions. This article describes a C++ implementation of the operator that follows the original formulation of Avidan and Shamir (2007), including the optional forward-energy criterion subsequently introduced by Rubinstein, Shamir and Avidan (2008). The implementation supports image reduction, image enlargement via ordered seam insertion, multi-pass enlargement for large scale factors, a user-supplied weight mask for object protection and removal, along with dumping of energy maps and visualisation of seams. We detail the algorithm, its parameters and its computational complexity, discuss design choices with respect to the original descriptions, and illustrate the behaviour of the operator on natural images.
107. 【2608.04314】Adversarial Attacks for Good: A Survey of Proactive Protection across the Visual Content Lifecycle
链接：https://arxiv.org/abs/2608.04314
作者：Jiaming Zhang,Boyang Chen,Zherui Li,Fuyao Zhang,Xinyu Yan,Hong Xi Tae,Wenwen He,Xuan Wang,Siqi Guo,Junhao Dong,Kun Wang,Hanxun Huang,Yige Li,Xingjun Ma,Yang Cao,Lingjuan Lyu,Wei Yang Bryan Lim
类目：Cryptography and Security (cs.CR); Computer Vision and Pattern Recognition (cs.CV)
关键词：visual content enters, content enters, retains little technical, technical interventions, Abstract
备注：
点击查看摘要
Abstract:Once visual content enters an AI pipeline, its owner often retains little technical control over how it is used. Legal and regulatory remedies can address misuse, but many technical interventions must be applied earlier, when content is released or accessed. This survey examines the protective paradigm that has grown around this intervention point, which we call \emph{adversarial attacks for good}. Perturbations and structured signals long studied as attacks on learned models are instead applied by data owners, creators, platforms, or auditors to disrupt unauthorized automation or support later accountability. Five research communities have arrived at this inversion largely independently, each addressing a different stage of a visual asset's lifecycle: privacy filters against unwanted recognition at sharing time, unlearnable examples against unauthorized training, generative safeguards against malicious editing or imitation, adversarial CAPTCHAs for access control against automated agents, and provenance mechanisms for post-circulation attribution. Although developed in separate venues with incompatible success criteria, many of these methods exploit persistent gaps between human perception, semantic interpretation, and machine inference, suggesting that the paradigm remains relevant as visual pipelines evolve toward multimodal models and autonomous agents. To make their claims comparable, we evaluate all five families along shared axes of transferability, adaptability, and deployment readiness. Across the lifecycle, we find that most protections are still validated mainly against static or weakly adaptive adversaries, while evidence beyond controlled benchmarks remains scarce. We close by consolidating cross-stage countermeasures and open problems for robust, composable, and deployable owner-side protection.
108. 【2608.04302】CLIP-CC-Bench: Evaluating Paragraph-Level Video Descriptions in Video-Language Models
链接：https://arxiv.org/abs/2608.04302
作者：Mukhtiar Ali,Harsh Dubey,Sugam Mishra,Chulwoo Pack
类目：Computer Vision and Pattern Recognition (cs.CV); Information Retrieval (cs.IR); Multimedia (cs.MM)
关键词：generate accurate long-form, Benchmarking video-language models, Benchmarking video-language, long-form video description, single-sentence metrics
备注： Accepted and presented at EvalMG 2026, the Second Workshop on Evaluation for Multimodal Generation, co-located with ACM SIGIR 2026
点击查看摘要
Abstract:Benchmarking video-language models has largely focused on short clips and single-sentence metrics, leaving open whether current systems can generate accurate long-form, paragraph-level descriptions. We introduce CLIP-CC-Bench, an evaluation suite for long-form video description built from 5 hours of movie content segmented into 90-second clips, each paired with an expert-written paragraph-style reference. The evaluation suite employs an ensemble of five state-of-the-art LLM-based embedding models to increase reliability and mitigate single-model bias, and applies two complementary methodologies: (i) coarse-grained semantic matching and (ii) fine-grained semantic matching to compare model-generated descriptions against CLIP-CC-Bench references. Using this framework, we evaluate 17 state-of-the-art video-language models and report both their Borda-aggregated rankings and their average scores on CLIP-CC-Bench. We further quantify the protocol's internal reliability through inter-judge agreement and bootstrap ranking stability. We release standardized evaluation scripts, model outputs, and aggregation tools at this https URL to support reproducibility. CLIP-CC-Bench provides a practical evaluation framework for long-form video description, filling a gap left by existing short-clip and QA-only benchmarks.
109. 【2608.04292】Binding Biometrics with AI Agent Identifiers for Delegation of Authority
链接：https://arxiv.org/abs/2608.04292
作者：Joseph Geo Benjamin,Anil K Jain,Karthik Nandakumar
类目：Computer Vision and Pattern Recognition (cs.CV)
关键词：agentic artificial intelligence, artificial intelligence, systems has raised, proliferation of agentic, agentic artificial
备注： Accepted in IJCB sessions 2026
点击查看摘要
Abstract:The proliferation of agentic artificial intelligence (AI) systems has raised serious questions about the accountability for tasks performed by AI agents. Ideally, an AI agent must not be allowed to perform critical tasks without explicit authorization by a human operator. Since biometric recognition is one of the most reliable approaches for authenticating individuals, it has the potential to enable authenticated delegation of authority to AI agents. In this work, we present a framework called BIND, which leverages ideas from the field of biometric cryptosystems, to securely bind biometric data of the human user to the AI agent identity (ID) and authority scope (task-specific constraints) at the time of agent authorization. This token/identifier can be presented by the AI agent to an Identity Auditor, who simultaneously performs biometric authentication and recovers the agent ID and scope, thereby enabling real-time user authentication and establishing a non-repudiable proof of human control and delegation of authority. We also provide a practical implementation of the proposed BIND framework based on face features extracted using standard deep neural network models. To facilitate this implementation, we propose a feature adaptation module that transforms real-valued feature embeddings into fixed-length binary representations suitable for a fuzzy commitment construct based on turbo error correcting codes. Experiments demonstrate the practical feasibility of the proposed face cryptosystem, achieving a True Match Rate of $96\%$ at zero False Match Rate and supporting $1024$-bit agent tokens.
110. 【2608.04246】SAFECAST: Robust Failure Detection for VLA Policies with Contrast-Set Training and Calibration
链接：https://arxiv.org/abs/2608.04246
作者：Harshitha Rajaprakash,Aditeya Prajapati,Rong Xue,Abrar Anwar,Jesse Thomason
类目：Robotics (cs.RO); Computer Vision and Pattern Recognition (cs.CV)
关键词：altered initial states, distractor objects, deployment-time distribution shifts, policies often fail, altered initial
备注：
点击查看摘要
Abstract:Vision-language-action policies often fail under deployment-time distribution shifts such as clutter, distractor objects, lighting changes, novel objects, altered initial states, and reworded instructions. Hidden-state-based risk probes combined with functional conformal prediction can detect rollout failures, but their reliability depends on calibration data matching deployment conditions. We introduce SAFECAST, which leverages contrast set perturbations to improve hidden-state probe training and calibration for deployment time shift. SAFECAST statistically significantly improves failure detection ROC-AUC scores over a state of the art baseline in both real-world DROID and LIBERO simulation experiments across multiple VLM backbones. We further find that SAFECAST benefits most when both visual and language contrast set perturbations are used to augment data, and that with contrast set perturbations, sim-to-real calibration leads to better probes than using real rollout data only.
111. 【2608.04244】SIGNPOST-Bench: Benchmarking Text-Vision Conflict Resolution in Multimodal Large Language Models
链接：https://arxiv.org/abs/2608.04244
作者：Sirun Li,Minghao Liu,Ling Dai,Yong Li,Haoxin Lyu,Junting Zhou,Fan Zhang
类目：Computer Vision and Pattern Recognition (cs.CV); Computation and Language (cs.CL)
关键词：existing benchmarks rarely, benchmarks rarely reveal, Multimodal large language, make grounded predictions, make grounded
备注： 27 pages, 25 figures
点击查看摘要
Abstract:Multimodal large language models (MLLMs) make grounded predictions in real-world scenes by combining visual and textual cues, yet existing benchmarks rarely reveal how they arbitrate between these evidence sources when they conflict. We introduce SIGNPOST-Bench, a controlled counterfactual benchmark for evaluating text-vision conflict resolution. Each source image is transformed into a counterfactual quintuplet of Original, Blank, Similar, Random, and Adversarial variants. Synthetic, localized scene-text interventions are designed to preserve non-textual content, enabling paired measurements of changes in localization performance and directed shifts toward geographic targets introduced by conflicting text. SIGNPOST-Bench contains 5,111 counterfactual groups and 25,555 image variants from four datasets. We evaluate 20 MLLMs from seven providers. Compared with Original images, Adversarial variants raise median localization error from 282 km to 1,347 km, a 4.8-fold increase. Among geocodable adversarial samples, 6.5-20.1% of predictions lie less than 50 km from the injected target across models, and every evaluated model exhibits a positive mean paired reduction in target distance from Blank to Adversarial. Compatible, unrelated, and conflicting text replacements produce distinct effects on model predictions, while clean-input localization performance does not fully predict robustness to conflicting text. These results establish visual geolocation as a continuous diagnostic of scene-text arbitration and provide a controlled framework for evaluating how MLLMs resolve conflicting multimodal evidence.
112. 【2608.04230】ransferable Dual-Stream Representations for Mesoscale-Preserving Sea Surface Temperature Downscaling
链接：https://arxiv.org/abs/2608.04230
作者：Parth Doshi,Priyanka Aravindan,Vaishnav Vaidheeswaran,Md Mahbub Alam,Gabriel Spadon
类目：Machine Learning (cs.LG); Computer Vision and Pattern Recognition (cs.CV)
关键词：Deep learning models, minimize reconstruction error, physically meaningful multi-scale, meaningful multi-scale structure, preserve physically meaningful
备注：
点击查看摘要
Abstract:Deep learning models for scientific spatio-temporal downscaling often minimize reconstruction error while failing to preserve physically meaningful multi-scale structure. For sea surface temperature prediction, this can yield outputs that are numerically plausible yet overly smooth, missing mesoscale variability critical to regional ocean dynamics. Existing methods often focus on pixel-wise objectives or single-context conditioning, which limits their ability to preserve spectral fidelity and generalize across regions. To address this, we propose EddyFlow, a representation learning framework for kilometer-scale sea surface temperature downscaling that balances predictive accuracy, scale-dependent structure, and regional generalization. EddyFlow is trained on the Gulf of St.~Lawrence and evaluated in zero-shot and few-shot settings on the Bay of Fundy and the Gulf of Mexico. EddyFlow demonstrates that physics-informed representation learning reduces zero-shot RMSE by 21%, achieves up to 85.6% skill relative to persistence on unseen domains, and maintains near-ideal spectral fidelity with a PSD ratio of $\approx 1.00$.
113. 【2608.04224】OmniVR: Joint Video-Audio Conditional Generation for Restoring Degraded Historical Films
链接：https://arxiv.org/abs/2608.04224
作者：Xin Lu,Zihao Fan,Mingchen Zhong,Jie Huang,Xueyang Fu,Zheng-Jun Zha
类目：Computer Vision and Pattern Recognition (cs.CV)
关键词：Historical films suffer, leaving quality gaps, existing methods restore, modality independently, cross-modal inconsistency
备注：
点击查看摘要
Abstract:Historical films suffer from co-occurring visual and audio degradations---blur, noise, flicker, hiss, clipping, and dropout---yet existing methods restore each modality independently, leaving quality gaps and cross-modal inconsistency. We present OmniVR, the first joint audio-video generative restoration model. Built upon a 22B-parameter audio-video generation backbone, OmniVR formulates restoration as conditional generation within a unified multimodal DiT: the low-quality video and audio are encoded as latent conditions, combined with a fixed restoration prompt, and jointly denoised to recover visual structure, temporal motion, and acoustic detail under one coordinated objective. Three key designs enable this adaptation: (1) a joint audio-video degradation pipeline that simulates real old-film characteristics from Internet-collected data; (2) an architecture-preserving text-to-audio-video (T2AV) to audio-video-to-audio-video (AV2AV) transition with prompt annealing that maximally retains the generative prior; and (3) first-frame image-to-video (I2V) anchoring with loss reweighting and waveform supervision for long-video extrapolation and audio fidelity. We also propose OmniVRBench, the first benchmark that evaluates audio-video restoration across visual quality, audio quality, temporal consistency, and audio-visual synchrony on 200 real historical clips. OmniVR surpasses all prior methods on all six visual metrics, achieves the best audio quality, and produces natural colorization---the first method to jointly address all three aspects. Code and weights will be publicly released. Project Page: this https URL
114. 【2608.04210】PADFormer: Pose-agnostic Anomaly Detection from Sparse View Images
链接：https://arxiv.org/abs/2608.04210
作者：Ruiqi Wang,Yiming Qian,Fenggen Yu,Yuxuan Lu,Dakuo Wang,Hao Zhang,Jing Huang
类目：Computer Vision and Pattern Recognition (cs.CV)
关键词：Pose-agnostic Anomaly Detection, Pose-agnostic Anomaly, remains challenging, arbitrary viewpoints, significant pose variations
备注： Accepted to ECCV 2026 (oral)
点击查看摘要
Abstract:Pose-agnostic Anomaly Detection (PAD) remains challenging as anomalies can appear under arbitrary viewpoints, requiring methods to handle significant pose variations. Existing approaches rely on complex 3D reconstruction, which are computationally expensive and require extensive multi-view data. We propose PADFormer, a novel image-space approach that leverages Vision Transformer (ViT) to directly reconstruct anomaly-free versions of query images while preserving pose information. Our key insight is to adapt cross-view masked reconstruction for anomaly detection through training exclusively on normal data, combined with dynamic patch selection and spatial alignment mechanisms that enable effective learning from sparse reference views under significant pose variations. During inference, we perform multiple forward passes with different masking patterns to generate an ensemble of anomaly-free reconstructions, ensuring comprehensive coverage of the query image. Anomalies are detected by comparing these reconstructions with the query image. PADFormer achieves state-of-the-art results on the PAD benchmark while maintaining comparable performance on classic few-shot anomaly detection (FSAD) tasks, demonstrating superior efficiency and generalization without requiring 3D reconstruction.
115. 【2608.04196】SiMDex: Mining Similar Egocentric Videos for Cross-Embodiment Dexterous Manipulation
链接：https://arxiv.org/abs/2608.04196
作者：Nie Lin,Takehiko Ohkawa,Sijin Chen,Ruoshi Wen,Zhuohang Li,Liqun Huang,Zhengming Zhu,Yiming Bao,Yunfei Li,Minjie Cai,Xiao Ma,Wei Xu,Yoichi Sato
类目：Robotics (cs.RO); Computer Vision and Pattern Recognition (cs.CV); Machine Learning (cs.LG)
关键词：Recent years, benefits dexterous manipulation, scaling ego-centric human, ego-centric human videos, years have witnessed
备注： 12 pages, 4 figures. Project page: [this https URL](https://lin-nie.github.io/SiMDex/)
点击查看摘要
Abstract:Recent years have witnessed an explosive trend of scaling ego-centric human videos for robot manipulation, yet it remains unclear which data actually benefits dexterous manipulation. We present SiMDex, a similarity-based data mining framework that casts human data selection for VLA post-training in dexterous manipulation as a recommendation problem. For each robot demonstration, SiMDex employs a three-layer recall-ranking-re-ranking pipeline to extract task-relevant subsets from a pool of ~32M egocentric human samples, operating in a morphology-agnostic action space that requires no changes to VLA architecture or training. Against a strong baseline trained with an equal amount of randomly sampled human data, SiMDex uses only ~1.49M mined samples (5% of the pool) yet improves the overall success rate from 47.7% to 61.1%, showing that selective curation outperforms indiscriminate data mixing.
116. 【2608.04190】Adversarially Robust Abductive Fusion of Pre-trained Transformer-based Perception Models
链接：https://arxiv.org/abs/2608.04190
作者：Mario Leiva,Yue Ma,Qinru Qiu,Gerardo Simari,Paulo Shakarian
类目：Artificial Intelligence (cs.AI); Computer Vision and Pattern Recognition (cs.CV); Machine Learning (cs.LG); Logic in Computer Science (cs.LO)
关键词：Deploying pre-trained perception, majority voting trade, voting trade recall, Deploying pre-trained, pre-trained perception models
备注：
点击查看摘要
Abstract:Deploying pre-trained perception models in novel environments degrades their accuracy under distributional shift, and assembling them alone does not recover it: combiners such as majority voting trade recall for precision and are brittle to coordinated failures. Prior metacognitive methods learn logical rules that flag a model's errors, but rely on hand-authored domain-knowledge cues (object-size priors, segmentation masks) that do not transfer to genuinely novel scenes. We show that this metacognitive layer can be learned without any domain knowledge by exploiting vector-space geometry: per-model Label Vector Pools (LVP), built from each model's own training embeddings, yield error-detection rules from the geometry of detections relative to training-determined prototypes, reaching parity with domain-knowledge rules to within $0.002$ every F1 on test set. Because the approach remains neurosymbolic, these geometric rules share a single logical framework and can still be complemented by domain knowledge when available. We frame the fusion of multiple imperfect ViT-based detectors as a consistency-based abduction problem solved at test time by an exact Integer Program (IP) and a polynomial-time heuristic. On an aerial-imagery benchmark of 15 weather-shifted test sets and six ViT detectors, our domain-knowledge-free layer matches the strongest majority-vote variant on clean data (within $0.005$ F1) and, unlike every majority-vote baseline, retains its performance under a coordinated label-flipping attack: at a $90\%$ flip rate it averages $0.42$ F1 versus $0.35$ for MV-Plurality (a $22\%$ relative gain) and attains the highest F1 on \emph{every} test set once the flip rate exceeds $0.4$
117. 【2608.04175】riCLE: Tri-Modal Vision-Language Reasoning for Edge-Deployed Fine-Grained Clustering
链接：https://arxiv.org/abs/2608.04175
作者：Kishor Datta Gupta,Md. Mahfuzur Rahman,Fahad Rahman,Ahmed Rafi Hasan,Faysal Mehrab Chowdhury,Mohd Ariful Haque,Roy George
类目：Computer Vision and Pattern Recognition (cs.CV)
关键词：interpret aircraft imagery, limited memory, limited compute, intermittent connectivity, aerial observation
备注：
点击查看摘要
Abstract:Edge platforms used for aerial observation must interpret aircraft imagery under limited memory, limited compute, and intermittent connectivity. This setting is difficult for standard RGB-only recognition models and general-purpose vision-language models, especially when calibrated thermal and LiDAR aircraft data are unavailable. We present TriCLE, an application-oriented tri-modal vision-language system for aircraft taxonomic grouping under edge constraints. From a single RGB aircraft image, TriCLE generates a structure-preserving FLIR-style thermal view and a pseudo-LiDAR depth projection, then fuses the aligned views with task instructions in a compact Qwen3-VL backbone. The model is aligned to an expert aircraft taxonomy based on propulsion, airframe family, size, design era, and configuration, so its outputs reflect engineering-relevant similarity rather than only surface appearance. We evaluate supervised fine-tuning, rotation-preserving SFT, and three policy-alignment strategies: GRPO, GSPO, and DAPO. Sequence-level GSPO gives the strongest validation performance, reaching 88.33\% validation accuracy and 0.91 weighted F1 on valid aircraft outputs. On a held-out aircraft test partition, GSPO achieves 78.00\% accuracy and 0.793 weighted F1 while preserving 94.00\% parseable output formatting. After 4-bit quantization and attention-memory optimization, the aligned 4B model fits an 8GB deployment target and processes each tri-modal triplet in 1.48 seconds. These results support TriCLE as a practical prototype for interpretable, edge-feasible aircraft grouping, while emphasizing the need for further validation on real aligned thermal and LiDAR sensor streams.
118. 【2608.04154】RNet: Topography-Guided Frequency Rectification and Structure-Aware Decoding for Multimodal Paddy Rice Segmentation
链接：https://arxiv.org/abs/2608.04154
作者：Kaiwen Xiao,Chunlong Fu,Liping Zheng,Yanfeng Su
类目：Computer Vision and Pattern Recognition (cs.CV); Artificial Intelligence (cs.AI)
关键词：visually similar vegetation, alters optical appearance, terrain alters optical, similar vegetation, mountainous and hilly
备注： 16 pages, 9 figures, 6 tables
点击查看摘要
Abstract:Mapping paddy rice from very-high-resolution imagery in mountainous and hilly regions is difficult because terrain alters optical appearance and increases confusion with visually similar vegetation. We present TRNet for 0.5-m GaoJing-1 red--green--blue (RGB) imagery, a 5-m TanDEM-X digital elevation model (DEM), and derived slope. Separate visual and terrain encoders preserve modality-specific features. At an early encoder stage, Topographic Energy-Spectral Rectification applies terrain-conditioned low-frequency modulation and asymmetric high-frequency regulation to suppress steep-slope clutter and conditionally enhance compatible low-slope rice cues. The Topography-guided Paddy Structure Decoder combines semantic, rice--background boundary, and interior cues, using coarse terrain as context. Experiments used an Area A internal test set and held-out Area B, which had steeper terrain and lower rice prevalence. TRNet achieved rice intersection-over-union (IoU) values of 85.10\% and 80.68\%, exceeding the original Dual-Encoder U-Net by 9.15 and 18.83 percentage points, respectively. Ablation and slope-stratified results linked these gains to frequency rectification, structure learning, and fewer steep-terrain false positives. The results support coarse topography as a contextual prior for very-high-resolution paddy rice mapping.
119. 【2608.04147】LiNC: Lightweight Noise Correction via Per-Sample Trust and Gaussian Mixture Modeling
链接：https://arxiv.org/abs/2608.04147
作者：Abhishek Moturu,Babak Taati,Anna Goldenberg
类目：Machine Learning (cs.LG); Artificial Intelligence (cs.AI); Computer Vision and Pattern Recognition (cs.CV)
关键词：imaging datasets due, medical imaging datasets, annotation errors, inter-rater variability, common in medical
备注：
点击查看摘要
Abstract:Label noise is common in medical imaging datasets due to factors such as inter-rater variability, annotation errors, and ambiguous cases. This can severely undermine the reliability and clinical effectiveness of machine learning models trained using those datasets. To address this challenge, we introduce Lightweight Noise Correction (LiNC), which adds a single trainable trust parameter per training sample and learns when to use the observed label and when to defer to the model during a standard training loop. The key idea is to train using a convex combination of the observed label and the model's own predictive distribution, controlled by a per-sample trust parameter. We show that the gradient of this objective drives trust values in opposite directions for clean versus noisy samples in the early training phase, yielding separable trust distributions. We use a 3-component Gaussian Mixture Model over the trust values to separate them into clean, ambiguous, and noisy cases and then execute a short soft-correction phase on the noisy cases and a final hard correction phase. Experiments on ten 2D datasets from MedMNISTv2 under label noise of up to 50% show consistent gains in accuracy and strong mislabel detection. LiNC adds negligible asymptotic overhead: the training-time complexity remains dominated by the base network, with additional memory growing linearly with the size of the training set.
120. 【2608.04132】RUTA: Principled Visual Token Allocation via Rate-Utility Optimization
链接：https://arxiv.org/abs/2608.04132
作者：Jian Zou,Xiaoyu Xu,Zhihua Wang,Yilin Wang,Balu Adsumilli,Kede Ma
类目：Computer Vision and Pattern Recognition (cs.CV)
关键词：sequences make large, make large language, large language model-side, language model-side computation, resulting long visual
备注：
点击查看摘要
Abstract:High-resolution images and long videos provide vision-language models with rich context for multimodal reasoning and fine-grained perception, but the resulting long visual token sequences make large language model-side computation and memory costly. Existing visual token reducers often operate at prescribed rates, while recent methods adapt token counts across inputs using method-specific learned thresholds or importance predictors. We introduce RUTA, a principled Rate-Utility Token Allocation method that performs pre-LLM reduction by jointly learning which tokens to retain and how many to allocate to each image-query pair. RUTA constructs query-conditioned candidate tokens and predicts a retention probability for each candidate. During training, these probabilities parameterize independent Bernoulli gates, while their sum provides a differentiable training-time estimate of the token count for each pair. Retained tokens serve as anchors that aggregate information from non-retained tokens according to semantic affinity and spatial proximity. RUTA is optimized with a penalized rate-utility objective that balances downstream task loss against expected token usage. Averaged across five benchmarks and measured relative to each backbone's full-token baseline, RUTA uses only $2.0\%$ and $4.2\%$ of visual tokens while preserving $88.2\%$ and $94.4\%$ of task performance on LLaVA-NeXT-7B and Qwen3-VL-8B, respectively.
121. 【2608.04130】Radar4D-VLM: Proposal-Grounded Temporal 4D Radar Reasoning Across Frozen Language Models
链接：https://arxiv.org/abs/2608.04130
作者：Jiaju Han,Xuemeng Sun,Qike Zhang,Xiang Chen,Luwei Yang,Jiahuan Long,Yiwei Wei,Jiujiang Guo,Chengyin Hu
类目：Computer Vision and Pattern Recognition (cs.CV)
关键词：autonomous driving primarily, driving primarily rely, standalone perceptual modality, radar largely unexplored, temporal vision-language model
备注：
点击查看摘要
Abstract:Vision-language models for autonomous driving primarily rely on cameras and LiDAR, leaving 4D radar largely unexplored as a standalone perceptual modality despite its robustness to adverse visibility and direct measurement of radial velocity. We introduce Radar4D-VLM, a radar-only temporal vision-language model that reasons from ten consecutive 4D-radar point-cloud sweeps without camera or LiDAR input. Radar4D-VLM extracts geometrically grounded object proposals and organizes radar evidence into a compact hierarchy of object, scene, and kinematic tokens. A parameter-efficient projector maps these tokens into frozen language backbones, while auditable prediction heads jointly model object count, spatial distribution, motion state, collision risk, semantic category, and radial velocity. Radar4D-VLM combines proposal-grounded temporal object tokenization, global scene context, and explicit kinematic tokens within a unified frozen-backbone interface. On sequence-isolated K-Radar development validation, its Top-64 proposal recall reaches 98.13% at 4 m, exceeding fixed-lattice and uniform-random controls by 6.40 and 22.83 percentage points, respectively. We further evaluate 24 matched runs spanning eight frozen Qwen, Phi, Mistral, Llama, and Gemma backbones under an identical adaptation budget. The radar-token interface remains compatible across all five language-model families, while matched aligned, permuted, and no-language controls show sensor dependence but no stable direct-head gain from aligned language supervision. These results establish a reproducible foundation for radar-only multimodal scene and motion reasoning while separating interface compatibility from the benefit of language supervision.
122. 【2608.04127】aching Foundation Models to Read mmWave: Pose-Guided Kinematic Representation for Human Behavior Understanding
链接：https://arxiv.org/abs/2608.04127
作者：Duo Zhang,Zhehui Yin,Zhiyun Yao,Haotong Qin,Xusheng Zhang,Hongliu Yang,Jianyu Sun,Junzhe Wang,Zizhou Fan,Michele Magno,Daqing Zhang
类目：Computer Vision and Pattern Recognition (cs.CV)
关键词：Large language model, Large language, language model agents, Large, perceive human behavior
备注： 16 pages, 7 figures
点击查看摘要
Abstract:Large language model agents need to perceive human behavior in physical environments. Millimeter-wave (mmWave) radar provides a privacy-friendly and contactless sensing modality, but radar observations are difficult to align with language. Existing radar-language methods often rely on synthetic data or lack explicit supervision for human body structure and motion. We present mmMind, a radar-language model that uses synchronized 3D pose as training-only supervision. A spatio-temporal radar encoder is pretrained to capture body configuration and motion dynamics, after which the pose head is removed so that inference requires radar alone. The learned radar representations are then aligned with an LLM for behavior captioning and spatio-temporal question answering. We also introduce mmMind-Bench, a real-world mmWave-language benchmark containing 17.9 hours of recordings from 23 participants across seven indoor environments. Experiments on captioning, question answering, and unseen-action generalization show that mmMind consistently outperforms existing radar-language baselines, while ablations confirm the importance of pose-guided pretraining.
123. 【2608.04124】Perception Before Reasoning: Dynamic Latent Reasoning for Video Understanding and Question Answering
链接：https://arxiv.org/abs/2608.04124
作者：Haotian Xia,Zilin Xiao,Junbo Zou,Vicente Ordonez,Hanjie Chen
类目：Computer Vision and Pattern Recognition (cs.CV); Artificial Intelligence (cs.AI)
关键词：question answering requires, visual evidence, answering requires models, ground language queries, reasoning latents
备注：
点击查看摘要
Abstract:Video question answering requires models to ground language queries in visual evidence and, when necessary, reason over that evidence across time. Existing methods typically rely on long textual chain-of-thought rationales, even though many questions can be answered as soon as the relevant object, action, or frame is localized. We propose Dynamic Latent Reasoning (DyLaR), which first grounds a question in a short block of perception latents (continuous hidden states that encode query-relevant visual evidence), and then adaptively decides whether to append reasoning latents (continuous thoughts that reason over this evidence in latent space) before answering. DyLaR learns this behavior by grounding perception latents in verified visual evidence and distilling verified rationales into reasoning latents, followed by reinforcement learning that further refines when to reason. Across nine video benchmarks and four multimodal language model backbones, DyLaR improves average accuracy over same-backbone baselines while generating fewer than 20 tokens per query. On Qwen3-VL-4B, for example, DyLaR improves average accuracy over Qwen3-VL-4B-Thinking from 54.0 to 58.2 while reducing response length from 1,220.7 to 18.5 tokens per query. Ablations further show that grounded perception latents, rationale-supervised reasoning latents, and adaptive routing each improve accuracy.
124. 【2608.04121】Interpretable Fuzzy Inference for UAV Target Tracking Using Bounding-Box Geometry
链接：https://arxiv.org/abs/2608.04121
作者：Reza Ahmari,Ahmad Mohammadi,Vahid Hemmati,Nicholas Edmond,Hossein Z. Saghazadeh,Olusola Odeyomi,Parham Kebria,Abdollah Homaifar
类目：Robotics (cs.RO); Artificial Intelligence (cs.AI); Computer Vision and Pattern Recognition (cs.CV)
关键词：unmanned aerial vehicles, supports cooperative aerial, onboard vision remains, vision remains challenging, unmanned ground vehicles
备注：
点击查看摘要
Abstract:Vision-based guidance of unmanned aerial vehicles (UAVs) toward unmanned ground vehicles (UGVs) supports cooperative aerial--ground robotics, but reliable continuous yaw estimation from onboard vision remains challenging because of sensing uncertainty, limited computation, and the need for interpretable control. Existing deep-learning and geometric-reconstruction approaches often require large datasets, external localization, or complex modeling assumptions, reducing transparency and deployment suitability on resource-constrained platforms. We present an interpretable fuzzy-inference framework that generates continuous yaw commands from low-dimensional features extracted from YOLO boxes: target centroid location, area, and aspect ratio. No explicit geometric modeling is required. A Mamdani fuzzy system serves as an interpretable baseline using a shoulder--triangle--shoulder input partition. It is followed by a first-order Takagi--Sugeno model with three antecedent membership terms per input, whose parameters are derived from training-set quantiles, yielding a compact 27-rule structure. Evaluation uses 6{,}169 labeled samples from a VICON motion-capture environment. Across five randomized train--test splits, the Takagi--Sugeno model achieves a test-set mean absolute error of $0.140^\circ \pm 0.003^\circ$, a root mean squared error of $0.200^\circ \pm 0.008^\circ$, and a maximum absolute error of $1.254^\circ \pm 0.121^\circ$. Within-threshold accuracies are $99.676% \pm 0.270%$ for $\pm1^\circ$ and $100.000% \pm 0.000%$ for both $\pm3^\circ$ and $\pm5^\circ$. Directional consistency between image-plane horizontal displacement and predicted yaw sign reaches $90.254% \pm 0.612%$. These results show that the framework is transparent, data-efficient, computationally lightweight, and suitable for real-time vision-based UAV guidance toward mobile ground targets.
125. 【2608.04111】GEB-Bench: Abstract Structures Told in Many Voices
链接：https://arxiv.org/abs/2608.04111
作者：Tong Zhang,Zhiyuan Shi,Yun Peng,Tao Xie
类目：Computer Vision and Pattern Recognition (cs.CV); Computation and Language (cs.CL); Logic in Computer Science (cs.LO)
关键词：river delta, lightning bolt, abstract structural motif, Mobius twist, spirit of Godel
备注：
点击查看摘要
Abstract:Can a model look at a river delta and a lightning bolt and see that they share a structure? We introduce GEB-Bench, a benchmark whose unit is an abstract structural motif--self-reference, a strange loop, a Mobius twist--in the spirit of Godel, Escher, Bach. Each motif is told in several voices: a natural scene whose composition is the structure, a folk story whose telling enacts it through a mechanically checkable form device, a mathematical theorem, and a programmatic skeleton; surface parameters are declared nuisance variables and never scored. Motifs, voices, and the structural changes between them form a small cross-modal category, and GEB-Bench's tasks are its questions. Evaluating twelve open and proprietary models, we find that abstraction failure is lawful. The central finding is a gap between recognition and cross-voice mapping: models identify a structure within one voice far better than they carry it across voices; every model pays this tax, and mapping strong enough to narrow it appears only at the frontier tier. Two patterns support it. Errors align more strongly with the designed formal geometry than with measured perceptual geometries, and frontier models from different vendors converge on the same wrong answers; and surface complexity taxes every model that reads structure, with capacity buying headroom rather than immunity. GEB-Bench is fully generative and released with its pipeline.
126. 【2608.04106】LoRetta: A Foundation Model and Extensive Dataset for Global-Scale Remote Sensing Dense Image Matching
链接：https://arxiv.org/abs/2608.04106
作者：Siwei Yu,Han Guo,Zhenwei Shi,Zhengxia Zou
类目：Computer Vision and Pattern Recognition (cs.CV); Image and Video Processing (eess.IV)
关键词：underpins broad applications, establishes pixel-wise correspondences, matching establishes pixel-wise, image matching establishes, vision and photogrammetry
备注： 17 pages, 12 figures, 6 tables. Submitted to IEEE Transactions on Pattern Analysis and Machine Intelligence. Project page: [this https URL](https://siweiyu.com/work/loretta/)
点击查看摘要
Abstract:Dense image matching establishes pixel-wise correspondences and underpins broad applications in computer vision and photogrammetry. However, extending dense matching to global-scale remote sensing remains challenging because image pairs may differ in acquisition time, season, viewpoint, spatial resolution, and land-cover state. The resulting large geometric offsets, partial overlap, and intrinsically unmatchable regions make direct dense correspondence prediction unreliable and inefficient. We thus reformulate dense matching as localization-and-registration: first localizing the matchable overlap and affine geometry, then refining dense residuals within the aligned frame. Based on this formulation, we propose LoRetta, a foundation model coupling matchability-aware affine localization with guided dense registration. We also introduce LEVIR-GM, a global-scale multi-temporal optical matching benchmark with dataset-native matchability labels (103K aligned, 827K augmented pairs, six continents, five years, 0.5-1024 m resolution). We further establish a unified evaluation protocol for sparse, semi-dense, and dense matchers. On LEVIR-GM, LoRetta achieves an area under the curve (AUC) of 83.3%, outperforming the strongest baseline RoMa v2 by 1.6 points, with larger percentage of correct keypoints (PCK) gains of 6.5 and 8.2 points at 1 and 2 pixels, while reducing inference latency by 47.8%. Astronaut-to-satellite and unmanned aerial vehicle (UAV)-to-satellite geolocalization experiments further demonstrate its transferability as a reusable geometric aligner.
127. 【2608.04084】SpecDrop: Parameter-Free Category-Conditioned Routing for Modular Specialization
链接：https://arxiv.org/abs/2608.04084
作者：Boyao Wang,Zhihan Lei
类目：Machine Learning (cs.LG); Computation and Language (cs.CL); Computer Vision and Pattern Recognition (cs.CV)
关键词：networks pursue specialization, budgets learned routers, total-parameter budgets learned, underperform equal-weight No-Routing, learned routers
备注： 35 pages, 6 figures. Code: [this https URL](https://github.com/Beryex/SpecDrop)
点击查看摘要
Abstract:Mixture-of-experts (MoE) networks pursue specialization through learned routers, gates, and load-balancing losses, yet at matched total-parameter budgets learned routers can underperform equal-weight No-Routing baselines. Is the bottleneck the routing algorithm, or the alignment between training-signal granularity and the target categories? We probe the question with SpecDrop, a fixed parameter-free routing scheme: each of $K$ branches receives weight $p_a$ for its assigned category and a small leakage $p_i 0$ otherwise, merged through a category-independent fixed denominator, with no learned routing parameters and no auxiliary losses; the category label is required at inference. On vision tasks where each image has one superclass label (CIFAR-100 on ResNet-110; ImageNet-1K on ViT-S/16), SpecDrop reaches 79.23% on CIFAR-100 and 79.89% on ImageNet-1K, exceeding parameter-matched baselines that do not use the label (+4.75 over dense on CIFAR-100; +6.53 over the No-Routing+SE control on ImageNet-1K). These gains quantify what category supervision buys when deployed through routing -- not an advantage over label-aware deployments of the baselines: given the same label, masking a dense model's outputs is stronger for accuracy alone (85.2 / 83.7). SpecDrop's contribution is converting the label into trained-in modular structure: 58%/100% branch-category alignment, and masking gains of 0.00 (CIFAR) / +1.06 (ImageNet) -- the output-space restriction is largely internalized during training. On fuzzy partitions, where training units span multiple categories (SlimPajama-6B language modeling with a 30M Transformer; SuperNI instruction tuning over Llama-3.2-1B with LoRA), the routing mechanism reduces to the matched No-Routing controls within seed noise, the null our thesis predicts. Granularity alignment, not algorithm choice, localizes when routing helps. Code: this https URL
128. 【2608.04061】Advancing Utility Pole and Sign Detection Through Deep Learning
链接：https://arxiv.org/abs/2608.04061
作者：Carl Dickinson,Gaetano Di Caterina
类目：Computer Vision and Pattern Recognition (cs.CV)
关键词：critical public services, support power distribution, power distribution systems, public services, essential part
备注：
点击查看摘要
Abstract:Utility poles are an essential part of the infrastructure used to support power distribution systems and other critical public services. Their regular inspection is crucial to ensure the stability and safety of the electrical grid. A deep learning framework is presented for the automated detection, segmentation and lean angle estimation of wooden utility poles, and classification of attached electrical warning signs, using ground-level imagery. The system is trained on a custom dataset of 4,570 annotated images extracted from Google Street View, featuring challenging real-world scenes with visually ambiguous wooden poles lacking distinctive features. The proposed model is based on the Detection Transformer (DETR), suitably modified and trained on the custom dataset. The model outperforms standard object detectors (RetinaNet, Faster R-CNN, YOLOv3-Tiny), achieving a mean average precision of 90.43% for pole detection and 88.26% for sign detection. Extending this model with a segmentation head enables per-instance mask generation, which is then used to estimate pole lean angle. The model accurately estimates lean for 1,367 out of 1,433 test-set poles, with a mean absolute error of 1.01 degrees. Moreover, the custom dataset created in this work is also made publicly available to be used as a benchmark.
129. 【2608.04052】When Modalities Fail to Tango: Conformal Backdoor Detection in Multimodal Contrastive Learning
链接：https://arxiv.org/abs/2608.04052
作者：Yiming Chen,Kemou Li,Haiwei Wu,Jiantao Zhou
类目：Cryptography and Security (cs.CR); Computer Vision and Pattern Recognition (cs.CV); Machine Learning (cs.LG)
关键词：pre-trained MCL models, multimodal contrastive learning, garnered growing attention, downstream tasks critically, tasks critically depend
备注：
点击查看摘要
Abstract:Backdoor attacks in multimodal contrastive learning (MCL) have garnered growing attention in recent years, as many downstream tasks critically depend on pre-trained MCL models. Existing detection-based defenses predominantly rely on the CLIPScore metric, under the assumption that poisoned pairs exhibit lower semantic similarity between the image and the caption. However, we identify two critical flaws remaining in existing methods: (1) the substantial overlap between CLIPScore distributions of benign and poisoned pairs undermines the reliability of this metric, and (2) fixed-threshold detection cannot provide statistical guarantees for ambiguous samples within overlapping regions. To overcome these limitations, we propose integrating conformal prediction (CP), a statistical framework that quantifies uncertainty through nonconformity scores (NCSs), to establish provable confidence bounds for detecting poisoned image-caption pairs. Building on CP, we introduce CASCADE, a novel two-stage Coarse-to-Fine Conformal Backdoor Detection framework. The coarse-grained stage uses cross-modality consistency to identify high-confidence benign and poisoned pairs. In the fine-grained stage, a reference set is constructed from high-confidence poisoned pairs, and instance-level NCSs based on text-space similarity are computed for each sample in the unidentified subset. These NCSs measure conformity to the poisoning distribution and enable precise identification of latent poisoned pairs within the unidentified subset. Extensive experiments on the large-scale CC3M dataset demonstrate that CASCADE achieves an average FPR of 5.79% at 100% TPR and an average AUROC of 0.9867 across diverse attacks, while remaining effective against adaptive attacks.
130. 【2608.04030】NuclearDiffusion: Text-to-Image Foundation Models for Learning Nuclear Energy Concepts
链接：https://arxiv.org/abs/2608.04030
作者：Mohammed I. Radaideh,Jeremy Moon,Andre Gala-Garza,Emma Son,Yug Shah,Majdi I. Radaideh
类目：Graphics (cs.GR); Artificial Intelligence (cs.AI); Computer Vision and Pattern Recognition (cs.CV); Computers and Society (cs.CY); Machine Learning (cs.LG)
关键词：remains largely unexplored, domains remains largely, Generative artificial intelligence, engineering domains remains, artificial intelligence
备注： 29 pages, 10 figures, and 4 tables
点击查看摘要
Abstract:Generative artificial intelligence (AI) has transformed text-to-image synthesis, yet its ability to represent specialized engineering domains remains largely unexplored. As an exmaple in nuclear engineering, general-purpose foundation models frequently generate physically incorrect or conceptually inconsistent images because they lack domain-specific knowledge. This work presents one of the first systematic studies of domain adaptation for nuclear text-to-image generation through fine-tuning of open-source diffusion models. We curate a dataset of 1,000 captioned nuclear energy images spanning reactors, fuel cycles, radiation, and related concepts, and use it to fine-tune three state-of-the-art open-source models: Stable Diffusion XL (SDXL), SD-v3.5-Medium, and the flow-matching Flux.1 model. Their performance is evaluated using both quantitative image-similarity metrics and qualitative expert assessment against the corresponding zero-shot models. Fine-tuning substantially improves the fidelity of SDXL, provides only limited gains for SD-v3.5-Medium, and yields no measurable improvement for Flux.1, demonstrating that adaptation effectiveness depends strongly on the underlying generative architecture rather than model scale alone. We further compare the fine-tuned models against three leading commercial systems--GPT-Image-2, Gemini-3.1-Flash-Image, and Midjourney. Although GPT-Image-2 and Gemini generate convincing images for broad nuclear concepts, they frequently fail on specialized engineering prompts, where the fine-tuned open-source models produce more accurate and technically consistent outputs. These results establish domain-specific fine-tuning as a practical pathway for developing trustworthy generative AI tools for domain-specific applications.
131. 【2608.05054】MarsCast: Transfer Learning of AI Weather Foundation Models to Planetary Atmospheres
链接：https://arxiv.org/abs/2608.05054
作者：M.L. Carroll,J. Li,S.D. Guzewich,G.Villanueva,J.A. Caraballo-Vega,M.J. Frost
类目：Earth and Planetary Astrophysics (astro-ph.EP); Artificial Intelligence (cs.AI); Computer Vision and Pattern Recognition (cs.CV); Machine Learning (cs.LG)
关键词：GraphCast graph neural, Mars Climate Database, graph neural weather, Earth weather foundation, neural weather forecasting
备注：
点击查看摘要
Abstract:We investigate the transferability of Earth weather foundation models to planetary atmospheres by adapting the GraphCast graph neural weather forecasting model to Mars. While GraphCast achieves state-of-the-art performance for terrestrial forecasting, its applicability to non-Earth environments remains unexplored. Using the Mars Climate Database (MCD), which provides global atmospheric fields across vertical altitude levels (similar to Earth pressure levels), we evaluate zero-shot and fine-tuned GraphCast predictions of Martian temperature and wind fields. Zero-shot forecasts produce a surprisingly accurate depiction of current conditions but fail to reproduce diurnal variability and rapidly decay toward climatological mean states. To address this limitation, we fine-tune GraphCast using MCD variables and top-of-atmosphere solar radiation forcing while holding humidity constant. Fine-tuning enables rapid learning of Martian thermal variability. Within as few as 10 training epochs, the model begins to capture the diurnal cycle and forecasts up to 10 days reproduce seasonal and vertical temperature structure. Prediction quality improves with training sample size and exhibits sensitivity to seasonal initialization. These results demonstrate that Earth-trained AI weather models can be adapted to simulate Martian atmospheric dynamics, providing a pathway toward rapid planetary weather prediction to support mission operations, dust storm risk mitigation, and future human exploration.
