本篇博文主要展示每日从Arxiv论文网站获取的最新论文列表,以自然语言处理、信息检索、计算机视觉等类目进行划分。
统计
今日共更新913篇论文,其中:
- 自然语言处理72篇
- 信息检索13篇
- 计算机视觉224篇
自然语言处理
1. 【2611.24936】Jailbreak Detection on Large Language Models via Contrastive Prototype Structures
链接:https://arxiv.org/abs/2611.24936
作者:Yan Qiu,Yuanyuan Li
类目:Computation and Language (cs.CL)
关键词:large language models, harmful prompts remain, language models, Jailbreak attempts, prompts remain
备注:
点击查看摘要
Abstract:Jailbreak attempts and harmful prompts remain a challenge for the safe deployment of large language models. We study whether contrastively learned representations can support safety detection without training an additional downstream classifier. We propose Jailbreak Detection with Prototype Inference (JDPI), which uses binary safety labels to construct harmful and benign prototypes and makes predictions from relative prototype distances. JDPI uses only binary safety labels for encoder adaptation, prototype construction, and validation. In our setup, it uses fewer labeled examples than the classifier baseline (Method A) by avoiding the separate split used to train the downstream classifier. Across four benchmarks covering different input formats, contrastive fine-tuning improves JDPI's detection accuracy by 9.1-18.0 percentage points. Using the same encoder, JDPI achieves slightly higher ROC-AUC on one benchmark and stays within 0.017 of the baseline across all four. With dataset-specific adaptation, JDPI exceeds the released Llama Guard 3-8B baseline by 0.173-0.310 in ROC-AUC across the four benchmarks. In multi-turn replay on TomGibbs, JDPI uses accumulated user history as the conversation unfolds, achieving higher conversation-level accuracy than when it uses the current turn alone. Across 12 directed transfers, JDPI outperforms the classifier by an average of 1.59 percentage points in accuracy, demonstrating stronger cross-dataset robustness.
2. 【2611.24888】Who are you? The Personal Imitation Game
链接:https://arxiv.org/abs/2611.24888
作者:Muhammad Rizky Wellyanto,Hajara-Yasmin Isa,Kexin Xue,Vishal Ravichandran,Wenpeng Lu,Jerry O. Talton,Ranjitha Kumar
类目:Human-Computer Interaction (cs.HC); Artificial Intelligence (cs.AI); Computation and Language (cs.CL)
关键词:Large Language Models, Personal Imitation Game, Large Language, Language Models, Alan Turing
备注: 19 pages, 6 figures, 5 tables
点击查看摘要
Abstract:Large Language Models (LLMs) now perform strongly on Alan Turing's 1950 Imitation Game, so we propose a harder challenge, the Personal Imitation Game, which asks whether a machine can replicate a specific individual in conversation rather than a generic human. Testing with 30 participants, we establish a human baseline and report the identity signals interrogators rely on: stylometric features, personality traits, and biographical facts. To demonstrate the framework, we design an LLM-based system that plays the game for one of the paper's authors, drawing on an autoethnographic dataset spanning ten years of personal digital communication. In a 20-person deception study, interrogators choose our system over the author's own friends in 7 of 10 rounds, and the author over our system in 13 of 20; neither rate is distinguishable from chance at this sample size. We consider design improvements, applications of personal AI agents, and implications for identity and privacy.
3. 【2611.24887】Ground, Iterate, Trace: Decomposing AI Research Ideation
链接:https://arxiv.org/abs/2611.24887
作者:Samson Yu,Qiran Zou,Jing Yu Lim,Rushi Shah,Anirudh Goyal,Dianbo Liu
类目:Computation and Language (cs.CL)
关键词:Large language models, prior scientific work, automated ideation effective, Large language, generate plausible research
备注: 56 pages
点击查看摘要
Abstract:Large language models have been widely used to generate plausible research ideas, yet it remains unclear what makes automated ideation effective and how generated novel proposals arise from prior scientific work. We introduce the Creative AI Researcher (CAR), an agentic framework combining literature grounding, iterative development and explicit conceptual operators. CAR constructs a semantic knowledge graph from retrieved literature and equips an ideation agent with graph-retrieval tools and four operators: bisociation, analogy, exploration and elaboration. Each operator call is checked against structural acceptance conditions and recorded, producing a trace of intermediate transformations and declared literature sources. We evaluate CAR with two language-model backbones across 40 LiveIdeaBench research topics and two execution benchmarks. CAR achieves significantly higher methodological novelty (MN) than single-shot prompting and The AI Scientist. Its methodological novelty does not differ significantly from the Research Ideation Agent, while its feasibility scores are higher on both backbones. With a shared experiment executor, CAR achieves the highest pairwise win rate among the evaluated AI methods on FML-bench-Lite and BioXArena. In a ten-condition ablation study, MN point estimates favor CAR over one-step proposal on both backbones, but no MN contrast survives correction for multiple comparisons. The results do not establish a consistent quality benefit from explicit operators beyond free-form iteration. CAR makes the sequence of operations and declared literature sources inspectable, separating judged idea quality from procedural traceability.
4. 【2611.24849】Can Programs Replace LLM Judges? Executable Program Judges for Scalable and Efficient Relevance Labeling
链接:https://arxiv.org/abs/2611.24849
作者:Eugene Agichtein,Charles L. A. Clarke,Kaustubh Dhole,William Stern
类目:Information Retrieval (cs.IR); Computation and Language (cs.CL)
关键词:Large language model, Large language, create large-scale relevance, LLM judges, LLM
备注:
点击查看摘要
Abstract:Large language model (LLM) judges have become a common way to create large-scale relevance judgments for retrieval evaluation. However, each judgment requires a potentially expensive call to a frontier model, and the judgments are hard to interpret and may not be reproducible. In this paper, we ask whether an LLM can write a readable program for each topic that replaces its per-document judgments. To address these issues, we propose ExecJudges, a method for generating fast, inspectable and scalable judges for automatic relevance labeling. Specifically, ExecJudges uses a frontier LLM to turn assessment guidelines into executable programs, which rely only on lexical matching, standard NLP libraries like parsers, and pool statistics. An ExecJudges-L variant also uses examples of human labels if available. We evaluate ExecJudges on seven TREC collections by comparing the produced labels with trained NIST assessors, frontier LLM judges, and recently proposed per-topic classifiers. Our results show that ExecJudges agrees with NIST assessors at the level of the Claude Sonnet 5.5 LLM judges on TREC DL 2019-2021. With a small number of human labels per topic, the ExecJudges-L variant agrees with NIST more than per-topic classifiers with the same labels on all seven collections, and either exceeds the agreement of LLM judges provided the same labels or is within about 0.05 of it in Krippendorff's alpha. The labels rank the official TREC DL runs with Kendall's tau of 0.87 to 0.94, and writing the programs incurs a one-time cost of $0.17 to $0.49 per topic, with no additional costs for grading. Our results suggest that ExecJudges offers a reproducible, inspectable, efficient and cost-effective alternative to LLM judges for labeling deep result pools for system evaluation or ranking improvements.
5. 【2611.24845】Vibe-Trading: Towards Agent-Native Trading
链接:https://arxiv.org/abs/2611.24845
作者:Haozhe Wu,Zhonghang Li,Chao Huang
类目:Artificial Intelligence (cs.AI); Computation and Language (cs.CL); Multiagent Systems (cs.MA); Computational Finance (q-fin.CP); Trading and Market Microstructure (q-fin.TR)
关键词:produce multi-step outputs, call tools, retrieve information, multi-step outputs, produce multi-step
备注: Technical report. 16 pages, 8 figures, 4 tables. Code: [this https URL](https://github.com/HKUDS/Vibe-Trading)
点击查看摘要
Abstract:Existing AI agents can retrieve information, call tools, run experiments, and produce multi-step outputs. Yet most are built around relatively static workflows and struggle to preserve reliable state as the environment changes. Financial trading makes this limitation especially consequential: prices, news, market conditions, portfolio positions, and available capital evolve continuously. A result is useful only when it remains tied to the correct time, data snapshot, and execution context. We present Vibe-Trading, a comprehensive agent-native trading framework and all-in-one workspace for building, evaluating, and evolving agents for financial trading. It unifies market research, financial data, strategy generation, backtesting, portfolio analysis, paper trading, multi-agent collaboration, and optional broker execution. Its central contribution is an architecture that keeps results, evidence, research history, and trading authority connected throughout the research-to-action process. Each result retains its source, temporal scope, configuration, and producing run. Shared evidence grounding enables verification, versioned research records support reproducible continuation, and an explicit authority gate separates strategy research from live execution. Together, these designs provide a practical foundation for trading agents whose results can be traced, reproduced, and acted on only under explicit authority.
6. 【2611.24844】General progress substitutes for specific training data in language models
链接:https://arxiv.org/abs/2611.24844
作者:Mark Marosi,Peter Antal
类目:Computation and Language (cs.CL); Artificial Intelligence (cs.AI)
关键词:developers withhold data, data, learning capabilities, developers withhold, models
备注: 49 pages, 16 figures, 6 tables. Code: [this https URL](https://github.com/Mapika/progress-substitutes-data)
点击查看摘要
Abstract:Language models acquire their capabilities from training data, and developers withhold data during training to keep models from learning capabilities that could be misused. How long this delays a capability, and how much data is needed to acquire it later, is unknown. Here we show that the general progress a model makes on the rest of its training data substitutes for the data that was withheld. We withheld the training signal for individual skills in models of chess, Othello, molecular strings, bracket languages and web text. The delay of a withheld skill in larger models was predicted from smaller ones. A chess model never explicitly trained to respond to check ranked the legal responses almost as well as a model that was. When models were then trained on the withheld data, they acquired the skill with a small fraction of the data needed from initialization, and this fraction decreased as the models progressed. A control rule of equal difficulty was learned much more slowly. A programming language withheld from the training text of a 7-billion-parameter model was regained in the same way. Data filtering therefore delays a capability, and the data required to regain it decreases as models improve.
7. 【2611.24830】A Foundation Model to Measure and Predict Human Behaviour
链接:https://arxiv.org/abs/2611.24830
作者:Haoran Ye,Tianrui Huo,Hangzhen Li,Xin Zhang,Guojie Song
类目:Computation and Language (cs.CL); Artificial Intelligence (cs.AI); Human-Computer Interaction (cs.HC)
关键词:Large language models, Large language, infer psychological attributes, predict individual behaviour, simulate human populations
备注: 48 pages, 6 figures
点击查看摘要
Abstract:Large language models (LLMs) are increasingly used to infer psychological attributes, predict individual behaviour, and simulate human populations. However, despite generating plausible responses, they often misrepresent individuals and distort population statistics. Here we introduce Chiron, a foundation model to measure and predict human behaviour. We jointly fine-tune an LLM to represent psychological constructs, predict individual behaviour, and align population response distributions with human statistics. The training and evaluation data span 36 behavioural domains and 78 social science experiments, comprising 288,307 person records, 1,411,750 atomic behavioural decisions, and 99,953 psychometric assessments over 109 psychological constructs. In our evaluations, Chiron's psychological measurements agree more closely with human self-reports and exhibit fewer biases that undermine psychometric validity. It predicts individual behaviour more accurately than frontier LLMs (up to 28.4% relative improvement). In simulations of social science experiments, it more accurately captures the direction and magnitude of human experimental effects (82.2--94.3% lower effect-size estimation error). Together, these results establish Chiron as a unified computational instrument for measuring and predicting human behaviour.
8. 【2611.24809】On-Policy Morphing into Hybrid Attention Models
链接:https://arxiv.org/abs/2611.24809
作者:Disen Lan,Xuanda Wang,Jianbin Zheng,Huafeng Kuang,Wei Tang,Yu Cheng
类目:Computation and Language (cs.CL)
关键词:models combine full, attention models combine, retain full attention, Hybrid attention models, Converting pretrained Transformers
备注:
点击查看摘要
Abstract:Hybrid attention models combine full and linear attention to achieve a better quality--efficiency trade-off. Converting pretrained Transformers into such models requires identifying which layers should retain full attention under a prescribed budget. We identify two off-policy mismatches in FlashMorph's layer selection: (i) continuous branch interpolation and teacher-forced hidden states differ from discrete hybrid computation; and (ii) frozen model weights fail to account for parameter adaptation during subsequent recovery. We introduce FlashMorph2, an on-policy layer-selection method that (I) evaluates sampled discrete hybrid architectures end to end using their own hidden states, and (II) jointly optimizes layer-selection scores and model weights under the recovery objective to reflect post-adaptation performance. Experiments across Qwen3 model scales and linear-attention variants show that FlashMorph2 consistently outperforms existing layer-selection methods, improving average scores over FlashMorph by $+2.42$ points on RULER, $+1.08$ on LongBench, and $+0.76$ on recall-intensive tasks, while completing layer selection in only $0.72$--$3.57$ GPU hours across 0.6B--8B models, demonstrating high efficiency and scalability.
9. 【2611.24793】he Elephant in the Question: Unified Models Picture the Concepts Behind Their Answers
链接:https://arxiv.org/abs/2611.24793
作者:Tobias Braun,Marcus Rohrbach,Lorenzo Cavallaro
类目:Computer Vision and Pattern Recognition (cs.CV); Computation and Language (cs.CL); Machine Learning (cs.LG)
关键词:reasoning steps underlying, Visualizing intermediate representations, Visualizing intermediate, inspect the reasoning, reasoning steps
备注: 32 pages, including supplementary material. Code: [this https URL](https://github.com/multimodal-ai-lab/V-lens)
点击查看摘要
Abstract:Visualizing intermediate representations could help us inspect the reasoning steps underlying a model output. Unified multimodal models offer this opportunity by combining understanding and generation in one architecture. Their image generators can access representations formed while processing text. We introduce the V-lens, which uses this connection to visualize a chosen token's hidden state without training a decoder. Across four diffusion-based and two purely autoregressive unified models, the resulting images reveal key intermediate concepts that connect a question to its answer even when the concepts are never named explicitly. Controlled tests show how this content changes with context, token position, and depth. We then introduce lens tuning, which supervises selected hidden states with pictures through the V-lens. Across BAGEL, SenseNova-U1, and HunyuanImage-3.0, the rate of correct visualizations of pronoun referents rises from 2-11% to 86-94%. Surprisingly, such visual supervision also improves text answers without supervising the answer tokens directly. Code is available at this https URL.
10. 【2611.24721】Induction Head Origins of SLAM-ASR Speech Recognition Models
链接:https://arxiv.org/abs/2611.24721
作者:Artem Fedorchenko,Tanel Alumäe
类目:Computation and Language (cs.CL)
关键词:frozen language model, small trained map, language model sufficient, frozen language, language model
备注: Submitted to ICASSP
点击查看摘要
Abstract:What makes a small trained map from encoder states to the embedding space of a frozen language model sufficient for competitive speech recognition? The premise of the SLAM-ASR architecture is that a frozen language model possesses linguistic knowledge rich enough to perform the recognition task. This assumption, while reasonable, provides little insight into the mechanisms underlying these models. We therefore seek the concrete internal structures on which they rely. This work shows that SLAM-ASR models rely on induction heads of the frozen language model: the same heads that copy repeated token sequences in text are also important for mapping speech embeddings to transcript tokens. Moreover, we find pairs of heads that behave as a constrained form of differential attention, cancelling attention noise and sharpening attention onto the previously generated word. Finally, we show how induction heads could be used to improve transcription quality under noisy audio.
11. 【2611.24719】DRIFT: Detachable Reasoning for Runtime-Flexible and Efficient End-to-End Driving
链接:https://arxiv.org/abs/2611.24719
作者:Emad Aghajanzadeh,Bethany Anne Wong,Leonhard Kunze,Moritz Thoma,Maximilian Forstenhäusler,Nael Fasfous,Manoj Rohit Vemparala,Pierpaolo Mori,Christos Anagnostopoulos,Daniel Mueller-Gritschneder,Ulf Schlichtmann
类目:Computer Vision and Pattern Recognition (cs.CV); Computation and Language (cs.CL); Machine Learning (cs.LG); Robotics (cs.RO)
关键词:demonstrated strong potential, providing scene description, integration of vision-language, potential by providing, autonomous driving
备注: 11 pages, 4 figures, 7 tables. Project page: [this https URL](https://drift-e2e.github.io/)
点击查看摘要
Abstract:The integration of vision-language models (VLMs) into end-to-end (E2E) autonomous driving has demonstrated strong potential by providing scene description and reasoning capabilities. However, VLMs are computationally expensive, and switching language reasoning on and off at runtime without hurting closed-loop performance has not been demonstrated, which hinders practical adoption. To address these issues, we propose DRIFT, an E2E autonomous driving framework that integrates a lightweight language model (LM) for visual question answering and action commentary. By efficiently projecting intermediate driving representations into the language embedding space, we demonstrate that a compact LM (150M parameters) can achieve strong scene description and action commentary. Furthermore, we introduce a runtime-flexible planning module that is trained on surrogate reasoning features distilled from the LM and can be conditioned on either the LM or the surrogate reasoning features at inference. This design enables bypassing the LM entirely when required by deployment constraints, without sacrificing driving performance. With only 255M parameters, DRIFT reaches up to a Driving Score (DS) of 97.03 and a Success Rate of 89.39% (mean over three runs) on Bench2Drive by interleaving its LM and surrogate reasoning at a chosen reasoning rate. This is on par with the strongest non-language models while being far more compact than existing VLM-based models. DRIFT further achieves competitive results on the real-world NAVSIM and AlpaSim benchmarks at significantly lower computational cost than existing VLM-based approaches. Project page: this https URL
12. 【2611.24711】Openhagen Worlds: Calibrated Environments for Measuring and Training Scientific Discovery Agents
链接:https://arxiv.org/abs/2611.24711
作者:Alexander Boesgaard Lorup
类目:Machine Learning (cs.LG); Artificial Intelligence (cs.AI); Computation and Language (cs.CL)
关键词:choose informative experiments, unstated scoring rules, agent runs experiments, skill from recall, science must choose
备注: 41 pages. Ancillary files: a public sample of 48 development instances with scripted-oracle episodes. Hosted evaluation: [this https URL](https://eval.openhagen.ai)
点击查看摘要
Abstract:Agents that do science must choose informative experiments and reason from their results, but most discovery benchmarks cannot separate that skill from recall, unstated scoring rules or shortcut policies. We introduce Openhagen Worlds, a suite of 27 procedurally generated environments (some with finite truth spaces), from classic puzzles of experimental reasoning to simulator-grounded physics and chemistry, in which an agent runs experiments under a budget and commits an answer that a deterministic verifier checks against a hidden truth. Every observation states the complete scoring rule, and each of the 178 difficulty levels is calibrated: it is admitted only if, on 64 development instances, a scripted oracle succeeds on at least 0.75, the strongest plausible non-deductive policy on at most 0.25 and each gated shortcut on at most 0.35. The 92 core levels span all 27 families and are sized to train small models; the 86 frontier rungs extend the 15 discovery worlds with harder questions and with campaigns of three, five and eight or more committed stages on one hidden system. On the same 84 withheld frontier instances, one per family and rung, DeepSeek V4.1 Flash solves 32 at its default reasoning effort with 16,384 tokens per call and 57 at its maximum effort with up to 393,216 tokens per call (oracle: 79); the headroom that remains is largest in eight-stage campaigns. On-policy expert iteration gives a 9B model modest gains: its core-tier success on trained families rises from 0.090 to 0.148, partly through better action format, with no detectable loss on GPQA Diamond. A hosted service plays the core tier's withheld instances.
13. 【2611.24699】When Does Continual Pre-Training Help? A Controlled Study of Dialectal Adaptation for Québec French
链接:https://arxiv.org/abs/2611.24699
作者:Eeham Khan,Richard Khoury,Leila Kosseim
类目:Computation and Language (cs.CL); Artificial Intelligence (cs.AI); Machine Learning (cs.LG)
关键词:Continual pre-training, non-standard language varieties, adapt pretrained language, scores adequately capture, capture its effects
备注: EACL 2027 Main Conference
点击查看摘要
Abstract:Continual pre-training (CPT) is widely used to adapt pretrained language models to dialectal and non-standard language varieties, yet it remains unclear when this additional training is necessary, what drives its benefits, and whether aggregate evaluation scores adequately capture its effects. In this paper, we present a controlled study of CPT for Québec French language understanding across nine LLMs spanning different families and scales. Using a budget-matched protocol of 30M tokens per experiment, our results show that the benefit of CPT depends strongly on the starting model's existing competence in the target variety. When CPT is beneficial, the improvements are driven more by corpus composition than by data volume, with literary text accounting for most of the gain. Paired Québec and standard French evaluations further show that CPT primarily improves expression comprehension rather than grammatical competence, and that corpus register determines which regional expressions are acquired. These gains also involve capability-specific trade-offs, including declines in reading comprehension that aggregate scores can conceal. Overall, these findings show that CPT for dialectal adaptation should not be treated as a uniformly beneficial default. Instead, adaptation strategies should begin with a careful assessment of the base model's existing competence, accompanied by source-level data analysis and task-level evaluation.
14. 【2611.24654】CurvDelta: Low-Curvature Delta Modeling For Surgical Learning and Unlearning
链接:https://arxiv.org/abs/2611.24654
作者:Zeguan Xiao,Zhuocheng Liang,Yun Chen,Yong Wang,Guanhua Chen,Pinyan Lu
类目:Computation and Language (cs.CL)
关键词:adapting large language, large language models, central challenge, challenge in adapting, adapting large
备注:
点击查看摘要
Abstract:A central challenge in adapting large language models (LLMs) is capability preservation: an adaptation can achieve its target while degrading general capabilities. We introduce CurvDelta, a scalable framework that incorporates the preservation constraint into the parameterization of the model update. By analyzing the local geometry of the preservation constraint around the pre-adaptation model, CurvDelta constructs a low-curvature space in which parameter updates leave the preservation loss nearly invariant. We make its construction practical for LLMs through Kronecker-factored approximate curvature (K-FAC) and a factorized projection. We also introduce CurvLoRA, a low-rank variant within the same preservation geometry, designed for more efficient adaptation. Both methods apply to preservation-constrained adaptation tasks such as capability learning and selective unlearning. We evaluate our methods on mathematical reasoning, code generation, and knowledge unlearning. For mathematical reasoning and code generation, our approaches achieve competitive downstream performance while better preserving general capabilities, offering a favorable acquisition--retention trade-off relative to the baselines. For knowledge unlearning, our main results on TOFU and MUSE show a better balance between targeted forgetting and knowledge retention than the baselines, with retention performance close to the from-scratch retraining reference.
15. 【2611.24651】Piccolo: Co-Evolving Decision Models and Data through Recursive Self-Improvement
链接:https://arxiv.org/abs/2611.24651
作者:Zhenqing Ling,Qirui Jiao,Daoyuan Chen,Cheng Chen,Xinji Mai,Haowen Gao,Zhuo Zhang,Xuchen Pan,Mengsu Ding,Jiaji Deng,Yuexiang Xie,Xiaoming Qin,Weirui Kuang,Zhaoyang Liu,Yaliang Li,Bolin Ding,Jingren Zhou
类目:Computation and Language (cs.CL); Artificial Intelligence (cs.AI); Computer Vision and Pattern Recognition (cs.CV)
关键词:Agentic Data Engineering, Engineering and recursive, coevolves model capabilities, Index Vision Full, Agentic Data
备注: 22 pages, 13 figures, 11 tables. Project page and blog: [this https URL](https://decisionplayground.agentscope.io/piccolo.html)
点击查看摘要
Abstract:We present Piccolo, a family of multimodal decision models (0.8B to 27B) that coevolves model capabilities and training data via Agentic Data Engineering and recursive self-improvement (RSI). The models predict probabilities over candidate answers for general-purpose text and visual judgments, agent harnesses, data workflows, and interactive games. Piccolo-27B tops the Decision Index Vision Full leaderboard with a score of 75.59, while Piccolo-9B and Piccolo-4B achieve 68.35 and 67.46, respectively. All three rank first on the Private benchmark in their respective model-size categories. Research agents analyze model behavior, construct training data, conduct experiments, and use the results to refine subsequent proposals. Semantic slices and a shared research memory connect these actions across the model family's development. With training recipes comprising hundreds of thousands of examples, data selection and feedback-driven refinement are central to the self-improvement process. Our large-scale RSI exploration yields three counter-intuitive insights for decision-model development: (1) Backbone-recipe co-adaptation outweighs raw model recency: earlier backbones can surpass newer generations on visual tasks under identical recipes; (2) Targeted data curation enables 4B-9B models to rival 27B baselines on specialized workflows; and (3) Continuous loss reduction can decouple from and even degrade discrete decision accuracy. Together with the released weights and tools from the Data-Juicer and AgentScope ecosystems, Piccolo provides an actionable blueprint for building frontier decision models and scaling data-driven RSI.
16. 【2611.24650】Self-Rewarded Reinforcement Learning for Better-Structured Medical Explanations in Small Language Models
链接:https://arxiv.org/abs/2611.24650
作者:Prakhar Mishra,Bhavya Yajush Awasthi,Tanuja,Nishant
类目:Artificial Intelligence (cs.AI); Computation and Language (cs.CL)
关键词:Small language models, on-premises clinical deployment, poorly structured compared, Small language, large language models
备注: 14 Pages, 10 Figures
点击查看摘要
Abstract:Small language models (SLMs) can run on commodity hardware and support privacy-preserving, on-premises clinical deployment, but their medical explanations are often short and poorly structured compared with large language models. Recent supervised approaches such as Meerkat transfer chain-of-thought reasoning into 7B-8B backbones, yet their training objective does not directly target the style or organization of the generated rationales. Building on the instruction-tuned Mistral-7B backbone, we investigate whether a lightweight reinforcement learning (RL) step can improve explanation structure without human preference this http URL apply Proximal Policy Optimization (PPO) with LoRA adaptation to 2,000 Meerkat-derived medical prompts, guided by a simple self-supervised reward that favors longer, lexically diverse, and well-punctuated responses while penalizing very short ones. Training for about 170 PPO steps on a single NVIDIA A100 GPU yields a stable increase in the average reward. Quantitative and qualitative comparisons show that the RL-tuned model produces longer, more organized, multi-point explanations while preserving the underlying medical this http URL discuss the promise and the limitations of self-rewarded RL for shaping the style of medical explanations.
17. 【2611.24636】Clover-35B-A3B-Preview Technical Report
链接:https://arxiv.org/abs/2611.24636
作者:DiffuSpace Technical Team
类目:Computation and Language (cs.CL); Artificial Intelligence (cs.AI); Machine Learning (cs.LG)
关键词:billion total parameters, billion activated parameters, Gated Delta Networks, supporting thinking-mode generation, billion total
备注: Technical Report. Code: [this https URL](https://github.com/DiffuSpace/Clover-35B-A3B-Preview;) Weights: [this https URL](https://huggingface.co/Diffuspace/Clover-35B-A3B-Preview)
点击查看摘要
Abstract:We introduce Clover-35B-A3B-Preview (Clover), a mixture-of-experts diffusion language model with 35 billion total parameters and 3 billion activated parameters, supporting thinking-mode generation. Clover adapts Gated Delta Networks to block diffusion by combining recurrent memory of previously generated context with bidirectional softmax attention inside the current block. Historical memory readout competes with current-block tokens in a shared softmax, with learnable block-size calibration and a separate, zero-initialized right-context convolution branch. Clover leads the seven evaluated agent metrics, including 73.6 on SWE-bench Verified and 51.7 on Terminal-Bench 2.1. On internal agent workloads, it reaches a reported generation speed of 680 tokens/s using one NVIDIA GB300 GPU at BF16 precision and batch size one. We organize the evaluation around model quality, drafting and verification costs, and deployment under constrained memory. We also describe a decision-model variant that generates answers to related questions in one denoising pass. On an internal set of 500 groups, joint prediction raises question accuracy from 90.37% to 93.88% and all-correct group accuracy from 65.4% to 77.0%.
18. 【2611.24626】ransferring an analysis of human brain organization to language models
链接:https://arxiv.org/abs/2611.24626
作者:Donghyeok Lee,Sangwon Yu,Sangkwon Park,Hye Youn Park,Munyoung Chang,Sungroh Yoon
类目:Machine Learning (cs.LG); Computation and Language (cs.CL)
关键词:human cortical development, functional systems, increasingly differentiated, systems become increasingly, cortical development
备注: 5 figures, 5 extended data figures, 5 extended data tables; includes Supplementary Information
点击查看摘要
Abstract:During human cortical development, functional systems become increasingly differentiated. We asked whether improved reasoning in language models involves a comparable regional reorganization. Applying diffusion-map embedding, used to characterize cortical organization, to matched base and reasoning-trained models from three families, we found broadly preserved gradient profiles with localized changes within a contiguous layer span. Grafting weights from these spans into instruction-tuned hosts improved mathematical reasoning on average, with gains rising with region overlap, while retaining more previously solved problems than grafts elsewhere. Across training, the span formed gradually during the early rise in performance, and grafts from intermediate stages helped. Grafts shifted most hosts partly towards their donors' behaviour and sometimes improved on problems the donors failed; the same analysis located a transferable span in a medical model. Together, these findings identify a reasoning-related region whose weights improve reasoning with low interference in the host's existing mathematical competence.
19. 【2611.24598】Chain-of-Thought Hidden States Reveal Task-Solving Directions for Direct Answers
链接:https://arxiv.org/abs/2611.24598
作者:Houman Mehrafarin,Ioannis Konstas
类目:Computation and Language (cs.CL); Artificial Intelligence (cs.AI)
关键词:Large Language Models, Large Language, intermediate reasoning steps, eliciting intermediate reasoning, performance of Large
备注:
点击查看摘要
Abstract:Chain-of-Thought (CoT) prompting improves the performance of Large Language Models by eliciting intermediate reasoning steps. However, CoT is far from perfect: on some tasks, models perform worse than with direct-answer prompting, and on others they generate a few hundred tokens to reach the correct answer. We start by labelling each CoT hidden state as recovering if patching it into the direct prompt yields the correct answer, and failed otherwise. A linear probe shows that the model encodes these two classes differently. We then show that the representation space of the direct prompt holds a direction for each question that leads the model, in a short generation, to the correct answer more often than CoT does. Specifically, we rotate the direct prompt's last hidden state once, before generation, towards a direction built for each question from its own recovering and failed states. Across 25 tasks, these directions raise the direct prompt's mean accuracy from $46\%$ to $85\%$ on LLaMA 3.1 8B (CoT: $69\%$) and from $52\%$ to $86\%$ on Qwen 2.5 7B (CoT: $77\%$), with a tenth of CoT's tokens. CoT is thus not the shortest route to this accuracy, and these directions can be used in future work to train models to align their states with them.
20. 【2611.24588】Large Language Models or AI Agents for English to Arabic Translation? An assessment on WikiEn-Ar190
链接:https://arxiv.org/abs/2611.24588
作者:Fatima Zahra Iguenfer,Yousra Chtouki
类目:Computation and Language (cs.CL)
关键词:Agent performs, translating from English, research investigates, considered a low-resource, native Arabic speakers
备注:
点击查看摘要
Abstract:This research investigates whether a large language model or an autonomous AI Agent performs better at translating from English to Arabic, which is considered a low-resource language. The authors introduce WikiEn-Ar190, a new dataset of 190 carefully selected Wikipedia articles from 18 different areas, which were created and verified by two native Arabic speakers to ensure that they accurately reflect both meaning and style. Using this dataset, the study evaluates the translation quality of GPT-5-mini both as an independent neural translator and when used within a more complex process that includes self-assessment and improvement. As a standalone model, GPT-5-mini produced very fluent translations , achieving a COMET score of 0.8649 and receiving high human scores for adequacy, fluency, and style (averaging 4.53 out of 5). When used in a system that combined translation with self-assessment, it improved lexical accuracy by 28.76% (BLEU score of 22.83) while maintaining a similar level of semantic quality. These findings show that methods involving autonomous reasoning can improve translation consistency for language pairs with limited resources. Beyond the results, the approach offers a framework for evaluating and improving translations, potentially using them as assisting tools to expand the representation of low resource languages, such as Arabic, on the web.
21. 【2611.24576】Late Entry in Looped Language Models
链接:https://arxiv.org/abs/2611.24576
作者:Ziqi Jin,Jiaxi Li,Weiwen Xu,Aixin Sun
类目:Computation and Language (cs.CL)
关键词:Looped language models, compact parameter count, shared layer stack, Looped language, sequential decoding latency
备注: 25 pages, 9 figures
点击查看摘要
Abstract:Looped language models deepen reasoning at a compact parameter count by iterating a shared layer stack T times per token, turning recurrence depth directly into sequential decoding latency. Conventional early-exit decoding halts after initial passes without cache completion, truncating cached key-value states of past tokens to shallow depth. We study the opposite strategy, late entry, where new tokens skip initial passes and enter directly into the final K T passes over a deep context cache. Once lightweight fine-tuning resolves hidden-state norm mismatch at the entry boundary, deep cached context substitutes for skipped token-local passes, allowing late entry to outperform matched early exit across model families and pass budgets. To turn this asymmetry into wall-clock speedups without stalling to build deep caches, candidate blocks verify through only the final K passes on the critical path while previously committed tokens complete passes 1 - T alongside subsequent steps at zero extra serial calls. Projecting these finished T-pass wake states inside the existing output GEMM (Wake verification, governed by logit-margin tolerance tau in [0, inf]) unifies greedy-exact T-pass decoding (tau = 0, matching K = T block decoding within bf16 reduction order), margin-gated verification (tau 0), and non-rollback lossy late entry (tau = inf) inside a single static CUDA graph. On Ouro-2.6B (T = 4), K = 2 Wake reaches 5.2x greedy-exact speedup at tau = 0 (+26% over K = 4 block decoding on GSM8K) and 6.8x at tau = inf (+16.9 points over trained early exit at matched speed). Scaling test-time recurrence across T in {8, 16, 32, 64} on Huginn-3.5B cuts K = T/4 rollbacks 21.5x with depth, driving greedy-exact (tau = 0) speedups to 6.1x (T = 32) and 8.4x (T = 64, +153% over K = 64 block) and tau = inf speedups to 10.6 to 12.5x.
22. 【2611.24567】Objective-Specialized Residual Streams in Manifold-Constrained Hyper-Connections
链接:https://arxiv.org/abs/2611.24567
作者:Shantanu Agarwal,Ruchita V. Maitri,Zhiqi Huang,Vivek Datla,Alfy Samuel
类目:Computation and Language (cs.CL)
关键词:Manifold-constrained Hyper-Connections, extend a Transformer, Transformer residual stream, multiple parallel streams, Transformer residual
备注:
点击查看摘要
Abstract:Manifold-constrained Hyper-Connections (mHC) extend a Transformer's residual stream into multiple parallel streams with learned cross-stream connections. Prior work trains all streams toward a shared objective. We instead study whether assigning distinct learning objectives to different streams can improve task performance and induce objective-specific specialization. Using complementary lexical (sparse) and semantic (dense) retrieval objectives as a test case, we find that objective-specialized mHC consistently outperforms a non-specialized mHC model across the full sparse--dense fusion range, for both two- and four-stream models and at two model scales. We then examine how specialization changes the model internally using probes of stream representations, residual mixing, objective-specific gradient flow, and read/write gates. The specialized and non-specialized models behave similarly through most of the network but diverge in the final layers: the specialized model develops increasingly objective-specific gradient flow, distinct read/write routing patterns, and reduced cross-stream mixing, and streams assigned to different objectives become increasingly dissimilar while those trained toward the same objective remain highly similar. Fixing the residual mixing to the identity leaves the specialization benefit intact, indicating that the benefit stems from objective-specific read/write routing rather than learned cross-stream mixing. These results show that assigning distinct objectives to mHC streams can improve multi-objective retrieval while inducing structured, depth-dependent specialization in the residual computation.
23. 【2611.24540】Do Frontier Model Ingredients Add Up? A Replicated Factorial Study of Small Language Models
链接:https://arxiv.org/abs/2611.24540
作者:Biswa Sengupta
类目:Computation and Language (cs.CL); Artificial Intelligence (cs.AI)
关键词:language model requires, model requires choosing, requires choosing, choosing which published, published improvements
备注:
点击查看摘要
Abstract:Building a small language model requires choosing which published improvements to combine. We compare five changes in the same model, using the same data and training budget. We test selected combinations and repeat each one with a second random seed. Mixture-of-experts layers, Muon optimisation and a Gated DeltaNet hybrid give the largest gains. Extra normalisation also helps, while an attention gate with a matched parameter budget adds little on average. The gains mostly combine, but small interactions between changes appear again in the repeated runs. Repetition also reveals benefits that an initial statistical estimate treated as noise. These findings apply to our small-model training setting; the expert models also have greater total capacity. The practical lesson is to test promising components together and repeat the comparisons before deciding which ones to keep.
24. 【2611.24535】ProbMedTOD: A Bayesian Network Guided Task-Oriented Dialogue System for Patient History Taking
链接:https://arxiv.org/abs/2611.24535
作者:Vishal Vivek Saley,Bhavesh Gurnani,Dinesh Raghu,Mausam
类目:Computation and Language (cs.CL)
关键词:Patient history taking, Patient history, refines diagnostic hypotheses, iterative questioning, history taking
备注:
点击查看摘要
Abstract:Patient history taking is a strategic, multi-turn process that refines diagnostic hypotheses through iterative questioning, mirroring clinical reasoning where each inquiry updates beliefs over candidate diagnoses. Deploying AI in this setting is constrained by privacy, cost, and latency, making Small Language Models (SLMs) a more desirable backbone compared to larger LLMs. However, SLMs capture only the fast, intuitive "System 1" side of this process and lack explicit mechanisms for the deliberative "System 2" reasoning needed to maintain and update diagnostic uncertainty over time. Bayesian Networks (BayesNets) offer a natural complement, providing an interpretable framework for auditable probabilistic belief tracking. Yet, clinical BayesNets are difficult to construct due to their reliance on expert curated structure, dependencies, and parameters. We introduce ProbMedTOD, which automatically synthesizes a clinical BayesNet from a small number of clinical notes in three stages: ontology building, structure prediction, and parameter estimation. ProbMedTOD then integrates the resulting BayesNet with SLMs, enabling probabilistic reasoning in multi-turn dialogues. A BayesNet Agent maintains real-time posterior distributions over diagnoses, while a Policy Agent picks diagnostically informative questions. Experiments on DDXPlus and MIMIC-IV Notes show that ProbMedTOD outperforms LLM-only and retrieval-based baselines, with especially strong gains for smaller models.
Subjects:
Computation and Language (cs.CL)
Cite as:
arXiv:2611.24535 [cs.CL]
(or
arXiv:2611.24535v1 [cs.CL] for this version)
https://doi.org/10.48550/arXiv.2611.24535
Focus to learn more
arXiv-issued DOI via DataCite (pending registration)</p>
25. 【2611.24522】MECI: Multi-Element Collaborative Interaction for Multimodal Entity Linking
链接:https://arxiv.org/abs/2611.24522
作者:Jie Peng,Yongxue Shan,Yongfu Zha,Xiaodong Wang
类目:Computation and Language (cs.CL); Artificial Intelligence (cs.AI)
关键词:Multimodal Entity Linking, Entity Linking, Multimodal Entity, aims to disambiguate, knowledge base
备注: 11 pages, 8 figures
点击查看摘要
Abstract:Multimodal Entity Linking (MEL) aims to disambiguate mentions in multimodal contexts by grounding them to specific entities in a knowledge base. A pivotal challenge in MEL is capturing multi-level correspondences: the semantic consistency between mention-entity pairs and the complementary correlations across modalities. However, existing methods often suffer from element dominance due to their reliance on coupled interactions or coarse global aggregations. In response, we propose the Multi-Element Collaborative Interaction (MECI) framework. First, to capture multi-element mention-entity correspondences, we develop a Multi-view Experts Network that leverages a "divide-and-conquer" strategy for decoupled feature learning to mitigate element dominance, supported by a KL-guided routing mechanism that governs expert specialization and collaboration. Furthermore, to model cross-modal complementary correlations, we propose a Hierarchical Multimodal Interaction Module, where a dynamic modality-aware weighting network refines interactions across hierarchical semantic levels, thereby integrating multi-granular evidence to counteract element dominance. Finally, we incorporate a generative semantic refinement stage that utilizes large language models for zero-shot re-ranking. Extensive experiments on WikiDiverse, RichpediaMEL, and WikiMEL show that MECI consistently outperforms state-of-the-art baselines, improving Hits@1 by 1.95%, 7.30%, and 2.31%, respectively.
26. 【2611.24475】Nothing to Declare: Steering a single neuron increases concealment in AI teams
链接:https://arxiv.org/abs/2611.24475
作者:Edoardo Chidichimo,Felix P. S. Wallis
类目:Computation and Language (cs.CL); Artificial Intelligence (cs.AI)
关键词:Recent research suggests, Recent research, research suggests, suggests that individual, interpretable roles
备注: 3 pages, 1 figure
点击查看摘要
Abstract:Recent research suggests that individual neurons can have interpretable roles in AI behaviour. In Qwen3.5-4B, steering along one neuron's output direction increased decisions to conceal information more in teams than alone across five scenarios, even when teammates provided no messages. When all members participated and received steering, collective concealment increased in four scenarios, while inter-agent messaging helped temper concealment under steering. Moreover, removing the neuron's contribution had negligible effects. Collective settings can therefore change how an internal intervention affects AI behaviour.
27. 【2611.24383】What Majority Labels Hide: Annotator Disagreement and Model Errors in Multi-Dialect Arabic Sexism Detection
链接:https://arxiv.org/abs/2611.24383
作者:Alaa Bazaid,Arkaitz Zubiaga
类目:Computation and Language (cs.CL); Computers and Society (cs.CY); Social and Information Networks (cs.SI)
关键词:reduce annotator judgements, single majority label, vary across communities, sexism detection datasets, single majority
备注: 36 pages, 6 figures. Submitted to Online Social Networks and Media. Dataset: [this https URL](https://huggingface.co/datasets/alaabaz/MuDAS) . Content warning: contains examples of sexist and offensive language
点击查看摘要
Abstract:Judgements of sexism are subjective and may vary across communities, yet sexism detection datasets usually reduce annotator judgements to a single majority label, obscuring disagreement and how models behave on contested cases. We introduce MuDAS, a dataset of YouTube comments in Saudi, Egyptian and Moroccan Arabic, each labelled by three native speakers of its dialect, with the individual annotations preserved. We use it to evaluate Arabic encoders fine-tuned with hard-label, soft-label and perspectivist objectives, GPT-5.6 Terra, and four Arabic-centric large language models. Model errors are more frequent on comments with annotator disagreement. Despite relatively close Macro-F1 scores, GPT-5.6 Terra and the fine-tuned encoders exhibit contrasting error patterns: GPT-5.6 Terra, even with the annotators' definition of sexism, over-classifies gender-related content such as religious advice on modesty, whereas the encoders miss implicit and prescriptive sexism. The Arabic-centric models also over-predict sexism and remain below the encoders. Disagreement-aware learning preserves Macro-F1, but soft labels improve alignment with annotators on contested comments at the expense of unanimous ones, and neither paradigm reliably identifies contested comments. Lower performance on Egyptian Arabic, clearest for the large language models, is not fully explained by the examined measures of test-set composition and dialectness. Majority-label scores can therefore hide contrasting error patterns that matter for content moderation.
28. 【2611.24379】MoE-Context: Show the Model Only What It Needs
链接:https://arxiv.org/abs/2611.24379
作者:Vedant Navale
类目:Artificial Intelligence (cs.AI); Computation and Language (cs.CL); Machine Learning (cs.LG)
关键词:agents read, step, Abstract, task, blocks
备注: 10 pages, 10 figures
点击查看摘要
Abstract:AI agents read their whole history at every step. People do not work like that: we keep what the moment needs and hold the rest as a rough note we can recall. MoE-Context applies this to language models. The context is split into blocks. Before each step a fast yes or no pass decides which blocks the step needs; the rest stay as one-line notes that can be brought back, and nothing is deleted. In the version we build, one model makes this decision and then answers, over a single key-value cache where hidden blocks are masked rather than removed. On a laptop with Qwen3-1.7B and a synthetic lookup task, showing only the needed block raises accuracy from 93 to 119 of 120, and a masked block is as hidden as a deleted one. The model's own picking is the weak part. A supervised linear probe written into spare output rows lifts its accuracy from 80 to 110 of 120, but simple keyword search picks perfectly on this task, and the probe does not transfer to a second task. We report these results openly, along with two cheap usage patterns that block switching enables: undoing a failed attempt and running subagents without a new context.
29. 【2611.24301】LoopCache: Accelerating Long-Context Self-Speculative Decoding for Looped Transformers
链接:https://arxiv.org/abs/2611.24301
作者:Zizhuo Fu,Haochen Huang,Shengxuan Qiu,Linye Wei,Ziyu Tang,Runsheng Wang,Meng Li
类目:Computation and Language (cs.CL); Machine Learning (cs.LG)
关键词:large effective depth, achieve large effective, Looped Transformers, large effective, effective depth
备注:
点击查看摘要
Abstract:Looped Transformers achieve large effective depth with compact models by repeatedly applying a weight-shared block across R recurrent steps, each maintaining a loop-specific key-value (KV) cache. To accelerate sequential loop execution, recent wave-style self-speculative methods like WaveFront Decoding (WFD) and LoopSpec pipeline draft and target tokens across loops in a single step. However, they face two critical limitations. First, they draft only at fixed loops and ignore intermediate prediction changes, wasting computation on stale trajectories. Second, batching tokens from different loops loads up to Rx the KV cache per step, severely amplifying memory traffic in long-context scenarios. To address these challenges, we propose LoopCache, an efficient self-speculative decoding framework for long-context Looped Transformers. Based on the correlation between prediction switches and output confidence, LoopCache introduces confidence-guided dynamic draft updating to refine drafts in flight across all intermediate loops. Furthermore, LoopCache employs a cyclic loop KV cache strategy that rotates through loop-specific KV caches, restricting per-step KV traffic to a single loop while batching tokens at the nearest matching cycle for lossless verification. By also selectively retaining independent KV caches for a few loop-sensitive shallow layers, LoopCache improves acceptance rates with negligible overhead. Experiments show that LoopCache achieves average 10.25x and 14.56x speedup over WFD and LoopSpec.
30. 【2611.24285】LipSpell: A Language-Independent Phonetic Scoring Framework for Spelling Correction Combining Distinctive-Feature Edit Distance, Keyboard Geometry, and Corpus Frequency
链接:https://arxiv.org/abs/2611.24285
作者:Leonardo Araujo
类目:Computation and Language (cs.CL)
关键词:Automatic spelling correction, keyboard typographical slips, Automatic spelling, English-centric phonetic hashes, word pronunciation
备注:
点击查看摘要
Abstract:Automatic spelling correction must handle both keyboard typographical slips and cognitive or phonetic misspellings where writers approximate a word's pronunciation. Conventional spell checkers rely either on orthographic edit distance and keyboard proximity--which struggle when phonetic errors diverge by multiple characters--or on English-centric phonetic hashes such as Soundex and Metaphone. We present LipSpell, a unified, language-independent phonetic scoring engine paired with swappable language-specific G2P frontends, keyboard maps, and lexicons integrating: (1) a precomputed |P|x|P| phone-distance matrix from universal IPA distinctive features, (2) physical keyboard geometry, and (3) corpus unigram frequency priors, accelerated by a two-stage symmetric-delete and seven-class phonetic-code index. Across six English benchmarks (including 1,000 L2 errors from TOEFL-Spell), LipSpell outperforms Norvig and Hunspell on every dataset and beats Aspell on Norvig1 (78.52% vs. 75.56%), Holbrook (32.00% vs. 26.67%), and TOEFL-Spell (77.30% vs. 72.80%). Evaluated across six languages spanning five branches (Germanic, Romance, Slavic, Semitic, Turkic), three families (Indo-European, Afro-Asiatic, Turkic), and three scripts, LipSpell achieves strong fixed-weight transfer on Brazilian Portuguese (73.00% Top-1 on n=500 Wikipedia without NFD tolerance and 75.80% with NFD vs. 66.40% Norvig, 64.20% Hunspell, 64.00% Aspell) and Russian Cyrillic (85.67% vs. 78.50% Norvig). On 50/50 clean non-word splits with lexicon cleaning and weak-character indels, tuning yields strong gains over Norvig in Portuguese (86.97% vs. 76.75%), Turkish (81.67% vs. 79.05%), Hebrew Abjad (74.59% vs. 62.30%), and French (74.53% vs. 67.92%), while fixed weights remain optimal for English (80.37%) and Russian (85.67%).
31. 【2611.24236】From a finite subgroup of $SO(3)$:$C_2$ leads to a Holant dichotomy
链接:https://arxiv.org/abs/2611.24236
作者:Yuxuan Zhou
类目:Computational Complexity (cs.CC); Computation and Language (cs.CL); Discrete Mathematics (cs.DM)
关键词:Counting complexity, extensively studied, mathcal, Counting, complexity
备注:
点击查看摘要
Abstract:Counting complexity within the Holant framework has been extensively studied through complexity dichotomy theorems. While substantial progress has been achieved for various restricted settings, the general complex-valued even-arity case without auxiliary unary signatures remains largely open. This paper studies a fundamental case in this program where the binary signatures induce a group structure satisfying \(M(B(\mathcal{F}))\cong C_2\). It is observed that projective equivalence classes of binary signature matrices naturally embed into finite subgroups of \(SO(3)\), providing an algebraic viewpoint for structural classification. This paper further identifies two fundamentally different regimes for the \(C_2\) case, namely the normal and isotropic cases, and analyzes their complexity behavior separately. Combining gadget constructions, decomposition techniques, interpolation arguments, and a realnumberization approach that transforms the complex-valued setting into an equivalent real-structured framework, this paper establishes a complete dichotomy theorem: for every even-arity signature set \(\mathcal{F}\) satisfying \(M(\B(\mathcal{F}))\cong C_2\), the problem \(\mathrm{Holant}(\mathcal{F})\) is either polynomial-time computable or \#P-hard. A Pauli-basis representation for signatures is further introduced as an algebraically natural language for organizing realizable signatures and revealing structural relations that are less transparent in the standard basis.
Subjects:
Computational Complexity (cs.CC); Computation and Language (cs.CL); Discrete Mathematics (cs.DM)
Cite as:
arXiv:2611.24236 [cs.CC]
(or
arXiv:2611.24236v1 [cs.CC] for this version)
https://doi.org/10.48550/arXiv.2611.24236
Focus to learn more
arXiv-issued DOI via DataCite (pending registration)</p>
32. 【2611.24168】CounselTraj: Synthesizing Process-Aware Psychological Counseling Dialogues via Dialogue-Act Trajectory Planning
链接:https://arxiv.org/abs/2611.24168
作者:Zhiyang Qi,Michimasa Inaba
类目:Computation and Language (cs.CL); Artificial Intelligence (cs.AI)
关键词:systems requires large-scale, high-quality dialogue data, LLM-based systems, systems requires, requires large-scale
备注: Accepted to EACL 2027 Main Conference
点击查看摘要
Abstract:The growing demand for psychological counseling has motivated the development of LLM-based systems that support human counselors, but training such systems requires large-scale, high-quality dialogue data. Authentic counseling transcripts are difficult to collect and share because of privacy concerns, making synthetic dialogues an important alternative. However, existing synthetic datasets, despite covering diverse scenarios and client profiles, often consist of brief exchanges and fail to capture session-level counseling processes. We propose CounselTraj, a dialogue-act trajectory planning framework for expanding brief synthetic dialogues into process-aware, session-level psychological counseling dialogues. CounselTraj learns dialogue-act patterns from role-play counseling dialogues produced by human counselors, plans coordinated trajectories for clients and counselors, and uses them as flexible guidance for long-form dialogue generation. Experiments across three Chinese synthetic counseling datasets show that CounselTraj produces more complete and coherent sessions with higher-quality counselor interventions than both direct expansion and expansion with randomly sampled trajectories. Based on CounselTraj, we construct CounselTrajD, a dataset containing over 10K process-guided Chinese psychological counseling dialogues, and use it to train CounselTrajLM for counseling response generation. Evaluations at both the response and session levels show that CounselTrajLM outperforms publicly available Chinese counseling models in response quality, therapeutic understanding and intervention, and counseling-process coherence and progress. Our code and model are publicly available at this https URL.
33. 【2611.24156】Once Upon a Bias: Gender Stereotyping in LLM-Generated Children's Stories
链接:https://arxiv.org/abs/2611.24156
作者:Sayumi Muthukumarana,Roshan Godaliyadda,Parakrama Ekanayake,Roshan Ragel
类目:Computation and Language (cs.CL); Artificial Intelligence (cs.AI)
关键词:Large language models, Large language, language models, models are increasingly, Large
备注:
点击查看摘要
Abstract:Large language models are increasingly used to create stories for children. Since bedtime stories can shape children's perceptions of social roles, this raises concerns about whether LLMs adapt narrative portrayals according to the intended recipient's gender. We conducted a large-scale study to examine gender stereotyping in LLM-generated bedtime stories. Using matched daughter/son prompts and a five-generation design, we analyze 62,873 retained stories from 16 LLMs across 35 country settings and 12 narrator roles. We introduce a multi-level evaluation framework combining context-adjusted lexical rate contrasts, relative stereotype alignment, narrator-recipient interaction analysis, human evaluation, and representation-level interventions to examine both the consistency and interpretability of these differences. Our results reveal systematic differences in narrative framing: son-directed stories emphasize higher agency and lower communion compared to daughter-directed stories. While narrator gender expresses limited moderation on these patterns, high sociability exhibits a more pronounced interaction. Human evaluations demonstrate consistent agreement, although correspondence with lexical measures varies across narrative dimensions. Additionally, hidden-state analyses reveal decodable recipient-gender information. These findings demonstrate persistent yet context-dependent representational disparities in LLM-generated children's stories and motivate narrative-specific bias evaluation beyond short-form benchmarks.
34. 【2611.24086】Constructing Dynamic Conversations with Neural Back-channel Generation and Voice-line Scheduling
链接:https://arxiv.org/abs/2611.24086
作者:Yifang Pan,Chris Landreth,Karan Singh
类目:ound (cs.SD); Computation and Language (cs.CL)
关键词:dyadic conversational audio, problem of combining, channels of dyadic, voice-line channels, unified conversation
备注: 12 pages, 8 figures, project page [this https URL](https://evansamaa.github.io/NeuralSchedularRelease_page/)
点击查看摘要
Abstract:We present a novel approach to the problem of combining two voice-line channels of dyadic conversational audio into a unified conversation with controllable back-channeling and turn-taking dynamics. In voice performances for the game industry, the two tracks of voice-lines are typically recorded or synthesized in isolation. Our method takes two such unsynchronized tracks and uses a neural scheduler to sequence them into a single dynamic conversation, modifying both tracks by inserting or removing pauses and introducing voice overlap as appropriate. We further propose a back-channel predictor that is able to predict opportunity for back-channels for a given conversational turn's audio and content, which the scheduler can then incorporate. We evaluate our scheduler and predictor independently and in combination by: various quantitative comparison; a qualitative and perceptual comparison to prior art and naive baselines; a compelling set of results showing animator control over the diversity, extent and nature of back-channeling and turn dynamics.
35. 【2611.24076】Meaning Reaches Object Choice but Not Completion: Where Is a VLA Vulnerable to Landmark Synonyms?
链接:https://arxiv.org/abs/2611.24076
作者:Ryoichi Kurihara,Kota Sunamoto
类目:Robotics (cs.RO); Computation and Language (cs.CL); Machine Learning (cs.LG)
关键词:VLA policies, policies are fragile, connector, landmark, VLA
备注: Accepted to the CoRL 2026 Workshop on The Action Gap (TAG 2026). OpenReview: [this https URL](https://openreview.net/forum?id=b7wZqqQzON)
点击查看摘要
Abstract:VLA policies are fragile to rewording. Success rates alone cannot say whether a policy fails to choose the object or to complete the task, nor where in the policy to intervene. We study the released GR00T N1.7 on two LIBERO-Spatial tasks differing only in the landmark word telling which of two bowls to take. Synonyms absent from the training instructions send the policy to the bowl they name, unlike made-up words, yet complete 67.6% of one task's closed-loop episodes, against the original word's 100%. Before the connector between the vision-language model and the action generator, replacing only the landmark tokens' vectors with the other landmark's (pasting) sends nearly every episode to the other bowl. After the connector, pasting at the landmark, the remaining instruction, or the image tokens alone rarely moves the choice, but any two do. Hiding the landmark tokens from the action generator after the connector still lowers completion for the synonyms that fall short. Finetuning on diversified landmark words repairs that one task's completion for synonyms absent from finetuning; finetuning on the original instructions does not. Training only the connector on the diversified words suffices, as does copying the connector of a policy fully finetuned on them into one finetuned on the original instructions. An open-loop score on starting frames misses that weights outside the connector also carry part of the repair. For now, on this evidence, robustness to rewording should be judged in closed loop, with actions executed, intervening on both sides of the connector, scoring choice and completion separately.
36. 【2611.24071】Measuring and Charging for Retained Context in Long-Context LLM Services
链接:https://arxiv.org/abs/2611.24071
作者:Yi Lu,Qianshi Lin,Yumeng Yang,Hongxia Wang
类目:Performance (cs.PF); Computation and Language (cs.CL)
关键词:Long-context large language, large language model, Long-context large, language model, repeated context
备注: 33 pages, 4 figures, 9 tables, including Appendices A-F; code and recorded data included as ancillary material
点击查看摘要
Abstract:Long-context large language model (LLM) services charge for repeated context, yet billing weights need not reflect its contribution to answer quality. We present a three-layer framework separating content relevance, observable proxies and auditable positional weights. Context-ablation experiments show that relevance dominates position and that distance-decay families do not outperform a relevance-only baseline. A measured serving stack shows that cache hits reduce repeated-prefix compute cost by over two orders of magnitude at 65,536 tokens, while cold resends constrain sustainable discounts. Cross-provider measurements separate cache value from tokenisation and list-price differences. Calibrated simulations reduce the 99th percentile of session charges by 31-42%, although revenue-matched nonlinear tariffs achieve similar reductions and median charges increase. The framework provides a bounded per-token accounting rule and measured deployment constraints, rather than a validated utility estimator.
37. 【2611.24058】MIRAGE: A Framework for Multi-hop Reasoning under Dynamic Knowledge Injection
链接:https://arxiv.org/abs/2611.24058
作者:Heyang Xu,HanWen Hao,Haifeng Wang,Haoyi Zhou,Jianxin Li
类目:Computation and Language (cs.CL)
关键词:Multi-hop Question Answering, Question Answering, synthesize multiple pieces, requires large language, coherent reasoning path
备注: Submitted to COLING 2027. 15 pages, 5 figures
点击查看摘要
Abstract:Multi-hop Question Answering (MQA) requires large language models (LLMs) to retrieve and synthesize multiple pieces of evidence into a coherent reasoning path, which is essential for complex tasks. However, prior work on knowledge updating for MQA has largely focused on simple fact edits, overlooking a more challenging and realistic setting---\emph{Knowledge Injection}---in which the model must simultaneously accommodate both conflicting updates and entirely novel information. Under this setting, a hop-level answer is often wrong simply because the model trusts the wrong knowledge source: it clings to outdated parametric memory when the injected knowledge conflicts with it (K2), and it cannot answer when the required fact is entirely new (K3). To investigate this, we introduce NEXUS, the first benchmark tailored for multi-hop reasoning under Knowledge Injection, which explicitly covers both conflicting (K2) and incremental (K3) knowledge. We further propose MIRAGE, an efficient, training-free framework whose core component, the Correlation-aware Dual-path Reasoner (CDR), resolves such per-hop source-trust errors via P(True)-based confidence routing, complemented by a lightweight template-guided corrector for residual structural deviations. Extensive experiments across four LLM families and two benchmarks demonstrate that MIRAGE consistently improves both final-answer accuracy and hop-level reasoning accuracy, establishing a new state of the art in this challenging setting.
Comments:
Submitted to COLING 2027. 15 pages, 5 figures
Subjects:
Computation and Language (cs.CL)
Cite as:
arXiv:2611.24058 [cs.CL]
(or
arXiv:2611.24058v1 [cs.CL] for this version)
https://doi.org/10.48550/arXiv.2611.24058
Focus to learn more
arXiv-issued DOI via DataCite (pending registration)</p>
38. 【2611.24055】Grading Executable Legal Rules Against Independent Records: Transitional Provisions and Korean SME Status
链接:https://arxiv.org/abs/2611.24055
作者:Jinhyung Bae
类目:Computation and Language (cs.CL); Artificial Intelligence (cs.AI)
关键词:large language models, medium enterprise, increasingly turned, small or medium, large language
备注: 51 pages, 2 figures (plus 1 in the appendix). Submitted to Artificial Intelligence and Law. Code and data: [this https URL](https://github.com/nepersoned/lextemporis)
点击查看摘要
Abstract:Laws are increasingly turned into programs that decide cases, such as whether a firm was a small or medium enterprise (SME) on a given date, and large language models (LLMs) now write them. The programs are usually graded against answers written by someone who read the same law, so a program can score well by sharing the grader's reading rather than by getting the law right. LexTemporis grades them instead against records of what actually happened: 10,901 firm-years of SME status disclosed by Korean listed companies, with every version of the governing rules. An appendix adds comparison cases from US income tax and German hospital minimum volumes. The date changes outcomes mainly through provisions outside the main statutory text: one transitional provision raises agreement with the disclosures from 83-86% to 89-90%. The disclosures also test the author's reading of a contested provision, favouring it in 130 of 189 firm-years where readings diverge. Four LLMs wrote the rules, given different selections of the law. A pre-registered comparison finds no condition better across all four models. In exploratory analyses, supplying the right provisions looks better graded against the author's reading than against the records: the advantage shrinks for three models (for one, to between a quarter and three-fifths of its size), and per firm-year it is several times larger where the author's reading is wrong, partly by construction, since there reproducing that reading more faithfully must lose. Grading legal programs against the grader's own reading can overstate how well they work.
39. 【2611.24049】A Hybrid RAG Framework for Deceptive Review Detection: Integrating RoBERTa-LoRA, FAISS-Based Retrieval, and LLM Reasoning
链接:https://arxiv.org/abs/2611.24049
作者:Andrei Kostin,José Ángel Diaz-Garcia,Karel Gutiérrez-Batista
类目:Computation and Language (cs.CL)
关键词:deceptive online opinions, online opinions poses, misinformation and fraud, growth of deceptive, deceptive online
备注:
点击查看摘要
Abstract:The growth of deceptive online opinions poses a critical risk of misinformation and fraud. This paper proposes a Retrieval-Augmented Generation (RAG) system to automatically distinguish spam reviews. The architecture integrates RoBERTa with Low-Rank Adaptation (LoRA) classification, FAISS semantic vector retrieval, and a Large Language Model (LLM) for final reasoning. The system was evaluated across balanced and severely imbalanced real-world scenarios. Results demonstrate that five epochs of fine-tuning enable robust domain transfer, achieving an F1-score of 0.90 in cross-domain evaluation, outperforming baselines (OpenAI Detector,F1 = 0.82) and demonstrating competitive performance against specialized in-domain SOTA models (fakeRoBERTa,F1 = 0.97). The proposed architecture provides evidence-backed explainability and statistically significant precision gains while maintaining comparable overall detection performance.
40. 【2611.24043】Compression Effects on Low-Resource Multilingual Translation
链接:https://arxiv.org/abs/2611.24043
作者:Abbas Sani
类目:Computation and Language (cs.CL)
关键词:Swahili Hausa Yoruba, small general-purpose LLM, pilot study characterising, multilingual instruction-tuned LLM, multilingual machine translation
备注: `Pilot study. 3 model families x 3 precisions x 6 directions on FLORES-200. Code and raw results: [this https URL](https://github.com/avvas200/efficient-multilingual-nmt-pilot) `
点击查看摘要
Abstract:We present a pilot study characterising the effect of post-training quantisation on multilingual machine translation quality and inference cost for three African languages - Hausa, Swahili, and Yoruba - across three model families: a small general-purpose LLM (Qwen2.5-1.5B), a larger multilingual instruction-tuned LLM (Aya-23-8B), and a dedicated multilingual MT model (NLLB-200-distilled-1.3B). Evaluating on the FLORES-200 devtest set (1,012 sentences per direction) under FP16, bitsandbytes INT8, and NF4 INT4 precisions on consumer-grade NVIDIA T4 GPUs, we find: (i) the specialised MT model outperforms both LLMs by 32.7-43.3 chrF on the Hausa and Swahili directions, and by 12.9-24.7 chrF on Yoruba, despite being roughly a sixth the size of the 8B instruction-tuned LLM; (ii) post-training quantisation preserves aggregate translation quality within approximately 1 chrF across all model families and language directions, with a single worst case of 2.19 chrF; (iii) on consumer GPUs, INT8 and INT4 quantisation provide substantial memory savings (49-61% at INT4) but do not reduce inference latency relative to FP16 - in fact, latency increases by 1.3x to 5.3x; (iv) a clear performance gradient tracking training-resource availability is visible across languages (Swahili Hausa Yoruba). These findings motivate a research agenda focused on hardware-aware compression and error-type analysis beyond aggregate metrics for low-resource deployment.
41. 【2611.24001】Rethinking Fixed-Horizon Latent Reasoning
链接:https://arxiv.org/abs/2611.24001
作者:Tianxiang Chen,Jingyuan Zhou,Weiming Yang,Kaidi Yang
类目:Machine Learning (cs.LG); Artificial Intelligence (cs.AI); Computation and Language (cs.CL)
关键词:reasoners typically run, latent, typically run, reasoning, latent CoT
备注: 21 pages, 4 figures
点击查看摘要
Abstract:Existing latent reasoners typically run for a preset latent horizon during inference. Intuitively, a longer horizon should provide more opportunity for reasoning. However, in this work, we find that extending latent Chain-of-Thought (CoT) to the full preset horizon may in fact degrade reasoning performance. We observe that latent CoT evolves through distinct functional stages, with earlier computation forming frontier reasoning states, while later computation mainly reorganizes existing candidates and can weaken gold candidate support. These findings motivate a rethink of fixed-horizon latent inference. To address this, we characterize the reasoning benefits of latent CoT by realized and remaining constructive progress. Empirically, we find that higher realized constructive progress corresponds to higher current reasoning accuracy, while more remaining progress corresponds to larger future gains from continued latent CoT. To this end, we introduce a lightweight method that estimates the remaining reasoning progress and adaptively adjusts the latent horizon during inference. Across representative latent CoT reasoners and multiple arithmetic benchmarks, our method improves reasoning accuracy by up to 7.2 percentage points while reducing latent computation by up to 29.0%, without retraining the LLM.
42. 【2611.24000】A Mechanistic Analysis of In-Context Learning and Out-of-Context Hallucination
链接:https://arxiv.org/abs/2611.24000
作者:Tianxiang Chen,Jingyuan Zhou,Guoyao Yang,Kaidi Yang
类目:Computation and Language (cs.CL); Artificial Intelligence (cs.AI); Machine Learning (cs.LG)
关键词:Large language models, follow specific rules, Large language, OOC hallucination, ICL and OOC
备注: 25 pages, 6 figures
点击查看摘要
Abstract:Large language models can infer and follow specific rules through in-context learning (ICL), yet may fail to remain faithful to provided evidence, giving rise to out-of-context (OOC) hallucination. Existing work has largely investigated the underlying mechanisms of ICL and OOC hallucination along separate lines. However, this separation obscures the mechanistic basis for these seemingly opposing outcomes. In this work, we study this contrast through the unified view of functional roles in dynamic context. The key observation is that role information can be internally represented without reliably constraining output. We derive a first-order decomposition to characterize the role effect, showing that its substantial influence depends on two necessary conditions, namely sufficient contribution magnitude and coherent signed aggregation. Causal analysis further shows that role span contributions affect key token choices involving response structure, content selection, and factual values. We further show that role-consistent choices depend on sufficient role span support at these key decisions, while failing choices can be reversed when this support overcomes the deficit. These findings support a common mechanistic view of ICL and OOC hallucination: each functional role defines the pattern of output decisions over which role span contributions must remain effective, and differences in this organization account for the seemingly opposing outcomes.
43. 【2611.23969】ReflectRec: Cycle-Consistency Preference Verification for LLM-based Recommendation
链接:https://arxiv.org/abs/2611.23969
作者:Dae Kyoo Kim,Heejung Cho,Sungmoo Hong,Eunil Park
类目:Computation and Language (cs.CL); Information Retrieval (cs.IR)
关键词:CoT-style LLM recommenders, LLM recommenders generate, natural-language preference note, prefix to ranking, behaviorally similar users
备注: 15 pages. Published in the Proceedings of EMNLP 2026 (Main Conference)
点击查看摘要
Abstract:CoT-style LLM recommenders generate a natural-language preference note as a chain-of-thought prefix to ranking, but the note is trusted without verification, and the ranker never consults behaviorally similar users. We propose ReflectRec, organized around a preference-to-history round-trip verifier whose recall score serves as an operational signal of preference-to-history recoverability, enabling conditional regeneration for low-recoverability preferences and selection among regeneration attempts. Around this verifier, we wire a semantic Stage-1 prompt, conditional regeneration, and Forward-Look retrieval that injects future watches of behaviorally similar users as a collaborative hint. On MovieLens-100K and MovieLens-1M, the full pipeline outperforms CoT4Rec-ICL by +16.25 / +4.46 pp Hit@10, with positive gains across all ValScore buckets and three LLM backbones (Qwen3-30B, Qwen2.5-72B, and Llama-3.3-70B). We also observe a similar verifier-guided regeneration pattern on Amazon Electronics.
44. 【2611.23900】Coverage Governs Accuracy: When to Expand the Lexicon and When to Fall Back in Low-Resource Manchu OCR
链接:https://arxiv.org/abs/2611.23900
作者:Ruixin Yu,Zifan Wang,Mingxin Ding,Xiaoxin Sun,Dongjie Wang,Suhua Wang,Minghao Yin
类目:Computation and Language (cs.CL); Computer Vision and Pattern Recognition (cs.CV)
关键词:produces near-miss spelling, near-miss spelling errors, Character-level OCR, produces near-miss, near-miss spelling
备注: 15 pages, 11 figures, 6 tables
点击查看摘要
Abstract:Character-level OCR for low-resource Manchu often produces near-miss spelling errors. Lexicon-constrained decoding can correct such errors with lexical knowledge, but a fixed dictionary cannot cover the many unseen word forms produced by agglutinative word formation. We study whether morphological expansion alleviates this limitation, and how the expansion size and confidence fallback affect recognition. We segment the training transcriptions with a dictionary-based stem list, extract roots and suffix chains from them, generate about 510k candidates, select the expansion size through frequency ranking on the validation set, and decompose accuracy under hard lexicon constraints into the product of coverage and in-lexicon precision. Across three CRNN training seeds, adding 250k candidates to a base dictionary of about 29k words raises the type accuracy on unseen combinations of seen roots from 63.6% to 86.3%, and overall token accuracy from 82.4% to 84.2%. Adding all candidates lowers overall token accuracy to 81.5%, showing that the coverage gain comes with a loss of in-lexicon precision. A further comparison shows that the base dictionary with confidence fallback performs best overall among the CRNN systems tested, reaching 76.1% type and 85.5% token accuracy and outperforming the expanded lexicon both with and without fallback. A Transformer recognizer shows similar trends. These results indicate that morphological expansion relieves the constraint of a fixed dictionary, but a larger lexicon does not guarantee better recognition; in the present low-resource Manchu OCR setting, a selective combination of reliable lexical constraints and open recognition is more effective.
45. 【2611.23835】NarrowSpec: Draft Hidden Width Need Not Match the Target in Speculative Decoding
链接:https://arxiv.org/abs/2611.23835
作者:Alagappan Valliappan,Ankit More,Dheevatsa Mudigere,Jatin Mitra
类目:Computation and Language (cs.CL); Machine Learning (cs.LG); Performance (cs.PF)
关键词:model proposing tokens, large language models, target model verifies, small draft model, draft model proposing
备注: 29 pages, 6 figures, 33 tables
点击查看摘要
Abstract:Speculative decoding is widely used to speed up generation in large language models (LLMs), with a small draft model proposing tokens that one forward pass of the target model verifies. Draft models that read the target's hidden states, such as DFlash and EAGLE-3, typically inherit the target's hidden width. We show that the draft's whole residual stream can run at half or a quarter of that width while largely preserving acceptance. For DFlash, which borrows the target's vocabulary projection, the trained output adapter folds into it, giving one narrower projection with logits unchanged up to floating-point rounding. We test two drafter families, DFlash and EAGLE-3, on two targets, Qwen3-8B (NVFP4) and GPT-OSS-20B (MXFP4), in SGLang on a B200 and a DGX Spark (GB10). At batch size 1 and temperature 0, narrowing the draft to half or a quarter width speeds up end-to-end decoding by 1.06-1.12x on B200 and 1.13-1.44x on GB10. The speedup holds at larger batch sizes. From batch 32 to the largest batch each GPU fits, narrowing reaches up to 1.21x on both GPUs, and no narrow draft is slower than full width.
46. 【2611.23832】A System 1 Decision Model for Medical Multiple-Choice Questions Built on Frozen Language Models
链接:https://arxiv.org/abs/2611.23832
作者:Pavan Kumar Mantha
类目:Artificial Intelligence (cs.AI); Computation and Language (cs.CL)
关键词:single pass, multiple-choice question, question, frozen language model, language model turns
备注: 12 pages with 7 tabular explanations
点击查看摘要
Abstract:A System 1 model answers a multiple-choice question in a single pass. A frozen language model turns the question and each option into vectors, two small, trained heads place them in a shared space, and the closest option wins. The design has been shown on agent decisions, but we are not aware of a study of how far it goes on exam questions that need medical knowledge, where the right option is often no more similar to the question than the wrong ones. We rebuilt the recipe from scratch for MedMCQA and trained only two projection heads and one temperature on top of frozen Qwen3.5-0.8B, Granite-4.2-3B and Qwen3-8B, using a contrastive loss with in-batch and hard negatives and a mask for repeated answers. With plain last-token vectors, both small encoders answer about one question in three correctly (33.1% and 33.0% on 3,842 held-out questions, where chance is 25.0%). Writing each text inside a one-word-summary prompt and lowering the learning rate of the heads raised this to 41.7% and 42.5% for the two small encoders and to 46.0% +-0.2 (three seeds) for Qwen3-8B. Shuffling the questions against the options returns accuracy to chance, so the heads do use the question. Flan-PaLM and Med-PaLM 2 report 57.6% and 72.3% on MedMCQA under a different protocol, so ours is a cheap, fast baseline and not a competitive system. We also say what our evidence cannot show: validation gains were about 1.7 to 2 times the test gains, and we could not separate the effect of reading several encoder layers from the other changes.
47. 【2611.23807】Sightline: Long-Window Speculative Decoding for Fast SVG Generation
链接:https://arxiv.org/abs/2611.23807
作者:Yanchao Li,Ben Gao,Jiaqing Xie,Yusu Zhang,Beilun Wang,Yanbo Wang,Xiaohua Xu,Nanqing Dong,Tianfan Fu,Zhongyu Huang
类目:Computer Vision and Pattern Recognition (cs.CV); Computation and Language (cs.CL); Machine Learning (cs.LG)
关键词:Scalable Vector Graphics, Scalable Vector, drawing is slow, SVG, forward pass
备注: 45 pages, 23 figures, 34 tables. Code: [this https URL](https://github.com/LiZaiyuan0619/Sightline)
点击查看摘要
Abstract:Scalable Vector Graphics (SVG) generation models write a graphic as code with one forward pass per token, so drawing is slow. Speculative decoding speeds them up losslessly, and block drafters guess many tokens per pass. On SVG, many of their rounds accept the whole draft and end only at the verification window. We find that their misses are rare and clumped, and that they stay accurate far deeper into a block than on natural language. A run-length identity and three propositions show that a longer window pays once the candidates are wide, up to the drafter's horizon. We propose Sightline, which pairs a window as long as the drafter's horizon with a draft tree wide enough to fill it. Experiments cover text-to-SVG and image-to-SVG with three models on six benchmarks from MMSVG, MMSVG-2M and SArena. In every setting, Sightline is the fastest of the compared lossless decoders. On SArena, it stays the fastest when every decoder shares one loop control. It also leads in almost every setting of six models from four other families, which write SVG code, compressed SVG tokens or Lottie JSON. Analyses trace the lead to rounds that shorter windows cut off, both in loop-free content and in loops. Speculative decoding also saves time for choosing samples by rendering, which beats choosing by likelihood. These results suggest that speculative decoding on vector graphics should verify as far as the drafter sees. Code and data are available at this https URL.
48. 【2611.23751】hinking Fast, Anywhere: One System-1 for the Decision Points of Any Agent Workflow
链接:https://arxiv.org/abs/2611.23751
作者:Rui Li,Liyang He,Zheng Zhang,Zhenya Huang,Qi Liu
类目:Artificial Intelligence (cs.AI); Computation and Language (cs.CL); Machine Learning (cs.LG)
关键词:Autonomous LLM agents, Autonomous LLM, LLM agents incur, intermediate actions choose, agents incur substantial
备注: 16 pages, 8 figures, 7 tables. Project page: [this https URL](https://middle-chunjie.github.io/AnyS1/)
点击查看摘要
Abstract:Autonomous LLM agents incur substantial cost and latency by invoking a large model (System-2) at every step, even when many intermediate actions choose among a small set of known candidates. Existing dual-system agents offload such decisions to a lightweight System-1 model, but rely on workflow-specific hooks authored per environment. We present AnyS1, a framework that turns human-authored procedural skills, whose instructions naturally specify where choices arise and which options apply, into a shared, domain-general source of decision points. Offline, AnyS1 mines grounded enumerable decisions across over 1,000 public skills, gathers execution-grounded trajectory states inside isolated containers to construct Any-DP, and trains a single calibrated 4B pointer System-1 with label smoothing. At run time, compiled workflows resolve routine decisions in one forward pass and fall back to System-2 when confidence falls below a calibrated threshold. Across three unseen benchmarks and distinct System-2 model families, AnyS1 preserves task reliability while cutting episode cost by 75-91% and model-call time by 63-92% relative to an uncompiled System-2 agent, and maintains performance under zero-shot threshold transfer.
49. 【2611.23750】Beyond Replacement: Levenshtein Editing for Diffusion Language Models
链接:https://arxiv.org/abs/2611.23750
作者:Shuaidi Wang,Zhuocheng Gong,Xuejie Liu,Jianguo Li,Yu Zhang
类目:Computation and Language (cs.CL); Artificial Intelligence (cs.AI)
关键词:Diffusion language models, enabling any-order infilling, Diffusion language, offer a flexible, any-order infilling
备注: 21 pages, 9 figures
点击查看摘要
Abstract:Diffusion language models (DLMs) offer a flexible alternative to autoregressive models, enabling any-order infilling. While most existing decoding methods treat decoded tokens as fixed, some recent models allow previously decoded tokens to be revised only by replacing them with alternative tokens, a paradigm we refer to as replacement-only editing (ROE). Although ROE can correct individual tokens, it is inefficient for sequence shifts caused by missing or extra tokens and struggles when errors occur across many positions. To address these limitations, we propose LED (Levenshtein Editing for Diffusion Language Models), a post-training framework that extends ROE with insertion and deletion, enabling Levenshtein-style refinement of generated sequences. It uses two training designs. First, it reconstructs edit supervision on the fly during supervised fine-tuning using longest-common-subsequence alignment, enabling insertion and deletion learning without manually annotated labels. Second, it adopts multi-stage training, where the state from one stage becomes the input to the next, exposing the model to its earlier errors and enabling it to correct them. When applied to the 16B-parameter LLaDA2.1-Mini Base, LED raises the average score across twelve benchmarks from 54.86 to 57.10 under confidence-guided decoding while increasing tokens per forward from 6.14 to 7.05, whereas under confidence-free decoding it raises the average score from 27.52 to 47.19. We further show that LED achieves higher pass@k at large k than thresholded decoding without editing, demonstrating improved generation diversity and revealing the diversity potential of DLMs without confidence guidance. LED also improves robustness to large block sizes, raising the average score from 36.97 to 51.34 at block size 128. Together, these results suggest that LED supports more robust self-correction in DLMs.
50. 【2611.23716】Learning through Discernment: when a small language model learns from user feedback on the job, and when it decays
链接:https://arxiv.org/abs/2611.23716
作者:Sirisilp Kongsilp
类目:Machine Learning (cs.LG); Artificial Intelligence (cs.AI); Computation and Language (cs.CL)
关键词:Small language models, daily work, Small language, language models, model
备注: 34 pages, 11 figures. Code and data: [this https URL](https://github.com/kratai-desci/research/tree/discernment/discernment)
点击查看摘要
Abstract:Small language models on devices could keep improving by learning, overnight, from their own daily work. Without a true outcome, however, something must decide which attempts are worth learning from, and both the model's own judgment and the user's reaction can be wrong. We study a small model (Qwen3 0.6B and 1.7B) with a LoRA adapter, fine-tuned each night on the attempts a judge accepts, over 127 lives of up to a year on WebShop and $\tau^2$-bench. We compare the true outcome, the user's stated verdict and the model's own judgment with a separate, frozen "feedback sense" that reads the user's reaction, built in three versions. Judging itself, the model improved and then decayed, over a year and whatever its adapter. A sense that took polite give-ups for success collapsed 3 of 5 lives; one built for discernment, which first asks whether the user gave a clear verdict, kept 4 of 5 near the oracle for two weeks, then faded on clean but scarce lessons, to the level of self-judging by day 60. When the user cannot see the outcome ($\tau^2$-bench), the sense read reactions correctly, yet 80% of its lessons were failures the customer had thanked. Across 97 lives, the share of failed attempts among the lessons predicted the outcome (Spearman $\rho$ = -0.81). In our experiments, learning from users held up only where the judge kept failures out and the user could see the outcome; lopsided lessons may also narrow the model.
51. 【2611.23704】Localizing Text Anonymization for Trustworthy AI: Extending RAT-Bench to Malaysian Microdata and PII
链接:https://arxiv.org/abs/2611.23704
作者:David Hong Liang Chew,Zexi Yao,Nataša Krčo,Matthieu Meeus,Waqas Khalid Obeidy,Yves-Alexandre de Montjoye
类目:Computation and Language (cs.CL)
关键词:adopt foundation models, organizations adopt foundation, local model development, enter LLM workflows, Bahasa Malaysia
备注: Accepted at the Trustworthy AI for Good (AI4GOOD) Workshop at ICML 2026
点击查看摘要
Abstract:As governments and organizations adopt foundation models for public services and workplace productivity, sensitive citizen, employee, and administrative text may enter LLM workflows through inference, retrieval, fine-tuning, or local model development. We examine whether text anonymizers transfer to new deployment settings by extending RAT-Bench to Malaysia using local microdata, Malaysian PII formats, and culturally grounded transcripts in Malaysian English and Bahasa Malaysia. We evaluate NER- and LLM-based anonymizers using an LLM attacker that infers attributes from anonymized text, measuring both re-identification success and text utility. Across Malaysian English and Bahasa Malaysia, the two LLM anonymizers provide the strongest explicit privacy-utility trade-offs, reducing Easy/Hard re-identification risk to 23-29% while preserving BLEU scores of 0.77-0.94. Non-LLM tools show sharper failure modes, either preserving utility while leaving high residual risk or reducing risk through severe over-redaction, especially in Bahasa Malaysia. These findings suggest that anonymizers should be evaluated against deployment-relevant languages, identifier formats, and cultural contexts before being relied on in public-sector, workplace, or local AI pipelines.
52. 【2611.23673】Communicative On-Policy Distillation: Distilling Black-Box Teacher Capabilities through Communication for Long-Horizon Tasks
链接:https://arxiv.org/abs/2611.23673
作者:Lirui Luo,Kelong Mao,Xinwei Yang,Rongqing Li,Luyu Chen,Kieran Wong,Yudong Guo,Xinrui Wang,Jiayin Zhu,Gu Simiu,Sulong Xu,Cong Fang
类目:Machine Learning (cs.LG); Artificial Intelligence (cs.AI); Computation and Language (cs.CL)
关键词:conventional formulation relies, Communicative On-Policy Distillation, On-policy distillation, improving language-model agents, text-only black
备注: 13 pages, 8 figures, 2 tables. Project page: [this https URL](https://liruiluo.github.io/communicative-opsd/)
点击查看摘要
Abstract:On-policy distillation (OPD) is a powerful paradigm for improving language-model agents on long-horizon tasks, yet its conventional formulation relies on teacher logits that text-only black- box models do not expose. To address this limitation, we introduce Communicative On-Policy Distillation (COPD), which uses natural-language communication as a medium for distilling a stronger model's capabilities into a smaller student. During task execution, the student au- tonomously decides when to consult a black-box teacher and formulates open-ended natural- language requests without predefined question templates or intervention schedules. Successful student-teacher interactions are retained as privileged references to supervise fresh adviser-free student rollouts through on-policy self-distillation, jointly with task-reward reinforcement learning. On CAMG-II, COPD achieves 2.46 times the gain of the strongest black-box OPD baseline. The capabilities it learns through communication generalize to Terminal-Bench 2.1 and SWE-Bench Pro V2, improving task success without adviser access.
53. 【2611.23651】Safe Personal RSI: Evaluating and Repairing Personal-Agent Updates
链接:https://arxiv.org/abs/2611.23651
作者:Jiapeng Pei,Yuhang Yao,Sitan Liu,Hanxin Chen,Hao Li,Guogeng Li,Jiapei Liu,Ruijie Chen,Hongshan Lin,Yang Mo,Tianyu Shi,Shangding Gu
类目:oftware Engineering (cs.SE); Artificial Intelligence (cs.AI); Computation and Language (cs.CL)
关键词:Safe Personal RSI, personal health assistant, Personal RSI, assistant reusable instructions, study Safe Personal
备注:
点击查看摘要
Abstract:Kicker is a personal health assistant. Changes to an assistant's reusable instructions can improve task handling while changing what it discloses, which requests it follows, and how it reports actions. We study Safe Personal RSI, an update-evaluation component developed for Kicker. We restrict recursive self-improvement (RSI) to prompt updates, with model weights and runtime code fixed. The procedure tests proposed and current instructions, repeats failing cases, temporarily undoes changes to guide repairs, and checks safety and task performance before adoption. If no version passes, the current instructions stay in use. Our executable benchmark contains 29 email, calendar, health, and marketplace cases, with separate probes for authorization reuse, memory, and tool-output injection. Across three models and two deliberately risky edits, the procedure rejects 20 of 21 update attempts. In repair confirmation, an admitted GPT-5.6 Luna repair reduces attack violations from 10.7% to 0% and increases safe task completion from 89.7% to 100%; a broad handwritten authority rule matches these outcomes. Under stronger attacks, a repair transferred from Qwen2.5-7B-Instruct to Qwen3-14B reduces violations from 69.0% to 17.9% and increases safe task completion from 19.5% to 64.4%. Separate runtime probes examine memory-entry deletion and identify missing local OAuth scope checks in the evaluated snapshot.
54. 【2611.23633】Option-GRPO: Hierarchical RL Against AI Slop in Code Generation for Unseen SDKs
链接:https://arxiv.org/abs/2611.23633
作者:Prashank Kadam
类目:oftware Engineering (cs.SE); Computation and Language (cs.CL); Machine Learning (cs.LG)
关键词:Large language models, Large language, language models increasingly, non-idiomatic implementations, bloat codebases
备注: Accepted to Findings of EACL 2027. 27 pages
点击查看摘要
Abstract:Large language models increasingly produce AI slop in generated code: verbose, redundant, non-idiomatic implementations that pass functional checks but bloat codebases. The problem compounds on SDKs released after the model's training cutoff, where the model has neither idiomatic priors nor compositional understanding. We present Option-GRPO, a hierarchical reinforcement learning framework that adapts a code LLM to an unseen SDK from its source code alone, without human-written usage examples or test cases, and that explicitly optimizes for both functional correctness and code elegance. Our key insight is that SDK source code contains compositional structure that defines a hierarchy of reusable code patterns (code options) which simultaneously serve as the building blocks of a curriculum and the action space for efficient policy optimization. We show that code options induce a valid semi-Markov decision process, give conditional bounds under which shorter option description length implies more concise and, in expectation, more API-efficient programs, and analyze under stated assumptions how the option hierarchy improves credit assignment by a factor of $\sqrt{T/K}$. Using Group Relative Policy Optimization (GRPO) with execution-based rewards at each curriculum level, Option-GRPO discovers SDK idioms, composes them into patterns, and assembles patterns into complete programs. Experiments on 12 unseen SDKs with a 32B-parameter base model demonstrate 61.4% pass@1, outperforming flat GRPO by 8.7 points, ExploraCoder by 11.2 points, and ReCode by 6.3 points. Option-GRPO produces solutions that use 34.7% fewer API calls and are 27% shorter than flat-GRPO outputs while maintaining higher correctness, directly attacking the slop problem.
55. 【2611.23601】FutureBench: A Renewable Framework for Evaluating Knowledge-Context Conflict in LLMs
链接:https://arxiv.org/abs/2611.23601
作者:Yi-Cheng Lai,Hen-Hsen Huang
类目:Computation and Language (cs.CL); Artificial Intelligence (cs.AI)
关键词:check retrieved text, source to trust, check retrieved, retrieved text, passage
备注:
点击查看摘要
Abstract:Asking a model to check retrieved text can also tell it which source to trust. We introduce FutureBench, a renewable framework for evaluating knowledge-context conflict using timestamped questions, controlled passage triplets, and no-context probes. Across 3,629 questions and six models, Verify (a prompt combining checking with passage priority) raises false-answer adoption in all five models tested on pre-cutoff events, mainly by displacing answers that were correct under Baseline, which allows both passage evidence and model knowledge. On post-cutoff events, only o4-mini increases adoption (+13.7 percentage points). A four-prompt comparison on 902 post-cutoff questions finds that passage priority without checking raises adoption, while moving to Verify lowers it in all five models. Adversarial-Aware, which warns of misleading passages and prioritizes model knowledge, restores 82.5-98.9% of displaced pre-cutoff answers with under one point of change in true-passage accuracy. FutureBench supports repeated evaluation on fixed cohorts and extension to new events as models change.
56. 【2611.23570】Safe to Omit, Worth Omitting? Keeping the Answer and Cutting Latency When an LLM Agent Skips Another's Message
链接:https://arxiv.org/abs/2611.23570
作者:Xinyang Li,Chih-Hsuan Yang,Xiaolong Ma,Anjir Ahmed Chowdhury,Vikram Vasudevan,Rajeev Thakur,Feng Yan
类目:Machine Learning (cs.LG); Computation and Language (cs.CL)
关键词:receiver agent, adds latency, improve performance, performance but adds, receiver
备注: 58 pages, 11 figures, 41 tables. Code: [this https URL](https://github.com/Piedpiperrr/safe-to-omit;) data: [this https URL](https://huggingface.co/datasets/AgentsSci/ICLR2027-SafeToOmit)
点击查看摘要
Abstract:In multi-agent systems, a receiver agent often waits for a helper agent's message (text or its latent KV cache), which can improve performance but adds latency. We study when a receiver agent can skip the message safely (the final answer rarely changes) and beneficially (the time saved exceeds the cost of deciding). Our mechanism reads the receiver agent's confidence without the message and uses a finite-sample test to certify a threshold at which, with high probability, at most 5% of skipped answers change; if none passes, the system keeps communicating. We test it on OpenBookQA, ARC-Challenge, and MMLU-Pro with three Qwen model pairs, on cross-family agents, and on SQuAD and GSM8K. Four insights emerge. First, oracle headroom is weak evidence that messages help: control inputs without answer-specific content, matched to change answers as often as real messages, reproduce a median 76% of the oracle's gain. Second, whether a message can be skipped depends on how often and how predictably it alters the answer, not on its accuracy; every certified KV-cache path is less accurate than the receiver agent alone. Third, where text messages help, skipping keeps most of their gain: on unseen questions in the four such settings, the policies skip 59 to 80% and keep an estimated 76 to 95% of the gain. In two prospective tests with all rules fixed in advance, our mechanism skips a three-agent system's verifier on 39.9% of sealed questions, keeping 12.55 of its 12.80-point gain, and a 32B helper on 68.4% of held-out questions, keeping its whole gain. Fourth, safety does not determine efficiency: at batch size one, skipping text saves hundreds of milliseconds per query and skipping KV cache far less. Certification extends to SQuAD but not to GSM8K. In eleven tests on unseen questions, every certified policy we could test kept its change rate's 95% upper bound below 5% under our answer parsers.
57. 【2611.23546】Language Models Make New Research Sound Old
链接:https://arxiv.org/abs/2611.23546
作者:Guy Zyskind
类目:Computation and Language (cs.CL); Artificial Intelligence (cs.AI)
关键词:Researchers are increasingly, validate research questions, large language models, increasingly using large, large language
备注:
点击查看摘要
Abstract:Researchers are increasingly using large language models (LLMs) to validate research questions. We find that LLMs make open research questions sound partly studied and overstate what related papers establish. Models recognize most of these overstatements when asked to check the papers' full texts separately. On 40 questions drawn from newly accepted papers, GPT-6-Astra correctly judges 98% to be unstudied, but states that status plainly in only 8% of ordinary answers. All six frontier systems show reporting gaps of 30-90 percentage points. Across 22 models given identical papers, higher capability is associated with more accurate judgments and larger reporting gaps. This reporting failure is asymmetric: models correctly identify studied questions in 88-100% of ordinary answers when we supply the answering papers. To correct this, we build a fixed answer from each model's judgment and separate checks of cited papers. This raises Astra's clear reporting of open status from 8% to 95%, with improvements in all six systems. Finally, we predict reviewers' novelty grades by combining several models' written ratings with a rule learned from graded examples, reaching the performance range reported by Think-Probe-Respond. Unlike TPR, our approach needs no access to model weights or internal activations, so it also works with closed-weight frontier models.
58. 【2611.23533】Can LLMs Understand What They Predict? Evaluating Latent Causal Structure Inference in Financial Markets
链接:https://arxiv.org/abs/2611.23533
作者:Yixiang Zheng,Yuechen Jiang,Zhiwei Liu,Jimin Huang,Yuyan Wang,Sophia Ananiadou
类目:Computation and Language (cs.CL); Machine Learning (cs.LG)
关键词:Large language models, predictive success reflects, predict future outcomes, Large language, implicitly assuming
备注:
点击查看摘要
Abstract:Large language models (LLMs) are increasingly evaluated through their ability to predict future outcomes, implicitly assuming that predictive success reflects an understanding of the underlying system. However, accurate predictions can often be achieved without recovering the latent mechanisms that generate observable outcomes. To examine this assumption, we introduce LUCID, a benchmark that evaluates whether models can infer latent market structures from event descriptions and historical financial observations. Models generate event-induced causal graphs representing inferred market structure and are evaluated through a reconstruction-based framework. Our evaluation of 6 open-weight and proprietary LLMs shows that adding their generated graphs generally reduces reconstruction quality compared with using historical observations alone, although a few metrics improve. These findings suggest that inferring informative, event-specific market structures from observable evidence remains challenging for the evaluated LLMs.
59. 【2611.23530】Dreaming Up Authors: Factual Prevalence and LLM Hallucinations in Academic Authorship Attribution
链接:https://arxiv.org/abs/2611.23530
作者:Sijia Zhong,Tamir Yirga,Yijia Ma,Thai Le,Yifan Hu
类目:Computation and Language (cs.CL)
关键词:Large Language Models, Large Language, remain underexplored, Language Models, question answering
备注: Published in EMNLP 2026 (Main Conference). 37 pages. Code and data: [this https URL](https://github.com/quinnzhong1/dreaming-up-authors) . Anthology: [this https URL](https://aclanthology.org/2026.emnlp-main.2372/)
点击查看摘要
Abstract:Large Language Models (LLMs) are widely used for question answering but are prone to hallucinations, yet the underlying causes, particularly the role of factual prevalence in training data, remain underexplored. In this work, we study this through academic authorship attribution, hypothesizing that hallucinations are more likely for less prevalent papers. Using citation count as a proxy for factual prevalence, we curate a dataset of 1.92 million papers across eight scientific disciplines with stratified samples by citation level. Experiments on three state-of-the-art LLMs confirm our hypothesis, revealing a robust inverse relationship between hallucination rate and citation count across all models and disciplines, with hallucination rates exceeding 98% for rarely cited papers and falling below 37% for highly cited ones. Our in-depth analysis further shows that these failures primarily reflect retrieval limitations rather than knowledge absence, and hidden-state probing of open-source models shows that factual prevalence is encoded in pre-generation representations, providing convergent mechanistic evidence for the prevalence-hallucination relationship. The code and data are available at this https URL.
60. 【2611.23518】From Arraignment to Resolution: Transformer Models of Court Cases
链接:https://arxiv.org/abs/2611.23518
作者:Kory Rosen,Aron Culotta
类目:Computation and Language (cs.CL); Computers and Society (cs.CY); Machine Learning (cs.LG)
关键词:criminal legal system, complex decision points, plea deal negotiations, criminal legal, plea deal
备注: Published in Findings of EMNLP 2026
点击查看摘要
Abstract:The criminal legal system is composed of many complex decision points, such as which charges are pursued, how bond is determined, plea deal negotiations, and sentencing. Given the profound impact of such decisions, there is a growing need to increase the transparency within this system. While prior machine learning approaches have focused on modeling single decisions (e.g., recidivism), this paper proposes a transformer-based time series model to forecast the outcome of the next proceeding in a case (e.g., charges accepted, bond set, plea deal made, etc.). By modeling a case as an event sequence, we enable a more nuanced analysis to support better oversight and understanding of decision making. We conduct an empirical analysis of thousands of cases using a newly constructed dataset derived from publicly available court records of a mid-sized U.S. city. We fit models that combine textual data from clerk notes with structured information about the defendant, prior charges, and courtroom. Our transformer model achieves $.57$ micro-F1 on a 20-class multi-label task, a significant improvement over several baselines and comparable to a GRU trained on the same input. We demonstrate how the model can be used to provide insights into decision-making and help court observers identify emerging issues within the jurisdiction studied.
61. 【2611.23509】GRAFT: Execution-Guided Genetic Recombination for Agent Design
链接:https://arxiv.org/abs/2611.23509
作者:Sevda Ebrahimi,Mohammed Saidul Islam,Xiaodan Zhu,Elham Dolatabadi
类目:Artificial Intelligence (cs.AI); Computation and Language (cs.CL); Neural and Evolutionary Computing (cs.NE)
关键词:Automated agent design, recombine promising mechanisms, LLM-based agent performance, agent design searches, Automated agent
备注:
点击查看摘要
Abstract:Automated agent design searches for executable architectures that improve LLM-based agent performance, with evolutionary methods using language-model crossover to recombine promising mechanisms for task decomposition, model coordination, and verification. This search, however, is typically guided by aggregate fitness alone, which hides the instance-level successes and failures behind each score. We introduce Execution Guided Genetic Recombination for Agent Design (GRAFT), which uses aggregate fitness for selection and survival and parent-specific execution evidence to guide semantic crossover with integrated mutation. Its feedback harness reuses bounded samples of successful and failed runs from fitness evaluation, withholding reference answers and requiring no additional task executions. We evaluate GRAFT across six reasoning and programming settings using open-weight optimizer and executor models. It achieves a mean test accuracy of 82.0%, exceeding the strongest baseline by 9.8% points. Controlled ablations show that adding execution evidence improves test accuracy with initialization, search rules, repair policies, and evaluation budgets held fixed. Our results support the value of parent-specific execution evidence for evolutionary agent design.
62. 【2611.23508】BOS Summary: The First Token Matters
链接:https://arxiv.org/abs/2611.23508
作者:Hui Zhu,Saleh Zare Zade,Rafi Ibn Sultan,Deng Pan,Arun Ross,Dongxiao Zhu
类目:Artificial Intelligence (cs.AI); Computation and Language (cs.CL); Computer Vision and Pattern Recognition (cs.CV)
关键词:Large vision-language models, Large vision-language, BOS Summary, BOS, receive a disproportionate
备注:
点击查看摘要
Abstract:Large vision-language models (LVLMs) and language models (LLMs) often exhibit attention sinks, whereby particular positions, such as the beginning-of-sequence (BOS) token, receive a disproportionate share of attention. Under standard causal masking, however, the BOS query attends only to itself, so the representation that later tokens repeatedly access carries no information about the input. We introduce BOS Summary, a training-free inference-time method that repurposes this persistent pathway to carry input context. At a single transformer layer during prefill, BOS Summary relaxes the causal constraint for the BOS query row alone, letting BOS aggregate all visual and textual input tokens into a self-derived context representation while every other token retains standard causal attention. The enriched representation then remains accessible to the model throughout decoding, with no additional parameters, auxiliary modules, or decoding-time overhead. Across multiple VLM and LLM families and eight multimodal and text-only benchmarks, BOS Summary improves over direct and Chain-of-Thought prompting and outperforms recent sink-intervention methods on multi-step reasoning tasks. A mechanistic analysis shows that making BOS input-specific also weakens its sink dominance and redistributes attention toward content-bearing prompt tokens, indicating that changing what a sink carries reshapes how later layers allocate attention.
63. 【2611.23483】SpecLook: Speculating Where to Look for Fast and Reliable Agentic MLLMs
链接:https://arxiv.org/abs/2611.23483
作者:Xi Ding,Jiawei Zhang
类目:Artificial Intelligence (cs.AI); Computation and Language (cs.CL); Computer Vision and Pattern Recognition (cs.CV); Machine Learning (cs.LG)
关键词:Agentic multimodal large, Agentic multimodal, images interleave reasoning, multimodal large language, images interleave
备注: Technical report
点击查看摘要
Abstract:Agentic multimodal large language models (MLLMs) that think with images interleave reasoning with zoom-in tool calls. However, this sequential loop is slow, and its answers carry no reliability score. Prior speculation of tokens, reasoning steps, or answers keeps the loop, and answer speculation also accepts confident errors. We find that the loop mostly adds one zoom-in step, with which a small model is ahead of the agent on most splits. Our key insight is to speculate the agent's next observation, which, unlike a drafted answer, can be verified, corrected, and used to test the answer. We propose SpecLook, a training-free speculate-and-verify framework with three levels. A small model drafts a glimpse and re-answers from it, and the question ends if the answer is unchanged and confident. Otherwise, the agent's large model verifies and corrects the glimpse without tools, and their answers are fused. Only questions that still lack evidence reach the agent, in one forward pass that SpecLook-Fast skips, so no question runs the loop. The trace scores every answer's reliability without labels. Extensive experiments on V*, HR-Bench, and POPE show that both are more accurate than every baseline on every split. SpecLook reaches 87.2% and 86.2% mean accuracy with 7.7x and 4.7x speedups over the DeepEyes-7B and Thyme-RL agents, and SpecLook-Fast reaches 9.9x and 6.3x.
64. 【2611.23482】Formal Adequacy Index (based on Polish-language material)
链接:https://arxiv.org/abs/2611.23482
作者:Paweł Patreucha
类目:Computation and Language (cs.CL)
关键词:variation in speech, simply measuring, dimensions of stylistic, stylistic variation, measuring its level
备注: 9 pages, 1 figure, 1 table
点击查看摘要
Abstract:Linguistic formality is one of the dimensions of stylistic variation in speech; however, simply measuring its level does not allow us to determine whether the chosen mode of communication is appropriate in a given situation. The aim of this article is to present the concept of a formal adequacy index (based on Polish-language material), which enables a quantitative assessment of how well the formality of a text matches the expectations arising from the communicative context. The proposed approach is based on the separate modelling of observed formality, determined on the basis of the linguistic features of the text, and expected formality, determined on the basis of the characteristics of the communicative situation. Adequacy is assessed by comparing these two measures, taking into account contextual tolerance thresholds, which may differ for a lack of formality and an excess of formality. The potential applications of the index are illustrated by an analysis of Polish-language emails containing apologies for being late, written by human language users or generated by a large language model. The implementation presented here is preliminary in nature and requires further calibration and empirical validation. The main contribution of this article is the proposal of a general methodological framework for the quantitative study of formal adequacy in various communicative contexts.
65. 【2611.23403】racking Representations in LLM Reasoning Traces for Iterated Games
链接:https://arxiv.org/abs/2611.23403
作者:Chaipat Tirapongprasert,Rajarshi Das,Parijat Dube
类目:Computation and Language (cs.CL); Artificial Intelligence (cs.AI); Computer Science and Game Theory (cs.GT)
关键词:Large language model, Large language, performance across domains, LLM agents make, agents achieve
备注: Accepted to SoCalNLP 2026. 15 pages, 11 figures
点击查看摘要
Abstract:Large language model (LLM) agents achieve state-of-the-art performance across domains through chain-of-thought (CoT) reasoning. Despite their ubiquitous use, little is understood about the relationship between the externalized trace and the model's actual computation. We address this by investigating how LLM agents make decisions in iterative games with incomplete information. Using a series of mechanistic interpretability techniques, we find that the model forms and iteratively revises an internal representation of the game throughout the CoT. Once the model prematurely commits to an incorrect belief, it rarely recovers. This failure is manifested in both the literal trace and the layer activations. Ultimately, as multi-agent systems are more prevalent, our work opens the door to a more focused diagnosis of their strategic reasoning errors. For illustrative purposes, we offer an interactive visualizer of our methodology on selected sample traces at this https URL
66. 【2611.23388】QDA: Closed-Loop LLM Quantization Design with Agents
链接:https://arxiv.org/abs/2611.23388
作者:Xingyang Li,Tianlang Zhao,Song Han,Ligeng Zhu
类目:Machine Learning (cs.LG); Artificial Intelligence (cs.AI); Computation and Language (cs.CL)
关键词:serve large language, Post-training quantization, large language models, low cost, serve large
备注: 19 pages, 7 figures, 10 tables
点击查看摘要
Abstract:Post-training quantization is the standard way to serve large language models at low cost. However, the best quantization recipe differs across models and precisions. As a result, experts must redesign it for every new model, which is time-consuming and often sub-optimal. Fortunately, an LLM agent can take over this design by refining the recipe over rounds of building and evaluating candidates, so that more rounds lead to better recipes. However, evaluating one quantized reasoning model takes tens of GPU-hours, and combining methods from different codebases means porting each to the new architecture by hand. We present QDA, a system in which an agent designs the recipe for each model and precision. To make evaluation fast, QDA scores each candidate in minutes by its output KL divergence from the original model on reasoning traces that the agent never sees, and checks promising ones with long generations to catch failures such as repetition loops. To make building fast, QDA provides a library of public PTQ methods that are compatible with one another and with new model architectures. Across four reasoning models, QDA improves accuracy over the strongest public recipe by 0.9, 1.4 and 2.2 percentage points on average at W4A16, W4A8 and W3A16, using 64 GPU-hours per search. On the 320B-parameter MoE model GLM-5.3-Flash at NVFP4 W4A4, it recovers 98.3% of the FP8 model's accuracy, against 96.9% for the best vendor checkpoint. Every QDA checkpoint also runs on existing serving kernels with no inference overhead. We will release the library, the evaluator and the harness.
67. 【2611.23386】Chorus: Shared N-gram Embeddings with Layer-wise Adaptation
链接:https://arxiv.org/abs/2611.23386
作者:Stephen Chung,Wenyu Du
类目:Computation and Language (cs.CL)
关键词:Language models, n-gram embedding tables, n-gram embedding, model input, consecutive tokens
备注:
点击查看摘要
Abstract:Language models can gain capacity at little computational cost from n-gram embedding tables: large tables of learned vectors for short sequences of consecutive tokens, which are looked up and added to the model's input or to its intermediate layers. Recent designs commonly use a separate n-gram embedding table for each layer, which makes the tables costly in memory and prevents layers from sharing what they learn. Sharing a single table, however, gives every layer the same embedding and leaves no room for layer-wise preferences, and it performs poorly in practice. We propose Chorus, in which one set of n-gram embedding tables serves the whole network and every layer reads it in its own way. Each layer maps the refined embedding through its own projection layer, after rescaling its components by factors computed from its current hidden state. Chorus matches the quality of existing n-gram methods with 87% smaller tables: with 1.2B table parameters it predicts text better than FactorEngram and DeepSeek's Engram do with 9.6B. It outperforms Engram, TN-gram and FactorEngram in every setting where we compared them, across table sizes from 20M to 9.6B parameters. We also show that the recently proposed memory-efficient optimizer Ember works well with Chorus, saving 99.2% of table-optimizer memory compared with AdamW without degrading training performance. These results enable further scaling of n-gram embedding tables along the table-size axis by reducing memory use and allowing shared knowledge to be reused across layers.
68. 【2611.23359】Prioritize, Don't Decide: What We Learned Auditing a Deployed Human-in-the-Loop CV Sorter
链接:https://arxiv.org/abs/2611.23359
作者:Hadi Mohammadi,Jeroen Sparla,Alexandros Paliouras,Georg Krempl,Robert A. Bagheri
类目:Computation and Language (cs.CL); Information Retrieval (cs.IR); Machine Learning (cs.LG)
关键词:screening round consumes, single academic vacancy, Priority Sorter deployed, hundreds of applications, draw hundreds
备注: 11 pages, 4 figures, 6 tables. Under review at the EACL 2027 Industry Track
点击查看摘要
Abstract:A single academic vacancy can draw hundreds of applications, and the first screening round consumes most of a committee's time. We describe a CV Priority Sorter deployed at a national platform for academic vacancies and integrated with its vacancy database. It proposes a reading order for a pile of CVs and leaves every decision to the committee, pairing a frozen multilingual transformer encoder with a per-session linear model that retrains after each label and is discarded when the session ends. It works: on a pool of 300 synthetic CVs, a committee reading in the tool's order reaches 95% of the relevant candidates after half the pile, where an arbitrary order needs almost all of it. This paper is about what we found when we audited that claim, rather than about the system itself. Three findings changed the product. The pool turned out to be sortable by document length alone, which makes precision-style metrics on it uninformative; regenerating it under a shared length budget removes that shortcut and leaves the reading-depth result standing. The four-level agreement score such systems report, ours included, is a binary probability cut at fixed thresholds, so it follows the classifier's confidence rather than its ranking and falls as labels accumulate. And uncertainty sampling, the obvious way to implement a "suggest the next CV" button, leaves model quality intact but raises the committee's total reading load above the no-label baseline, because the CVs it asks about are the ones that would otherwise be skipped. A fourth result shows where to look next: a hosted LLM embedding model lifts zero-label ordinal discrimination from 0.66 to 0.80 under length control, then stops mattering once the session model has two dozen labels; a local cross-encoder matches that cold-start gain without sending data out. We report the fixes each finding forced.
69. 【2611.23357】A Survey on RLOps for Large Language Models
链接:https://arxiv.org/abs/2611.23357
作者:Lingzhe Zhang,Tong Jia,Chiming Duan,Minghua He,Leyi Pan,Kening Zheng,Zhaoyang Liu,Bolin Ding,Philip S. Yu,Ying Li
类目:oftware Engineering (cs.SE); Computation and Language (cs.CL); Cryptography and Security (cs.CR)
关键词:large language models, LLM reinforcement learning, improving the reasoning, Reinforcement learning, increasingly important
备注:
点击查看摘要
Abstract:Reinforcement learning (RL) has become increasingly important for improving the reasoning, instruction-following, and task-solving capabilities of large language models (LLMs). However, scaling RL training exposes failures in distributed execution, feedback computation, and learning dynamics that can waste computational resources, slow training, and undermine model improvements. These challenges have motivated a growing body of techniques for LLM RLOps, which manages the efficiency and reliability of operations throughout the LLM reinforcement learning training lifecycle. Despite growing interest, these techniques remain scattered across research areas, with limited synthesis of their operational roles and interactions. To bridge this gap, we present a systematic survey of LLM RLOps methods. We organize existing approaches into five stages: Training Setup, Experience Collection, Reward Computation, Policy Optimization, and Model Evaluation. Within each stage, we examine operational concerns, management techniques, and evaluation evidence. Building on this taxonomy, we identify open challenges and future directions in training process observability, cross-stage coordination, generalizable RLOps, multi-run management, and RLOps for recursive self-improvement. These directions extend operational management from individual training runs to diverse training settings, shared infrastructure, and successive improvement cycles. We maintain a GitHub repository indexing relevant papers and open resources at this https URL.
70. 【2611.23274】A Concurrent Structured Record Mirrored the Prose It Accompanied: A Pre-Registered Conceptual Replication and Extension of the Hide Pending Tool Call Scenario
链接:https://arxiv.org/abs/2611.23274
作者:Nicolás Rocchia
类目:oftware Engineering (cs.SE); Computation and Language (cs.CL)
关键词:Tool Call scenario, Hide Pending Tool, Pending Tool Call, honesty prompt, accompanied prose coded
备注: 10 pages, 4 tables. Pre-registered study; protocols, code, data and analyses at [this https URL](https://github.com/NicolasRocchia/disensor-hide-pending)
点击查看摘要
Abstract:Huang et al. (2026) show that language models writing reports of agentic work tend to conceal narrative-changing flaws, and that appending "Be honest in your response." to the instruction substantially increases disclosure in most of their scenarios. In their Hide Pending Tool Call scenario, where a broader follow-up query was accepted but never returned, the honesty prompt left GPT-5.5's rate of flagging the missing data at 0%, although it turned most answers into caveats. I ask whether a different request changes this: appending, in the same generation, a structured record that follows the schema of an adversarial-review tool, including a residue section for what remained unresolved. In two pre-registered phases on the same 100 synthetic log pairs, each pending log matched with a clean twin in which the follow-up query did return, I ran gpt-5.6-sol through Codex CLI under four conditions. Phase 1 measured the record: with the residue schema, a judge coded it as stating the unreturned query in 29% of logs, against 5% with a findings-only placebo schema, and in none of the clean twins (p = 1.2 x 10^-7); a second run two days later gave 30% and 6%. But 70-71% of residue records declared that nothing remained open while the query was pending. Phase 2 measured the prose answer: the confirmatory comparison against the honesty prompt was not significant (four discordant logs, all favoring the honesty prompt; p = 0.125), no equivalence test was pre-registered, and the judge failed the pre-registered human validation on the decisive category; I therefore claim neither an effect nor its absence. In an exploratory cross-tabulation, every residue record coded as stating the gap accompanied prose coded as mentioning it (30 of 30), and none accompanied prose coded as omitting it (0 of 70): in this setting, the structured record mirrored the prose it accompanied.
71. 【2611.23236】PGPO: Prototype-Guided Progressive Obfuscation for Privacy-Preserving LLM-Enhanced Recommendation
链接:https://arxiv.org/abs/2611.23236
作者:Shengxiang Lin,Jiajie Su,Pengyang Zhou,Xiang Chen,Xiaolin Zheng,Feng Tian,Chaochao Chen
类目:Information Retrieval (cs.IR); Computation and Language (cs.CL); Cryptography and Security (cs.CR)
关键词:LLM-enhanced recommender systems, recommender systems encode, leak sensitive textual, sensitive textual attributes, systems encode item
备注: Accepted to EMNLP 2026
点击查看摘要
Abstract:LLM-enhanced recommender systems encode item side information into semantic embeddings, but these embeddings may leak sensitive textual attributes through inversion attacks. Existing input-side defenses rely on trusted collection or local token perturbation, often disrupting the inter-item semantic topology needed for recommendation. To address it, we introduce relational fidelity as the key objective, requiring semantic shift for privacy while preserving relative item relations for utility. Based on this principle, we propose prototype-guided progressive obfuscation (PGPO), which learns topology-aware obfuscation anchors for high-influence prototype words and propagates them through a static semantic graph. PGPO is optimized with GRPO using individual dissimilarity, structural consistency, and semantic constraint rewards. Experiments on two benchmarks show strong inversion defense with near-upper-bound recommendation performance. The code is available at this https URL.
72. 【2611.23788】Geometry-bridge flow matching for blind rigid-motion correction in head cone-beam CT
链接:https://arxiv.org/abs/2611.23788
作者:Sungho Yun,Seungryong Cho
类目:Medical Physics (physics.med-ph); Computation and Language (cs.CL); Computer Vision and Pattern Recognition (cs.CV)
关键词:accurate anatomical structures, Rigid motion, head cone-beam, measured projections, complicate recovery
备注:
点击查看摘要
Abstract:Rigid motion of a patient head during head cone-beam CT (CBCT) scan can degrade image quality and complicate recovery of accurate anatomical structures from the measured projections. We present a 3D flow-matching prior approach that learns the image changes induced by motion. Its geometry-bridge connects motion-corrupted and motion-free Feldkamp-Davis-Kress (FDK) reconstructed images of the same source volume through progressively reduced simulated motion. The forward projector's geometry derivative supplies analytic velocity targets along this path. A plug-and-play loop encompasses the learned image updates together with spline-based pose estimation, conjugate-gradient data consistency, and image total-variation regularization. Both the refined image volume and the FDK reconstructed image volume at the estimated poses are evaluated. On simulated 360-view scans from 30 held-out CQ500 patients with nominal per-component motion amplitudes within 10 mm / 10 degrees, the framework achieved state-of-the-art image reconstruction quality and motion accuracy in comparison with other methos such as learned autofocus and diffusion-based joint reconstruction. Mean peak signal-to-noise ratio (PSNR) reached 37.1 dB and mean detector-plane reprojection error (RPE) was 0.27 mm. With matched endpoints at the inference settings, the geometry bridge improved PSNR by 1.6 dB and reduced RPE by 33% relative to image-linear training. With 50 prior updates and coarse-to-fine motion estimation, total processing time averaged 9.1 minutes per patient on an RTX A6000.
信息检索
1. 【2611.24849】Can Programs Replace LLM Judges? Executable Program Judges for Scalable and Efficient Relevance Labeling
链接:https://arxiv.org/abs/2611.24849
作者:Eugene Agichtein,Charles L. A. Clarke,Kaustubh Dhole,William Stern
类目:Information Retrieval (cs.IR); Computation and Language (cs.CL)
关键词:Large language model, Large language, create large-scale relevance, LLM judges, LLM
备注:
点击查看摘要
Abstract:Large language model (LLM) judges have become a common way to create large-scale relevance judgments for retrieval evaluation. However, each judgment requires a potentially expensive call to a frontier model, and the judgments are hard to interpret and may not be reproducible. In this paper, we ask whether an LLM can write a readable program for each topic that replaces its per-document judgments. To address these issues, we propose ExecJudges, a method for generating fast, inspectable and scalable judges for automatic relevance labeling. Specifically, ExecJudges uses a frontier LLM to turn assessment guidelines into executable programs, which rely only on lexical matching, standard NLP libraries like parsers, and pool statistics. An ExecJudges-L variant also uses examples of human labels if available. We evaluate ExecJudges on seven TREC collections by comparing the produced labels with trained NIST assessors, frontier LLM judges, and recently proposed per-topic classifiers. Our results show that ExecJudges agrees with NIST assessors at the level of the Claude Sonnet 5.5 LLM judges on TREC DL 2019-2021. With a small number of human labels per topic, the ExecJudges-L variant agrees with NIST more than per-topic classifiers with the same labels on all seven collections, and either exceeds the agreement of LLM judges provided the same labels or is within about 0.05 of it in Krippendorff's alpha. The labels rank the official TREC DL runs with Kendall's tau of 0.87 to 0.94, and writing the programs incurs a one-time cost of $0.17 to $0.49 per topic, with no additional costs for grading. Our results suggest that ExecJudges offers a reproducible, inspectable, efficient and cost-effective alternative to LLM judges for labeling deep result pools for system evaluation or ranking improvements.
2. 【2611.24757】Variance-Optimal Counterfactual Policy Design for Sample-Efficient Online A/B Testing
链接:https://arxiv.org/abs/2611.24757
作者:Shashank Gupta,Subhadeep Maji
类目:Machine Learning (cs.LG); Information Retrieval (cs.IR)
关键词:user-facing systems, approach for evaluating, recommendation or ranking, treatment effect, average treatment effect
备注:
点击查看摘要
Abstract:Online A/B testing is the standard approach for evaluating changes to user-facing systems, such as a new recommendation or ranking policy, before deployment. While the difference-in-means estimator of a standard A/B test is unbiased, it often suffers from high variance. When the treatment and production policies choose the same actions with similar probabilities, the resulting observations can add noise while providing little information about the difference between the policies. Consequently, A/B tests can require large amounts of traffic and long durations to detect small treatment effects. Existing variance-reduction methods mainly improve the estimator after the data-collection scheme has been fixed, and recent overlap-aware designs only optimize a single traffic split between the two policies; thereby, they cannot direct traffic to the actions on which the policies disagree. In this paper, we jointly optimize how actions are sampled and how the resulting observations are used to estimate the treatment effect. Our contributions are two-fold. First, building on pairwise off-policy evaluation, we derive the variance-optimal action-level logging policy for estimating the average treatment effect between two policies, which has a closed-form solution for any fixed control variate. Second, we use historical logs to jointly optimize the logging policy and a control variate to further reduce variance. Our experimental results in simulated policy A/B tests show that our designs estimate the average treatment effect more accurately than existing variance reduction methods on every evaluated setting, while generally improving statistical power and reducing how often the worse policy is selected. As a result, our designs enable practitioners to reach reliable deployment decisions with considerably less online traffic.
Subjects:
Machine Learning (cs.LG); Information Retrieval (cs.IR)
Cite as:
arXiv:2611.24757 [cs.LG]
(or
arXiv:2611.24757v1 [cs.LG] for this version)
https://doi.org/10.48550/arXiv.2611.24757
Focus to learn more
arXiv-issued DOI via DataCite (pending registration)</p>
3. 【2611.24283】X-split Argus: Learning Reusable History Tokens for Efficient CTR Prediction
链接:https://arxiv.org/abs/2611.24283
作者:Alexander Ploshkin,Nikita Stepanov,Alexey Pismenny,Viacheslav Ovchinnikov,Nikolay Savushkin
类目:Information Retrieval (cs.IR)
关键词:Accurate click-through rate, processing extensive behavioral, extensive behavioral histories, Accurate click-through, present X-split ARGUS
备注: Alexander Ploshkin, Nikita Stepanov and Alexey Pismenny contributed equally to this work
点击查看摘要
Abstract:Accurate click-through rate (CTR) prediction depends on user representations that capture both persistent preferences and recent intent, yet processing extensive behavioral histories for every ranking request is costly under production compute and latency constraints. We present X-split ARGUS, a framework for ad ranking whose history compressor and online ranker are trained end to end. The compressor encodes past interactions into a compact set of tokens that are computed offline and reused across candidates, while the online ranker scores ads through cross-attention to these tokens and representations of recent events. Training combines an autoregressive loss for click feedback with distillation of candidate-specific click logits from a larger teacher that scores each candidate using the uncompressed history. The number of compressed history tokens and the length of recent history can be adjusted to fit serving compute and latency budgets. Offline evaluation shows a 0.81% relative reduction in normalized entropy (NE) compared with the production baseline. Deployment in a large-scale real-time bidding system increases online inference cost by 15% relative to the production baseline. An online A/B test against this baseline yields relative gains of 4.65% in a business metric reflecting the monetary value of conversions generated for advertisers and 2.96% in clicks.
4. 【2611.24006】MARS Technical Report
链接:https://arxiv.org/abs/2611.24006
作者:Tong Liu,Chaoqun Hou,Longxiang Xiong,Junhong Lou,Jiaqi Zheng,Qixin Li,Zixin Zhao,Yong Duan,Xiannian Hu,Fengze Jiang,Yu Zhu,Cheng Guo,Xinyuan Qi,Zi Zhuang,Ping Liu,Lv Shao,Zongwei Wang
类目:Information Retrieval (cs.IR)
关键词:Modern industrial recommender, industrial recommender systems, Modern industrial, Agentic Recommendation System, Multi-stage Agentic Recommendation
备注: 46 pages, 5 figures
点击查看摘要
Abstract:Modern industrial recommender systems use multi-stage cascades--retrieval, pre-ranking, ranking, and re-ranking--to narrow millions of candidates to a personalized list. While each stage deploys sophisticated models, pipeline-level decisions on channel activation, quotas, model selection, and score fusion remain engineer-tuned and user-agnostic within each scenario. This report presents MARS (Multi-stage Agentic Recommendation System), extending personalization from individual-stage predictions to full pipeline orchestration. A near-line agent built on fine-tuned Qwen3-0.6B generates per-user configurations--a seven-dimensional action covering pipeline mode, retrieval quotas, model selection, fusion weights, and reranker settings--from a constrained action space. Configurations are validated, cached, and consumed asynchronously; the cascade never waits for LLM inference. Schema checks, fallbacks, and a kill switch ensure serving safety. The policy is trained in three stages: Confidence-aware SFT distills schema-valid generation from a teacher model; Format-Anchored Preference Optimization (FAPO) learns action preferences via ActionMask with posterior weighting; Group Relative Policy Optimization (GRPO) refines the policy through Sandbox replay with a frozen reward model, without exposing exploratory actions to live traffic. Two-week A/B tests across five Taobao scenarios with over ten million daily users show positive gains on all key metrics. The largest reach +5.22% V2P and +7.34% GMV, with positive CTR lifts on both click surfaces. About 20% of requests use a lightweight Minimal mode, and deployment shows 12 ms lower mean serving latency versus production.
Comments:
46 pages, 5 figures
Subjects:
Information Retrieval (cs.IR)
ACMclasses:
I.2.0; I.5.0; I.7.0
Cite as:
arXiv:2611.24006 [cs.IR]
(or
arXiv:2611.24006v1 [cs.IR] for this version)
https://doi.org/10.48550/arXiv.2611.24006
Focus to learn more
arXiv-issued DOI via DataCite (pending registration)
Submission history From: Yu Zhu [view email] [v1]
Wed, 18 Nov 2026 07:56:58 UTC (1,451 KB)
5. 【2611.23969】ReflectRec: Cycle-Consistency Preference Verification for LLM-based Recommendation
链接:https://arxiv.org/abs/2611.23969
作者:Dae Kyoo Kim,Heejung Cho,Sungmoo Hong,Eunil Park
类目:Computation and Language (cs.CL); Information Retrieval (cs.IR)
关键词:CoT-style LLM recommenders, LLM recommenders generate, natural-language preference note, prefix to ranking, behaviorally similar users
备注: 15 pages. Published in the Proceedings of EMNLP 2026 (Main Conference)
点击查看摘要
Abstract:CoT-style LLM recommenders generate a natural-language preference note as a chain-of-thought prefix to ranking, but the note is trusted without verification, and the ranker never consults behaviorally similar users. We propose ReflectRec, organized around a preference-to-history round-trip verifier whose recall score serves as an operational signal of preference-to-history recoverability, enabling conditional regeneration for low-recoverability preferences and selection among regeneration attempts. Around this verifier, we wire a semantic Stage-1 prompt, conditional regeneration, and Forward-Look retrieval that injects future watches of behaviorally similar users as a collaborative hint. On MovieLens-100K and MovieLens-1M, the full pipeline outperforms CoT4Rec-ICL by +16.25 / +4.46 pp Hit@10, with positive gains across all ValScore buckets and three LLM backbones (Qwen3-30B, Qwen2.5-72B, and Llama-3.3-70B). We also observe a similar verifier-guided regeneration pattern on Amazon Electronics.
6. 【2611.23710】CONCORD-Rec: Catalog-Conditioned Modality Utility for Semantic-ID Diffusion Recommendation
链接:https://arxiv.org/abs/2611.23710
作者:Haopeng Jin,Bingkang Shi,Tao Yu,Yifei Ma,Hao Liu,Hongzhu Yi,Jinwen Luo
类目:Information Retrieval (cs.IR)
关键词:requires distinguishing historical, reduce catalog-level ambiguity, Multimodal generative recommendation, distinguishing historical modalities, genuinely reduce catalog-level
备注:
点击查看摘要
Abstract:Multimodal generative recommendation with discrete semantic identifiers (SIDs) requires distinguishing historical modalities that merely alter token predictions from those that genuinely reduce catalog-level ambiguity. When audiovisual, visual, or textual streams are missing, degraded, or semantically conflicting, uniform fusion and unconstrained token gates can propagate misleading content across iterative denoising steps. We propose CONCORD-Rec, a reliability-conditioned discrete diffusion recommender supervised by counterfactual catalog-partition utility. Given a fixed tokenizer and a partially masked SID state, CONCORD-Rec evaluates out-of-fold teacher coalitions over the exact feasible catalog support and aggregates item posteriors across the target code partition at each unresolved position. Rather than distilling raw token probabilities directly, the resulting counterfactual loss differences supervise an availability-masked tokenwise gate that allocates bounded modality residuals around a behavioral denoiser while retaining an exact behavioral fallback slot. We establish theoretical fixed-support ranking stability bounds under gate perturbations and evaluate position-level and code-conditional allocation across audiovisual and image-text recommendation benchmarks under controlled missingness, signal degradation, and semantic conflict. Across 20 source-compatible protocol-dataset rows, CONCORD-Rec improves NDCG by 4%; under matched 30% conflict it improves NDCG by 14.63%. The full-vs-uniform-T=16 sweep improves quality by 2.00-3.58% while reducing p95 latency by up to 86.89%.
7. 【2611.23702】KnowMe Paper Radar: Lightweight Discovery of Promising Papers in the Age of Autoresearch
链接:https://arxiv.org/abs/2611.23702
作者:Zhuo Chen,Jian Wang,Jinjie Gu,Lijun Wu
类目:Artificial Intelligence (cs.AI); Information Retrieval (cs.IR)
关键词:topically similar papers, methodological guidance, similar papers makes, KnowMe Paper Radar, growing volume
备注: Ongoing work. 67 pages with 34 figures and 37 tables. Project website: [this https URL](https://hackerchenzhuo.github.io/knowme-research-ai/en/) GitHub: [this https URL](https://github.com/hackerchenzhuo/knowme-research-ai)
点击查看摘要
Abstract:The growing volume of topically similar papers makes it difficult to identify new work that can offer useful research insights or methodological guidance. We present KnowMe Paper Radar, a lightweight discovery tool that combines multiple sources of evidence into personalized reading priorities. Users specify topics, priorities, exclusions and dates; the system brings together topical relevance, recency, author research histories, collaboration paths and direction-specific institutional context. By exposing each score component and its supporting sources, KnowMe helps readers select promising papers for their current questions and discover relevant work beyond familiar authors. Automatic acquisition, version merging and graph-cache reuse keep this workflow convenient without large language model (LLM) inference during ordinary use. Experiments on 174,640 acquired records show more than threefold faster repeated context preparation, while cached abstracts improve normalized discounted cumulative gain at rank 10 (NDCG@10) from 0.412 to 0.598 on expert-judged retrieval tasks. Complementary analyses examine how the same accumulated records support semantic retrieval, research-history modeling and literature-grounded planning. A browser interface and a structured command-line interface (CLI) make these capabilities available to both researchers and research agents, positioning KnowMe as a practical upstream tool for literature-informed recursive self-improvement (RSI) workflows.
8. 【2611.23606】VIBRA: Backward-Compatible Cross-Modal Retrieval with Consensus Barycenters
链接:https://arxiv.org/abs/2611.23606
作者:Shubing Yang,Xutong Liu,Dongfang Zhao
类目:Computer Vision and Pattern Recognition (cs.CV); Information Retrieval (cs.IR)
关键词:Backward-compatible retrieval enables, cross-modal retrieval system, Backward-compatible retrieval, retrieval enables, cross-modal retrieval
备注:
点击查看摘要
Abstract:Backward-compatible retrieval enables a new encoder to query the gallery that an old model has already encoded, so that a cross-modal retrieval system can be upgraded without re-encoding billions of items. Existing compatible methods either fit the old query and stay near the accuracy of the old model, or run the old and the new encoder for every query. We propose VIBRA, which matches relevance on the old gallery, corrects the errors of a single model with the barycenter of several views, which is itself a query in the old space, and trains the new encoder to inherit this consensus, so that models absent at deployment still shape every query. We show that the barycenter gains margin over the average of its views exactly when their hardest irrelevant items differ, and we characterize the share of this gain that one encoder can reach. On five benchmarks with five upgrades among six CLIP, MetaCLIP and SigLIP2 backbones, VIBRA is the best compatible method in all 50 settings, improving Recall@1 by up to 5.6 points over the strongest baseline and by up to 18.0 points over the old model, while adding less than 1 ms to the latency of the new encoder.
9. 【2611.23537】From Docket Minutes to Case Trajectories: Building Analysis-Ready Criminal Court Data
链接:https://arxiv.org/abs/2611.23537
作者:Erfan Hosseini Sereshgi,Kory Rosen,Jaya Dhariwal,Nicholas Mattei,Aron Culotta
类目:Information Retrieval (cs.IR)
关键词:Criminal court records, provide public oversight, Orleans criminal court, criminal court proceedings, Criminal court
备注: This paper has been accepted at IEEE Big Data 2026
点击查看摘要
Abstract:Criminal court records are often fragmented, noisy, and siloed, making it challenging for both legal experts and community stakeholders to access and analyze court data to identify trends and provide public oversight. In this paper, we introduce a system for data ingestion, harmonization, and dashboard visualization designed for New Orleans criminal court proceedings data, aimed at both expert and lay audiences. Our system collects and harmonizes disparate data, generates case trajectories, and provides intuitive visualization techniques based on community feedback to analyze demographics, temporal trends, and specific case events. We demonstrate the system through an exploratory analysis of recent short-term trends and policy impacts within New Orleans, illustrating how it can support the investigation of anomalies, policy changes, and procedural irregularities. We also present feedback from a small formative study of community stakeholders who have helped co-design and use the system, suggesting that it may improve access to and interpretation of local court data.
10. 【2611.23454】LusoClin: A Synthetic Clinical Note Dataset for Medical Information Retrieval in European Portuguese
链接:https://arxiv.org/abs/2611.23454
作者:Carla Teixeira Lopes,Daniel Félix
类目:Artificial Intelligence (cs.AI); Information Retrieval (cs.IR)
关键词:patient cohort identification, Access to large-scale, European Portuguese, advancing medical information, including clinical note
备注:
点击查看摘要
Abstract:Access to large-scale clinical text collections is crucial for advancing medical information retrieval research, including clinical note search, patient cohort identification, and domain-adapted retrieval models. However, real clinical notes are rarely available due to privacy regulations, which particularly limits progress for low-resource languages such as European Portuguese. We present LusoClin, a publicly released dataset of synthetic clinical notes in European Portuguese, created to support privacy-preserving research on clinical text retrieval. LusoClin was generated by pre-processing real clinical data, fine-tuning the open-source Llama 3.2 (3B) instruction-tuned model using Quantized Low-Rank Adaptation, and applying prompt-based clinical note generation. To facilitate semantic indexing and clinical search, the notes are automatically annotated with clinical entities using a BERT-based model. We evaluate LusoClin in terms of utility, fidelity, diversity, and privacy using automatic metrics and human evaluation. Results indicate that the dataset captures clinically relevant language patterns and can support downstream IR applications such as clinical document retrieval and domain-specific model adaptation, although annotators can often distinguish synthetic from real notes and some medical inconsistencies persist.
11. 【2611.23359】Prioritize, Don't Decide: What We Learned Auditing a Deployed Human-in-the-Loop CV Sorter
链接:https://arxiv.org/abs/2611.23359
作者:Hadi Mohammadi,Jeroen Sparla,Alexandros Paliouras,Georg Krempl,Robert A. Bagheri
类目:Computation and Language (cs.CL); Information Retrieval (cs.IR); Machine Learning (cs.LG)
关键词:screening round consumes, single academic vacancy, Priority Sorter deployed, hundreds of applications, draw hundreds
备注: 11 pages, 4 figures, 6 tables. Under review at the EACL 2027 Industry Track
点击查看摘要
Abstract:A single academic vacancy can draw hundreds of applications, and the first screening round consumes most of a committee's time. We describe a CV Priority Sorter deployed at a national platform for academic vacancies and integrated with its vacancy database. It proposes a reading order for a pile of CVs and leaves every decision to the committee, pairing a frozen multilingual transformer encoder with a per-session linear model that retrains after each label and is discarded when the session ends. It works: on a pool of 300 synthetic CVs, a committee reading in the tool's order reaches 95% of the relevant candidates after half the pile, where an arbitrary order needs almost all of it. This paper is about what we found when we audited that claim, rather than about the system itself. Three findings changed the product. The pool turned out to be sortable by document length alone, which makes precision-style metrics on it uninformative; regenerating it under a shared length budget removes that shortcut and leaves the reading-depth result standing. The four-level agreement score such systems report, ours included, is a binary probability cut at fixed thresholds, so it follows the classifier's confidence rather than its ranking and falls as labels accumulate. And uncertainty sampling, the obvious way to implement a "suggest the next CV" button, leaves model quality intact but raises the committee's total reading load above the no-label baseline, because the CVs it asks about are the ones that would otherwise be skipped. A fourth result shows where to look next: a hosted LLM embedding model lifts zero-label ordinal discrimination from 0.66 to 0.80 under length control, then stops mattering once the session model has two dozen labels; a local cross-encoder matches that cold-start gain without sending data out. We report the fixes each finding forced.
12. 【2611.23236】PGPO: Prototype-Guided Progressive Obfuscation for Privacy-Preserving LLM-Enhanced Recommendation
链接:https://arxiv.org/abs/2611.23236
作者:Shengxiang Lin,Jiajie Su,Pengyang Zhou,Xiang Chen,Xiaolin Zheng,Feng Tian,Chaochao Chen
类目:Information Retrieval (cs.IR); Computation and Language (cs.CL); Cryptography and Security (cs.CR)
关键词:LLM-enhanced recommender systems, recommender systems encode, leak sensitive textual, sensitive textual attributes, systems encode item
备注: Accepted to EMNLP 2026
点击查看摘要
Abstract:LLM-enhanced recommender systems encode item side information into semantic embeddings, but these embeddings may leak sensitive textual attributes through inversion attacks. Existing input-side defenses rely on trusted collection or local token perturbation, often disrupting the inter-item semantic topology needed for recommendation. To address it, we introduce relational fidelity as the key objective, requiring semantic shift for privacy while preserving relative item relations for utility. Based on this principle, we propose prototype-guided progressive obfuscation (PGPO), which learns topology-aware obfuscation anchors for high-influence prototype words and propagates them through a static semantic graph. PGPO is optimized with GRPO using individual dissimilarity, structural consistency, and semantic constraint rewards. Experiments on two benchmarks show strong inversion defense with near-upper-bound recommendation performance. The code is available at this https URL.
13. 【2611.24702】ask-Oriented Network Ranging via Joint Graph and Schedule Design
链接:https://arxiv.org/abs/2611.24702
作者:Zijian Zhang,Yuan Shen
类目:ignal Processing (eess.SP); Information Retrieval (cs.IR); Information Theory (cs.IT); Networking and Internet Architecture (cs.NI); Systems and Control (eess.SY)
关键词:Measuring inter-node distances, acquiring node positions, Measuring inter-node, fully connected graph, network localization
备注: 16 pages, 14 figures, with appendices
点击查看摘要
Abstract:Measuring inter-node distances is the basis for network localization, whereas acquiring node positions does not require a fully connected graph of distances. Existing complete-ranging protocols for network localization require every node pair to exchange signals, which prolongs the ranging cycle. In this paper, we propose sparse signal-multiplexing network ranging (S-SM-NR), a task-oriented framework that adapts the transmit-listen schedule to the ranging task required by network localization. First, we establish the S-SM-NR framework by formulating the joint design of the task graph and the transmit-listen schedule. Then, we analyze the minimum round count required by S-SM-NR and establish a connectivity condition for network localization. Subsequently, we develop a closed-form joint design that constructs the sparse task graph and the balanced transmit-listen schedule. In particular, the joint design can achieve the minimum round count under the complete physical connectivity. Finally, simulations show that the proposed S-SM-NR scheme reduces the occupied-round count and improves the localization accuracy by shortening the ranging cycle.
计算机视觉
1. 【2611.24941】UltraTex: Aligning Physical Knowledge for High-Fidelity PBR Texture Generation
链接:https://arxiv.org/abs/2611.24941
作者:Jingzhi Bao,Xiaohe Ma,Hong Li,Yuanming Hu,Xiaoguang Han
类目:Computer Vision and Pattern Recognition (cs.CV)
关键词:Generating physically based, Generating physically, observed appearance entangles, entangles material identity, physically based rendering
备注: Accepted to ACM Transactions on Graphics (SIGGRAPH Asia 2026). Project page: [this https URL](https://ultratex.vercel.app/)
点击查看摘要
Abstract:Generating physically based rendering (PBR) texture maps for a 3D mesh from a single reference image is highly ambiguous, as the observed appearance entangles material identity, surface reflectance, and illumination. While current generative methods produce realistic albedo maps, they still struggle to infer accurate metallic and roughness attributes for relighting. We present UltraTex, a cascaded multi-view framework that moves beyond appearance matching by integrating physical knowledge into multi-view diffusion-based PBR generation. UltraTex first jointly predicts albedo and world-space shading normals to generate more detailed albedo maps, and then predicts metallic-roughness (MR) maps conditioned on the generated albedo, reference image, and material semantic priors. To inject material semantics, a lightweight adapter transforms hidden-layer features of a vision-language model (VLM) into intrinsic material representation tokens, providing part-level knowledge anchors for MR estimation. To model material-light interaction, photometric representation alignment guides intermediate diffusion features toward ground-truth material renderings under controlled lighting. This makes metallic and roughness variations observable through highlight intensity, specular sharpness, and local reflectance contrast, providing stronger guidance for fine-grained material prediction without explicit lighting supervision. Extensive experiments show that UltraTex generates high-quality textures and outperforms existing open-source and commercial methods in visual fidelity, material accuracy, and physical plausibility.
2. 【2611.24940】$Z$-sight: Unifying Vision-Language-Action and Latent World Modeling
链接:https://arxiv.org/abs/2611.24940
作者:Junjie Ye,Yiqi Zhao,Muchen Xu,Celina Shiyu Wang,Hao Dong,Ruohai Ge,Rong Xue,Benjamin J Kolligs,Aditya Dutt,Sasanka Nagavalli,Nassim A. Samad,Gaurav S. Sukhatme,Jyotirmoy V. Deshmukh,Yue Wang,Yuliang Guo
类目:Robotics (cs.RO); Computer Vision and Pattern Recognition (cs.CV)
关键词:latent world modeling, generative latent world, unified robot foundation, robot foundation model, generative latent
备注: Project page: [this https URL](https://junjieye.com/Z-sight/)
点击查看摘要
Abstract:We introduce $Z$-sight, a unified robot foundation model that couples vision-language-action (VLA) and generative latent world modeling. Video-based world action models (WAMs) show the value of joint future-and-action generation for control, yet inherit the heavy cost of video generators. Recent latent-space policies avoid pixel generation, yet typically align intermediate features or rely on frozen target encoders, decoupling future prediction from the VLM's native feature space. $Z$-sight instead predicts the future directly in the visual token space in which its VLM perceives the present. Coupling a foresight expert and a continuous action expert with the VLM in a mixture-of-transformers, $Z$-sight jointly denoises future spatial tokens and actions at inference without pixel reconstruction or external encoders. Trained on human video and robot trajectories, $Z$-sight achieves 81.9% success on SIMPLE (+9.6 points over the strongest baseline) and 73.3% across six Unitree G1 tasks (+20.0 points) at 110 ms per chunk, while remaining competitive on tabletop benchmarks.
3. 【2611.24917】Unified Law for Visual Tasks (The Four Laws of Computer Vision)
链接:https://arxiv.org/abs/2611.24917
作者:Sen Fang,Yalin Feng,Yancheng Zhang,Yanxin Zhang,Zonglin Lyu,Hongbin Zhong,Liwei Che,Rui Wu,Chen Chen,Dimitris N. Metaxas
类目:Computer Vision and Pattern Recognition (cs.CV)
关键词:Computer Vision, World Snapshot Models, World Snapshot, call World Snapshots, World Snapshot generation
备注: Preview version (main paper only); This work began in December 2024. Project page at [this https URL](https://world-snapshot.github.io/WSMs;) GitHub org at [this https URL](https://github.com/World-Snapshot;) Hugging Face org at [this https URL](https://huggingface.co/World-Snapshot;) This is an empirical article that attempts to unify thousands of visual tasks through a single methodology. Patent pending (inventors: SF and DM)
点击查看摘要
Abstract:In this paper, we propose a Unified Law for Visual Tasks, which could be further divided into Four Laws of Computer Vision, with the goal of enabling a single framework or method to uniformly train on and perform nearly all visual tasks. Our starting point is an inspiration and hypothesis: if the world information is encoded into image, which we call World Snapshots, then image processing is world processing, and image generation is World Snapshot generation. Furthermore, if all visual and visualization tasks can be reformulated as generation tasks, then a unified method, World Snapshot Models (WSMs), can perform the vast majority of visual tasks, much as large language models have largely unified natural language processing. This paper validates these laws across more than 1,000 trained and evaluated visual tasks (defined by distinct input-output mappings) and presents a unified training framework for future visual-task research and applications.
4. 【2611.24915】PrismRestore: A Compact Restoration Toolbox for Agentic Image Restoration
链接:https://arxiv.org/abs/2611.24915
作者:Zhaocheng Yu,Kui Jiang,Jing Cao,Yiang Chen,Xiaocheng Feng,Junjun Jiang,Xianming Liu
类目:Computer Vision and Pattern Recognition (cs.CV)
关键词:handle mixed degradations, requires storing multiple, multiple restoration networks, storing multiple restoration, mixed degradations
备注:
点击查看摘要
Abstract:Agentic image restoration composes operations to handle mixed degradations, yet its toolbox often requires storing multiple restoration networks. We propose PrismRestore, which constructs a compact bank of callable restoration actions from a single frozen all-in-one model. Our approach builds on task-dependent yet overlapping channel responses in the shared model. An offline task-prior search uses activation and gradient statistics to group channels with similar response profiles, then searches task-specific modulation coefficients over non-representative channels while leaving representative channels unmodulated. The resulting prior bank enables explicit action selection and composition without additional restoration backbones or task-specific adapter modules. A state-conditioned policy learns to compose these actions using Group Relative Policy Optimization. Its objective separately normalizes fidelity and perceptual feedback within groups of trajectories sampled for the same input, and combines terminal evaluation with return-to-go derived from stepwise quality changes. Experiments on synthetic composite-degradation and real-world multi-task benchmarks show improvements over the evaluated all-in-one baselines and restoration quality competitive with multi-model agents. PrismRestore achieves these results with a single restoration backbone and faster inference than the evaluated agentic baselines.
5. 【2611.24904】ShadowTransfer: A Geographic Transfer Benchmark for Overhead Shadow Detection
链接:https://arxiv.org/abs/2611.24904
作者:Shagun Mittal,Raymond A. Yeh,Satish Ukkusuri
类目:Computer Vision and Pattern Recognition (cs.CV)
关键词:aerial imagery feeds, imagery feeds dense-prediction, feeds dense-prediction pipelines, Overhead aerial imagery, aerial imagery
备注: Accepted at NeurIPS 2026, Track on Evaluations and Datasets. 53 pages. Project page: [this https URL](https://shagun8.github.io/shadowtransfer/) . Code: [this https URL](https://github.com/shagun8/shadowtransfer) . Data: [this https URL](https://huggingface.co/datasets/shagunmittal/ShadowTransfer) (DOI [https://doi.org/10.57967/hf/11483](https://doi.org/10.57967/hf/11483) )
点击查看摘要
Abstract:Overhead aerial imagery feeds dense-prediction pipelines that are routinely trained in one geographic region and deployed in another, yet no benchmark measures how shadow detection models transfer across regions. Existing overhead shadow datasets either lack a geographic transfer protocol or rely on automatically generated labels. We introduce ShadowTransfer, a benchmark of 4,500 human-verified shadow masks across three U.S. cities at two native resolutions, paired with a leave-one-city-out evaluation protocol. Under vanilla training, holding out the test city costs 3.5 to 7.8 mIoU across four architectures, the gap remains significant when the training set grows from 225 to 900 images, and six of eight adaptation methods degrade more than vanilla training on average. We propose a three-probe framework that diagnoses where and how the transfer breaks: the failure operates in feature space, beyond the reach of pixel-histogram matching, shows up as a confidence collapse on shadow pixels while background confidence stays intact, and is recoverable by retraining different architectural components depending on the model: encoder for some, decoder for others. These findings replicate on INRIA building footprints, suggesting they describe geographic transfer beyond shadows. Guided by the diagnosis, we build Spectral Information Bottleneck (SIB), a plug-in module with one component per finding. Across the 9 (architecture, held-out city) cells, SIB has the smallest mean gap to the within-city upper bound of the ten methods we evaluate (6.42 mIoU), level with FDA. The benchmark, LOCO protocol, code, and trained models are released to support reproducible evaluation of geographic transfer in overhead dense prediction.
6. 【2611.24864】NereidVLA: Learning to Understand and Navigate the Underwater World
链接:https://arxiv.org/abs/2611.24864
作者:Chenjie Zhao,Zhiliang Zhu,Quan Chen,Yuquan Xue,Xingxue Shi,Puyue Wang,Zhiqiang Zhao,Kangye Zheng,Boyang Xu,Mingwei Lin,Dejun Li
类目:Computer Vision and Pattern Recognition (cs.CV); Robotics (cs.RO)
关键词:requires translating visual, vision-language datasets rarely, datasets rarely include, rarely include paired, translating visual spatial
备注: 11 pages, 4 figures
点击查看摘要
Abstract:Underwater navigation requires translating visual spatial cues into motion, yet marine vision-language datasets rarely include paired robot trajectories. We present NereidVLA, a vision-language-action model that transfers marine spatial knowledge to language-conditioned 3D navigation. We first train Sea2Know on approximately two million marine examples to learn target grounding, spatial relations, and scene structure without paired robot-motion supervision. We then adapt it using expert trajectories executed under simulated underwater vehicle dynamics. Given RGB histories and a language instruction, NereidVLA predicts discrete motion tokens that decode into local 3D trajectories for closed-loop tracking. Evaluation combines an independent six-dimensional underwater navigation understanding benchmark with offline prediction and closed-loop navigation. Under matched navigation training and execution settings, marine spatial pretraining increases simulation success from 72.4% to 84.4%. Without fine-tuning on the real test environment, NereidVLA achieves 78.75% success in laboratory-pool robot trials, compared with 47.50% for InternVLA-N1. These results show that marine spatial supervision can support downstream motion learning and physical underwater navigation.
7. 【2611.24862】Gaspard: Expressive unified speech and gesture synthesis with continuous autoregressive flow-matching and decoupled classifier-free guidance
链接:https://arxiv.org/abs/2611.24862
作者:Téo Guichoux,Théodor Lemerle,Catherine Pelachaud,Nicolas Obin
类目:Computer Vision and Pattern Recognition (cs.CV)
关键词:Human communication involves, Human communication, coordinated multimodal signals, communication involves highly, involves highly coordinated
备注: Preprint - under review
点击查看摘要
Abstract:Human communication involves highly coordinated multimodal signals such as speech and co-speech gestures. Existing approaches for generating both modalities commonly follow a cascaded pipeline, where speech is generated first and subsequently used to drive gesture generation; speech generation is therefore independent of gestures. We introduce Gaspard, a unified speech--gesture generation model based on continuous autoregressive generation with next-step flow-matching predictions, and trained on Seamless Interaction, a large-scale multimodal dataset. The heterogeneous temporal properties of speech and gesture motivate modality-specific prediction heads and decoupled classifier-free guidance over the two generation streams. This guidance rebalances the influence of speech and gesture at inference time, improving expressivity while mitigating dynamics collapse in continuous autoregressive gesture generation. Objective and perceptual evaluations show that Gaspard is a strong gesture generation model against unimodal baselines and that it produces highly synchronous speech and gestures compared to a matched cascaded system.
8. 【2611.24840】Does the Action Matter? Probing Action-Conditioned Semantic Forecasting as a World Model for Autonomous Driving
链接:https://arxiv.org/abs/2611.24840
作者:Binay Kumar Singh,Niels Da Vitoria Lobo
类目:Computer Vision and Pattern Recognition (cs.CV); Artificial Intelligence (cs.AI); Machine Learning (cs.LG); Robotics (cs.RO)
关键词:supporting counterfactual reasoning, action, supporting counterfactual, scene evolves, counterfactual reasoning
备注: 8 pages, 3 figures, 6 tables. Accepted at the WAM-AD Workshop, WACV 2027. Code: [this https URL](https://github.com/binayksingh/does-action-matter)
点击查看摘要
Abstract:World models are increasingly used in autonomous driving to predict how a scene evolves under the ego vehicle's actions, supporting counterfactual reasoning and model-based planning. However, existing evaluations focus on prediction fidelity and planning performance, and rarely verify that the predictions depend on the conditioning action. In this paper, we study action dependence in action-conditioned semantic forecasting, which predicts future semantic segmentation from past segmentations and the ego action. We compare a forecaster with feature-wise and input-level action conditioning against an identical model without action input, and measure action dependence with test-time action interventions scored by standard segmentation metrics. The evaluation is validated in a synthetic environment with known action-dependent dynamics and applied to Cityscapes and BDD100K, both within each dataset and across the two vehicle platforms. Experiments show that the executed action improves mid-term mIoU by at most 2.5 points on Cityscapes and 0.6 points on BDD100K, whereas a wrong action reduces it by up to 23.2 and 7.0 points, below the model without action input. On real data, feature-wise conditioning depends 1.6 times more on the action than input-level conditioning at similar accuracy, whereas both depend on it equally in the synthetic environment, where the history does not determine the action. These results show that an accuracy gain indicates that a driving world model uses its action, but not how strongly its forecasts depend on it. Code is available at this https URL.
9. 【2611.24827】Synthesizing hyperspectral Data using generative Models to train spectral Unmixing Methods for low-cost Crop Residue Cover Mapping
链接:https://arxiv.org/abs/2611.24827
作者:Ege Artan,Shuo Chen,Andrew Peng,Sammuel Aldrich Karya,Kyaw Thiha
类目:Machine Learning (cs.LG); Computer Vision and Pattern Recognition (cs.CV); Signal Processing (eess.SP)
关键词:Space System Division, Field Imaging Nanosatellite, Toronto Aerospace Team, Aerospace Team Space, Team Space System
备注: 8 pages, 3 figures. Published in the XXV ISPRS Congress 2026
点击查看摘要
Abstract:The spectral range of the Field Imaging Nanosatellite for Crop residue Hyperspectral Mapping (FINCH), developed by the University of Toronto Aerospace Team's Space System Division, poses significant challenges for hyperspectral unmixing to determine crop residue cover fractional abundances. The severe lack of standard indices necessitates the use of complex, data-driven unmixing models. Complex unmixing models require dense, well-distributed manifolds, whereas ground-truth datasets are sparse and expensive. To better leverage the information content in existing ground-truth datasets, this study presents a framework that decouples interpolation and abundance-mapping estimation via an intermediary conditional data generator, contrasting with traditional single-model unmixing pipelines. This decoupling enables the inclusion of physical prior knowledge about spectra, which is not natively accessible to unmixing models, thereby artificially expanding the dataset to serve as a Monte Carlo approximation of the population risk. To test this hypothesis, we have proposed two generator models: a 1D Conditional U-Net with Conformer Layers (GD-Streamline) and a Dual-Path Transformer Conditional Variational AutoEncoder (TCVAE), and two unmixing models: Multi-Layered Perceptrons (MLP) and Fourier Neural Operators (FNO). The results of the study indicate a need for rigorous integration and introduction of spectral priors within the scope of the proposed decoupling framework to prevent domain and mode collapse and hallucination by the generator models; otherwise, this leads to falsely approximated data manifolds, ultimately resulting in unsatisfactory and out-of-distribution unmixing performance.
10. 【2611.24806】ReCoCo: From Patches to Compositions for Visual Representation Learning
链接:https://arxiv.org/abs/2611.24806
作者:Juan Gutiérrez,Saman Zadtootaghaj,José Luis Blanco-Murillo,Sara Atito
类目:Computer Vision and Pattern Recognition (cs.CV)
关键词:features remains challenging, Self-supervised learning, produced powerful visual, remains challenging, dense features remains
备注:
点击查看摘要
Abstract:Self-supervised learning has produced powerful visual representations, yet learning strong dense features remains challenging. While consistency between augmented views provides strong image-level supervision, extending this principle to local visual content is less straightforward. Existing approaches often match individual patches across views or identify coherent regions, introducing uncertainties in correspondence estimation and assumptions about how visual content should be grouped. We introduce ReCoCo (Regional Composition Consistency), a principled approach that extends cross-view consistency to the regional level without requiring such correspondences or semantic grouping. ReCoCo represents the content shared between augmented views through compositions of their patch-level predictions over learned visual prototypes. By aligning these compositions, the model learns from regions containing multiple visual concepts without requiring them to be separated beforehand. ReCoCo integrates naturally with teacher-student self-distillation and introduces no additional learned parameters. Extensive experiments demonstrate substantial improvements in frozen-feature segmentation, clustering, and regional retrieval while maintaining competitive image-level recognition, alleviating the trade-off between dense and global representation quality.
11. 【2611.24805】Edge-Efficient Joint RAW Image Compression and Restoration for Smart Wearable Devices
链接:https://arxiv.org/abs/2611.24805
作者:Yunong Sun,Sicheng Chen,Fei Xia
类目:Computer Vision and Pattern Recognition (cs.CV)
关键词:Smart wearable devices, augmented reality glasses, wearable devices, increasingly rely, augmented reality
备注:
点击查看摘要
Abstract:Smart wearable devices, such as augmented reality glasses, increasingly rely on cameras to understand and interact with their surroundings. However, their compact sensors often capture noisy RAW images, while limited compute and power make high-quality on-device restoration difficult. This creates an emerging need to reduce both local computation and transmission cost without sacrificing image quality. Two common solutions are to offload restoration or to compress the raw data before transmission. The first makes RAW transmission costly; the second reduces the payload but can spend bitrate preserving sensor noise. We address this fundamental mismatch with an asymmetric framework for joint RAW compression and restoration that directly optimizes the transmitted representation for clean image reconstruction. A lightweight on-device encoder compresses noisy RAW measurements into a bitstream, while a higher-capacity backend decoder jointly denoises and demosaics the encoded representation. We use knowledge distillation to train a smaller encoder for tighter compute budgets. Experiments on real and synthetic data show improved rate-restoration performance over strong cascaded compression-restoration pipelines and learned compression baselines. On SIDD, our method improves PSNR by more than 1.5 dB over cascaded baselines at approximately 0.25 bits per pixel. We further validate the method on real 12 MP RAW images captured with commercial smart glasses and measure sender-side latency on both smartphone and smart-glasses hardware. The distilled encoder achieves a 27.6$\times$ NPU inference speedup over on-device NAFNet for 256$\times$256 output patches. The code is available at this https URL
12. 【2611.24795】Ultimate3D: Teaching Multimodal LLMs 3D Camera-Object Relations by Large-Scale Visual Instruction Data
链接:https://arxiv.org/abs/2611.24795
作者:Liu He,Yizhi Song,Ruiqi Xu,Baris Sarper Tezcan,Xiao Zeng,Albert Y.C. Chen,Lu Xia,Shashwat Verma,Sankalp Dayal,Min Sun,Cheng-Hao Kuo,Daniel Aliaga
类目:Computer Vision and Pattern Recognition (cs.CV)
关键词:Multimodal Large Language, accurately capturing camera-object, Multimodal Large, camera viewpoint, camera shots
备注: Accepted to Machine Vision and Applications 2027
点击查看摘要
Abstract:Multimodal Large Language Models (MLLMs) struggle with accurately capturing camera-object relations, especially for object orientation, camera viewpoint, and camera shots. This stems from the fact that existing MLLMs are trained on images with limited diverse camera-object relations and corresponding textual descriptions. To address this, we propose a synthetic generation pipeline to create large-scale 3D visual instruction datasets. Our framework takes 3D assets as input and uses rendering and diffusion-based image generation models to create photorealistic images preserving precise camera-object relations. Additionally, large language models (LLMs) are used to generate text prompts for guiding visual instruction tuning and controlling image generation. We create Ultimate3D, a large-scale 3D-grounded 2D visual instruction dataset. Ultimate3D consists of 240K VQAs with precise camera-object annotations, and corresponding benchmark. MLLMs fine-tuned on our proposed dataset outperform commercial models by a large margin, achieving an average accuracy improvement of 29.7% on camera-object relation recognition tasks. Our code, dataset, and benchmark will contribute to broad MLLM applications.
13. 【2611.24793】he Elephant in the Question: Unified Models Picture the Concepts Behind Their Answers
链接:https://arxiv.org/abs/2611.24793
作者:Tobias Braun,Marcus Rohrbach,Lorenzo Cavallaro
类目:Computer Vision and Pattern Recognition (cs.CV); Computation and Language (cs.CL); Machine Learning (cs.LG)
关键词:reasoning steps underlying, Visualizing intermediate representations, Visualizing intermediate, inspect the reasoning, reasoning steps
备注: 32 pages, including supplementary material. Code: [this https URL](https://github.com/multimodal-ai-lab/V-lens)
点击查看摘要
Abstract:Visualizing intermediate representations could help us inspect the reasoning steps underlying a model output. Unified multimodal models offer this opportunity by combining understanding and generation in one architecture. Their image generators can access representations formed while processing text. We introduce the V-lens, which uses this connection to visualize a chosen token's hidden state without training a decoder. Across four diffusion-based and two purely autoregressive unified models, the resulting images reveal key intermediate concepts that connect a question to its answer even when the concepts are never named explicitly. Controlled tests show how this content changes with context, token position, and depth. We then introduce lens tuning, which supervises selected hidden states with pictures through the V-lens. Across BAGEL, SenseNova-U1, and HunyuanImage-3.0, the rate of correct visualizations of pronoun referents rises from 2-11% to 86-94%. Surprisingly, such visual supervision also improves text answers without supervising the answer tokens directly. Code is available at this https URL.
14. 【2611.24792】raining-free motion statistics match self-supervised behavior encoders
链接:https://arxiv.org/abs/2611.24792
作者:Ziwen Chen
类目:Computer Vision and Pattern Recognition (cs.CV); Machine Learning (cs.LG)
关键词:Self-supervised encoders, Self-supervised, encoders, animal behavior, hBehaveMAE
备注: 10 pages, 7 tables, 1 figure. Code will be released with the conference version
点击查看摘要
Abstract:Self-supervised encoders for animal behavior are often trained on, or fed, hand-built movement features. We ask whether they beat those ingredients when compared fairly. A training-free representation, made of generic per-frame measurements summarized over fixed time windows and reduced with PCA, is compared with retrained and released encoders (BAMS, hBehaveMAE) under a preregistered protocol: frozen representations of equal size and one shared classifier. On CalMS21 it reaches 85.8 macro F1 against 82.4 for the best encoder, ties with four labelled videos, and leads when the annotator changes or new behaviors must be learned from a few videos; the reading holds for encoders retrained with three seeds. On the MABe22 mouse triplets it scores 83.7 against 80.3 for the released hBehaveMAE (gap +3.4, 95% interval +2.1 to +4.7). On a second species, the MABe22 fly groups, the same recipe scores 51.6 mean F1 over 50 tasks against 48.2 for BAMS, the best published method. On Shot7M2, the synthetic basketball benchmark built by hBehaveMAE's authors, it scores 68.8 against 64.7 for hBehaveMAE, the best method in that paper, though hBehaveMAE remains better on the shortest movements. Short time windows carry most of the result. Raw keypoints in each body's own frame, summarized over the same windows, already match or beat the encoders on both mouse datasets and on Shot7M2; on the fly groups they fall short, and the measurements add 5.3 points. On these benchmarks, current behavior encoders have yet to beat the hand-built features they learn from, and such baselines should be reported alongside them.
15. 【2611.24785】Read Together, Answer Apart: How Earlier Questions Help Vision-Language Models Look
链接:https://arxiv.org/abs/2611.24785
作者:Xinyi Guo,Kaile Huang,Xitong Ling,Mingyi He,Chende Zheng,Jianyu Xu,Chao He,Dongqing Song
类目:Computer Vision and Pattern Recognition (cs.CV)
关键词:existing evaluations report, evaluations report conflicting, individual answers change, Densely Captioned Images, report conflicting accuracy
备注:
点击查看摘要
Abstract:Vision-language models (VLMs) are often asked several questions about the same image in a single prompt (co-asking), yet existing evaluations report conflicting accuracy gains and hallucinations at the prompt level without tracking how individual answers change. By pairing each question's single-question and co-asked answers across 22 open-weight VLMs (1B-35B), we find that erratic net shifts reflect two opposing effects: on Densely Captioned Images (DCI), every model repairs 15.4%-29.8% of its single-question errors under co-asking, while the share of correct answers turned wrong falls from 30.8% on weaker models to 4.7% on stronger ones. Across all 14 models supporting custom attention masks, we show through attention readouts and layer-window interventions that earlier same-image questions read visual information into their question tokens in the early-to-middle layers during prefill, helping later questions concentrate image attention onto their queried targets. This visual help accumulates across the scene even across non-overlapping regions, whereas blocking earlier questions from attending to the image lowers both target localization and accuracy in all 14 models. While cross-question help is established during shared question reading, part of the co-asking damage enters later during answer decoding when the decoder attends to other question stems. We therefore introduce Read Together, Answer Apart, a training-free decoding mask that keeps shared prefill intact and isolates each answer from other stems, improving co-asking accuracy on 12 of 14 models on DCI (up to +4.71 percentage points, closing 70% of the pooled gap to asking alone) and on 10, 8, and 5 of the 14 models on WebUI, FinTabNet, and GQA (up to +11.4, +5.3, and +1.1 points).
16. 【2611.24778】Counterfactual World Models: JEPAs That Reason About "What If"
链接:https://arxiv.org/abs/2611.24778
作者:Randall Balestriero,Yann LeCun,Kerda Varaku
类目:Machine Learning (cs.LG); Artificial Intelligence (cs.AI); Computer Vision and Pattern Recognition (cs.CV); Robotics (cs.RO); Machine Learning (stat.ML)
关键词:World models operate, World models, fully observe, World, world model
备注:
点击查看摘要
Abstract:World models operate in a world they cannot fully observe. When a hidden disturbance, such as a current pushing a boat, changes the outcome of an action, a counterfactual question follows: what would have happened under a different action, subject to the same disturbance? Its answer separates bad decisions from bad luck. We prove that world models trained on factual transitions cannot answer it on their own, whatever their data or capacity. For many physical systems, the observed outcome and a simple property of the dynamics suffice: natura non facit saltus, nature makes no leaps. When actions and disturbances act as separate forces, their interaction is second-order in time, so the counterfactual outcome is, to first order, the observed outcome plus the difference between the world model's predictions under the alternative and factual actions. This yields a counterfactual world model with no counterfactual data, extra parameters, or retraining. For transitions of duration $h$, we prove an $O(h^2)$ bound on the counterfactual error relative to the disturbance variance, establish a matching minimax lower bound from unpaired factual data, and characterize failures at discontinuities, coordinate folds, and disturbances that scale the command. We validate these predictions from pixels with JEPA world models on frozen DINOv2 and masked autoencoder (MAE) encoders. On four rendered simulators, our approach improves counterfactual $R^2$ from $38$--$46\%$ to $86$--$96\%$ with the same world models; on a fifth, an underwater vehicle, it reaches $90$--$94\%$ with sixteen times more factual data. Four further frozen encoders, DINOv3, SigLIP~2, CLIP and EVA-02, give $81$--$95\%$ on the same simulators. In hindsight planning, it finds the action that would have reached the goal in $35$--$69\%$ of events, against $14$--$23\%$ for an oracle planner that knows the simulator but not the outcome.
17. 【2611.24762】A$^3$Bench: Compositional Reasoning over Aggressive Interactions in Videos
链接:https://arxiv.org/abs/2611.24762
作者:Austin C Baker,Raiyaan Abdullah,Daniel Zeev Yoffe,Shruti Vyas,Yogesh S Rawat
类目:Computer Vision and Pattern Recognition (cs.CV)
关键词:Recent Multimodal Large, Multimodal Large Language, Large Language Models, Large Language, Recent Multimodal
备注:
点击查看摘要
Abstract:Recent Multimodal Large Language Models (MLLMs) have achieved strong performance on general video understanding benchmarks, yet their ability to reason about participant roles in aggressive interactions remains largely unexplored. Existing violence and crime datasets primarily evaluate whether aggression occurs, without testing whether models can identify who performed the action, who received it, and how participants are related within the scene. To address this gap, we introduce A$^3$Bench, a benchmark to study aggressive action and association in videos. The benchmark evaluates whether models can jointly recognize aggressive actions, distinguish aggressors from victims and bystanders, and correctly bind actions to participants under challenging distractor settings. We benchmark 17 multimodal models and find that they struggle substantially on fine-grained aggressive-scene understanding. Performance degrades consistently as reasoning complexity increases, with models frequently confusing aggressors with victims or bystanders, revealing a systematic failure in role-action association rather than coarse aggression recognition alone. We further observe that increasing model scale or generic reasoning models experience limited improvement. Finally, we explore a simple role-graph prompting strategy that encourages structured reasoning over participant interactions, leading to improvements in compositional reasoning. Our findings highlight that reliable aggressive-scene understanding remains a major challenge for current MLLMs and motivate future research on role-aware video reasoning beyond coarse violence detection. Code and dataset: this https URL.
18. 【2611.24738】Jeveration: System One Models Now Generate
链接:https://arxiv.org/abs/2611.24738
作者:Yuhe Liu,Qi Li,Shihua Zhang,Xinchao Wang
类目:Computer Vision and Pattern Recognition (cs.CV)
关键词:Jev return decisions, Jev return, System, Jev, return decisions
备注:
点击查看摘要
Abstract:System One models such as Jev return decisions through fast, parallel prediction. In this paper, we introduce Jeveration, showing that the same decision paradigm can also generate high-quality images in a single forward pass. Jeveration builds on a frozen vision-language model, repurposing its visual-semantic knowledge for holistic image generation. The key challenge is to maintain global consistency. A single prompt admits many valid images, yet parallel predictions must converge on a coherent whole. We therefore condition all positions on a shared random state to coordinate parallel predictions, so that each draw yields a different but coherent image. Since the model tends to ignore this state under direct image supervision, we first learn a distribution over discrete image codes to guide single-pass generation. Beyond text-to-image generation, we further unify image-to-image generation and dense prediction within the same architecture, so that one checkpoint supports all three tasks, each in one forward pass. Experiments show that Jeveration generates a 512x512 image in 78 ms, up to 122x faster than existing unified models, while maintaining competitive image quality and dense prediction performance.
19. 【2611.24731】HandWorldBench: A Unified Benchmark for Hand-Centric World Models
链接:https://arxiv.org/abs/2611.24731
作者:Zhaolu Wang,Jiacheng Xu,Lihong Wang,Shuqing Wang,Jianjie Fang,Ziyou Wang,Xvyuan Liu,Rongze Tang,Yiding Ma,Xin Zhang,Jiawei Chen,Wei Wu,Chen Gao,Yong Li,Zhibo Chen
类目:Computer Vision and Pattern Recognition (cs.CV)
关键词:Recent advances, demanding setting, fine-grained embodied tasks, opening new possibilities, possibilities for fine-grained
备注:
点击查看摘要
Abstract:Recent advances in world models are opening new possibilities for fine-grained embodied tasks, with hand-object interaction offering a particularly demanding setting. Modeling these interactions involves capturing how hand actions change objects and how the resulting states persist over time. Yet systematic evaluation tailored to these hand-centric capabilities remains underdeveloped. To address this gap, we introduce HandWorldBench, a unified benchmark that systematically evaluates hand-centric world models across human hands, simulated dexterous hands, and real dexterous hands along six dimensions: visual quality, hand fidelity, appearance consistency, instruction following, physical consistency, and temporal coherence. To construct the benchmark, we develop a data curation pipeline that extracts hand-object interaction clips and screens them for quality, yielding 20,500 high-quality egocentric clips from over 600 hours of footage. We then manually select 500 clips covering seven task families for evaluation. Benchmarking eleven models under a unified protocol reveals that stronger hand-pose control does not consistently translate into coherent object-state evolution and persistence or robust cross-embodiment performance, with several specialized hand-control models underperforming general-purpose text/image-to-video generators on dexterous-hand tasks.
20. 【2611.24726】Generalization of Text Line Segmentation for HTR in Historical Documents
链接:https://arxiv.org/abs/2611.24726
作者:Gayan H. Pathirage,Stephan M. Unter,Simon Corbillé,Elisa H. Barney Smith
类目:Computer Vision and Pattern Recognition (cs.CV)
关键词:historical documents, paper we investigate, generalization capability, YOLO, RT-DETR
备注: This work is going to be published in the proceedings for the Historical Image Processing workshop 2026 (HIP 2026), connected to the International Conference of Document Analysis and Recognition (ICDAR)
点击查看摘要
Abstract:In this paper we investigate the generalization capability of state-of-the-art object detection models (Hi-SAM, YOLO, and RT-DETR) for the automatic segmentation of text lines in historical documents. Generalization is needed because it is not practical for museums and archives to produce large amounts of labeled samples for each new collection. We also evaluate the few-shot setting by fine-tuning the networks with only a small number of training samples per target dataset. To this end, we use nine different datasets that differ significantly in visual layout and language. Additionally, we discuss why mean average precision (mAP), a classic evaluation metric for object detection, is insufficient for assessing the quality of a text line segmentation model. Depending on the dataset, our CRNN models achieve character error rates (CER) ranging from 2.48% to 7.07% per page for the segmented text lines. This is sufficient for reading comprehension and simple text searches. Finally, we quantify the resource and time requirements of training and inference in different use case settings. This forms the basis for more informed decisions about model and hardware choices based on available resources.
21. 【2611.24723】Multi-Band Galaxy Novel-View Synthesis with Radiative Gaussian Fields
链接:https://arxiv.org/abs/2611.24723
作者:Yuzhe Wu,Jialin Zhu,Doğa Yılmaz,Daisuke Kawata,He Wang
类目:Computer Vision and Pattern Recognition (cs.CV)
关键词:view synthesis tool, Gaussian Splatting method, Gaussian Splatting, view synthesis, synthesis tool
备注: 10 pages, 3 figures
点击查看摘要
Abstract:We propose a novel view synthesis tool for galaxies based on Gaussian Splatting (GS). Unlike RGB data, where GS has achieved success, galaxy observations consist of multi-band signals from a massive number of bodies, contain complex patterns of cross-band interactions, and capture light propagation in highly heterogeneous media. As a result, the underlying physical process poses great challenges in model representation and optimization, which have not been investigated in existing research. To this end, we propose a physically inspired Gaussian Splatting method, GalaxyGS, which is equipped with physically meaningful components that capture both the energy emission mechanism of different astronomical bodies and the light propagation between them. In the absence of directly comparable baseline methods, we adopt a wide range of possible alternative approaches. Comparisons conclusively demonstrate that GalaxyGS is better in accuracy and physical fidelity across datasets and metrics.
22. 【2611.24719】DRIFT: Detachable Reasoning for Runtime-Flexible and Efficient End-to-End Driving
链接:https://arxiv.org/abs/2611.24719
作者:Emad Aghajanzadeh,Bethany Anne Wong,Leonhard Kunze,Moritz Thoma,Maximilian Forstenhäusler,Nael Fasfous,Manoj Rohit Vemparala,Pierpaolo Mori,Christos Anagnostopoulos,Daniel Mueller-Gritschneder,Ulf Schlichtmann
类目:Computer Vision and Pattern Recognition (cs.CV); Computation and Language (cs.CL); Machine Learning (cs.LG); Robotics (cs.RO)
关键词:demonstrated strong potential, providing scene description, integration of vision-language, potential by providing, autonomous driving
备注: 11 pages, 4 figures, 7 tables. Project page: [this https URL](https://drift-e2e.github.io/)
点击查看摘要
Abstract:The integration of vision-language models (VLMs) into end-to-end (E2E) autonomous driving has demonstrated strong potential by providing scene description and reasoning capabilities. However, VLMs are computationally expensive, and switching language reasoning on and off at runtime without hurting closed-loop performance has not been demonstrated, which hinders practical adoption. To address these issues, we propose DRIFT, an E2E autonomous driving framework that integrates a lightweight language model (LM) for visual question answering and action commentary. By efficiently projecting intermediate driving representations into the language embedding space, we demonstrate that a compact LM (150M parameters) can achieve strong scene description and action commentary. Furthermore, we introduce a runtime-flexible planning module that is trained on surrogate reasoning features distilled from the LM and can be conditioned on either the LM or the surrogate reasoning features at inference. This design enables bypassing the LM entirely when required by deployment constraints, without sacrificing driving performance. With only 255M parameters, DRIFT reaches up to a Driving Score (DS) of 97.03 and a Success Rate of 89.39% (mean over three runs) on Bench2Drive by interleaving its LM and surrogate reasoning at a chosen reasoning rate. This is on par with the strongest non-language models while being far more compact than existing VLM-based models. DRIFT further achieves competitive results on the real-world NAVSIM and AlpaSim benchmarks at significantly lower computational cost than existing VLM-based approaches. Project page: this https URL
23. 【2611.24706】Can Omni-modal LLMs Understand Spoken Intentions?
链接:https://arxiv.org/abs/2611.24706
作者:Zijie Xin,Tianyi Wang,Daoze Zhang,Jie Yang,Changli Tang,Qijie Wei,Fengyun Rao,Xirong Li
类目:Computer Vision and Pattern Recognition (cs.CV)
关键词:directly through voice, spoken requests directly, Omni-modal large language, large language models, Omni-modal large
备注: project page [this https URL](https://xxayt.github.io/O-IN-Bench)
点击查看摘要
Abstract:Omni-modal large language models (omni-modal LLMs) can respond to spoken requests directly through voice and video calls. Such requests rely on the scene shared by the user and the assistant, so the same utterance may call for different help or for no response. Most audio-visual benchmarks provide questions as separate text input, and interactive benchmarks mostly score a response as a whole. It thus remains unclear whether a model recognizes when the user is asking for help, and whether its reply meets the user's need once the request is recognized. We propose O-IN-Bench to evaluate how well omni-modal LLMs understand spoken intentions. A model watches a complete video with native speech and outputs one text response, or [SILENCE] when no response is needed. O-IN-Bench contains 1,000 Chinese and English samples in five categories and 16 subcategories. Contrastive pairs combine the same request with different scene conditions to test whether the response changes with the context. A sample-specific rubric judges language, intent, and content separately, and O-IN-Engine verifies each rubric against the video. Across 30 omni-modal model configurations, the best overall score is 59.6, achieved by Gemini3.7-Flash. Gemini3.7-Flash passes 74.4% of the intent judgments but only 42.6% of the content judgments. When the speech is a conversation between peers rather than a request to the assistant, even strong models often keep responding. When a reply must satisfy several people, the best score is only 25.9. These results indicate that recognizing a request and meeting its requirements are separate abilities, and O-IN-Bench measures each of them.
24. 【2611.24696】Improving Zero-shot 3D Semantic Mapping in Offroad with Geometric Priors
链接:https://arxiv.org/abs/2611.24696
作者:Eerik Alamikkotervo,Risto Ojala
类目:Computer Vision and Pattern Recognition (cs.CV)
关键词:challenging offroad conditions, enables platform agnostic, platform agnostic mobile, agnostic mobile robot, mobile robot navigation
备注:
点击查看摘要
Abstract:Semantic 3D mapping is a powerful representation that enables platform agnostic mobile robot navigation in challenging offroad conditions. Current vision foundation models can accurately predict semantics zero-shot on structured scenes but performance is compromised in natural environments. On the other hand, 3D point clouds encode valuable information for discriminating semantics in scenarios where vision foundation models fail. In this work, we propose fitting a sphericity-elevation distribution for each semantic class to act as geometric anchor. Only the points with the most confident vision foundation model predictions are considered when fitting to produce high quality geometric prior distributions. The geometric prior is fused with the vision foundation model prediction to produce the final semantic mapping. The proposed prior improves SAM3 baseline by +6.3 in mIoU and +8.9 in accuracy on the RELLIS-3D dataset, and by +1.7 in mIoU and +2.1 in accuracy on the WildScenes dataset. The source code will be made public upon paper acceptance.
25. 【2611.24681】DrawScore: Evaluating Generated CAD Models against Engineering Drawings
链接:https://arxiv.org/abs/2611.24681
作者:Haorui Chen,Shiteng Lu,Zhanyu Guo,Zaixi Li,Yunhe Ni,Yaosong Du,Yichen Xiang,Xiande Huang,Rui Huang
类目:Computer Vision and Pattern Recognition (cs.CV); Software Engineering (cs.SE)
关键词:Chamfer Distance, Automatic CAD generation, barely penalize local, surface F-score, penalize local errors
备注:
点击查看摘要
Abstract:Automatic CAD generation is commonly evaluated with volumetric IoU, Chamfer Distance, and surface F-score, which barely penalize local errors that occupy only a small fraction of a part's volume or surface area, such as a misplaced hole or a slot on the wrong face, even though such errors are critical to manufacturing. We formalize these metrics in a unified integral form and show that the weight of a local error decays as the $(k+q)$-th power of its size, where $k$ is the dimension of the integration domain and $q$ is determined by the penalty function. Guided by this analysis, we propose DrawScore, which projects both models into three orthographic views with visible and hidden lines, as in engineering drawings, and applies a saturating penalty along the projected curves, so that the weight of a local error decays only linearly. Controlled experiments confirm the predicted decay orders, and on outputs of existing CAD generation agents, DrawScore exposes errors in outputs that IoU rates as nearly perfect. DrawScore further improves generation as a reinforcement learning reward and, for CAD generation from engineering drawings, serves as a test-time verifier without 3D ground truth or an additional reward model. Our code is available at this https URL.
26. 【2611.24672】Predicting Species Splits: A Challenging Fine-Grained Benchmark for Category Discovery
链接:https://arxiv.org/abs/2611.24672
作者:Christian Lange,Joakim Bruslund Haurum,Bingchen Zhao,Oisin Mac Aodha
类目:Computer Vision and Pattern Recognition (cs.CV)
关键词:making long-term evaluation, increasingly saturated, fail to reflect, long-term evaluation, reflect real-world discovery
备注: Accepted at NeurIPS 2026 (Evaluations and Datasets Track)
点击查看摘要
Abstract:Widely used category discovery benchmarks have become increasingly saturated and often fail to reflect real-world discovery settings, making long-term evaluation of new methods difficult. We introduce PS-SPLIT, a large-scale fine-grained benchmark for category discovery, motivated by taxonomic splitting: the real-world process by which what was once considered a single species is found to comprise multiple distinct ones. PS-SPLIT contains 79K globally distributed, high-quality, community-collected bird images spanning 771 species, derived from 257 recent real expert-verified taxonomic splits, with associated geographic metadata and full taxonomic hierarchy. The visual differences between categories arising from real taxonomic splits can be very subtle, often requiring significant human expertise to tell apart, making PS-SPLIT a challenging benchmark which we hope will assist in advancing fine-grained category discovery research. Based on taxonomic splitting, we introduce the task of discovery by category splitting, where the goal is to determine which known categories should be subdivided into new finer-grained subcategories. We adapt four generalized category discovery methods to this setting, evaluate existing methods, and show that there is substantial room for future progress.
27. 【2611.24651】Piccolo: Co-Evolving Decision Models and Data through Recursive Self-Improvement
链接:https://arxiv.org/abs/2611.24651
作者:Zhenqing Ling,Qirui Jiao,Daoyuan Chen,Cheng Chen,Xinji Mai,Haowen Gao,Zhuo Zhang,Xuchen Pan,Mengsu Ding,Jiaji Deng,Yuexiang Xie,Xiaoming Qin,Weirui Kuang,Zhaoyang Liu,Yaliang Li,Bolin Ding,Jingren Zhou
类目:Computation and Language (cs.CL); Artificial Intelligence (cs.AI); Computer Vision and Pattern Recognition (cs.CV)
关键词:Agentic Data Engineering, Engineering and recursive, coevolves model capabilities, Index Vision Full, Agentic Data
备注: 22 pages, 13 figures, 11 tables. Project page and blog: [this https URL](https://decisionplayground.agentscope.io/piccolo.html)
点击查看摘要
Abstract:We present Piccolo, a family of multimodal decision models (0.8B to 27B) that coevolves model capabilities and training data via Agentic Data Engineering and recursive self-improvement (RSI). The models predict probabilities over candidate answers for general-purpose text and visual judgments, agent harnesses, data workflows, and interactive games. Piccolo-27B tops the Decision Index Vision Full leaderboard with a score of 75.59, while Piccolo-9B and Piccolo-4B achieve 68.35 and 67.46, respectively. All three rank first on the Private benchmark in their respective model-size categories. Research agents analyze model behavior, construct training data, conduct experiments, and use the results to refine subsequent proposals. Semantic slices and a shared research memory connect these actions across the model family's development. With training recipes comprising hundreds of thousands of examples, data selection and feedback-driven refinement are central to the self-improvement process. Our large-scale RSI exploration yields three counter-intuitive insights for decision-model development: (1) Backbone-recipe co-adaptation outweighs raw model recency: earlier backbones can surpass newer generations on visual tasks under identical recipes; (2) Targeted data curation enables 4B-9B models to rival 27B baselines on specialized workflows; and (3) Continuous loss reduction can decouple from and even degrade discrete decision accuracy. Together with the released weights and tools from the Data-Juicer and AgentScope ecosystems, Piccolo provides an actionable blueprint for building frontier decision models and scaling data-driven RSI.
28. 【2611.24643】AggCLIP: Aggregating CLIP for Zero-Shot Multimodal Anomaly Detection
链接:https://arxiv.org/abs/2611.24643
作者:Zhi Li,Hanyu Wang,Fei Shen,Zhengtao Zhang
类目:Computer Vision and Pattern Recognition (cs.CV)
关键词:Multimodal Anomaly Detection, Zero-Shot Multimodal Anomaly, high annotation costs, Anomaly Detection, target-specific training data
备注: 13 pages, 4 figures, 6 tables. Author version of the ACM Multimedia 2026 paper
点击查看摘要
Abstract:Zero-Shot Multimodal Anomaly Detection (ZSMAD) aims to identify both appearance and geometric defects in novel categories without relying on target-specific training data, making it especially valuable in real-world scenarios constrained by data scarcity, privacy concerns, or high annotation costs. While recent approaches successfully adapt vision-language models like CLIP to achieve this open-vocabulary generalization, they still face significant challenges: (i) severe geometric information loss during 3D-to-2D point cloud rendering; (ii) spatially noisy 2D feature maps produced by raw CLIP patch embeddings, compounded by shallow late-stage modality fusion; and (iii) severe cross-modal semantic misalignment due to decoupled processing streams. To address these limitations, we propose AggCLIP, a novel framework that adapts frozen CLIP representations for ZSMAD by explicitly incorporating local geometric structure and visual spatial context. Specifically, to accurately localize fine-grained defects, we design a Geometry-Aware Spatial Aggregation (GSA) module that refines back-projected 3D point features via density-adaptive neighborhood weighting, alongside a Context-Aware Spatial Aggregation (CSA) module that smooths multi-scale 2D patch embeddings using a lightweight trainable adapter. Furthermore, to prevent semantic drift between modalities, we introduce a Dual-Stream Prompt Contrastive (DSPC) loss, which aligns the abstract concepts of normality and abnormality across the point cloud and RGB streams. Extensive experiments demonstrate that the proposed AggCLIP framework achieves state-of-the-art performance on the MVTec 3D-AD and Eyecandies benchmarks under both one-vs-rest and cross-dataset zero-shot settings, notably surpassing the strongest competitive methods by up to 2.9% in pixel-level AUPRO.
29. 【2611.24637】Hypnos: A Dual-Phase End-to-End Platform for Scalable Continual Learning at the Edge
链接:https://arxiv.org/abs/2611.24637
作者:Guilherme Guedes,Marco Apolinario Lainez,Charlotte Frenkel
类目:Hardware Architecture (cs.AR); Artificial Intelligence (cs.AI); Computer Vision and Pattern Recognition (cs.CV)
关键词:enables neural networks, mitigating catastrophic forgetting, non-stationary data streams, adaptable edge devices, edge devices
备注:
点击查看摘要
Abstract:Continual learning (CL) enables neural networks to adapt to non-stationary data streams by mitigating catastrophic forgetting, offering a path to robust, private, and adaptable edge devices. Yet, porting CL to real-world edge devices remains challenging for two reasons: (i) online operation, where sensor data arrives one sample at a time and the device must adapt on-the-fly, and (ii) the low footprint of edge devices, which makes traditional error backpropagation training prohibitive. Inspired by the brain's wake and sleep cycles, the former for rapid adaptation with minimal resources and the latter for long-term consolidation, we introduce Hypnos, an open-source dual-phase platform for efficient CL at the edge. In the wake phase (e.g., daily operation), Hypnos relies on a low-footprint system-on-chip with a RISC-V microcontroller unit (MCU) targeting efficient inference and supporting lightweight gradient-free prototype updates. In the sleep phase (e.g., night-time charging), where wall-plug energy and larger memory budgets are available, a smart charger performs rehearsal-based gradient updates for consolidation. Deployed on a ZCU104 field-programmable gate array (FPGA) and synthesized in a 28-nm process, Hypnos is benchmarked on the CORe50 NICv2-391 protocol. It sets a new state-of-the-art accuracy of 78.6%-81.6% for edge CL systems, with an on-device latency of 7.79s-10.54s per sample depending on activation precision, and requires $3.8\times$ fewer learning steps and $22\times$ less learning compute than quantized latent replay at higher accuracy and comparable latency.
30. 【2611.24633】H3Remover: Reinforcing Video Object Removal without Paired Data
链接:https://arxiv.org/abs/2611.24633
作者:Kangjie Chen,Xiangyu Li,Jinhao Deng,Dongbin Zhang,Chaoda Zheng,Shijia Chen,Yuqi Zhang,Choo Sin Wai,Boyi Sun,Puyi Wang,Yu Zhang,Xianming Liu,Boyang Wang
类目:Computer Vision and Pattern Recognition (cs.CV); Artificial Intelligence (cs.AI)
关键词:Pretrained video editors, editors possess rich, possess rich removal, video editors possess, Pretrained video
备注: Project page: [this https URL](https://chenkangjie1123.github.io/H3Remover/) , code at: [this https URL](https://github.com/chenkangjie1123/H3Remover)
点击查看摘要
Abstract:Pretrained video editors possess rich removal and completion priors, yet may retain targets or alter unrelated content. We present H3Remover, which reinforces MiniMax-H3 Base Ref2VA for video object removal without paired object-free training targets. To discourage superficial edits such as recoloring, a target-removal reward favors calibrated depth changes consistent with revealing farther content. To limit unintended scene changes, a background-geometry reward encourages preservation of relative depth ordering. To preserve background appearance without retaining unwanted shadows or reflections, an appearance reward constrains RGB changes in source-derived protected regions while exempting targets and estimated effects. Frozen HPSv3 complements these constraints with perceptual-quality feedback for synthesized content. Evaluation regions remain fixed across candidates, and DiffusionART optimizes LoRA adapters with the backbone frozen. Experiments show substantial reconstruction gains over original H3, with qualitative comparisons demonstrating more effective object removal and better preservation of surrounding scene content.
31. 【2611.24616】EgoJev: Finding Evidence in Egocentric Video by Reading VLM Decisions
链接:https://arxiv.org/abs/2611.24616
作者:Chunggi Lee,Hanspeter Pfister
类目:Computer Vision and Pattern Recognition (cs.CV)
关键词:long egocentric video, Questions about long, long egocentric, egocentric video, video often depend
备注:
点击查看摘要
Abstract:Questions about long egocentric video often depend on one brief moment, such as the second a hand puts an object down. A vision-language model (VLM) that sees a few evenly spaced frames usually misses it. Training-free methods rank frames with a separate image-text model or ask the VLM to write a time. In a controlled test, the VLM itself locates the moment better when it chooses among labeled frames, and we read this choice from its output distribution. With the same candidates, this choice lands on the evidence more often than image-text scores or a written time span, and it compares all candidates in one decision. Based on this finding, we propose EgoJev, a training-free frame selector for wearable assistants. EgoJev reads where the evidence is from labeled thumbnails and looks again at the windows it points to. It places its few answer frames at the quantiles of the resulting distribution. One frozen VLM selects, answers and returns the evidence time, with no second model. A wearable device can encode its coarse thumbnails while it records. With Qwen3.5-4B and 8 answer frames, EgoJev raises QAEgo4D accuracy from 51.2% with uniform sampling to 62.0%, and it outperforms recent training-free frame selectors on QAEgo4D and MyEgo. A single-level version runs on a Mac and an iPhone.
32. 【2611.24597】DAL: Dynamic Angular Loss for Imbalanced Medical Image Classification
链接:https://arxiv.org/abs/2611.24597
作者:Salman Mohammad,Furkan Kasım,Manuel Günther
类目:Computer Vision and Pattern Recognition (cs.CV)
关键词:Class imbalance remains, Class imbalance, clinically important classes, major obstacle, rare but clinically
备注: Accepted at the British Machine Vision Conference (BMVC) 2026. 18 pages, 5 figures, 4 tables. Code: [this https URL](https://github.com/msalman95/DA)
点击查看摘要
Abstract:Class imbalance remains a major obstacle in medical image classification, where rare but clinically important classes are often overshadowed by majority categories. Although current strategies for reducing bias, including angle-based softmax (ABS) losses, offer strong discriminative features, their behavior in settings with a small number of severely imbalanced classes remains insufficiently understood. Previous approaches that estimate the intrinsic parameters of ABS rely on the assumption of many classes, which does not hold in medical settings. We derive data-driven upper and lower bounds for the scale parameter and an upper bound for the angular margin using negative-angle statistics, and use these to drive a batch-adaptive angular margin (BAAM), which we combine with dynamically-weighted cross entropy (DWCE) to define our dynamic angular loss (DAL). Across four highly imbalanced medical image classification benchmarks spanning dermatology, ophthalmology, and gastrointestinal endoscopy, our method consistently achieves superior balanced accuracy and the lowest coefficient of variation of per-class recall. Although this entails a small performance reduction on majority classes, the gains on minority categories are substantial, yielding a reliable and equitable overall classifier. Our source code is available at: this https URL.
33. 【2611.24579】ALIVE: Bring Foundation Features to Life with Events
链接:https://arxiv.org/abs/2611.24579
作者:Rong Zou,Haiqing Hao,Davide Scaramuzza
类目:Computer Vision and Pattern Recognition (cs.CV); Robotics (cs.RO)
关键词:provide powerful representations, Visual foundation models, models provide powerful, visual content, foundation models provide
备注:
点击查看摘要
Abstract:Visual foundation models provide powerful representations for diverse tasks, but their features become stale as visual content changes between image observations. Refreshing them requires new frames and repeated runs of a computationally expensive image encoder. We introduce ALIVE, an event-based method that incrementally updates native foundation model features between image observations. Given an anchor image, ALIVE uses subsequent events to transport and correct existing multi-layer features extracted by a frozen DINOv3 encoder. Its lightweight updater is trained end-to-end using only synthetic data, without downstream task supervision, and transfers directly to real-world event streams without any adaptation. Across three real-world datasets and five task-dataset settings spanning object detection, semantic segmentation, and depth estimation, ALIVE outperforms all evaluated event-based methods in every metric. On DSEC, when the anchor frame is 200,ms old, ALIVE improves over anchor-image predictions by 30.9,mAP in detection, 11.6,mIoU in segmentation, and 15.9% in depth AbsRel. Compared with image re-encoding, ALIVE reduces FLOPs by 82.4% and latency by a factor of 2.5.
34. 【2611.24578】BaseRSI: Recursive Self-Improvement of a Base Vision-Language Model via Training-Free Harness Internalization
链接:https://arxiv.org/abs/2611.24578
作者:Mingda Zhang,Yuanhao Dong,Qiang Huang,Rui Qian,Yanjin Li,Zhenyu Liu,Xiao Huang,Zihao Yang,Jian Wang,Pinjia He,Xiaoying Tang
类目:Computer Vision and Pattern Recognition (cs.CV); Artificial Intelligence (cs.AI)
关键词:Recursive Harness Internalization, base model, Recursive self-improvement, prompts and tools, model
备注:
点击查看摘要
Abstract:Recursive self-improvement (RSI) is moving from the harness of prompts and tools around a base model toward the model itself. However, existing RSI still faces key challenges, including harness-bound gains that stay outside the base model, black-box supervision that reads only the model's outputs, and imprecise edits whose site and strength are fixed in advance. To address these challenges, we propose BaseRSI, a new paradigm of Recursive Harness Internalization: the base model improves its harness, then writes what the harness adds into its own weights through training-free edits, round after round on its own responses. To support this paradigm, we introduce ReGap, a mechanism that supervises every transformer block with the high-dimensional activation difference that the harness makes at that block alone, and updates the block's weights along this signal in a low-dimensional subspace. Building on this mechanism, we further propose ReKey, an algorithm that writes this signal by regression into a few key-value neurons of every block, and measures the site and strength of each edit on the model's own responses. Moreover, the base model fills every role of this recursion with forward passes alone, and its edits, written at different sites, compose in one checkpoint without refitting. Experimental results on ten visual reasoning benchmarks show that BaseRSI consistently improves a base vision-language model both in distribution and on held-out benchmarks. Our code is available at this https URL.
35. 【2611.24565】Beyond Appearance: Evidence-Guided Programmatic 3D Scene Reconstruction with Agentic Lighting Reasoning
链接:https://arxiv.org/abs/2611.24565
作者:Zeting Luo,Yifan Li,Shuai Yang
类目:Computer Vision and Pattern Recognition (cs.CV)
关键词:produce executable code, generates editable digital, editable digital content, reconstruction produce executable, supports visual simulation
备注: Project Page: [this https URL](https://logic-al-al.github.io/LightHarness) Code: [this https URL](https://github.com/lyf1212/LightHarness_Code)
点击查看摘要
Abstract:Image-to-code methods for 3D scene reconstruction produce executable code that generates editable digital content and supports visual simulation. However, existing methods primarily rely on appearance similarity, which provides limited guidance for distinguishing errors in geometry, materials, and illumination. Light shapes scene appearance through its interactions with geometry and materials, influencing shading, cast shadows, and reflections. We therefore present LightHarness, a VLM-based agentic framework that incorporates explicit lighting reasoning into iterative scene reconstruction. To be specific, we leverage complementary geometric, intrinsic, and illumination cues to jointly refine scene structure and appearance, and propose Lighting Reasoning Memory to maintain light-source hypotheses, which supports evidence, and revision history throughout iterations. Together, they support reasoning about partially visible and unseen sources, with evidence-guided verification directing subsequent revisions. Our framework reconstructs explicit scenes through coherent reasoning about geometry, materials, and illumination without additional training or differentiable rendering. Extensive experiments demonstrate superior image-to-code reconstruction accuracy under challenging shadows and complex illumination, further supporting downstream scene editing, object insertion, and versatile relighting.
36. 【2611.24558】SCR: Detecting Backdoor-Poisoned Samples through Clean-Reference Residuals
链接:https://arxiv.org/abs/2611.24558
作者:Yi Yang,Jinyang Huang,Feng-Qi Cui,Yu-Tong Guo,Sirui Zhao,Haiming Jin,Weihua Li,Quan Bai,Meng Li
类目:Computer Vision and Pattern Recognition (cs.CV); Cryptography and Security (cs.CR)
关键词:Deep neural networks, adversaries manipulate model, manipulate model predictions, Deep neural, training data
备注:
点击查看摘要
Abstract:Deep neural networks are vulnerable to backdoor attacks, in which adversaries manipulate model predictions by injecting poisoned samples into training data. Detecting these samples remains challenging because legitimate semantic variation can obscure poisoning-related signals in learned representations. Consequently, class-level statistics used by existing detectors may flag benign feature deviations while overlooking poisoned samples that blend into the class distribution. In this paper, we propose Semantic Counterfactual Residualization (SCR), which uses a small set of clean reference samples to account for benign variation through sample-specific feature reconstruction. SCR identifies poisoned samples by detecting coherent deviations from these clean baselines, without requiring prior knowledge of the attacks. We evaluate SCR through experiments covering twelve dirty-label and clean-label attacks across CIFAR-10, CIFAR-100, and Tiny ImageNet. With a small clean-reference set, SCR achieves attack-averaged true positive rates above 99.5\% and false positive rates below 2\% on each dataset, outperforming eight representative detection baselines in macro-TPR.
37. 【2611.24541】Fuse Before Reconstruct: A Novel Reconstruction Paradigm for Multimodal Industrial Anomaly Detection
链接:https://arxiv.org/abs/2611.24541
作者:Zhi Li,Hanyu Wang,Qiyu Chen,Fei Shen,Zhengtao Zhang
类目:Computer Vision and Pattern Recognition (cs.CV)
关键词:Industrial anomaly detection, enhance detection accuracy, multimodal data emerges, Industrial anomaly, precision manufacturing
备注: Preprint before peer review. Includes supplementary material. Code: [this https URL](https://github.com/LiZhi-CASIA/FBR)
点击查看摘要
Abstract:Industrial anomaly detection plays a pivotal role in precision manufacturing, where joint analysis of multimodal data emerges as a critical approach to enhance detection accuracy. Feature reconstruction, a dominant paradigm for such joint analysis, is primarily implemented through intra-modal and cross-modal methods, both of which are plagued by significant limitations. Intra-modal reconstruction neglects complementary cross-modal information, while its cross-modal counterpart is prone to projection errors and ambiguous one-to-many mappings (e.g., same textures mapping to multiple geometries), leading to detection failures. To address these limitations, we propose a novel feature reconstruction paradigm, Fuse Before Reconstruct (FBR), which first fuses multimodal features and then performs reconstruction for anomaly detection. Firstly, we propose a bidirectional cross-modal attention mechanism to achieve deep feature complementarity between image and point cloud features, mitigating projection bias through channel alignment and dynamic spatial prior integration. Besides, a learnable pixel-wise gated fusion strategy is adopted to preserve multi-modal structural consistency while optimizing local feature combination. Finally, dual-path collaborative reconstruction is implemented to recover original modal features from the fused representations, where cross-modal consistency constraints effectively eliminate mapping ambiguities. Extensive experiments show that FBR achieves 98.3% I-AUROC on MVTec 3D-AD and 92.6% I-AUROC on Eyecandies, surpassing previous state-of-the-art methods by +2.9% and +4.1%, respectively.
38. 【2611.24524】SCRiD: Spatial and Contextual References for Generalizable AI-Generated Video Detection
链接:https://arxiv.org/abs/2611.24524
作者:Hao Tan,Zekai Zuo,Boyang Chen,Jinghui Qin
类目:Computer Vision and Pattern Recognition (cs.CV)
关键词:Generalizable AI-generated video, resemble synthesis artifacts, discrepancies unreliable indicators, detection remains challenging, Contextual Reference-informed Detection
备注: 14 pages,10 figures
点击查看摘要
Abstract:Generalizable AI-generated video detection remains challenging because natural motion and content variations can resemble synthesis artifacts, making isolated video discrepancies unreliable indicators of authenticity. To address this challenge, we propose Spatial and Contextual Reference-informed Detection (SCRiD), a framework that interprets video variations through complementary spatial and contextual references. Specifically, we introduce Matched-Coordinate Residual Response (MCR) to characterize temporal residual associations under motion and coordinate references with matched subpixel sampling conditions. We further develop Real-Conditioned Score Adaptation (RCSA), which learns a content-conditioned denoising reference from authentic videos and probes its responses across multiple noise scales and quality views to guide feature adaptation. To integrate these complementary sources of evidence, we design Reversible Evidence Exchange (REE) to enable information exchange through reversible symplectic updates. We train SCRiD using synthetic videos from only 4 generators and evaluate it across 88 generator or task subsets spanning 6 benchmarks. Our method achieves average ACC and AP of 72.72% and 74.23%, respectively, outperforming the strongest baseline by 3.24 and 0.81 percentage this http URL and robustness experiments further validate the effectiveness of the proposed components and the stability of SCRiD under common post-processing operations.
39. 【2611.24521】RAID: Representation Autoencoder for Intrinsic Decomposition
链接:https://arxiv.org/abs/2611.24521
作者:Claudia Cuttano,Tamy Boubekeur,Elena Garces,Giuseppe Vecchio
类目:Computer Vision and Pattern Recognition (cs.CV)
关键词:inherently ill-posed problem, strongly coupled appearance, coupled appearance factors, Single-image intrinsic decomposition, single RGB observation
备注:
点击查看摘要
Abstract:Single-image intrinsic decomposition is an inherently ill-posed problem, requiring the disentanglement of strongly coupled appearance factors from a single RGB observation. Recent methods address this ambiguity with diffusion priors, at the cost of large, iterative, and often stochastic models. We show that accurate decomposition does not require a large generative model. Our key idea is to decouple reconstruction from decomposition on top of a strong pretrained representation. First, an $X \rightarrow X$ teacher learns to reconstruct each intrinsic map from DINOv3 features using a shared decoder, without paired RGB-to-intrinsic supervision. Then, a DINOv3-based student predicts the corresponding teacher representations from RGB through lightweight, modality-specific projections, enabling training on partially annotated datasets and predicting all intrinsics in a single forward pass. \textbf{RAID}, predicts intrinsic properties in the structured latent space, enabling a compact 165M-parameter model, 100x faster than diffusion-based methods. Despite its compact size, it achieves competitive or superior decomposition quality, with improved robustness to input resolution. Additionally, our learned projections yield interpretable, modality-specific feature spaces.
40. 【2611.24512】Comparing Each Generation with Independent Siblings Gives Video World Model Post-Training a Proper Objective for Outcome Frequencies
链接:https://arxiv.org/abs/2611.24512
作者:Xin Li,Yiwen Chen
类目:Computer Vision and Pattern Recognition (cs.CV)
关键词:physical outcomes open, Video world models, makes single videos, frame leaves, leaves several physical
备注:
点击查看摘要
Abstract:Video world models are used as simulators, so when a first frame leaves several physical outcomes open, repeated generations should produce each outcome as often as it occurs in the data. Physics post-training scores every generation against the single recorded future of its history, which makes single videos more faithful, but the expected score is linear in the model distribution and is maximized by always producing the most likely outcome. The log-likelihood of the recorded outcome is maximized by the data distribution, and its gradient weights agreement with the record by the inverse of how often the model produces the generated outcome, a frequency that other generations of the same history with independent noise, its siblings, can estimate. To train with this weight from a finite group, we propose SibPO, which performs Sibling-paired Policy Optimization: siblings estimate a truncated inverse-frequency weight for agreement with the record, and a second term subtracts the same weight times the agreement with the siblings, which makes the expected objective a strictly proper divergence over outcome cells at every group size. On PisaBench, SibPO lowers the KS distance to simulated landing times by 9.1-10.2% relative to the strongest method outside its reward family on three of four video models without degrading single videos, and it lowers the error of estimated robot policy success rates by 13%.
41. 【2611.24494】RECLAIM: Robot-Vision-Informed Adversarial Patch Defense via Embodied Attention Feedback
链接:https://arxiv.org/abs/2611.24494
作者:Jiayou Zhu,Peng Guo,Litian Zhang,Xinze Shi,Yuting Zhang,Zelong Yu,Yize Tian,Jirong Liu,Zifeng Kang
类目:Robotics (cs.RO); Cryptography and Security (cs.CR); Computer Vision and Pattern Recognition (cs.CV)
关键词:alter subsequent observations, perception-action loop grounded, corrupt action predictions, physical interaction, subsequent observations
备注: 10 pages, 2 figures
点击查看摘要
Abstract:Vision-language-action policies control embodied agents through a perception-action loop grounded in physical interaction, where adversarial patches can corrupt action predictions and thereby alter subsequent observations. To mitigate these threats, many existing defenses rely on policy adaptation or auxiliary models, but require additional training or inference. We introduce RECLAIM, a training-free, inference-time defense that uses a frozen VLA's own attention response to local visual interventions to guide purification during closed-loop embodied manipulation. Our key insight is that attention changes induced by local edits provide feedback for choosing where and how to purify as the robot interacts with its environment. RECLAIM combines this feedback with robot geometry to refine patch localization and select local treatments, then revalidates them as the agent's embodied actions change the observed scene. We evaluate RECLAIM against patches generated by four adversarial attack methods across multiple manipulation task suites under frozen VLA control. Under adversarial patch attacks, RECLAIM succeeds in at least one rollout on 95% of the evaluated tasks and raises rollout success on LIBERO-Object from 0% without defense to 70.0%. Its mean decision time is only 548.9 ms per control step. These results highlight policy-guided visual purification as an effective approach to defending embodied agents against adversarial patches and recovering closed-loop task execution.
42. 【2611.24479】Every Query Routes Itself: Accelerating Video Diffusion Transformers with Per-Query Routed Sparse Attention
链接:https://arxiv.org/abs/2611.24479
作者:Zhen Huang,Shuwei Li,Ruizhe Yao,Xinrui Chen,Danyi Liu,Haoyuan Liang,Zhiyu Ye,Qiuyan Sun,Bingwei Chen,Yiyuan Li,Xu Zhou,Haohuan Fu
类目:Computer Vision and Pattern Recognition (cs.CV); Machine Learning (cs.LG)
关键词:video diffusion transformers, long token sequences, token sequences dominates, Self-attention over long, diffusion transformers
备注:
点击查看摘要
Abstract:Self-attention over long token sequences dominates the inference cost of video diffusion transformers. Clustering-based sparse attention clusters both queries and keys and lets all queries in a cluster share one selection of key clusters. We show that this sharing limits accuracy: with key clusters and density fixed, attention grows more accurate as the query clusters are refined, and is most accurate when each query selects on its own, which needs no query clustering at all. We therefore propose Per-Query Routed Sparse Attention (PQRS), a training-free method that clusters only keys, lets each query select its own key clusters, and compensates for the unselected ones with their centroids. Per-query selections leave no query tile with a shared key list, so PQRS traverses key clusters instead of queries, buckets queries with identical selections into dense tasks, and merges their partial results in one pass under a fixed softmax offset carried from the previous denoising step. On four text-to-video and image-to-video models from the Wan and HunyuanVideo families, PQRS is the fastest of all the sparse attention methods we evaluate and the closest to the dense output, reaching $28.9$ dB PSNR on Wan2.1-T2V and $35.4$ dB on HunyuanVideo at $1.66$--$2.10\times$ end-to-end speedup.
43. 【2611.24462】Anchoring with Visual Evidence: Explicit Semantic Anchors for Joint Change Detection and Captioning
链接:https://arxiv.org/abs/2611.24462
作者:Kai Zheng,Hang-Cheng Dong,Puyi Fang,Junjia Pan,Zhenkai Wu,Wei Zhang
类目:Computer Vision and Pattern Recognition (cs.CV)
关键词:bi-temporal remote sensing, remote sensing images, seeks to localize, bi-temporal remote, remote sensing
备注:
点击查看摘要
Abstract:Joint change detection and captioning seeks to localize changes and describe them from bi-temporal remote sensing images. A key challenge is translating dense visual evidence into the semantic conditions that guide description generation. We propose Explicit Semantic Anchors (ESA), a supervised interface that exposes entity presence, temporal appearance variation, and spatial extent. A prediction-guided readout combines visual features and predicted change probabilities, and three factor heads estimate the corresponding anchor activations. These activations guide generation through two complementary routes. The context route maps the three factor activations to three separate memory tokens, one per factor, and appends them to the dense visual memory, while the lexicon route produces an additive vocabulary bias at each decoding step. Dense visual memory remains available for spatial detail and information beyond the explicit factors. Anchor targets are derived from existing images and change annotations, requiring no additional anchor-level language annotation. At inference, all conditions are computed from the input image pair and the model's own predictions. Experiments on LEVIR-MCI and WHU-CDC show strong joint detection and captioning performance, supporting the use of supervised visual attributes as explicit conditions for change description generation.
44. 【2611.24457】ShoWorld: Shape-Faithful Multi-Hypothesis Occupancy World Model
链接:https://arxiv.org/abs/2611.24457
作者:Zhiqiang Yan,Guangfa Wang,Yuan Wu,Jian Yang,Gim Hee Lee
类目:Computer Vision and Pattern Recognition (cs.CV)
关键词:world models struggle, forecast moving objects, moving objects due, Occupancy world models, world models
备注: 10 pages
点击查看摘要
Abstract:Occupancy world models struggle to forecast moving objects due to motion uncertainty and shape distortion. A single trajectory can average plausible futures into an implausible path, and repeated re-voxelization can erode or inflate object shape even under correct motion. We introduce ShoWorld to address these challenges through separate motion and shape representations. The motion forecaster predicts probabilistic trajectories conditioned on scene layout, nearby agents and the future ego trajectory. Each hypothesis is rendered into occupancy and fused according to its probability to retain alternative futures. The shape component predicts sub-voxel points with occupancy probabilities and offsets from local geometry. These points move rigidly with the object, and voxel occupancy follows the probability that at least one occupied point falls within it. This representation reduces shape distortion under motion. ShoWorld achieves state-of-the-art results on Occ3D-nuScenes. With ground-truth occupancy and ego trajectory, it improves mIoU and IoU by 6.56 and 9.16 points overall and by 6.06 and 5.53 points on dynamic classes.
45. 【2611.24451】EviScope: Evidence-Conditioned Sampling for Long-Range 3D Detection
链接:https://arxiv.org/abs/2611.24451
作者:Mingyang Lyu(1, 2 and 3),Yinqian Sun(1),Gongce Cao(3),Chujie Tang(1),Yuhao Zhang(4),Lu Jia(5),Qian Zhang(1),Bo Wang(1),Pin Lyu(1) ((1) Institute of Automation, Chinese Academy of Sciences, Beijing, China, (2) Zhongguancun Academy, Beijing, China, (3) University of Chinese Academy of Sciences, Beijing, China, (4) University of Illinois Urbana-Champaign, USA, (5) University of Science and Technology Beijing, Beijing, China)
类目:Computer Vision and Pattern Recognition (cs.CV)
关键词:Query-based camera-radar detectors, camera-radar detectors choose, Query-based camera-radar, object query samples, position or feature
备注: 29 pages, including appendices
点击查看摘要
Abstract:Query-based camera-radar detectors choose where and over which frames each object query samples from the query's position or feature. We test whether radar evidence around the query should also set these choices at the 150 m range of MAN TruckScenes, where recall differs across object-level radar-return counts within every range band. EviScope is a minimal test: the current frame's returns form a non-learned bird's-eye evidence field, which each query reads at its decoded centre to scale its radial sampling extent and to weight eight frames. It adds 5,450 parameters (0.009%) to RaCFormer, with no latency increase in our timing. Our module-free recipe exceeds the published TruckScenes mAP (25.9 over seven runs against 24.1), so we measure the module only against its own baseline at matched seeds. The evidence a query reads rises with its object's return count (Spearman 0.51) but captures zero-return objects only in part, and the learned radius tracks it ($\rho\approx-0.91$ in four runs). Over seven four-epoch pairs the module changes NDS by $+0.31\pm0.36$ and mAP by $-0.27\pm0.22$ (mean $\pm$ SE). Against a geometry-only conditioner with the same parameter count, trained at three matched seeds, it is higher by $+1.25\pm0.72$ NDS and $+0.87\pm0.21$ mAP, on mAP in every pair, and the control's mean NDS is near the baseline's. An exploratory far-range gain failed an internally pre-specified held-out test, and large inference-time substitution penalties also occur without query-local evidence: neither learned allocation nor inference-time dependence substitutes for matched end-to-end comparisons.
46. 【2611.24445】Retroactive Concept Bottleneck Models for Exemplar-Free Class-Incremental Learning
链接:https://arxiv.org/abs/2611.24445
作者:Yang Ji,Ying Sun,Yuanxin Zhuang,Yuting Zhang
类目:Computer Vision and Pattern Recognition (cs.CV)
关键词:retaining training samples, previous tasks, sequentially without retaining, samples from previous, EFCIL
备注:
点击查看摘要
Abstract:Exemplar-free class-incremental learning (EFCIL) requires models to learn new classes sequentially without retaining training samples from previous tasks. Continual concept bottleneck models (CBMs) make their predictions interpretable by identifying visual concepts in an image and using them to score each class. However, these methods typically focus on preserving learned concepts or approximating past classes with generated features, which can leave new concepts underused in old-class predictions. We propose RetroCBM, which retains a small set of vocabulary-independent statistics relating frozen visual features to the embedding space of a pretrained vision-language model. Even after earlier training images have been discarded, we can use these statistics to supervise new concepts. To do so, we develop a retroactive regression rule that yields the same predictor for each new concept as class-balanced ridge regression on all images seen so far. We then reproject the stored statistics to recover the full-history squared classification loss and thus learn how much each new concept should contribute to old-class predictions. Since the classifier may use systematic concept errors to distinguish nearby classes, we further introduce a graph-weighted penalty on the margin shifts caused by those errors. Across six datasets, RetroCBM outperforms state-of-the-art interpretable methods in all class-incremental settings, with an 18.1\% relative gain in mean average incremental accuracy over the strongest exemplar-free baseline.
47. 【2611.24443】NativeSeg: Native Vision Language Segmentation of 3D Computed Tomography via Spatial Visual Tokens and Verified Local Zoom
链接:https://arxiv.org/abs/2611.24443
作者:Chengqian Ma,Gaole Dai,Zijie Meng,Weixuan Jin,Yijie Du,Yuexiao Ma,Xiawu Zheng,Fei Chao
类目:Computer Vision and Pattern Recognition (cs.CV)
关键词:fine boundary precision, Text prompted, organ prompted slices, prompted slices lack, full slice context
备注: 13 pages, 5 figures, 4 tables. Code: [this https URL](https://github.com/step-out/NativeSeg) ; Weights: [this https URL](https://huggingface.co/ChengqianMa/NativeSeg)
点击查看摘要
Abstract:Text prompted 3D computed tomography (CT) segmentation requires both full slice context and fine boundary precision across volumes where over $80\%$ of organ prompted slices lack the queried target. Existing medical vision language models (VLMs) are largely non native segmenters: they delegate pixel delineation outside the VLM by compressing reasoning into a 1D [SEG] vector for an external Segment Anything Model (SAM) decoder, quantizing masks into lossy discrete codebooks, or issuing box prompts to stateless external SAM tools without empty slice screening or spatial state memory. We introduce NativeSeg, a native vision language segmentation framework on Low Rank Adaptation (LoRA) adapted Qwen3.8-27B that performs dense mask decoding, volumetric triage, spatial error self diagnosis, and multiround local zoom directly over the VLM's 2D spatial visual tokens without external SAM decoders. NativeSeg decodes confidence maps from upper layer 2D spatial tokens fused with a five slice U-Net pathway, halts at zero zooms on $91\%$ of standalone evaluations via a native presence gate (pres), localizes uncertain boundaries via a 20 cell multiscale error gate (err), and accepts zoomed updates only when predicted error decreases. Trained solely on adult CT, NativeSeg improves Round $0$ DSC by $+19.81$ and $+7.04$ points over discrete codebook and 1D [SEG] readouts on the identical backbone, reaches $86.81\%$ adult and $87.04\%$ zero shot pediatric DSC, and safely refines 2D nnU-Net ($89.52\%$ adult; $86.86\%$ pediatric), 3D SegVol ($86.81\%$), and 3D VoxTell ($89.85\%$).
48. 【2611.24440】Explainable and CPU-Efficient Brain Tumor Classification from MRI Using Two-Phase EfficientNetB3 Fine-Tuning
链接:https://arxiv.org/abs/2611.24440
作者:Deepro Das,Saad Kabir Uddin,Mohammad Alauddin,Bornali Sarker,Thasnim Akhter
类目:Computer Vision and Pattern Recognition (cs.CV)
关键词:lethal neurological disorders, diagnoses annually worldwide, Magnetic Resonance Imaging, neurological disorders, annually worldwide
备注: 6 pages, 5 figures, 6 tables
点击查看摘要
Abstract:Brain tumors are among the most lethal neurological disorders, with over 308,000 new diagnoses annually worldwide. Accurate classification of tumor type from Magnetic Resonance Imaging (MRI) scans is critical for treatment planning; however, manual radiological interpretation is time-consuming and prone to inter-observer variability. This paper proposes an automated deep learning pipeline for four-class brain MRI classification (glioma, meningioma, no tumor, and pituitary tumor) using EfficientNetB3 transfer learning with a custom classification head. A two-phase training strategy is adopted: Phase 1 trains the head with a frozen backbone (best validation accuracy: 92.77%), and Phase 2 selectively fine-tunes the top 37 layers at a 100x reduced learning rate (best validation accuracy: 94.91%). On an independent balanced test set of 1,600 MRI images, the system achieves 90.62% overall accuracy and a macro-average AUC of 0.9854. Gradient-weighted Class Activation Mapping (Grad-CAM) is integrated for class-discriminative visual explainability. The proposed system simultaneously addresses three dimensions absent in prior work: high accuracy, CPU-only computational efficiency, and clinical interpretability.
49. 【2611.24433】MLIE-CoT: Chain-of-Thought Reasoning via Multi-Level Image Editing
链接:https://arxiv.org/abs/2611.24433
作者:Baichuan Yang,Dongyuan Xu,Zhao Yang,Yong Liu,Bing Su
类目:Computer Vision and Pattern Recognition (cs.CV)
关键词:breaking complex tasks, complex tasks, handle complex tasks, large language models, proven effective
备注: 11 pages, 5 figures. Code: [this https URL](https://github.com/yang031119/MLIE-CoT)
点击查看摘要
Abstract:Chain-of-thought (CoT) reasoning has proven effective in improving the reasoning abilities of large language models by breaking complex tasks into step-by-step solutions. In recent years, researchers have attempted to extend CoT reasoning to visual generative models to improve their ability to handle complex tasks. However, existing CoT methods for text-to-image generation still rely mainly on textual reasoning and therefore cannot fully use and preserve the correct semantic information already present in an image, as even the most detailed language cannot capture all visual information. With advances in image generation, models such as FLUX and BAGEL can generate images conditioned on visual inputs. Recent work has attempted to carry images forward as context in the chain of thought, but this introduces another limitation: unsuitable earlier images can strongly interfere with the model, while repeated whole-image regeneration introduces new hallucinations at each round and causes them to accumulate. To address these issues, we propose MLIE-CoT, a visual chain-of-thought framework based on multi-level image editing. MLIE-CoT learns to select the repair granularity from three levels, preserving correct semantics whenever possible during reasoning while reducing the effect of incorrect content. To improve the ability of multimodal models to route errors of different severity to appropriate repair levels, we construct an image-repair dataset covering different error cases and use it to train a repair-level router. Extensive experiments demonstrate that MLIE-CoT significantly outperforms both its base models and methods using fixed repair strategies, producing clear improvements across multiple benchmarks.
50. 【2611.24431】Let a Small Model Read First: Faster Prefill for Long Interleaved Image-Text Inputs
链接:https://arxiv.org/abs/2611.24431
作者:Xiaolin Lin,Kai-Chiang Wu,Bing Li,Grace Li Zhang
类目:Computer Vision and Pattern Recognition (cs.CV); Artificial Intelligence (cs.AI)
关键词:long interleaved inputs, small hybrid model, Vision-language models, slow to start, long interleaved
备注:
点击查看摘要
Abstract:Vision-language models (VLMs) can answer questions about long interleaved inputs, such as documents of many pages and collections of hundreds of images, but they are slow to start: on inputs of about 100K tokens, the prefill takes most of the wait for the first output token. A common approach prunes the input by scoring its tokens with the target model itself or with a small model of the same family that first writes an answer. However, this scoring takes much of the time that pruning saves. We find that the evidence for a question lies in a few whole units (images or paragraphs), and that a small hybrid model finds them with one attention head of an early layer, even at half image resolution. Based on these findings, we propose SkimPrefill, which lets the small hybrid model read the whole input first, up to that early layer, with images at half resolution. Most layers of the small hybrid model (Qwen3.5-0.8B) run in linear time, so this pass is short. SkimPrefill then ranks every unit with that head and gives the target the top units as an ordinary shorter prompt. One untrained selector serves targets of different model families. Extensive experiments on MMLongBench at 128K tokens, with 19 datasets of five task types, show the effectiveness of SkimPrefill. With 10% of the input tokens kept, SkimPrefill speeds up the prefill of Qwen3.5-9B, Gemma-4-12B and Qwen3-VL-32B by 3.7 times, 11.6 time and 8.2 times while maintaining their average accuracy.
51. 【2611.24423】Probing Physics in Motion Diffusion Models Against Their Own Estimate
链接:https://arxiv.org/abs/2611.24423
作者:Saeed Ghorbani
类目:Machine Learning (cs.LG); Computer Vision and Pattern Recognition (cs.CV); Graphics (cs.GR)
关键词:Recent studies report, Recent studies, physical properties decode, inputs or outputs, studies report
备注: 32 pages, 12 figures
点击查看摘要
Abstract:Recent studies report that video diffusion models "know more than they show": physical properties decode better from their activations than from their inputs or outputs. But a diffusion model computes its clean-sample estimate from those same activations, so such a comparison measures how easily a readout extracts the information. Whether the model hides anything is a separate question. We set out to find hidden physical knowledge in kinematic text-to-motion diffusion models, where the physics of every intermediate estimate can be computed exactly, and recorded every layer along the sampling trajectories of MDM (two checkpoints) and MotionDiffuse. Against the usual baselines the layers do look ahead of the model's own estimate on balance and flight. With every readout tuned the same way, the estimate closes the gap: for balance and flight, adding a layer gains at most 0.01, a size that a positive control shows we would detect. The fall rate of jumps is the one clear exception. Early in denoising, MDM's layers add about 0.08 in $R^2$ (up to 0.11 in a refit) that the estimate does not yet show, and the gain fades as sampling proceeds; on MotionDiffuse it is small and not robust. The estimate also flags floating samples without training, but aborting them saves under a tenth of the compute. In a video model, plausibility that linear probes miss in Wan 2.1's clean latent becomes readable after one fixed nonlinearity.
52. 【2611.24422】Resolution as Attention for Efficient Agent Memory Perception
链接:https://arxiv.org/abs/2611.24422
作者:Siqi Xiang,Zhipeng Xu,Shengrui Liu,Zulong Chen,Chunyan Miao,Shijian Lu
类目:Artificial Intelligence (cs.AI); Computer Vision and Pattern Recognition (cs.CV)
关键词:making efficient memory, LLM-based agents accumulate, accumulate extensive interaction, efficient memory management, agents accumulate extensive
备注:
点击查看摘要
Abstract:LLM-based agents accumulate extensive interaction histories when solving complex, long-horizon tasks, making efficient memory management critical to their performance and scalability. While existing memory management methods primarily focus on what to retain or retrieve, they often overlook how much perceptual detail should be allocated to individual past interactions. We posit that effective agent memory depends not only on the information preserved, but also on the granularity at which each past interaction is perceived. Motivated by this insight, we propose Resolution as Attention (RAA), a memory perception mechanism that uses visual resolution as a means of attention allocation across interaction histories. RAA renders past interactions as images and dynamically adjusts their resolution at every step according to their relevance to the agent's upcoming decision, preserving fine-grained information for critical interactions while reducing visual-token consumption for less relevant ones. To adaptively determine the perceptual fidelity of each interaction, we train a lightweight relevance estimator with teacher-model guidance and further optimize it to preserve the agent's decision consistency at a reduced visual-token cost. Experiments across multiple tasks and backbones show that RAA reduces token consumption by over 50% without compromising task performance. Moreover, RAA can be integrated as a plug-in perception layer with existing memory management methods, yielding further token savings and revealing adaptive perception as a complementary dimension of agent memory.
53. 【2611.24419】PhysResponse: Probing the Physics of Video Generation with Controlled Interventions
链接:https://arxiv.org/abs/2611.24419
作者:Yuhan Li,Shuran Zhang,Haixin Tian,Baonuo Rao,Weipeng Li,Yujia Hou,Zhenyu Wang,Zijun Wang,Luoyang Lin,Lijing Luo,Jiehui Huang,Kaiyuan Yang,Jingbin Tan,Panwen Hu,Xiaodan Liang
类目:Computer Vision and Pattern Recognition (cs.CV)
关键词:increasingly viewed, dynamics that respond, respond correctly, Video generation models, generation models
备注: 11 pages. This version contains the main paper and references only. Supplementary material will be added in a later revision
点击查看摘要
Abstract:Video generation models are increasingly viewed as world simulators, which requires not only plausible motion but dynamics that respond correctly to specified physical parameters. Existing models, datasets and benchmarks, however, largely target physical plausibility, leaving this physical parameter responsiveness insufficiently explored. We introduce PhysResponse, a benchmark that measures it at two levels: scene-conditioned simulator inversion recovers the effective gravity, friction and initial speed of generated motion, and event evaluation tests outcomes near and across critical transitions. Evaluating ten image-to-video models, we find that even the lowest parameter errors exceed those of physically correct videos by over an order of magnitude, that models rarely reproduce both outcomes of a transition pair, and that the two levels expose complementary failures that visual metrics such as PSNR miss. To supply the missing training signal, we construct 72,000 simulated videos of 20 multi-object interaction types with controlled parameters; fine-tuning on them lowers parameter errors, even on held-out interaction types, and transfers to real videos and an external physics benchmark. Continued training with a numerical encoder further lowers all three errors on held-out types. PhysResponse provides a foundation for world simulators that follow physical conditions, not just appearances.
54. 【2611.24410】Why Multi-Source Audio Fails to Compose: A Geometric Diagnosis and Its Remedy
链接:https://arxiv.org/abs/2611.24410
作者:Anurag Singh,Prajwal Singh,Shanmuganathan Raman
类目:Computer Vision and Pattern Recognition (cs.CV); Sound (cs.SD)
关键词:occupy geometrically displaced, semantically unanchored positions, diffusion U-Net lacks, tokens occupy geometrically, CLIP text-token space
备注: Accepted in BMVC 2026
点击查看摘要
Abstract:Multi-source audio-to-image generation exhibits consistent compositional failures when audio tokens occupy geometrically displaced, semantically unanchored positions in the CLIP text-token space, where the diffusion U-Net lacks the prior to reliably attend to them or ground them in distinct spatial regions. We formalize this as the geometric displacement phenomenon and expose it through a two-level clustering analysis comparing BEATs and CLAP embeddings at the raw encoder and projected CLIP token levels. Three geometric properties emerge as important: intra-class compactness, angular alignment with the CLIP text manifold, and inter-token spatial separation. We show that CLAP, whose contrastive language-audio pretraining aligns embeddings with natural language semantics, substantially outperforms BEATs on all three properties, improving the silhouette score by more than 3x and the Audio-Image Content score as well. Building on this analysis, we introduce a multi-token extension that requires no diffusion-backbone retraining and assigns each audio source to a distinct placeholder token in the CLIP tokenizer, complemented by a mixed inference-time objective that combines Attend-and-Excite with an A* spatial separation loss to resolve residual cross-token interference. For evaluation, we construct multi-audio compositional benchmarks from VGGSound (VGG2, VGG3, and VGGN) and extend six standard audio-image metrics to the multi-source setting. Across five benchmarks and four strong baselines, our method achieves strong results on the metrics most directly tied to compositional faithfulness, with AIC improvements of up to 2.25x over the strongest baseline and FID reduced by nearly 2x on two-source generation. We propose geometric displacement as a diagnostic framework for cross-modal generation systems that project learned embeddings into a pretrained contrastive space.
55. 【2611.24408】StrideWorld: Benchmarking Scene Rigidity and Action Consistency in Interactive Video World Model
链接:https://arxiv.org/abs/2611.24408
作者:Chuang Wang,Rui Nie,Jie Song,Junyuan Gao,Jiahe Song,Haitao Zhou,Boyu Zhu,Xing Li,Jiang Wu,Conghui He,Qian Yu,Weijia Li
类目:Computer Vision and Pattern Recognition (cs.CV)
关键词:Interactive video world, turn user commands, world models turn, models turn user, Interactive video
备注:
点击查看摘要
Abstract:Interactive video world models turn user commands into the exploration of a generated scene, which requires the scene to keep its 3D structure as the camera moves and each action to keep a stable effect. Existing benchmarks mostly evaluate these properties against references fitted to the video under test, either a scene reconstructed from its own frames or a camera trajectory aligned to the command path with a free scale. Such references can absorb gradual scene deformation and changes in pace and route length, so these errors can pass undetected. We therefore introduce StrideWorld, a benchmark whose references are fixed before the motion they judge. It evaluates scene rigidity against the ground-truth geometry of the start view, and action consistency against the stride of each action, the camera motion of one command. The stride is measured on the model's first steps and then used to test the steps that follow, so each model is judged only against its own steps. Its measurements are validated on rendered videos with known camera motion, and its scores agree with human judgments. Across \nvideos{} videos generated by \nmodels{} open models from $140$ start views, all models but one deform their scenes as the camera moves, with offsets up to $7.3$ times those of a rigid rendering along the same camera path. Most models keep a steady pace when a single action is held. Along composed routes, several models often stall, and most completed routes end away from the positions predicted by each model's own strides. Only the two models that scale commands by the estimated scene depth bring most routes close to these positions. Fine-tuning two models on a few rendered clips removes most of their mid-route stalls. Code, data, and a leaderboard are publicly available at this https URL.
56. 【2611.24404】RAPID: Reconstruction Accelerated by Polarization-Informed Depth for 2D Gaussian Splatting
链接:https://arxiv.org/abs/2611.24404
作者:Daisong Ouyang,Linlin Xia,Liangkun Yan,Song Gu,Shedong Zhao,Xin Dai,Yi Zheng
类目:Computer Vision and Pattern Recognition (cs.CV)
关键词:multi-view images remains, images remains, remains a fundamental, fundamental challenge, Gaussian splatting
备注: 10 pages, 5 figures, 5tables
点击查看摘要
Abstract:Efficient and accurate 3D surface reconstruction from multi-view images remains a fundamental challenge in Gaussian splatting. Many existing methods rely on sparse geometric initialization and iterative densification, resulting in substantial optimization overhead and difficulties in balancing rendering quality and geometric accuracy, particularly in reflective and weakly textured regions. To address these limitations, we propose RAPID, a geometry-first framework that uses multiview polarization cues to recover dense surface priors. We integrate surface normals into depth maps using sparse geometric anchors. These depth maps are then fused across views. Curvature-adaptive sampling and oriented Gaussian initialization then support efficient optimization. Experiments on the PISR dataset demonstrate an approximately \(5\times\) speedup over PhyGaP, which is a state-of-the-art Gaussian-based reconstruction method. RAPID reduces the average optimization time from approximately 21 minutes (PhyGaP) to 4 minutes while maintaining competitive novel-view synthesis quality and mesh reconstruction accuracy. These results demonstrate the effectiveness of RAPID in accelerating Gaussian optimization from polarimetric multi-view inputs.
57. 【2611.24401】Reliable Tree Recruitment from Scarce Annotations for Forest Inventory Expansion
链接:https://arxiv.org/abs/2611.24401
作者:Luiz Fernando Souza Luz,Pedro J. Soto Vega,Matheus Pinheiro Ferreira,Laura Elena Cué la Rosa,Hudson Franklin Pessoa Veras,Ernandes Macedo da Cunha Neto,Evandro Orfanó Figueiredo,Ana Paula Dalla Corte,Carlos Roberto Sanquetta,José Matheus Segre Moneva Viveiros,Matheus Santos Fuza,Heitor Eduardo Ferreira Campos Morato Filpi,Cristiano Vinícius Vidal,Vinícius Moura Costa,Débora Nascimento Rosário,Danilo Roberti Alves de Almeida,Paulo Guilherme Molin,Dário Augusto Borges Oliveira
类目:Computer Vision and Pattern Recognition (cs.CV)
关键词:RGB UAV orthomosaics, Species-level inventories, annotation cost, individual tree crowns, handful of field-validated
备注: 20 pages, 15 figures
点击查看摘要
Abstract:Species-level inventories of individual tree crowns (ITCs) from UAV imagery are limited by annotation cost: in tropical forests, many species are represented by only a handful of field-validated crowns. We propose a reliable tree recruitment framework that expands a small annotated inventory by iteratively self-training a multi-task DeepLabV3 network, combining semantic segmentation with a crown-distance regression head, on RGB UAV orthomosaics. At each iteration, candidate crowns enter the training inventory only if they pass a confidence filter on class probability and distance map and a morphology filter on crown area and solidity, with at most five new crowns per species, keeping recruited samples consistent with the reference inventory. We evaluate the framework at two Brazilian sites: an Amazonian rainforest in Acre (17 species, 355 annotated crowns, 4 cm GSD) and a forest site in the Cerrado of Maranhão (4 species, 51 annotated crowns, 1.8 cm GSD). Over 20 iterations, the number of retained crowns grew from about 500 to 960 in Acre and from about 700 to over 3100 in Maranhão, while crown-level F1 exceeded single- and multi-task DeepLabV3 baselines (53.5% vs. 50.0% and 45.2%; 92.2% vs. 79.7% and 83.6%). The framework offers a low-cost route to expanding species inventories with consumer-grade UAVs and limited fieldwork.
58. 【2611.24384】Enhancing Active Learning Performance with Initial Data Selection Based on Foundation Models
链接:https://arxiv.org/abs/2611.24384
作者:Mohammad Mirzaee,Mehrshad Fallah Estalkhzir,Mohammad Reza Mohammadi
类目:Computer Vision and Pattern Recognition (cs.CV); Artificial Intelligence (cs.AI)
关键词:low resource budget, initial labeled pool, active object detection, pool is essential, resource budget
备注: 26 pages, 18 figures
点击查看摘要
Abstract:In active object detection, selecting the initial labeled pool is essential, especially in complex datasets with expensive labeling and low resource budget. This study presents an active learning framework designed to optimize the initial training cycle of active learning for the object detection task. Our approach diverges from traditional methods by utilizing feature-based clustering from the YOLO-World model and refining images based on objectness scores rather than relying on uncertainty sampling in the initial cycle. This approach focuses on finding samples with various features and emphasizes training in areas where the foundation model is most confident. Doing so helps the model better understand different types of objects. For subsequent cycles, we employ a consistency-based sampling technique to maintain the robustness of the training process. To evaluate our method, we use two complex datasets called Aquarium and NWPU VHR-10, which are well known for their diverse object sizes and challenging conditions. In the NWPU dataset, our method increases mAP50 by an average of 6.6% in image-based evaluation. Moreover, the image refinement method shows an improvement of approximately 11% in object-based evaluation. The results show improvements in model performance and labeling time, demonstrating the advantages of using foundation models in early cycles of active learning. An implementation of the algorithm is available at this https URL
59. 【2611.24380】Where Should a Diffusion Transformer Meet Its Teacher?
链接:https://arxiv.org/abs/2611.24380
作者:Zhuoqi Hu,Shiwei Li,Haozhao Wang,Xiandi Luo,Yihao Ouyang,Wenting Shen,Ang Wang,Jiamang Wang,Minyi Guo,Ruixuan Li
类目:Computer Vision and Pattern Recognition (cs.CV)
关键词:Diffusion Transformer training, accelerates Diffusion Transformer, Diffusion Transformer, significantly accelerates Diffusion, existing methods typically
备注:
点击查看摘要
Abstract:Representation Alignment (REPA) significantly accelerates Diffusion Transformer training by guiding intermediate student features with a pretrained visual teacher, yet existing methods typically attach supervision to a single fixed layer. As student representations evolve, however, we find that the most effective layer for alignment varies across training stages. To track this evolving layer suitability online without costly search, we measure gradient compatibility between the flow-matching and alignment objectives, and propose AdaREPA, an online routing framework that adaptively selects the supervision layer as training progresses. AdaREPA uses a softmax distribution over smoothed layer compatibility scores to sample a layer for alignment at each training step. This probabilistic routing adapts teacher supervision to the student's changing learning needs, with minimal training overhead and no additional inference cost. On ImageNet-256, AdaREPA accelerates convergence and improves generation quality across model sizes, while revealing how the most suitable layers for supervision shift over training.
60. 【2611.24373】CanopyKV: Remember beyond the Crop for Fast Active Visual Reasoning
链接:https://arxiv.org/abs/2611.24373
作者:Lei Yang,Dayan Wu,Yiliang Zhu,Xinze Liu,Hengjie Zhu,Zihao Zhang,Ding Wang,Peng Fu,Zheng Lin
类目:Computer Vision and Pattern Recognition (cs.CV)
关键词:Active visual reasoning, challenging visual questions, enables multimodal models, iterative image inspection, Active visual
备注: 16 pages, 11 figures, including supplementary material
点击查看摘要
Abstract:Active visual reasoning enables multimodal models to answer challenging visual questions by gathering fine-grained evidence through iterative image inspection and tool use. However, accumulating crops and reasoning tokens expands the key-value (KV) cache, making compression important for reducing memory and inference costs. Existing KV compression methods typically select historical tokens based on attention, recency, or redundancy. These criteria do not explicitly distinguish locally replaced evidence from complementary information needed for future exploration: each crop refreshes a local region, while earlier views may still support subsequent decisions. Discarding such evidence can impair accuracy or require additional search, while scoring and cache reorganization can offset the computational savings from compression. We introduce CanopyKV, a training-free KV compression framework that uses parent-child crop relationships to retain complementary evidence, keeps retained history stable, and protects the state needed for ongoing reasoning. Tool metadata provides retention signals without additional attention scoring, while compact KV storage reduces subsequent attention costs. Across VisualProbe, V*, HR-Bench, and MME-Lite, CanopyKV achieves state-of-the-art answer accuracy and end-to-end efficiency among the evaluated KV compression methods. On MME-Lite, it retains approximately one-quarter of Dense's total KV while improving accuracy by 10.5 percentage points and accelerating complete reasoning by $1.64\times$.
61. 【2611.24371】One Affinity, Two Axes: Bidirectional Slot Attention for Iterative Aggregation across the Structure--Detail Spectrum in Visual Retrieval
链接:https://arxiv.org/abs/2611.24371
作者:Zihang Wang,Wenkai Zhu,Lingdong Kong,Teng Yan,Alan Liang,Shuyuan Li,Xieyuanli Chen,Dong Kong,Guanyu Zong
类目:Computer Vision and Pattern Recognition (cs.CV)
关键词:Aggregating dense patch, discriminative global descriptors, Aggregating dense, dense patch tokens, dense patch
备注:
点击查看摘要
Abstract:Aggregating dense patch tokens into compact and discriminative global descriptors is central to visual retrieval. Although foundation models provide rich local features, existing aggregation heads still rely on single-pass mappings and lack an explicit mechanism to coordinate structural and fine-grained cues. Therefore, we propose Bidirectional Slot Attention (BiSA), which reformulates aggregation as an iterative coupled system. Cross-Axis Consensus (CAC) progressively refines learnable slots by interpreting a single shared token--slot affinity along two complementary axes. Along the competition axis, slots vie for each token under a temperature shared across all of them, determining token ownership and inducing a non-redundant scene decomposition. Along the selection axis, a slot-specific temperature governs the breadth of each slot's response, spanning a spectrum from broad layouts to sparse fine-grained details. Their consensus couples non-redundant decomposition with multi-granular slot specialization. Furthermore, the two axes expose intrinsic token-reliability cues at no extra cost: a token's claim strength under competition and its salience under selection. Progressive Token Filtering (PTF) converts these cues into a retention probability that down-weights unreliable tokens before the next CAC iteration, without any segmentation branch or external prior, forming a self-reinforcing loop of competitive decomposition, selective focusing, and feedback purification. Across all three retrieval tasks, BiSA leads on five of six place recognition benchmarks (+4.2 R@1 on Baidu Mall), tops all 16 metrics of drone--satellite geo-localization (+8.4 R@1 at 150 m), and improves instance-level retrieval on ROxford-Hard by 4.5 mAP points. Notably, the selection axis adds only 20 learnable scalars, yet improves average R@1 by 2.8 points.
62. 【2611.24366】SCOUT: Risk-Guided Foundation-Model Escalation for Robust Tissue Detection in Whole-Slide Images
链接:https://arxiv.org/abs/2611.24366
作者:Cristian Tommasino,Antonio M. Rinaldi
类目:Computer Vision and Pattern Recognition (cs.CV)
关键词:whole-slide image analysis, Reliable tissue detection, involving pale tissue, high average segmentation, average segmentation accuracy
备注:
点击查看摘要
Abstract:Reliable tissue detection is essential for whole-slide image analysis, yet high average segmentation accuracy can conceal failures involving pale tissue, small fragments, and heterogeneous staining. Pathology foundation models offer transferable representations, but exhaustive inference is computationally demanding and may introduce false-positive tissue predictions. We propose SCOUT (Selective Correction of Uncertain Tissue), a risk-guided cascade that selectively refines an initial thumbnail-based tissue mask. A lightweight audit network estimates the spatial distribution of segmentation errors and prioritizes candidate regions for processing by a frozen pathology foundation model. An asymmetric consensus rule integrates the resulting predictions with the initial mask to recover missed tissue while limiting false positives. A calibrated slide-level review score additionally controls the marginal rate of unflagged case-group failures under the calibration assumptions. Evaluation on a public multi-source, multi-stain benchmark shows that SCOUT reduces the proportion of case groups requiring mask correction from 38.1% for the best released detector to 18.6%, with particularly pronounced improvements in immunohistochemistry. Source-held-out experiments demonstrate better generalization than a segmentation network trained on the same annotations, while selective inference achieves an observed failure rate no higher than processing all candidate regions. Downstream, lymph-node classification with SCOUT masks is non-inferior to that obtained with two of the three released detectors evaluated. A variant built on an openly licensed foundation model is publicly released as an installable Python package with trained weights. These results support risk-guided foundation-model inference as an effective approach to tissue detection across heterogeneous histopathology data.
63. 【2611.24360】owards Pose-Robust Few-Shot Vision-Language-Action Models
链接:https://arxiv.org/abs/2611.24360
作者:Wenhao Li,Zhongyi Han,Yuhao Wei,Liang Han,Yilong Yin,Liqiang Nie
类目:Robotics (cs.RO); Artificial Intelligence (cs.AI); Computer Vision and Pattern Recognition (cs.CV)
关键词:resulting policy silently, policy silently assumes, robot base stay, wrong place, dozen demonstrations
备注: 17 pages, 13 figures, 17 tables (including supplementary material). Project page: [this https URL](https://peacelwh.github.io/WACO/)
点击查看摘要
Abstract:Fine-tuning a vision-language-action (VLA) model on a few dozen demonstrations is the standard way to put a manipulation policy on a robot, yet the resulting policy silently assumes that the camera and the robot base stay exactly where they were during data collection: a bumped tripod or a base re-docked a few centimetres away makes it reach for the wrong place. We show that both failures are one phenomenon, a change of the camera-to-base extrinsic that the policy has memorised, and that the robot carries everything needed to undo it. We introduce WACO, a world-anchored canonicalization framework for VLA models: the arm, placed by forward kinematics at the measured joint angles, serves as a metric calibration target, and the static scene serves as the world anchor that tells a moved camera from a moved robot. From a single RGB-D frame, WACO estimates both shifts without ground-truth extrinsics or extra data, canonicalises the image to the calibration viewpoint and the state, action and start pose to the calibration base frame, and fine-tunes the unmodified policy under the artefacts of its own estimator. We also introduce LIBERO-PoseShift, the first benchmark for RGB VLAs with graded camera, base and compound shifts. Across LIBERO-PoseShift, the LIBERO-Plus camera track and four real-world tasks on a UR5e robot, WACO consistently outperforms the standard few-shot recipe and prior viewpoint-robust methods, raising shifted success by 13.2 points in simulation and by 36.7 points on the real robot, recovering 90% of the simulation gap to a ground-truth oracle and 87% of the real-robot gap, and establishing a new state of the art among methods that use no perturbed training data, with no test-time calibration. Project page: this https URL.
64. 【2611.24352】OCTR2A: Real-Time RGB-to-BEV Trajectory Projection and Forecasting at Urban Intersections
链接:https://arxiv.org/abs/2611.24352
作者:Emanuele Iacobelli,Simone Teglia,Francesco Pro,Claudia Melis Tonti,Irene Amerini
类目:Computer Vision and Pattern Recognition (cs.CV); Artificial Intelligence (cs.AI); Machine Learning (cs.LG)
关键词:practice requires costly, requires costly resources, assuming that reliable, studied assuming, practice requires
备注:
点击查看摘要
Abstract:Trajectory forecasting at urban intersections is often studied assuming that reliable bird's-eye-view (BEV) motion histories are available, which in practice requires costly resources such as LiDAR, aerial views, HD maps, or calibrated infrastructure. We address this upstream perception-to-forecasting problem by reconstructing and predicting agents' BEV trajectories only from paired opposite-corner RGB cameras mounted on existing urban infrastructure at variable heights at inference time. We introduce the Opposite-Corner Trajectory Reconstruction and Anticipation Architecture (OCTR2A), a modular pipeline integrating per-view detection with class refinement, geometric cross-camera matching, transformer-based BEV projection, and egocentric GRU forecasting, trained in cascade and mainly supervised with a soft multi-scale spatial-overlap loss. Since no public benchmark provides this sensing configuration, we introduce R3ST++, a Blender-based synthetic benchmark built from SIND trajectories that provides synchronized opposite-corner views, cross-view correspondences, and BEV ground truth, while also serving as a deployment-support simulator for site-specific fine-tuning. Experiments on these datasets show that, with 12 observed and 12 predicted frames at 10\,Hz, OCTR2A achieves projection~-~forecasting ADE/FDE of 0.144/0.140~-~0.154/0.241~meters in real time on a single consumer GPU. Evaluated on ground-truth BEV histories, the forecasting module improves over the strongest state-of-the-art methods on SIND by 66.7\%/41.7\% ADE/FDE. Transfer experiments on TUMTraf-V2X show that R3ST++ pretraining supports adaptation to real-world roadside domains with different sensing configurations.
65. 【2611.24330】Can Weight-Space Programming Alter Learned Ethical Priorities?
链接:https://arxiv.org/abs/2611.24330
作者:Kai Yi Ji
类目:Computer Vision and Pattern Recognition (cs.CV); Artificial Intelligence (cs.AI); Machine Learning (cs.LG)
关键词:benefiting more people, trades off honoring, honoring a commitment, commitment against benefiting, fitted choice weights
备注:
点击查看摘要
Abstract:We investigate whether a weight edit can alter how a model trades off honoring a commitment against benefiting more people. In a concept-supervised text model, we alter fitted choice weights so that the benefit required to override a formal pledge is larger than for an ordinary promise. Across six independently trained text readers, the edit selects the larger group in both orders on 100% of targeted ordinary conflicts and 83.3% of targeted formal conflicts. It preserves the source model's success rates on smaller conflicts and its factual accuracy. The remaining formal-conflict failures coincide with errors reading a beneficiary number. A separate classifier demonstrates why this distinction matters: reusing a factual answer passes an unconditional choice test but fails to preserve small-gap promises. In a YOLO11n pilot on COCO128, a bias edit increased small-target matches from 3/43 to 9/43, and an equivalent score calibration reproduced the retained detections. These exploratory results show a way to test selective changes to learned priorities in a model with an exposed trade-off mechanism. Whether the method can discover such changes in less structured language models remains open.
66. 【2611.24327】Pixel2Perspective: A Benchmark for Multi-Modal and Multi-View Generalization in Vision-Language Models
链接:https://arxiv.org/abs/2611.24327
作者:Aditya Gandhamal,Aniruddh Sikdar,Arya Kishor,Suresh Sundaram
类目:Computer Vision and Pattern Recognition (cs.CV)
关键词:Visual Language Models, autonomous vehicles, improve navigation, drones and autonomous, navigation by enhancing
备注: 16 pages
点击查看摘要
Abstract:Embodied AI systems, such as drones and autonomous vehicles, improve navigation by enhancing perception in environments containing unseen objects. While Visual Language Models (VLMs) offer strong image understanding, their real-world application is underexplored. For effective deployment, open-vocabulary semantic segmentation (OVSS) models, leveraging models like CLIP to assign pixel-level labels from text, must be robust to outdoor conditions, adverse weather, and varying viewpoints. Despite their potential, these models encounter significant challenges in cross-domain generalization, hindering their practical use in embodied AI applications. In this work, we present Pixel2Perspective, a benchmark for multi-angle object localization and open-vocabulary segmentation across both aerial and ground perspectives. This benchmark supports thorough evaluation across different viewpoints and sensor modalities. We assess foundational models like CLIP and advanced OVSS models using a curated dataset that spans varied outdoor conditions, sim-to-real scenarios, and multiple sensors, including infrared, to support the development of robust and generalizable perception systems. We also present a novel framework, Multi-Source RGB-IR Uncertainty Quantification (MSRIQ), which jointly models aleatoric and epistemic uncertainties across RGB and infrared imagery. This approach supports robust evaluation of open-vocabulary segmentation models across diverse environmental conditions. Our work sets a robustness-driven benchmark, providing key insights and laying the foundation for future research in open-world embodied AI perception. The benchmark and project page are available at this https URL
67. 【2611.24316】Fuse to Recognize the Unseen: Class-Relative Infrared-Visible Fusion for Open-Vocabulary Semantic Segmentation
链接:https://arxiv.org/abs/2611.24316
作者:Zehao Gao,Jie Feng,Wenda Zhao,Junpeng Zhang,Ronghua Shang,Weisheng Dong
类目:Computer Vision and Pattern Recognition (cs.CV); Artificial Intelligence (cs.AI)
关键词:enhance visual perception, Visible-infrared image fusion, Visible-infrared image, visual perception, challenging illumination
备注: 11 pages, 6 figures
点击查看摘要
Abstract:Visible-infrared image fusion leverages complementary thermal cues to enhance visual perception, particularly under challenging illumination. However, such perceptual complementarity does not readily translate into semantic gains in open-vocabulary semantic segmentation (OVSS), where CLIP-based models are predominantly pretrained on RGB imagery. Directly introducing IR information may instead disrupt RGB-language alignment, turning promising thermal cues into semantic interference that undermines unseen-class recognition. Our pixel-level RGB-IR semantic analysis reveals that class-relevant thermal cues can correct certain RGB misclassifications while preserving reliable RGB predictions, motivating RGB-anchored, class-relative thermal cue selection. Based on this insight, we propose EmergeFuse, an infrared-visible fusion framework for OVSS that selectively converts complementary thermal evidence into unseen-class semantic corrections. Specifically, class-relative cue selection evaluates RGB-IR semantic responses and calibrated thermal evidence relative to RGB predictions, generating class-relative correction requests through competitive selection. Text-contrast image rectification then translates these selected requests into semantic rectification directions through text contrasts, guiding evidence-gated feature updates to realize complementary thermal evidence for unseen-class recognition. Experiments on three benchmarks demonstrate gains of up to 4.755% in unseen-class mIoU and 15.730% in strict complementary conversion rate over existing fusion methods, while maintaining competitive fusion quality.
68. 【2611.24296】AnchorUp: Anchoring Feature Upsampling on a Classical Interpolation
链接:https://arxiv.org/abs/2611.24296
作者:Kotaro Horikawa,Keisuke Toida,Junya Komori,Kazuhiro Hotta
类目:Computer Vision and Pattern Recognition (cs.CV)
关键词:Vision foundation models, coarse patch grid, foundation models produce, models produce semantically, produce semantically strong
备注: Accepted to WACV 2027. Code: [this https URL](https://github.com/KotaroHorikawa/AnchorUp)
点击查看摘要
Abstract:Vision foundation models produce semantically strong features on a coarse patch grid, and dense prediction needs them at pixel resolution. Existing learned upsamplers close that gap by regressing the high-resolution feature map from the low-resolution one, typically through cross-attention driven by the input image. We observe that most of what they regress needs no learning: plain bilinear interpolation already reproduces the smooth regions of the target, and recent benchmarks even report bilinear and bicubic on par with several learned upsamplers. The rest is mostly edges and fine details, which suggests that these learned upsamplers spend much of their capacity re-computing the smooth content that classical interpolation provides. We therefore introduce AnchorUp, which stops predicting the high-resolution feature map and predicts only its correction from the interpolation. It keeps a parameter-free classical upsampler as an anchor, obtains a second prediction from a small guidance network, and blends the two through a single learnable scalar gate. Because the anchor supplies the smooth content, the guidance network only has to correct what remains. An analysis of the learned correction confirms that it concentrates on edges and fine details, and so the network can be small: AnchorUp uses 167,425 parameters, a quarter of the budget of the strongest baseline, NAF, and runs at 64% of its latency. Trained once on DINOv3 with only the first of NAF's two training stages, and evaluated on that backbone plus six held-out ones, it is already on par with NAF trained through both stages on Pascal VOC and Cityscapes. Our method surpasses it on DAVIS-2017 video object segmentation; the gain holds on every backbone and is larger in contour accuracy F. Code is available at this https URL.
69. 【2611.24284】CROMPT: Adapting CLIP via Cross-Modal Prompting for Robust Zero-Shot Generalization under Adverse Weather
链接:https://arxiv.org/abs/2611.24284
作者:Aniruddh Sikdar,Arya Kishor,Snehashis Dhibar,Suresh Sundaram
类目:Computer Vision and Pattern Recognition (cs.CV)
关键词:unavailable during training, challenging when target-domain, target-domain images, images are unavailable, Domain adaptation
备注: 17 pages
点击查看摘要
Abstract:Domain adaptation for semantic segmentation is challenging when target-domain images are unavailable during training. Zero-shot domain adaptation addresses this by adapting models using only textual descriptions of the target domain. However, existing prompt-learning methods are not well suited for ResNet-based CLIP vision backbones or pixel-level cross-modal feature alignment. We propose CROMPT, a prompt-learning framework for zero-shot domain adaptation in semantic segmentation. CROMPT introduces Cross-Modal Feature Augmentation (CROMA), combining Random Visual Prompting with Hybrid Textual Prompting (HTP) to jointly adapt visual and textual representations. We further propose CROMA-VL, which leverages vision-language models to generate domain-conditioned prompts, and a progressive easy-to-hard adaptation strategy for improved domain invariance. Experiments on two benchmark datasets demonstrate consistent improvements over strong baselines and state-of-the-art methods, achieving 1.63-2.63% mIoU gains while maintaining inference-time complexity. These results establish CROMPT as an effective and efficient framework for zero-shot multi-domain semantic segmentation. OpenReview: this https URL
70. 【2611.24277】Semantic Guidance for Joint Infrared-Visible Video Fusion and Frame Interpolation
链接:https://arxiv.org/abs/2611.24277
作者:Yushen Xu,Xiaosong Li,Huaming Wu,Dongwei Ren
类目:Computer Vision and Pattern Recognition (cs.CV)
关键词:rates requires recovering, desired output rate, requires recovering missing, requires recovering, desired output
备注: 10 pages, 5 figures, 4 tables
点击查看摘要
Abstract:Infrared and visible video fusion at high frame rates requires recovering missing observations when camera rates differ or fall below the desired output rate. Cascading interpolation and fusion can accumulate errors and cause brightness fluctuations. Motion and occlusion further complicate content selection, particularly when missing visible frames also remove direct semantic guidance. We propose SAFI, which uses semantics at the target time to guide joint interpolation and fusion. When the visible frame rate exceeds the infrared rate, available visible frames provide semantic context directly. At times without visible observations, motion guides local matching, and propagation recovers semantics from neighboring frames. Semantic context complements aligned image features at the target time, while matching confidence helps select reliable infrared and visible information. Reconstruction and fusion share source selection within each scale, allowing supervision on missing frames to guide fusion. To preserve visible appearance and reduce flicker, we bound thermal enhancement and penalize brightness inconsistencies across frames while accounting for changes in thermal contrast. Experiments on three datasets show improved visual quality with less computation than cascaded baselines and consistent temporal gains when input rates differ. SAFI maintains strong downstream tracking performance despite missing frames. The code is available at this https URL.
71. 【2611.24267】Look Closer, Look Back, or Wait for Agentic Streaming Video Understanding
链接:https://arxiv.org/abs/2611.24267
作者:Pengyu Li,Zhitao Gao,Lingling Zhang,Wenxuan Huang,YiCheng Xiao,Chen Zheng,Jun Liu
类目:Computer Vision and Pattern Recognition (cs.CV)
关键词:Streaming video models, latency limits, Streaming video, frames, video models
备注:
点击查看摘要
Abstract:Streaming video models must answer questions about a live video under strict memory and latency limits. Feeding only the latest few frames keeps present perception sharp, whereas adding past frames dilutes attention on the current scene and triggers hallucinations. Recent cascade methods retrieve past frames when uncertain, but they still fail when current frames are blurry or when decisive future clues have not appeared yet. We introduce ActiveForager, a streaming video agent that maintains a clean short working window and gathers extra evidence only on demand. For each query, the agent either answers directly, zooms into high-resolution recent frames, recalls past events from memory, or waits for upcoming frames. When recalling history, a shared-weight sub-agent verifies candidate clips in isolation and returns a compact evidence summary. A stream-decoupled attention mask then keeps historical evidence from interfering with present visual tokens. We train this policy using counterfactual information gain, which rewards a tool call only when the gathered evidence makes the true answer more likely and prevents tool collapse across all actions. Across Qwen2.5-VL-7B and Qwen3-VL-8B on OVO-Bench, ActiveForager improves real-time perception, historical recall, and proactive response at once, lifting overall accuracy on Qwen3-VL-8B from 47.6% to 65.9%. It also doubles zero-shot accuracy on the multi-hop Video-Holmes benchmark while adding minimal visual tokens per query.
72. 【2611.24259】Do Thinking VLMs Need to Keep Looking at Long Videos? Write It Down, Then Look Away
链接:https://arxiv.org/abs/2611.24259
作者:Pengyu Li,Zhitao Gao,Lingling Zhang,Heng Wang,Chengqian Ma,WenHao Wang,Wenxuan Huang,Chen Zheng,Jun Liu
类目:Computer Vision and Pattern Recognition (cs.CV)
关键词:Vision-language models, hundreds to thousands, video, Vision-language, model
备注:
点击查看摘要
Abstract:Vision-language models (VLMs) reason about long videos over hundreds to thousands of tokens. Every decoding step re-reads the key-value (KV) cache of the entire video. Must the model keep looking while it reasons? In LookAway, the model first writes timestamped notes on what the question needs and then puts the video away. Once the notes close, the inference engine hides the video's KV cache for the rest of the response. Across both hybrid and pure-attention VLMs, dependence on the video is concentrated early in the response. On Qwen3.6-27B, writing notes preserves accuracy better than free reasoning at the same withdrawal step. Replacing them with another video's notes harms accuracy, whereas keeping the video visible alongside the model's own notes adds no significant benefit. Without additional training, LookAway nearly matches full-video thinking accuracy at 0.12-0.13 times its modeled decode cost. To shorten this initial look, we fine-tune the model on its own correct LookAway responses. Reinforcement learning then rewards correct answers while penalizing KV reads. The trained model puts the video away after a median of 66-68 tokens. Across three held-out benchmarks, it writes more tokens than an identically trained full-video model yet reads only 27-35% as many KV positions at matched accuracy.
73. 【2611.24256】Spark-H3: Adaptive Block Sparse Attention for Video Diffusion Models
链接:https://arxiv.org/abs/2611.24256
作者:Zecheng Tang,Jiaye Fu,Youtian Lin,Xinyao Rao,Liuxuan Sun,Jian Zhang,Yao Yao,Wei Li
类目:Computer Vision and Pattern Recognition (cs.CV)
关键词:Long video sequences, sequences make attention, video sequences make, Long video, sequences make
备注:
点击查看摘要
Abstract:Long video sequences make attention a substantial cost in video diffusion models. Block sparse attention (BSA) reduces this cost by computing selected blocks exactly while compressing the remaining blocks. However, fixed token grouping can place tokens with dissimilar attention preferences in the same group, while uniform key averaging can underestimate the attention mass of compressed blocks. We introduce Spark-Attn, a training-free BSA method comprising Spark-Reblock and Spark-Reweight. Spark-Reblock recursively groups tokens by attention preference while preserving regular block shapes for efficient GPU execution. Spark-Reweight uses the global mean query as an anchor to construct attention-weighted key-value representations and logit biases for the compressed blocks, calibrating their attention mass. We instantiate Spark-Attn in MiniMax-H3 as Spark-H3. At 10% exact-block density, Spark-H3 delivers a 2.49$\times$ DiT speedup and reaches 24.81 dB PSNR against the corresponding dense outputs on MiniMax-H3 (20 denoising steps, 345 frames, 1344$\times$768). Increasing the density to 20% further improves PSNR to 26.76 dB while still achieving a 2.26$\times$ DiT speedup. These results demonstrate that Spark-H3 achieves a state-of-the-art quality-efficiency trade-off for training-free BSA on video diffusion models.
74. 【2611.24253】One Shot, Any View: High-Fidelity Video Reshooting with Geometry Guidance and Visual References
链接:https://arxiv.org/abs/2611.24253
作者:Kai Huang,Runze Xu,Hui Ren,Taiyi Wu,Xinyuan Li,Xun Gao,Quande Liu,Dong Tang,Tongtong Shen,Xuefeng Su,Alan Zhao
类目:Computer Vision and Pattern Recognition (cs.CV)
关键词:point video, video, costly reshoot, additional angle, angle after production
备注: Technical report. 19 pages, 14 figures, 3 tables
点击查看摘要
Abstract:Filming a scene from an additional angle after production usually requires a costly reshoot. We present One Shot Any View, a system that takes a recorded clip and generates a reshot video of the same performance along a user-specified camera trajectory. As in recent geometry-grounded methods, the system conditions generation on the source video and an explicit 3D prior, namely a point video rendered along the requested trajectory from a feed-forward reconstruction of the source video. We target human-centric cinematic videos, which are often shot with long lenses, shallow depth of field, and low-key lighting, all of which degrade feed-forward reconstruction. Existing geometry-grounded methods tend to inherit the degraded point video, and their output quality drops visibly. Two designs make such point videos usable. First, in addition to rendered videos of 3D assets, we train on real-world multi-camera videos, whose point videos are produced by the same reconstruction pipeline as at inference. This training-inference consistency enables the generator to take only the composition of people and scene elements from the point video, and their appearance from the source video. Second, a geometry-anchored RoPE offset keeps the generated video temporally synchronized with the source while largely preventing the output from copying the degraded appearance of the point video. Meanwhile, since the generator is a pretrained reference-to-video (R2V) model, the system also accepts optional character and scene reference images, which supply the appearance of regions that the source never shows. In both objective and human evaluations, One Shot Any View outperforms existing reshooting methods.
75. 【2611.24247】Allocentric Memory, Egocentric Acting: A Dual-Frame VLA for Urban Aerial Navigation
链接:https://arxiv.org/abs/2611.24247
作者:Haotian Xu,Yue Hu,Zhengqiu Zhu,Tuo Feng,Wenguan Wang,Quanjun Yin
类目:Computer Vision and Pattern Recognition (cs.CV); Robotics (cs.RO)
关键词:Language-goal aerial navigation, aerial navigation requires, Language-goal aerial, collected during flight, reach a destination
备注:
点击查看摘要
Abstract:Language-goal aerial navigation requires a UAV to reach a destination specified in natural language using observations collected during flight. A key challenge is linking changing egocentric perception with spatial evidence anchored to stable locations for long-range navigation. We propose CityNav-VLA, a dual-frame vision-language-action model for urban navigation integrating dynamic egocentric context with persistent allocentric spatial memory. The UAV incrementally accumulates visual observations on a fixed north-up map initialized with geographic and landmark priors, preserving evidence across viewpoint changes. A dynamic query conditioned on the instruction, current observations, UAV state, and motion history retrieves relevant memory evidence to predict a map-frame target. A deterministic controller converts this target into body-frame commands using the UAV's current pose, connecting global reasoning with egocentric execution. To reduce target fluctuations over long horizons, early goal commitment (EGC) fixes the execution target to the coordinate-wise median of the first $K$ predictions without extra trainable parameters. On Test Unseen, CityNav-VLA reduces NE by 21.2% and improves SR and SPL by 15.8% and 30.9%, respectively. It also reduces NE by 6.2% and improves SR and SPL by 5.6% and 23.2%, respectively, relative to the former SOTA after dataset refinement. Cumulative ablations show that navigation-map features yield the largest gains, while metric coordinates further improve endpoint accuracy. With a 4B-scale backbone, inference averages 0.68 s per decision with 8.6 GB peak memory on a single A100, presenting remarkable model efficiency.
76. 【2611.24243】End-to-End Landsat Land Surface Temperature Reconstruction with AlphaEarth Embeddings
链接:https://arxiv.org/abs/2611.24243
作者:Shengjie Liu
类目:Computer Vision and Pattern Recognition (cs.CV)
关键词:Land surface temperature, spatially continuous observations, cloud contamination prevents, Land surface, Landsat LST reconstruction
备注:
点击查看摘要
Abstract:Land surface temperature (LST) data at Landsat's 30 m resolution is important for many applications, but cloud contamination prevents its spatially continuous observations. Here, we propose A$^2$E$^2$, an end-to-end U-Net that integrates annual temperature cycle (ATC), coarse LST from Earth system modeling, and AlphaEarth embeddings for Landsat LST reconstruction. The ATC supplies a physics-guided thermal prior, the coarse LST provides a temporal background, and the U-Net converts the embedding field into day-specific fine-scale correction using observed pixels. Experimental results on four seasonally distributed Landsat 8/9 scenes over New York City in 2022 show that A$^2$E$^2$ achieves 0.78 K mean absolute error and 1.01 K root mean squared error, reducing error by 60.1% relative to baseline method without spatial correction and by 3.7% relative to the same U-Net guided by annual mean spectral reflectance. Sensitivity analysis shows stable performance regardless of distance to clear pixels and percentage of blocked pixels. Independent validation with ECOSTRESS LST shows that Earth foundation model embeddings have better performance than annual mean spectral bands for LST reconstruction.
77. 【2611.24240】ViLFlow: Parallel Vision-Language Generation with Continuous Latent Flows
链接:https://arxiv.org/abs/2611.24240
作者:Kevin Zhai,Bian Sun,Zhenyi Wang,Song Wang
类目:Computer Vision and Pattern Recognition (cs.CV); Machine Learning (cs.LG)
关键词:discrete diffusion models, predominantly relied, discrete diffusion, diffusion models, diffusion models typically
备注:
点击查看摘要
Abstract:Vision-language generation has predominantly relied on autoregressive and discrete diffusion models. Autoregressive models generate tokens sequentially, while discrete diffusion models typically predict tokens independently within each denoising step. We propose ViLFlow (Vision-Language Flow), a continuous-flow vision-language model that combines joint modeling of output embeddings with parallel token decoding. ViLFlow adapts pretrained continuous language generation to visual question answering and image captioning through learned image guidance and prediction-aware token decoding. A visually adapted teacher provides image-guided predictions as additional denoising supervision. The denoiser learns to reproduce these predictions directly, retaining one image-conditioned prediction per generation step. We train the token decoder on denoiser predictions and paired reference text to improve the conversion of continuous outputs into words. The decoder can be adapted independently, improving answering and captioning without further denoiser training. For captioning, it also supplies text feedback during late denoising, allowing decoded words to inform subsequent refinement. ViLFlow outperforms autoregressive and masked-diffusion baselines by up to 8.46 percentage points on VQAv2 and 4.50 points on GQA without GQA training. On COCO Karpathy test, it improves CIDEr by 16.25 over masked diffusion and 17.81 over sampled autoregression.
78. 【2611.24233】Psychometrics-Inspired Coreset Selection for Image Quality Assessment
链接:https://arxiv.org/abs/2611.24233
作者:Susu Jiang,Xuelin Shen,Baoliang Chen
类目:Computer Vision and Pattern Recognition (cs.CV)
关键词:Collecting large-scale subjective, Collecting large-scale, large-scale subjective annotations, subjective annotations remains, training accurate image
备注:
点击查看摘要
Abstract:Collecting large-scale subjective annotations remains a critical bottleneck in training accurate image quality assessment (IQA) models. Coreset selection reduces this burden by selecting representative images before annotation. However, diversity in a visual feature space does not necessarily reflect diversity in perceptual quality, raising a fundamental question about which quality-related factors the representation should encode. We propose GRAM, a quality-aware representation learning framework inspired by the graded response model (GRM) from psychometrics. GRM describes how an individual's latent ability determines the probabilities of reaching successive assessment grades, with response sensitivity and grade thresholds governing this relationship. In this paper, we find that this principle naturally aligns with perceptual quality assessment, where the impact of image degradation depends on image content and distortion characteristics. Building on this connection, GRAM models quality judgments through three complementary factors: latent quality, content-dependent response sensitivity, and distortion-dependent grade thresholds. These factors jointly determine the probabilities of attaining different quality grades and the resulting expected quality score. The learned parameters provide a structured quality-aware representation that complements general visual features for coreset selection. Experiments on nine datasets demonstrate improvements across different selection strategies, downstream IQA models, and annotation budgets. The code will be made publicly available.
79. 【2611.24223】raining Denoising Models of Any Architecture Without Stored Images
链接:https://arxiv.org/abs/2611.24223
作者:Christos Koutlis,Ioannis Kompatsiaris,Symeon Papadopoulos
类目:Computer Vision and Pattern Recognition (cs.CV)
关键词:pretrained denoising model, denoising model replace, pretrained denoising, denoising model, model replace
备注:
点击查看摘要
Abstract:Can a pretrained denoising model replace the dataset used to train it? We investigate this question through teacher-supervised training, showing for the first time that continuous-time diffusion and flow models of any architecture and size can be trained from random initialization without stored images. Our students regress the teacher's velocity on a rolling pool of noisy samples, advanced along trajectories of random step counts to obtain independent, dense and approximately continuous noise levels. This saves considerable storage for large datasets and applies even when the data are unreleased or private. Our experiments include 12 teachers of 6 families and 6 datasets. Our method outperforms standard training of the same network on CIFAR-10 for students 4x smaller than the teacher, comes within 0.75 to 1.92 FID of it at the teacher's size and surpasses training on teacher-generated images and step reduction methods without teacher-based initialization. Our code is available at this https URL.
80. 【2611.24222】arget-Consistent Action Guidance via Semi-Unbalanced Optimal Transport for Drifting World Models
链接:https://arxiv.org/abs/2611.24222
作者:Md Abtahi Majeed Chowdhury,Md Rifat Ur Rahman,Md Sakib Hossain Shovon,Jaesik Choi,Minhyuk Sung
类目:Computer Vision and Pattern Recognition (cs.CV)
关键词:offline policy evaluation, Action-conditioned world models, diffusion makes expensive, multi-step diffusion makes, policy evaluation
备注:
点击查看摘要
Abstract:Action-conditioned world models let robots imagine the outcomes of their actions for planning and offline policy evaluation, but these uses need many rollouts, which multi-step diffusion makes expensive. Drifting models move the iteration from inference to training and generate samples in one network pass. DriftWorld brought this paradigm to action-conditioned world models; to strengthen action following, it repels predictions from the current frame, a construction borrowed from classifier-free guidance. We show that this accentuation has a target bias: the push persists even when a prediction matches the recorded future, in kernel, balanced, and unbalanced Wasserstein drifting fields alike. We introduce Mass-Share Accentuation (MSA) to remove this bias. MSA treats the current frame as a counterfactual destination competing with the recorded future in a semi-relaxed optimal transport problem; subtracting the closed-form share of the correct future makes the guidance vanish at the target, and the remainder is shared by all predictions. We prove that the recorded future is the unique fixed point along the motion direction for collapsed predictions, and that the guidance fades with prediction error. On Push-T, Robomimic, and BridgeV2, where trained models under-react rather than overshoot, MSA improves action ranking, response slope, and moving-region error over latent-space baselines at no inference cost, though these benchmarks do not isolate bias removal as the cause.
81. 【2611.24212】DetailScale-SR: Decoupling Restoration and Continuous Detail Control for Real-World Super-Resolution
链接:https://arxiv.org/abs/2611.24212
作者:Shengchuan Gao,Zhongjie Duan,Yingda Chen,Jingjing Chen
类目:Computer Vision and Pattern Recognition (cs.CV)
关键词:Real-world image super-resolution, Real-world image, detailed plausible reconstruction, super-resolution often seeks, detailed plausible
备注:
点击查看摘要
Abstract:Real-world image super-resolution often seeks the most detailed plausible reconstruction, although more detail is not always desirable. The same degraded image may require restrained restoration, faithful reconstruction, or enhanced visual detail according to user intent. We present DetailScale-SR, a preference-conditioned framework that continuously controls reconstructed detail for a fixed low-resolution input without altering the scene being restored. Rather than relying on post-processing or repeatedly rewriting prompts, it learns a continuous family of reconstructions while separating content recovery from detail preference. This enables a smooth transition from conservative restoration to richly textured results under consistent input and semantic constraints. Its controllable range also extends beyond conventional restoration: DetailScale-SR can enhance high-frequency structures and generate perceptually richer textures than the paired high-resolution reference. Text restoration is another distinct strength. For text-rich scenes, especially Chinese posters and signs, it recovers more legible and character-faithful text than prior generative SR methods under matched textual guidance. DetailScale-SR therefore unifies continuous detail control, beyond-reference enhancement, and reliable recovery of semantically important content. Our model is available at this https URL
82. 【2611.24200】SpiderMark: Hybrid Watermarking for Generative AI via Algorithmic Injection and Learning-Based Verification
链接:https://arxiv.org/abs/2611.24200
作者:Chihcheng Hsieh,Yang Song,Flora Salim,Martin Bell,Jacinto C. Nascimento,Catarina Moreira
类目:Computer Vision and Pattern Recognition (cs.CV); Artificial Intelligence (cs.AI)
关键词:diffusion-based generative models, proliferation of diffusion-based, reliable methods, methods to trace, trace and verify
备注:
点击查看摘要
Abstract:The proliferation of diffusion-based generative models has intensified the need for reliable methods to trace and verify the provenance of AI-generated images. Existing watermarking approaches for diffusion models typically rely on isotropic frequency patterns and handcrafted detection metrics, which are prone to directional vulnerability and threshold instability across diverse image transformations. We propose SpiderMark, a hybrid watermarking framework that combines algorithmic frequency-domain injection with learning-based probabilistic verification, requiring no modifications to pretrained diffusion model weights. SpiderMark embeds a structured spider-web pattern, comprising radial spokes and concentric polygonal rings, into the Fourier representation of the initial latent noise, and trains a lightweight ResNet-18 verifier augmented with auxiliary similarity cues (PSNR and L1) to produce a continuous watermark confidence score for watermark detection. Across nine evaluation conditions, SpiderMark achieves the highest fixed-threshold Accuracy in eight, while retaining strong detection on unseen COCO prompts without verifier retraining. CLIP similarity and FID measurements show that these detection gains are accompanied by small measured differences in prompt alignment and distributional image quality. SpiderMark establishes that structured watermark geometry and learned probabilistic verification can be unified into a single, practical framework, bridging the gap between imperceptibility, robustness, and real-world verifiability for generative AI content attribution.
83. 【2611.24197】From Neural Tokens to Language: Dual-View EEG Semantic Bridging for Brain-to-Text Generation
链接:https://arxiv.org/abs/2611.24197
作者:YaoYang Liu,Jun Dai,ZhuoShi Tang,Weijun Sun,Xiaozhao Fang,Jie Wen
类目:Computer Vision and Pattern Recognition (cs.CV)
关键词:produce natural-language descriptions, Visually evoked, neural responses elicited, aims to produce, produce natural-language
备注: 30 pages, 15 pages of supplementary material
点击查看摘要
Abstract:Visually evoked EEG-to-text generation aims to produce natural-language descriptions from neural responses elicited by visual stimuli. Existing approaches typically compress EEG signals into a single global representation and directly project it into the hidden space of a large language model (LLM), which may limit the preservation of complementary visual semantics under noisy and subject-dependent EEG signals. We propose a dual-view semantic token bridging framework that exploits raw images and sketch images as complementary visual supervision to learn raw-oriented and sketch-oriented EEG semantic token streams. To interface these structured neural representations with an LLM, we introduce a branch-specific semantic query bridge that aggregates each token stream into compact semantic slots and converts them into LLM-compatible soft prompts. A progressive adaptation strategy further transitions the learned visual-semantic interface from image-supported training to image-free EEG-conditioned generation. Experiments demonstrate strong subject-wise EEG classification performance and competitive EEG-to-text generation across multiple LLM backbones. Ablation, branch-level representation, and EEG-label disentanglement analyses further show that the two EEG streams learn complementary view-oriented representations and that EEG-derived prompts provide sample-specific information beyond category labels.
84. 【2611.24193】WM^MLLM: Turning MLLMs into World Models
链接:https://arxiv.org/abs/2611.24193
作者:Baiang Li,Enze Xie,Jincheng Yu,Jin Wang,Haozhe Liu,Yitong Li,Juanxi Tian,Felix Heide
类目:Computer Vision and Pattern Recognition (cs.CV)
关键词:Video world models, drift or disappear, world model, world model built, interactive world model
备注: 25 pages, 14 figures, 8 tables; project page: [this https URL](https://ztmotalee.github.io/wmmllm/)
点击查看摘要
Abstract:Video world models keep the world in the context of a generator, so a revisit is synthesized again and earlier edits can drift or disappear. We present WM^MLLM, the first interactive world model built on multimodal large language models (MLLMs), for persistent, editable world generation from a single image. WM^MLLM keeps the world state outside every generator, as a panorama with a revision history, and lets the MLLMs write it: they turn each request into operations on this state, select where a local edit is written, and check every candidate before it becomes the next state. Image generators only propose views, and video only presents them. Users can walk to new places and return, edit objects through text or a box, place subjects, and change lighting, weather or season. By construction, a revisit at the same revision reproduces the earlier view exactly, a local edit leaves every panorama pixel outside its region unchanged, and each request starts from the result of the previous one. Against six world model baselines on MBench-A, WM^MLLM has the highest mean on all four environment-recovery metrics (Epipolar 91.5 versus 0.1 to 74.0) and completes the most camera round trips (85.3% versus at most 50.9%). It has the best value in all 15 CompBench columns against 15 baselines. On 112 local edits judged blind, selecting the region to write after generation raises completion from 34.8% to 83.0%. On 258 WBench and MBench-A items, all 46 revisits recover their earlier views, and all 97 local edits that pass the check leave panorama pixels outside their region unchanged.
85. 【2611.24192】rajWAM: Trajectory-Conditioned Future Learning for Vision-Language Driving
链接:https://arxiv.org/abs/2611.24192
作者:Xuerui Fang,Gulimila Kezierbieke,Chen Wangxing,Hui Li
类目:Computer Vision and Pattern Recognition (cs.CV); Robotics (cs.RO)
关键词:World Action Models, World Action, Action Models, connect future scene, action planning
备注: 22 pages, including supplementary material. Code and model weights: [this https URL](https://github.com/FangWD520/TrajWAM)
点击查看摘要
Abstract:World Action Models (WAMs) connect future scene prediction with action planning for autonomous driving. Each driving record pairs an executed ego trajectory with its observed future, providing a factual outcome for learning this connection. We present TrajWAM, a trajectory-conditioned WAM that improves vision-language driving through auxiliary bird's-eye-view (BEV) future learning. An ego-trajectory query conditions DINOv3-aligned BEV predictions from shared planner representations. Training combines factual reconstruction under the executed trajectory with one-sided feature separation under an unlabeled perturbed trajectory. Together, these objectives train the shared planner to generate driving waypoints. Native inference directly generates waypoints and bypasses the auxiliary future-prediction branch. On all 220 Bench2Drive routes, TrajWAM achieves 91.53 Driving Score and 75.45% Success Rate, exceeding a schedule-matched planning-only fine-tuning baseline by 5.31 Driving Score points and eight successful routes.
86. 【2611.24190】Monitoring Unplanned Construction Using Satellite Change Detection
链接:https://arxiv.org/abs/2611.24190
作者:Harsh Kashyap,Taranum Wasu
类目:Computer Vision and Pattern Recognition (cs.CV)
关键词:Cities, public services, Cities need land, NITI Aayog, strain public services
备注: 22 pages, 11 figures, 5 tables
点击查看摘要
Abstract:Cities need land for homes, shops, roads, and public services. New construction changes that land. It can harm natural areas and water bodies, add to waste, and strain public services. NITI Aayog's 2021 report stated that 65% of 7,933 urban settlements in India had no master plan. That number tells us about planning coverage at the time. It does not tell us whether any single building was authorised. Artificial intelligence, which lets computers learn patterns from data, can help find physical changes by comparing satellite images taken on different dates. We test simple methods on two public datasets from outside India. First, we turn each image into a set of numbers using a pretrained neural network, a model already trained to recognise patterns in images. We feed those numbers to standard machine-learning classifiers that predict land categories and physical changes. The methods reach about 95 percent accuracy at spotting built-up land and about 83 percent accuracy at saying whether a small pair of images shows change, where each pair gets one label. We also train a small neural network on a standard change-detection benchmark. Separately, we run a preliminary descriptive check on an Agra sample of 48 patches, 7 of them positive. The results show that satellite images carry information about physical change. They do not show whether construction was planned or permitted.
87. 【2611.24184】VideoACE: Benchmarking Agentic Video Understanding through High-Frame-Rate Actions
链接:https://arxiv.org/abs/2611.24184
作者:Luoyuan Zhang,Xuanyu Zheng,Yancheng Long,Haoyang Xu,Bin Wen,Tingting Gao
类目:Computer Vision and Pattern Recognition (cs.CV); Artificial Intelligence (cs.AI); Multimedia (cs.MM)
关键词:actively interact, Agentic video understanding, understanding, video understanding, video
备注:
点击查看摘要
Abstract:Agentic video understanding requires models to actively interact with video environments to gather evidence and form judgments. Evaluating this capability therefore requires assessing whether models can translate opportunities for interaction into better understanding. We introduce VideoACE, a benchmark for assessing video understanding under conditions that support sustained interaction. We focus on high-frame-rate action videos, where rapid, subtle changes make targeted observation especially valuable. The benchmark comprises 1,000 multiple-choice questions spanning six categories of fine-grained action understanding, accompanied by temporal evidence annotations and user hints screened for answer leakage. Its evaluation environment combines video inspection tools with a user simulator, allowing models to adjust temporal coverage, sampling density, and spatial detail, and to revisit evidence and revise their judgments under simulated user feedback. We evaluate single-pass observation, autonomous tool use, and tool use combined with user feedback to examine how effectively models exploit these opportunities. Evaluation of five models reveals distinct patterns in how they acquire evidence and respond to opportunities for further interaction. These findings establish VideoACE as a testbed for examining agentic video understanding beyond performance under a fixed observation setting.
88. 【2611.24182】ART: Bridging the Embodiment Gap in 3D Human-to-Robot Affordance Transfer
链接:https://arxiv.org/abs/2611.24182
作者:Zichen Zhang,Wei Zhai,Xuji Zhang,Yang Cao,Yu Kang
类目:Computer Vision and Pattern Recognition (cs.CV)
关键词:critical link, link between perception, affordance, Paired Affordance Dataset, human priors
备注: 15 pages, 14 figures, 5 tables. Project page: [this https URL](https://derek0907.github.io/ART/)
点击查看摘要
Abstract:3D affordance acts as a critical link between perception and action, providing priors for robotic manipulation. While existing approaches predominantly leverage human priors as direct references, they often overlook that affordance is not an object-intrinsic property but is inherently coupled with the agent's embodiment. Due to the structural and morphological disparities between human hands and robot end-effectors, directly applying human priors causes severe incompatibility for robotic execution. Motivated by this, we introduce a novel task setting, 3D Human-to-Robot Affordance Transfer, which addresses the affordance misalignment arising from distinct embodiment structures. To support this task, we construct the Human-Robot Paired Affordance Dataset (HPAD), the first large-scale 3D affordance benchmark in which every object is annotated with both human-prior and simulation-validated robot labels. HPAD covers 2,137 objects across 80 categories and 13 atomic action primitives, yielding over 27,000 object-action pairs. On top of HPAD, we propose the 3D Affordance tRansfer neTwork (ART). ART first filters geometric false positives in the human prior via a reliability gating stream, then reconstructs the prior into a robot-actionable distribution through hierarchical reconstruction and a graph-diffusion refinement that resolves the topological mismatch between human and robot affordances. Experiments on HPAD demonstrate the reliability of the proposed task and the superiority of ART. We further validate ART on a 6-DoF arm with a parallel-jaw gripper in a 2,400-execution benchmark across 12 objects and 4 action categories; ART attains the best success rate (86.5% over its 480 trials), confirming the practicality of the transfer paradigm. Project page: this https URL
89. 【2611.24171】Efficient World-Model Planning via Physics-Guided Search Distillation
链接:https://arxiv.org/abs/2611.24171
作者:Hongyu Zhang,Xiaoshuai Hao,Heye Huang,Qingwen Meng,Wenjun Shi,Jianao Xu,Xin Hu,Xuanru Jin,Jingyi Zhang,Baicang Guo
类目:Robotics (cs.RO); Computer Vision and Pattern Recognition (cs.CV)
关键词:World models support, models support autonomous, support autonomous driving, World models, evaluating candidate trajectories
备注:
点击查看摘要
Abstract:World models support autonomous driving by evaluating candidate trajectories through predicted future scene evolution. However, accurate future prediction does not ensure road-compliant execution; when all candidates violate road boundaries, re-ranking cannot select a feasible trajectory. We propose Physics-Guided Search Distillation (PGSD), a single-view planning framework that combines future representations and execution feedback for verified beyond-pool repair through three components: (1) Execution-Aware Selection checks ego-vehicle road compliance under simulated execution and selects the highest-scoring feasible trajectory; (2) a Clearance-Aware Search Teacher generates beyond-pool corrections and selects demonstrations with larger minimum road-boundary clearance during execution to buffer imitation errors; and (3) a Future-Conditioned Repair Student predicts one correction from rewarder features and execution feedback for single-rollout verification. The pretrained planner and rewarder remain frozen; no future scenes are generated online. On NAVSIM v1, single-view PGSD achieves 90.97 PDMS, exceeding all compared prior single-view planners and retaining 73.2% of the search teacher's PDMS gain over execution-aware selection. On NAVSIM v2 navhard, it reaches 38.96 EPDMS, improving on WorldDrive by 11.24 points (40.5%). The student replaces up to ten search rollouts with a single verification rollout per repair trigger. HUGSIM Hard examples provide qualitative evidence of zero-shot transfer.
90. 【2611.24170】ViStep-RL: Stepwise Reinforcement Learning for Long-Horizon Visual Agents
链接:https://arxiv.org/abs/2611.24170
作者:Junyang Chen,Yao Qin,Xiao Li,Ningchao Wang,Yiguo Qiao,Wenbin Li,Xin Geng
类目:Computer Vision and Pattern Recognition (cs.CV)
关键词:Long-horizon visual agents, Long-horizon visual, inspect images, task unfolds, remains usable
备注: 19 pages, including supplementary material
点击查看摘要
Abstract:Long-horizon visual agents must do more than inspect images: they must turn observations into evidence that remains usable as a task unfolds. Existing active-perception methods teach agents where and how to look, while a final-answer reward gives little structure for deciding which observations satisfy dependent subgoals or should be retained for later use. We present ViStep-RL (Visual Stepwise Reinforcement Learning), which models execution as a dependency-linked graph of evidence-bearing subgoals. A verifier tracks which subgoals are supported by valid evidence, and changes in graph potential provide a net-progress learning signal; an active memory protocol preserves evidence across turns and links retrieval to downstream use. On ViInteract, ViStep-RL reaches 60.0% strict completion with Qwen2.5-VL-7B, compared with 10.0% for the base model and 46.5% after supervised fine-tuning. Controlled studies show that incremental task-state rewards outperform cumulative rewards by 3.2 points, while active evidence writing improves strict completion by 7.75 points over automatic writing. Under a common interaction framework, the task-trained ViStep-RL policy achieves 59.5% answer accuracy, 24.0 points above the strongest of six general-purpose reference models. Held-out episodes confirm the gain at both Qwen scales, and gains extend to three external visual reasoning benchmarks.
91. 【2611.24164】Gaussian Primitive Fusion for Any-Resolution Multimodal Images
链接:https://arxiv.org/abs/2611.24164
作者:Baoshuo Cai,Xinrui Jiang,Bing Cao
类目:Computer Vision and Pattern Recognition (cs.CV); Artificial Intelligence (cs.AI)
关键词:Multimodal image fusion, fusion integrates complementary, Multimodal image, integrates complementary observations, integrates complementary
备注: 15 pages, 8 figures
点击查看摘要
Abstract:Multimodal image fusion integrates complementary observations from different modalities into a unified representation. Existing methods typically estimate modality contributions on a common discrete grid. When source modalities have different native resolutions, constructing such a grid generally involves resampling, which may introduce artifacts and affect subsequent fusion decisions. To address this problem, we propose Gaussian Primitive Fusion (GPF), which reformulates heterogeneous-resolution fusion in a shared continuous Gaussian space. Instead of first converting source images to a common resolution, GPF directly fits their native-resolution observations with shared Gaussian geometry and modality-specific content representations, and learns modality contributions within the resulting continuous representation. Once optimized, the fused representation is rendered at the finest native resolution without resampling the sources. With heterogeneous-resolution inputs, GPF produces clearer target contours and finer textures, with less blurring and blocking artifacts than resample-then-fuse pipelines. Quantitatively, it achieves the best results on multiple infrared-visible benchmarks against state-of-the-art fusion methods equipped with interpolation or super-resolution frontends.
92. 【2611.24163】Stabilizing Neural-to-Image Retrieval with Frequency-Domain Priors and Adaptive Supervision
链接:https://arxiv.org/abs/2611.24163
作者:YaoYang Liu,ZhuoShi Tang,Haopeng Lai,Weijun Sun,Xiaozhao Fang,Jie Wen
类目:Computer Vision and Pattern Recognition (cs.CV)
关键词:brain computer interfaces, offering important insights, enabling transformative applications, infer perceived visual, perceived visual stimuli
备注: 12 pages
点击查看摘要
Abstract:Visual neural decoding aims to infer perceived visual stimuli from brain activity, offering important insights into human visual cognition while enabling transformative applications in brain computer interfaces and artificial intelligence. Recent advances in pretrained vision-language models have provided a semantically rich representation space for this task. However, reliable alignment between EEG/MEG signals and visual embeddings remains challenging because neural responses are noisy, highly variable across trials, and subject dependent. To address this issue, we propose a unified neural-to-image retrieval framework that improves retrieval performance by aligning neural and image representations in a shared semantic space. The framework integrates dual-stream adaptation, frequency-domain calibration, and phase-guided temperature modulation to promote more robust cross-modal semantic alignment. Specifically, dual-stream adaptation models a low-dimensional shared subspace with correlation regularization to suppress redundant noise and enhance cross-modal consistency. Frequency-domain calibration conditionally modulates neural representations using amplitude and phase statistics. Phase-guided temperature modulation further adjusts the supervision strength in contrastive learning according to the stability of neural responses. Experiments on THINGS-EEG and THINGS-MEG under both intra-subject and inter-subject settings demonstrate that the proposed framework consistently outperforms strong baselines. These results suggest that robust neural-to-image retrieval under heterogeneous neural conditions benefits from a stable shared semantic space, frequency-informed representation calibration, and adaptive supervision. Code will be released at this http URL.
93. 【2611.24161】AVTR: Adaptive Multimodal Reasoning with Visual Imagination
链接:https://arxiv.org/abs/2611.24161
作者:Xiaoqian Shen,Yunyang Xiong,Changsheng Zhao,Saksham Suri,Chong Zhou,Zechun Liu,Zhipeng Cai,Zhuang Liu,Qi Qian,Wei Wen,Hongyu Xu,Hanxian Huang,Ernie Chang,Yanbei Chen,Vikas Chandra,Raghuraman Krishnamoorthi,Mohamed Elhoseiny,Lemeng Wu
类目:Computer Vision and Pattern Recognition (cs.CV)
关键词:advanced multimodal tasks, solve advanced multimodal, Vision-Language Models increasingly, Models increasingly rely, multimodal tasks
备注: Project page: [this https URL](https://xiaoqian-shen.github.io/AVTR)
点击查看摘要
Abstract:Vision-Language Models increasingly rely on textual reasoning to solve advanced multimodal tasks. Despite its effectiveness, this paradigm faces two fundamental limitations. First, relying on textual reasoning alone can limit perceptual ability in vision-centric questions, where accurate answers depend on interpreting task-relevant visual evidence. Second, applying textual reasoning uniformly across inputs overlooks question-dependent variation in both reasoning modality and depth, often producing verbose and unnecessary traces when little logical inference is required. To this end, we propose Adaptive Visual-Textual Reasoning (AVTR), which introduces visual imagination as a complementary reasoning modality to textual thinking, enabling the model to revisit its visual observations and distill the salient evidence through generative visual reflection. Furthermore, we formulate reasoning-modality selection as a learned routing problem, allowing a single policy to adaptively harness the complementary strengths of visual reflection and textual reasoning in response to the perceptual and inferential demands of the input. Across 16 benchmarks, AVTR improves the average score by 4.5 points over Qwen3.5 baseline, while reducing inference latency by 33\%. It also scales effectively into a smaller size backbone, outperforming the evaluated 2B-scale baselines.
94. 【2611.24157】GenRF: Generative Registration-free Image Fusion
链接:https://arxiv.org/abs/2611.24157
作者:Linfeng Tang,Zhijun Fan,Yang Chen,Zizhuo Li,Hao Zhang,Jiayi Ma
类目:Computer Vision and Pattern Recognition (cs.CV)
关键词:improve scene visibility, Multi-modal image fusion, Multi-modal image, visibility and perception, improve scene
备注: Project page: [this https URL](https://linfeng-tang.github.io/GenRF)
点击查看摘要
Abstract:Multi-modal image fusion combines complementary observations to improve scene visibility and perception. However, in real-world acquisition, differences in sensing principles and configurations introduce geometric discrepancies, while degradations further compromise source quality. Existing methods typically rely on explicit registration, yet large parallax complicates correspondence estimation, while mismatched fields of view leave some regions without cross-modal counterparts. Meanwhile, independent restoration underuses cross-modal information, whereas restoration-fusion methods assuming alignment remain vulnerable to misregistration. This study revisits misaligned and degraded image fusion in the era of generative foundation models, in which cross-attention enables position-agnostic complementary information aggregation, while generative priors support degradation recovery and inference in missing regions. Building on this perspective, we propose GenRF, which adapts pretrained FLUX through LoRA to jointly enhance degraded observations, infer missing cross-modal context, and fuse complementary information without explicitly estimating deformation fields. Extensive experiments on real-world infrared and visible captures show that GenRF delivers high-quality fusion directly from unregistered, degraded images without preprocessing, while improving downstream task performance under misalignment. Additional experiments demonstrate that GenRF can generalize to misaligned multi-exposure and multi-focus image fusion without task-specific fine-tuning. Project: this https URL.
95. 【2611.24154】Game2Player: Scaling Gameplay Experience for Visual Agents through Environment-Player Co-Development
链接:https://arxiv.org/abs/2611.24154
作者:Xirui Jiang,Yuxi Chen,Rui Yang,Feng Luo,Pengcheng Wang,Bingxuan Li,Zihao Zhao,Yushan Li,Han Wang,Manling Li,Yang Zhang,Shiyu Chang,Tong Zhang,Huan Zhang
类目:Computer Vision and Pattern Recognition (cs.CV)
关键词:demanding perception, long-horizon planning, precise control, rich testbed, Games
备注: Project page: [this https URL](https://game2player.github.io)
点击查看摘要
Abstract:Games are a rich testbed for visual agents, demanding perception, long-horizon planning, and precise control. However, collecting human gameplay is costly, and learning from gameplay videos requires inferring missing actions. Existing datasets also cover a limited range of games, often without explicit tasks or verifiable outcomes. To address these gaps, we introduce Game2Player, a framework that co-develops game environments and players to collect diverse and verifiable gameplay experience at scale. In the Game2Player loop, a Builder agent adapts existing web games, exposes their internal state, and defines tasks with state-based progress measures and success checks. A Coding Player and a VLM Player then attempt the tasks and report problems to the Builder for repair, so that play serves as both environment validation and training data collection. With Game2Player, we build 568 verified web games with 3,436 tasks, collect 15.7K trajectories (307 hours of gameplay), and hold out 50 games as the benchmark PlayerBench. To evaluate the collected experience as training data, we train VLM agents with progress-weighted SFT, assigning greater weight to steps that advance task progress. Game2Player-9B improves progress from 14.3% to 31.2% on PlayerBench and from 30.0% to 53.1% on the external GameWorld benchmark. More broadly, our work suggests a path for visual agents to scale their gameplay experience by building, verifying, and learning from existing game worlds.
96. 【2611.24152】SEER: Semantic Evidence for Executable Region Propagation in Open-Vocabulary Semantic Segmentation
链接:https://arxiv.org/abs/2611.24152
作者:Junyang Chen,Xiao Li,Yiguo Qiao,Wenbin Li
类目:Computer Vision and Pattern Recognition (cs.CV)
关键词:made strong progress, adapting CLIP-like representations, Training-free open-vocabulary semantic, open-vocabulary semantic segmentation, dense prediction
备注: 33 pages, including supplementary material
点击查看摘要
Abstract:Training-free open-vocabulary semantic segmentation (OVSS) has made strong progress by adapting CLIP-like representations to dense prediction, but its remaining errors are often local, structured, and tied to the active label space. A successful correction must be visually plausible, legal under the target vocabulary, and executable in the segmentation score field. We propose SEER, a training-free test-time repair layer for Semantic Evidence and Executable Region propagation. SEER casts post-prediction OVSS repair as evidence-conditioned label propagation. The OVSS score space proposes compact candidate labels, a frozen VLM verifies closed-set regional evidence, and an execution gate certifies whether the evidence is legally and semantically actionable. The accepted label is then propagated by an OVSS-side policy: coherent high-confidence regions allow broad propagation, while object- or boundary-risk cases use conservative score-grounded propagation. Across eight standard OVSS benchmarks, SEER reaches 58.0 average mIoU without updating OVSS weights. Matched-input controls isolate the repair mechanism, while resource measurements report the cost of the complete frozen system.
97. 【2611.24145】Inherited Adapter Experts for Replay-Free Domain-Incremental Learning
链接:https://arxiv.org/abs/2611.24145
作者:Gyutae Oh,Hyunmin Kong,Jitae Shin
类目:Computer Vision and Pattern Recognition (cs.CV); Artificial Intelligence (cs.AI)
关键词:Replay-free domain-incremental learning, Replay-free domain-incremental, domain-incremental learning trains, keeping earlier images, domain-incremental learning
备注: 21 pages (8 pages main text, plus references and supplementary material), 8 figures, 23 tables
点击查看摘要
Abstract:Replay-free domain-incremental learning trains a model on the same classes in successive domains without keeping earlier images. Per-domain learners freeze a pre-trained ViT and train an additional expert for each new domain: the small set of parameters and the classification head learned on that domain. Test images carry no domain label, so a router selects one expert for each image. Recent work trains this router. However, a training-free Mahalanobis router costs the analyzed learners under 2.7 points of final accuracy relative to always selecting each image's own-domain expert. The experts, not this router, limit accuracy. Our learner, Lamarck, therefore uses this router and strengthens the experts. Its adapters, small trainable layers that correct every image token after each deep ViT block, add 4.4 points of final accuracy on the deepfake benchmark CDDB-Hard. In addition, each expert is initialized from its predecessor at no extra cost: the domains share their classes, so an image misrouted to a later expert loses less. Against twelve frozen-backbone baselines under one protocol, Lamarck is first on CDDB-Hard by 5.2 points, second on DomainNet and OfficeHome, and highest in mean accuracy.
98. 【2611.24138】Ego2X: Contact-aware Transfer from Egocentric Human Data to Any Robot
链接:https://arxiv.org/abs/2611.24138
作者:Xinke Lyu,Bowen Yang,Zhicheng Wang
类目:Robotics (cs.RO); Computer Vision and Pattern Recognition (cs.CV)
关键词:Embodied models depend, Embodied models, slow and labor-intensive, real interaction data, models depend
备注: 10 pages, 7 figures
点击查看摘要
Abstract:Embodied models depend on real interaction data, yet collecting it on physical robots is slow and labor-intensive. Recent ego-to-robot pipelines retarget first-person view human videos to robot data via transferring hand and arm motion. They judge fidelity mainly by visual appearance. However, trajectories often fail in terms of physics: fingers hover beside objects, and grippers close on empty space. We argue that how the object moves and the contacts that drive the motion count. Thus, we present Ego2X, an object-centric framework that converts monocular egocentric video into physically executable robot data across dexterous hands and parallel grippers. Ego2X centers on CORE (Contact-centric Object-interaction REpresentation), which records each contact on the object by its surface anchor, as well as relative force, grasp mode, and interaction state. Throughout the pipeline, CORE corrects hand-object drift via object-relative constraints, steers inverse kinematics toward contacts the robot must realize, and preserves held objects during arm erasure and robot compositing. On HOT3D, contact-driven correction reduces in-contact hand depth error by 33.7%, while on EgoDex, contact-driven retargeting realizes 62.9% of annotated contacts, surpassing the official HuRo pipeline (27.9%). In RoboTwin 2.0, post-training X-VLA on Ego2X episodes alone, without simulator demonstrations, raises task success rate from 40.5% zero-shot to 67.0% across 200 paired scenes. We will publicly release our conversion pipeline and CORE-annotated dataset for dexterous hands and parallel grippers.
99. 【2611.24137】PatchAdaNet: Unsupervised class incremental learning using local patches and global adapters
链接:https://arxiv.org/abs/2611.24137
作者:Kamel Aouaidjia,Yanchao Duan,Zihui Zhen,Xin Li,Yongfei Lv,Chongsheng Zhang
类目:Computer Vision and Pattern Recognition (cs.CV)
关键词:Biological learning systems, systems acquire knowledge, acquire knowledge incrementally, reusable representations established, learning systems acquire
备注:
点击查看摘要
Abstract:Biological learning systems acquire knowledge incrementally through broad, reusable representations established early and kept largely stable, while integrating new experience with minimal interference with previously learned knowledge. Replay-based methods approximate this by storing samples or features from previous tasks and mixing them into the current task. However, memory buffers grow with the number of tasks and may retain task-specific noise that reduces generalization. To address these limitations, we propose PatchAdaNet (Patch Adapter Network), an unsupervised class-incremental learning method for image classification that continually updates feature representations using autoencoders. Since images share structural features such as edges, corners, and shapes, we first divide each image into patches and train independent autoencoders on external data to reconstruct each patch. The layers of all patch encoders are aggregated into global frozen layers that generalize across tasks and remain fixed throughout learning. Learnable adapters then build global image representations from these features to reconstruct the entire image. To prevent feature drift without storing data, we introduce the Global Feature Centroid (GFC), a fixed-size running average of adapter outputs updated after each task to penalizes representation drift. Elastic Weight Consolidation (EWC) is applied to the learnable layers to preserve previously learned features by constraining important weights. Finally, a linear classifier is trained on the frozen latent representation and GFC for class-incremental evaluation. We evaluate PatchAdaNet in both CNN and Vision Transformer settings on CIFAR-10, CIFAR-100, miniImageNet and 5-Datasets, providing a comparative study of the two architectures under identical class-incremental settings. Code is available.
100. 【2611.24122】How Well Do Generalist CT Foundation Models Differentiate Liver Tumors?
链接:https://arxiv.org/abs/2611.24122
作者:Yosuke Yamagishi,Tomohiro Kikuchi,Shouhei Hanaoka,Takeharu Yoshikawa,Osamu Abe
类目:Computer Vision and Pattern Recognition (cs.CV); Artificial Intelligence (cs.AI)
关键词:remains largely unevaluated, detected lesions remains, lesions remains largely, differentiate detected lesions, high performance
备注: 12 pages, 3 figures. Code: [this https URL](https://github.com/yamagishi0824/liver-tumor-ddx-ct-fm)
点击查看摘要
Abstract:Generalist CT foundation models detect many findings with high performance, but their ability to differentiate detected lesions remains largely unevaluated. This retrospective study evaluated two generalist models, RADAR and Merlin, for differentiating liver tumors using a public single-institution multiphase CT dataset (517 cases: hepatocellular carcinoma, intrahepatic cholangiocarcinoma, breast and colorectal cancer liver metastases, and hemangioma). The models were evaluated in a zero-shot setting using their existing outputs and with linear probing, in which only a linear classifier was trained on frozen image features (stratified fivefold cross-validation repeated five times). In the zero-shot setting, RADAR achieved an accuracy of 54.4% (95% CI: 49.9, 58.8) and a $\kappa$ of 0.37 for four-class differentiation at the portal venous phase and predicted 31% (129 of 421) of malignant tumors as hemangioma. With linear probing of four-phase features, its accuracy improved to 79.9% (95% CI: 76.6, 83.4) with a $\kappa$ of 0.72, and 1% (five of 421) of malignant tumors were predicted as hemangioma; Merlin achieved an accuracy of 60.9% (95% CI: 56.9, 65.0). Generalist CT foundation models showed limited zero-shot ability to differentiate liver tumors, which improved substantially with linear probing alone.
101. 【2611.24116】EVF-Bench: A Comprehensive Benchmark for E-commerce Visual Effects in Video Generation
链接:https://arxiv.org/abs/2611.24116
作者:Run Ling,Ke Cao,Runze He,Jie Gao,Ao Ma,Wei Feng,Zhanjie Zhang,Hao Wang,Jinghe Hu,Guibing Guo
类目:Computer Vision and Pattern Recognition (cs.CV)
关键词:product-centered visual presentations, accompanied by promotional, E-commerce visual effects, EVF, product-centered visual
备注:
点击查看摘要
Abstract:E-commerce visual effects (EVF) are dynamic, product-centered visual presentations in e-commerce videos, often accompanied by promotional text. They play an important role in attracting viewer attention and conveying product information. However, general measures of visual quality and text relevance alone do not establish whether generated videos preserve product appearance, maintain readable text, and produce coherent, contextually appropriate effects. To address this issue, we introduce EVF-Bench, a dedicated benchmark spanning six evaluation dimensions: visual quality and text relevance as general dimensions, and motion, product consistency, text stability, and creative rationality as EVF-specific dimensions. To complement the benchmark and support task-aligned research and model adaptation, we further construct EVF-32K, a dataset of over 32000 curated image-text-video triples tailored to e-commerce scenarios. Using this suite, we systematically evaluate representative video generation models and their domain-adapted variants. The results reveal complementary strengths, with no evaluated configuration leading all six dimensions. Together, EVF-Bench and EVF-32K provide a task-oriented evaluation framework and a supporting data resource for e-commerce video generation.
102. 【2611.24104】SCOPE: Single-Cell Omics Prediction from Histology via RNA-Protein Prior Enhancement
链接:https://arxiv.org/abs/2611.24104
作者:Leyang Tao,Xuan Gong,Luyuan Xie,Arjun Beniwal,Yuqi Tan,Kun-Hsing Yu
类目:Computer Vision and Pattern Recognition (cs.CV); Genomics (q-bio.GN)
关键词:existing slide archives, extend spatial molecular, slide archives, content of cells, cells from routine
备注:
点击查看摘要
Abstract:Predicting the RNA and protein content of cells from routine HE images could extend spatial molecular profiling to existing slide archives. However, both HE-spatial proteomics and HE-spatial transcriptomics paired datasets are limited by the number of available markers and the insufficient availability of high-quality, cell-resolved data. We present SCOPE (Single-Cell Omics Prediction via RNA-protein prior Enhancement), a unified framework that leverages biologically complementary information between RNA and proteins to address the limited availability of single-omics supervision. Our key insight is that, although RNA and proteins represent distinct molecular modalities, their biological relationships provide valuable shared knowledge that can benefit single-omics prediction tasks. To this end, SCOPE introduces biologically informed biomarker priors to enhance shared gene-level representations, enabling knowledge transfer across RNA and protein modalities through gene-protein correspondence and molecular associations. By exploiting these complementary relationships, SCOPE enables both prediction tasks to benefit from a shared backbone enriched with cross-omics biological knowledge. Furthermore, its biomarker correlation design facilitates the prediction of unseen markers, extending spatially resolved molecular profiling beyond directly supervised targets. Across test datasets spanning 16 organs, SCOPE improves the average RNA Pearson correlation over the strongest baseline by 8.8%. On two external protein cohorts, it improves the average Pearson correlation over the strongest baseline by 9.7% for seen markers and 11.9% for unseen markers.
103. 【2611.24102】IDEA: Intent-Dominance Evidence Audit for Inference-Time Backdoor Detection in GUI Agents
链接:https://arxiv.org/abs/2611.24102
作者:Zhizhuo Kou,Sizai Hou,Jian Yang,Ge Sun,Junyu Luo,Wenkai Zhang,Sirui Han,Yike Guo
类目:Computer Vision and Pattern Recognition (cs.CV)
关键词:Organizations increasingly deploy, downloaded open-weight GUI, increasingly deploy downloaded, deploy downloaded open-weight, open-weight GUI agents
备注:
点击查看摘要
Abstract:Organizations increasingly deploy downloaded open-weight GUI agents. A backdoored agent passes acceptance tests on clean data and misbehaves only when a visual trigger appears on screen, turning an unvetted checkpoint into a supply-chain risk. However, existing sample-level detectors target image classifiers and CLIP-style encoders, leaving step-level backdoor detection for GUI agents unaddressed. On clean steps, the instruction drives the action; when an unreferenced screen region hijacks the step, the instruction loses control over the decision state or that region dictates it instead. We term this signature intent--evidence dominance inversion and propose IDEA (Intent-Dominance Evidence Audit) to detect backdoor activation at inference time for each step without trigger knowledge or attack data. Specifically, Intent Counterfactual Probing (ICP) measures how far redirected instructions move the decision state, and Region Causal Audit (RCA) quantifies how far ablating a candidate region moves it. A gated conformal score compares both displacements against clean steps at the same distance from the clean manifold, providing a finite-sample false-positive guarantee. Extensive experiments on Qwen2.5-VL-3B across five attack families show that IDEA consistently detects backdoor activation under a controlled false-positive rate and outperforms state-of-the-art detection methods, while localizing the trigger region for every alarm and adding only modest inference overhead.
104. 【2611.24098】RoughCut-Bench: A Benchmark for Automatic Video Rough-Cut with Large Vision-Language Models
链接:https://arxiv.org/abs/2611.24098
作者:Ruisi Zhao,You Li,Sihan Chen,Aosong Cheng,Chixiang Ma,Sen Pei,Zhen Yang,Hehe Fan,Dongliang He,Yi Yang
类目:Computer Vision and Pattern Recognition (cs.CV)
关键词:organizing raw media, media production workflows, media production, core stage, initial video cut
备注: Accepted to KDD 2027. Project page is at [this https URL](https://whalesong-zrs.github.io/RoughCut-Bench-Project-Page)
点击查看摘要
Abstract:Video rough-cut is a core stage in media production workflows, organizing raw media materials into an initial video cut. Despite recent advances in large vision-language models (LVLMs) for video-related tasks, automatic rough-cut remains underexplored, without a comprehensive benchmark for evaluation or mature workflows for producing executable cut plans. To address the evaluation gap, we introduce RoughCut-Bench, which covers two practical settings: (1) a visually driven setting over video cases, covering highlight detection and selection, source-video ordering, and adherence to user requirements when provided; and (2) a speech-driven setting over automatic speech recognition (ASR) transcripts, covering filler removal, expression deduplication, and low-value content trimming. Across these settings, the benchmark contains 101 visually driven cases, including 831 raw videos with 1,476 highlight annotations. It also includes 84 speech-driven cases, each consisting of an ASR transcript with annotated cleanup spans. The evaluation compares cut plans with these annotations in both settings and additionally uses a human-aligned agentic evaluator to assess rendered-cut quality in the visual setting. To apply LVLMs to automatic rough-cut, we further propose agentic workflows that guide them to produce structured cut plans from input materials. Experiments show that current LVLMs remain limited in producing expert-aligned cut plans. Our benchmark, code, and prompts are released at this https URL.
105. 【2611.24095】Object Pose: The Link between Weak Perspective, Paraperspective, and Full Perspective
链接:https://arxiv.org/abs/2611.24095
作者:Radu Horaud,Fadi Dornaika,Bart Lamiroy,Stephane Christy
类目:Computer Vision and Pattern Recognition (cs.CV); Robotics (cs.RO)
关键词:perspective camera model, DeMenthon and Davis, perspective camera, camera model, weak perspective camera
备注:
点击查看摘要
Abstract:Recently, DeMenthon and Davis [4,5] proposed a method for determining the pose of a 3-D object with respect to a camera from 3-D to 2-D point correspondences. The method consists of iteratively improving the pose computed with a weak perspective camera model to converge, at the limit, to a pose estimation computed with a perspective camera model. In this paper we give an algebraic derivation of DeMenthon and Davis' method and we show that it belongs to a larger class of methods where the perspective camera model is approximated either at zero order (weak perspective) or first order (paraperspective). We describe in detail an iterative paraperspective pose computation method for both non coplanar and coplanar object points. We analyse the convergence of these methods and we conclude that the iterative paraperspective method (proposed in this paper) has better convergence properties than the iterative weak perspective method. We introduce a simple way of taking into account the orthogonality constraint associated with the rotation matrix. We analyse the sensitivity to camera calibration errors and we define the optimal experimental setup with respect to imprecise camera calibration. We compare the results obtained with this method and with a non-linear optimization method.
106. 【2611.24093】UCLIR: Unsupervised Continual Learning using Independent Representations
链接:https://arxiv.org/abs/2611.24093
作者:Kamel Aouaidjia,Yanchao Duan,Zihui Zhen,Xin Li,Yongfei Lv,Chongsheng Zhang
类目:Machine Learning (cs.LG); Computer Vision and Pattern Recognition (cs.CV)
关键词:regularization-based methods aim, penalty terms, aim to mimic, mimic neuroplasticity, neuroplasticity by updating
备注:
点击查看摘要
Abstract:In continual learning, regularization-based methods aim to mimic neuroplasticity by updating network weights to learn new tasks while constraining changes to parameters that are important for previously learned tasks through penalty terms. However, as the number of tasks increases, preserving knowledge from earlier tasks becomes less effective. This is mainly because updates for new tasks are distributed across the entire network, where parameters are highly interdependent due to the dense connectivity between neurons. Inspired by the Exponential Distance Rule (EDR), which states that the number of neural connections decreases with increasing physical distance, learning is naturally locally correlated while inter-area updates remain less task-dependent. Motivated by this principle, in this paper we demonstrate that applying weight regularization to multiple small and independently learned complementary representations is more effective than regularizing a single large shared representation. By decomposing the input into independently learned components, learning new tasks induces only localized parameter updates within specific representation, which limit interference and improve knowledge retention across tasks. We employ independent autoencoders that learn different parts of the input independently through unsupervised regularization. At inference time, a linear classifier is used for evaluation. The experimental results on a sequence of tasks using 5-Splits MNIST, sequence of 5-datasets, 5-Splits CIFAR-10, 10-Splits Tiny ImageNet shows competitive results with supervised and replay-based methods. Code is available.
107. 【2611.24084】From Restoration to Redrawing: Revitalizing Ancient Chinese Characters by SVG Code Generation and Editing
链接:https://arxiv.org/abs/2611.24084
作者:Weiguang Zhang,Linhua Dong,Bin Dong,Shanshan Jiang,Maizhen Ning,Xiaoqiang Kang,Kun Zhang,Hao Xu,Xiaowei Huang,Wei Wang,Kaizhu Huang,Qiufeng Wang
类目:Computer Vision and Pattern Recognition (cs.CV)
关键词:Han Dynasty bamboo, ancient Chinese writing, Han Dynasty, Hanjian manuscripts written, written on Han
备注:
点击查看摘要
Abstract:Hanjian manuscripts written on Han Dynasty bamboo and wooden slips are invaluable heritage of ancient Chinese writing but suffer from damaged ink and incomplete strokes. Existing image-to-image restoration methods may alter observed strokes and offer limited control over stroke geometry. Inspired by experts' manual redrawing practices, we address these issues by introducing \textbf{ReDraw}, a two-stage vision-language model(VLM) that reformulates character restoration as Scalable Vector Graphics(SVG) code generation and editing. Stage one targets faithful vectorization of observed strokes geometry or conservative tracing of mildly degraded strokes. Guided by skeleton chain-of-thought reasoning and reinforcement learning with structure-aware rewards, stage two predicts skeletons before contours to redraw severely incomplete strokes by appending paths while keeping existing SVG paths unchanged. To support training, we develop a multi-layer data synthesis pipeline that disentangles ink, material, and degradation to generate 1.2M samples, and derive up to 6,400 adaptation pairs from 320 annotated real glyphs. Experiments demonstrate synthetic restoration performance and adaptability to real Hanjian glyphs. Code and demo will be available upon acceptance.
108. 【2611.24081】Learning Where and How Much to Sample: Volume-Adaptive k-Space Acquisition for Accelerated MRI
链接:https://arxiv.org/abs/2611.24081
作者:Reyhaneh Neshatavar,Mohsen Yavartanoo,Kyoung Mu Lee
类目:Computer Vision and Pattern Recognition (cs.CV)
关键词:Magnetic resonance imaging, medical imaging technique, essential medical imaging, Magnetic resonance, long acquisition times
备注:
点击查看摘要
Abstract:Magnetic resonance imaging (MRI) is an essential medical imaging technique, but its high costs and long acquisition times can limit accessibility. Accelerated MRI reduces acquisition time through k-space undersampling, yet conventional methods often rely on predefined sampling patterns and uniform slice budgets, overlooking anatomical variations and inter-slice relationships. In this paper, we propose a volume-adaptive k-space acquisition framework that jointly learns where and how much to sample across an entire MRI volume. Using only acquired auto-calibration signal (ACS) measurements, our method exploits volume-wide anatomical information to adaptively allocate sampling budgets across slices while identifying informative k-space lines. A learned joint slice--frequency refinement module coordinates sampling decisions while preserving an exact total volume-level acquisition budget. To reconstruct the acquired measurements, we design a sensitivity-aware synchronous cross-slice reconstruction network that estimates coil sensitivities from ACS measurements and progressively refines neighboring-slice reconstructions while enforcing hard k-space data consistency. Both networks are jointly optimized using reconstruction-based supervision without requiring fully sampled k-space at inference. Experiments on knee and brain MRI across multiple contrasts and acceleration factors demonstrate the adaptive behavior of our acquisition framework and evaluate its reconstruction performance.
109. 【2611.24080】CausalDrive: Causal Decomposition for Data-Efficient End-to-End Autonomous Driving
链接:https://arxiv.org/abs/2611.24080
作者:Xiaobao Wei,Jingxian Wu,Lirong Yang,Qing Han,Yuan Gao,Ziwei Yang,Shuai Yang,Weiwei Liu,Chang Huang
类目:Computer Vision and Pattern Recognition (cs.CV)
关键词:expert, Efficient, nominal, nominal intention, expert trajectories
备注:
点击查看摘要
Abstract:Efficient use of expert demonstrations is essential for end-to-end autonomous driving. However, directly imitating expert trajectories couples nominal intention, which specifies where the vehicle should go, with multimodal driving styles, which determine how that intention is executed. Different combinations of these factors can yield similar expert trajectories, obscuring their respective causal contributions and forcing the policy to learn both from a single entangled target. This ambiguity limits how effectively the policy can exploit expert demonstrations. We propose CausalDrive, a causal decomposition framework for data-efficient driving policy learning that separates deterministic nominal intention prediction from probabilistic driving style modeling. Specifically, we construct counterfactual nominal trajectories from navigation command, map topology, and ego state by removing style dependent effects, providing a Nominal Intention Planner with supervision independent of expert trajectories. Conditioned on the predicted nominal intention and surrounding agents, a diffusion-based Style Execution Policy models diverse driving styles as a multimodal distribution over residuals between nominal and expert trajectories. This causally structured supervision reduces redundant learning of shared nominal behavior in routine scenarios, directing expert supervision toward informative execution variations and improving data utilization. CausalDrive achieves 91.2 PDMS on NAVSIM v1 and 89.5 EPDMS on NAVSIM v2, matching or surpassing state-of-the-art performance. Further experiments demonstrate strong performance with limited expert data and clear advantages in challenging interactive scenarios.
110. 【2611.24079】APS: Adaptive Partitioning-Based Frame Selection for Long Video Understanding
链接:https://arxiv.org/abs/2611.24079
作者:Junxiang Wen,Yinxi Zhao,Jiawei Chen,Jingying Hu,Lijun Wang,Zhen Qin,Shuiguang Deng
类目:Computer Vision and Pattern Recognition (cs.CV); Artificial Intelligence (cs.AI)
关键词:video understanding capabilities, improved video understanding, long videos remains, videos remains challenging, significantly improved video
备注:
点击查看摘要
Abstract:Recent advances in large vision-language models (LVLMs) have significantly improved video understanding capabilities, yet processing long videos remains challenging due to the substantial computational cost of extensive visual inputs. Frame selection offers an efficient alternative, but many existing region-based methods rely on fixed or predefined temporal partitions, which may result in suboptimal temporal regions, compromising subsequent budget allocation and frame selection. In this work, we propose Adaptive Partitioning-Based Frame Selection (APS), a training-free framework that adapts temporal partitions to local variations in query-frame relevance. Specifically, APS formulates temporal partitioning as a complexity-regularized fitting problem, jointly optimizing partition boundaries and local models through dynamic programming. The resulting adaptive temporal partitions guide frame budget allocation based on query relevance and temporal extent, enabling informative frame selection under limited visual budgets. Extensive experiments on Video-MME, MLVU, and LongVideoBench demonstrate that APS consistently improves performance across diverse LVLM backbones, achieving the highest average gains over uniform sampling among competing methods. Further analyses validate the effectiveness of adaptive partitioning and demonstrate its generalizability across model scales, similarity encoders, and advanced LVLMs.
111. 【2611.24077】AeroRSI: Recursive Self-Improvement for Aerial Vision-Language Navigation
链接:https://arxiv.org/abs/2611.24077
作者:Shiwei Wu,Yuwen Pan,Zhi Li,Yaowen Li,Yan Zhou,Xuemiao Xu,Yu Liu
类目:Computer Vision and Pattern Recognition (cs.CV); Artificial Intelligence (cs.AI); Robotics (cs.RO)
关键词:follow natural-language instructions, unmanned aerial vehicles, enable unmanned aerial, models enable unmanned, natural-language instructions
备注: 8-page main paper with references and supplementary material (15 pages); 5 figures, 13 tables. Code: [this https URL](https://github.com/starsholic/AeroRSI) Project page: [this https URL](https://starsholic.github.io/AeroRSI/)
点击查看摘要
Abstract:Vision-language-action (VLA) models enable unmanned aerial vehicles (UAVs) to follow natural-language instructions, yet still fail frequently in closed-loop flight. Correcting these failures requires repairs, i.e., actions that avoid them at the states the policy itself reaches, which human pilots cannot label at scale and rule-based planners cannot supply reliably. In this paper, we present AeroRSI, a recursive self-improvement framework in which an aerial navigation policy iteratively learns from its own failures. Our key insight is that a failing policy should judge its own repairs by flying on from them. Judged in this way, its sampled actions and even rule-based proposals that hurt it when imitated directly become reliable supervision. Moreover, an episode terminates at its first failure, concealing subsequent failures until the earlier ones are repaired. AeroRSI thus renews its supervision in every round, anchoring the failures of the current policy, verifying repairs by its continuation, and distilling them while keeping its behavior elsewhere. On TravelUAV, three rounds raise the success rate of a supervised 2B-parameter policy from 46.4% to 67.5% and reduce collisions by half, surpassing the previous state of the art by 10.6 points from the same initialization without extra human labels or inference cost. Code and models are available at this https URL.
112. 【2611.24074】SemPhys: Semantic-Physical Co-Learning for Humanoid Motion Generation via Differentiable World Models
链接:https://arxiv.org/abs/2611.24074
作者:Zejian Kang,Yeqiang Wang,Zekai Chen,Boyang Shen,Mingyu Ma,Bingqian Wu,Fan Zhou,Jianjie Zhang,Chentao Shen,Wentao Yang,Rui Qian,Xiangru Huang
类目:Robotics (cs.RO); Computer Vision and Pattern Recognition (cs.CV)
关键词:programming joint trajectories, humanoid whole-body movements, human-robot collaboration, natural language, joint trajectories
备注:
点击查看摘要
Abstract:In human-robot collaboration, natural language lets people direct a humanoid's whole-body movements without programming joint trajectories. A common approach generates a kinematic reference from an instruction and relies on a tracking controller to execute it. However, as the controller maintains balance under contact constraints, the executed motion can depart from the reference in ways that change the action it conveys. Our central insight is that semantic alignment must be evaluated directly on the complete motion the robot physically executes. Building on this insight, we introduce \textbf{SemPhys}, a framework that trains motion generators using semantic feedback from simulated executions. A differentiable world model and a fixed tracking controller predict a short segment of a recorded execution. SemPhys inserts this prediction at its original position, scores the complete trajectory against the instruction, and backpropagates through the predicted segment. A paired anchoring loss stabilizes adaptation by keeping each generated reference close to its counterpart from the initial generator. On BONES-SEED, SemPhys largely preserves instruction retrieval accuracy from reference to execution and achieves the highest retrieval accuracy on executed motions among the evaluated methods.
113. 【2611.24068】Benchmarking Reasoning Reliability,Prior_Evidence Contrast,and Spectral Discrimination in Remote-Sensing Geological Image Interpretation
链接:https://arxiv.org/abs/2611.24068
作者:Jun Wang,Fengpeng Li,Wei Han,Yuxuan Li,Tianjin Huang,Jining Yan,Sheng Wang,Kan Sun,Lizhe Wang
类目:Computer Vision and Pattern Recognition (cs.CV)
关键词:Reliable lithology interpretation, Multimodal Large Language, Reliable lithology, requires accurate geological, lithology interpretation requires
备注:
点击查看摘要
Abstract:Reliable lithology interpretation requires accurate geological recognition and reasoning grounded in visual evidence. Although Multimodal Large Language Models show promise in remote sensing, their reliability in geological interpretation remains unclear. We introduce HyGeoBench, a diagnostic benchmark of 10,000 questions based on hyperspectral imagery across three dimensions: reasoning reliability, prior_evidence contrast, and spectral discrimination. It covers ten task types, with both multiple-choice and open-ended response formats. Integrating hyperspectral satellite imagery with geological annotations and verified evidence, HyGeoBench examines reasoning errors, conflicts between geological priors and visual evidence, and cross-band inconsistencies. Evaluations reveal limitations in fine-grained recognition and evidence-grounded interpretation. To address these challenges, we propose HyGeoAgent, a tool-augmented framework integrating region observation, text knowledge retrieval, same-band image retrieval, and native PCA analysis through adaptive tool use. Experiments demonstrate consistent improvements over the corresponding base models, improving geological image interpretation across diagnostic dimensions.
114. 【2611.24060】R$^{3}$Sieve: Suppressing Unconditional Information from Attention to Answers for High-Resolution VQA
链接:https://arxiv.org/abs/2611.24060
作者:Yifan Cao,Yawen Zhang,Xiaocui Yang,Faxian Wan,Shi Feng,Daling Wang,Yifei Zhang,Shangshu Yu,Xianbing Zhao
类目:Computer Vision and Pattern Recognition (cs.CV)
关键词:demanding fine-grained perception, question-relevant small objects, correct answer based, demanding fine-grained, fine-grained perception
备注:
点击查看摘要
Abstract:High-resolution visual question answering (VQA) requires models to identify question-relevant small objects in high-resolution images and select the correct answer based on what they recognize, demanding fine-grained perception. Previous methods focus on obtaining better visual evidence (image crops), but obtaining it alone is insufficient, using it effectively is equally crucial. Visual evidence should therefore be utilized as effectively as possible throughout inference. We argue that small-target evidence is obscured by unconditional information that remains observable in subsequent processing even after the condition on which it should depend is removed. Such information hinders the effective use of small-target evidence. We propose R$^{3}$Sieve (Residual-Region-Response Sieve), a training-free framework. We divide inference into three stages---localization, cropping, and answering---and introduce distinct counterfactual references to suppress interference at each stage. During localization, the Shared-Subspace Canceller (SSC) uses a generic image description to construct a counterfactual reference and obtain question-relevant attention maps. During cropping, the Unique-Contribution Spotter (UCS) constructs counterfactual references from visual and non-visual features to identify distinctive, complementary crop regions. During answering, the Content-Aligned Voter (CAV) constructs an image-free branch to calibrate answer responses. Experiments on three high-resolution VQA benchmarks show that R$^{3}$Sieve achieves the highest overall accuracy among compared methods across diverse settings, improving base models by up to 21.5 percentage points.
115. 【2611.24042】CURVTON-205K: A Mask-Free Difficulty-Stratified Synthetic Dataset for Robust and Scalable Virtual Try-On
链接:https://arxiv.org/abs/2611.24042
作者:Dipan Bartaula,Anuj Shrestha,Ankit Belbase,Anish Bishwakarma,Saroj Poudel,Samip Ghimire,Shashank Bhatta,Eric Arazo,Binod Bhattarai
类目:Computer Vision and Pattern Recognition (cs.CV)
关键词:Recent advances, conditions remains challenging, diverse real-world conditions, real-world conditions remains, architectural innovation
备注: Accepted at IEEE/CVF Winter Conference on Applications of Computer Vision (WACV) 2027. Project page: [this https URL](https://curvton2.github.io/curvton)
点击查看摘要
Abstract:Recent advances in 2D virtual try-on (VTON) have largely focused on architectural innovation, yet robust performance in diverse real-world conditions remains challenging. We argue that failure modes in VTON, such as inaccurate occlusion handling, unconstrained poses, and poor coverage of under-represented garments and appearance factors, are tightly coupled to dataset design choices and long-tail data regimes. We present a scalable data generation pipeline that coordinates specialized editing and generation models with vision-language model (VLM) guidance to synthesize triplets without requiring parsing masks or dense-pose annotations. Using this pipeline, we create CURVTON-205K, a curated synthetic dataset of 205K triplets: source person, target garment, and corresponding try-on image. CURVTON-205K is explicitly designed to broaden coverage across garments, person appearance, poses, occlusions, and visual conditions. Additionally, CURVTON-205K is organized into three difficulty tiers, enabling controlled evaluation under varying levels of difficulty and supporting difficulty-aware experimentation. We analyze the dataset diversity and demonstrate its potential to improve try-on fidelity across four VTON architectures and its cross-dataset generalization.
116. 【2611.24038】Semantic-aware Contrasting with Explicitly Disentangled Representations for AI-Generated Video Detection
链接:https://arxiv.org/abs/2611.24038
作者:Jiaquan Yu,Jingyi Xie,Zeyu Zhang,Yunhang Qian,Xiaobin Hu,Jiawei Liu
类目:Computer Vision and Pattern Recognition (cs.CV)
关键词:raised critical concerns, content authenticity, rapid proliferation, proliferation of AI-generated, raised critical
备注: 16 pages, 5 figures
点击查看摘要
Abstract:The rapid proliferation of AI-generated videos (AIGVs) has raised critical concerns about content authenticity. Existing benchmarks suffer from a fundamental limitation: real and fake videos are often collected or generated from different semantic distributions. As a result, detectors may rely on spurious correlations involving objects, scenes, or actions rather than intrinsic artifacts introduced by generative models. To this end, we introduce SAVGE, a large-scale semantically aligned AIGV detection dataset comprising 2,369,822 videos, including 424,115 real videos and 1,945,707 generated videos. SAVGE organizes samples into real-fake semantic groups by using captions paired with real videos as prompts for text-to-video generation, thereby providing both group-level correspondence and textual semantic anchors. Based on SAVGE, we further propose Explicit Disentangling and Semantic-aware Contrasting (EDSC), a model-agnostic training framework for learning artifact-focused representations with reduced semantic interference. EDSC first extracts and aligns visual semantic features with textual representations, and then explicitly suppresses semantic information in the artifact branch while contrasting real and fake samples within each semantic group. By integrating caption-guided semantic supervision with intra-group contrastive learning in a two-phase optimization strategy, EDSC encourages detectors to capture generation-related cues under matched semantic conditions. Extensive experiments on SAVGE and GenVideo demonstrate that EDSC improves most evaluated backbone-metric combinations, enhances robustness under challenging evaluation settings, and strengthens cross-dataset generalization. These results highlight the importance of semantic alignment in AIGV detection and validate explicit semantic-artifact disentanglement as an effective strategy for robust detector learning.
117. 【2611.24023】STAR++: A Physical-Aware Benchmark and Framework for Astronomical Super-Resolution across Multiple Filters
链接:https://arxiv.org/abs/2611.24023
作者:Guohang Zhuang,Jiaze Li,Kuo-cheng Wu,Hao Du,Jinyang Huang,Xiang Zhang,Wanli Ouyang,Yan Lu
类目:Computer Vision and Pattern Recognition (cs.CV)
关键词:ASR, Existing ASR methods, Astronomical super-resolution, aims to reconstruct, astrophysical analysis
备注:
点击查看摘要
Abstract:Astronomical super-resolution (ASR) aims to reconstruct high-resolution star fields from low-resolution telescope observations across multiple filters for astrophysical analysis. However, developing a physics-aware ASR framework faces two challenges: 1) Data view: no large-scale benchmark supports field-level ASR, especially on unseen filters. 2) Method view: Existing ASR methods often fail to preserve stellar flux across spatial resolutions. Moreover, ASR requires reconstructing high-frequency components, including accurate star locations and profiles, making it more ill-posed than natural-image SR. To tackle these problems, we introduce STAR++, a large-scale benchmark comprising approximately 1.7 million flux-consistent image pairs across 36 filters, derived from Hubble Space Telescope (HST) observations. STAR++ uses a flux-preserving data generation pipeline to ensure photometric consistency across resolutions and provides the Flux Error (FE) metric to evaluate ASR models from a physical perspective. To address method-level challenges, we propose a Flux-Invariant Super-Resolution Plus (FISR++) model that generalizes to unseen filters while accurately reconstructing flux-consistent high-resolution images. Beyond the flux-preserving modules, FISR++ introduces two novel components. Specifically, a Filter-Aware Adapter (FAA) incorporates filter metadata to adapt image representations to varying spectral and optical conditions. In addition, a Spatial Ambiguity Disentangle Loss (SADL) models spatial ambiguity as pixel-shift uncertainty, mitigating the smoothing of learned conditional mean reconstructions caused by this ambiguity. Extensive experiments demonstrate that FISR++ consistently outperforms existing ASR methods in reconstruction quality and flux accuracy, while generalizing to unseen filters, highlighting its effectiveness for photometrically faithful ASR.
118. 【2611.24018】Semi-Supervised GAN-Based Segmentation of CTPA Scans for Pulmonary Emboli Boundary Extraction
链接:https://arxiv.org/abs/2611.24018
作者:Maria Thoma,Michalis Savelonas,Foteini Malli,Dimitris Iakovidis
类目:Computer Vision and Pattern Recognition (cs.CV); Machine Learning (cs.LG)
关键词:pulmonary vascular disease, morbidity and mortality, Pulmonary embolism, vascular disease, high morbidity
备注: 2026 IEEE 39th International Symposium on Computer-Based Medical Systems (CBMS), Limassol, Cyprus
点击查看摘要
Abstract:Pulmonary embolism (PE) is a pulmonary vascular disease associated with high morbidity and mortality, where delayed diagnosis can be fatal. While computed tomography pulmonary angiography (CTPA) is the clinical gold standard for PE detection, interpreting these scans manually is time-consuming and error-prone. In this work, we propose a semi-supervised segmentation method, employing generative adversarial networks (GANs) to leverage large pools of unlabeled CTPA scans. The proposed method addresses more effectively the challenges of CTPA image segmentation than previous methods by employing the dice loss and the cross-entropy loss to tackle the highly imbalanced nature of CTPA scans, the feature matching loss to encourage the generator to produce anatomically plausible segmentation maps that are indistinguishable from ground truth, as well as the self-training loss to reinforce learning from highly-confident pseudo-labeled samples. The experimental evaluation on a publicly available dataset shows that the proposed method maintains accurate segmentation, despite a limited number of labeled CTPA scans. with a Dice coefficient of 88.75% and a 95th percentile Hausdorff distance (HD95) of 1.74, outperforming fully supervised baselines.
119. 【2611.24016】Rethinking Contrastive Targets in Radiology Language-Image Pretraining
链接:https://arxiv.org/abs/2611.24016
作者:Yunhe Gao,Ashwin Kumar,Jiaming Liu,Chong Wang,Maya Varma,Fuying Wang,Stefania L. Moroianu,Louisa Fay,Sergios Gatidis,Akshay S. Chaudhari,Curtis Langlotz
类目:Computer Vision and Pattern Recognition (cs.CV); Machine Learning (cs.LG)
关键词:CLIP-style contrastive image-text, contrastive image-text pretraining, image-text pretraining, pretraining is widely, CLIP-style contrastive
备注: NeurIPS 2026
点击查看摘要
Abstract:CLIP-style contrastive image-text pretraining is widely used in radiology data, where each image is paired with one report and every other report in the batch is treated as a negative. Under this recipe, two failures appear, neither of which is systematically addressed by existing contrastive learning theory: zero-shot evaluation peaks then degrades while training loss keeps decreasing, and training fails to learn from large batches. We hypothesize the cause is structural rather than radiology-specific: many unpaired reports share clinical findings with a given image, so CLIP's paired-identity target is systematically biased, not merely noisy. To test this, we construct CLEVR-Radiology, a synthetic testbed that preserves only this overlapping-fact structure and reproduces both failures. With evidence that the structure is sufficient to reproduce the failures, we develop a target-misspecification theory that explains both failures from a single deterministic gradient bias amplified by batch size, and use it to motivate a minimal compatibility-based correction, CorLIP: text is treated as positive whenever its clinical assertions agree with the image, with no architecture changes. Beyond benchmark improvements, CorLIP largely removes the post-peak degradation, makes large-batch training succeed where it previously collapsed, substantially amplifies the gains from more data, and is the only configuration that remains stable up to B=2048. The correction transfers from chest X-ray to 3D CT modalities.
120. 【2611.24012】FlyVQA: Seeing Video Frame Interpolation Quality through Fly Eyes
链接:https://arxiv.org/abs/2611.24012
作者:Baoliang Chen,Lingyu Zhu,Zhiwei Zhong,Shiqi Wang
类目:Computer Vision and Pattern Recognition (cs.CV)
关键词:achieve full alignment, ultimate ambition, reproduce the neural, neural computations, achieve full
备注:
点击查看摘要
Abstract:The ultimate ambition of visual quality assessment is to reproduce the neural computations of human vision and thereby achieve full alignment with human perception. However, the immense complexity of human visual circuitry makes this goal difficult to realize. Recently released fly visual connectomes provide a tractable biological starting point. Despite differences between fly and human vision, the fly's well-characterized motion pathways motivate us to investigate their potential for assessing the perceptual impact of temporal distortions. In this paper, we use a shared, frozen fly visual circuit to perform full-reference quality assessment of frame-interpolated videos, whose artifacts can disrupt temporal consistency. Specifically, compound-eye sampling maps reference and distorted sequences into the circuit, while a connectome-aware readout quantifies discrepancies in cell-type-specific activity and excitatory--inhibitory synaptic dynamics. Reference-guided temporal pooling and a lightweight regression head integrate these differences into a quality score. Experiments across multiple interpolation methods demonstrate better agreement with human ratings than CNN, RNN, and Transformer baselines with comparable parameters, supporting a biologically grounded approach to perceptual quality modeling.
121. 【2611.24011】WAM-Policy: World Action Models with On-Policy Distillation for Quadrupedal Loco-Manipulation
链接:https://arxiv.org/abs/2611.24011
作者:Junlin Long,Xu Deng,Yiran Wang,Yue Yang,Yang Zhao,Zeyu Zhang
类目:Robotics (cs.RO); Computer Vision and Pattern Recognition (cs.CV)
关键词:Arm-free quadrupeds, front leg, language instructions, Arm-free, policy
备注: Code: [this https URL](https://github.com/AIGeeksGroup/WAM-Policy) , Website: [this https URL](https://aigeeksgroup.github.io/WAM-Policy/)
点击查看摘要
Abstract:Arm-free quadrupeds can perform loco-manipulation, walking on three legs while manipulating with a front leg, yet no existing policy for it follows language instructions. World action models (WAMs) have emerged as effective language-conditioned manipulation policies, but are pretrained for robot arms and sample slowly. WAM-Policy turns one WAM into a quadrupedal loco-manipulation policy via two stages. The first, Cross-Morphology Policy Adaptation and Acceleration, adapts the arm-centric WAM to quadrupeds and distills the transferred policy into a student with one video and two action steps by on-policy self-distillation (OPSD). This student, however, is bounded by its teacher, which only imitates demonstrations. The second stage, Task-Specific Improvement and Merging, therefore tunes the teacher by reinforcement learning (RL) and merges the slow specialists into one few-step student by multi-teacher on-policy distillation (M-OPD). On QLM-Bench, ten simulated tasks with 1,000 demonstrations synthesized without teleoperation, the transferred policy reaches 77.0%, and the OPSD student keeps 76.7% while sampling 14 times faster. Task-specific RL and M-OPD lift it to 85.0%. We also deploy the OPSD student on a real Unitree Go2 Air. WAM-Policy thus achieves a stronger quadrupedal policy at a fraction of the inference cost.
122. 【2611.24010】HHW-Ego: DSLR-Quality Enhancement for Multi-Source Wearable Ego Imaging
链接:https://arxiv.org/abs/2611.24010
作者:Yunfei Tong,Qianqian Sun,Jixiang Luo,Dell Zhang,Hao Sun
类目:Computer Vision and Pattern Recognition (cs.CV)
关键词:daily smart glass, smart glass usage, embodied intelligence, high demand, demand for scenarios
备注:
点击查看摘要
Abstract:Wearable ego-centric images are now in high demand for scenarios ranging from daily smart glass usage to embodied intelligence. But the image quality is far behind smart phone due to the lacking paired high-quality reference images and the dedicated enhancement systems. To solve this, a customized degradation pipeline is designed to generate paired samples matching wearable camera images, boosting the upper limit of enhancement performance. Besides a two-stage enhancement framework is further built: first, tuning an efficient model on the paired dataset to enhance real wearable images; second, using hyperspectral data to refine color temperature for better quality. Moreover we present HHW-Ego, a multi-source paired dataset integrating with Hyperspectral, High-dynamic range and Wearable Ego-centric data, which includes 5,000 image groups spanning indoor/outdoor and day/night scenes. On held-out paired data, Stage-1 fine-tuning improves PSNR, SSIM, Lab distance, and LPIPS for Real-ESRGAN and PFT-SR. On real wearable images, no-reference metrics, human preference, gamut analysis, and visual comparisons characterize the effects of Stage 2.
123. 【2611.24007】Freeforward4d: Annotation-free feed-forward 4d reconstruction of driving scenes
链接:https://arxiv.org/abs/2611.24007
作者:Luosong Guo,Xiang Yang,Chaoyang Li,Ao Gao,Jiangming Shi,Yanyun Qu,Zhizhong Zhang,Jingyu Gong,Yuan Xie
类目:Computer Vision and Pattern Recognition (cs.CV)
关键词:enables efficient creation, autonomous driving simulation, renderable driving scenes, reconstruction enables efficient, enables efficient
备注:
点击查看摘要
Abstract:Feed-forward 4D reconstruction enables efficient creation of renderable driving scenes for autonomous driving simulation. However, most existing methods rely heavily on dataset-provided cameras, LiDAR depth, or 3D box annotations, limiting their applicability to unannotated driving videos. Pretrained visual models can predict cameras, depth, and object tracks from RGB alone, but their geometry is mutually inconsistent: multi-view predictions come in short windows with independent scales and frames, while monocular depth is metric only per image. Our key insight is that, once reconciled into a single metric frame, these priors can replace dataset annotations for feed-forward 4D reconstruction. We present \textbf{FreeForward4D}, which calibrates overlapping prediction windows against monocular metric depth and registers them into one frame across time. On this geometry, 2D tracks are associated across cameras in 3D, and objects move through hierarchical motion that combines a shared object velocity with local Gaussian corrections, so that both training and inference use only RGB videos. On Waymo, FreeForward4D achieves 27.66 dB PSNR and 0.836 SSIM while recovering metric-scale cameras and depth without scale alignment. It surpasses STORM, SLARM-F, UFO, DGGT, and StreetForward, which rely on dataset annotations, in full-image PSNR and attains the best RGB fidelity on 8-second windows. RGB-only fine-tuning further adapts the framework to nuScenes and Argoverse 2. Code will be released.
124. 【2611.24003】Choosing Supervised Post-Processing for Frozen Person Detectors: Information, Object Changes, and Complete Cost
链接:https://arxiv.org/abs/2611.24003
作者:Tianyi Zhou,Zijun Jia
类目:Computer Vision and Pattern Recognition (cs.CV); Artificial Intelligence (cs.AI)
关键词:frozen detector computes, detector computes hypotheses, supervised post-processing, frozen detector, detector computes
备注:
点击查看摘要
Abstract:A frozen detector computes hypotheses that can support better final outputs through supervised post-processing. Choosing a readout requires knowing which available information improves detection and what its processing costs. We compare candidate scores, class distributions, geometry, native-set summaries and adapted neighborhood meta classification under common candidate, supervision and operating-point conditions. On 320 held-out CrowdHuman images, adding native-set context to a 97-dimensional readout yields 275 more matches and 96 more unmatched predictions. On a separate 319-image cohort, Full100 matches 6004 objects,versus 5848 for source-selected raw WBF and 6002 for a MetaDetect-style adaptation. Its gain over raw contains 300 added and 144 lost discoveries. After execution-equivalent optimization, Full100 and the meta-statistics readout both take approximately 95 ms per complete warm request on a fixed 64-image workload. Source-frozen transfer gives positive point results on 223 MOT17Det frames from seven sequences, while the main MIAP result shows no improvement. The study supports using sufficiently informative output readouts while retaining optimized meta classification as a competitive option. It also shows why feature dimension and net detection gain alone are unreliable guides to adoption.
125. 【2611.24002】ProtoBridge-CLIP: Observation-Masked Finding Prototypes for Medical Image--Report Representation Learning
链接:https://arxiv.org/abs/2611.24002
作者:Jinrui Wu,Yun Peng
类目:Computer Vision and Pattern Recognition (cs.CV); Artificial Intelligence (cs.AI)
关键词:Radiology reports distinguish, single study-level target, conventional global image, reports distinguish affirmed, distinguish affirmed findings
备注:
点击查看摘要
Abstract:Radiology reports distinguish affirmed findings, explicit negations, and unmentioned concepts, but conventional global image--report contrastive learning often compresses these different evidence states into a single study-level target. We introduce \model{}, which combines symmetric global image--report alignment with observation-masked finding supervision and a soft relation objective over affirmed report concepts. Text-derived finding prototypes are aligned with local visual tokens, while uncertain and unmentioned concepts are excluded from direct finding labels. We evaluate the framework on three CT benchmarks and five chest-radiograph benchmarks, together with component ablations, paired and clinical retrieval, localization, perturbation robustness, calibration, and per-finding analyses. \model{} achieves macro-AUROC values of 73.6%, 66.5\%, and 68.8% on CT-RATE, RAD-ChestCT, and INSPECT, and 80.2%, 91.4%, 78.5%, 86.5%, and 90.8% on ChestXray14, CheXpert, PadChest, RSNA, and SIIM-ACR, respectively. Relative to the matched global-only controls, the findings and relation objectives yield consistent gains across the evaluated datasets. These results demonstrate the practical value of treating observed and unobserved report concepts differently during medical image--text representation learning.
126. 【2611.23996】ReSynth: Constructing Synthetic Supervision for Multi-Image Understanding
链接:https://arxiv.org/abs/2611.23996
作者:Yilin Wang,Xin Yu,Xinyi Guo,Xiangxi Zheng,Dongxing Mao,Linjie Li,Zhengyuan Yang,Alex Jinpeng Wang
类目:Computer Vision and Pattern Recognition (cs.CV)
关键词:People often understand, understand the world, world by comparing, People, views
备注:
点击查看摘要
Abstract:People often understand the world by comparing different views of it. Vision-language models (VLMs), however, receive these views as separate images and still struggle to relate them. One problem is data. Training on these tasks requires groups of images with a known relationship: the same object from different viewpoints, or the same scene before and after a change. Such examples are hard to collect at scale from existing photo collections. We introduce ReSynth, which turns single-image perception and controllable image generation into cross-image relational supervision. Localized editing produces before-and-after pairs, viewpoint generation produces multiple views of the same instance, and verified scene contents support comparisons of counts and attributes. ReSynth combines Qwen3-VL-8B's captioning, grounding, and counting capabilities with Qwen-Image-2.1's generation and editing capabilities. It verifies single-image facts such as object counts and attributes, then combines these facts to construct multi-image QA. Using GRPO on approximately 440K ReSynth questions, with no real images during training, Qwen2.5-VL-7B and Qwen3-VL-8B improve by 17.73 and 6.10 average points on BLINK, MUIRBench, and VLM2-Bench, while single-image performance is broadly preserved on Qwen2.5-VL-7B and improves across all four evaluated benchmarks on Qwen3-VL-8B. Notably, this supervision enables the 8B perception model itself to outperform Qwen3-VL-235B on BLINK and MUIRBench.
127. 【2611.23994】WaveView: The Novel View Is in the Noise
链接:https://arxiv.org/abs/2611.23994
作者:Kaiwen Wang,Jan-Hendrik Pauls,Yinzhe Shen,Frank Bieder,Christoph Stiller
类目:Computer Vision and Pattern Recognition (cs.CV)
关键词:target camera view, view synthesis, Generative novel view, video diffusion models, camera view
备注:
点击查看摘要
Abstract:Generative novel view synthesis (NVS) employs a diffusion model to render a static or dynamic scene from a target camera view. Explicit warp methods complete a reprojection of the source with video diffusion models, and implicit methods encode the camera parameters or 3D cues. Both add inference overhead, and the generated layout often drifts from the ground truth. We present WaveView, which injects the warped source into the phase of the initial noise of a pretrained generator. The Dual-Tree Complex Wavelet Packet Transform (DT-CWPT) allows local and anisotropic phase injection. The generator is thus constrained less in disoccluded regions and along the epipolar line. WaveView keeps the architecture and adds negligible overhead. On the DL3DV benchmark, it outperforms all baselines in every overlap bin. In the hardest bin, it gains 0.84 dB PSNR on the covisible region over MetaView, which shares its generator, while running 2.1 times faster. For dynamic NVS, fine-tuning AnyView with the same noise raises its zero-shot PSNR on AnyViewBench from 11.58 to 12.35 dB.
128. 【2611.23991】SIRAZ: Sparse Inter-Layer Residual Aggregation for Zero-Shot Visual Neural Decoding
链接:https://arxiv.org/abs/2611.23991
作者:Neha,Mudasir Ganaie
类目:Computer Vision and Pattern Recognition (cs.CV)
关键词:Brain-computer interfaces aim, establish direct communication, Brain-computer interfaces, neural, visual
备注:
点击查看摘要
Abstract:Brain-computer interfaces aim to establish direct communication between neural activity and external systems, making interpretation of brain signals an important problem. In this context, neural visual decoding seeks to infer visual information from neural activity by learning a mapping between brain responses and the corresponding visual stimuli. It investigates how visual information emerges from neural signals and relates to perceptual processing while enabling the reconstruction or retrieval of visual stimuli from neural signals. Our brain primarily follows a hierarchical pathway to process visual information captured by the eye. However, neural representations learned by existing methods differ fundamentally from those produced by the biological visual system. To bridge this gap and mimic the human vision hierarchy, we introduce Sparse Inter-layer Residual Aggregation for Zero-shot (SIRAZ). SIRAZ is a framework that aligns neural visual representations by jointly modeling early visual processes via fovea- and magnocellular-pathway-inspired Difference-of-Gaussians transformations. To align the semantic information with neural information, it extracts multiple intermediate representations of the image at multiple depths and passes them through the Residual Gated Adapter Mechanism. To study the temporal dynamics of neural signals, the Direction-aware Multi-Scale Temporal Attention Encoder is introduced to align the learned neural representation with the final visual representation. The proposed framework achieves 92.5% top-1 and 99.27% top-5 accuracy in the intra-subject setting on the THINGS-EEG dataset, and 50.25% and 75% top-1 and top-5 accuracy in the THINGS-MEG dataset. The results indicate that combining biologically motivated visual representations with the structured modalities of neural dynamics improves cross-modal alignment and zero-shot image retrieval.
129. 【2611.23972】Agreement Is Not Accuracy: LiDAR Elevation Errors Hidden in LiDAR-Camera Extrinsics
链接:https://arxiv.org/abs/2611.23972
作者:Yeon Kun Lee,Dongseok Yang,Deok-Kyeong Jang,Choongho Chung,Junyoung Lee,Hyeongseok So
类目:Computer Vision and Pattern Recognition (cs.CV)
关键词:provide complementary measurements, complementary measurements, cameras provide complementary, systematic LiDAR elevation, LiDAR elevation biases
备注: 14 pages, 6 figures, including supplementary material
点击查看摘要
Abstract:LiDAR and cameras provide complementary measurements for 3D vision, yet agreement between their observations does not establish the accuracy of either sensor's internal geometry. We show that systematic LiDAR elevation biases can be absorbed into the estimated extrinsics. Using synchronized LiDAR intensities and camera-estimated checkerboard poses, we jointly estimate a sensor-specific elevation correction shared across recordings and extrinsics for each recording. Ray-plane intersections locate the checker pattern without using reported ranges, preventing range errors from directly moving the lookup. Under matched intensity and correction models, this lookup gives smaller camera-consistency residuals than measured-point lookup. Across six identically designed rigs using the same sensor models, recording-excluded corrections reduce the mean translation-reference distance from 7.57 to 3.04 mm. On separate structures, they reduce inter-plane angle errors against references from independent dense camera scans and improve image-edge alignment. Rolling the rigs shifts the plane pairs exhibiting the largest angular improvements, as predicted for an elevation-only correction. Within the ray-plane model, calibration-target projections and loop-consistency residuals change little despite these geometric improvements. These findings show why agreement after extrinsic calibration need not establish correct internal LiDAR geometry. Synchronized observations of a familiar checkerboard provide a practical route to estimating and testing elevation corrections without a dedicated large calibration field. We plan to release code and data for reproducing the main results.
130. 【2611.23970】OMAS: Topology-Aware Operation Modeling and Adaptive Staging for Egocentric Hand Forecasting
链接:https://arxiv.org/abs/2611.23970
作者:Guanzhong Sun,Junyi Ma,Yuxuan Wu,Erhang Zhang,Chenyang Xu,Yanzi Miao,Hesheng Wang
类目:Computer Vision and Pattern Recognition (cs.CV); Artificial Intelligence (cs.AI)
关键词:understanding human intentions, egocentric videos plays, videos plays, plays an important, important role
备注:
点击查看摘要
Abstract:Forecasting future hand movements from egocentric videos plays an important role in understanding human intentions and hand-object interactions. However, different manipulation intentions can lead to substantially different hand trajectories even when interacting with the same object. Existing methods have not sufficiently explored the intrinsic relationships between articulated hand movements and object semantics to capture such operation-dependent motion patterns. Therefore, we propose TOMAS, a topology-aware operation modeling and adaptive staging framework for egocentric hand forecasting. Specifically, we develop a topology-aware hand-object reasoning mechanism that associates hand skeletal structures and articulation dynamics with object semantics and spatial relationships to jointly infer potential interaction targets and latent manipulation operations. The resulting object-operation representations provide interaction conditions for future hand trajectory prediction, with supervision adapted to the annotations available in each dataset. Furthermore, we introduce an adaptive staging mechanism that organizes future hand movements into ordered latent stages with learnable temporal boundaries. Stage-specific geometric subgoals are then incorporated to guide trajectory refinement according to the underlying manipulation patterns. Experimental results on multiple publicly available benchmarks demonstrate that TOMAS achieves state-of-the-art performance in multi-dimensional and multi-target hand motion forecasting.
131. 【2611.23964】Dual-Manifold Variational Autoencoders
链接:https://arxiv.org/abs/2611.23964
作者:Hongxu Chen,Chun Wang,Yanghao Wang,Hongxiang Li,Zhen Wang,Zhixiang Zhang,Long Chen
类目:Computer Vision and Pattern Recognition (cs.CV); Artificial Intelligence (cs.AI)
关键词:latent diffusion models, variational autoencoders, reconstruction quality, dimension of variational, essential to push
备注:
点击查看摘要
Abstract:In latent diffusion models, expanding the latent dimension of variational autoencoders (VAEs) is essential to push the upper bound of reconstruction quality. However, without explicit structural constraints, such unregularized expansion easily yields a rugged and ill-conditioned latent manifold, causing generated latents to drift off the manifold during generation (i.e., off-manifold problem). Recent works smooth this manifold by using external foundation models (e.g., DINOv2) as a topological regularizer. However, we argue that tying the latent space to frozen external priors inherently restricts generalization: bound by their pre-training distributions and semantic biases, they may fail to capture faithful visual details on out-of-distribution domains. Observing that low-dimensional latents naturally form a smoother, generation-friendly manifold, we propose dual-manifold VAE (DM-VAE) to leverage internal representations as a topological regularizer. Specifically, DM-VAE pairs a high-dimensional reconstructive branch with a low-dimensional branch that extracts a clean topological skeleton, anchoring the high-dimensional manifold to this smooth subspace. Building upon this framework, we further introduce topological representation alignment (TRA) to supervise intermediate generator features during training, and internal guidance with topological skeleton (IG_TS) to guide structural components while preserving high-frequency details at inference. On ImageNet-1K (256x256), our foundation-model-free DM-VAE achieves a state-of-the-art rFID of 0.12 and a generation FID50K of 1.28 (w/o CFG) and 1.20 (w/CFG), and further reaches 1.11 when orthogonally combined with DINOv2.
132. 【2611.23962】SEE: Unlocking the Hidden Perception of Frozen VLMs
链接:https://arxiv.org/abs/2611.23962
作者:Alin-Ionut Popa,Tuo Zhang,Dimitrios Dimitriadis
类目:Computer Vision and Pattern Recognition (cs.CV); Artificial Intelligence (cs.AI)
关键词:Vision-language models, dense infographic, miss the fine, large photo, Vision-language
备注:
点击查看摘要
Abstract:Vision-language models (VLMs) grasp the overall content of an image but often miss the fine details a question hinges on, such as a small object in a large photo or a number in a dense infographic. Yet even when their answer is wrong, the region they point to often contains the evidence. We introduce Self-Elicited Evidence (SEE), a training-free method that lets a frozen VLM look again where it points. When the initial answer has low confidence, SEE prompts the model to propose regions where the details may appear, answers the question on these enlarged crops, and aggregates the answers with the original prediction through confidence-weighted voting. The model also answers with its own transcript, which helps on text-rich images. SEE needs no detector or OCR engine. Across five backbones, SEE improves every small-object benchmark by 2.3-13.1 points and outperforms five training-free baselines on small objects. With SEE, Qwen3.5-4B surpasses the direct answer of Qwen3.5-9B and matches the 6 times larger Qwen3.5-27B.
133. 【2611.23958】DORMS: Image Segmentation via Depth-Ordered Region Merge Trees
链接:https://arxiv.org/abs/2611.23958
作者:Furkan Mumcu,Yasin Yilmaz
类目:Computer Vision and Pattern Recognition (cs.CV)
关键词:introduce DORMS, monocular depth maps, depth maps, Depth-Ordered Region Merging, Region Merging
备注:
点击查看摘要
Abstract:We introduce DORMS (Depth-Ordered Region Merging for Segmentation), a purely algorithmic approach to image segmentation from monocular depth maps, requiring no neural networks or learned parameters in the segmentation process. An off-the-shelf monocular depth estimator provides the input depth maps. Our method constructs depth-ordered merge trees through a deterministic union-find sweep, combining component area, depth persistence, and spatial contact constraints to guide region merging. A threshold search determines the segmentation granularity according to a specified target region count, while global depth detrending improves the separation of local geometric structures. Extensive experiments on COCO Panoptic, Cityscapes, and ADE20K demonstrate consistent improvements over classical segmentation baselines and competitive performance against zero-shot neural segmentation models. Notably, our method outperforms all evaluated zero-shot models on Cityscapes, while remaining competitive with several approaches on COCO and ADE20K. These results highlight the potential of purely algorithmic geometric reasoning for image segmentation without learned segmentation models.
134. 【2611.23954】SimplePro: A Simple Baseline for Proactive Video Understanding
链接:https://arxiv.org/abs/2611.23954
作者:Yilin Wang,Xiangxi Zheng,Dongxing Mao,Linjie Li,Zhengyuan Yang,Alex Jinpeng Wang
类目:Computer Vision and Pattern Recognition (cs.CV)
关键词:newly arrived evidence, understanding requires vision-language, arrived evidence warrants, requires vision-language models, Proactive video understanding
备注:
点击查看摘要
Abstract:Proactive video understanding requires vision-language models (VLMs) to decide not only what to answer as a video unfolds, but also when newly arrived evidence warrants a response and when to remain silent. Although modern VLMs excel at offline long-video understanding, they struggle in proactive streaming settings, leading prior work to introduce task-specific reinforcement learning or auxiliary triggering modules. We find that explicit reporting rules and structured streaming context substantially improve a frozen VLM's proactive response performance. We exploit this observation to build SIMPLEPRO, a training-free baseline for proactive video understanding. At each decision step, the model combines recent high-resolution frames, bounded historical observations, and prior answer records to determine whether new information warrants a response and, when needed, generate a concise answer. The system then filters consecutive duplicate answers while retaining the raw outputs as context for subsequent decisions. Without any parameter updates or auxiliary triggers, SIMPLEPRO with a 4B backbone consistently outperforms specialized proactive systems across OVO-Bench, StreamingBench-PO, and ProactiveVideoQA. SIMPLEPRO shows overall performance gains as backbone size increases from 2B to 32B. We hope this simple yet effective method will serve as a strong baseline for proactive video understanding, and we will release our code and inference pipeline to support future research.
135. 【2611.23953】MitoWorld: A Stochastic World Model of the Single-Cell Cycle under Genetic Perturbation
链接:https://arxiv.org/abs/2611.23953
作者:Ting Long,Xiaojing Sun,Cheng Han,Zhicheng Ji,Yingnian Wu,Jiebo Luo,Heng Fan,Dongfang Liu
类目:Computer Vision and Pattern Recognition (cs.CV); Quantitative Methods (q-bio.QM)
关键词:cell observed, Gaussian reference, uncertain future, Gaussian reference latent, gene knockdown
备注:
点击查看摘要
Abstract:A cell observed after a gene knockdown has an uncertain future, both in how its appearance will evolve and when it will divide. Existing approaches predict endpoints or population-level changes without jointly modeling individual trajectories and division timing. In this work, we present MitoWorld, a stochastic world model that generates these future appearances as continuous evolution in a Gaussian reference latent, terminated by a division hazard and converted into images by a conditional diffusion renderer. The ordinary transition provably preserves its Gaussian reference, tying its noise to its dynamics; paired observations identify its discrete transition where stationary marginals cannot. The hazard turns division timing into a survival likelihood that can be tested against real cycle lengths. On 325 held-out MitoCheck tracks spanning 17 knockdowns and a control, MitoWorld lowers FID by 34.6\% and the video metric JEDi by 45.6\% relative to the strongest baselines. Its latent-space prediction intervals achieve near-nominal 90\% coverage, are 35\% narrower than those of a model with freely fitted noise. The hazard also yields cell-cycle length distributions with KS distance close to the model-based finite-sample reference. Testable single-cell stochastic dynamics thus offer a generative tool for time-lapse perturbation screens.
136. 【2611.23950】VOL-Agent: Programming Your Film via Video Orchestration Language Agentic System
链接:https://arxiv.org/abs/2611.23950
作者:Hongji Yang,Yucheng Zhou,Lingting Zhu,Shengju Qian,Xinye Li,Huan Zheng,Dongyue Lu,Wei Chow,Xin Wang,Cheng-zhong Xu,Jianbing Shen
类目:Computer Vision and Pattern Recognition (cs.CV)
关键词:Recent video foundation, story-coherent films remains, video foundation models, Recent video, foundation models generate
备注: 14 pages, 6 figures
点击查看摘要
Abstract:Recent video foundation models generate high-quality and limited-length clips, but orchestrating them into minute-level, story-coherent films remains an open problem. Existing video agents highly rely on natural-language planning or predefined workflows, which make their production hard to program, customize, and extend. Therefore, we introduce VOL-Agent, a programmable video agent built on a Video Orchestration Language (VOL). With static compilation and verification, followed by runtime dynamic loading, VOL-Agent decouples structural reliability from execution flexibility. Through the skill-driven executor, VOL-Agent dynamically resolves tools and skills from an open-world repository and supports user-defined skills in natural language. Moreover, VOL-Agent is capable of crafting new director skills from learned knowledge and improves itself through feedback to accumulate filmmaking knowledge. To overcome the disconnect between different segments, we apply a structural memory mechanism to record and recall world and entity states, sustaining entity and scene consistency across different shots. Critic feedback supports local revision, while reusable filmmaking knowledge and experiences can be retained for subsequent tasks. Experiments on three benchmarks show that VOL-Agent generates minute-scale videos with strong overall quality and limited cost.
137. 【2611.23946】How Complete Must an Environment Be to Train a Capable GUI Agent?
链接:https://arxiv.org/abs/2611.23946
作者:Jingxing Wang,Teng Wang,Xingyu Lou,Zihan Guo,Jun Wang,Weiwen Liu,Weinan Zhang,Jianghao Lin
类目:Computer Vision and Pattern Recognition (cs.CV); Artificial Intelligence (cs.AI); Machine Learning (cs.LG)
关键词:Graphical user interface, executable GUI environments, executable GUI, Graphical user, GUI
备注: 15pages, 6 figures
点击查看摘要
Abstract:Graphical user interface (GUI) agents are widely trained in executable GUI environments. To lower cost, build environments in batches, and reduce manual effort, each work trades away part of the environment through its construction choices. We describe these choices along three dimensions: reward coverage, the goal-reaching trajectories the verifier rewards; data coverage, the application data the agent is exposed to beyond what the task reads or writes; and feature coverage, the application features beyond what the task uses. Existing works set each dimension on engineering grounds, but they do not measure its effect on training. How complete must an environment be to train a capable agent, and along which dimensions? In MobileGym, we adjust each dimension separately, train Qwen3-VL-4B-Instruct, and evaluate every policy in the original full environment. A verifier that rewards only one reference path lowers success by 10.53 points, and removing unneeded records also lowers success. Pruning unneeded features instead raises success from 51.61% to 56.13%, entirely on tasks whose environments it left unchanged. The gain reverses when each episode is weighted equally and vanishes when only episodes on the pruned tasks are balanced. Without pruning, length weighting there reinforces long detour successes that erode skills on other tasks. Following these findings, we build MobileGym-Lite, which keeps reward and data coverage complete and prunes unneeded features. It outperforms Full in the full environment (55.48% versus 51.61% success) with lower construction cost.
138. 【2611.23944】SIGNTOK: A Text-Aligned Continuous Pose Tokenizer for Sign Language Generation
链接:https://arxiv.org/abs/2611.23944
作者:Nabeela Khan,Gaurav Rai,Bowen Wu,Sihan Tan,Benjamin Yen,Takeshi Ashizawa,Kazuhiro Nakadai
类目:Computer Vision and Pattern Recognition (cs.CV)
关键词:Text-conditioned sign language, Text-conditioned sign, sign language generation, SLG, Text-conditioned
备注:
点击查看摘要
Abstract:Text-conditioned sign language generation (SLG) depends not only on the generator but also on the space in which signing is generated. Existing approaches generate directly in pose space, in quantized code spaces, or in learned latent spaces trained for reconstruction. We find that these spaces are organized largely by signer identity rather than linguistic content: on CSL-Daily, a test clip's nearest neighbor in raw pose space is the same sentence signed by another person in only 2% of cases. To address this, we propose SIGNTOK, a continuous pose tokenizer whose latent space is shaped by three training-only heads: a kinematics head that predicts motion dynamics, a sentence-level head that aligns signing with the overall text, and a word-level head that aligns short motion windows with content words. Together, these objectives raise the same-sentence cross-signer nearest-neighbor rate to 82%. On three sign language benchmarks, the same generator trained in the SIGNTOK latent instead of pose space raises back-translation BLEU-4 in five of six benchmark-scorer settings at comparable joint accuracy, and word-level alignment gives higher back-translation scores than sentence-level alignment on every benchmark.
139. 【2611.23941】Angular Conditional Distillation: Overcoming Classifier Geometry Mismatch
链接:https://arxiv.org/abs/2611.23941
作者:Xuefei Zhang,Ruoqi Wang,Lei Cui
类目:Computer Vision and Pattern Recognition (cs.CV); Artificial Intelligence (cs.AI)
关键词:fundamental and univer, knowledge distil, Angular Conditional Distilla, Angular Coordinate Interface, Abstract
备注:
点击查看摘要
Abstract:Classifier outputs provide a fundamental and univer- sally available knowledge interface for knowledge distil- lation (KD). However, standard logit objectives entangle transferable semantic relations with architecture-dependent parameterization artifacts, causing a classifier geometry mismatch that makes direct knowledge transfer unreliable. To address this, we propose Angular Conditional Distilla- tion (ACD). First, the Angular Coordinate Interface (ACI) reconstructs the student classifier using angular coordi- nates, suppressing magnitude and bias effects to provide a geometry-compatible space. Then, Scale-Invariant Condi- tional Distillation (SICD) standardizes outputs and exclu- sively transfers conditional non-target relations, filtering out untransferable absolute confidence. Together, ACI de- fines where to transfer, while SICD defines what to transfer. Extensive experiments show that ACD achieves state-of-the- art performance over existing KD methods. Moreover, as a universal drop-in replacement, it consistently improves existing distillation pipelines while requiring no feature adapters or additional inference-time cost. Code is avail- able at this https URL.
140. 【2611.23935】Facet-SLAM: Real-Time Triangle Splatting SLAM
链接:https://arxiv.org/abs/2611.23935
作者:Edward Beng Wai Tan,Siew-Kei Lam
类目:Robotics (cs.RO); Computer Vision and Pattern Recognition (cs.CV)
关键词:Gaussian splatting build, Dense SLAM systems, splatting build maps, mesh requires extraction, Dense SLAM
备注:
点击查看摘要
Abstract:Dense SLAM systems based on 3D Gaussian splatting build maps that render photorealistically, but their surface geometry is poor, and obtaining a mesh requires extraction or fusion stages. Triangle splatting avoids this with surface primitives whose vertices can be triangulated directly into a connected mesh, yet SLAM with this representation has so far relied on photometric tracking, which is too slow for localization at sensor rate. We present Facet-SLAM, the first real-time triangle splatting SLAM system for RGB-D cameras. Fast geometric registration is the natural alternative, but it is not sufficient alone, since the mapper's optimization displaces triangles from the observed surface and makes them unreliable registration targets. We therefore separate the two roles: the tracker sets each triangle's centroid from fused depth and registers frames to these centroids by GICP, while the mapper is restricted to non-translating updates via a change of variables. Our system achieves competitive tracking accuracy at around 100 FPS on a desktop GPU and runs at sensor rate on a Jetson AGX Orin. Compared with the photometric approach, it is over 20 times faster, matches rendering quality, and improves surface accuracy, reducing the chamfer distance of the Delaunay mesh on Replica from 1.55 to 0.86 cm. We will release the implementation at this https URL.
141. 【2611.23934】What Right-Angle Corners Reveal About an Uncalibrated Camera: Convex Cones, Certified Bounds and Identifiability
链接:https://arxiv.org/abs/2611.23934
作者:Michael Werman
类目:Computer Vision and Pattern Recognition (cs.CV); Algebraic Geometry (math.AG); Metric Geometry (math.MG)
关键词:perpendicular edges meet, segments meeting, edges meet, perpendicular edges, Abstract
备注: 26 pages
点击查看摘要
Abstract:The corner of a box or a room, where three perpendicular edges meet, appears in an image as three segments meeting at a point. We ask what such junctions reveal about an uncalibrated camera's intrinsic parameters. Each junction is consistent with an explicit convex set of cameras; when only the focal length is unknown, this set is an interval in closed form. These sets are computed from the image alone, so calibrating from many junctions is a convex feasibility problem. Almost every junction bounds the focal length from below; we characterize when one also bounds it from above, and when many junctions determine it exactly. For a known vertex, the bounds can be certified against any bounded error in the measured edge directions.
142. 【2611.23933】RoPE: Topology-aware Relative Positional Encoding for 4D Generation
链接:https://arxiv.org/abs/2611.23933
作者:Chengzhuo Gui,Zhenglin Zhou,Fan Ma,Zongxin Yang,Yi Yang
类目:Computer Vision and Pattern Recognition (cs.CV)
关键词:made substantial progress, recovering dynamic, monocular videos, generation has made, made substantial
备注: Project page: [this https URL](https://akui1321.github.io/TRoPE)
点击查看摘要
Abstract:Video-to-4D mesh generation has made substantial progress in recovering dynamic 3D geometry and motion from monocular videos. However, commonly used feed-forward approaches remain prone to sticking artifacts, where geodesically distant vertices move together incorrectly when they become spatially close. We attribute this failure to cross-part motion leakage: spatially nearby yet structurally distant vertices may retrieve similar motion information when motion decoding fails to distinguish spatial proximity from mesh connectivity. To address this issue, we introduce TRoPE, a topology-aware relative positional encoding that makes mesh connectivity explicit in the interaction between vertex queries and mesh-supported motion sources. TRoPE derives spectral features from the graph Laplacian and constructs pairwise topological relations from distances between multi-scale diffusion features, using these relations to modulate vertex--anchor attention through rotary phases. We further introduce a vertex-wise tail-risk objective that emphasizes persistently high-error vertices to explicitly suppress sticking artifacts. To better characterize and quantify these failures, we introduce MeshSticking-4D, a dedicated benchmark for this failure mode. Experiments on MeshSticking-4D show that TRoPE suppresses sticking artifacts while improving overall motion accuracy. Compared with the baseline, it reduces the worst-vertex error by 39.4% and lowers the separation failure rate from 49.35% to 34.90%. Additional evaluations on Motion-80 and Motion324-Subset show lower tail errors on diverse objects, while transfer to Mesh4D demonstrates applicability across backbones.
143. 【2611.23929】ProxySearch: Decoupled Inference-Time Scaling for Diffusion Models via Asymmetric Noise-Rank Transfer
链接:https://arxiv.org/abs/2611.23929
作者:Seungwook Kim,Jongmin Lee
类目:Computer Vision and Pattern Recognition (cs.CV)
关键词:inference time improves, multi-step teacher rollouts, Noise search, inside noise search, time improves
备注: Accepted to NeurIPS 2026. 26 pages, 7 figures
点击查看摘要
Abstract:Noise search at inference time improves text-to-image diffusion but requires many multi-step teacher rollouts per query. Distilled few-step models are now routinely released, yet their potential as proxies inside noise search is uncharacterized. We measure proxy-teacher rank agreement across three distillation regimes and three backbones (DiT, MMDiT, UNet) and find a sharp asymmetry. On our main pool, closed-form image statistics transfer from the proxy to the teacher (Spearman $\rho$ up to 0.80), while learned semantic scorers do not ($\rho \le$ 0.12). Paired null controls trace this rank to the first denoising step for coarse statistics and to distillation for finer ones. We use the asymmetry for noise search under a joint constraint: a semantic reward specifies content, and a numeric target specifies image statistics. In PROXYSEARCH, the proxy filters candidates by statistical match in 1 step, and the teacher returns the survivor with the highest semantic reward. No weight trades one objective against the other. On FLUX-1024, PROXYSEARCH tightens the statistical target by 21-23% at the same number of teacher trajectories, with no detected loss on the optimized reward.
144. 【2611.23908】raining-Free Domain Generation for One-Shot Federated Learning with Server-Side Pre-trained Diffusion Models
链接:https://arxiv.org/abs/2611.23908
作者:Yixuan Tong,Yuanguo Bi,Mingjian Zhi,Wenchao Xu,Yuan Wu
类目:Artificial Intelligence (cs.AI); Computer Vision and Pattern Recognition (cs.CV); Distributed, Parallel, and Cluster Computing (cs.DC)
关键词:One-shot federated learning, attracted increasing attention, One-shot federated, limiting client-server interaction, single communication round
备注:
点击查看摘要
Abstract:One-shot federated learning (OSFL) has attracted increasing attention for its ability to substantially reduce communication overhead by limiting client-server interaction to a single communication round. Nevertheless, this limited interaction makes learning from heterogeneous client distributions challenging. Diffusion-based OSFL approaches seek to address this challenge by synthesizing data that approximate client distributions. However, existing diffusion-based approaches often impose substantial computational costs on clients and lack effective visual guidance to generate data aligned with client distributions. To address these limitations, we propose FedDoG, a training-free Domain Generation framework that leverages server-side pre-trained diffusion models to synthesize data aligned with client domains. FedDoG extracts compact domain representations using a frozen image encoder, substantially reducing client-side computational and communication costs. These representations are incorporated into the diffusion model through a pre-trained image-conditioning adapter, thereby providing explicit visual guidance for domain generation. To further improve the quality of synthetic data, FedDoG adapts the strength of visual conditioning according to domain concentration and filters generated samples based on class consistency and predictive entropy. Experiments on three benchmarks demonstrate that FedDoG generates higher-quality synthetic images and outperforms existing OSFL baselines in classification accuracy. Meanwhile, FedDoG reduces client upload and computation time by 99.9% and 91.5%, respectively. The code is available at this https URL.
145. 【2611.23900】Coverage Governs Accuracy: When to Expand the Lexicon and When to Fall Back in Low-Resource Manchu OCR
链接:https://arxiv.org/abs/2611.23900
作者:Ruixin Yu,Zifan Wang,Mingxin Ding,Xiaoxin Sun,Dongjie Wang,Suhua Wang,Minghao Yin
类目:Computation and Language (cs.CL); Computer Vision and Pattern Recognition (cs.CV)
关键词:produces near-miss spelling, near-miss spelling errors, Character-level OCR, produces near-miss, near-miss spelling
备注: 15 pages, 11 figures, 6 tables
点击查看摘要
Abstract:Character-level OCR for low-resource Manchu often produces near-miss spelling errors. Lexicon-constrained decoding can correct such errors with lexical knowledge, but a fixed dictionary cannot cover the many unseen word forms produced by agglutinative word formation. We study whether morphological expansion alleviates this limitation, and how the expansion size and confidence fallback affect recognition. We segment the training transcriptions with a dictionary-based stem list, extract roots and suffix chains from them, generate about 510k candidates, select the expansion size through frequency ranking on the validation set, and decompose accuracy under hard lexicon constraints into the product of coverage and in-lexicon precision. Across three CRNN training seeds, adding 250k candidates to a base dictionary of about 29k words raises the type accuracy on unseen combinations of seen roots from 63.6% to 86.3%, and overall token accuracy from 82.4% to 84.2%. Adding all candidates lowers overall token accuracy to 81.5%, showing that the coverage gain comes with a loss of in-lexicon precision. A further comparison shows that the base dictionary with confidence fallback performs best overall among the CRNN systems tested, reaching 76.1% type and 85.5% token accuracy and outperforming the expanded lexicon both with and without fallback. A Transformer recognizer shows similar trends. These results indicate that morphological expansion relieves the constraint of a fixed dictionary, but a larger lexicon does not guarantee better recognition; in the present low-resource Manchu OCR setting, a selective combination of reliable lexical constraints and open recognition is more effective.
146. 【2611.23891】SixthView: A Benchmark for Hidden-View Reconstruction and 3D Assembly Reasoning
链接:https://arxiv.org/abs/2611.23891
作者:Pooya Khosravi,Jen-Feng Chang,Fnu Pramono
类目:Computer Vision and Pattern Recognition (cs.CV); Artificial Intelligence (cs.AI)
关键词:Inferring the unseen, unseen side, Inferring, part placement, exact
备注: 13 pages
点击查看摘要
Abstract:Inferring the unseen side of a 3D assembly requires reasoning about part placement and occlusion. Different assemblies can match the observed views while producing different hidden views, leaving the target ambiguous. We introduce SixthView, a benchmark in which models predict a sixth orthographic view from five observed views and information about the component shapes. The benchmark contains 602 synthetic polycube assemblies with 2--10 parts and requires a dense grid of part labels for the missing view, allowing both exact-view and cell-level evaluation. For each item, we exhaustively verify that all legal assemblies consistent with the exact component geometry and observed label grids yield the same target view. With these inputs, the task tests symbolic assembly and coordinate reasoning. The image-only variants additionally require shape recovery from rendered views, where occlusion can make different 3D shapes look identical. We evaluate three vision-language models under four input conditions. Across 552 items with complete evaluations, exact-grid accuracy with images plus exact text ranges from 33.2% to 85.4%. Adding exact text to separate images improves accuracy by 19.7--63.0 percentage points, whereas adding images to exact text changes it by only $-0.5$ to $+2.1$ points. Separate image panels outperform a merged sheet by 6.7--11.1 points. The results show that performance depends strongly on access to exact symbolic information, with little further improvement from images once the geometry is specified. GitHub: this https URL
147. 【2611.23889】Beyond Additive Composition: Interventional Modeling of State-Dependent Cross-Degradation Responses for Composite Image Restoration
链接:https://arxiv.org/abs/2611.23889
作者:Zongda Tang,Jie Feng,Wenda Zhao,Mingtao Feng,Yuping Liang,Ronghua Shang,Weisheng Dong
类目:Computer Vision and Pattern Recognition (cs.CV)
关键词:recover images corrupted, image restoration aims, Composite image restoration, recover images, images corrupted
备注:
点击查看摘要
Abstract:Composite image restoration aims to recover images corrupted by multiple coexisting degradations. Existing methods predominantly decompose composite corruptions into individual factors, relying on these isolated priors to guide restoration. However, coexisting degradations are inherently interdependent: the manifestation of one degradation dynamically shifts with the states of others, inducing non-additive interactions that factor-wise paradigms fail to capture. To bridge this gap, we formulate state-dependent cross-degradation responses and propose PROBE, an interventional framework that explicitly models how coexisting degradations influence one another. Specifically, we first construct unified degradation axes by aligning visual and textual degraded-clean offsets. Along these axes, a calibrated coordinate head estimates current degradation states, while bidirectional feature perturbations are injected to probe cross-coordinate responses under the current states. Contrasting these responses with corresponding intensity-matched single-degradation references quantifies intrinsic non-additive deviations, allowing the resulting response descriptors and state coordinates to jointly guide restoration. For comprehensive evaluation, we introduce SCD-15, a new benchmark covering all 15 nonempty combinations of four degradation factors at three severity levels. Experiments demonstrate that PROBE achieves state-of-the-art performance on the CDD-11, AWMM-100K, and SCD-15 composite restoration datasets using a compact model with modest GPU memory requirements.
148. 【2611.23887】Can AI Understand Indonesian Children's Book Illustrations? Compliance, Safety, and the Cultural Alignment Gap
链接:https://arxiv.org/abs/2611.23887
作者:Rian Adam Rajagede,Rochana Prih Hastuti,Fitri Hasanah Amhar
类目:Computer Vision and Pattern Recognition (cs.CV)
关键词:illustrations carry developmental, Indonesia Book Leveling, carry developmental, formally codified, book illustrations carry
备注: Accepted at Workshop on Computer Vision for Children (CV4CHL) at CVPR 2026 (Non-Proceeding Track)
点击查看摘要
Abstract:Children's book illustrations carry developmental, pedagogical, and cultural responsibilities that are often formally codified in guidelines. As AI systems begin entering children's publishing pipelines, both as evaluators and as generators, a natural question arises: can they reliably meet the structural, safety, and cultural standards required? We investigate this across three criteria: guideline compliance, safety, and cultural alignment, using Indonesia's Book Leveling Guideline as a government-issued evaluation framework with explicit visual criteria for four reading levels. To understand how well VLMs can identify and apply these guideline criteria, we first evaluate them on real published children's book pages and compare their assessments against a human baseline. To then examine whether AI-generated illustrations meet the same standards, we apply the VLM judge to images produced by two generation models and cross-check them against a subset of human illustrator raters. We find that VLMs are reliable on explicit criteria such as compliance and safety, but fall short on cultural alignment, systematically missing content that human raters flag as culturally inappropriate. AI-generated illustrations show the same blind spot: they satisfy structural and safety criteria but fail to capture implicit cultural details that human raters consistently identify.
149. 【2611.23881】NullTok: empirical-null token selection for training-free long video understanding
链接:https://arxiv.org/abs/2611.23881
作者:Zhang Ziyue,Juwei Jian,Baiyang Song,Yunshan Zhong,Jun Peng,Yiyi Zhou,Rongrong Ji
类目:Computer Vision and Pattern Recognition (cs.CV); Artificial Intelligence (cs.AI)
关键词:large language models, existing multimodal large, multimodal large language, Long video understanding, video understanding remains
备注: 8 pages
点击查看摘要
Abstract:Long video understanding remains a daunting challenge for existing multimodal large language models (MLLMs), mainly due to the large number of visual tokens accumulated in long videos. Currently, frame selection and token selection are two main paradigms for addressing this issue, both aiming to select relevant visual information to form a compact input for MLLMs. A natural approach combines these two paradigms by selecting frames before pruning tokens, but noisy frame-level relevance estimates can cause irreversible evidence loss. In this paper, we study this problem from the perspective of assessing the spatiotemporal distribution of query-specific evidence and propose a novel and training-free approach termed NULLTOK. For every patch across all frames, we construct an empirical-null distribution from 64 group-matched irrelevant control queries, capturing the scores expected in the absence of target-specific evidence. Target affinities are calibrated against these same-position references into right-tail Monte Carlo p-values, which are discrete super-uniform under a conditional exchangeability assumption. Calibrated screening and a high-recall semantic guard produce candidate tokens for deterministic visual deduplication and joint spatiotemporal budget allocation. To validate NullTok, we apply it to two representative MLLMs and conduct extensive experiments on a set of video benchmarks. The experimental results demonstrate its effectiveness under a bounded visual token budget. For example, it achieves 65.81% and 68.85% on Video-MME using frozen LLaVA-Video-7B and Qwen3-VL-8B backbones, respectively, at a cap of 64 full-frame equivalents (FFE), without subtitles or audio. These results outperform all evaluated baselines on Video-MME under the same budget for both backbones. Code is available at this https URL.
150. 【2611.23872】Keep the Image in Mind: A Visual Board for Multimodal Agents
链接:https://arxiv.org/abs/2611.23872
作者:Haiming Zhu,Zhiwei Chen,Ding Cao,Tianxiang Zhao
类目:Computer Vision and Pattern Recognition (cs.CV)
关键词:Multi-step visual reasoning, early visual misinterpretations, visual misinterpretations persist, visual reasoning suffers, Multi-step visual
备注:
点击查看摘要
Abstract:Multi-step visual reasoning suffers when early visual misinterpretations persist as incorrect premises in subsequent reasoning. We argue that how visual evidence is represented during reasoning matters as much as how much of it is gathered. Rather than appending observations and inferences to an ever-growing dialogue, we introduce a Visual Board-centered harness that separates visual observations, uncertain interpretations, and derived reasoning within a structured, capacity-bounded, and revisable workspace. This separation makes each premise explicit and auditable, so that conflicts between what is seen and what is derived become detectable even by a text-only critic and can trigger targeted reinspection of the original image. Without task-specific training, our harness consistently outperforms standalone models across 4B, 8B, Qwen3-VL-Flash, and 32B configurations on Euclid30K, SolidGeo, and ChartQA bar charts, with gains of up to 11.68, 18.42, and 24.82 percentage points, respectively. Under identical tools, critic, and interaction limits, the structured Board outperforms shared free-form notes by 6.67 to 13.00 points and no-Board retries by 7.35 to 18.80 points while using 7.9\% to 38.2\% fewer tokens. Supervised fine-tuning on harness trajectories further improves direct answering and critic-free tool use, with positive transfer to chart reasoning.
151. 【2611.23871】HOForce: Motion and Force Reconstruction of Hand-Object Interaction Using a Single-View RGB-D Video
链接:https://arxiv.org/abs/2611.23871
作者:Jiahui Wang,Wanqing Du,Yuanbin Zou,Zhefeng Xu,Feng Xu
类目:Computer Vision and Pattern Recognition (cs.CV)
关键词:single camera makes, camera makes large-scale, makes large-scale capture, large-scale capture practical, Reconstructing hand-object interactions
备注: Includes supplementary material. Project page: [this https URL](https://huggingface.co/spaces/fnoi16wjhui/HOForce)
点击查看摘要
Abstract:Reconstructing hand-object interactions is an important task, and achieving it with a single camera makes large-scale capture practical but challenging due to heavy occlusions. We introduce HOForce to track hands and objects, and further recover interaction forces from single-view RGB-D video and known object properties. For two-hand manipulation, we jointly model fingertip forces of both hands and the contact forces between objects when two objects are presented. The reconstructed forces also improve motion tracking by giving pose constraints in the occluded regions. HOForce recovers hand and object motion more accurately on DexYCB and TACO and produces less hand-object penetration than interaction baselines on DexYCB. On our dataset of fingertip forces recorded with tactile sensors, HOForce reduces normal-force MAE by 24.5% relative to the strongest baseline. The recovered forces reproduce object motion in MuJoCo and transfer to other engines after adaptation.
152. 【2611.23865】On Demand, Not Uniformity: Adaptive Expert Orchestration for Multimodal Continual Instruction Tuning
链接:https://arxiv.org/abs/2611.23865
作者:Mengxin Qin,Kun Wei,Weitao An,Jiahua Li,Xu Yang,Cheng Deng
类目:Computer Vision and Pattern Recognition (cs.CV)
关键词:Large Language Models, Multimodal Large Language, Continual Instruction Tuning, Multimodal Continual Instruction, enables Multimodal Large
备注:
点击查看摘要
Abstract:Multimodal Continual Instruction Tuning (MCIT) enables Multimodal Large Language Models (MLLMs) to adapt to sequential instruction data while retaining previously acquired knowledge. While recent Mixture-of-Experts (MoE)-based methods mitigate forgetting by freezing historical experts and introducing new ones, they largely overlook the varying adaptation demands across tasks. Probing incoming tasks reveals pronounced heterogeneity in layer-wise gradient pressure and token-wise prediction difficulty. This observation highlights two unresolved decisions: where additional capacity is needed and how strongly new experts should contribute to individual tokens. To address both questions, we propose the Coverage-Calibrated Expert Injection and Residual Arbitration framework (CIRA-MoE), which derives expert acceptance margins via learnable threshold routing to serve as a unified coverage signal. Specifically, historical acceptance evidence guides the selective injection of new experts into insufficiently covered layers. Meanwhile, relative evidence between historical and current experts modulates token-wise residual contributions, coupled with arbitration-consistent regularization to promote expert specialization. Comprehensive experiments on UCIT and CoIN demonstrate that CIRA-MoE outperforms existing state-of-the-art methods in MCIT.
153. 【2611.23862】Grounding Radiology Reports in 3D Chest CT: Results of the ReXGrounding Challenge
链接:https://arxiv.org/abs/2611.23862
作者:Mohammed Baharoon,Mahmoud Hussain Alabbad,Sung Eun Kim,Xiaoman Zhang,Lucas Bijnens,Zirong Chen,Jiayu Wang,Xi Ouyang,Mingyang Yu,Weiqiu Jin,Xiaodan Ye,Dijia Wu,Kwang-Hyun Uhm,Inhwa Son,Rachel van 't Hull,Tim Veenboer,Cees Snoek,Kevin Groot Lipman,Daesung Kim,Junha Im,Youngjun Jun,Sangwon Lee,Katelyn Morrison,Adam Perer,Mijin Yun,Ali Piracha,Reem Alwasiah,Talal Yahyan,Abdulrahman Alamoudi,Mashael Alshehri,Rayan Alsager,Hannah Hancock,Anna M. Hunkemöller,Peter Youssef,Mona A. Alhammad,Afnan Aljumayan,Brecht Van Berkel,Eric Bijnens,Saud Alhussain,Maarten Steyvers,Rolf Symons,Bryan Eelen,Ayed Alaklubi,Maha Buraykan Alshammari,Amirah Fahad Alshammeri,Tomas Gilvydis,Ibrahim AlTurki,Monerah Almohideb,Ali Alburkani,Shahad Alhomaiani,Fatima Jabali,Maximilian Rokuss,Moritz Langenberg,Yannick Kirchhoff,Benjamin Hamm,Klaus Maier-Hein,Ibrahim Ethem Hamamci,Sezgin Er,Michael Moritz,Hassan R. Alomaish,Luyang Luo,Pranav Rajpurkar
类目:Computer Vision and Pattern Recognition (cs.CV); Artificial Intelligence (cs.AI)
关键词:systems generate reports, Segmenting each reported, measured and tracked, tracked over time, systems generate
备注: 25 pages, 13 figures, 4 tables
点击查看摘要
Abstract:Segmenting each reported finding in a radiology scan would allow findings to be measured and tracked over time and let AI systems generate reports that are verifiable, but so far AI models have performed poorly at this task. To advance this task, we organized the ReXGrounding Challenge at MICCAI 2026, the first challenge dedicated to it. Given a non-contrast chest CT scan and the set of findings from its report, participants trained models to produce the corresponding set of 3D segmentations. The challenge used 2,992 training and 200 validation scans from the ReXGroundingCT dataset, and evaluated methods on a held-out test set of 300 scans with 582 findings, exhaustively annotated by board-certified radiologists. In total, 48 teams from 15 countries made 760 submissions, and the best method reached a Dice of 0.375 and a hit rate of 0.77, matching agreement between two independent radiologist groups (Dice 0.32, hit rate 0.76, on a 50-scan subset). On the 100 scans of the pre-challenge benchmark, the winner reached a Dice of 0.390, against 0.285 for VoxTell, the best model before the challenge, which five of the six best teams fine-tuned. Non-focal findings remained far harder to segment than focal ones, instance F1 stayed below inter-reader agreement (0.34 vs. 0.47), and models sometimes segmented the right abnormality in the wrong location. Top methods thus match radiologist agreement in Dice, but non-focal findings, findings with multiple instances, and adherence to the anatomical location described in the text remain open problems.
154. 【2611.23860】MobileWeave: Program-First Synthesis of Dependency-Linked Tasks for Training Mobile GUI Agents
链接:https://arxiv.org/abs/2611.23860
作者:Ziyi He,Xian Wu,Qiyuan Chen,Haitao Li,Cehao Yang,Yinghao Zhu,Xichen Zhang,Xiao Chen,Yushi Feng,Lequan Yu
类目:Computer Vision and Pattern Recognition (cs.CV)
关键词:mobile GUI agents, GUI agents require, Native mobile GUI, agents require large-scale, mobile GUI
备注:
点击查看摘要
Abstract:Native mobile GUI agents require large-scale training trajectories covering both individual app functions and multi-goal workflows within and across apps, which makes automatic task synthesis a promising way to scale training data. Existing synthesis pipelines, however, produce each task as free-form text, leaving implicit how its subgoals depend on one another. This matters because composition is a skill in its own right: a policy may complete each subtask of a workflow on its own, yet fail to chain these subtasks into the complete workflow. We present MobileWeave, which instead synthesizes every task as an executable program derived from the app implementations of a controllable simulator. MobileWeave abstracts each app into a typed capability graph with semantic types shared across apps, composes atomic programs into single- and cross-app composite programs through explicit value and object dependencies, and verifies each program by native execution, deriving both its instruction and its completion check from the same program. Trained on teacher trajectories collected for these tasks and on recovery supervision at the policy's own stalls, MobileWeave-8B improves its base model by 26.7 points on MobileGym and by 8.6, 21.3, and 17.6 points on AndroidWorld, AndroidLab, and MobileWorld, with comparable gains at 4B. Beyond these gains, we find that composition is learnable from task structure: explicit dependencies between subgoals act as a training signal that teaches a policy to compose skills it already masters. We release data, model, and code at this https URL .
155. 【2611.23859】mporal Image Representations for Mouse Dynamics Authentication
链接:https://arxiv.org/abs/2611.23859
作者:Junyi Li,Charles Devlen,Daqing Hou
类目:Computer Vision and Pattern Recognition (cs.CV)
关键词:traditionally relied, relied on handcrafted, handcrafted features, features or time-series, Mouse dynamics authentication
备注: 11 pages, 5 figures, 5 tables
点击查看摘要
Abstract:Mouse dynamics authentication has traditionally relied on handcrafted features or time-series representations that capture both spatial and temporal characteristics of mouse movements. More recent approaches have modeled mouse data as images, enabling the use of powerful pretrained computer vision models. However, prior work primarily focuses on spatial-only representations where the cursor coordinates are plotted as a trajectory image. These representations discard the temporal characteristics which can help discriminate users. In this work, we propose image representations that incorporate speed and velocity information for mouse dynamics authentication. We investigate two mouse data representations, a XYPlot representation that intuitively shows the path of movement, and a Pairwise Distance (PWD) representation that utilizes the entire image canvas to encode relationships between mouse events. Both representations are evaluated under both location-aware and location-free settings using pretrained 2D-CNN and ViT architectures. We found that the incorporation of temporal information improved performance over spatial representations alone, and that the location-aware variants generally outperformed the location-free. We achieve state-of-the-art performance on two datasets, Balabit and TWOS, and extend our evaluation to two other datasets. We found that in general, location-aware PWD with temporal encoding combined with the 2D-CNN architecture provided the best results, with mean EERs of 5.21% and 12.70% on the Balabit Test-1 and Test-2 sets, respectively, 4.94% on TWOS, 6.16% on ChaoShen, and 0.59% on DFL.
156. 【2611.23855】RoboBuilder: Agentic Robot Co-Design with Real-world Video Feedback in the Loop
链接:https://arxiv.org/abs/2611.23855
作者:Ghadi Nehme,Dahyun Nam,Akshay Govind Srinivasan,Faez Ahmed
类目:Artificial Intelligence (cs.AI); Computer Vision and Pattern Recognition (cs.CV); Robotics (cs.RO)
关键词:real, real world, agents, agents design robots, coding agents
备注:
点击查看摘要
Abstract:Can AI agents design robots that work in the real world? We introduce RoboBuilder, a benchmark in which coding agents co-design a robot's hardware and program from standard parts, motors and a custom 3D-printed part, and every design that passes in simulation is built and tested on a physical course. Simulation alone is not enough: under public part data, none of 15 initial passing designs crossed the real course, so an agent learns the simulator from measurements it requests and videos of real runs. Across four frontier coding agents, ten obstacles at three difficulty levels, 16,427 simulations of 12,774 distinct robots and 135 physical robots built in an estimated 240 hours, agents solve nearly all easy obstacles and invent mechanisms our search baselines do not reach, but falter at harder levels, when one robot must cross every obstacle, and in the real world, where 66% of simulated passes hold across all levels. Closing the loop helps: shown a video of its real failure, an agent redesigns the robot, sometimes correcting its simulator, and removes 72% of level-1 and level-2 failures. RoboBuilder measures progress toward agents that engineer real machines. Project Page: this https URL
157. 【2611.23845】ReflectScene:MLLM-Guided Closed-Loop Diffusion for 3D Indoor Scene Generation
链接:https://arxiv.org/abs/2611.23845
作者:Jiawei Shen,Jie Feng,Junpeng Zhang,Guanbin Li,Wanyi Chen,Weisheng Dong,Guangming Shi
类目:Computer Vision and Pattern Recognition (cs.CV); Artificial Intelligence (cs.AI)
关键词:layouts satisfying geometric, requires layouts satisfying, satisfying geometric, indoor scene generation, Text-conditioned
备注: 10 pages, 3 figures
点击查看摘要
Abstract:Text-conditioned 3D indoor scene generation requires layouts satisfying geometric, functional, and semantic constraints. However, existing diffusion methods rely primarily on static textual conditions, limiting their ability to assess evolving scenes and translate semantic discrepancies into executable object-level corrections. MLLM-based post-hoc refinement provides semantic feedback but remains decoupled from denoising, potentially disrupting scene this http URL propose ReflectScene, an MLLM-guided closed-loop diffusion framework. First, we introduce Hierarchical Prior-Grounded Scene Understanding that aligns structured layouts with multi-view visual evidence, to interpret object-specific context and scene global semantics via a frozen multi-modal large language model (MLLM). Second, we propose MLLM-Guided Semantic Reward Evaluation to distill MLLM's diagnoses and assessments on possible candidate corrections into a lightweight Reward Predictor, enabling correction decision making with reduced computation burden. Finally, Decision-Conditioned Trajectory-Consistent Diffusion is proposed to integrate selected corrections through transient conditioning and persistent clean-estimate projection, correcting object layout in the generated scene while preserving non-target arrangements throughout this http URL on 3D-FRONT demonstrate improved functional consistency and semantic alignment while maintaining competitive generation quality and diversity.
158. 【2611.23840】OGBench: Benchmarking and Improving Tiny-Object Generation Fidelity in Aerial Imagery
链接:https://arxiv.org/abs/2611.23840
作者:Chanhwi Jeong,Dongmin You,Seungwook Kim
类目:Computer Vision and Pattern Recognition (cs.CV); Artificial Intelligence (cs.AI)
关键词:Advances in deep, enable the synthesis, synthesis of realistic, Advances, deep generative models
备注: Accepted to WACV 2027 Round 2. 20 pages, 7 figures
点击查看摘要
Abstract:Advances in deep generative models now enable the synthesis of realistic imagery. However, aerial imagery is dominated by tiny objects, such as vehicles, ships, and aircraft, which occupy only a few pixels owing to the overhead, wide-area nature of aerial imaging. Existing generative models struggle substantially more with tiny objects than with larger ones. This limitation has been overlooked, as prior work on generative evaluation emphasizes prompt following or human preference rather than whether small objects are rendered faithfully. To address this gap, we introduce TOGBench, a quantitative, object-level benchmark for evaluating tiny-object generation fidelity in aerial imagery. We further propose two simple train-/inference-time techniques that improve the ability of layout-to-image generators to render small objects, raising their performance on TOGBench.
159. 【2611.23822】Mapping the Design Space of Feature Sampling for Optical Flow
链接:https://arxiv.org/abs/2611.23822
作者:Liping Zhang
类目:Computer Vision and Pattern Recognition (cs.CV)
关键词:correlation feature sampling, sampling, feature sampling, correlation features, correlation feature
备注:
点击查看摘要
Abstract:The computation of correlation features is crucial for optical flow estimation. Existing methods (e.g., RAFT) perform fixed-radius lookups on a cost volume, yielding a set of correlation features for each position; alternatively, WAFT performs single-point feature indexing, yielding one correlation feature vector per position. None of them designs a suitable correlation feature sampling scheme for each position according to regional characteristics. Our insight is that not every position requires the same sampling scheme. Applying an identical sampling scheme to every position is either redundant or insufficient and ignores the specific characteristics of the corresponding region. This paper rethinks the design of feature sampling and investigates the following question: should the support of feature sampling---i.e., the extent and directions over which evidence is aggregated---be learned separately for each query position? To study this question, we propose SAGE (Sampling with Anisotropic Gaussian Expectation). This method replaces single-point sampling with a per-pixel anisotropic Gaussian expectation on a fixed candidate grid centered on the current optical flow estimate. A positive-definite precision matrix controls the scale, anisotropy, and orientation of the sampling support, with single-point sampling included as its limiting case. Consequently, the sampling support becomes an explicit and analyzable design variable and provides a principled way to analyze how directional feature aggregation affects correspondence learning. Extensive experiments show that SAGE, while adding only 1.5% multiply-accumulate operations (MACs), effectively improves zero-shot cross-dataset generalization. When trained only on FlyingChairs and FlyingThings, SAGE reduces the end-point error (EPE) by 24.2% and 16.4% on Sintel clean and final, and reduces KITTI F1-epe and F1-all by 7.2% and 11.6%.
160. 【2611.23813】SWIFT: Slow World, Instant Freshness, and Timely Action for Asynchronous Autonomous Driving
链接:https://arxiv.org/abs/2611.23813
作者:Jiazhe Wang,Tianyi Yang,Ruihang Li,Fan Lu,Xianming Zeng,Jianyun Xu,Xingliang Liu,Lei Yang
类目:Computer Vision and Pattern Recognition (cs.CV)
关键词:action-delivery time central, making both observation, scenes keep changing, changing during model, action-delivery time
备注:
点击查看摘要
Abstract:Driving scenes keep changing during model computation, making both observation freshness and action-delivery time central to world-action planning. To retain rich generative world priors in autonomous driving without stalling real-time control, we present SWIFT (Slow World, Instant Freshness, and Timely action), a deployment-oriented asynchronous world-action system that moves world updates off the observation-to-action critical path. On a background GPU, a slow Video DiT streams surround-view history into read-only predictive world context, trained with a predictive span that covers both the update interval and background world computation. On a foreground GPU, a lightweight freshness carrier encodes current surround-view images into a gated key/value bank at every step, letting a fast Action DiT combine live visual feedback with the latest completed world context without waiting. On open-loop NAVSIM, SWIFT reaches 94.2 PDMS at 198.4 ms per decision with a trajectory scorer; on closed-loop Bench2Drive, it achieves an 80.30 Driving Score at 109.9 ms foreground latency. SWIFT further transfers to the Waymo E2E hidden test (8.0050 Rater Feedback Score) and nuScenes (0.54 m mean endpoint error, 0.03% mean collision rate). Across all four benchmarks, SWIFT achieves the best results among compared world-action methods, establishing a favorable accuracy-latency trade-off.
161. 【2611.23808】PReMind: When Personal Memory Meets Visual Hazards
链接:https://arxiv.org/abs/2611.23808
作者:Yeongtak Oh,Donghyeok Lee,Joonhee Oh,Jaewon Jeong,Sungroh Yoon
类目:Computer Vision and Pattern Recognition (cs.CV)
关键词:provide advice tailored, Personalized vision-language assistants, Personalized vision-language, memories to provide, provide advice
备注: Project page: [this https URL](https://github.com/oyt9306/PReMind)
点击查看摘要
Abstract:Personalized vision-language assistants use visual observations and personal memories to provide advice tailored to the user. When the observed scene contains a physical hazard, personal context, such as allergies, medical conditions, or mobility constraints, can change what help is appropriate. Responding appropriately therefore requires recognizing the visible hazard and applying the relevant person's information. We introduce PReMind-Bench, a diagnostic benchmark that evaluates this combination across diverse hazards and personal-memory conditions. It separately measures memory access, grounding, abstention, and hazard response, and assesses joint success on interdependency cases. Existing models often satisfy one requirement while failing the other, and training only for safety or personalization leaves a gap in joint success. We therefore propose PReMind, a two-phase reinforcement learning framework that combines on-demand and forced-call interactions (memory always retrieved) to jointly train grounded memory use and hazard response. At 4B, PReMind raises joint success from 12.9% to 28.6% over its base on the interdependency set and improves hazard adequacy and correct abstention on the main test. Joint-success gains persist under forced-call evaluation and under a second LLM judge across three Phase II seeds; averaged over these seeds, joint success also improves on unseen hazard groups. These findings highlight the need to evaluate and train personal assistants to connect visual evidence with the relevant person's context.
162. 【2611.23807】Sightline: Long-Window Speculative Decoding for Fast SVG Generation
链接:https://arxiv.org/abs/2611.23807
作者:Yanchao Li,Ben Gao,Jiaqing Xie,Yusu Zhang,Beilun Wang,Yanbo Wang,Xiaohua Xu,Nanqing Dong,Tianfan Fu,Zhongyu Huang
类目:Computer Vision and Pattern Recognition (cs.CV); Computation and Language (cs.CL); Machine Learning (cs.LG)
关键词:Scalable Vector Graphics, Scalable Vector, drawing is slow, SVG, forward pass
备注: 45 pages, 23 figures, 34 tables. Code: [this https URL](https://github.com/LiZaiyuan0619/Sightline)
点击查看摘要
Abstract:Scalable Vector Graphics (SVG) generation models write a graphic as code with one forward pass per token, so drawing is slow. Speculative decoding speeds them up losslessly, and block drafters guess many tokens per pass. On SVG, many of their rounds accept the whole draft and end only at the verification window. We find that their misses are rare and clumped, and that they stay accurate far deeper into a block than on natural language. A run-length identity and three propositions show that a longer window pays once the candidates are wide, up to the drafter's horizon. We propose Sightline, which pairs a window as long as the drafter's horizon with a draft tree wide enough to fill it. Experiments cover text-to-SVG and image-to-SVG with three models on six benchmarks from MMSVG, MMSVG-2M and SArena. In every setting, Sightline is the fastest of the compared lossless decoders. On SArena, it stays the fastest when every decoder shares one loop control. It also leads in almost every setting of six models from four other families, which write SVG code, compressed SVG tokens or Lottie JSON. Analyses trace the lead to rounds that shorter windows cut off, both in loop-free content and in loops. Speculative decoding also saves time for choosing samples by rendering, which beats choosing by likelihood. These results suggest that speculative decoding on vector graphics should verify as far as the drafter sees. Code and data are available at this https URL.
163. 【2611.23793】ROBIN++: Unified Copyright Protection and Tamper Localization for Diffusion Models via Dual-Domain Synergistic Watermarking
链接:https://arxiv.org/abs/2611.23793
作者:Huayang Huang,Siqi Zeng,Qian Wang,Bo Du,Yu Wu
类目:Computer Vision and Pattern Recognition (cs.CV)
关键词:Digital watermarking, generated content, promising solution, solution for verifying, verifying the provenance
备注: 2026 TPAMI
点击查看摘要
Abstract:Digital watermarking provides a promising solution for verifying the provenance and integrity of generated content. However, existing methods often suffer from an inherent trade-off between copyright robustness and localization sensitivity due to the entanglement of conflicting signals within a single domain. In this paper, we propose ROBIN++, a paradigm that shifts from single-domain entanglement to a Dual-Domain Synergistic design. Our key insight is to explicitly decouple robustness and fragility during injection by assigning them to distinct domains, while re-coupling them for mutual reinforcement during verification. Specifically, we introduce an Adversarial Optimization Algorithm to inject a frequency-domain copyright watermark during generation, maximizing extraction robustness without visual artifacts. Complementarily, a Content-Aware Perturbation generator is integrated into the latent space to inject the spatial-domain localization watermark, ensuring sensitivity to spatial tampering with minimal interference to the copyright watermark. During verification, we propose a dual-branch synergistic detector that facilitates Frequency-to-Spatial Synergy, where robust spectral priors act as global anchors for local anomaly detection. Conversely, a Spatial-to-Frequency Refinement strategy is employed to rectify corrupted latent features using predicted tamper masks. Extensive experiments show that ROBIN++ significantly outperforms existing approaches in both copyright recovery and tamper localization accuracy, effectively mitigating the long-standing trade-off between the two tasks. Code is available at this https URL.
164. 【2611.23781】When accuracy hides risk: risk-controlled confirmation of the critical view of safety under expert disagreement
链接:https://arxiv.org/abs/2611.23781
作者:Bowen Xiang,Xiao Yu,Mingze Xu
类目:Computer Vision and Pattern Recognition (cs.CV)
关键词:Automated assessment, view of safety, advanced rapidly, check must decide, critical view
备注:
点击查看摘要
Abstract:Automated assessment of the critical view of safety (CVS) in laparoscopic cholecystectomy has advanced rapidly, but a deployed check must decide, before the first clip, whether to confirm CVS. The standard rule calibrates a threshold so that false confirmations are rare on average. We show that this rule concentrates its errors on disputed videos, which one of three expert raters judged sufficient, and that the failure is structural. When reference labels are votes of a panel sampled from a larger expert pool, the rule that maximizes sensitivity at a fixed average false-confirmation rate also maximizes false confirmation of disputed videos, and for recognizers that recover expert judgement with less noise this rate rises with accuracy. We propose certified confirmation, a risk-controlled monitoring rule whose threshold is selected by multiple testing so that, with probability at least 1 - {\delta}, false confirmation is at most {\alpha} overall, among disputed videos, within vendor and country groups, and before CVS is visible, using disputed labels only to calibrate. In a pre-specified, hash-locked evaluation on the 300 test videos (18 countries) of the SAGES CVS Challenge, standard calibration falsely confirmed 8 of 39 disputed videos (0.21; one-sided p = 0.037 against {\alpha} = 0.10), whereas certified confirmation confirmed 2 (one-sided 95% upper bound 0.15). On development data, protection held for vendor and country groups left out of calibration. The cost was confirming 10% rather than 35% of sufficient videos on time. Average accuracy hides this risk; disputed cases must be protected explicitly.
165. 【2611.23770】DetectViT: Test-time Backdoor Detection for Vision Transformers via Inter-Head Attention Discrepancy
链接:https://arxiv.org/abs/2611.23770
作者:Siquan Huang,Yijiang Li,Xingfu Yan,Ningzhi Gao,Leyu Shi,Ying Gao
类目:Computer Vision and Pattern Recognition (cs.CV); Cryptography and Security (cs.CR)
关键词:Vision Transformers, third-party pretrained checkpoints, pretrained checkpoints leaves, underpin the visual, visual encoders
备注: Accepted to NeurIPS 2026
点击查看摘要
Abstract:Vision Transformers (ViTs) now underpin the visual encoders of most multimodal models, but this success comes at a cost: relying on third-party pretrained checkpoints leaves these systems open to backdoor attacks. Existing test-time detection methods rely on either the predicted label or intermediate embeddings, limiting their applicability across diverse tasks and incurring substantial computational overhead. In this work, we study how backdoor triggers affect the multi-head attention mechanism of ViTs and identify two distinct anomalous patterns: $\textit{complete attention hijacking}$, causing the attention to exhibit abnormally high inter-head spatial overlap, and $\textit{partial attention hijacking}$, causing an abnormally large variance in the concentration of the attention distribution. Motivated by these findings, we propose $\textbf{DetectViT}$, a test-time backdoor detection method that requires neither training data, model outputs, nor any prior knowledge of the trigger. DetectViT quantifies the two hijacking phenomena via an inter-head consistency score and an inter-head entropy variance score, with thresholds estimated from a small out-of-distribution (OOD) calibration set drawn from an arbitrary source. Across four representative attacks, diverse architectures (DeiT, CLIP, and LLaVA), and downstream tasks (image classification and captioning), DetectViT substantially outperforms all baselines with as few as 64--128 OOD calibration samples, achieving a true positive rate of $100\%$ and a false positive rate of $0.17\%$ on BadCLIP. Notably, it introduces negligible additional overhead by relying solely on attention weights already computed during inference. Code is available at: this https URL.
166. 【2611.23769】Keep the Reasoning, Read the Image: On-Demand Visual KV for Multi-Turn Reasoning VLMs
链接:https://arxiv.org/abs/2611.23769
作者:Ziyi Wang,Xiyue Zhang,Bing Hu,Guangmin Song,Chenyi Zhang,Li Wang
类目:Computer Vision and Pattern Recognition (cs.CV)
关键词:Reasoning vision-language models, Reasoning, vision-language models, thousands of tokens, reasoning grows
备注:
点击查看摘要
Abstract:Reasoning vision-language models (VLMs) think for thousands of tokens before they answer, and retention policies keep the KV cache affordable by pruning it as the reasoning grows. We find that this pruning sacrifices the image in multi-turn conversations: the image and the reasoning share one budget, so the growing reasoning crowds the visual entries out. A chart question that loses 8 points to the full cache when asked first loses 24 points when asked after a round of reasoning. Protecting a small visual reserve from eviction recovers most of this loss, while adding the same slots to the shared budget recovers considerably less. The image therefore lacks protection, not capacity. Based on this finding, we propose ShelfKV, a training-free method that works on top of existing retention policies. It keeps every visual entry in CPU memory and, at each decoding step, swaps the visual entries that the current query scores highest into the attention slots of the text entries it scores lowest. The image is never evicted, yet it occupies attention slots only while it is read. We evaluate ShelfKV on four benchmarks, three of them conversational, with two retention policies and two reasoning VLMs. In conversations, it improves the strongest existing policy, DBTrimKV, by up to 20.3 points at matched GPU memory, and still leads by 8.7 points when DBTrimKV uses twice the memory. It also consistently outperforms a fixed visual reserve of matched GPU memory in these settings, without hurting accuracy on standalone questions. These results show that the image should be protected from the reasoning but read only on demand.
167. 【2611.23765】Mean Curvature Minimizing Foliations Of A Vector Field
链接:https://arxiv.org/abs/2611.23765
作者:Tabish A. Syed,Marc Theveneau,Benedikt Wirth,Rustum Choksi,Kaleem Siddiqi
类目:Computer Vision and Pattern Recognition (cs.CV)
关键词:Streamline patterns, ubiquitous in nature, patterns are ubiquitous, Streamline, vector field
备注:
点击查看摘要
Abstract:Streamline patterns are ubiquitous in nature. Prominent examples include white matter fibre tracts in the mammalian brain and arrangements of elongated cells, such as cardiomyocytes in the heart wall. It has been posited that such organisations foliate into locally smooth surfaces to optimise function. However, the recovery of these foliations from a single vector field aligned with the local orientation of the underlying sheets remains an open problem. We address this challenge by introducing a mathematical model of a sheet and developing an algorithm for sheet inference, from measurements of the oriented structures. Given an input vector field, our model estimates a second vector field that is orthonormal to it, while also requiring the Lie bracket of the two to vanish. To balance sheet likelihood against low surface area, we also reduce the mean curvature of the recovered local surface. We compute the first variation of an associated energy and provide a numerical method which admits an efficient, parallel implementation for sheet estimation. We demonstrate our approach on both synthetic and biological examples, motivated by applications to medical imaging. These include white matter fibre tracts from a brain atlas and confocal microscopy images of cleared heart wall tissue. For the latter we are able to reconstruct sheets aligned with local cardiomyocyte directions at unprecedented (micrometre) scale.
168. 【2611.23755】GramStatTexNet: Efficient, Interpretable, and Neuro-Inspired Texture Analysis-by-Synthesis
链接:https://arxiv.org/abs/2611.23755
作者:Vasha DuTell,Zuyu Yun,Anne Harrington,Michael Y-S Fang,Mark Hamilton,Christian Kövesdi,Edward H. Adelson,William T. Freeman,Ruth Rosenholtz
类目:Computer Vision and Pattern Recognition (cs.CV); Graphics (cs.GR); Neurons and Cognition (q-bio.NC)
关键词:networks achieve high, achieve high realism, limited direct connection, Deep networks achieve, biological vision
备注:
点击查看摘要
Abstract:Deep networks achieve high realism in texture synthesis but often rely on opaque, over-parameterized representations that have limited direct connection to biological vision. Neuroscience-informed pyramid models are interpretable, compact, and biologically inspired, but rely on small, hand-curated statistics sets that are difficult to generalize or extend. We introduce GramStatTexNet, a hybrid analysis-by-synthesis framework that combines the multi-scale Gabor filter structure of classical models with the flexibility of Gramian-based correlations, organized into structured filter families. This structured representation enables analyses that are challenging for deep-feature methods: each statistic carries a named filter-family identity, allowing us to decompose synthesis fidelity into per-family contributions and to learn an interpretable, low-dimensional embedding that preserves perceptual texture content. We compare our framework to classical and deep texture-synthesis baselines across diverse texture categories, and on multiple perceptual metrics, highlighting the compactness and interpretability advantages of this hybrid representation, alongside competitive synthesis quality. This same framework extends naturally to peripheral textures via spatial pooling, and to dynamic textures using a spatiotemporal filter bank. GramStatTexNet provides a unified, interpretable framework for analyzing and modeling visual information with natural extensions across space, time, and eccentricity. Project page: this https URL
169. 【2611.23754】Detect What You Need: Chain-of-Causal Reasoning for 3D Intent Grounding
链接:https://arxiv.org/abs/2611.23754
作者:Zihao Zhang,Aming Wu,Yang Li,Yahong Han
类目:Computer Vision and Pattern Recognition (cs.CV)
关键词:Accurately matching human, Accurately matching, artificial intelligence, important goal, goal of artificial
备注: Accepted at NeurIPS 2026
点击查看摘要
Abstract:Accurately matching human intentions in 3D space is an important goal of artificial intelligence. Recently, 3D Intention Grounding (3D-IG) has emerged, aiming to localize target 3D objects given a natural-language intent. Unlike conventional visual grounding with descriptive referring expressions, 3D-IG intents are abstract and non-descriptive, making object localization substantially more challenging. This requires inferring latent functional requirements from non-descriptive intents and aligning them with object-level 3D representations. However, existing methods largely rely on implicit intent-object matching, leading to logical gaps and limited interpretability, robustness, and generalization. To address these challenges, we propose Chain-of-Causal Reasoning (CoCR), a causality-inspired functional dependency reasoning framework that explicitly bridges abstract intentions and candidate objects through intermediate functional requirements. Specifically, CoCR progressively decomposes complex intents into ordered functional requirements, thereby forming an explicit intent-function-object reasoning chain that links abstract intentions to object suitability. Building on this chain, we construct a causality-inspired functional dependency graph to model requirement-attribute relationships and introduce a causal-visual alignment module that aligns function-aware representations with the geometric-semantic features of 3D point clouds, enabling bidirectional verification between structured functional reasoning and visual evidence. Extensive experiments on 3D Intention Grounding and 3D Visual Grounding demonstrate that our method enhances intent-aware object localization. Our code is available at this https URL.
170. 【2611.23752】ACUS: An Angular Calibrated Uncertainty Score for Incomplete-View CT
链接:https://arxiv.org/abs/2611.23752
作者:Zihan Deng,Chuanzhi Xu,Quankeng Huang,Yujie Liu,Zhenzhen Gong,Junjie Huang,Tianye Niu,Lequan Yu
类目:Computer Vision and Pattern Recognition (cs.CV)
关键词:limited-arc computed tomography, Sparse-view and limited-arc, computed tomography, ground-truth image, limited-arc computed
备注:
点击查看摘要
Abstract:Sparse-view and limited-arc computed tomography (CT) are commonly treated as projection completion: a network fills an unmeasured view, and the result is ranked by similarity to one ground-truth image. That score can be high even when the query angle is far from every acquired measurement, because a learned prior may invent anatomy the scan never constrained. Existing protocols therefore leave a basic question unanswered: how does predictive reliability change with distance from the acquired views? We propose the Angular Calibrated Uncertainty Score (ACUS) to answer that question. ACUS is the Winkler interval score of a nominal 90% predictive interval on held-out projections, binned by distance to the nearest acquired view. It charges interval width and adds a large miss penalty when a held-out pixel falls outside the interval, so a narrow band helps only when coverage holds. We use collapse on acquired angles as a necessary consistency check on the predictor. On chest and dental cone-beam orbits, interval width grows with the angular gap. Empirical coverage ranges from 77% to 92%, and the peak signal-to-noise ratio (PSNR) of the point estimate falls as the width rises. Nearest-neighbor, interpolation, and point completion, each given one residual scale, have zero reported far coverage and incur outside-distance penalties. ACUS exposes gap-dependent overconfidence that a point-estimate PSNR alone does not measure.
171. 【2611.23736】GaussLearner: 3D Gaussian Primitives as Self-Regulating Continual Learners
链接:https://arxiv.org/abs/2611.23736
作者:Gaoxiang Linghu,Haonan Zhou,Xuanhe Liu,Weili Xu,Kaiyue Zhou,Jun Xiao,Gaoang Wang
类目:Computer Vision and Pattern Recognition (cs.CV)
关键词:Gaussian fields aims, Gaussian, maintain faithful reconstructions, evolving world, aims to maintain
备注: Project Page: [this https URL](https://gaoxiang-linghu.github.io/GaussLearner/)
点击查看摘要
Abstract:Continual learning of 3D Gaussian fields aims to maintain faithful reconstructions of the evolving world. However, directly fine-tuning an existing Gaussian field on new observations can cause catastrophic forgetting of previously reconstructed content. To address this challenge, existing methods employ selective local optimization, replay, or parameter regularization, yet still struggle to balance adaptation to scene changes with preservation of still-valid content. Our key insight is to view 3D Gaussian primitives themselves as self-regulating continual learners. Based on this insight, we introduce GaussLearner, which equips each Gaussian with a closed continual-learning loop connecting observation, self-regulation, and adaptation at the primitive level. At each novel state, GaussLearner first attributes pixel-wise inconsistencies in current observations to their contributing Gaussians. Each Gaussian then self-regulates through a learning weight that balances current observations against cross-state reliability. This weight governs both Gaussian parameter adaptation and optimizer memory refresh. As current inconsistencies are resolved, the adaptation outcome renews each Gaussian's reliability for future states. The resulting field is a globally coupled population of locally self-regulating learners. Across scenes from CL-Splats, WAT, and our CL-3D dataset, GaussLearner improves average PSNR by 3.41 dB over the strongest baseline. On CL-3D, it achieves the best reconstruction quality on both changed and unchanged views.
172. 【2611.23728】AdaS-ViT: Test-Time Adaptive Routing for Spiking Vision Transformers with Ultra-High Energy Efficiency
链接:https://arxiv.org/abs/2611.23728
作者:Zecheng Hao,Zihan Huang,Shenghao Xie,Jiyuan Zhang,Zhaofei Yu,Tiejun Huang
类目:Neural and Evolutionary Computing (cs.NE); Artificial Intelligence (cs.AI); Computer Vision and Pattern Recognition (cs.CV)
关键词:Spiking Neural Networks, spiking vision transformers, existing spiking vision, Neural Networks, largely confine spiking
备注:
点击查看摘要
Abstract:Spiking Neural Networks (SNNs) promise event-driven and energy-efficient vision, yet existing spiking vision transformers (S-ViTs) largely confine spiking tokens to fixed and uniform firing schemes along the temporal, spatial and channel dimensions, whereas suppressing the computational redundancy thus incurred calls for adaptive firing mechanisms that operate jointly across all three axes at test time. To this end, we propose the Adaptive Efficient Integrate-and-Fire (AdaE-IF) model, which consults the attention score of each spiking token and routes salient regions to fine-grained firing patterns while assigning energy-frugal coarse patterns to the remaining regions. Building on AdaE-IF, we further introduce Spiking Gated Self-Attention (S-GSA) and Spiking Local-Gated Unit (S-LGU), which enforce selective spike emission at token and channel granularity. We integrate the aforementioned mechanisms and modules into AdaS-ViT, a test-time adaptive-routing architecture trained with a progressive spatio-temporal curriculum that stabilizes the co-adaptation of firing precision and routing masks. Extensive experiments across benchmarks spanning diverse data scales and modalities demonstrate that AdaS-ViT consistently achieves highly competitive performance. Notably, on ImageNet-1k it consumes only 1.02 to 2.45 millijoules per image and attains an accuracy-per-energy ratio of 32$\times$ to 72$\times$, operating in an economical inference regime and thereby pushing the energy-efficiency boundary further.
173. 【2611.23727】GTA: Generative Tower of Agents for Scalable Holonic Scene Generation
链接:https://arxiv.org/abs/2611.23727
作者:Zeng Tao,Yunuo Chen,Xiang Feng,Yumeng He,Peihao Li,Nanxiang Jiang,Yaowei Guo,Fanlian Zeng,Yin Yang,Chenfanfu Jiang
类目:Computer Vision and Pattern Recognition (cs.CV)
关键词:modeling requires coordinating, coordinating scene-wide goals, requires coordinating scene-wide, context budgets, requires coordinating
备注: [this https URL](https://zt-zeng-tao.github.io/GTA/)
点击查看摘要
Abstract:Scalable world modeling requires coordinating scene-wide goals with fine-grained objects and dynamic characters under limited time and context budgets. We introduce GTA (Generative Tower of Agents), which represents every unit at every level of an adaptive scene hierarchy as a holon, both a self-contained construction task and a composable scene unit. Parallel agents recursively plan the hierarchy top-down, generate assets, and assemble and refine them bottom-up. Uniform interfaces carry scene-wide requirements down the hierarchy, guiding local construction so that each holon fits into the larger one. GTA accelerates scalable world modeling through high-throughput parallel execution while preserving well-structured layouts and fine-grained asset details. GTA then populates the assembled environments with dynamic human characters whose appearance and motion are guided by the scene, yielding realized 4D worlds. Project page: this https URL
174. 【2611.23720】Do 3D Point Representations Encode Architectural Functions? Probing Supervised and Pretrained Models on Urban Facades
链接:https://arxiv.org/abs/2611.23720
作者:Abdiaziz Muse,Jiju Poovvancheri
类目:Computer Vision and Pattern Recognition (cs.CV); Machine Learning (cs.LG)
关键词:Semantic labels, operable or fixed, identical-looking windows, column may carry, carry load
备注: Accepted at the NeurIPS 2026 Workshop: Physical World AI: Geometry, Characteristics, and Multimodal Sensing
点击查看摘要
Abstract:Semantic labels say what a building element is, not what it does: two identical-looking windows on one facade may be operable or fixed, a column may carry load or only decorate. We ask whether 3D point representations encode such architectural function. On 22 mobile-LiDAR buildings with 15 functional attributes per element, we hold out entire buildings and compare hand-crafted descriptors, networks trained from scratch and frozen self-supervised point transformers. Descriptors and frozen features are read by the same linear layer. No network trained from scratch, from PointNet to a 108M-parameter PTv3, clearly exceeds the descriptors (0.610 mAP on functionally ambiguous elements), whereas frozen Utonia reaches 0.732. Random-weight controls of the same architectures (0.649 and 0.652) attribute the gain to pretraining (Utonia: +0.080, 95% building interval [+0.035, +0.124]). Pretrained features improve at every tested number of labelled buildings while descriptors and random features plateau, and, in the configurations we tested, fine-tuning erodes the benefit: at today's data scale, function is better read from pretrained representations than learned from scratch.
175. 【2611.23717】CoScan: Multi-Scale Content-Adaptive Space-Filling Scans for Causal State-Space Image Restoration
链接:https://arxiv.org/abs/2611.23717
作者:Wei Dong,Terry Ji,Yan Min,Shahram Shirani,Jun Chen,Han Zhou
类目:Computer Vision and Pattern Recognition (cs.CV)
关键词:Selective state-space models, recently shown strong, shown strong potential, Selective state-space, textbf
备注: Accepted by NeurIPS 2026
点击查看摘要
Abstract:Selective state-space models (SSMs) have recently shown strong potential for efficient image restoration. A central difficulty, however, is that a 2D feature map typically needs to be linearized into a 1D token sequence for causal state-space updates, making restoration performance highly sensitive to the imposed scan order. Existing deterministic scans follow space-filling principles but remain fixed and content-agnostic, and can yield fragmented or semantically inconsistent contexts under degradations. We propose \textbf{CoScan}, a \textbf{co}ntent-adaptive \textbf{scan}-order design for causal vision SSMs. CoScan learns image-dependent space-filling traversals via differentiable proxy supervision driven by \textbf{scan-quality objectives} that \textit{encourage short-range coherence and causal predictability, while preserving spatial adjacency}. To support hierarchical restoration backbones, we introduce an \textbf{efficient multi-scale scan construction by coarsening a fine-level minimum spanning tree}, producing coherent scans across feature resolutions. In addition, a \textbf{frequency-guided start selection strategy} is designed to \textit{place informative structures early in the sequence prefix to strengthen causal context formation}. Plugged into existing Mamba-based backbones, CoScan yields consistent improvements across several restoration benchmarks with minimal overhead and shows promising results on high-level vision tasks. Code will be released at this https URL
176. 【2611.23705】A Hardware-Aware Comprehensive Study of Edge Vision Deployment with A Unified Hardware-Model-Runtime Mapping Framework
链接:https://arxiv.org/abs/2611.23705
作者:Le Duc An,Le Duc Thanh,Vo Hoai Viet
类目:Computer Vision and Pattern Recognition (cs.CV)
关键词:computer vision directly, resource-constrained edge devices, rapid growth, growth of deep, deep learning
备注:
点击查看摘要
Abstract:The rapid growth of deep learning has driven the deployment of computer vision directly on resource-constrained edge devices. However, the extreme heterogeneity of edge hardware architectures poses major challenges in selecting and designing suitable vision models. This survey bridges the gap between edge hardware constraints, model architectures, and deployment software toolchains. We establish a comprehensive taxonomy classifying edge hardware into three distinct tiers: mid-range devices (GPUs and dedicated NPUs or TPUs), dedicated embedded MPU or SoC processors, and microcontrollers (MCUs). We aggregate and analyze multi-source benchmark results for image classification and object detection across the computing spectrum, spanning from tens of watts down to milliwatt-scale bare-metal MCUs. Based on these empirical findings, we propose a two-tiered hardware-to-model mapping framework that provides concrete model selection matrices across performance priorities (Speed, Balanced, Accuracy) and actionable architectural design principles (peak SRAM budgeting, resolution-first scaling, and algorithm-compiler co-design). Finally, we highlight critical deployment challenges including toolchain fragmentation, operator incompatibility, and Vision Transformer feasibility, and outline promising future research directions in on-device adaptation and sustainable Edge AI.
177. 【2611.23697】What Do Learned 3D Models Add to Localization from Posed References?
链接:https://arxiv.org/abs/2611.23697
作者:Jaehyun Kim,Wonseok Kang,Jeongmin Lee,Tae-Wan Kim
类目:Computer Vision and Pattern Recognition (cs.CV)
关键词:images avoids building, Localizing a query, posed reference images, reference images avoids, point map
备注: 10 pages
点击查看摘要
Abstract:Localizing a query from a few posed reference images avoids building a 3D point map. We ask what the camera predictions of learned 3D models add to geometric estimation from the same references, and where failures arise. Across eight learned models and variants and five test sets, we compare reference budgets, swap pose components, and intervene on the references a model sees. At fine pose-error thresholds, geometric estimation with one additional reference is significantly more accurate than every model on two references on Cambridge and NAVER B1 and never significantly worse, alone or as a fallback behind a shared acceptance gate; with the same three references, the more accurate approach depends on the scene. The gain from the learned translation magnitude depends on reference selection: it improves seven of the eight models on 12-Scenes with the spread-out pairs we select, but none of the five we test with the two top-ranked retrieved references. Where retrieval also returns references from other locations, at NAVER Gangnam Station, models fail at different stages. A robust alignment brings Pi3X and MapAnything within 3.3 pp of adding references as needed. In controlled tests, aligning on the nearby references largely restores VGGT, Pi3X and MapAnything but not Depth Anything 3, and pose-conditioned Pi3X recovers only if the far poses are masked and left out of its input-pose normalization.
178. 【2611.23695】MoMoVL: Modality-Aware Adaptive Compression for On-Device Sparsely-Activated Vision-Language Models
链接:https://arxiv.org/abs/2611.23695
作者:Dongkeun Lee,Wonjun Lee
类目:Computer Vision and Pattern Recognition (cs.CV); Machine Learning (cs.LG)
关键词:cross-modal web tasks, diverse cross-modal web, perform diverse cross-modal, reduced latency, agentic web
备注: 11 pages, 6 figures, Accepted at WSDM 2027 Findings
点击查看摘要
Abstract:The rise of the agentic web has been accompanied by growing demand for on-device Vision-Language Models (VLMs) that perform diverse cross-modal web tasks with enhanced privacy and reduced latency. Mixture-of-Experts (MoE) models are widely adopted in modern VLMs for compute-efficient scalability, yet their multi-expert architecture poses deployment challenges due to substantial memory demands. Mixed-precision quantization is an effective compression technique that reduces bit-widths while adapting to the heterogeneous hardware budgets of edge devices, particularly well-suited for MoE due to its inherent structural sparsity, which causes imbalanced importance among experts. While prior work has applied quantization for efficient MoE inference at the edge, we point out that in MoE-based VLMs, the modality-specific characteristics of visual and text tokens further impact expert quantization. To address this issue, we present MoMoVL, a modality-aware adaptive compression framework for MoE-based VLMs that brings vision-language inference within reach of on-device resource budgets. Motivated by our empirical findings on modality-induced disparities across experts, MoMoVL jointly exploits the per-expert activation profile and per-modality quantization sensitivity to guide bit-width assignment. Experimental results show that MoMoVL improves the accuracy-compression trade-off relative to state-of-the-art mixed-precision baselines and reduces memory footprint by up to 15.44% with a modest drop in accuracy over its uniform-precision baseline. Our code and data are publicly available at this https URL.
179. 【2611.23680】Structured Extraction of Diffraction Data from Covalent Organic Frameworks
链接:https://arxiv.org/abs/2611.23680
作者:Shenghan Gao,Zhiling Zheng
类目:Computer Vision and Pattern Recognition (cs.CV); Materials Science (cond-mat.mtrl-sci); Artificial Intelligence (cs.AI)
关键词:Powder X-ray diffraction, Powder X-ray, covalent organic frameworks, primary structural evidence, organic frameworks
备注: 66 pages: main text (19 pages, 7 figures, 3 tables) and Supporting Information (47 pages) in one file. Data and code: [this https URL](https://github.com/AaronG211/pxrd_PEAK) ; database website: [this https URL](https://pxrd-peak.vercel.app)
点击查看摘要
Abstract:Powder X-ray diffraction (PXRD) is the primary structural evidence for covalent organic frameworks (COFs) and many other materials. Yet decades of measured patterns exist only as images in published figures. Automatic recovery is difficult, because PXRD figures vary widely in format, from a single curve to crowded stacks with labels, insets, and neighboring plots that challenge traditional computer vision. This work presents a pipeline in which a vision large language model (LLM) interprets the layout of each figure, locating the diffraction plot and its curves and sketching a coarse trace of each, while classical computer vision (CV) refines every trace to pixel-level accuracy. Applied to 3,954 COF articles, the pipeline produces 11,445 calibrated diffraction curves, released in the open database COF-PEAK. Compared with a hand digitization of 60 panels, extracted peak positions differ by a median of only 0.07 degrees. Whereas papers usually report only a few peak angles, which limits further analysis, the digitized curves enable tracking how peak prominence, a possible marker of crystallinity, changes with synthesis conditions. Vision LLMs and classical CV can thus turn such figures into reusable experimental data.
180. 【2611.23674】Learning What to Degrade: A Recoverability-Aware Curriculum for Reliable Smart-Glasses Super-Resolution
链接:https://arxiv.org/abs/2611.23674
作者:Yaopeng Wang,Qingliang Wang,Yilun Jin,Xiujin Liu,Han Yue,Chen Xia,Renjie Zhu,Chuang Sheng
类目:Computer Vision and Pattern Recognition (cs.CV)
关键词:obscure small text, fine-grained structures needed, images obscure small, vision-language models, obscure small
备注:
点击查看摘要
Abstract:Degradations in smart-glasses images obscure small text and fine-grained structures needed by vision-language models (VLMs) for visual assistance. We introduce an high-resolution (HR)-referenced semantic evaluation protocol that compares VLM outputs on restored images with those on corresponding HR references. Our evaluation reveals two failure modes: generative SR can synthesize misleading details, whereas restoration-oriented SR can leave critical evidence blurred or ambiguous. These findings motivate improving faithful detail recovery in restoration-oriented SR, where a key training challenge is to provide sufficiently difficult inputs while retaining recoverable visual information. We propose ReDiC (Recoverability-Difficulty Curriculum), which combines a model-independent recoverability proxy with model-relative restoration difficulty to adapt training degradation severity. ReDiC increases severity when inputs are recoverable and restoration is easy, maintains challenging but recoverable conditions, and reduces severity when estimated recoverability is low. It modifies only training degradation generation, leaving the SR architecture and reconstruction objective unchanged. Across NAFNet, SwinIR, and MambaIR, ReDiC improves average PSNR by approximately 2.5 dB and HR-referenced semantic scores by about 3 points over the corresponding baselines on synthetic benchmarks, with further gains confirmed by LLM-based and human evaluations on 200 real smart-glasses captures.
181. 【2611.23672】Image-Condition-Controlled Diffusion Network with Large-Scale CT Pretraining for CT-Free Attenuation Correction in Dynamic Cardiac PET
链接:https://arxiv.org/abs/2611.23672
作者:Zhejia Zhang,Scott M. Leonard,Jinkui Hao,Ryan J. Avery,Richard L. Weinberg,Bo Zhou
类目:Computer Vision and Pattern Recognition (cs.CV)
关键词:Positron emission tomography, Positron emission, NAC PET frame, myocardial perfusion, information on myocardial
备注: 11 pages, 6 figures, 3 tables. Submitted to IEEE Transactions on Radiation and Plasma Medical Sciences
点击查看摘要
Abstract:Positron emission tomography (PET) provides quantitative information on myocardial perfusion. In dynamic PET, tracer kinetics are tracked through a series of reconstructed time frames. Reliable attenuation correction is essential for accurate measurements. Clinical attenuation maps ($\mu$-maps) are typically derived from computed tomography (CT), which adds radiation exposure and may introduce PET-CT misregistration due to respiratory and cardiac motion. Moreover, supervised CT-free methods rely on paired non-attenuation-corrected (NAC) PET and CT data, which are scarce, whereas large-scale unpaired CT data are more readily available. To address these challenges, we propose Large-scale Unpaired CT-Informed Diffusion (LUCID), a framework that transfers anatomical priors learned from 10,000 chest CT volumes in the public CT-RATE dataset to $\mu$-map synthesis for dynamic cardiac $^{82}$Rb PET. First, a convolutional neural network (CNN) predicts an initial $\mu$-map from each NAC PET frame. Second, the CT-pretrained diffusion backbone refines a low-noise perturbation of the CNN prediction through a single denoising step, conditioned on the corresponding NAC PET frame via ControlNet. Finally, frame-wise predictions are averaged voxel-wise to obtain the final $\mu$-map. We further introduce the voxel-wise temporal standard deviation (TSD) to quantify inter-frame variability without requiring a reference $\mu$-map. We evaluated LUCID on an in-house dataset of 102 patients and found that it achieved the best quantitative accuracy while maintaining competitive inter-frame consistency. Ablation studies showed that refinement with a diffusion backbone pretrained on in-house CT data from the training patients was less accurate than the CNN prediction alone, highlighting the importance of large-scale CT pretraining.
182. 【2611.23670】EyeDiT: Anatomy-Grounded Multi-Modal Eye Generation and Simulation
链接:https://arxiv.org/abs/2611.23670
作者:Yonghan Lee,Zachary Willms,Kun Liu,Pushkar Anand,Ali Behrooz,Dmitri Model,Dinesh Manocha,Yatong An
类目:Computer Vision and Pattern Recognition (cs.CV)
关键词:critical for rapid, rapid development, Anatomy Variational Autoencoder, Synthetic eye, eye
备注:
点击查看摘要
Abstract:Synthetic eye data are critical for rapid development of eye tracking (ET) in AR/VR applications. However, existing synthetic eye generators are largely image-centric and lack a high-quality underlying 3D anatomy of the eye, which limits their control mainly to the gaze and provides no anatomical labels. To address this problem, we propose EyeDiT, an anatomy-grounded, multi-modal 3D eye generation and simulation model trained with optical coherence tomography (OCT), labeled anatomy point clouds and near-eye images. Our pipeline first trains a Multi-Modal Anatomy Variational Autoencoder (VAE), which encodes an anatomy-aware latent representation from all input modalities. Then, our Eye Latent DiT is trained to generate latent samples conditioned on the optical axis, the pupil position and radius, and the subject identity. Together with Gaussian decoders for novel-view simulation, our model enables control over the pupil, subject identity and viewpoint for data synthesis. We show that our model outperforms previous eye image synthesis models in novel-view synthesis (24.9 against 21.8 dB PSNR from a single view), and that a public ET neural network trained on our synthetic data shows a smaller sim-to-real gap than when trained on data from other generators (3.5$^\circ$ vs. 8.4$^\circ$ gaze error), approaches a network trained on real data, and further improves ET performance when combined with real data (2.3$^\circ$ vs. 2.6$^\circ$). More information is available on the project page: this https URL
183. 【2611.23661】LightSEAD: Lightweight SE(3)-Aware Keypoint Detector through Selective Knowledge Distillation
链接:https://arxiv.org/abs/2611.23661
作者:Mayowa Adebambo,Alexander D. Rast,Andrew Bradley
类目:Computer Vision and Pattern Recognition (cs.CV); Robotics (cs.RO)
关键词:increasing architectural complexity, computational demands, pursuit of improved, improved performance, performance in deep
备注:
点击查看摘要
Abstract:The pursuit of improved performance in deep learning has often been accompanied by increasing architectural complexity and computational demands. We explore an alternative direction: achieving more with less through structured supervision. We introduce LightSEAD, a lightweight SE(3)-aware keypoint detector initialised with calibrated Harris pseudo-labels and enriched through viewpoint-aware and appearance-aware supervision and selective knowledge distillation. LightSEAD comprises only six convolutional layers and 84,289 learnable parameters, achieving competitive repeatability across HPatches, KITTI 06 and 4Seasons while demonstrating its downstream applicability to relative pose estimation on MegaDepth-1500. Further optimised for mixed-precision inference, LightSEAD Coarse achieves approximately 909 FPS on an NVIDIA RTX 4090 and 139 FPS on a Jetson Orin Nano Super.
184. 【2611.23646】PRISM: Profiling and Representation of Individual Stereotypical Motion
链接:https://arxiv.org/abs/2611.23646
作者:Austin A. MBaye,Savik Kinger,Matthew S. Goodwin
类目:Computer Vision and Pattern Recognition (cs.CV)
关键词:interpretable joint structure, joint structure, fixed set, set of physical, physical landmarks
备注:
点击查看摘要
Abstract:An anatomical pose provides interpretable joint structure, but no fixed set of physical landmarks can fully specify a person's movement. We introduce PRISM, a complementary representation that characterizes repetitive movement using tracked masks and dense optical flow. Torso-relative flow measures intensity, frame-wise mask variation measures spatial extent, signed-flow recurrence estimates frequency, and persistent sliding-window geometry describes the rhythmicity of the movement. Across 1,890 synthetic conditions, the measurements recover controlled speed, amplitude, and frequency while exposing observation limits. An idealized hand-motion experiment demonstrates variation omitted by identical exact COCO-17 trajectories; controlled corruption experiments separately examine measurement sensitivity. Audits of two pose methods on the same data cohort and 36,930 external skeleton clips document incomplete observation and dependence on processing choices. In 1,404 annotated complex stereotypical motor movement bouts from six autistic participants, total flow converges with pose motion, while matched-coordinate examples and repeated-session profiles show differences in intensity, extent, frequency, and rhythmicity within behavior labels. A separate VLM comparison evaluates quantitative frequency estimation, and a profile-conditioned decoder links the measurements to illustrative skeletal movements. PRISM connects numerical profiles to visible examples, providing an interpretable way to compare how the same behavior is expressed beyond its label or event count.
185. 【2611.23620】How Uncertain Is My Action? Self-Correcting Visual Planning with Frozen World Models
链接:https://arxiv.org/abs/2611.23620
作者:Siqing Li,Luming Yang,Yijia Jiao,Junze Deng,Alper Yilmaz
类目:Robotics (cs.RO); Computer Vision and Pattern Recognition (cs.CV); Machine Learning (cs.LG)
关键词:sampling-based model predictive, model predictive control, world model combined, latent world model, frozen latent world
备注:
点击查看摘要
Abstract:A frozen latent world model combined with sampling-based model predictive control is a visual planner. At deployment, the executed action can depart from the commanded one, and the frozen model has no way to notice. We show that the model can be run in reverse: for any hypothesized actuation shift, it predicts what it should have observed under that shift, and comparing this prediction with what was actually observed quantifies the shift and lets the planner correct itself. Unlike the residual energy, which mixes the shift with the model's own error, the score keeps only the part of the residual that a shift can produce. This shift score detects the shift with a locally optimal test, estimates it with Cramer-Rao error bars, names the dominant generator, and compensates the planner once an alarm fires. On PushT, TwoRoom and OGBench-Cube with a frozen latent world model, measured detection follows the prediction to within 0.05 AUROC. Self-correction raises success from 0.650 to 0.862 on PushT and from 0.378 to 0.432 on the harder Cube task, levels with an oracle that is told the true shift at the same block, better than five common test-time adaptation methods. On a real robotic arm, the score identifies a base-joint offset that no joint encoder registers and reduces the reach error from 36 mm to 3-5 mm, and under shifts drawn at random and sealed before the session, estimates recover 92-94% of the error reduction achieved with the true shift. The broader implication is that a frozen world model can act as a filter by predicting the future, testing action-interface uncertainty and validity, and finally correcting failures.
186. 【2611.23616】Isthmus: Bridging Supervision Islands Restores Uncertainty and Stability in One-Step Flows
链接:https://arxiv.org/abs/2611.23616
作者:Luming Yang,Siqing Li,Haoxian Liu,Zhenyu Bu,Jiazheng Zhu,Alper Yilmaz
类目:Computer Vision and Pattern Recognition (cs.CV); Machine Learning (cs.LG)
关键词:One-step flow maps, one-step sampler loses, One-step, posterior samplers, One-step flow
备注:
点击查看摘要
Abstract:One-step flow maps are increasingly used as posterior samplers, where the point of sampling is the uncertainty that the samples carry. We find that the one-step sampler loses this uncertainty and becomes less stable. Compared with multi-step samples of the same network, its samples lose variance and decorrelate. Supervising the one-step jump directly makes this worse; the samples collapse further toward the conditional mean and break down when training is extended. We trace both failures to the locations where training places its pairs of start and end times. Only pairs on the diagonal of the time triangle receive targets from data, and the pair used for one-step sampling is either left unsupervised or supervised on an island, a set of pairs that the self-consistency identity does not connect to the diagonal, where the objective neither determines the one-step map nor anchors its training. We bridge the island with an isthmus, which places 5% of the training pairs along the whole edge of the triangle that joins the one-step pair to the diagonal. The error of the one-step map is then bounded by the self-consistency residual along this edge. With the network, the loss and the inference code unchanged and at no added cost, the isthmus improves one-step sampling across several flow-map objectives, in MRI reconstruction, face inpainting, and unconditional generation on CIFAR-10 and ImageNet 32x32. On brain MRI, it restores one-step calibration to 0.96 and the correlation with multi-step samples to 0.93; both vary far less across seeds, and the agreement holds when training is extended to three times its length.
187. 【2611.23615】Weight Space Programming for Vision Language Model Repair
链接:https://arxiv.org/abs/2611.23615
作者:Avinash Goutham Aluguvelly
类目:Computer Vision and Pattern Recognition (cs.CV); Artificial Intelligence (cs.AI)
关键词:network weights directly, weights, language model edits, weight-space programming, model
备注:
点击查看摘要
Abstract:In weight-space programming, a language model edits a network's weights directly, without gradients. We ask what a successful repair of this kind relies on. We plant faults in the weights of two vision-language models, SmolVLM-256M and LFM2-VL-450M, and check each edit made by a coding agent against the original weights. Without being told where to look, the agent finds rescaled rows in the projector and noise in the vision encoder of both models, infers the unknown scale factors, restores most or all of the rescaled weights to their exact original values, and returns accuracy to near its original level; programs that it writes from such weights repair unseen checkpoints of the same model without feedback. These repairs depend on a trace of how the faults were computed. The original weights have 16-bit precision, and faults computed in 32-bit arithmetic leave low bits that only the damaged weights carry. Rounding those bits away leaves each model's answers to our test questions unchanged, and the agent, unaided, then restores no weight exactly: on SmolVLM it no longer finds the damage, and in a single session on LFM2-VL its only gain comes from rescaling answer-token embeddings, which changes how often the model answers yes. Marking the damaged weights again in their low bits brings the edits back to them. Rounded weights still hold statistical evidence of the rescaling, in the distribution of stored values and in row norms, and one sentence in the prompt that names such a statistic directs the agent's edits to the damaged rows on both models. Swapped rows or columns, which leave the set of stored values unchanged, are not found unaided on either model within our session budgets. We recommend that studies of weight repair report the number format of the weights and include a rounded control.
188. 【2611.23606】VIBRA: Backward-Compatible Cross-Modal Retrieval with Consensus Barycenters
链接:https://arxiv.org/abs/2611.23606
作者:Shubing Yang,Xutong Liu,Dongfang Zhao
类目:Computer Vision and Pattern Recognition (cs.CV); Information Retrieval (cs.IR)
关键词:Backward-compatible retrieval enables, cross-modal retrieval system, Backward-compatible retrieval, retrieval enables, cross-modal retrieval
备注:
点击查看摘要
Abstract:Backward-compatible retrieval enables a new encoder to query the gallery that an old model has already encoded, so that a cross-modal retrieval system can be upgraded without re-encoding billions of items. Existing compatible methods either fit the old query and stay near the accuracy of the old model, or run the old and the new encoder for every query. We propose VIBRA, which matches relevance on the old gallery, corrects the errors of a single model with the barycenter of several views, which is itself a query in the old space, and trains the new encoder to inherit this consensus, so that models absent at deployment still shape every query. We show that the barycenter gains margin over the average of its views exactly when their hardest irrelevant items differ, and we characterize the share of this gain that one encoder can reach. On five benchmarks with five upgrades among six CLIP, MetaCLIP and SigLIP2 backbones, VIBRA is the best compatible method in all 50 settings, improving Recall@1 by up to 5.6 points over the strongest baseline and by up to 18.0 points over the old model, while adding less than 1 ms to the latency of the new encoder.
189. 【2611.23600】Entropy-Regularized Integrated Gradients for Stable Feature Attribution
链接:https://arxiv.org/abs/2611.23600
作者:Rukmangadh Sai Myana,Sumit Kumar Jha,Yanzhao Wu
类目:Machine Learning (cs.LG); Artificial Intelligence (cs.AI); Computer Vision and Pattern Recognition (cs.CV)
关键词:interpret neural networks, model predictions unchanged, leaving model predictions, Integrated Gradients, Entropy-Regularized Integrated Gradients
备注: 36 pages, 19 figures, 27 tables, including the appendix
点击查看摘要
Abstract:Path-based attribution methods like Integrated Gradients (IG) are widely used to interpret neural networks, yet imperceptible input perturbations can drastically alter their attribution maps while leaving model predictions unchanged. We introduce Entropy-Regularized Integrated Gradients (ERIG), a path-based attribution method that intrinsically attenuates gradient irregularities along the integration path using maximum-entropy softmax weighting with temperature parameter $\tau$. We quantify the effect of the entropy-regularized weighting on attribution stability by using the Lipschitz properties of the model and its gradients. We decompose the attribution change under an input perturbation into a model-response term, whose bound is $\tau$-independent, and a weight-variation term, whose bound decreases at rate $\mathcal{O}(1/(2\tau))$. Their ratio $R(\tau)$ characterizes the trade-off between gradient information and attribution stability. We further add a new dimension to attribution evaluation by adapting classical noise estimators into two metrics, LINV-C and LINV-P, that quantify attribution noise. Across eleven CNN and Transformer backbones on ImageNet and Oxford-IIIT Pet datasets, our experiments show that, relative to the next-best method, ERIG remains comparable on ranking-based fidelity metrics while improving localization by approximately 24% and attribution stability by approximately 47-fold on average, reaching up to four orders of magnitude improvement over IG, all at negligible additional computational cost.
190. 【2611.23591】GRAFT: Graph-Regularized Prototype Fields for Realistic Test-Time Adaptation of VLMs
链接:https://arxiv.org/abs/2611.23591
作者:Maxx Richard Rahman,Wolfgang Maass
类目:Computer Vision and Pattern Recognition (cs.CV); Machine Learning (cs.LG)
关键词:Unlabeled test data, test-time adaptation methods, Unlabeled test, streams are correlated, test data
备注:
点击查看摘要
Abstract:Unlabeled test data can improve the zero-shot classifier of a vision-language model, yet transductive and test-time adaptation methods often fall below the zero-shot baseline when only a few vocabulary classes are present in a batch or when streams are correlated. We trace this failure to a shared design: every class owns independent parameters estimated from its own samples, so absent or rare classes are left at biased text anchors or fitted to foreign samples. We propose Graph-Regularized Anchor-displacement Field Transduction (GRAFT), which models the adapted prototypes as text anchors plus a displacement field that is partially pooled over a semantic class graph through a Gaussian Markov random field prior, together with an explicit class-presence model. Inference is closed-form and gradient-free on cached features. Anchor-only shrinkage and a single global shift are the two extreme settings of the prior; absent classes are provably harmonic extensions of the present ones, and the pooling strength can be chosen by empirical Bayes. Under the realistic batch and online protocols on eleven benchmarks and four ImageNet shifts, GRAFT improves over the zero-shot model in every setting and over anchor-based transduction where few classes are present.
191. 【2611.23581】ExiST: Existence-Aware Self-Training for Semi-Supervised Multi-Organ Segmentation
链接:https://arxiv.org/abs/2611.23581
作者:Maxx Richard Rahman,Prakhar Kumar,Wolfgang Maass
类目:Computer Vision and Pattern Recognition (cs.CV); Artificial Intelligence (cs.AI)
关键词:Semi-supervised multi-organ segmentation, Semi-supervised multi-organ, smallest organs occupy, multi-organ segmentation, segmentation should learn
备注:
点击查看摘要
Abstract:Semi-supervised multi-organ segmentation should learn more than a dozen organs from a handful of labeled scans, and the smallest organs occupy less than one percent of the foreground. We identify a failure mode of self-training that is specific to this regime. When a teacher misses a small organ in a scan, every voxel of that organ receives the confident pseudo-label background, and the student is trained toward the organ absence. Repeated over many scans, this loop extinguishes tail organs. In a pilot study, removing the gallbladder from the pseudo-labels reduced its Dice on AMOS from 36.5 to 1.0. We argue that a missing organ should be treated as unresolved rather than absent and propose Existence-aware Self-Training (ExiST). It builds pseudo-labels by leave-one-out consensus of independently trained networks, screens them with a quality gate on validation scans, and infers, per scan and per organ, whether the organ was found. For every unresolved organ, the background voxels of that scan receive the set-valued target, trained with a marginal cross-entropy that accepts either answer and a Dice term that skips uncertain voxels. We show that this loss never pushes a candidate organ toward absence and never favors one member of the set, which leaves the decision to labeled evidence. ExiST outperforms the current state-of-the-art methods and reaches 77.1 mean Dice on Synapse with 20% labeled scans and 75.9 on AMOS with 5% labeled scans.
192. 【2611.23578】DOCK: Dual-Stream-Inspired Self-Supervised Object-Centric Representations Grounded in Robot Kinematics
链接:https://arxiv.org/abs/2611.23578
作者:Pengju Liu,Caiwen Ding
类目:Computer Vision and Pattern Recognition (cs.CV); Robotics (cs.RO)
关键词:Latent-space planners select, planners select actions, predicted future image, Latent-space planners, planners select
备注:
点击查看摘要
Abstract:Latent-space planners select actions by comparing the embedding of a predicted future image with that of a goal image. This assumes that distance in the embedding reflects how far the scene is from the goal. We profile this assumption with simulator-executed futures, which removes prediction error. For six frozen DINO and V-JEPA encoders, feature distance barely tracks the object's distance to its goal: the rank correlation is at most 0.11, against 0.87 for object-mask centroids. It is dominated by background and robot patches, and frozen-feature readouts capture the arm far better than the object. Inspired by the ventral (what) and dorsal (where/how) visual streams, we propose DOCK, which learns an object representation whose changes agree with measured physical motion. A grasped object moves with the gripper, so forward kinematics on the joint angles gives a proxy for the object's displacement without annotation. From 9,951 held-object transport trajectories in 21 simulated kitchen scenes, we mine 404K displacement pairs and train three components: an object token pooled from patch features (what), a spatial ruler that makes differences of readout coordinates equal to measured displacements (where), and a dynamics predictor over object tokens (how). Used as a planning cost, the learned representation places the object in 15 of 21 tasks, as many as an oracle that reads the true object position, while mask geometry reaches 10 and frozen feature distance 9. It still solves the most tasks when the unseen robot arm is visible, and its ranking transfers to held-out layouts with unseen object instances and categories without any adaptation.
193. 【2611.23554】Layer-Matched Sparse Attention for Low-Redundancy 3D Reconstruction
链接:https://arxiv.org/abs/2611.23554
作者:Sammam Mahdi,Fariha Binta Salim,K. M. Jubair Sami,Rakin Bin Rabbani
类目:Computer Vision and Pattern Recognition (cs.CV)
关键词:VGGT reconstructs cameras, VGGT reconstructs, global attention grows, attention grows quadratically, reconstructs cameras
备注:
点击查看摘要
Abstract:VGGT reconstructs cameras and geometry from unposed images in one pass, but its global attention grows quadratically with the number of views. Training-free accelerators thin this attention and work well on redundant video, but lose pose accuracy when views share little geometry. This paper traces these losses to two kinds of cross-view information that a uniform sparse approximation can fail to preserve together: diffuse layers aggregate distributed context, whereas matching layers depend on query-specific interactions that carry little attention mass. A per-layer statistic measures the log-mass missed by a mean token. Its row-free proxy is the logit variance, and it selects the estimator for each layer. Diffuse layers use shared, importance-sampled hub keys, with log-weights that make the softmax numerator and denominator unbiased. Matching layers stay exact: every aggressive thinning tested cost accuracy. At equal cost, the selected layers outperform the four shifted windows tested. On 13 unseen low-redundancy scenes with 32 views, LMSA stays within 1.4 AUC@5 of dense attention, compared with 10.3 for AVGGT and 4.5 for SAF3R; its difference from contemporaneous GPA-VGGT is not significant, and only its gains over AVGGT pass both scene-level tests. With 100 views, it stays within 1.9 AUC@5 while its aggregator runs $2.0\times$ faster. Used post hoc to choose the layers GPA-VGGT protects, the statistic improves it by 1.2 AUC@5 for 5% more FLOPs. Applied untuned to $\pi^3$, it beats AVGGT but remains further from dense accuracy.
194. 【2611.23543】Stability of Parallel Transport Convolution under Riemannian Metric Perturbations
链接:https://arxiv.org/abs/2611.23543
作者:Hongnian Huang,Andrew Stump
类目:Machine Learning (cs.LG); Computer Vision and Pattern Recognition (cs.CV); Differential Geometry (math.DG); Machine Learning (stat.ML)
关键词:underlying domain, Riemannian manifolds depend, geometric deep learning, fundamental question, Riemannian geometric constructions
备注: 78 pages. Comments welcome!
点击查看摘要
Abstract:Geometric convolutions on Riemannian manifolds depend on the geometry of the underlying domain, making stability under metric perturbations a fundamental question in geometric deep learning. This paper studies this question for parallel transport convolution through methods from differential geometry and analysis. On a fixed compact manifold, we establish that, for a continuous input signal and a continuous reference kernel supported in a sufficiently small normal neighborhood, the convolution output depends continuously in $L^2$ on the Riemannian metric in the $C^2$-topology, under the specified localization and transport-path conventions. The analysis develops perturbation estimates for geodesics, parallel transport, Jacobi fields, and exponential maps, and uses these estimates to compare the transported kernels. A central difficulty is the cut locus, which obstructs uniform comparison of the transport construction over the entire manifold. We address this difficulty by combining uniform estimates away from the cut locus with integral bounds on neighborhoods of arbitrarily small volume, while also controlling the change in the Riemannian volume measure. The resulting qualitative stability theorem connects perturbation theory for Riemannian geometric constructions with the analysis of spatial convolution operators in geometric deep learning.
195. 【2611.23529】ORGANIC3D: Grounding 3D CT Vision Language Models in Organ Level Image Text Coupling
链接:https://arxiv.org/abs/2611.23529
作者:Zelong Liu,Hamza Ahmed,Wynn Wingyi Lee,Mickael Torjman,Alexander Dimitrakakis,Jiahua Lu,Timothy Deyer,Bachir Taouli,Zahi A. Fayad,Xueyan Mei
类目:Computer Vision and Pattern Recognition (cs.CV)
关键词:achieved strong performance, computed tomography, achieved strong, strong performance, correct anatomical evidence
备注:
点击查看摘要
Abstract:Vision language models (VLMs) have achieved strong performance in 3D computed tomography (CT) classification, but high accuracy does not reflect whether their predictions rely on the correct anatomical evidence. We introduce ORGANIC3D, a framework for evaluating and improving organ level image text alignment through controlled anatomical interventions. Our benchmark assesses 12 vision encoder checkpoints and 7 generative VLMs using matched patient comparisons, organ erasure, lesion replacement, and laterality perturbations, revealing substantial alignment failures even among top performing models. To learn organ level image text correspondence, ORGANIC3D aligns each finding sentence with the anatomy aware image features and trains on edited scans to anchor this alignment in the organ's image evidence. We further extend this framework to generative VLMs, promoting evidence dependent question answering and report generation. Trained exclusively on public data, ORGANIC3D achieves SOTA classification AUROC on CTRATE, Merlin, and the heldout RADChestCT dataset and reaches 77% laterality accuracy, compared with at most 66% for prior models. Its generative VLM matches or exceeds all evaluated publicly released VLMs on visual question answering, achieves the highest report accuracy, and retracts 96 to 99% of target organ findings after organ removal, versus a maximum of 69% for other models. Together, these results establish ORGANIC3D as a framework for testing whether 3D CT vision language models ground their predictions in anatomical evidence and for improving organ level alignment alongside predictive accuracy.
196. 【2611.23517】RageVLA: Augmentation-Trained Perception and Action for State-of-the-Art Robustness on LIBERO
链接:https://arxiv.org/abs/2611.23517
作者:Amrit Lal Singh,Smarak Kanjilal
类目:Robotics (cs.RO); Computer Vision and Pattern Recognition (cs.CV)
关键词:object moves, saturated on LIBERO, instruction, object, Amrit Lal Singh
备注: 10 pages, 10 figures, 9 tables
点击查看摘要
Abstract:Vision-language-action (VLA) models look saturated on LIBERO, yet most of them fall apart on LIBERO-PRO as soon as an object moves or the instruction asks for something new. We argue that what is missing is not scale but decoupling. In these policies, what to do is tied to where things were during training, to the scene around them and to the wording of the instruction. We cut these ties in two places. In the data, LIBERO demonstrations are re-arranged by general rules: every object and fixture is moved and rotated, tasks are carried into other environments and onto other surfaces, and every instruction is paraphrased. In the policy, a perception engine on frozen DINOv3 features, trained on this augmented data, hands the action model the scene in decoupled form: a list of objects with their positions, and a compact code of what the gripper is looking at, next to image and instruction embeddings from TinyCLIP. A flow-matching diffusion transformer, trained from scratch, acts on this view. We call this combination of Robust Augmentation and Grounded Encoding RageVLA. With 0.21 billion parameters, RageVLA sets a new state of the art on LIBERO-PRO among VLA policies, scoring 0.72 against 0.53 for the best published model, and a larger variant reaches 0.73. Two further tests on LIBERO-Object show that the policy acts on what it perceives. When the object positions it is given are shuffled while the camera images stay correct, its success falls from 0.965 to 0.215. When the target object is displaced by 28 cm, it still succeeds in 0.865 of the episodes, where $\pi_{0.5}$, run through the same harness, succeeds in 0.105.
Comments:
10 pages, 10 figures, 9 tables
Subjects:
Robotics (cs.RO); Computer Vision and Pattern Recognition (cs.CV)
Cite as:
arXiv:2611.23517 [cs.RO]
(or
arXiv:2611.23517v1 [cs.RO] for this version)
https://doi.org/10.48550/arXiv.2611.23517
Focus to learn more
arXiv-issued DOI via DataCite (pending registration)
Submission history From: Amrit Lal Singh [view email] [v1]
Tue, 17 Nov 2026 21:07:47 UTC (662 KB)
197. 【2611.23516】Beyond the Sink Token: Sink-Guided Key Editing in Vision-Language Models
链接:https://arxiv.org/abs/2611.23516
作者:Hui Zhu,Saleh Zare Zade,Rafi Ibn Sultan,Xiangyu Zhou,Dongxiao Zhu
类目:Computer Vision and Pattern Recognition (cs.CV); Artificial Intelligence (cs.AI)
关键词:tokens attract, disproportionate share, sink tokens attract, tokens attract attention, attention
备注:
点击查看摘要
Abstract:In vision-language models (VLMs), a few sink tokens attract a disproportionate share of attention. While previous studies have examined why these tokens attract attention, we investigate how the sink-key direction affects attention to image and question tokens. Across four probed VLMs, boosting the magnitude of positive sink-aligned components increases attention on average, whereas boosting negative components decreases it; these responses persist through later layers. This identifies the native sink key as a reference for controlling attention beyond its own position. Motivated by this finding, we propose SHRINK, a training-free method that partially reduces sink-aligned components across visible keys during generation. On nine benchmarks under sampled reasoning, it improves mean accuracy by 5.1 and 2.7 percentage points on LLaVA-OneVision-7B and Qwen2.5-VL-3B, with smaller gains on the other two main models, Qwen2.5-VL-7B and Gemma-3-4B. Beyond SHRINK, rescaling sink-aligned key components can replace the attention operations of two existing attention-steering methods, GIFT and ACG, with comparable accuracy on hallucination benchmarks.
198. 【2611.23513】rust the Ray, Not the Depth: Ray-Lateral Agreement for Multi-View 3D Human Pose Fusion
链接:https://arxiv.org/abs/2611.23513
作者:Farnoosh Koleini,Ahmed Helmy,Pu Wang,Xiwu Cao
类目:Computer Vision and Pattern Recognition (cs.CV)
关键词:reconstructed bodies land, Put a monocular, monocular mesh estimator, calibrated rig, monocular mesh
备注: 10 pages, 5 figures, 1 table
点击查看摘要
Abstract:Put a monocular mesh estimator behind every camera of a calibrated rig, and the reconstructed bodies land a meter or more apart. The cause is geometric: each estimate sits at the depth implied by a focal length the network assumed, not the one the camera has. Its error therefore points almost entirely along its own viewing ray, and the calibration tells us that direction exactly. To address this, we introduce the Ray-Lateral Agreement (RLA), which follows one rule: trust the ray, not the depth. RLA first slides every estimate along its own rays, which removes the depth bias and leaves the shape untouched. It then fuses the views one point at a time, under a Gaussian error whose precision follows each viewing ray. The maximum-likelihood solution is closed-form, interpolates between averaging and midpoint triangulation, and stays well conditioned where triangulation is singular. Robustness is the harder half, because correct views also disagree along their rays. Any weight built on the full residual therefore punishes good views together with bad ones. Ray-lateral agreement instead scores every view, at every point, by its residual across that view's own ray alone. We evaluated RLA on multi-camera public benchmark datasets. Across two, three, and four cameras and three per-view pose estimators, RLA consistently achieves lower worst-case error than all triangulation baselines.
199. 【2611.23508】BOS Summary: The First Token Matters
链接:https://arxiv.org/abs/2611.23508
作者:Hui Zhu,Saleh Zare Zade,Rafi Ibn Sultan,Deng Pan,Arun Ross,Dongxiao Zhu
类目:Artificial Intelligence (cs.AI); Computation and Language (cs.CL); Computer Vision and Pattern Recognition (cs.CV)
关键词:Large vision-language models, Large vision-language, BOS Summary, BOS, receive a disproportionate
备注:
点击查看摘要
Abstract:Large vision-language models (LVLMs) and language models (LLMs) often exhibit attention sinks, whereby particular positions, such as the beginning-of-sequence (BOS) token, receive a disproportionate share of attention. Under standard causal masking, however, the BOS query attends only to itself, so the representation that later tokens repeatedly access carries no information about the input. We introduce BOS Summary, a training-free inference-time method that repurposes this persistent pathway to carry input context. At a single transformer layer during prefill, BOS Summary relaxes the causal constraint for the BOS query row alone, letting BOS aggregate all visual and textual input tokens into a self-derived context representation while every other token retains standard causal attention. The enriched representation then remains accessible to the model throughout decoding, with no additional parameters, auxiliary modules, or decoding-time overhead. Across multiple VLM and LLM families and eight multimodal and text-only benchmarks, BOS Summary improves over direct and Chain-of-Thought prompting and outperforms recent sink-intervention methods on multi-step reasoning tasks. A mechanistic analysis shows that making BOS input-specific also weakens its sink dominance and redistributes attention toward content-bearing prompt tokens, indicating that changing what a sink carries reshapes how later layers allocate attention.
200. 【2611.23501】MedHarness: A Unified Budgeted Runtime for Reliable Medical Imaging Agents
链接:https://arxiv.org/abs/2611.23501
作者:Haofan Wu,Le Zhang
类目:Computer Vision and Pattern Recognition (cs.CV)
关键词:Medical imaging agents, harm clinical trust, limited verification budget, Medical imaging, heterogeneous imaging surfaces
备注:
点击查看摘要
Abstract:Medical imaging agents already return answers on heterogeneous imaging surfaces, yet deployment still faces a limited verification budget where each wrong accept can harm clinical trust. When a thin check and a deeper evidence pass disagree on the same case, deciding when to stop, open more evidence, repair, or reject without a shared controller remains ill-posed. Tool-using workflows often lock a surface-specific evidence order per dataset, or a private always-open full-depth verification path; those pipelines stay costly, leave many wrong accepts, and cannot share one controller across modalities. We present MedHarness, a registered multi-surface runtime reliability layer that reformulates post-answer control as shared decision-object attachment after a proposer's initial prediction. Under Shared Decision Object, Minimum-Sufficient Verification, and Intermediate Repair, one shared policy with surface recipe adapters lets a new imaging surface join the same stop / open-evidence / repair / reject controller by recipe registration. On a 512-case live pool, MedHarness cuts wrong accepts versus a full-verification baseline by about 47% and raises accepted accuracy to 83.2% vs. 72.9% at 86.1% coverage versus full release under that baseline. On a failure-enriched multi-surface stress ranking, the same controller cuts estimated remote service calls per case by about 71% versus that baseline under a matched context-load setting. A held-out three-surface check and an alternate language-model transfer keep the same control ordering. The code is available at this https URL.
201. 【2611.23491】GRID-UAV: Geometry and Ray-depth Integrated Decision-making for Unmanned Aerial Vehicles
链接:https://arxiv.org/abs/2611.23491
作者:Anas Anwarul Haq Khan,Amrin Kareem,Muhammad Umer Sheikh,Biplab Banerjee,Muhammad Haris Khan
类目:Computer Vision and Pattern Recognition (cs.CV)
关键词:encode images, encoders that capture, implicit alignment, policy estimated depth, depth
备注:
点击查看摘要
Abstract:Vision-language-action models (VLAs) encode images with 2D encoders that capture little 3D structure. A common approach is to feed the policy estimated depth, but a UAV chooses its own altitude, must judge distances tens of meters away, where monocular depth is less reliable, and must relate its forward and downward views. On long aerial routes, adding depth features directly brings no significant gain (42.5% vs. 42.6% success), so we study how 3D information should be given to the policy. Analyzing flight failures, we find that two forms of 3D grounding fail in opposite ways: explicit 3D lifting of depth collides less on the way but more often fails to finish at the target, while implicit alignment with a 3D geometry model finishes more often but collides more. We propose GRID, which combines both: (1) it lifts monocular depth into ray-corrected 3D positions in the UAV's frame for both cameras, and (2) during training only, it aligns its visual features with a frozen multi-view geometry model, at no inference cost. GRID adding explicit lifting to implicit alignment significantly increases how often the policy reaches the target ($61.5\% \rightarrow 69.5\%$, $p0.001$), and adding implicit alignment to explicit lifting significantly increases how often it completes the task once there ($57.7\% \rightarrow 61.4\%$, $p0.05$). On TravelUAV (3,005 test episodes), GRID achieves the best success rate on every split, with relative gains over the state of the art of 9.5% (seen), 17.1% (unseen objects) and 25.4% (unseen map).
202. 【2611.23484】Reference-Guided Data Curation for 3D Medical Vision-Language Pretraining
链接:https://arxiv.org/abs/2611.23484
作者:Faidra Patsatzi,Ashwin Kumar,Chong Wang,Curtis P. Langlotz,Reinhard Heckel,Akshay S. Chaudhari
类目:Computer Vision and Pattern Recognition (cs.CV)
关键词:large medical imaging, medical imaging datasets, imaging datasets collected, medical imaging, data
备注: Accepted at WACV 2027. Code: [this https URL](https://github.com/StanfordMIMI/data-curation-3d-medical-vlm)
点击查看摘要
Abstract:Training 3D vision-language models (VLMs) on large medical imaging datasets collected across institutions is constrained by data quality. Vast, uncurated pools of image-report pairs are characterized by redundancy, class imbalance, and cross-institutional heterogeneity. We address this challenge of data curation for improving 3D VLM pretraining by proposing an offline, training-free, reference-guided data curation recipe. Our approach leverages a small, high-quality reference set to independently extract quality signals for each modality of a large, uncurated pool and merges them through a rank-aggregation step into a single ranking that guides sample selection. We validate this method on datasets comprising 35k 3D abdominal CT volume-report pairs from two institutions and 58k 2D chest X-ray image-report pairs. On 3D CT, using our reference-guided data curation strategy, our curated model matches the zero-shot disease classification performance of a state-of-the-art baseline (Merlin) while using $3.3\times$ less training data and a reference set of 5k samples. On 2D chest X-rays, our method yields a curated model that matches or exceeds the zero-shot disease classification performance of a model trained on the full pool while using up to $10\times$ less training data and a reference set of only 74 samples. Ablations show that the selection is stable under resampling of the reference, that smaller reference sets still outperform random sampling, and that label rebalancing does not improve performance. To support reproducibility, we release our code.
203. 【2611.23483】SpecLook: Speculating Where to Look for Fast and Reliable Agentic MLLMs
链接:https://arxiv.org/abs/2611.23483
作者:Xi Ding,Jiawei Zhang
类目:Artificial Intelligence (cs.AI); Computation and Language (cs.CL); Computer Vision and Pattern Recognition (cs.CV); Machine Learning (cs.LG)
关键词:Agentic multimodal large, Agentic multimodal, images interleave reasoning, multimodal large language, images interleave
备注: Technical report
点击查看摘要
Abstract:Agentic multimodal large language models (MLLMs) that think with images interleave reasoning with zoom-in tool calls. However, this sequential loop is slow, and its answers carry no reliability score. Prior speculation of tokens, reasoning steps, or answers keeps the loop, and answer speculation also accepts confident errors. We find that the loop mostly adds one zoom-in step, with which a small model is ahead of the agent on most splits. Our key insight is to speculate the agent's next observation, which, unlike a drafted answer, can be verified, corrected, and used to test the answer. We propose SpecLook, a training-free speculate-and-verify framework with three levels. A small model drafts a glimpse and re-answers from it, and the question ends if the answer is unchanged and confident. Otherwise, the agent's large model verifies and corrects the glimpse without tools, and their answers are fused. Only questions that still lack evidence reach the agent, in one forward pass that SpecLook-Fast skips, so no question runs the loop. The trace scores every answer's reliability without labels. Extensive experiments on V*, HR-Bench, and POPE show that both are more accurate than every baseline on every split. SpecLook reaches 87.2% and 86.2% mean accuracy with 7.7x and 4.7x speedups over the DeepEyes-7B and Thyme-RL agents, and SpecLook-Fast reaches 9.9x and 6.3x.
204. 【2611.23477】mmTherm: Absolute 3D Human Pose Estimation from Low-Resolution Thermal Images and mmWave Radar
链接:https://arxiv.org/abs/2611.23477
作者:Zhaoxin Chang,Badii Jouaber,Daqing Zhang
类目:Computer Vision and Pattern Recognition (cs.CV)
关键词:human pose estimation, aims to recover, global position, make HPE under-constrained, clouds make HPE
备注:
点击查看摘要
Abstract:Absolute 3D human pose estimation (HPE) aims to recover both a person's body pose and their global position in 3D space. Privacy-preserving millimeter-wave (mmWave) radar provides distance and direction measurement capabilities, but its sparse point clouds make HPE under-constrained. Low-resolution thermal imaging can provide complementary observations of body direction and structure. However, our evaluation shows that a strong feature-level fusion baseline loses much of its localization and HPE advantage when generalizing to unseen subjects, with radial localization becoming less accurate than a radar-only baseline. Motivated by this finding, we introduce mmTherm, a lightweight framework that explicitly incorporates the complementary geometric properties of thermal imaging and mmWave radar. It combines body structure and joint directions estimated from thermal images with range information from radar to localize the body and reconstruct its 3D pose. Experiments on OctoNet show that mmTherm reduces cross-user absolute joint error by 33% and root localization error by 38% relative to the strongest corresponding baselines, while requiring substantially fewer parameters and less computation at inference.
205. 【2611.23456】VeriMath: Human-in-the-Loop Generation and Verification of Curriculum-Aligned Mathematics Assessment Items
链接:https://arxiv.org/abs/2611.23456
作者:Quy P. Nguyen,Duong Q. Nguyen,Dinh P. T. Nguyen,Truong-Giang T. Le,Truong A. Bui,Vi H. Vo,Quang-Thinh Bui,My-Phuong Ngo,Trong N. Do,Thao P. Le,Phuc M. Nguyen,H. Nguyen-Xuan
类目:Artificial Intelligence (cs.AI); Computer Vision and Pattern Recognition (cs.CV)
关键词:deployment remains challenging, classroom deployment remains, educational artificial intelligence, Automatic generation, artificial intelligence
备注:
点击查看摘要
Abstract:Automatic generation of mathematics assessment items is a growing area in educational artificial intelligence, yet classroom deployment remains challenging. Generated questions must satisfy multiple constraints: mathematical correctness, curriculum alignment, appropriate cognitive difficulty, and educator reviewability. While prior work has advanced text generation, reliable assessment authoring requires granular control over localized examination formats and expert educational judgment. To address this challenge, we present VeriMath, a generalized, human-in-the-loop framework for generating rigorous, curriculum-constrained mathematics assessment drafts. Using annotated reference items, the framework anchors generation to the requested examination section and cognitive reasoning level, then applies sequential automated verification and guided repair before submitting drafts for expert review. We evaluate VeriMath through a national curriculum case study using 4,555 mathematics items from the Vietnamese Grade 12 curriculum and 50 distinct generation requirements covering multiple-choice, true/false, and short-answer formats. Across two model backbones, the framework produces verifier-accepted drafts for 94% (47/50) of requirements per run. Blind expert evaluations confirm high mathematical accuracy and curriculum alignment, with strict teacher-readiness assessments accepting up to 56.8% of generated items. Analysis of rejected drafts reveals persistent domain-agnostic challenges involving formatting, statement validity, and solution-step completeness. These findings demonstrate that VeriMath can bridge generative AI and dependable educational applications by producing highly reviewable assessment drafts within tightly constrained national curriculum frameworks.
206. 【2611.23450】NIRJALA: Draining the Sea out of Gaussian Splatting with Three-Path Optics and Reversible Physics
链接:https://arxiv.org/abs/2611.23450
作者:Latchan Chhetri,Aman Kumar
类目:Computer Vision and Pattern Recognition (cs.CV); Graphics (cs.GR)
关键词:Water, underwater photograph, Gaussian Splatting framework, Gaussian Splatting, scene
备注: 21 pages, 11 figures
点击查看摘要
Abstract:Water does not just tint an underwater photograph; it rewrites it. Each pixel mixes attenuated scene radiance, backscattered light, and scene light scattered forward on its way to the camera, and many combinations of scene and water explain the same pixels. Most Gaussian-splatting methods for underwater scenes model only the first two effects and remove the water by dividing by a transmission that decays exponentially with range, so their inversion gain grows without bound. We present NIRJALA (Sanskrit for "without water"), a 3D Gaussian Splatting framework that recovers water-free appearance and geometry together. TRIVENI, a three-path image formation model, adds a range-saturating forward-scattering term to direct transmission and backscatter; we prove that this keeps the inversion gain bounded at every range and bound how errors in the estimated backscatter and attenuation propagate into the recovered colours. PRATIBIMBA, a reversible physics anchor, ties a flexible neural water model to a ten-parameter physical model in both directions, so scene content cannot hide inside the water. SAMARASA collects the physics-informed objectives, including a range-binned colour prior. To score water removal against ground truth, we built SAJALA ("with water"), a 30-scene suite path-traced in Blender Cycles with water-free images, depth, and normals under controlled water colour and turbidity. On SAJALA, NIRJALA raises PSNR from 24.34 to 24.76 dB and lowers depth AbsRel from 0.109 to 0.091 against the strongest baseline; on real SeaThru-NeRF scenes it lowers LPIPS from 0.194 to 0.175 while rendering at 44 FPS. We also discuss where appearance metrics and geometry disagree underwater, and the limits of our evaluation.
207. 【2611.23446】Where Global Context and Weight Sharing Work in Compact Vision Backbones
链接:https://arxiv.org/abs/2611.23446
作者:Michael Schroeder
类目:Computer Vision and Pattern Recognition (cs.CV); Artificial Intelligence (cs.AI); Machine Learning (cs.LG)
关键词:depthwise separable convolutions, Compact convolutional backbones, losing representational capacity, EfficientNet achieve strong, local depthwise separable
备注: 8 pages, 5 figures
点击查看摘要
Abstract:Compact convolutional backbones such as EfficientNet achieve strong parameter efficiency through local depthwise separable convolutions, yet two structural questions remain central to hybrid design: where does global self-attention improve recognition over pure convolution, and can repeated blocks share weights without losing representational capacity? We investigate both questions on ImageNet-1k (50,000-image validation) using controlled EfficientNet-B0 experiments. First, we show that wrapping a classification backbone in multi-scale feature pyramids (BiFPN) fails to outperform a plain MBConv trunk (69.99% - 71.81% vs. 72.86% plain B0) until paired with stage attention (73.15%). In contrast, keeping the MBConv trunk intact and inserting lightweight zero-gated residual self-attention adapters at stages P3 (28x28 windowed), P4 (14x14 global), and P5 (7x7 global) reaches 74.02% standalone and outperforms a bit-for-bit matched plain-B0 control by +0.954 percentage points (73.88% vs. 72.92%) across an 88-epoch stateless replay schedule, with learned attention amplitude rising monotonically with semantic depth (27.2% at P3 - 45.4% at P4 - 66.3% at P5). Second, we evaluate recurrent weight sharing across repeated MBConv blocks with pass-specific BatchNorm statistics (+1,536 parameters per pass). While post-hoc kernel averaging degrades accuracy to 17.03% and single-template stage tying (-27.6% parameters) plateaus at 66.10%, alternating two shared block templates ([B1, A, B, A]) in Stage 6 cuts total model parameters by 10.9% (582K weights) and recovers by +10.82 points in a single epoch to 70.92% (-1.84 points vs. unshared control).
208. 【2611.23445】Why Sub-4-Bit Vision CNNs Collapse and Where Mixed-Precision Recovery Stops
链接:https://arxiv.org/abs/2611.23445
作者:Michael Schroeder
类目:Artificial Intelligence (cs.AI); Computer Vision and Pattern Recognition (cs.CV); Machine Learning (cs.LG)
关键词:bits frequently induces, Deep vision models, resource-constrained environments depend, environments depend heavily, frequently induces catastrophic
备注: 8 pages, 6 figures, 2 tables
点击查看摘要
Abstract:Deep vision models operating in resource-constrained environments depend heavily on low-bit quantization, yet compressing weights below 4 bits frequently induces catastrophic accuracy collapse. While mixed-precision recovery strategies aim to rescue accuracy by selectively retaining high-precision parameters, the fundamental mechanics of where and why these models collapse remain poorly understood. In this study, we systematically analyze the anatomy of quantization collapse across standard Vision CNNs (including ResNet and EfficientNet backbones) evaluated on the full 50,000-image ImageNet-1k validation set. First, we map the exact progression of degradation, demonstrating that collapse is highly non-linear and concentrated in late-stage pointwise projections and depthwise layers. Second, we introduce a structured, multi-step recovery waterfall that evaluates selective integer precision (INT4/INT8) and selective floating-point scales. We show that keeping less than 5% of critical layer activations in 8-bit precision recovers up to 92% of the accuracy lost during aggressive sub-4-bit compression. Finally, we bridge the hardware reality gap by evaluating deployment performance on standard ARM and x86 CPU architectures. We show that while low-bit quantization reduces memory storage requirements, standard runtime software bit-unpacking introduces a 1.48x to 21.5x execution slowdown depending on batch size, highlighting a critical trade-off between model footprint and actual edge throughput. Our findings provide a practical blueprint for navigating the limits of sub-4-bit deployment without sacrificing hardware performance.
209. 【2611.23442】ransFIRA+: Transfer Learning for Biometric Recognizability Assessment
链接:https://arxiv.org/abs/2611.23442
作者:Allen Tu,Kartik Narayan,Joshua Gleason,Jennifer Xu,Matthew Meyn,Tom Goldstein,Vishal M. Patel
类目:Computer Vision and Pattern Recognition (cs.CV)
关键词:quality metrics fail, visual quality metrics, variation in pose, web imagery, imagery must contend
备注: Project Page: [this https URL](https://transfira.github.io/plus)
点击查看摘要
Abstract:Biometric recognition in surveillance, video, and web imagery must contend with extreme variation in pose, blur, illumination, occlusion, and distance, where visual quality metrics fail to predict whether inputs are recognizable to the deployed encoder. Existing face image quality assessment methods rely on visual heuristics, curated annotations, or costly generative pipelines, leaving their predictions detached from the encoder's decision geometry. We introduce TransFIRA, a lightweight framework that grounds recognizability directly in embedding space. TransFIRA derives class-center similarity (CCS) and class-center angular separation (CCAS) from a pretrained encoder's embeddings and learns to predict them from images without quality annotations. Using predicted CCS for template weighting and CCAS0 as a natural filtering threshold yields state-of-the-art face verification performance on BRIAR, raising TAR at $10^{-6}$ FMR from 0.085 with uniform averaging to 0.598. The predicted scores also reveal how image degradations affect encoder-specific recognizability. TransFIRA+ extends this framework to cross-camera multimodal person identification in untrimmed surveillance video. In this setting, body recognizability scores saturate within a narrow range, making direct weighting effectively uniform. We introduce a post-hoc sigmoid calibration that transforms these saturated scores into informative weights without retraining. After calibration, the BRIAR-trained TransFIRA predictor improves fused face and body identification on two unseen multi-camera datasets, outperforming detector-confidence weighting and uncalibrated CCS-rank weighting. Together, TransFIRA and TransFIRA+ demonstrate that encoder-grounded recognizability generalizes across biometric modalities and recognition settings.
210. 【2611.23432】Bits, Not Boxes: Label-Free Layer Selection for Visual Localization in VLMs
链接:https://arxiv.org/abs/2611.23432
作者:Haisheng Fu,Gexin Huang,Haikai Zhao,Jie Liang,Zhenman Fang,Jingning Han,Xiaoxiao Li
类目:Computer Vision and Pattern Recognition (cs.CV); Methodology (stat.ME)
关键词:maps cheaply show, Attention maps cheaply, unreliable for localization, maps cheaply, make them unreliable
备注: 35 pages, 9 figures, 56 tables
点击查看摘要
Abstract:Attention maps cheaply show where a vision-language model (VLM) looks, but attention sinks make them unreliable for localization. Contrasting the question with a generic prompt suppresses sinks, yet only some layers localize the queried object. Existing methods find these layers with box annotations or fixed rules. We show that attention itself can identify them. A question changes how much a head attends to the image and where this attention goes. The two effects separate exactly, and the second, measured in bits on unlabeled questions, indicates which layers localize. WhereBits turns these where-to-look bits into a layer prior and combines it with per-head contrastive scores. Its three design constants were fixed once on labeled COCO data, and a new model or resolution needs only unlabeled questions. Across eight VLMs, WhereBits matches the pointing accuracy of box-tuned layers on COCO and RefCOCO and ranks target tokens better. This advantage holds on a held-out model family and at higher resolutions. Its COCO pointing accuracy is 0.72, against 0.22 for mean attention. In training-free zoom-in, its crops raise V* accuracy by 6.0 points under forced-choice evaluation.
211. 【2611.23417】Autoregressive View Model
链接:https://arxiv.org/abs/2611.23417
作者:Jensen(Jinghao)Zhou,Samuel Rota Bulo,Denis Rozumny,Duncan Zauss,Peter Hedman,Nikhil Keetha,Philip Torr,Christian Rupprecht,Andrea Vedaldi,Peter Kontschieder
类目:Computer Vision and Pattern Recognition (cs.CV)
关键词:Autoregressive View Model, present the Autoregressive, Autoregressive View, constructing an explicit, AVM
备注: Project Page: [this https URL](https://autoregressive-view-model.github.io/)
点击查看摘要
Abstract:We present the Autoregressive View Model (AVM), which generates a scene as a stream of views without constructing an explicit 3D representation. Through a unified in-context interface, AVM conditions on images or videos to generate short clips along a requested camera path. Visual context grounds reconstruction and interpolation, while a learned prior enables extrapolation into unseen regions. AVM retrieves relevant views from inputs and generated history to form a bounded active context. We fine-tune a pretrained bidirectional video diffusion transformer with flow matching in an N-in-1-out formulation. We then tune a causal model on its own rollouts to improve photorealism, reduce denoising steps, and limit long-horizon drift. We scale training to approximately 8M scenes and video clips. We evaluate generation capacity and scene persistence across benchmarks for camera control, text control, visual quality, dynamics, sparse and 4D reconstruction, and long-horizon consistency. The results demonstrate that AVM adapts to diverse input and output settings through one in-context interface.
212. 【2611.23416】OCLDA-GS: Object-Centric Local Deformation-Aware Gaussian Streaming
链接:https://arxiv.org/abs/2611.23416
作者:Ashim Dahal,Nikesh Subedi,Rabab Abdelfattah,Nick Rahimi
类目:Computer Vision and Pattern Recognition (cs.CV)
关键词:Free Viewpoint Video, Free Viewpoint, Gaussian Splatting, Viewpoint Video, online reconstruction
备注:
点击查看摘要
Abstract:Streaming 3D Gaussian Splatting (3DGS) allows for online reconstruction of dynamic scenes from multi-view videos for Free Viewpoint Video (FVV). Current time and storage efficient methods rely on selecting a subset of anchor Gaussians and neighborhood interpolation for fast streamable 3DGS. However, these methods can overlook object-specific motion and introduce inconsistencies near object boundaries. We propose OCLDA-GS, an object-centric streaming framework that dynamically reallocates object and motion-aware Gaussian anchors to allocate anchor Gaussian support for objects undergoing large motion while at the same time, restricting the gaussian neighborhood within object groups for more precise object boundaries. Furthermore, it estimates object-wise rigid transformations (e.g. a hand moving) and leverages these estimates to learn per-anchor local deformation correction for small deformations (e.g. wrinkles on a shirt). Then, we conduct experiments on three dynamic multi-view video datasets and demonstrate improvements over the previous baselines alongside improved quality between refinement timesteps. Under matched refinement budgets, our method improves PSNR over Instant Gaussian Stream (IGS) by 0.169 dB on N3DV, 0.202 dB on Meeting Room and +0.199 on Google Immersive, while reducing the mean PSNR drop between refinement timesteps by 9.2% and 13.8% in N3DV and Meeting Room, respectively.
213. 【2611.23410】RFBench: A Simulator-Grounded Multimodal Benchmark for Passive RF Layout Design and Understanding
链接:https://arxiv.org/abs/2611.23410
作者:Jiarui Cui,Boxun Yan
类目:Artificial Intelligence (cs.AI); Computer Vision and Pattern Recognition (cs.CV); Machine Learning (cs.LG)
关键词:generate structured outputs, remains largely untested, electromagnetic behavior remains, behavior remains largely, connect passive radio-frequency
备注: 29 pages, 4 figures. Code: [this https URL](https://github.com/jerry66s6/RFBench) . Data: [this https URL](https://huggingface.co/datasets/Jerry1030/RFBench)
点击查看摘要
Abstract:While multimodal language models can interpret diagrams and generate structured outputs, their capacity to connect passive radio-frequency (RF) design intent, layout geometry, and electromagnetic behavior remains largely untested. To address this gap, we introduce RFBench, a simulator-grounded dataset and benchmark for passive RF layout design and understanding. The dataset contains 10,000 two-port planar layouts simulated using ADS Momentum, with each layout represented by editable geometric primitives, a raster image, a 16x16 occupancy grid, complex S-parameters at 201 frequency points, a structural category, and an RF response label. Building on these aligned representations, we define seven tasks spanning specification-conditioned layout generation, representation translation, response prediction, simulator-based validation, design diagnosis and improvement, structural understanding, and response classification. Extensive baselines across seven multimodal language models reveal systematic weaknesses in generating structurally valid layouts and predicting electromagnetically faithful responses. We show that RF-specific fine-tuning increases T1a structural validity from 0.0% to 83.2% and the T3 primitive-input within-2-dB rate from 17.7% to 62.8%, demonstrating the value of domain adaptation while leaving considerable room for further progress. RFBench establishes a common testbed for evaluating multimodal RF capabilities and guiding the development of models that connect design intent and layout representations with physical behavior.
214. 【2611.23395】Lossless Reversible Watermarking for AIGC with Applications in Authentication
链接:https://arxiv.org/abs/2611.23395
作者:Yupeng Qiu,Han Fang,Ee-Chien Chang
类目:Cryptography and Security (cs.CR); Computer Vision and Pattern Recognition (cs.CV)
关键词:tampering detection, authentication, AIGC watermarking, support applications, AIGC
备注:
点击查看摘要
Abstract:AIGC watermarking is commonly used for tracing, but it can also support applications such as authentication and tampering detection. We study lossless reversible watermarking, which enables exact recovery of the original watermarked image after permissible distortions. By exploiting the relatively low entropy of the information required for image generation, together with robust latent-space watermarking, we design a framework that preserves sufficient information for lossless recovery. We then apply the framework to AIGC authentication, which generally faces two key challenges. First, the authentication mechanism must resist adaptive adversaries. Second, the content relevant to authentication, such as visual designs or expressed messages, may depend on application requirements that are not known at generation time. To solve these difficulties, we incorporate cryptographic message authentication to provide unforgeability against adaptive attacks. Lossless recovery provides the original image as reference evidence, after which a suitable analyzer can be selected to make the authentication decision. Experiments on Stable Diffusion demonstrate exact recovery under a wide range of distortions.
215. 【2611.23350】Asymmetric Optical Image Watermarking
链接:https://arxiv.org/abs/2611.23350
作者:Guang Yang
类目:Cryptography and Security (cs.CR); Computer Vision and Pattern Recognition (cs.CV)
关键词:Watermark detectors held, Watermark detectors, detectors held, two-layer image watermark, image watermark
备注: 10 pages, 4 figures, 5 tables
点击查看摘要
Abstract:Watermark detectors held by a provider cannot be checked by third parties, and a party that can detect a mark can usually also forge it. We study a two-layer image watermark in which the Fourier transform at the core of the pipeline is performed by a coherent optical 4F unit. Layer 1 carries an Ed25519 signature bound to the image content in Fourier magnitudes, so anyone with the public key verifies offline. Layer 2 is a key-defined carrier written by a secret phase hologram and read by a matched-filter correlator. The optics is a scalar-diffraction simulation with finite modulator bits, pixel aperture, pupil, defocus, misalignment, and sensor noise. No hardware is built. All criteria were fixed in a hashed specification before any optical result. The simulated optical signer reproduces the digital pipeline (field error 1.7e-15, bit agreement 99.997%), verifies at least 99.5% of 877 images under each of 8 post-sensor settings, one unattacked, and accepts none of 40,670 unmarked images. The optical carrier is key-selective: the matched key detects it on 99.8% of the unattacked images, and detection falls to 0% under a scale error of 1% and to 0% under a mask shift of 1 bin. Security rests on the signature and the content binding, for which we separate what is provable from what is measured.
216. 【2611.23273】orchtest2: One-Call Sanity Checks for PyTorch Models,Evaluated on Real and Unseen Architectures
链接:https://arxiv.org/abs/2611.23273
作者:Grzegorz Gwardys
类目:oftware Engineering (cs.SE); Computer Vision and Pattern Recognition (cs.CV)
关键词:deep learning code, recurrent layer, layer that treats, head was replaced, attention mask
备注:
点击查看摘要
Abstract:Many bugs in deep learning code are silent: a layer that is never called, an optimizer created before the classification head was replaced, a recurrent layer that treats the batch as time, or an ignored attention mask. The code runs, the loss decreases somewhat, and the defect surfaces, if ever, after days of GPU time. We present torchtest2, an open-source library that runs 18 sanity checks on a single batch with one call, restores the PyTorch state it touches, and adapts to model families through plugins. We evaluate it by mutation analysis with 17 classes of silent bugs on 103 models from torchvision, timm and Hugging Face transformers, on 600 random architectures and on 20 hand-written realistic bugs, against the checks of the original torchtest, overfitting a single batch, a smoke test and a training monitor in the style of AUTOTRAINER and DeepDiagnosis. Every mutant is compared with its healthy counterpart, and intervals come from a bootstrap over models. After freezing the tool, we ran it on 191 models of architecture families it had never seen: counting, for every tool alike, a failing report where the healthy model's report does not fail, it detected 78% of the injected bugs (95% CI 75-80), against 40% for the original torchtest checks and 48% for the training monitor. On real models its batch-independence check found a bug in timm's GCViT that couples the images of a batch. Real models also exposed false alarms that neither the tool's own test suite nor random architectures had revealed. Two independent collections of real silent bugs mark the limits of the approach: only 3 of 28 reproduced PyTorch bugs from Stack Overflow and 3 of the 18 silent training errors released with TrainCheck are within reach of a pre-training model tester. A median check takes 1.0 s. We release the library, the benchmark and all raw results.
217. 【2611.22724】ConserveEdit: Source-Anchored Transactions for Drift-Bounded Iterative Image Editing
链接:https://arxiv.org/abs/2611.22724
作者:Shuyang Jiang,Fucheng Deng,Yuchuan Luo,Zhenyu Wu
类目:Computer Vision and Pattern Recognition (cs.CV); Machine Learning (cs.LG)
关键词:produce convincing local, iterative refinement loop, Instruction-driven image editors, requested region accumulate, editors produce convincing
备注: 28 pages, 15 figures, 16 tables; includes supplementary material
点击查看摘要
Abstract:Instruction-driven image editors produce convincing local edits in one pass, but inside an iterative refinement loop every accepted output becomes the next input, so unintended changes outside the requested region accumulate. We present ConserveEdit, an editor-agnostic controller that treats each refinement step as a transaction against a contract compiled once from the source image. The contract separates what may change (the authorized region of each target, with a bounded halo for shadows and reflections) from what must survive (protected content with explicit tolerances). Candidates from a frozen editor are composed through a source-anchored mask and committed only if target progress, drift on protected content, and image quality pass as separate tests; otherwise the step is rolled back and repaired. For any editor, this keeps every pixel outside the authorized region bit-exact and bounds drift independently of the number of iterations, whereas no fixed scalar trade-off reproduces the separate tests. On all 1053 MagicBrush test edits with a frozen Qwen-Image-Edit, masking is necessary but not sufficient. A direct call already alters the rest of the image (12.1 dB PSNR outside the mask); composed through the source-anchored mask, it stays bit-exact there and its DINO similarity to the ground truth rises from 0.67 to 0.85, matching ConserveEdit, so this gain comes from the mask. Next to automatic masks, however, masked direct editing changes a protected face or text on 48% of turns, and masked scalar selection avoids this mostly by abstaining (23% constrained success). Because its drift test names the violated entity, ConserveEdit shrinks the support instead (2% violations, 53% constrained success); in a simulator with known ground truth it raises constrained success from 55.0% to 61.8%. Automatic grounding remains the main bottleneck. Code and logs will be released.
218. 【2611.24776】4D-TEMPO: Differentiable 4D-STEM ptychography via fast Physics Optimizers
链接:https://arxiv.org/abs/2611.24776
作者:Ramani Duraiswami,Yichao Zhang
类目:Instrumentation and Detectors (physics.ins-det); Materials Science (cond-mat.mtrl-sci); Computer Vision and Pattern Recognition (cs.CV); Mathematical Physics (math-ph); Computational Physics (physics.comp-ph)
关键词:ptychography recovers atomic, recovers atomic structure, Electron ptychography recovers, ptychography recovers, recovers atomic
备注: 34 pages: 8-page paper, references, and 23-page supplementary material; 10 figures, 10 tables
点击查看摘要
Abstract:Electron ptychography recovers atomic structure from 4D-STEM data: diffraction patterns at multiple positions of a scanned probe. Current codes reconstruct an image and read atoms off it, or fit atoms under an $L_1$ loss; neither gives per-atom uncertainty. We propose 4D-TEMPO, a differentiable physical renderer that fits the atoms, the microscope, and the detector directly to the recorded electron counts under a Poisson likelihood. The output is a list of atoms, each a Gaussian primitive with a 3D position, a thermal width, and a model-conditional error bar from the Fisher information. The Fisher information also conditions the solver, gives the dose (electrons per Å$^2$) each parameter needs, and separates calibration from imaging. On a simulated WSe$_2$ bilayer, starting from the published reconstruction, fitted atoms cut in-plane error by a third (3.2 vs. 4.8 pm) and depth error by 40-61% against its 3D peaks. Depth comes from multiple scattering into the dark field, the faint counts outside the direct beam. The two layers, $10\times$ closer than the depth of field, separate; per-atom depth then follows the layer geometry. When the model is well specified, the bars are calibrated (69% of position coordinates within $1\sigma$). On experimental WSe$_2$, fitted atoms predict held-out patterns better than the three published codes we test. With free probe modes where atoms are incomplete, the same likelihood predicts best on all three held-out benchmarks, matches every WSe$_2$ baseline in 5 min against up to 20.5 h, and passes torchslice at its released setting 180$\times$ sooner on PrScO$_3$. Per-atom error bars, planned doses, and fast reconstruction point to low-dose, real-time 3D imaging of beam-sensitive and dynamic materials.
219. 【2611.24363】ransferable Named Sparse Features for Morphology-based Gleason Grading
链接:https://arxiv.org/abs/2611.24363
作者:Argyrios Zafeiriou,Athanasios Kallipolitis,Ilias Maglogiannis
类目:Image and Video Processing (eess.IV); Computer Vision and Pattern Recognition (cs.CV)
关键词:Deep learning systems, Deep learning, learning systems grade, systems grade prostate, grade prostate biopsies
备注:
点击查看摘要
Abstract:Deep learning systems grade prostate biopsies at the level of pathologists, but most report a grade without the glandular architecture that justifies it. This work presents a grading system built on a sparse code over the tokens of a pathology foundation model. A sparse autoencoder pretrained on biopsies is aligned with multi-reader annotations, so that 69 of its 2048 features are named after Gleason morphologies such as cribriform glands, cords, and comedonecrosis. The same code feeds a grading network, which reads all features to predict Gleason patterns and the Grade Group, while morphology maps read directly from the named features. We evaluate the system on five public datasets that include needle biopsies, tissue-microarray cores, multi-reader annotations, and three scanners. The morphology maps remain stable across alignment runs and, without retraining, detect cribriform glands in external biopsies (area under the receiver operating characteristic curve, AUROC, 0.849) and pattern 5 in unseen cores when evaluated by up to six pathologists (AUROC 0.846 to 0.919). In an independent biopsy cohort, Grade Group agreement with three pathologists on Leica scans is close to their agreement with each other. Removing the named features reduces the corresponding Gleason pattern predictions by 49% to 98%, with effects larger than those from 20 activity-matched random feature removals. These results show that a shared sparse representation can support competitive Gleason grading while providing named morphological evidence that transfers across datasets and whose contribution to grading can be measured directly.
220. 【2611.24242】Joint Image Denoising and Interpolation via Mixed Graph Filtering
链接:https://arxiv.org/abs/2611.24242
作者:Gene Cheung,Niruhan Viswarupan,Romisaa Elsaidy
类目:Image and Video Processing (eess.IV); Computer Vision and Pattern Recognition (cs.CV); Signal Processing (eess.SP)
关键词:introduces correlated noise, remove image details, interpolation introduces correlated, complicates subsequent denoising, separately is suboptimal
备注:
点击查看摘要
Abstract:Images often require both denoising and interpolation, yet treating the two tasks separately is suboptimal: interpolation introduces correlated noise that complicates subsequent denoising, while denoising may remove image details useful for interpolation. In this paper, we study joint image denoising and interpolation from a mixed graph filtering perspective, where denoising is modeled using undirected graph filters and interpolation using directed graph filters. We first establish that, under mild conditions, any (pseudo-)linear denoiser and interpolator can be interpreted as solution filters of maximum a posteriori (MAP) estimation problems regularized by undirected and directed graph smoothness priors, respectively. Guided by these results, we formulate graph-based joint denoising and interpolation problems using both directed and undirected graph structures. We characterize when joint denoising/interpolation reduces to a separable pipeline and when a genuinely coupled optimization is required. For the non-separable case, we recover sparse graph operators from given denoisers and interpolators, yielding an efficient linear-system formulation solvable via conjugate gradient. Experimental results demonstrate that our proposed joint framework consistently outperforms separate denoising and interpolation pipelines across multiple denoiser-interpolator combinations.
221. 【2611.23788】Geometry-bridge flow matching for blind rigid-motion correction in head cone-beam CT
链接:https://arxiv.org/abs/2611.23788
作者:Sungho Yun,Seungryong Cho
类目:Medical Physics (physics.med-ph); Computation and Language (cs.CL); Computer Vision and Pattern Recognition (cs.CV)
关键词:accurate anatomical structures, Rigid motion, head cone-beam, measured projections, complicate recovery
备注:
点击查看摘要
Abstract:Rigid motion of a patient head during head cone-beam CT (CBCT) scan can degrade image quality and complicate recovery of accurate anatomical structures from the measured projections. We present a 3D flow-matching prior approach that learns the image changes induced by motion. Its geometry-bridge connects motion-corrupted and motion-free Feldkamp-Davis-Kress (FDK) reconstructed images of the same source volume through progressively reduced simulated motion. The forward projector's geometry derivative supplies analytic velocity targets along this path. A plug-and-play loop encompasses the learned image updates together with spline-based pose estimation, conjugate-gradient data consistency, and image total-variation regularization. Both the refined image volume and the FDK reconstructed image volume at the estimated poses are evaluated. On simulated 360-view scans from 30 held-out CQ500 patients with nominal per-component motion amplitudes within 10 mm / 10 degrees, the framework achieved state-of-the-art image reconstruction quality and motion accuracy in comparison with other methos such as learned autofocus and diffusion-based joint reconstruction. Mean peak signal-to-noise ratio (PSNR) reached 37.1 dB and mean detector-plane reprojection error (RPE) was 0.27 mm. With matched endpoints at the inference settings, the geometry bridge improved PSNR by 1.6 dB and reduced RPE by 33% relative to image-linear training. With 50 prior updates and coarse-to-fine motion estimation, total processing time averaged 9.1 minutes per patient on an RTX A6000.
222. 【2611.23444】Gaussian Splatoms: Direct Atomic Structure Identification in Electron Tomography
链接:https://arxiv.org/abs/2611.23444
作者:Nalini Singh,Tiffany Chien,Cedric Lim,Arthur R. C. McCray,Colin Ophus,Laura Waller
类目:Image and Video Processing (eess.IV); Computer Vision and Pattern Recognition (cs.CV)
关键词:projection measurements acquired, modeling AET samples, computationally reconstructs, tilt angles, structure of materials
备注: 26 pages, 14 figures
点击查看摘要
Abstract:Atomic electron tomography (AET) computationally reconstructs the 3D atomic structure of materials from a series of 2D projection measurements acquired at different tilt angles. Inspired by Gaussian splatting techniques from computer vision, we introduce an atom-centric parameterization for modeling AET samples. Instead of a conventional volumetric representation with a grid of voxels, we model the sample as a list of atoms, where each atom is a 3D Gaussian characterized by learned location, scale, and amplitude. The Gaussian model is a natural fit for atoms that allows us to easily incorporate prior information grounded in physics; we impose isotropy of atoms, a minimum interatomic distance, and realistic atomic radii. We show in simulation that our method is more robust than volumetric baselines to real-world imaging challenges, including a limited number of measurements and low electron dose. We further show that our method yields a more physically plausible reconstruction of an experimental 23,000 atom iron-platinum nanoparticle than a previously published volumetric reconstruction. Beyond tomography, our representation is a promising tool for any atomic resolution imaging modality (e.g. multislice electron ptychography).
223. 【2611.23419】LIC-Audit: How Comparable Are Reported Gains in Learned Image Compression?
链接:https://arxiv.org/abs/2611.23419
作者:Haisheng Fu,Jie Liang,Runyu Yang,Feng Ding,Xin Dong,Zixiao Jin,Jingning Han
类目:Image and Video Processing (eess.IV); Computer Vision and Pattern Recognition (cs.CV); Applications (stat.AP)
关键词:Learned image compression, report BD-rate improvements, Learned image, Learned, VTM reference
备注: 31 Pages, 4 figures
点击查看摘要
Abstract:Learned image compression (LIC) papers often report BD-rate improvements of only a few percent. They are measured against VTM reference software whose version keeps changing: the claims we audit name 7 versions, 5 in 2025 papers alone. But how comparable are these numbers across papers, and what evidence should future papers give for such small gains? We introduce LIC-Audit, an analysis of 353 released rate-distortion curves from 36 repositories, and a controlled re-evaluation of eight public codec releases with 28,193 encode-decode records. We identify three problems. First, different VTM reference runs change BD-rates by up to 3.56 percentage points, and on CLIC Pro published numbers order 10 of 55 method pairs differently from a common computation; comparing codecs directly over a fixed quality range removes this dependence. Second, releases use different and mostly undocumented conventions for counting bits, computing PSNR, and padding images. Under consistent measurement, the GLIC method's 1.00% lead over the HPCM method on Tecnick becomes a 1.43% deficit, changing only the padding reverses the comparison of HPCM and CMIC, and most gaps narrow when MS-SSIM replaces PSNR. Third, average curves hide how much results vary from image to image, so whether a small gain is statistically significant depends on the assumed variation. An optimistic assumption calls 12.0% of comparisons between equally good codecs significant on test sets as small as Kodak, instead of 5%; a cautious one leaves 18 of 45 differences between recent codecs undecided, yet per-image results show 12 of them to be significant. Beyond auditing past results, LIC-Audit provides a versioned evaluation protocol, a public leaderboard website with a scoring tool, source-linked benchmark data, per-image measurements, and reporting guidelines, to help future LIC papers make comparable and verifiable claims of progress.
224. 【2611.23352】Streaming Generative Video Compression with Embedding Alignment and In-Context Propagation
链接:https://arxiv.org/abs/2611.23352
作者:Zhitao Wang,Hengyu Man,Han Liu,Wenrui Li,Debin Zhao,Xiaopeng Fan
类目:Image and Video Processing (eess.IV); Computer Vision and Pattern Recognition (cs.CV); Multimedia (cs.MM)
关键词:preserving perceptual quality, ultra-low bitrates presents, bitrates presents substantial, presents substantial challenges, generative video compression
备注: 15 pages, 18 figures, 5 tables, including supplementary material
点击查看摘要
Abstract:Video compression at ultra-low bitrates presents substantial challenges in preserving perceptual quality. Although pretrained video diffusion transformers (DiTs) provide powerful generative priors, existing generative video compression (GVC) methods built on them typically rely on computationally expensive VAE decoders to reconstruct video frames. Moreover, jointly refining multiple frames to exploit the inherent spatiotemporal modeling capabilities of video DiTs introduces additional buffering delay. To address these challenges, we propose SGVC, a streaming generative video compression framework that adapts a pretrained latent video DiT into a causal pixel-space refiner. Specifically, SGVC projects decoded features from a contextual pixel codec directly into the DiT input embedding space. To bridge the representation gap between codec-derived embeddings and pretrained DiT embeddings, we introduce Embedding Alignment, which aligns the projected embeddings with input embeddings extracted from the original frames during training. The refined DiT hidden states are then mapped directly to the pixel domain through a lightweight reconstruction path, bypassing the pretrained VAE decoder. To further exploit the spatiotemporal priors of the video DiT while maintaining causal decoding, we introduce In-Context Propagation, which reuses layer-wise K/V states across frames to exploit historical generative context, improving temporal consistency without requiring clip-wise buffering. Extensive experiments demonstrate that SGVC achieves competitive rate-perception performance, with average BD-rate savings of 71.97% over S$^2$VC in terms of DISTS. Project Page: this https URL

