Prépublication
· 2026 arXiv
EN
Autoregressive language models produce, at every prediction step, a vocabulary-sized logit vector whose structure determines the conditional next-token distribution. We empirically identify that the ordered vocabulary-head logits follow a remarkably consistent...
Swathi Shree Narashiman, Aravind Ramana Venkatramanan et al.
#cs.CL
Prépublication
· 2026 arXiv
EN
Knowledge distillation trains student models on teacher-generated responses to transfer task capability, but those responses may also carry behavioral tendencies of their source. We investigate whether the source teacher can shape student behavior beyond the c...
Yu-Chien Tang, An-Zi Yen
#cs.CL
Prépublication
· 2026 arXiv
EN
Clinical trial protocols serve as the source of truth for study design, but are mostly written and used as unstructured PDF documents. Digitizing protocols turns static content into structured, machine-readable data that clinical software can analyze, validate...
Jiacheng Lin, Geoffrey Low et al.
#cs.CL
Prépublication
· 2026 arXiv
EN
Autoformalization (AF) requires more than compilation: formal statements must preserve the meaning of their informal sources, a property known as faithfulness. An emerging frontier is the automated evaluation of this property, yet most existing methods operate...
Yuming Feng, Tadd Mao et al.
#cs.CL
Prépublication
· 2026 arXiv
EN
Recursive self-improvement (RSI) aims to transform interaction experience into persistent improvements, yet external feedback is costly and world-model predictions may become unreliable as policies evolve. We introduce RSI-Anything, a domain-general framework...
Taoyong Cui, Pinxian Zeng et al.
#cs.CL
Prépublication
· 2026 arXiv
EN
Vision-Language Models (VLMs) increasingly analyze and reason about images, yet it remains unclear whether their internal representations faithfully encode the structured relations required for such reasoning. We introduce EICS (Evaluation-Invariant Concept St...
Shubh Garg, Chen Shani
#cs.CL
Prépublication
· 2026 arXiv
EN
Neural semantic parsers generate meaning representations token by token, yet well-formedness depends on references, scope, and graph relations that unfold during generation. Constrained decoding prevents locally invalid tokens but does not guarantee that a val...
Xiao Zhang
#cs.CL
Prépublication
· 2026 arXiv
EN
Large language models reuse the same attention heads across tasks, but identifying these shared heads does not explain what they do differently in each task. A natural source of inspiration is task-dependent interactions among shared brain regions, which motiv...
Sovesh Mohapatra, Dani S Bassett
#cs.CL
Prépublication
· 2026 arXiv
EN
A useful language model should know when to answer and when to say "I don't know." Can we improve that decision by training the model itself or by training a separate component to check its answers? We study both approaches on Qwen3-14B for factual questions,...
Juan Terven, Diana-Margarita Córdova-Esparza et al.
#cs.CL
Prépublication
· 2026 arXiv
EN
LLM developers post-train their models to exhibit prosocial values and behavioral traits, which are enumerated in an alignment target. However, while recent post-training developments have yielded models that score highly on alignment evaluations, training mod...
Andy Liu, Mehar Bhatia et al.
#cs.CL
Prépublication
· 2026 arXiv
EN
Conditional memory architectures such as DeepSeek Engram use input n-grams to look up learned embeddings, expanding the capacity of large language models (LLMs) with limited additional computation. Beyond model scaling, this architecture has demonstrated the p...
Hongru Cai, Ran Wei et al.
#cs.CL
Prépublication
· 2026 arXiv
EN
Prompted embedding models have recently received increasing attention, particularly for retrieval, where detailed retrieval instructions are provided as part of the retrieval prompt. Several new datasets and studies have examined this setting, showing that the...
Amanda Myntti, Jenna Kanerva et al.
#cs.CL