Paper-only Track
-
Beyond Sally-Anne: Evaluating Theory of Mind in Large Language Models using Epistemic Schelling Points
Text-based evaluations of Theory of Mind (ToM) in Large Language Models (LLMs) often involve cognitive tests akin to the Sally-Anne task that can be gamed due to exposure to relevantly similar tasks in pre-training and do not obviously test models' functional ToM abilities in ways that generalize to naturalistic settings. To address these issues, we introduce the Epistemic (A)symmetry Schelling Task (EAST), a two-player dialogue game designed to benchmark robust and generalizable ToM abilities. By requiring LLM-LLM dyads to independently converge on semantic Schelling points under varying states of epistemic transparency, we evaluate whether models can robustly apply ToM to achieve coordination. Our results reveal a significant capability gap in functional social reasoning, with only frontier models successfully navigating the varying epistemic demands of the tasks. Analysis of reasoning traces shows that coordination failures are primarily driven by epistemic tracking errors, such as conflating private knowledge with mutual knowledge. Despite high performance on traditional static benchmarks, our study shows that robust social reasoning and epistemic tracking remain a critical bottleneck, providing concrete targets for future LLM evaluation and development.
-
Can Faithful CoT Improve Multi-turn LLM Agents? Effectiveness, Boundaries, and Paradoxes
Faithful chain-of-thought (CoT) reasoning is widely regarded as a hallmark of capable language models. Yet its role in multi-turn interactive agents remains largely unexplored. We present the first systematic investigation of action-level CoT faithfulness in multi-turn Large Language Model (LLM) agents, decomposing it into two interpretable and verifiable dimensions: consistency, which measures CoT-action alignment, and determinacy, which captures the degree to which CoT uniquely determines the selected action. We evaluate across Qwen family models of varying scales on Wordle tasks with increasing difficulty. Strikingly, simple inference-time resampling based on CoT faithfulness unlocks non-trivial performance from a zero baseline in 3B models, and yields gains of up to 30 ClemScore points in larger models, all without any additional training. However, our results reveal that faithfulness is not universally beneficial: its effectiveness is bounded by model capability and task difficulty, and jointly enforcing multiple dimensions may introduce competing constraints. These findings suggest that CoT faithfulness should not be viewed as an inherently beneficial property, but rather as a nuanced mechanism whose impact depends on when and how it is utilized in multi-turn agent reasoning.
-
ProcessPrefBench: Evaluating Cross-Task Transfer of Individual Process Preferences from Naturalistic Human–AI Interaction
In natural human–AI dialogue, users' corrections, requests, and self-descriptions carry reusable signals about how they want work to be done. These signals reveal individual process preferences, user-specific standards for evidence, depth, analytical focus, procedure, and delivery. We ask whether a preference revealed in one dialogue can shape an assistant's later answer when the task does not restate it. ProcessPrefBench uses three months of natural dialogues from two knowledge workers to build 66 source–target cases separately judged for source evidence and target applicability. Sources are preselected, so the benchmark tests application rather than retrieval. We test three models with target-only, full-history, and distilled-preference prompts. Target-specific 0–2 rubrics score preference realization, and a task gate flags severe failures. Compared with target-only prompting, full-history and imperative distilled-preference prompting raise preference realization by 0.204 and 0.244 points on the 0–2 scale, respectively, with no reliable advantage from distillation. Fabricated-evidence failures rise from 5.8% under target-only prompting to 10.1% under full-history and 10.4% under imperative prompting. ProcessPrefBench highlights a tension in learning from dialogue, where drawing on prior interactions can improve preference realization yet also invite fabricated evidence. We hope it helps build assistants that honor a user's process preferences without sacrificing factual reliability.
-
Role-Specific Strengths and Alignment in LLM Cross-Play and Self-Play
Dialogue-game benchmarks usually pair a model with itself, and the resulting self-play score is treated as a measure of that model's ability. We test whether self-play scores measure partner-independent ability by comparing them with cross-play evaluations. Across nine Clembench games, we evaluate every possible seating of three popular open-weight models. The cross-play reading exposes additional model strengths and weaknesses that are not detectable in self-play. In four of nine games, a mixed pair of two different models beats the best self-play score; in another four cases, a model does better paired with itself than paired with the strongest available partner. Looking at the transcripts, two mechanisms explain most of this: some pairs succeed because one partner writes more detailed messages that keep the interaction on track, while some self-play scores are inflated because both copies of a model make the same mistake and agree by accident.
-
InterviewArena: Estimating Uninformative Participants through Interactive Dialogue under Incomplete Information
We introduce InterviewArena, an interview-style multi-agent framework for evaluating large language models (LLMs) under incomplete information. By treating missing information as a single factor, we disentangle missing-information identification from dialogue-based reasoning. Experimental results show that while LLMs can capture relative differences between participants, predicting specific missing information remains unstable, highlighting the need to evaluate these capabilities separately.
-
Protocol Competence of Small Language Models in Interactive Language Game Environments
Learning from interaction requires more than producing appropriate responses: learners must maintain a shared interaction with a partner, following its rules while adapting to feedback and contingencies. We argue that this depends on a prerequisite we call protocol competence: the ability to produce valid actions, maintain interaction structure, and respond appropriately across turns. In this paper, we investigate protocol competence in Small Language Models (SLMs) from the Qwen family across five interactive dialogue games from CLEMBENCH, finding 51.3% of episodes from Qwen3.5-0.8B terminate in protocol violations. However, teacher demonstrations from a larger language model can scaffold participation, raising Wordle play from 0/30 to 29/30. Once models can reliably participate, we find self-improving reinforcement learning becomes effective: a single round of self-generated reinforcement learning (self-ReST) yields large gains from a small number of successful trajectories. Comparing adaptive learning strategies, we find that credit-weighted training provides the most reliable improvements, reducing variance relative to uniform supervision. Together, these results suggest that interactive learning follows a staged trajectory: SLMs must first acquire the competence to participate before they can exploit the contingencies generated by interaction.
-
Pareto-Dominant Clarification: Post-Training Coding LLMs via PPO-Lagrangian Budget Constraints
Coding agents operating under ambiguous instructions or user prompts must decide whether to ask clarifying questions or attempt a solution directly. While clarification from the user may improve the correctness of the agent's solution, each back-and-forth interaction incurs user and system costs, forming an explicit accuracy vs. efficiency tradeoff. Existing works study clarification behavior but do not train policies under enforceable clarification budgets; penalty-based approaches typically require separate coefficient tuning swept across all clarification budget levels. We formulate clarification as a Constrained Markov Decision Process (CMDP) and post-train Qwen2.5-Coder-7B-Instruct with PPO-Lagrangian to optimize coding accuracy, subject to an expected question-budget constraint. Evaluated on HumanEvalComm with a GPT-4o-mini oracle simulator, the resulting policies reveal that untuned clarification behavior is Pareto-inefficient: budget-constrained policies can simultaneously achieve higher accuracy and lower clarification rates than the baseline model. Across budget levels, we observe a log-shaped Pareto frontier with diminishing returns to additional clarification. Gains arise not from simply asking more questions overall, but from improved question targeting and better code generation under ambiguity. Without explicit supervision, trained policies learn to allocate clarification budget non-uniformly, asking more frequently on tougher (multi-degradation) tasks. These results suggest that unconstrained interactive LLM systems may systematically use clarification inefficiently.
-
An Empirical Study on Improving an LLMs based Interviewer against Human Interviewees
This paper presents a semi-structured interview system composed of multiple LLM components, in which an LLM-based interviewer conducts dialogue and another LLM estimates interviewee persona attributes from the dialogue history and dynamically updated slot information. The system prioritizes questions related to important yet unconfirmed attributes to support efficient interviews. We evaluate persona attribute estimation performance when interviewing both human participants and LLM-based pseudo interviewees under zero-shot and few-shot prompting conditions. Experiments across three domains (IT engineer, store staff, and school teacher) with a total of 39 interviews show that persona attribute estimation is more challenging for human interviewees than for the LLM pseudo interviewees under zero-shot prompting. However, introducing few-shot prompting consistently improves estimation performance for human interviewees, primarily by increasing recall. On the other hand, its effect on the LLM pseudo interviewees is mixed across domains. A case study confirms that persona attribute estimation improves when illustrative examples are provided.
-
CMHA: A Counterfactual-Memory Hierarchical Architecture for Real-Time Strategy Games
Deploying large language models (LLMs) to real-time strategy (RTS) game environments for validating their reasoning and decision-making capabilities has become a popular research focus. Existing methods typically rely on traditional memory retrieval and unified execution via a single cloud-based LLM, which reveals severe bottlenecks in cognitive reasoning and inference latency. To address these limitations, we propose CMHA, a hierarchical agent architecture with counterfactual memory. At the macro level, CMHA employs a memory mechanism inspired by hypothetical reasoning to maintain long-term strategic consistency. At the micro level, we combine action pruning with a lightweight 2.8B tactical model fine-tuned via QLoRA to translate intents into executable actions. This architecture incurs minimal expensive deployment overhead, while sustaining tactical coherence and reducing inference latency. Experiments in the LLM-PySC2 environment demonstrate that our approach consistently outperforms cloud-based baselines while significantly reducing inference latency and token consumption.
Challenge Track
-
Enhancing LLM Performance via LoRA SFT and DPO for the LM Playschool Challenge
Large Language Models (LLMs) have achieved SOTA results in many natural language processing tasks. However, they are primarily trained for static text generation and often struggle in interactive, goal-oriented environments that require multi-turn reasoning, state tracking, and need to give structured output formats. This paper presents our team's submission to the LM Playschool Workshop Shared Task at EMNLP 2026. The LM Playschool Workshop Shared Task at EMNLP 2026 addresses this challenge by evaluating language models as interactive agents in structured dialogue games. We hereby explored different fine-tuning strategies for Qwen 4B and Qwen 7B models to improve their interactive capabilities and evaluated them. We perform a comparison of Supervised Fine-Tuning (SFT) and Direct Preference Optimization (DPO) using multi-turn training data. Our study analyzes the effectiveness of each optimization strategy for learning robust dialogue behaviors while maintaining models stability towards interactive games and also not to forget its language linguistics by different evaluating metrics. Experimental results show that although supervised fine-tuning provides consistent interactive performance, directly optimizing the foundation model using DPO reduces optimization errors and leads to improved agent behavior. Our best-performing model achieved a clemscore of 38.23 on the official LM Playschool leaderboard, evaluated using the Playpen workbench.
-
Adapting SCoRe to Single-Player Dialogue Games: Initial Experiments on PLAYPEN
Large Language Models (LLMs) are increasingly deployed as autonomous agents that solve tasks through multi-step interactions. However, most post-training methods are designed for single-turn reasoning or instruction-following tasks and provide limited supervision for long-horizon conversational decision making. In this work, we investigate whether Self-Correction via Reinforcement Learning (SCoRe), originally proposed for single-step mathematical reasoning and code generation, can improve agentic performance in multi-turn dialogue games. We adapt SCoRe to the PLAYPEN benchmark by fine-tuning Qwen3.5-9B on five single-player dialogue games using interaction-level rewards and iterative self-correction. Experimental results show that the resulting model substantially improves performance on the trained single-player games and further generalizes to unseen multi-player dialogue games, increasing the overall ClemScore from 41.18 to 43.31. Although a slight decrease is observed on the aggregate StatScore, the model largely retains its general language understanding and reasoning capabilities. These findings demonstrate that self-correction reinforcement learning is an effective approach for improving long-horizon conversational agent behavior beyond single-step reasoning tasks.
-
First Make It Playable, Then Make It Good: Staged Interaction Learning for Small Dialogue-Game Agents
We present Qwen-GuidePlay-2B, a 2B-parameter language model for dialogue-game interaction. We fine-tune Qwen3.5-2B using three steps: a) SFT on only successful game trajectories from Playpen, b) weighted turn-level SFT, and c) teacher-guided SFT. The teacher model (which is a larger model) is only used to fix formatting and evaluate examples, but does not create new gold actions. Our final model scores 57.12 clemscore and 42.68 statscore on the public Playpen validation. In the officially released challenge results, our model obtains the second-highest Playpen clemscore delta among submitted systems (which is ≈+36 over its base model). Our findings suggest that imitating full trajectories helps with playability, while turn-level and teacher-guided training usually improve decision-making and increase the overall score. Alternative procedurally heavy approaches like replay-repair and hard-example mining did not help, which suggests that small models are performant simply by using careful curation strategies rather than aggressive changes. We make available both the model and the code for reproducibility.
-
Raising a Small Language Model: From Imitation to Curiosity-Driven Learning in Dialogue Games
Children do not learn language from text corpora, rather they learn it in interaction by imitating competent speakers, receiving feedback, practicing, being corrected, and playing. The LM Playschool Challenge asks whether language models can profit from analogous regimes. We take the smallest recommended base model, Qwen3.5-2B (clemscore 13.63 in our evaluation environment), and traverse a sequence of five interaction-derived learning regimes that mirror a developmental progression: imitation of competent others (supervised fine-tuning on success-filtered transcripts), learning from outcome contrasts (direct preference optimisation), self-imitation (fine-tuning on the model's own successes), corrective feedback (contrastive pairs from the model's own failures against its own and expert successes), and intrinsically motivated trial-and-error (online GRPO with a novelty bonus). The offline regimes carry the model to 67.6 clemscore on our validation evaluation (+54.0 over the base model). On the organisers' held-out evaluation, our two submitted checkpoints achieve the largest out-of-domain gains of any submission in the challenge (+11.9 and +10.8 clemscore, against a maximum of +6.5 for any other entry), with in-domain gains of +33.2 and +37.3 and static-benchmark performance unchanged. Online GRPO regresses from its initialisation (−5.0) unless augmented with a random-network-distillation novelty bonus, which fully recovers the loss (+5.0 over the plain-GRPO control), evidence that the bonus primarily counteracts the forgetting that narrow online training induces on untrained games.
-
Teaching or Sharpening? An Exploration of the Potential of DPO Post-Training for Dialogue Games
Direct Preference Optimization (DPO) has emerged as a lightweight alternative to RLHF for post-training LLMs, yet it remains unclear whether it can teach novel skills or only sharpen existing ones. We investigate this question in the context of dialogue games, constructing multiple sets of on-policy preference pairs from the rollouts of supervised-fine-tuned Qwen3.5 models (2B and 9B) on the clembench benchmark, targeting rule adherence, strategic behaviour and excessive verbosity. Across all ablations, performance tracks a single variable, namely the distance of the chosen response from the policy's own distribution. Pairs built from the model's own winning moves raise the clemscore from 58.82 to 70.21 while pairs built around GPT-corrected moves land below the baseline at both scales. Transcript analysis points to a structural reason: bare preference pairs reduce conditional rules to unconditional preferences, so that DPO proves able to elicit successful behaviour already latent in the policy, but not to instill behaviour it does not yet exhibit.
-
Acquire, Repair, Preserve: A Diagnosis-Guided Post-Training Recipe for Small-Model Dialogue Game Agents
Interactive dialogue games test a capability that static benchmarks largely leave implicit: a model must carry state across turns, interpret feedback, and choose valid actions under changing constraints. We study this setting in the LM Playschool Challenge with a 2B open-weight model, and find that many failures are not only broad knowledge failures but also local decision failures: repeated guesses, malformed actions, and violations of feedback that the model has just seen. These diagnostics motivate a training recipe organized around three steps: acquire broad game participation through supervised fine-tuning, repair mechanically verifiable failures within one targeted dialogue-game family using turn-local preference pairs, and preserve general capabilities beyond these dialogue games. In the official final evaluation, our submission improves public clemscore from 10.67 to 38.92 and closed in-domain score from 13.41 to 41.17, while approximately preserving aggregate static performance (44.14 vs. 44.24 for the baseline). Out-of-domain clemscore remains low at 7.88, with the largest gains concentrated in unseen variants of the targeted family. Our results suggest that broad SFT brings most of the model's capability improvement; turn-local supervision can be effective when failure detection is precise, with observed transfer concentrated primarily within-family.