October 28th @
Invited Speakers
Talk Title
Language Learning through Social and Embodied Experience
Abstract
Human language learning is shaped by social and embodied experience. Through social interaction, we acquire the use of language from communicative context, and through sensorimotor experience with the physical world, we learn words that are grounded to perception and action. In this talk, I will discuss several studies in my lab that explore social and embodied context for language learning and grounding. I will introduce an interactive learning paradigm and show how demonstrations and corrective feedback from teachers and active trials from students can facilitate efficient word learning in language models.
About the Speaker
Joyce Chai is a Professor in the Department of Electrical Engineering and Computer Science at the University of Michigan. Her research interests span from natural language processing and embodied AI to human-AI collaboration. Her current work explores the intersection between language, vision, and action to enable situated communication with embodied agents. She is a recipient of the NSF Career Award and multiple awards with her students (e.g., Best Long Paper Award at ACL 2010, Outstanding Paper Awards at EMNLP 2021 and ACL 2023, and First Place in the Amazon Alexa AI Simbot Challenge 2023). She is a Fellow of ACL.
Talk Title
Learning to interact, interacting to learn
Abstract
As children learn to coordinate turns, produce contingent responses, and use multimodal behavior, they become increasingly active participants in linguistic interaction. Their participation changes the evidence available to them. It elicits adapted input, feedback, alignment, and repair, which create new opportunities for learning. Interaction therefore provides a context in which different learning signals arise and become informative. I present empirical and computational studies of this relationship. I use large-scale analyses of naturalistic child–caregiver interaction to examine the development of turn-taking, response contingency, and multimodal coordination. I then present work using child-like LMs and reinforcement learning to test whether and how different forms of naturally occurring feedback contribute to language learning. This developmental perspective argues that learning to interact and acquiring functional linguistic knowledge are coupled objectives. It has implications for machine language learning, particularly for how models can ground their acquired knowledge in situated language use and shared goals.
About the Speaker
Abdellah Fourtassi is an Associate Professor of Computer Science at Aix-Marseille University and a researcher at the Institute of Language, Communication and the Brain. His team studies how multimodal interaction shapes language learning. His research draws primarily on machine learning theory and methods to study human language development.
Talk Title
Data efficiency in children and language models
Abstract
Large language models show intriguing emergent behaviors, yet they receive at least three to four – and sometimes as much as six – orders of magnitude more language data than human children. What accounts for this vast difference in sample efficiency? I will describe steps towards a paradigm in which we can address this question. In particular, I’ll discuss the use of egocentric video (“baby headcam”) data for model training, and the use of developmental data for model evaluation. This paradigm provides a model-based framework for exploring the nature of children’s early data efficiency.
About the Speaker
Michael C. Frank is Benjamin Scott Crocker Professor of Human Biology in the Department of Psychology at Stanford University and Director of the Symbolic Systems Program. He received his PhD from MIT in Brain and Cognitive Sciences in 2010. He studies children’s language learning and development, with a focus on the use of large-scale datasets to understand the variability and consistency of learning across cultures. He is a founder of the ManyBabies Consortium, and has led open-data projects including Wordbank and the ongoing LEVANTE project. He has received awards including the Troland Award from the National Academy of Sciences and the FABBS Early Career Impact award. He served as President of the Cognitive Science Society, has edited for journals including Cognition and Child Development, and is current co-Editor in Chief of the Open Encyclopedia of Cognitive Science.
Talk Title
On causal & social world models in language models
Abstract
Recent works have suggested that “world models” emerge in large language models as a side effect of the compression necessary for their high performance on the language modeling task. While robust evidence for such world models exists for relatively simple (e.g., finite-state) problems, it remains unclear whether we should expect LMs to also evolve (veridical / human-like) world models for human-relevant but highly abstract latent variables such as causal information, discourse states, or social cognition. I discuss in-principle considerations based on an augmented optimal learner approach that also takes architectural constraints of the learning system into account. This architecturally-constrained efficiency approach predicts that abstract function-sharing can be a reason for emergent representations of the causal data-generating process. To explore these predictions, I discuss two case studies, one on function-sharing between Theory of Mind reasoning and pragmatic reasoning in large language models, and one on the emergence of generalizable Bayesian inference of common causes in small transformers.
About the Speaker
Michael Franke is a computational cognitive scientist, appointed as Professor of Pragmatics and Linguistics and coopted as a Professor of Computer Science at the University of Tübingen and head of the Cognitive Science & Pragmatics Lab. He is interested broadly in reasoning and language use in humans and machines. His work uses statistical, cognitive and evolutionary modeling, human-experimental methods, and simulation experiments. He is also interested in foundational questions concerning the relation between empirical data, abstract theory and explicit mathematical modeling. Currently, he is the coordinator of the DFG-funded Priority Area LaSTing and a co-speaker of the DFG-funded CRC Common Ground.
Schedule
| Start | End | Event |
|---|---|---|
| 09:00 | 09:15 | Welcome & introduction |
| 09:15 | 10:00 | Invited talk: Michael Franke |
| 10:00 | 10:30 | Flash talks for non-challenge papers (2min) |
| 10:30 | 11:00 | Coffee break |
| 11:00 | 11:45 | Invited talk: Abdellah Fourtassi |
| 11:45 | 12:00 | Flash talks for shared task submissions (2min) |
| 12:00 | 12:15 | Oral presentation for the Top 2 shared task submissions |
| 12:15 | 12:30 | Oral presentation for the Top 1 shared task submissions |
| 12:30 | 14:00 | Lunch |
| 14:00 | 14:45 | Invited talk: Joyce Chai |
| 14:45 | 15:30 | Poster session |
| 15:30 | 16:00 | Coffee break |
| 16:00 | 16:45 | Invited talk: Michael Frank |
| 16:45 | 17:30 | Panel discussion |
A Playschool for LLMs
The LM Playschool Workshop aims to bring together researchers interested in language learning from social interaction in conversational, collaborative, task-oriented, multi-turn environments. Relevant research strands include: (1) language games as a means of evaluating large language models and their ability to use language in interaction; (2) connections between human language acquisition and machine language learning, with a particular focus on the role of social interaction in data-efficient learning; (3) game-based and interactive environments for training large language models.
Shared Task
The workshop will host a shared task, the LM Playschool Challenge. The shared task focuses on training collaborative, goal-oriented language agents through language interaction data. Submitted systems will be evaluated for their ability to generalise to unseen “dialogue games”, i.e., goal-oriented tasks governed by multi-turn language interaction, as well as on traditional single-turn NLP benchmarks and cognitive tests. More details here.
Organizers
With invaluable contribution from:
Supported by:
