The Logic and Algebra Underlying Language Models

ESSLLI 2026

Course Description

Syllabus

Day 1: Preliminaries

Day 2

Day 3

Day 4

Day 5

Course Notes

Useful Literature

• Pascal Bergsträßer, Ryan Cotterell, and Anthony Widjaja Lin. Transformers are inherently succinct. In Proceedings of the Fourteenth International Conference on Learning Representations (ICLR), 2026.
• Chu-Cheng Lin, Aaron Jaech, Xin Luo, Matthew R. Gormley, and Jason Eisner. Limitations of autoregressive models and their alternatives. In Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL-HLT), pages 5147–5173, 2021.
• Andy Yang, David Chiang, and Dana Angluin. Masked hard-attention transformers recognize exactly the star-free languages. In The Thirty-eighth Annual Conference on Neural Information Processing Systems, 2024.
• Andy Yang, Anej Svete, Jiaoda Li, Anthony Widjaja Lin, Jonathan Rawski, Ryan Cotterell, and David Chiang. Probability distributions computed by autoregressive transformers. In Proceedings of the Fourteenth International Conference on Learning Representations (ICLR), 2026.