This course covers some of the algebra and logic underlying language models. We begin with algebraic preliminaries and the Krohn–Rhodes decomposition, showing how it characterizes the expressive power of RNNs, SSMs, and transformers. We then connect Krohn–Rhodes to linear temporal logic, and use that logical perspective to analyze the succinctness of transformers.
In recent years, NLP has become a part of our daily lives. Many of us use tools like Google Translate to understand sentences in languages we don’t know, and chatbots like ChatGPT to help draft essays and answer basic questions. However, even though most people recognize the utility of such tools, there are still many questions to be answered about their reliability and their impact on society. For example, to what extent can we or should we trust what ChatGPT says? Should chatbots ever be used in legal decision-making? What is the role that NLP should play in the education system? In this open-ended seminar, we will read and discuss opinions on the proper use of NLP in the real world, or as we term it, NLP in the wild!