Large Language Models, Spring 2026

ETH Zürich: Course catalog

Course Description

Large language models have become one of the most commonly deployed NLP inventions. In the past half-decade, their integration into core natural language processing tools has dramatically increased the performance of such tools, and they have entered the public discourse surrounding artificial intelligence. In this course, we start with the probabilistic foundations of language models, i.e., covering what constitutes a language model from a formal, theoretical perspective. We then discuss how to construct and curate training corpora, and introduce many of the neural-network architectures often used to instantiate language models at scale. The course discusses privacy and harms, as well as applications of language models in NLP and beyond.

Pre-requisites: While there are no formal pre-requisites for taking the course, we count on you being comfortable with probability theory, linear algebra, computational complexity, and machine learning.

Syllabus and Schedule

On the Use of Class Time

Lectures

There are two lecture slots for LLMs each week (3 hours total):

• Tuesdays 14:15-16:00 in HG E 3
• Fridays 10:15-11:00 in CAB G 61

In-person and Zoom

Lectures will be given in person and live broadcast on Zoom; the password is available on the course Moodle page.

Recordings: Lectures will be recorded—links to the Zoom recordings will be posted on the course Moodle page.

Tutorials

Tutorials will take place Thursdays 16-18 in NO C 60 and on Zoom.

Syllabus

Tutorial Schedule

Organization

Moodle as a Communications and Questions-answering Platform

We will use the course Moodle page for course communications and as a place where you can ask questions to the teaching staff. There are several forums you can use to ask specific questions and we encourage you to take advantage of that. We aim to response quickly.

Course Notes

We prepared an extensive set of course notes for the course. We will be improving them as we go this semester as well. Please report all errata you find in the course notes to the teaching staff in the Errata Google document linked on the course Moodle page.

Links to the course notes:

• LLM Course Notes Part 1
• LLM Course Notes Part 2 (up to date Overleaf link)
• LLM Course Notes Part 3

Other useful literature:

• ESSLLI 2023 Tutorial on the Expressivity of Neural Networks
• ESSLLI 2024 Tutorial on the Expressivity of Transformers
• Introduction to Natural Language Processing (Eisenstein)
• Deep Learning (Goodfellow, Bengio and Courville)
• AFLT Course Notes

Grading

Marks for the course will be determined by the following formula:

• 50% Final Exam
• 50% Assignments

Exam

The final exam is comprehensive and should be assumed to cover all the material in the slides and class notes. The date is determined by the ETH examinations office centrally and will be announced towards the end of the semester.

Remote exams: ETH offers a centralized system for taking exams remotely if you are an exchange student or under specific circumstances for ETH students as well. To find out more and arrange a remote exam, please follow the instructions on remote examinations here.

Exam review: After the grades have been announced, you will be able to sign up to the exam review session, which we will offer sometime in the first three weeks of the semester. During the session, you will have the opportunity to review your exam and assignments and understand how they were graded. You will also be able to take notes about the exam and solution, but no copies or photos can be taken. To sign up, we will publish a Google form after the grading conference. Note that we offer only one review session, so individual (or remote) sessions are not possible. See also here for more information about exam reviews in general.

Assignments

There will be three larger assignments in the course. Assignments are individual work, and you are expected to submit your own solutions—solutions that you wrote up yourself and did not copy from any of your peers. Each assignment might, however, follow a different policy on collaboration when it comes to discussing the problems with your peers—please refer to the specific assignment instructions for details.

We require the solutions to be properly typeset. We recommend using LaTeX (with Overleaf), but markdown files with something like MathJax for the mathematical expressions are also fine.

The first assignment will be of more theoretical nature and will be released shortly after the start of the semester. Assignments 2 and 3 will be of more practical nature and will be released in the second half of the semester.

Each of the three assignments contribute 1/3 to the final assignment grade (that is, the assignment grade will be the average of the three individual assignment grades; see the individual assignment instructions for the grading scales).

Assignment instructions:

• Assignment 1 Instructions: TBD
• Assignment 2 Instructions: TBD
• Assignment 3 Instructions: TBD

Assignment Deadlines

You will submit your assignments via Moodle.

• Assignment 1 is due on TBD.
• Assignment 2 is due on TBD.
• Assignment 3 is due on TBD.

Please be proactive with your time management and start early. Barring exceptional circumstances that do not only affect the last two weeks before the deadline (e.g., prolonged illness, family emergency, or severe mistakes in the assignment setup), we will not accept requests for deadline extensions—neither individual nor group requests. Late submissions will not be graded.