Machine Learning and Physics

Machine learning has become a transformational force in our society, and is profoundly impacting society in ways both good and bad. On the good side, machine learning fuels scientific breakthroughs such as solving the protein folding problem, or creating large language models that by now show sparks of artificial general intelligence. On the bad side, convincing deep fakes are used for manipulation, and machine learning supports surveillance in totalitarian states.

Contents

This course takes a two-pronged approach:

Physics of Machine Learning: Highlight physical ideas and concepts that drive ML

Machine Learning for Physics: Equip you with tools to help conduct, and interpret, future experiments

Curriculum

1. Introduction & linear dimension reduction
2. Nonlinear dimension reduction: connection to stat. mechanics; UMAP
3. Nonparametric density estimation (kernel density estimation; random values, expectations)
4. Linear regression
5. Regularized regression: ridge, lasso
6. Cross-validation, double descent
7. Statistical decision theory, classification
8. Parametric & generative methods: QDA. Discriminative: CART
9. Logistic regression, generalized linear models
10. Multi-layer perceptrons
11. Training of neural networks. Batchnorm
12. Auto-encoders & relation to PCA, Geometric Auto-encoder. Parametric UMAP.
13. SGD with momentum. ADAM. Backpropagation.
14. Convolutional neural networks
15. CNNs. Self-supervision, representation learning
16. Attention, transformers
17. Large language models (Letiția Pârcălăbescu)
18. Fine-tuning LLMs (Letiția Pârcălăbescu). Flow-based methods
19. Flow based methods
20. Graph neural networks (GNN). Miracles of biology :-)
21. AI Safety (Lennart Bürger, Erik Jenner)
22. Equivariant ML: symmetries, groups, representations (Peter Lippmann)
23. Spherical tensor product
24. ML in orbital-free density functional theory
25. Ethics of AI (Eva Winkler)
26. Q&A

Where and when

The main lectures are on Tuesdays and Thursdays from 9h00 (NOT 9h15!) until 10h45 in Großer Hörsaal, Philosophenweg 12.

FAQ

Q: Do I need prior knowledge in machine learning?
A: No.

Q: I just want to learn the basics. Is this the right course?
A: Students liked my 2022 rendition, but noted a steep learning curve; if you only want to cover the basics, please check for slower-paced alternatives such as the “Machine Learning Essentials”.

Q: Is this course about deep learning?
A: Neural networks will play an important role; but this course is more about principles. We will not discuss the latest architectures in any detail.

Q: Will this course be repeated next year?
A: Yes, like every MSc core course, though likely by a different professor.

Q: Is there a text book?
A: There is unfortunately not a single compact book covering the entire contents of the course. Most material is covered in the 2000 pages of Murphy’s volumes 1 and 2. I will try and make a script available for at least the bulk of the contents.

Q: Exam modalities?
A: To be admitted to the written exam at the end of the semester, you need to gain 50% of the points in the exercise sheets.

Q: 9AM is really early. Is it okay if I come a bit later?
A: No. I try and introduce the material of the day in a didactic manner at the beginning, and you disturb others by coming late. We start early so that we can finish by 10h45, allowing you to smoothly transit to your next lectures. (And yes, I think 9AM is early, too.)