Math and Machine Learning: Theory and Applications (Fall 2024)

• Register on the MPI course website.

• Location: Max Planck Insitute for Mathematics in the Sciences, Seminar Room E2 10 (Leon Lichtenstein)
• Organizers: Parvaneh Joharinad, Diaaeldin Taha
• Institutional Website: link
• Contact: To contact the organizers, email the lab at lab [at] mis [dot] mpg [dot] de.
• Mailing List: To stay informed of Lab activities, including this group's meetings, join the Lab mailing list.

Speaker: Diaaeldin Taha (Max Planck Institute for Mathematics in the Science, Germany)

Title: Graph and Topological Neural Networks I & II

Description: In these two sessions, we will provide an overview of deep learning with a focus on graph and topological neural networks. We will begin by reviewing neural networks, parameter estimation, and the universal approximation theorem. Then, we will discuss graphs and motivate graph convolutional neural networks by tracing their origins from spectral filters in signal processing. Lastly, we will review recent progress in topological deep learning, particularly focusing on simplicial, cellular, and hypergraph neural networks as extensions of graph neural networks. We will assume a basic familiarity with linear algebra and calculus; all relevant concepts from graph theory and topology will be introduced.

References:

• Bodnar, C., Frasca, F., Otter, N., Wang, Y., Lio, P., Montufar, G. F., & Bronstein, M. (2021). Weisfeiler and lehman go cellular: Cw networks. Advances in Neural Information Processing Systems.
• Bodnar, C., Frasca, F., Wang, Y., Otter, N., Montufar, G. F., Lio, P., & Bronstein, M. (2021). Weisfeiler and lehman go topological: Message passing simplicial networks. In International Conference on Machine Learning. PMLR.
• Goodfellow, I., Bengio, Y., & Courville, A. (2016). Deep learning. MIT Press.
• Hajij, M., Papamarkou, T., Zamzmi, G., Natesan Ramamurthy, K., Birdal, T., & Schaub, M. T. (2024). Topological deep learning: Going beyond graph data. Published online: https://tdlbook.org/
• Kipf, T. N., & Welling, M. (2017). Semi-supervised classification with graph convolutional networks. In International Conference on Learning Representations. PMLR.

Speaker: Parvaneh Joharinad

Title: Group Equivariant Neural Networks I & II

Speaker: Nico Scherf

Title: Deep Generative Models

Description: In this lecture, I will introduce key concepts underlying deep generative models and provide an overview of various model classes. The focus will then shift to generative adversarial networks (GANs), with a possible introduction to variational autoencoders (VAEs) if time allows. This presentation is conceptual in nature, emphasizing intuitive understanding over theoretical or implementation details. My goal is to offer a clear and accessible overview of these topics. The content is based on Simon Prince's freely available textbook, Understanding Deep Learning (https://udlbook.github.io/udlbook/).

Speaker: Jan Ewald

Title: On the (Underestimated) Importance of Objective/Loss Functions

Description: At the core of supervised and unsupervised learning are objective functions that are minimized (maximized) during the training of neural networks or by determining optimal strategies via mathematical modeling. However, despite their importance, they often find surprisingly little attention in publications and presentations to justify modeling and methodological AI decisions. In the lecture, we will discuss why they should get more awareness by exploring examples, summarizing objective/loss function types and ideas, as well as go through common pitfalls.

Speaker: Jan Ewald

Title: Autoencoder and Their Variants for Biomedical Data

Description: Powerful computational methods are crucial to make use of large-scale and high-dimensional data which is generated in biomedical research to gain insights into biological systems. In recent years autoencoders a family of deep learning-based methods show tremendous potential for biomedical research in various scenarios ranging from synthetic generation, translation of data modalities or the incorporation of biological knowledge to gain explainability of embeddings. In the lecture we will go from the basics, to use-cases and finally to a hands-on applying our framework AUTOENCODIX.

Speaker: Duc Luu

Title: Learning Dynamical Systems I & II

Description: Open challenges of learning data driven dynamical systems lie not only in their high nonlinearity and complexity, but also in nonautonomous and/or stochastic dependency on additive and multiplicative noises, etc. especially when the tasks are to predict short term future states or to learn the asymptotic structures like attractors or even to control the stability of the system. In these two lectures I will give an overview and report on recent developments in neural network and deep learning methods for learning dynamical systems, in particular for those generated from differential/difference equations.

Speaker: Robert Hasse

Title: Large Language Models: An Introduction

Description: Large Language Models (LLMs) are changing the way how humans interact with computers. This has impact on all scientific fields by enabling new ways to achieve for example data analysis goals. In this lecture we will be introduced to LLMs and dive into common applications in the science context. We will see how to generate text, code and images using LLMs and how LLMs can extract information from text and images. We will go through selected prompt engineering techniques enabling scientists to tune the output of LLMs towards their scientific goal and how to do quality assurance in this context.

Speaker: Guido Montufar

Title: An Overview of Theories for Feature Learning in Neural Networks I & II

Description: Feature learning, or learning meaningful representations of raw data, has been one of the guiding ideas behind deep learning. In principle and in practice deep neural networks can automatically learn hierarchies of data representations which successively distill the relevant parts of a problem and make it easier to arrive at a good solution. However, developing a theoretical framework to characterize feature learning and the properties of the data representations that are learned by a neural network is still an ongoing program. These lectures discuss some of the recent perspectives and advances in this direction.

Speaker: Paul Breiding

Title: Computing with Algebraic Varieties I & II

Description: These lectures will consider algebraic varieties from a computational point of view. They will cover topics such as dimension, degree, Gröbner bases and homotopy continuation. As a motivation, we will discuss varieties relevant in machine learning and data science.

Speaker: Angelica Torres

Speaker: Marzieh Eidi