mathematical_methods_machine_learning []

This page is read only. You can view the source, but not change it. Ask your administrator if you think this is wrong.
====== Math and Machine Learning: Theory and Applications (Fall 2024) ======

===== Registration =====

  * Register on the [[https://www.mis.mpg.de/events/event/mathematics-and-machine-learning|MPI course website]].

===== Organization =====

  * **Location**: [[https://www.mis.mpg.de/|Max Planck Insitute for Mathematics in the Sciences]], Seminar Room E2 10 (Leon Lichtenstein)
  * **Organizers**: Diaaeldin Taha and Parvaneh Joharinad
  * **Institutional Website**: [[https://www.mis.mpg.de/events/event/mathematics-and-machine-learning|link]]
  * **Contact**: To contact the organizers, email the lab at ''lab [at] mis [dot] mpg [dot] de''.
  * **Mailing List**: To stay informed of Lab activities, including this group's meetings, join the [[https://listserv.mis.mpg.de/listinfo/lab-newsletter|Lab mailing list]].

===== Schedule =====

^ Week        ^ Date          ^ Time         ^ Location   ^ Speaker              ^ Topic ^
^ Week 45 (2024) | Mon, 04.11.2024 | 15:00–16:30  | MIS, E2 10      | Diaaeldin Taha | Graph and Topological Neural Networks I   |
^ Week 46 (2024) | Mon, 11.11.2024 | 15:00–16:30  | MIS, E2 10      | Diaaeldin Taha | Graph and Topological Neural Networks II   |
^ Week 47 (2024) | Mon, 18.11.2024 | 15:00–16:30  | MIS, E2 10      | Parvaneh Joharinad | Group Equivariant Neural Networks I   |
^ Week 48 (2024) | Mon, 25.11.2024 | 15:00–16:30  | MIS, E2 10      | Parvaneh Joharinad | Group Equivariant Neural Networks II   |
^ Week 49 (2024) | Mon, 02.12.2024 | 15:00–16:30  | MIS, G3 10      | Nico Scherf           | Deep Generative Models   |
^ Week 50 (2024) | Mon, 09.12.2024 | ---  | ---      | ---           | **NO MEETING**   |
^ Week 51 (2024) | Mon, 16.12.2024 | 15:00–16:30  | MIS, E2 10      | Jan Ewald             | On the (Underestimated) Importance of Objective/Loss Functions   |
^ Week 3 (2025)  | Mon, 13.01.2025 | 14:00–15:30  | MIS, A3 01      | Jan Ewald             | Autoencoder and Their Variants for Biomedical Data   |
^ Week 4 (2025)  | Mon, 20.01.2025 | 14:00–15:30  | MIS, A3 01      | Duc Luu               | Learning Dynamical Systems I   |
^ Week 5 (2025)  | Mon, 27.01.2025 | 14:00–15:30  | MIS, A3 01      | Duc Luu               | Learning Dynamical Systems II   |
^ Week 6 (2025)  | Mon, 03.02.2025 | 14:00–15:30  | MIS, A3 01      | Robert Haase          | Large Language Models for Code Generation   |
^ Week 7 (2025)  | Mon, 10.02.2025 | 14:00–15:30  | MIS, A3 01      | Guido Montufar        | An Overview of Theories for Feature Learning in Neural Networks I   |
^ Week 8 (2025)  | Mon, 17.02.2025 | 14:00–15:30  | MIS, A3 01      | Guido Montufar        | An Overview of Theories for Feature Learning in Neural Networks II   |
^ Week 9 (2025)  | Mon, 24.02.2025 | 14:00–15:30  | MIS, A3 01      | Paul Breiding         | Computing with Algebraic Varieties I   |
^ Week 10 (2025) | Mon, 03.03.2025 | ---  | ---     | ---         | **NO MEETING**   |
^ Week 11 (2025) | Mon, 10.03.2025 | ---  | ---      | ---         | **NO MEETING**   |
^ Week 12 (2025) | Mon, 17.03.2025 | 14:00–15:30  | MIS, A3 01      | Angelica Torres       | Varieties in Machine Learning I   |
^ Week 13 (2025) | Mon, 24.03.2025 | 14:00–15:30  | MIS, A3 01      | Angelica Torres       | Varieties in Machine Learning II   |
^ Week 11 (2025) | Mon, 31.03.2025 | ---  | ---      | ---         | **NO MEETING**   |
^ Week 14 (2025)  | Mon, 07.04.2025 | 14:00–15:30  | MIS, A3 01      | Paul Breiding         | Computing with Algebraic Varieties II   |

===== Information =====

==== Weeks 45 & 46 (2024) ====

**Speaker**: Diaaeldin Taha (Max Planck Institute for Mathematics in the Science, Germany)

**Title**: Graph and Topological Neural Networks I & II

**Description**: In these two sessions, we will provide an overview of deep learning with a focus on graph and topological neural networks. We will begin by reviewing neural networks, parameter estimation, and the universal approximation theorem. Then, we will discuss graphs and motivate graph convolutional neural networks by tracing their origins from spectral filters in signal processing. Lastly, we will review recent progress in topological deep learning, particularly focusing on simplicial, cellular, and hypergraph neural networks as extensions of graph neural networks. We will assume a basic familiarity with linear algebra and calculus; all relevant concepts from graph theory and topology will be introduced.

**References**:
  * Bodnar, C., Frasca, F., Otter, N., Wang, Y., Lio, P., Montufar, G. F., & Bronstein, M. (2021). Weisfeiler and lehman go cellular: Cw networks. Advances in Neural Information Processing Systems.
  * Bodnar, C., Frasca, F., Wang, Y., Otter, N., Montufar, G. F., Lio, P., & Bronstein, M. (2021). Weisfeiler and lehman go topological: Message passing simplicial networks. In International Conference on Machine Learning. PMLR.
  * Goodfellow, I., Bengio, Y., & Courville, A. (2016). Deep learning. MIT Press.
  * Hajij, M., Papamarkou, T., Zamzmi, G., Natesan Ramamurthy, K., Birdal, T., & Schaub, M. T. (2024). Topological deep learning: Going beyond graph data. Published online: https://tdlbook.org/
  * Kipf, T. N., & Welling, M. (2017). Semi-supervised classification with graph convolutional networks. In International Conference on Learning Representations. PMLR.

==== Weeks 47 & 48 (2024) ====

**Speaker**: Parvaneh Joharinad

**Title**: Group Equivariant Neural Networks I & II

==== Week 49 (2024) ====

**Speaker**: Nico Scherf

**Title**: Deep Generative Models

**Description**: In this lecture, I will introduce key concepts underlying deep generative models and provide an overview of various model classes. The focus will then shift to generative adversarial networks (GANs), with a possible introduction to variational autoencoders (VAEs) if time allows. This presentation is conceptual in nature, emphasizing intuitive understanding over theoretical or implementation details. My goal is to offer a clear and accessible overview of these topics. The content is based on Simon Prince's freely available textbook, Understanding Deep Learning (https://udlbook.github.io/udlbook/).

==== Week 50 (2024) ==== 

**Speaker**: Jan Ewald

**Title**: On the (Underestimated) Importance of Objective/Loss Functions

**Description**: At the core of supervised and unsupervised learning are objective functions that are minimized (maximized) during the training of neural networks or by determining optimal strategies via mathematical modeling. However, despite their importance, they often find surprisingly little attention in publications and presentations to justify modeling and methodological AI decisions. In the lecture, we will discuss why they should get more awareness by exploring examples, summarizing objective/loss function types and ideas, as well as go through common pitfalls.

==== Week 3 (2025) ====

**Speaker**: Jan Ewald

**Title**: Autoencoder and Their Variants for Biomedical Data 

**Description**: Powerful computational methods are crucial to make use of large-scale and high-dimensional data which is generated in biomedical research to gain insights into biological systems. In recent years autoencoders a family of deep learning-based methods show tremendous potential for biomedical research in various scenarios ranging from synthetic generation, translation of data modalities or the incorporation of biological knowledge to gain explainability of embeddings. In the lecture we will go from the basics, to use-cases and finally to a hands-on applying our framework AUTOENCODIX.


==== Weeks 4 & 5 (2025) ====

**Speaker**: Duc Luu

**Title**: Learning Dynamical Systems I & II

**Description**: Open challenges of learning data driven dynamical systems lie not only in their high nonlinearity and complexity, but also in nonautonomous and/or stochastic dependency on additive and multiplicative noises, etc. especially when the tasks are to predict short term future states or to learn the asymptotic structures like attractors or even to control the stability of the system. In these two lectures I will give an overview and report on recent developments in neural network and deep learning methods for learning dynamical systems, in particular for those generated from differential/difference equations.         

==== Weeks 6 (2025) ====

**Speaker**: Robert Hasse

**Title**: Large Language Models: An Introduction

**Description**: Large Language Models (LLMs) are changing the way how humans interact with computers. This has impact on all scientific fields by enabling new ways to achieve for example data analysis goals. In this lecture we will be introduced to LLMs and dive into common applications in the science context. We will see how to generate text, code and images using LLMs and how LLMs can extract information from text and images. We will go through selected prompt engineering techniques enabling scientists to tune the output of LLMs towards their scientific goal and how to do quality assurance in this context.

==== Weeks 7 & 8 (2025) ====

**Speaker**: Guido Montufar

**Title**: An Overview of Theories for Feature Learning in Neural Networks I & II

**Description**: Feature learning, or learning meaningful representations of raw data, has been one of the guiding ideas behind deep learning. In principle and in practice deep neural networks can automatically learn hierarchies of data representations which successively distill the relevant parts of a problem and make it easier to arrive at a good solution. However, developing a theoretical framework to characterize feature learning and the properties of the data representations that are learned by a neural network is still an ongoing program. These lectures discuss some of the recent perspectives and advances in this direction.
==== Weeks 9 & 11 (2025) ====

**Speaker**: Paul Breiding

**Title**: Computing with Algebraic Varieties I & II

**Description**: These lectures will consider algebraic varieties from a computational point of view.  They will cover topics such as dimension, degree, Gröbner bases and homotopy continuation. As a motivation, we will discuss varieties relevant in machine learning and data science.
==== Weeks 12 & 13 (2025) ====

**Speaker**: Angelica Torres (MPI MIS)

**Title**: Varieties in Machine Learning I & II

**Description**:
  * **Varieties in Machine Learning I**: In this session we will build on Paul Breiding’s lectures and focus on the relation between the implicitization problem in Algebraic Geometry and the function space of neural networks.  We will review the work by Kohn, Trager, Kileel, Montufar, Li, and others, regarding the geometric properties of the function space of linear neural networks and polynomial neural networks. 
  * **Varieties in Machine Learning II**: In this session we focus on the algebraic and geometric properties of the varieties studied in the previous session. We will pay special attention to the Euclidean Distance Degree and its relation with the optimization problem in Machine Learning.