Welcome to the tutorial on Graph Learning: Principles, Challenges, and Open Directions!
This tutorial provides a comprehensive introduction to graph learning and GNNs, covering foundational concepts and recent advancements in the field. We begin with an overview of traditional graph representation and embedding methods, and then focus on modern approaches such as Graph Neural Networks (GNNs), Message Passing Networks (MPNNs), and Graph Transformers (GTs). The second part delves into the expressivity and generalizability of current GNN architectures. We will explore what functions and tasks GNNs can learn, referencing recent research that connects GNN expressivity with the Weisfeiler-Lehman (WL) graph isomorphism test. We will also discuss the generalizability of MPNNs, including their VC dimension and implications for model performance. Next, we address key information-flow challenges in graph learning architectures, such as under-reaching, over-smoothing, and over-squashing. We will highlight recent research aimed at understanding and alleviating these issues, including graph rewiring techniques. The tutorial will conclude with a panel discussion on future directions in graph machine learning. We will explore the limitations of the GNNs, graph foundation models and we will discuss the potential for integrating graph learning with large language models (LLMs) to enhance reasoning and complex data analysis capabilities.
Program
| Time | Session |
|---|---|
| July 22, 2024 | |
| 15:30 - 16:00 |
Introduction to Graph Machine Learning and Graph Neural Networks (GNNs) Speaker: Adrián Arnaiz-Rodríguez, Ameya Velingker |
| 16:00 - 16:30 |
Expressivity and Generalizability of GNNs Speaker: Ameya Velingker |
| 16:30 - 17:00 |
Challenges and Limitations of GNNs Speaker: Adrián Arnaiz-Rodríguez |
| 17:00 - 17:30 |
Panel Discussion on Future Directions of Graph Learning Panelists: Michael Bronstein, Michael Galkin (online), Christopher Morris, Bryan Perozzi Moderated by: Adrián Arnaiz-Rodríguez, Ameya Velingker |
Resources
We provide materials from the tutorial here. We will continue to add more content as it becomes available.
Slides
The slides from the presentation are below:
The slides are additionally available for download in multiple formats: PPTX, PDF
Video
A video of the tutorial is now available on YouTube!
Speakers
The tutorial features speakers with expertise in a variety of areas.
Organizers
-
Adrián Arnaiz-Rodríguez
PhD Student at ELLIS Alicante | website
Speaker Biography
Adrián Arnaiz-Rodríguez is a PhD Student at ELLIS in Alicante Unit and currently at MPI-SWS as part of the ELLIS PhD & PostDoc program. His research interest are in the area of the intersection of graph theory and GNNs (over-problems and graph rewiring), and it's application to trustworthy machine learning. He is supervised by Dr. Nuria Oliver and by Dr. Manuel Gómez Rodriguez in the exchange at MPI-SWS.
-
Ameya Velingker
Research Scientist at Google Research | website
Speaker Biography
Ameya Velingker is a research scientist at Google Research whose research interests are broadly in the area of machine learning and theoretical computer science. He is currently interested in machine learning on graph-structured and relational data and reasoning in ML models (e.g., AI for mathematics/science), combining tools from algorithms with machine learning. He is the author of popular model frameworks such as Exphormer, and his work has also been used in a number of systems such as AlphaProof (theorem proving), Google Maps (routing/navigation), and Gboard (privacy-preserving federated analytics). Velingker holds a Ph.D. from Carnegie Mellon University.
Panelists
-
Michael Bronstein
DeepMind Professor of Artifical Intelligence at the University of Oxford | website
Speaker Biography
Michael Bronstein is the DeepMind professor of AI at the University of Oxford. He previously served as Head of Graph Learning Research at Twitter. His main research interests are in geometric and graph ML and their applications. He has founded several successful startups including Fabula AI (acquired by Twitter in 2019), one of the pioneers of using graph neural networks for social media. He is on the scientific advisory board of Relation Therapeutics, Recursion Pharmaceuticals, and is the Chief Scientist at Vant AI.
-
Michael Galkin
AI Research Scientist at Intel Labs | website
Speaker Biography
Michael Galkin is a Senior Research Scientist at Intel AI Lab working on Graph Foundation Models and various graph ML applications including chip design, materials discovery, and knowledge graphs. Previously, he was a postdoc at Mila–Quebec AI Institute with Will Hamilton, Reihaneh Rabbany, and Jian Tang. Sometimes, Mike writes long blog posts on Medium about graph learning.
-
Christopher Morris
Assistant Professor at RWTH Aachen University | website
Speaker Biography
Christopher Morris completed his Ph.D. in computer science at TU Dortmund University in Germany, specializing in machine learning on graphs. Following his Ph.D., he spent a year as a postdoctoral fellow in the Department of Mathematical and Industrial Engineering at Polytechnique Montréal. He then continued his postdoctoral research in the Computer Science Department at McGill University and as a Mila–Quebec AI Institute member. Since June 2022, he has been a tenure-track assistant professor in the Computer Science Department at RWTH Aachen University in Germany, leading the Learning on Graphs group.
-
Bryan Perozzi
Research Scientist at Google Research | website
Speaker Biography
Bryan Perozzi is a Research Scientist in Google Research’s Algorithms and Optimization group, whose research focuses on learning expressive representations of structured data with neural networks. Bryan is an author of 40+ peer-reviewed papers at leading conferences in machine learning and data mining (such as NeurIPS, ICML, ICLR, KDD, and WWW). He’s the author of popular models in graph representation learning such as DeepWalk (random walk node embeddings), MixHops (graph neural networks) and more. Bryan’s current research focuses on the intersection of structure and generative AI (e.g. Talk Like a Graph). His doctoral thesis was awarded the prestigious SIGKDD Dissertation Award (2017) for making advances in neural network representations for graph data.
Venue
The Graph Learning: Principles, Challenges, and Open Directions tutorial will take place during the 2024 International Conference on Machine Learning (ICML), held at the Messe Wien Exhibition & Congress Center in Vienna Austria.