Segmentation, detection, and classification are major tasks in medical image analysis and image understanding. Medical imaging researchers heavily use the results of recent developments in machine learning approaches, and with deep learning methods they achieve significantly better results in many real-world problems compared to previous solutions. The course aims to enable students and professionals to apply deep learning methods to their data and problem. Using an interactive programming environment, participants of the course will explore all required steps in practice and learn tools and techniques from data preparation to result interpretation. We will work on example data and train models to segment anatomical structures, to detect abnormalities, and to classify them. Simple methods to explain predictions and assess network uncertainty will be discussed briefly as well. Participants will work in a prepared online environment providing selected deep learning toolkit installations, example data, and fully functional skeleton code as a basis for own experiments.

Compared to the tremendous success models like GPT4, Bard, or Llama, which are all based on transformers, had in the field of text analysis, machine translation, and de-novo text generation, it took longer for transformers to enter the scene of image analysis and image generation. Reasons being the high demands in terms of compute and data. More successful approaches in training transformers have led to a change that will likely be as impactful as the introduction of CNNs for image classification. Already, first (still limited) “image foundation models” have been published, and also in medical image analysis, several attempts are being made to parallel the semantic understanding and emergence seen in large language models for image models. In this course, we explain the thought model and elementary mathematics of the attention mechanism underlying transformers. You will learn in theory and explore in hands-on work the reason for their modeling capacity and understand why this creates the need of larger training datasets. The course traces the development of transformers for image analysis tasks and shows ways to pre-train transformers on weak or unlabeled data. The course concludes with examples of applications used in medical image analysis tasks.

This half-day deep dive course will guide researchers with some background knowledge, e.g. from the introductory course, SC1235 Introduction to Medical Image Analysis using Convolutional Neural Networks, through the most important concepts of generative adversarial networks (GANs) and show example applications to medical data. GANs are powerful appearance models, but GANs can also be used to map between different domains (such as between CT and MRI) or to help training better segmentation models. Adversarial training can be introduced into several learning tasks in medical image analysis. It has been shown to help make image analysis algorithms more robust to variability in the data and to reduce the probability of failure on unseen cases. GANs in their initial implementation have been known to be hard to configure and train, but recent advances have helped them catch ground in applications of classification and segmentation. We will introduce GANs conceptually and from a Variational Inference perspective, give an overview of their development towards the state of the art, and explain specific architectural decisions and developments that have been proposed to stabilize their training. We will show code examples and illustrate the course content with live demonstrations on example data, so that the participants gain some first-hand experience on the subject. The course is not designed as a hands-on workshop, though.