Transformer Models for Image Registration hos Datalogisk institut - Københavns Universitet

Image registration, a sub-field of Computer Vision, tries to find a geometric transformation to non-lineary align two images. For each pixel of one image, the transformation assigns a corresponding point in the other image. Common applications are in healthcare, for example to track disease progression, tumor growth, or to perform population analysis. Current research on image registration uses mostly unsupervised convolutional neural networks (CNNs) [1].

Transformer models, originally originating in NLP, are increasingly present in computer vision applications [3]. Compared to CNNs, transformer models add an ‘attention‘ layer to model significance relations between different parts of the input. This could help image registration models, where finding spatial relations between images is a prime objective. First attempts of using transformers in image registration succeed at offering incremental improvements over U-nets [2,4], but more work can be done.

The goal of the project is to develop vision transformers for the image registration task, and evaluate their performance in comparison to more traditional CNNs. The student will start the project by understanding the image registration problem, and vision transformer models. The novelty of the project is in adapting, modifying and evaluating vision transformers for the registration task. A large part of the project is the implementation, training and testing of the model (adjusting to our existing code base of pytorch + pytorch lightning would be a plus, but is not required). For evaluation, we do have a large dataset of preprocessed and annotated Brain-MRI scans, and multiple smaller 2d datasets. For an ambitious student, publishing the results of the project is feasible.

References

[1] Guha Balakrishnan et al. “VoxelMorph: A Learning Framework for Deformable Medical Image Registration”. In: IEEE Transactions on Medical imaging 38.8 (2019), pp. 1788–1800.

[2] Junyu Chen et al. “ViT-V-Net: Vision Transformer for Unsupervised Volumetric Medical Image Registration”. In: (2021).

[3] Alexey Dosovitskiy et al. “An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale”. In: (2020).

[4] ZihaoWang and Hervé Delingette. “Attention for Image Registration (AiR): an unsupervised Transformer approach”. In: (2021).

Contact information: 

Steffen Czolbe, per.sc@di.ku.dk 

 

IMAGE: Image Analysis, Computational Modelling, and Geometry

The work in the section ranges from theoretical analyses, over algorithm development, to solving concrete problems for science, industry and society within image analysis and processing, computer vision, computer simulation, numerical optimization, machine learning, computational modelling, and geometry.

The section is divided into seven areas:

  •        Applied Geometry

 

  •        Bioimaging

 

  •        Computer Graphics and Simulation

 

  •        Computer Vision

 

  •        Image Analysis

 

  •        Medical Image Analysis

Read more here: https://di.ku.dk/english/research/image/

 

 

Husk at nævne, at du fandt dette opslag på KU Projekt & Job