Back to Projects

AI-enabled Imaging, Emerging Imaging

MultiHeart: Fusing CT and MRI with Space-Time Transformation Networks

Project ID: 2022_044

Student: Yiyang Xu

1st Supervisor: Alistair Young, King’s College London
2nd Supervisor: Steve Niederer, King’s College London
Clinical Supervisor: Amedeo Chiribiri
Industrial Supervisor: Matt Sinclair (HeartFlow)

Aim of the PhD Project 

The overall goal is to develop a deep-learning based multi-modal spatio-temporal atlas, which will enable rapid co-registration of patient cardiac anatomy between different modalities. Specific aims are:

  • Automatically register CT and MRI exams within and between patients.
  • Predict motion abnormalities from CT
  • Predict extent of ischemic region from CT

Project description

Coronary artery disease is the leading cause of death worldwide. Coronary computed tomography angiography (CCTA) and cardiac magnetic resonance (CMR) are non-invasive imaging methods which are commonly used to evaluate patients to test whether invasive procedures are necessary [1]. However, they provide complimentary information, leading to the need for combining information across modalities for improved diagnosis [2-4]. CCTA provides high-resolution anatomical information allowing for the evaluation of coronary artery stenosis (narrowed arteries), including the characterization of plaque, calcium, percent stenosis [5], and flow limiting stenoses [6]. CMR provides information about myocardial function, including assessment of wall motion abnormalities (from cine-CMR), viability, ischemia and scar (from perfusion-CMR and delayed-enhancement CMR) [1]. The high temporal resolution of cine-CMR allows for wall motion abnormalities to be observed, while the high contrast sensitivity of perfusion-CMR allows for the quantification of myocardial perfusion, as well as identification of microvascular disease [7]. This information complements the assessment of coronary artery disease from CCTA and offers a more complete picture, helping to guide treatment options.

Fusion of CCTA and CMR examinations will give added clinical utility to existing analyses, by providing a deeper and concordant evaluation of anatomical and functional predictors. It would also enable better identification of which stenoses are significant. Perfusion imaging-derived myocardial blood flow could provide improved patient-specific boundary conditions, allowing total flow through the coronary tree to be matched with the measured flow at a whole-heart or territory level. Such co-registration would also allow for validation of a CCTA-derived computation of extent of ischaemic region [8] and allow better parametrization of a CCTA-derived simulated perfusion model [9]. In addition to this application, a joint CT-MRI spatio-temporal motion atlas could be learned. Such a spatio-temporal motion atlas could leverage the high temporal information in cine-MRI to inform the estimation of motion abnormalities from CCTA alone, where only limited temporal resolution (e.g., several frames across the cardiac cycle) might be available.

This PhD project will focus on the development of a deep-learning based multi-modal spatio-temporal atlas, which will also enable rapid co-registration of patient cardiac anatomy between different modalities. This work aims to build on other recent work in this space, such as Atlas-ISTN [10]. An effective multi-modal registration approach would further enable exciting applications of statistical population modelling of anatomy and function, leveraging the complementary information in heterogeneous datasets providing novel insights about normal and pathological variations.

The project would be suited to a computer science, engineering or applied mathematics student familiar with python, machine learning libraries, and medical image processing.

An overview of the Atlas-ISTN framework in training mode

Fig 1. An overview of the Atlas-ISTN framework in training mode


[1] Dewey et al. “Clinical quantitative cardiac imaging for the assessment of myocardial ischaemia”, Nature Reviews Cardiology, 2020.

[2] O. Gaemperli et al. “Cardiac Hybrid Imaging”, European Journal of Nuclear Medicine and Molecular Imaging, 2011.

[3] J. von Spiczak et al. “Multimodal Multiparametric Three-dimensional Image Fusion in Coronary Artery Disease: Combining the Best of Both Worlds”, Radiology, 2020.

[4] A. Coenen et al. “Integrating CT Myocardial Perfusion and CT-FFR in the Work-Up of Coronary Artery Disease”, JACC: Cardiovascular Imaging, 2017.

[5] R. Driessen et al. “Effect of Plaque Burden and Morphology on Myocardial Blood Flow and Fractional Flow Reserve”, JACC, 2018.

[6] C. Taylor et al. “Computational fluid dynamics applied to cardiac computed tomography for noninvasive quantification of fractional flow reserve: Scientific basis”, JACC, 2013.

[7] H. Rahman et al. “High-Resolution Cardiac Magnetic Resonance Imaging Techniques for the Identification of Coronary Microvascular Dysfunction”, JACC: Cardiovascular Imaging 2021.

[8] S. Malkasian et al. “Quantification of vessel-specific coronary perfusion territories using minimum-cost path assignment and computed tomography angiography: Validation in a swine model”, JCCT, 2018.

[9] L. Papamanolis et al. “Myocardial Perfusion Simulation for Coronary Artery Disease: A Coupled Patient-Specific Multiscale Model”, Annals of Biomedical Engineering, 2020.

[10] M. Sinclair et al. “Atlas-ISTN: Joint Segmentation, Registration and Atlas Construction with Image-and-Spatial Transformer Networks”, arXiv preprint, 2020.

[11] M. Hoffman et al. “SynthMorph: learning contrast-invariant registration without acquired images”, IEEE Transactions on Medical Imaging, 2021.

Back to Projects