University of Southern California Dissertations and Theses

Shift-invariant autoregressive reconstruction for MRI

(USC Thesis Other)

Shift-invariant autoregressive reconstruction for MRI

PDF

Download a page range

Download transcript

Copy asset link

Request this asset

Transcript (if available)

Content SHIFT-INVARIANT AUTOREGRESSIVE
RECONSTRUCTION FOR MRI
by
Tae Hyung Kim
A Dissertation Presented to the
FACULTY OF THE USC GRADUATE SCHOOL
UNIVERSITY OF SOUTHERN CALIFORNIA
In Partial Fulllment of the
Requirements for the Degree
Doctor of Philosophy
(Electrical Engineering)
December 2020
Copyright 2020 Tae Hyung Kim
Acknowledgments
Navigating through my doctoral program has always presented challenges, and writing this doctoral
thesis has been one of them. As I nalize my long journey at the University of Southern California
(USC), I would like to express my gratitude to many people who have helped me successfully
complete my Ph.D. program in many ways.
First, I would like to express my gratitude to Professor Justin P. Haldar, for whom I have
the greatest respect. He has been my best academic advisor and mentor and is one of the most
intellectual people I have ever met. He is also a great teacher, researcher, and presenter. He has
been my perfect role model, as he has not only taught me about all aspects of academics, but also
led me to appreciate and enjoy research.
I would also like to thank Professor Richard M. Leahy. He has always co-led the Biomedical
Imaging Group well and given me insightful feedback. By sharing new ideas and fostering discussion,
he taught me how to examine problems from various perspectives. I would also like to thank him
for serving as a doctoral committee member and providing me invaluable advice.
Professor Krishna S. Nayak, from whom I took my rst in-depth class on MRI, has also been
instrumental in my research. His course inspired me, and he oered me constructive advice and
encouragement on both academic and nonacademic matters. I am also grateful that I could have
him as my qualifying exam and doctoral committee member.
Thanks also to Dr. John C. Wood. He gave me extensive practical advice as an external
committee member for both my qualifying exam and doctoral defense, which helped me develop
my professional skills as a researcher.
As a proud member of the Biomedical Imaging Group, I had the good fortune of working with
exceptional colleagues, who I also consider my friends. I enjoyed spending time at Ronald Tutor
ii
Hall 317 with Dr. Divya Varadarajan, Dr. Daeun Kim, Rodrigo Lobos, Yunsong Liu, Jiayang
Wang and Chin-Cheng Chan. I am also grateful to other members who did not share the room,
including Prof. Anand A. Joshi, Dr. Chitresh Bhushan, Dr. Jian Li (Andrew), Dr. Minqi Chong,
Dr. Takfarinas Medani, Soyoung Choi, Hossein Shahabi, Dakarai McCoy, Haleh Akrami, Yijun
Liu, and Clio Gonzlez Zacaras. Thanks to these colleagues and friends, I was able to mature
academically, intellectually and personally.
Much of my research would not have been possible without great collaborators. I give special
thanks to Dr. Kawin Setsompop and Dr. Berkin Bilgic from Athinoula A. Martinos Center for
Biomedical Imaging, Massachusetts General Hospital/Harvard Medical School, with whom I co-
authored several papers on cutting-edge MRI research.
I would also like to express my appreciation to the Kwanjeong Educational Foundation and
Annenberg Fellowship for their nancial support. Thanks to them, I was able to work in a stable
and productive environment where I could devote myself to my research.
Lastly and most importantly, I would like to express my gratitude to my family members,
Father Weon Seok Kim, Mother Sungnam Kang, and Sister Jihye Kim for always providing me
with emotional and material support. Without them, I would never have been able to begin,
proceed, and complete my PhD.
Thanks to all the precious people I met, I was able to successfully complete my doctoral degree.
iii
Table of Contents
Acknowledgments ii
Table of Contents iv
List Of Tables vii
List Of Figures viii
Abbreviations xiv
Abstract xvi
Chapter 1: Introduction 1
1.1 Problem Statement . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3
1.2 Main Results . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5
1.3 Organization of the Document . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9
Chapter 2: Conventional MRI Reconstruction Methods 11
2.1 Parallel Imaging . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11
2.2 Phase-constrained Imaging . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 14
2.3 Sparsity-based Reconstruction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 16
2.4 Deep Learning in MRI Reconstruction . . . . . . . . . . . . . . . . . . . . . . . . . . 16
Chapter 3: Review of LORAKS 19
3.1 Shift-invariant autoregressive reconstruction . . . . . . . . . . . . . . . . . . . . . . . 19
3.2 LORAKS . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 21
Chapter 4: Accelerated MRI using LORAKS: Linear Shift-invariant Autoregressive
Reconstruction through Structured Low-Rank Matrix Modeling 27
4.1 Simultaneous Multislice Imaging (SMS-LORAKS) . . . . . . . . . . . . . . . . . . . 27
4.1.1 Background and Notation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 27
4.1.2 SMS-LORAKS . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 30
4.1.3 Experiments and Results . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 30
4.1.4 Conclusions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 32
4.2 Parallel and Phase-constrained Imaging with Sensitivity Encoding (SENSE-LORAKS) 33
4.2.1 Theory . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 33
4.2.2 Methods . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 35
iv
4.2.3 Results . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 41
4.2.4 Discussion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 48
4.2.5 Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 51
4.3 Highly accelerated 3D imaging with Wave-encoding (Wave-LORAKS) . . . . . . . . 52
4.3.1 Theory . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 52
4.3.2 Methods . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 59
4.3.3 Results . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 62
4.3.4 Discussion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 66
4.3.5 Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 71
4.4 Summary and Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 72
4.A Supporting Figures for SENSE-LORAKS . . . . . . . . . . . . . . . . . . . . . . . . 73
Chapter 5: LORAKI: A Nonlinear Shift-invariant Autoregressive MRI Reconstruc-
tion through Scan-specic Recurrent Neural Networks 85
5.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 85
5.2 Background . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 87
5.2.1 Autoregression and Autocalibration . . . . . . . . . . . . . . . . . . . . . . . 87
5.2.1.1 GRAPPA . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 88
5.2.1.2 RAKI . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 89
5.2.1.3 LORAKS and AC-LORAKS . . . . . . . . . . . . . . . . . . . . . . 91
5.3 Proposed LORAKI Approach . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 95
5.3.0.1 Training Considerations and Synthetic ACS data . . . . . . . . . . . 96
5.4 Evaluation Methods . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 96
5.4.1 Datasets . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 96
5.4.2 Comparisons and Performance Metrics . . . . . . . . . . . . . . . . . . . . . . 97
5.4.3 Implementation Details . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 98
5.4.3.1 LORAKI . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 98
5.4.3.2 GRAPPA . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 99
5.4.3.3 sRAKI . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 99
5.4.3.4 AC-LORAKS . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 99
5.5 Results . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 100
5.6 Discussion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 103
5.7 Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 109
Chapter 6: Ecient Iterative Solutions to Complex-Valued Nonlinear Least-Squares
Problems with Mixed Linear and Antilinear Operators 111
6.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 111
6.2 Background . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 114
6.2.1 Linear, Antilinear, and Real-Linear Operators . . . . . . . . . . . . . . . . . . 114
6.2.2 Real-Valued Transformation of Complex-Valued Least Squares . . . . . . . . 116
6.3 Main Results . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 120
6.4 Useful Relations for Common Real-Linear Operators . . . . . . . . . . . . . . . . . . 123
6.5 Numerical Example . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 123
6.6 Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 129
6.A Proof of Lemma 1 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 130
6.B Proof of Lemma 2 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 131
v
6.C Proof of Lemma 3 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 132
Chapter 7: Conclusion 135
Bibliography 137
Appendix A: The Fourier Radial Error Spectrum Plot: a More nuanced quantita-
tive evaluation of image reconstruction quality 147
A.A Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 147
A.B Creating the Fourier Radial Error Spectrum Plot . . . . . . . . . . . . . . . . . . . . 149
A.C Illustrative Application to MRI Reconstruction . . . . . . . . . . . . . . . . . . . . . 152
A.D Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 154
Appendix B: LORAKS Software Version 2.0: Faster Implementation and Enhanced
Capabilities 157
B.A Overview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 157
B.B Mathematical Description of Reconstruction Approaches Provided by the Software . 159
B.B.1 Problem Formulations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 159
B.B.1.1 Original Single-Channel Formulation . . . . . . . . . . . . . . . . . . 159
B.B.1.2 Enforcing Exact Data Consistency . . . . . . . . . . . . . . . . . . . 161
B.B.1.3 Multi-Channel Formulations: P-LORAKS and SENSE-LORAKS . . 162
B.B.1.4 Autocalibrated LORAKS . . . . . . . . . . . . . . . . . . . . . . . . 163
B.B.2 Algorithm Choices . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 165
B.B.2.1 Original Additive Half-Quadratic Majorize-Minimize Approach . . . 165
B.B.2.2 Multiplicative Half-Quadratic Majorize-Minimize Approach . . . . . 167
B.B.2.3 FFT-Based Computations . . . . . . . . . . . . . . . . . . . . . . . 169
B.C Software . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 171
B.C.1 P LORAKS.m . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 172
B.C.2 AC LORAKS.m . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 174
B.C.3 SENSE LORAKS.m . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 177
B.D Examples and Usage Recommendations . . . . . . . . . . . . . . . . . . . . . . . . . 180
B.D.1 Single-channel reconstruction . . . . . . . . . . . . . . . . . . . . . . . . . . . 180
B.D.2 Multi-channel reconstruction . . . . . . . . . . . . . . . . . . . . . . . . . . . 182
B.D.3 Choice of LORAKS matrix and neighborhood radius . . . . . . . . . . . . . . 184
B.D.4 Choice of algorithm . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 186
B.D.5 Choice of regularization parameter and the maximum number of iterations 189
vi
List Of Tables
4.1 The eect of the LORAKS neighborhood radius on NRMSE, memory usage, and
computation time for reconstruction of Dataset 1 with 12 accelerated VD CAIPI
sampling. For reference, values corresponding to Wave-CAIPI reconstruction are
also shown. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 72
6.1 Table of common real-linearA() operators and correspondingA

() operators. We
also provide expressions forA

(A()) in cases where the combined operator takes a
simpler form than applying each operator sequentially. In the last two rows, it is
assumed that the matrix A2C
M
1
N
, and that the vector y2C
M
is divided into
two components y
1
2C
M
1
and y
2
2C
MM
1
with y =
h
y
T
1
y
T
2
i
T
. In the last row,
we takeB(x), Cx D(Ex), with correspondingB

(y) = C
H
y E
H
(D
H
y). Note
that a special case of equivalent complex-valued operators associated with Eq. (6.7)
(with B chosen as the identity matrix) was previously presented by Ref. [2], although
without the more general real-linear mathematical framework developed in this work. 124
A.1 Conventional Error Metrics for SENSE-TV and SENSE-LORAKS. . . . . . . . . . . 153
B.1 Algorithm comparison results from ex4.m. . . . . . . . . . . . . . . . . . . . . . . . . 189
vii
List Of Figures
1.1 Undersampled MRI reconstruction pipeline. . . . . . . . . . . . . . . . . . . . . . . . 2
3.1 Autoregressive predictive relationship in k-space. . . . . . . . . . . . . . . . . . . . . 20
3.2 Shift-invariant autoregressive relationship induced from limited image support and
corresponding structured low-rank matrix modeling. . . . . . . . . . . . . . . . . . . 21
4.1 Single-channel reconstruction results. . . . . . . . . . . . . . . . . . . . . . . . . . . . 31
4.2 12-channel reconstruction results. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 32
4.3 Error (in the`
2
-norm) versus rank (r
C
andr
S
, respectively, for C and S) for dierent
slice encoding schemes. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 33
4.4 Gold standard (a,c,e) magnitude and (b,d,f) phase images used for evaluation. Im-
ages correspond to the (a,b) TSE, (c,d) MPRAGE, and (e,f) EPI datasets. . . . . . 36
4.5 Comparison between (a) SENSE with conventional Uniform sampling, (b) P-LORAKS
with conventional Uniform sampling, (c) SENSE-LORAKS with conventional Uni-
form sampling, and (d) SENSE-LORAKS with Uniform PF sampling for the TSE
data with 5.1 acceleration. Images for other acceleration factors are shown in
Supporting Figs. S1 and S2, while NRMSE values are plotted in Fig. 4(a). Re-
constructed images are shown using a linear grayscale (normalized so that image
intensities are in the range from 0 to 1), while error images are shown using the
indicated colorscale (which ranges from 0 to 0.25 to highlight small errors). NRMSE
values are shown underneath each reconstruction, with the best NRMSE values high-
lighted with red. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 40
4.6 Comparison between (a) SENSE with conventional Uniform sampling, (b) P-LORAKS
with conventional Uniform sampling, (c) SENSE-LORAKS with conventional Uni-
form sampling, and (d) SENSE-LORAKS with Uniform PF sampling for the MPRAGE
data with 5.0 acceleration. Images for other acceleration factors are shown in Sup-
porting Figs. S3 and S4, while NRMSE values are plotted in Fig. 4(b). Recon-
structed images are shown using a linear grayscale (normalized so that image inten-
sities are in the range from 0 to 1), while error images are shown using the indicated
colorscale (which ranges from 0 to 0.25 to highlight small errors). NRMSE values
are shown underneath each reconstruction, with the best NRMSE values highlighted
with red. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 41
viii
4.7 Plots of the reconstruction NRMSE as a function of the acceleration rate for the
(a) TSE data (corresponding images were shown in Fig. 2 and Supporting Figs. S1
and S2) and (b) MPRAGE data (corresponding images were shown in Fig. 3 and
Supporting Figs. S3 and S4). . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 42
4.8 Comparison of SENSE-LORAKS against (a) phase-constrained SENSE with AC
PF sampling, (b) P-LORAKS with Random PF sampling, and (c) SENSE+TV
with conventional Uniform sampling for the TSE dataset with 5.1 acceleration.
Reconstructions obtained using SENSE-LORAKS with Uniform PF sampling are
shown in (d). Images for other acceleration factors are shown in Supporting Figs.
S5 and S6, while NRMSE values are plotted in Fig. 7(a). The reconstructed images
are displayed using a linear grayscale (normalized so that image intensities are in
the range from 0 to 1). The error images are displayed using the indicated colorscale
(which ranges from 0 to 0.25 to highlight small errors). NRMSE values are shown
underneath each reconstruction, with the best NRMSE values highlighted with bold
text in each sampling pattern. The smallest NRMSE values for a given acceleration
rate are indicated in red. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 43
4.9 Comparison of SENSE-LORAKS against (a) phase-constrained SENSE with AC PF
sampling, (b) P-LORAKS with Random PF sampling, and (c) SENSE+TV with
conventional Uniform sampling for the MPRAGE dataset with 5.0 acceleration.
Reconstructions obtained using SENSE-LORAKS with Uniform PF sampling are
shown in (d). Images for other acceleration factors are shown in Supporting Figs.
S7 and S8, while NRMSE values are plotted in Fig. 7(b). The reconstructed images
are displayed using a linear grayscale (normalized so that image intensities are in
the range from 0 to 1). The error images are displayed using the indicated colorscale
(which ranges from 0 to 0.25 to highlight small errors). NRMSE values are shown
underneath each reconstruction, with the best NRMSE values highlighted with bold
text in each sampling pattern. The smallest NRMSE values for a given acceleration
rate are indicated in red. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 44
4.10 Plots of the reconstruction NRMSE as a function of the acceleration rate for the
(a) TSE data (corresponding images were shown in Fig. 5 and Supporting Figs. S5
and S6) and (b) MPRAGE data (corresponding images were shown in Fig. 6 and
Supporting Figs. S7 and S8). . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 45
4.11 Comparison between (left) SENSE with conventional Uniform sampling, (middle)
SENSE-LORAKS with conventional Uniform sampling, and (right) SENSE-LORAKS
with Uniform PF sampling for the EPI data. The left columns show reconstructed
images using a linear grayscale (normalized so that image intensities are in the range
from 0 to 1), while the right columns show error images using the indicated colorscale
(which ranges from 0 to 0.25 to highlight small errors). NRMSE values are shown
underneath each reconstruction. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 46
ix
4.12 SENSE-LORAKS reconstruction performance as a function of (a) rankr, (b) Tikhonov
regularization parameter
T
, and (c) LORAKS regularization parameter
S
. Except
where parameter values are being explicitly changed, the r,
T
, and
S
parameters
were set to their default values as described in the Methods section. . . . . . . . . . 47
4.13 Comparison between SENSE+TV, SENSE-LORAKS, and SENSE-LORAKS with
TV sampling for TSE data with Uniform PF sampling and 5.1 acceleration. Other
acceleration factors are shown in Supporting Fig. S10. The top row shows recon-
structed images using a linear grayscale (normalized so that image intensities are in
the range from 0 to 1), while the bottom row shows error images using the indicated
colorscale (which ranges from 0 to 0.25 to highlight small errors). NRMSE values
are shown underneath each reconstruction, with the best NRMSE value highlighted
in red. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 49
4.14 Gold standard magnitude and phase images for Dataset 1 (top row) and Dataset 2
(bottom row). Images are shown for three representative orthogonal views. . . . . . 58
4.15 Dierent 12 accelerated sampling patterns used with Dataset 1. (This gure con-
tains high resolution detail that may not print clearly on certain printers. Readers
may prefer to view the electronic version of this gure.) . . . . . . . . . . . . . . . . 62
4.16 Images showing the 3D aliasing patterns corresponding to the sampling patterns
from Fig. 4.15 for Dataset 1. We show (top row) axial, (middle row) coronal, and
(bottom row) sagittal views that are matched to the 3 orthogonal views shown in
Fig. 4.14. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 63
4.17 Dierent 12 accelerated sampling patterns used with Dataset 2. (This gure con-
tains high resolution detail that may not print clearly on certain printers. Readers
may prefer to view the electronic version of this gure.) . . . . . . . . . . . . . . . . 64
4.18 Images showing the 3D aliasing patterns corresponding to the sampling patterns
from Fig. 4.17 for Dataset 2. We show (top row) axial, (middle row) coronal, and
(bottom row) sagittal views that are matched to the 3 orthogonal views shown in
Fig. 4.14. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 65
4.19 Reconstructions of a representative sagittal slice from Dataset 1 using dierent re-
construction techniques and dierent 12 accelerated undersampling patterns. For
easier visualization, we have zoomed-in on a region that shows a variety of important
anatomical features that exhibit structure across a variety of dierent spatial scales,
including the brain stem, cerebellum, corpus callosum, and portions of the occipital
and parietal lobes. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 66
4.20 Maximum intensity projections of the 3D error images associated with reconstruc-
tions of Dataset 1 using dierent reconstruction techniques and dierent 12 accel-
erated undersampling patterns. The color scale has been normalized so that a value
of 1 corresponds to the maximum intensity of the image within the brain mask. . . . 67
x
4.21 Maximum intensity projections of the 3D error images associated with reconstruc-
tions of Dataset 2 using dierent reconstruction techniques and dierent 12 accel-
erated undersampling patterns. The color scale has been normalized so that a value
of 1 corresponds to the maximum intensity of the image within the brain mask. . . 68
4.22 Reconstructed k-space data obtained by applying dierent reconstruction methods
to a subsampled version of Dataset 1, using the 12 partial Fourier undersampling
pattern from Fig. 4.15. Both Wave-CAIPI and CS-Wave demonstrate signicant
errors in the high-frequency region of one side of k-space. This side of k-space was
not measured because of partial Fourier sampling. . . . . . . . . . . . . . . . . . . . 69
4.23 Maximum intensity projections of the 3D error images associated with reconstruc-
tions of Dataset 1 using 16 accelerated data. The color scale is normalized to match
Fig. 4.20. (This gure contains high resolution detail that may not print clearly on
certain printers. Readers may prefer to view the electronic version of this gure.) . 70
4.24 (top) NRMSE and (bottom) HFEN reconstruction error metrics as a function of the
(left) regularization parameter and (right) LORAKS matrix rank, for reconstruc-
tion of Dataset 1 with 12 accelerated VD CAIPI sampling. . . . . . . . . . . . . . . 71
5.1 Neural network representations of GRAPPA, RAKI, AC-LORAKS (with Landweber
iteration), and LORAKI. It should be noted that neither GRAPPA nor AC-LORAKS
were originally developed in the context of articial neural network models, but they
still admit neural network interpretations. . . . . . . . . . . . . . . . . . . . . . . . . 90
5.2 Representative reconstruction results for uniformly-undersampled T2-weighted data.
The top row shows reconstructed images for one slice in a linear grayscale, where the
gold standard image has been normalized to range from 0 (black) to 1 (white). The
bottom row shows error images with the indicated colorscale. NRMSE and SSIM
values are also shown below each image, with the best values highlighted in red. . . 99
5.3 Boxplots showing performance measures for (left column) T2-weighted data and
(right column) T1-weighted data. The results are compared using NRMSE (top
row) and SSIM (bottom row). . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 101
5.4 Error spectrum plots [1] corresponding to (left) the T2-weighted reconstruction re-
sults shown in Fig. 5.2 and (right) the T1-weighted reconstruction results shown in
Fig. 5.5. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 102
5.5 Representative reconstruction results for randomly-undersampled T1-weighted data.
The top row shows reconstructed images for one slice in a linear grayscale, where the
gold standard image has been normalized to range from 0 (black) to 1 (white). The
bottom row shows error images with the indicated colorscale. NRMSE and SSIM
values are also shown below each image, with the best values highlighted in red. . . 103
5.6 Reconstruction results for T2-weighted data with varying amounts of ACS data (with
xed total acceleration rate). Error images are shown using the same colorscale from
Fig. 5.2. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 104
xi
5.7 Reconstruction results for T1-weighted data with varying amounts of ACS data (with
xed total acceleration rate). Error images are shown using the same colorscale from
Fig. 5.5. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 105
5.8 AC-LORAKS and LORAKI reconstruction results for T2-weighted data with partial
Fourier sampling patterns. Error images are shown using the same colorscale from
Fig. 5.2. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 106
5.9 Evaluation of calibrationless reconstruction using synthetic ACS data. Error images
are shown using the same colorscale from Fig. 5.2. . . . . . . . . . . . . . . . . . . . 107
5.10 Comparison of GRAPPA, RAKI, and LORAKI with synthetic data training using
AC-LORAKS reconstruction. Error images are shown using the same colorscale from
Fig. 5.2. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 108
5.11 Evaluating the eects of dierent LORAKI network parameters on reconstruction
performance. The rst plot shows the eects of varying the number of hidden-
channel layersC while holding the kernel size xed at R
1
=R
2
= 5 and the number
of iterations xed at K = 5. The second plot shows the eects of varying R
1
, while
setting R
2
= R
1
and holding the other parameters xed at C = 64 and K = 5.
The nal plot shows the eects of varying K, while holding the other parameters
xed at C = 64 and R
1
= R
2
= 5. For reference, the NRMSE value for AC-
LORAKS reconstruction with optimized parameters is also shown (the AC-LORAKS
parameters are not varied in this plot). . . . . . . . . . . . . . . . . . . . . . . . . . . 109
6.1 Results for Landweber iteration. The plots show the total number of multiplications,
the normalized cost function value (normalized so that the initial value is 1), the
computation time in seconds, and the relative dierence between the solution from
the conventional method with matrices and solutions obtained with other methods. 126
6.2 Results for the conjugate gradient algorithm. The plots show the total number of
multiplications, the normalized cost function value (normalized so that the initial
value is 1), the computation time in seconds, and the relative dierence between the
solution from the conventional method with matrices and solutions obtained with
other methods. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 127
6.3 Results for the LSQR algorithm. The plots show the total number of multiplications,
the normalized cost function value (normalized so that the initial value is 1), the
computation time in seconds, and the relative dierence between the solution from
the conventional method with matrices and solutions obtained with other methods. 128
A.1 A gold-standard reference image (obtained from http://mr.usc.edu/download/data/)
along with three versions of this image that each present very dierent kinds of errors,
yet all share the same NRMSE value of 0.231 with respect to the gold standard. . . 148
A.2 Proposed ESPs corresponding to the \blurred," \noisy," and \ringing" images from
Fig. A.1. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 150
xii
A.3 (a) Magnitude and (b) phase images of the gold-standard T2-weighted reference
image. (c) 4.8-accelerated partial Fourier k-space sampling mask used for retro-
spective undersampling. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 152
A.4 (a) SENSE-TV and (b) SENSE-LORAKS reconstruction results from undersampled
data. (c) Fusion of SENSE-TV and SENSE-LORAKS reconstruction results. . . . . 153
A.5 ESPs for the SENSE-TV, SENSE-LORAKS, and their fusion shown in Fig. A.4. . . 154
B.1 Gold standard magnitude (left) and phase (right) images for the single-channel dataset.180
B.2 Sampling patterns used with single-channel data. . . . . . . . . . . . . . . . . . . . . 181
B.3 Reconstruction results from ex1.m. The top row shows results obtained with LO-
RAKS (using P LORAKS.m), while the bottom row shows results obtained with AC-
LORAKS (using AC LORAKS.m). . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 182
B.4 (a) Gold standard magnitude (top) and phase (bottom) images for each channel of
the multi-channel dataset. (b) The gold standard root-sum-of-squares combination
of the channels. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 183
B.5 Sampling patterns used with multi-channel data. . . . . . . . . . . . . . . . . . . . . 184
B.6 Reconstruction results from ex1.m. The top row shows results obtained with P-
LORAKS (using P LORAKS.m), while the middle row shows results obtained with
AC-LORAKS (using AC LORAKS.m), and the bottom row shows results obtained with
SENSE-LORAKS (using SENSE LORAKS.m). . . . . . . . . . . . . . . . . . . . . . . . 185
B.7 The eects of dierent choices of the LORAKS matrix type (C, S, or W), dierent
choices of the LORAKS neighborhood radius R, and not using virtual conjugate coils.187
B.8 The eects of dierent choices of the LORAKS matrix type (C, S, or W), dierent
choices of the LORAKS neighborhood radius R, and using virtual conjugate coils. . 188
B.9 The eects of the regularization parameter on AC-LORAKS reconstruction in ex5.m.190
B.10 The eects of the number of iterations on AC-LORAKS reconstruction in ex6.m. . . 191
xiii
Abbreviations
1D One Dimensional
2D Two Dimensional
3D Three Dimensional
ACS Autocalibrated Signal
ADMM Alternating Direction Method of Multipliers
AI Articial Intelligence
ANN Articial Neural Networks
CG Conjugate Gradient
CNN Convolutional Neural Networks
CS Compressed Sensing
CT Computed Tomography
DFT Discrete Fourier Transform
EPI Echo Planar Imaging
FT Fourier Transform
FFT Fast Fourier Transform
FOV Field of View
xiv
GD Gradient Descent
IFT Inverse Fourier Transform
MLE Maximum Likelihood Estimator
MM Majorize-Minimize
MNLS Minimum-Norm Least Squares
MR Magnetic Resonance
MRI Magnetic Resonance Imaging
MSE Mean Squared Error
NRMSE Normalized Root-Mean-Square Error
PET Positron Emission Tomography
PF Partial Fourier
RF Radio Frequency
RNN Recurrent Neural Networks
SGD Stochastic Gradient Descent
SMS Simultaneous Multi-Slice
SNR Signal-to-Noise Ratio
SPECT Single-Photon Emission Computerized Tomography
SSIM Structural Similarity Index
SVD Singular Value Decomposition
TV Total Variation
VCC Virtual Conjugate Coils
xv
Abstract
Magnetic resonance imaging (MRI) is a noninvasive biomedical imaging modality for visualizing
tissues inside the human body without ionizing radiation. It is a versatile diagnostic tool for
identifying anatomy (static and dynamic), functionality, and physiology of organs. Due to its
fundamental physical limitations, however, generating high-quality images is slow and expensive,
where researchers have been working for decades to overcome. Although there are many comple-
mentary approaches that can help achieve this goal, the main focus of this dissertation will be
on computational imaging methods that use mathematical modeling to obtain high-quality images
from incomplete data.
While there are many dierent kinds of computational imaging methods, this dissertation will
focus on methods that assume that MRI data is autoregressive and shift-invariant. To be specic,
"autoregressiveness" implies that an MRI data sample can be synthesized by taking linear/non-
linear combinations of its local neighbors, and "shift-invariance" indicates that such predictive
relationships are consistent across entire local MRI data regions. These conventional constraints
imply redundancies in MRI data and allow accelerated MRI scan by exploiting them.
In this study, we will rst present improved MRI reconstruction using LORAKS. LORAKS
(Low-rank modeling of local k-space neighborhoods) is a powerful linear shift-invariant autoregres-
sive reconstruction technique that is based on limited image support, slowly-varying phase, sparsity
in transform domains, and/or parallel imaging constraints, where such constraints are implemented
through structured low-rank matrix modeling. It will show many improved MRI applications us-
ing LORAKS, in simultaneous multislice imaging (SMS-LORAKS), sensitivity-encoded imaging
(SENSE-LORAKS), and highly-accelerated 3D imaging combined with Wave-CAIPI data acquisi-
tion (Wave-LORAKS).
xvi
It will then introduce LORAKI, a novel nonlinear shift-invariant autoregressive MRI reconstruc-
tion framework that is based on autocalibrated articial neural networks. The network structure
of LORAKI was adopted and motivated from an iterative algorithm solving LORAKS, comprising
recurrent neural networks (RNNs). As a result, LORAKI inherits many good features of LORAKS
while it also oers improved image qualities from its nonlinearities. It is a scan-specic network
that does not require external dataset, which can be useful and reliable in many scenarios where
large-scale training data are dicult to acquire.
Lastly, it will propose ecient iterative solutions to complex-valued nonlinear least-squares prob-
lems with mixed linear and antilinear operators. This study focuses on complex-valued least-squares
problems where the forward operator can be decomposed into linear and antilinear components.
While such formulations are nonlinear in its original complex-domain, previous literature addressed
them by mapping into equivalent real-valued linear least-squares and applying linear solvers (e.g.
Landweber iteration, Conjugate Gradient, LSQR). While this approach is valid, it may introduce
additional eorts in reformulation and ineciencies in computation. This work proposes theory and
computational methods that enable such problems to be solved iteratively using standard linear
least-squares tools, while retaining all of the complex-valued structure of the original inverse prob-
lem. The proposed algorithms can be widely applicable in solving many inverse problem scenarios,
including phase-constrained MRI reconstruction methods.
xvii
Chapter 1
Introduction
Since its introduction in the 1970s, magnetic resonance imaging (MRI) has become one of the most
prominent and productive biomedical imaging modalities. It is a noninvasive radiological diagnostic
tool that allows the visualization of high resolution images of tissues inside the body without the
risk of ionizing radiation. MRI has been widely exploited for diagnosing disease, understanding
organ anatomy and functionality, and even analyzing non-human related matters like the study of

uid dynamics.
Despite MRI's versatility, however, its slow data acquisition speed has been a long-standing
barrier for many MRI experiments for decades. Solving this problem is an essential but challenging
research topic because the lengthy process of data acquisition is fundamentally limited by both
physics and hardware. The long scan time creates trade-os between the scan speed, spatial/tem-
poral resolution, and signal-to-noise (SNR) ratio. It is also costly and inconvenient for patients.
There have been numerous approaches to overcoming these problems, e.g. adopting better
hardware such as higher and optimized coils; rened pulse sequencing; properly designed spatial
encoding, such as Cartesian and non-Cartesian trajectories; and advanced image reconstruction of
undersampled data based on signal processing theory. This study will mainly focus on the last
approach, accelerated MRI through image reconstruction. In particular, it will use the framework
of signal processing to investigate this research problem.
When a subject undergoes an MRI scan, data samples (k-space data) are acquired in the Fourier
domain through receiver coil arrays. The desired MR images can be obtained by acquiring sucient
data samples and then inverting the Fourier transform. However, in order to meet required image
1
Figure 1.1: Undersampled MRI reconstruction pipeline.
quality standards for specic criteria including eld of view (FOV), spatial/temporal resolution and
signal-to-noise ratio (SNR), many MRI experiments require a substantial number of data samples
which necessitates long data acquisition time accordingly. To overcome this limitation, the data
is often acquired in an undersampled manner, and missing data samples are recovered using MRI
reconstruction techniques. Specically, undersampled MRI reconstruction considers the following
sub-procedures, which are the scope of this study.
• Experiment design: Design of sampling trajectories (e.g. Cartesian, non-Cartesian), under-
sampling patterns (e.g. uniform, random, variable density ), and other acquisition-related
parameters (e.g
ip angle, repetition time) to maximize information from the limited scan
data.
2
• Forward model construction: Mathematical modeling of the physics of MRI data acquisition
describing the relationship between the desired image and the obtained MRI scan data.
• Inverse problem formulation: Construction of predication model that can estimate the desired
underlying MR image from the obtained data.
• Optimization/Computation algorithm: Selection of optimization algorithms that can solve
the constructed inverse problem and application of computational algorithms that eciently
save the computation time.
The main assumption we relied on for MRI reconstruction was shift-invariant autoregressive
characteristics in k-space [3{8, 23, 31, 59{64], where autoregression suggests that each k-space data
sample can be synthesized by taking combinations of its local neighbor samples, and shift-invariance
indicates that the synthesis rule is consistent across the entire local k-space regions. This conven-
tional method has been widely exploited in many MRI reconstruction studies. Such autoregressive
prediction can be either linear or nonlinear, where the linear autoregression is supported by well-
established theories [3, 4, 62], whereas the nonlinear autoregression has only been validated through
empirical evidence [55, 65, 66]. We will introduce improved MRI reconstruction methods exploiting
linear and nonlinear shift-invariant autoregression.
1.1 Problem Statement
The forward data acquisition in a single-coil MRI is represented as
d(k
m
) =
Z
(x)e
i2kmx
dx +n(k
m
); (1.1)
for m = 1; 2;:::;M, where (x) is the desired image to be estimated at spatial location x, d(k
m
)
is the data measured at themth k-space location k
m
, andn(k
m
) is the measurement noise (mostly
thermal Gaussian noise) at the mth k-space location with total M number of acquired k-space
data samples. This is a Fourier encoding where signal processing theory can be applied to recover
the underlying image. As a result, the data acquisition speed is intimately related to the Nyquist
3
rate, desired spatial / temporal resolution, and SNR. This means that the sucient number M of
distinct data samples should be acquired to satisfy those image constraints.
To make the analysis simple, let us consider a Nyquist-rate discretized formulation of Eq. (1.1),
d(k
m
) =
Q
X
q=1
e
i2kmxq
(x
q
) +n(k
m
); (1.2)
for m = 1; 2;:::;M and the Nyquist-rate number of samples Q. After the discretization, it can be
written compactly as a following matrix equation
d = E + n; (1.3)
where d2 C
M
is the vector of data samples corresponding to d(k
m
), E2 C
MQ
is the forward
matrix containing Fourier encoding and undersampling. 2 C
Q
is the vector of unknown voxel
coecients (x
q
), and n2C
M
is the vector of noise samples n
`
(k
m
). Because the thermal noise
in MRI is Gaussian and assuming the noise covariance is pre-whitened (the noise in single-coil
MRI is assumed to be independent and identically distributed Gaussian), the statistically-optimal
maximum likelihood estimator (MLE) for can be found by solving the simple least squares problem
^ = arg min
2C
Q
kE dk
2
2
; (1.4)
where the solution ^ exists and satises the normal equation,
E
H
E^ = E
H
d: (1.5)
However, ambiguity arises when the solution ^ is not unique, where the uniqueness requires that
the matrix E should have a full column rank. In practice, MRI data is often acquired in an
undersampled manner (i.e. M << Q) to reduce the scan time but it makes the problem to be
ill-posed. A commonly used minimum-norm least squares (MNLS) solution employing the Moore-
Penrose pseudoinverse ^ = (E
H
E)
y
E
H
d results in zero-padding in the k-space and aliasing in the
4
image. Our research has investigated how to recover missing k-space data samples and eliminate
image aliasing.
Approaches to overcoming the ill-posedness include increasing the number of acquired data
while not sacricing the overall scan time (e.g. parallel coil receiver arrays [21{25]), utilizing con-
straints or regularization derived from the prior knowledge of MRI (e.g. support [3], phase [29{32],
sparsity-based constraints [36{39]), and exploiting information learned from original data or ex-
ternal datasets (e.g. from machine learning/deep learning [49{55, 58]). In particular, this work
focused on shift-invariant autoregressive constraints, implying that k-space data are locally pre-
dictable [3{8, 23, 31, 59{64],
d(k
m
) =
X
`2A

`
d(k
m
`k); (1.6)
with a set of local neighbor indiciesA, the linear combination weights
`
, and k-space grid distance
k. There exist many reasons for such constraints to exist, such as limited image support [3], slowly-
varying phase [3, 31], transform domain sparsity [5, 62{64], and/or parallel imaging [7, 23, 59{61].
Extensive studies have been done based on those strategies. We will review some of those existing
studies and propose several novel methods.
1.2 Main Results
This study will rst introduce an improved MRI based on a linear shift-invariant autoregression
framework named LORAKS [3, 5{12]. LORAKS is a powerful constrained reconstruction framework
exploiting spatial support, smooth phase, sparsity, and/or parallel imaging constraints, which are
imposed by structured low-rank matrix modeling. We will show that many MRI applications can
be enhanced by using LORAKS in various aspects as follows:
• SMS-LORAKS [9]: This work addresses simultaneous multislice imaging (SMS). Unlike many
conventional SMS techniques requiring extra calibration, structured k-space sampling pat-
terns, coil sensitivity proles and particular slice RF encoding, SMS-LORAKS is
exible
enough to accommodate a number of dierent experimental variations: it supports both
single-channel and parallel imaging data, both calibration-based and calibrationless k-space
5
sampling trajectories, and Hadamard, Fourier, and random-phase non-Fourier encoding along
the slice dimension. The proposed framework was evaluated using real retrospectively under-
sampled k-space data. These evaluations validate the advantages of the proposed approach.
• SENSE-LORAKS [10]: Parallel imaging and partial Fourier acquisition are two classical ap-
proaches for accelerated MRI. Methods that combine these approaches often rely on prior
knowledge of the image phase, but the need to obtain this prior information can place prac-
tical restrictions on the data acquisition strategy. This work proposes and evaluates SENSE-
LORAKS, which enables combined parallel imaging and partial Fourier reconstruction with-
out requiring prior phase information. The proposed formulation is based on combining the
classical SENSE model for parallel imaging data with the LORAKS framework for MR image
reconstruction using low-rank matrix modeling. Previous LORAKS-based methods have suc-
cessfully enabled calibrationless partial Fourier parallel MRI reconstruction, but have been
most successful with nonuniform sampling strategies that may be hard to implement for
certain applications. By combining LORAKS with SENSE, we enable highly accelerated
partial Fourier MRI reconstruction for a broader range of sampling trajectories, including
widely-used calibrationless uniformly undersampled trajectories. Our empirical results with
retrospectively undersampled datasets indicate that when SENSE-LORAKS reconstruction is
combined with an appropriate k-space sampling trajectory, it can provide substantially bet-
ter image quality at high acceleration rates relative to existing state-of-the-art reconstruction
approaches.
• Wave-LORAKS [11]: Wave-CAIPI is a novel acquisition approach that enables highly-accelerated
3D imaging. This work investigates the combination of WaveCAIPI with LORAKS-based re-
construction to enable even further acceleration. Our previous LORAKS implementations ad-
dressed 2D image reconstruction problems. In this work, several recent advances in structured
low-rank matrix recovery were combined to enable large-scale 3D Wave-LORAKS reconstruc-
tion with improved quality and computational eciency. Wave-LORAKS was investigated
by retrospective subsampling of two fully-sampled Wave-encoded 3D MPRAGE datasets,
and comparisons were made against existing Wave reconstruction approaches. The results of
6
our experiments show that Wave-LORAKS can yield higher reconstruction quality with 16-
accelerated data than is obtained by traditional WaveCAIPI with 9-accelerated data. There
are strong synergies between Wave encoding and LORAKS, which enables Wave-LORAKS
to achieve higher acceleration and more
exible sampling compared to Wave-CAIPI.
Next, we will introduce LORAKI, a novel nonlinear shift-invariant autoregressive reconstruction
framework connecting shift-invariant autoregressive reconstruction with articial neural networks
(ANNs).
• LORAKI [13, 14]: We propose and evaluate a new magnetic resonance imaging (MRI) re-
construction method named LORAKI that trains an autocalibrated scan-specic recurrent
neural network (RNN) to recover missing Fourier (\k-space") data. Previous methods in-
cluding GRAPPA, SPIRiT, and AC-LORAKS assume that k-space data has a shift-invariant
autoregressive structure, and that the scan-specic autoregression relationships needed to
recover missing samples can be learned from fully-sampled autocalibration (ACS) data. Re-
cently, the structure of the linear GRAPPA method has been translated into a nonlinear
deep learning method named RAKI. RAKI uses ACS data to train an articial neural net-
work to interpolate missing k-space samples and often outperforms GRAPPA. In this work,
we apply a similar principle to translate the linear AC-LORAKS method (simultaneously
incorporating support, phase, and parallel imaging constraints) into a nonlinear deep learn-
ing method named LORAKI. Specically, LORAKI is built by taking the structure of the
AC-LORAKS reconstruction procedure (which was not originally developed in the context of
neural networks) and adding neural network features to it. Since AC-LORAKS is iterative
and convolutional, LORAKI takes the form of a convolutional RNN. This new architecture
can accommodate a wide range of sampling patterns, and even calibrationless patterns are
possible if synthetic ACS data are generated. The performance of LORAKI was evaluated
with retrospectively undersampled brain datasets. Results suggest that LORAKI can provide
improved reconstruction quality compared to other scan-specic autocalibrated reconstruc-
tion methods such as GRAPPA, RAKI, and AC-LORAKS.
7
In the last chapter, we will introduce ecient iterative solutions to complex-valued nonlinear
least-squares problems with mixed linear and antilinear operators. The suggested algorithms can
be widely applicable to any general complex-valued least-squares problems composed of mixed
linear and antilinear operators, such as solving phase-constrained MRI reconstruction methods [2{
4, 6, 15{20].
• We consider a setting in which it is desired to nd an optimal complex vector x2C
N
that
satisesA(x) b in a least-squares sense, where b2 C
M
is a data vector (possibly noise-
corrupted), andA() :C
N
!C
M
is a measurement operator. IfA() were linear, this reduces
to the classical linear least-squares problem, which has a well-known analytic solution as well
as powerful iterative solution algorithms. However, instead of linear least-squares, this work
considers the more complicated scenario whereA() is nonlinear, but can be represented as
the summation and/or composition of some operators that are linear and some operators
that are antilinear. Some common nonlinear operations that have this structure include
complex conjugation or taking the real-part or imaginary-part of a complex vector. Previous
literature has shown that this kind of mixed linear/antilinear least-squares problem can be
mapped into a linear least-squares problem by considering x as a vector in R
2N
instead of
C
N
. While this approach is valid, the replacement of the original complex-valued optimization
problem with a real-valued optimization problem can be complicated to implement, and can
also be associated with increased computational complexity. In this work, we describe theory
and computational methods that enable mixed linear/antilinear least-squares problems to be
solved iteratively using standard linear least-squares tools, while retaining all of the complex-
valued structure of the original inverse problem. An illustration is provided to demonstrate
that this approach can simplify the implementation and reduce the computational complexity
of iterative solution algorithms.
8
1.3 Organization of the Document
This dissertation is organized as follows. Chapter 2 covers conventional image reconstruction meth-
ods for MRI. Chapter 3 gives a review of shift-invariant autoregressive reconstruction methods and
the theory of LORAKS, a linear shift-invariant autoregressive reconstruction method through struc-
tured low-rank matrix modeling. Chapter 4 introduces accelerated MRI with LORAKS. Chapter
5 presents LORAKI, a novel autocalibrated recurrent neural network for nonlinear shift-invariant
autoregressive reconstruction. Chapter 6 proposes an iterative solution to nonlinear complex-valued
least-squares with mixed linear and antilinear operators. Chapter 7 concludes the dissertation.
9
This page intentionally left blank
Chapter 2
Conventional MRI Reconstruction
Methods
This chapter provides a review of existing and conventional MRI reconstruction techniques, which
are necessary to understand the later chapters in this dissertation. Section 2.1 covers parallel
imaging methods, Section 2.2 presents phase-constrained reconstruction, Section 2.3 gives sparsity-
based MRI reconstruction, and Section 2.4 illustrates deep learning methods for MRI
2.1 Parallel Imaging
Conventional approaches to accelerated MRI relied on k-space undersampling and parallel imag-
ing [21{25]. Parallel imaging exploits multiple receiver coils to acquire data, where they modulate
the underlying MR signal with dierent sensitivity proles. It is analogous to the lter bank: when
a signal is modulated with properly designed multiple dierent lters, it can be reconstructed from
the downsampling. SENSE [21, 22] and GRAPPA [23] are two most popular examples.
11
SENSE
SENSE utilizes known sensitivity proles of the receiver coils for the reconstruction. Sensitivity
proles can be estimated by the prescan, or from the fully sampled autocalibrated signal (ACS) [26].
The SENSE framework models parallel MRI data acquisition as [21, 22]
d
`
(k
m
) =
Z
s
`
(x)(x)e
i2kmx
dx +n
`
(k
m
); (2.1)
for m = 1; 2;:::;M and ` = 1; 2;:::;L, where (x) is the unknown image to be estimated (which
varies as a function of the spatial position x), d
`
(k
m
) is the data measured from the `th coil
at the mth k-space location k
m
, s
`
(x) is the sensitivity map for the `th coil, and n
`
(k
m
) is the
measurement noise from the`th coil at themth k-space location. We useM to denote the number
of k-space sampling locations and L to denote the number of channels.
Discretizing (x) in terms of Q dierent voxels leads to
d
`
(k
m
) =
Q
X
q=1
e
i2kmxq
s
`
(x
q
)(x
q
) +n
`
(k
m
); (2.2)
for m = 1; 2;:::;M and ` = 1; 2;:::;L, which can be written compactly as the matrix equation
d = E + n; (2.3)
where d is the length-LM vector of data samples from each coil d
`
(k
m
), E is the LMQ matrix
containing entries e
i2kmxq
s
`
(x
q
), is the length-Q vector of unknown voxel coecients (x
q
),
and n is the length-LM vector of noise samples n
`
(k
m
).
Assuming Gaussian thermal noise and that the inter-channel noise covariance has been whitened,
the SENSE approach nds the statistically-optimal maximum likelihood estimate for (when E
has full column rank) by solving the simple least squares problem [21, 22]
^ = arg min
2C
Q
kE dk
2
2
= (E
H
E)
1
E
H
d: (2.4)
12
The matrices E, E
H
, and E
H
E are generally too big to t within the memory limits of modern
computers, so are neither stored in memory nor directly inverted. Instead, the solution to Eq. (2.4)
can be found iteratively, using diagonal matrices, fast Fourier transforms (FFTs), and gridding (if
non-Cartesian trajectories are used) to eciently compute matrix-vector multiplications with E
and it's adjoint E
H
[22].
When k-space data is aggressively undersampled, Eq. (2.4) is frequently ill-posed, and it is
necessary to impose additional constraints to avoid severe noise amplication. A common approach
is to use Tikhonov regularization [27, 28]
^ = arg min
2C
Q
kE dk
2
2
+
T
kk
2
2
; (2.5)
which prefers reconstructions with smaller `
2
-norm power and has an eect of noise suppression.
The regularization parameter
T
can be chosen to adjust the relative contributions of the data
delity term and the regularization term to the nal reconstruction result.
GRAPPA
GRAPPA [23] is a direct linear interpolation method in k-space. When the undersampled k-space
data is acquired, the missing k-space data samples can be synthesized by taking linear combination
of its known k-space neighbor samples across the entire channels. Assuming multi-channel 2D
MRI, let d
`
(k
x
; k
y
) be a k-space sample at the k-space position (k
x
; k
y
) of the `th coil. Then, the
GRAPPA reconstruction of a missing k-space sample can be represented as
d
`
(k
x
; k
y
) =
C
X
c=1
X
(m;n)2N
w
`;m;n;c
d
c
(k
x
mk
x
; k
y
nk
y
); (2.6)
whereN is a set of indices indicating known local neighbors of d
`
(k
x
; k
y
) at every coils, w
`;m;n;c
is the GRAPPA kernel weights, and k
x
; k
y
are k-space spacing between the closest neighbor
samples. Because the GRAPPA kernel weightsw
`;m;n;c
are shared across the entire k-space region, it
is shift-invariant and therefore can be represented as a single convolution operation. The GRAPPA
13
kernel weights can be trained from autocalibrated signal (ACS) that is fully sampled regions in the
k-space.
2.2 Phase-constrained Imaging
Phase-constrained partial Fourier methods are based on the well-known characteristics that real-
valued images have conjugate-symmetric Fourier transforms [29{32]. As a result, it is unnecessary
to measure data from both sides of k-space for such images, which implies substantial potential ac-
celerations in data acquisition. In practice, MRI images are never real-valued, but instead typically
have slowly-varying phase in many cases. However, if the image phase is known in advance, it is
still possible to predict half of k-space from the other half [30{32], which still enables acceleration
by at most a factor of two. Therefore, phase constrained reconstruction methods consist of how to
measure the phase in advance and take advantage of conjugate symmetries in k-space.
Homodyne Reconstruction
Traditional partial Fourier methods [29, 30] take direct interpolation/extrapolation using conjugate
symmetries in k-space. Homodyne reconstruction [30] estimates the phase of image from the central
k-space region (that comprises low-frequency component of the image and has most of the slowly
varying phase information), which is used for the phase compensation. The conjugate symmetry
in k-space is applied to the phase-compensated images that is approximately real-valued.
Phase-constrained SENSE
Phase-constrained approaches have been previously combined with SENSE reconstruction [2, 15,
17, 20, 33, 34], and a typical approach is to use regularization to impose the phase constraints
[2, 15, 17, 20]. Let
^
(x
q
) denote the estimated phase for theqth image voxel(x
q
), be the image
we want to reconstruct (consists of (x
q
) entries), E be the SENSE encoding, d be the measured
14
data, and P be a QQ diagonal matrix of e
i
^
(xq )
values. Phase-constrained SENSE approaches
generally solve
^ = arg min
2C
Q
kE dk
2
2
+kImagfP
H
gk
2
2
; (2.7)
where ImagfP
H
g is the imaginary part of the phase compensated image P
H
and is another
regularization parameter. It is worth noting that if
^
(x
q
) is estimated accurately, then P
H
should
be real-valued and ImagfP
H
g 0.
Typically, the phase information
^
(x
q
) needed for phase-constrained SENSE is estimated by
acquiring additional phase calibration data from the low-frequency region of k-space.
Virtual Conjugate Coils
Virtual conjugate coils [8, 18, 20, 35] are another approach to partial Fourier imaging. In order to
exploit the correlation in conjugate positions in k-space, it generates virtual coils that is conjugate
symmetric to the original coils. Let d
`
(k
m
) be the k-space data from `th coil at the mth k-space
location k
m
, then its virtual coil data is constructed by
~
d
`
(k
m
) =d
`
(k
m
); (2.8)
and its virtual coil sensitivity prole satises
~ s
`
(x) =s
`
(x) exp(i2\(x)); (2.9)
wheres
`
(x) is the coil sensitivity prole at`th coil, and(x) is the underlying MRI image. Combin-
ing the virtual conjugate coils with original coils, VCC-SENSE and VCC-GRAPPA are performed
in the same way as in Eq. (2.4) and Eq. (2.6), respectively. It is worth noting that VCC-SENSE is
shown to be equivalent to the phase-constrained SENSE [20]. The concept of VCC-GRAPPA has
been extended to subsequent studies [8, 11, 35].
15
2.3 Sparsity-based Reconstruction
Compressed sensing [36{39] has demonstrated the feasibility to reconstruct a signal that is sparse in
some transform domains, from samples far less than the Nyquest rate when combined with properly
designed nonlinear reconstruction and incoherent sampling. Exact recovery (or recovery with some
error bounds) is guaranteed if certain conditions are satised, such as restricted isometry [40, 41],
incoherence [42{44], null-space [45, 46], or dual certicate [47, 48].
In MRI, one of the most common approaches is to use `
1
-regularization to promote transform-
domain sparsity (which is convex-relaxation of the original `
0
-formulation). Wavelet transform
and total variation are widely-used sparsifying transforms in MRI. For any forward model E, the
reconstruction can be formulated as
^ = arg min
2C
N
kE dk
2
2
+kR()k
1
; (2.10)
where R() represents the sparsity-promoting transforms, and is a user-selected regularization
parameter. This has been very successful over last 10+ years and is being implemented in recent
MR scanners.
2.4 Deep Learning in MRI Reconstruction
MRI reconstruction methods through deep learning [49{54] is becoming popular recent, and they
employ articial neural networks trained by dataset. Deep learning approach is dierent from the
conventional model-based reconstruction in the sense that it hardly assumes constraints designed by
humans' prior knowledge of MRI. Instead, it learns the prediction relationship automatically from
the data. In addition, neural network can capture nonlinearities in the prediction through nonlinear
activation functions, pooling, and other nonlinear components. There are two main approaches,
one [53, 54] is to train a very deep convolutional neural network using large number of dataset.
Once the network is trained, it enables very fast reconstruction, but it also necessitates large set of
training data. Another approach is to exploit small articial neural networks that can be trained
without the external database. RAKI [55] is an example, which is a three-layer articial neural
16
network that can be trained from the autocalibrated signal only. The experiment results indicate
that RAKI can be advantageous over conventional linear methods such as GRAPPA.
Recent trends in deep learning based approaches explore a hybrid of model-based and learning-
based reconstruction algorithms, aiming for their synergistic performance improvement. For exam-
ple, unrolled neural networks [50, 52, 56] embeds iterative algorithms of model-based reconstruction
to similar-form articial neural network models, where layers for data consistency steps are main-
tained but layers for regularization steps are subject to be trained. Plug-and-play methods [57, 58]
exploits iterative algorithms of model-based methods, but only replaces one or a few steps in the
iterative algorithms with articial neural networks (named as denoiser).
17
This page intentionally left blank
Chapter 3
Review of LORAKS
The goal of this chapter is to review the shift-invariant autoregressive reconstruction in MRI and
the corresponding structured low-rank matrix modeling. In particular, this chapter will focus on
the LORAKS technique as an example of a linear shift-invariant autoregressive MRI reconstruction
method, which is essential for understanding the following chapters.
3.1 Shift-invariant autoregressive reconstruction
Shift-invariant autoregressive properties have been one of the most popular constraint for under-
sampeld MRI reconstruction. As previous studies revealed, Fourier data acquired by MRI possesses
local correlation and therefore k-space samples can be predicted or synthesized from their adjacent
neighbor samples. A popular linear shift-invariant autoregression constraint indicates that, k-space
data ^ [n] dened in D dimensional grid n2Z
D
satises
^ [n] =
X
m2

m
^ [n m] 8n2Z; (3.1)
where is the set of indices for local k-space neighbor samplesand
m
is the combination weights.
Such autoregressive prediction is shift-invariant, as the combination weights
m
are shared for
all the k-space locations8n. This relationship implies redundancies in MRI data and enables
undersampled k-space data reconstruction. It is a convolutional relationship where there exist many
reasons why MRI data have such redundancies, some examples include that MRI data often satises
19
Figure 3.1: Autoregressive predictive relationship in k-space.
characteristics of limited support [3], slowly-varying phase [3, 31], parallel imaging [7, 23, 59{61],
and/or sparse in some transform doamin [5, 62{64].
To be more specic, let us consider an example of the limited support constraint. In many
cases, MR images [x] are support limited, i.e. they do not occupy the entire eld of view (FOV)
and have empty spaces. Therefore, there exist many functions g[x] that are dened outside the
support area, such that their element-wise multiplications are annihilated[x]g[x] 0. By taking
Fourier transform, the convolution relationships between the k-space data ^ [n] and the lter ^ g[n]
are derived (with an assumption that there exist band-limited lters ^ g[n]),
X
m2A
^ g[m] ^ [n m] 0 8n2Z; (3.2)
where A is the set of indices that ^ g[m] is non-zero. Equivalently, it can be represented as
^ [n] =
X
m2An0

^ g[m]
^ g[0]
^ [n m] 8n2Z; (3.3)
which is equivalent to Eq. (3.1) by
m
,
^ g[m]
^ g[0]
and , An0. Fig. 3.2 shows its graphical
representation. Please refer [4] for more details.
Although we only introduced linear autoregression in this section, there exist many methods
based on nonlinear autoregression [14, 55, 65, 66] (the predicative relationships are nonlinear).
20
Figure 3.2: Shift-invariant autoregressive relationship induced from limited image support and
corresponding structured low-rank matrix modeling.
Empirical results indicate that the nonlinear methods can also be promising in many MRI recon-
struction scenarios.
3.2 LORAKS
1
For the sake of simplicity and clarity, our description will focus solely on the theoretical justica-
tions, mathematical notation, and problem formulation conventions from our previous LORAKS
works [3, 5, 7, 8]. However, it is worth noting that there are several recent MR image reconstruc-
tion methods that are highly related to LORAKS and use very similar concepts and methodology
[61, 63, 64], but which have been derived and/or designed in somewhat dierent ways. Despite our
emphasis on LORAKS in this work, we would like to point out that some of the capabilities and
features we describe for LORAKS are also shared by these other methods.
LORAKS [3, 5{8, 10{12] is a powerful and
exible constrained reconstruction framework that
can integrate classical image support constraints, smooth phase constraints, sparsity constraints,
and parallel imaging constraints into a single unied subspace constraint. The basic foundation
of LORAKS is that, if one or more of the aforementioned constraints is applicable to a given
image, then trained reconstruction framework that can integrate classical image support constraints,
1
The text and gures in this chapter have been previously published in [67], and are copyright of the John Wiley
& Sons, Inc.
21
smooth phase he fully-sampled Fourier data of that image should be linearly predictable. In this
context, linear predictability implies that one k-space data sample can be accurately predicted as a
weighted linear combination of neighboring points in k-space. Furthermore, the linear combination
weights will be shift-invariant, meaning that if they can be learned somehow, then they can be
applied to interpolate or extrapolate unmeasured k-space data.
Linear-prediction relationships are nothing new in the constrained MR image reconstruction
literature. For example, linear prediction has been exploited in some of the earliest constrained
reconstruction papers in the eld [62, 68], and is the cornerstone of modern widely-used image
reconstruction methods like GRAPPA [23] and SPIRiT [59]. Oftentimes, existing linear-prediction
approaches make the assumption that some form of prior calibration information is available so
that the interpolation weights can be learned in advance. These types of approaches are powerful,
though have been less useful in scenarios for which the collection of calibration information would
be burdensome.
However, it's been known for a long time that linear predictability is intimately related to the
existence of structured low-rank matrices [62]. This fact opens new doors for MR image reconstruc-
tion, especially when combined with the explosion of recent theory and methods that demonstrate
the possibility of reconstructing low-rank matrices from sparsely sampled data [69]. LORAKS is
one such MR image reconstruction method, which uses the principles of linear predictability to con-
struct low-rank matrices, while simultaneously using the principles of structured low-rank matrix
recovery to infer missing data.
Compared to classical linear-prediction methods, LORAKS has been shown to have several
notable features:
• While linear-prediction relationships can exist in the data for various reasons (e.g., support,
phase, sparsity, parallel imaging, etc.), using LORAKS does not require specic prior knowl-
edge about which of these constraints are applicable. Instead, LORAKS attempts to learn
all of the relevant local linear-prediction relationships that may exist in k-space (regardless
of their source). LORAKS imposes all of these learned relationships simultaneously, while
remaining agnostic to the original source of these relationships. This allows LORAKS to be
22

exible and adaptable enough to work across a range of dierent image reconstruction scenar-
ios without the need for substantial adaptations in the problem formulation. This is distinct
from classical linear prediction methods, which usually choose a single linear-prediction re-
lationship to enforce based on prior information about a single specic constraint that is
expected to be compatible with the desired image.
• The use of low-rank matrix completion means that LORAKS is compatible with a wide range
of sampling patterns. In particular, LORAKS does not require calibration information, and
can successfully reconstruct images even in cases where the data sampling is highly uncon-
ventional. For example, previous work has demonstrated that LORAKS can successfully
reconstruct images from \silly" sampling patterns that have been selected for aesthetic pur-
poses (e.g., based on the logo of our institution [5, 8]). While we don't recommend the use of
\silly" sampling, these new capabilities oer exciting new opportunities for improving the de-
sign of k-space sampling patterns. In addition, this feature enables the use of linear-prediction
reconstruction in scenarios where acquiring calibration information is undesirable.
• We have frequently observed that LORAKS-based reconstruction outperforms sparsity-based
CS reconstruction [3, 5, 7, 8]. We also note that LORAKS is just regularization, and can
be synergistically combined with other regularization penalties for further improvements in
image reconstruction quality, although this generally comes at the expense of increased com-
putational burden.
• While LORAKS is compatible with calibrationless acquisition, it can also easily be used
with calibration information when it is available [6, 10, 12] for even further improvements in
reconstruction quality and computational eciency.
Without loss of generality, LORAKS reconstruction can be formed as follows,
^ = arg min
2C
N
kE dk
2
2
+J(P()); (3.4)
where the operatorP() : C
N
! C
QS
takes an image as input and constructs a structured
matrix (typically with Hankel and/or Toeplitz structure) out of the Nyquist-sampled k-space data
23
corresponding to this input, and J() :C
QS
!R is a cost function that penalizes matrices with
large rank. As before, denotes a regularization parameter. E is an any forward model, is an
ideal image to be reconstructed.
Because all the works introduced in the following sections are based on the S matrix, it will
introduce how S matrix is constructed (please refer [3] for other matrices). For simplicity, we will
assume for this description that(x) is a 2D image and that the discretization(x
q
),q = 1; 2;:::;Q
is dened on a rectilinear grid of sampling locations. Let F be the QQ unitary FFT matrix
such that f = F is the length-Q vector of Nyquist-grid samples of the Fourier transform of .
We will use the notation f[p] to denote the value of f at the grid point specied by the integer
vector p2Z
2
. The S matrix can be constructed from the vector f according to a linear operator
P
S
() :C
Q
!R
2T2N
R
dened by
P
S
(f),
2
6
6
4
S
r
+
(f) S
r

(f) S
i
+
(f) + S
i

(f)
S
i
+
(f) + S
i

(f) S
r
+
(f) + S
r

(f)
3
7
7
5
; (3.5)
where the matrices S
r
+
; S
r

; S
i
+
; S
i

2R
TN
R
have elements,

S
r
+
(f)

tq
= Realff[p
t
m
q
]g; (3.6)

S
r

(f)

tq
= Realff[p
t
m
q
]g; (3.7)

S
i
+
(f)

tq
= Imagff[p
t
m
q
]g; (3.8)

S
i

(f)

tq
= Imagff[p
t
m
q
]g: (3.9)
In this expression, the vectors m
q
,q = 1; 2;:::;N
R
are the full set of integer vectors within a radius
of R from the origin (i.e.,fm2Z
2
:kmk
2
2
Rg), and the vectors p
t
2Z
2
, t = 1; 2;:::;T are the
full set of integer vectors from a rectilinear Nyquist-sampled k-space grid. It should be observed
that the matrices S
r
+
; S
r

; S
i
+
; S
i

are all convolution matrices, and that each row from these
matrices is constructed from a local neighborhood of N
R
points in k-space. The neighborhood
radius R is a user-selected parameter that plays a similar role to the kernel size in GRAPPA and
24
related parallel imaging methods [23, 26, 60, 61]. It should also be noted that S
r
+
and S
i
+
are
formed using data from the opposite side of k-space relative to S
r

and S
i

, as would be expected
since the S matrix captures linear dependence relationships between opposite sides of k-space. It
has been shown that the S matrix will have low rank if the continuous image (x) has limited
spatial support or slowly-varying phase [3, 5].
P-LORAKS [6] extends LORAKS to parallel imaging data, leveraging the same inter-channel
linear dependence relationships used in previous Fourier-domain parallel imaging methods [23, 26,
60, 61]. Specically, if f
1
, f
2
;:::; f
L
are the Nyquist-sampled k-space data fromL dierent channels
in a parallel imaging experiment and f
P
denotes the length-LQ vector concatenating them, then
P-LORAKS constructs a modied S matrix using the linear operatorP
PS
() :C
LQ
!R
2T2LN
R
dened by
P
PS
(f
P
), [P
S
(f
1
);P
S
(f
2
);:::;P
S
(f
L
)]; (3.10)
i.e., the concatenation of the single-channel S matrices. The single-channel S matrices will have low
rank if the image has limited support or slowly-varying phase, and the concatenation will have even
better low-rank characteristics because of correlations between dierent channels. Importantly, this
enables parallel imaging reconstruction without requiring knowledge of the sensitivity maps and
without requiring a fully-sampled k-space calibration region.
The P-LORAKS matrix reduces to the LORAKS matrix when the number of channels L = 1,
so we will use P-LORAKS notation without loss of generality. In much of our previous work
[3, 5, 6, 9, 70], LORAKS/P-LORAKS image reconstruction was performed by minimizing
^
f
P
= arg min
f
P
2C
LQ

Uf
P
d

2
2
+
S

P
PS
(f
P
)L
r

P
PS
(f
P
)

2
F
; (3.11)
where U2C
LMLQ
is a block diagonal matrix representing the k-space subsampling operation for
each channel, andkk
F
denotes the Frobenius norm. The rst term in this expression is a standard
maximum likelihood data delity term, while the second term is a nonconvex regularization penalty
that encourages the S matrix to have low rank. The regularization parameter
S
controls the trade-
o between these two terms. The operatorL
r
: R
2T2LN
R
! R
2T2LN
R
computes the optimal
25
rank-r approximation of its argument (using truncation of the singular value decomposition [71]),
where r is a user-selected rank parameter.
The optimization problem in Eq. (3.11) is a nonconvex approach to imposing rank constraints
that incorporates prior knowledge of the matrix rank. We have previously shown [72] that this
kind of formulation can oer substantial performance advantages over alternative convex relaxation
approaches that are popular for imposing rank constraints [69]. This type of formulation is not only
found in previous LORAKS work [3, 5, 6, 9, 70], but has also proven successful for other structured
[61, 64] and unstructured [73] low-rank matrix completion problems in MRI.
26
Chapter 4
Accelerated MRI using LORAKS:
Linear Shift-invariant Autoregressive
Reconstruction through Structured
Low-Rank Matrix Modeling
4.1 Simultaneous Multislice Imaging (SMS-LORAKS)
4.1.1 Background and Notation
1
We assume that Q dierent slices are simultaneously excited (with the qth slice centered along
thez-axis at positionz
q
), and that for fully-encoded SMS MRI, each line of k-space is measuredP
dierent times under dierent slice encoding settings. In addition, we assume that data is measured
through an array of L dierent receiver coils. In this case, the fully-sampled SMS data acquisition
can be modeled as
d
p
`
(k
m
) =
Z Q
X
q=1
s
`
(x;z
q
)(x;z
q
)e
i(2kmx+pqm)
dx; (4.1)
form = 1;:::;M,p = 1;:::;P , and` = 1;:::;L. In this expression, d
p
`
(k
m
) denotes the measured
data from the`th coil,thepth repetition, and themthk-space location k
m
= (k
m
x
;k
m
y
); the variable
x = (x;y) denotes the within-slice spatial coordinates; s
`
(x;z
q
) denotes the sensitivity prole
1
The text and gures in this chapter have been previously published in [9], and are copyright of the IEEE.
27
of the `th coil for the qth slice; (x;z
q
) denotes the unknown image from the qth slice (to be
reconstructed); and the
pqm
parameters describe the eects of slice encoding for the qth slice,
the pth repetition, and the mth k-space sample. In an SMS experiment, the
pqm
parameters are
achieved by using either the gradient system or spatially-selective RF excitation to modulate the
phase of each individual slice for each measured line of k-space. This allows the dierent slices to
be disentangled from one another, even though their signals are superposed in (4.1). Common slice
encoding choices include:
1. Two-slice Hadamard encoding [74], in which Q = P = 2. In this case, the rst repetition
is acquired with both slices having the same phase (i.e., e
i
1qm
= +1 for all m), while the
second repetition is acquired with the slices having opposite phase (i.e., e
i
11m
= +1 and
e
i
12m
=1 for all m).
2. Fourier encoding [75], in which P = Q, and e
ipqm
= e
i2
(p1)(q1)
Q
for all m. Fourier
encoding is equivalent to two-slice Hadamard encoding when Q = 2.
3. Random RF encoding [76], in which
pqm
for each line of k-space is drawn independently
from the random uniform distribution over [0; 2].
In the conventional case where the s
`
(x;z
q
) are assumed known, (4.1) can be discretized as the
set of coupled matrix equations
d
`
=
Q
X
q=1
UP
q
FS
`

q
; (4.2)
for ` = 1;:::;L, where d
`
is the vector of measured data samples d
p
`
(k
m
) from the `th coil;
q
is
the vector of image coecients from the qth slice; S
`
is a diagonal matrix containing sensitivity
prole information for the `th coil; F is a Fourier transform matrix; P
q
contains thee
ipqm
phase
modulation parameters for each measurement from theqth slice; and U is an undersampling matrix
(if desired). The standard approach to reconstructing SMS data is equivalent to solving (4.2) in a
least-squares sense for the desired slice images (x;z
q
) [74, 77{79].
In this work, we consider the more dicult situation in which the s
`
(x;z
q
) are assumed to be
unknown. For this case, instead of trying to directly reconstruct the Q images (x;z
q
), our new
28
goal is to accurately reconstruct the LQ images
`
(x;z
q
), s
`
(x;z
q
)(x;z
q
). This is a standard
choice made by methods like GRAPPA [23], and is viable because it is relatively straightforward
to combine together images of the same anatomy from multiple coils (e.g., through a root sum-of-
squares (rSoS) procedure).
This modication also leads to a \simpler" data model:
d
p
`
(k
m
) =
Z Q
X
q=1

`
(x;z
q
)e
i(2kmx+pqm)
dx;
=
Q
X
q=1
e
ipqm
~
`
(k
m
;z
q
);
(4.3)
where
~
`
(k
m
;z
q
) =
Z

`
(x;z
q
)e
i2kmx
dx (4.4)
is the ideal Fourier k-space data for the `th channel. The discretized matrix-based version of (4.3)
can be represented as
d
`
=
Q
X
q=1
UP
q
~
q
`
; (4.5)
for ` = 1;:::;L, where ~
q
`
is the vector of sampled ~
`
(k
m
;z
q
) values for the `th coil and the qth
slice. Notice that while (4.2) and (4.5) have certain similarities, (4.5) is likely to be underdetermined
even if (4.2) was overdetermined. This is because the number of measurements is the same in both
equations, but there are L-times more unknowns in (4.5) than there are in (4.2). As a result,
additional constraints will generally be necessary to reconstruct the ~
q
`
in (4.5). In this work, we
will use LORAKS and P-LORAKS constraints [3, 7].
29
4.1.2 SMS-LORAKS
In our proposed SMS-LORAKS framework, image reconstruction is performed by solving
min
~
L
X
`=1

d
`

Q
X
q=1
UP
q
~
q
`

2
`
2
+
C
Q
X
q=1
J
C
(P
Ctot
(~
q
)) +
S
Q
X
q=1
J
S
(P
Stot
(~
q
)):
(4.6)
The rst term appearing in (4.6) is a least-squares data delity term corresponding to the data
model from (4.5). Notice that the data delity penalty includes coupling between dierent slices, but
there is no coupling between dierent coils. The nal two terms appearing in (4.6) are regularization
penalties that are designed to impose low-rank matrix constraints for the C
q
tot
and S
q
tot
matrices,
respectively. In contrast to the data delity penalty, there is no coupling between dierent slices, but
there is coupling between the dierent coil images from the same slice because of the P-LORAKS
constraints.
The regularization penalties in (4.6) are the same as used in previous LORAKS work. Speci-
cally, for some user-dened rank constraints r
C
and r
S
, we dene
J
C
(C
tot
) = min
rank(T)r
C
kC
tot
Tk
2
F
J
S
(S
tot
) = min
rank(V)r
S
kS
tot
Vk
2
F
:
(4.7)
Note that the penalties are equal to zero if the ranks of C
tot
and S
tot
are respectively smaller than
r
C
and r
S
, and are non-zero if the matrices have higher rank. See [3, 7] for additional discussion
of these penalties.
4.1.3 Experiments and Results
The following subsections present representative single-channel and multi-channel SMS-LORAKS
results.
30
(a) Original (b) Sampling (c) MN (HAD)
(d) MN (RAND) (e) Proposed (HAD) (f) Proposed (RAND)
Figure 4.1: Single-channel reconstruction results.
Single-Channel Results Complex k-space data from a real T2-weighted multislice (sequential,
not SMS) imaging experiment was used for simulation. Experimental data was acquired using a
12-channel array, but we only used data from one of the channels. SMS acquisition was simulated
from individual slices by linearly combining the data from each slice. We simulated SMS with
Q =P = 2 for both Hadamard (HAD) and random RF (RAND) slice encoding. Data subsampling
was simulated by randomly discarding phase encoding lines, for a 1.54 acceleration. Note that
the sampling scheme does not include a fully-sampled autocalibration region, and we are unaware
of any previous method for reconstructing data sampled in this way. Therefore, we have not
compared against other methods. Reconstruction results are shown in Fig. 4.1. For reference, we
have also included minimum-norm (MN) reconstructions (similar to zero-padded conjugate-phase
reconstruction for traditional MRI acquisition). SMS-LORAKS is capable of producing excellent
reconstructions of these images, despite the challenging data acquisition scheme.
Multi-Channel Results This simulation used all channels from the 12-channel data described
previously. We again used P = Q = 2 with Hadamard (HAD) and random RF (RAND) slice
encoding. We used a random partial-Fourier acquisition, which only samples from 5/8ths of k-
space, while leaving the remaining 3/8ths unsampled. This allows higher-density sampling than
31
(a) Original (b) Sam-
pling
(c) MN
(HAD)
(d) MN
(RAND)
(e) Proposed
(HAD)
(f) Proposed
(RAND)
(g) JTV
(HAD)
Figure 4.2: 12-channel reconstruction results.
could be achieved if we sampled all of k-space. We randomly measured from the sampled region
of k-space for a total of 3 acceleration. Reconstruction results are shown in Fig. 4.2, after
combining images using rSoS. Due to space constraints, only a single slice is shown. For reference,
we compare against joint total variation (JTV) reconstruction [80], a calibrationless parallel MRI
method based on joint sparsity. Results show that the proposed SMS-LORAKS approach yields
accurate reconstruction in this case, while more traditional approaches like JTV have diculties
with this challenging data acquisition scheme.
Fig. 4.3 shows a comparison between dierent sampling schemes and choices of r
C
and r
S
(S-based results are shown with
C
=0, and C-based results are shown with
S
=0). We observe
that there is not much dierence in quality between Hadamard and random RF slice encoding in
this case, though the dierent acquisition schemes have dierent optimal rank parameters. The
reconstruction error is relatively insensitive to r
C
and r
S
over a reasonably large range (e.g., for
random RF slice encoding, the reconstruction quality does not change much for 45 r
C
70 or
45r
S
80).
4.1.4 Conclusions
This work proposed and evaluated SMS-LORAKS, a novel low-rank matrix modeling framework for
reconstructing SMS MRI data. Unlike existing SMS approaches, SMS-LORAKS can reconstruct
both single- and multi-channel SMS images from novel,
exible calibrationless acquisition schemes
and high acceleration factors. We expect this approach to be useful in a range of MRI applications
32
Figure 4.3: Error (in the `
2
-norm) versus rank (r
C
and r
S
, respectively, for C and S) for dierent
slice encoding schemes.
where substantial improvements in imaging speed are required, especially when SMS-LORAKS is
combined with other constraints.
4.2 Parallel and Phase-constrained Imaging with Sensitivity Encoding
(SENSE-LORAKS)
4.2.1 Theory
2
The reconstruction approach proposed in this paper is a fusion of the SENSE and P-LORAKS
approaches, and takes advantage of their complementary advantages. Using the SENSE data
model is benecial, because it allows use of prior information about the coil sensitivity maps
that wouldn't normally be taken into account by P-LORAKS. Specically, the SENSE model can
be interpreted as imposing inter-coil linear dependence relationships in k-space [26] in a similar
way to P-LORAKS. However, P-LORAKS solves the much harder problem of estimating these
linear dependence relationships from undersampled data, while SENSE derives these inter-coil
relationships in a much simpler way by using prescan data. Although P-LORAKS can successfully
estimate the inter-coil relationships when data is sampled appropriately, there are certain common
structured k-space trajectories for which it is very challenging for P-LORAKS to learn the inter-
coil relationships successfully. For example, with uniform undersampling (i.e., the most common
2
The text and gures in this chapter have been previously published in [10], and are copyright of the John Wiley
& Sons, Inc.
33
undersampling trajectory provided by modern commercial MRI scanners), no two adjacent lines
of k-space are ever sampled simultaneously, which makes it nearly impossible to learn the inter-
coil k-space relationships between adjacent lines. Using SENSE modeling in combination with P-
LORAKS removes this ambiguity. Using the P-LORAKS model is also benecial to SENSE, because
it can incorporate support and phase constraints into SENSE reconstruction without requiring prior
information about the image support or phase, without requiring support or phase calibration data,
and without making any assumptions about the k-space sampling pattern.
Our proposed SENSE-LORAKS image reconstruction is obtained by solving the following op-
timization problem
^ = arg min
2C
Q
kE dk
2
2
+
T
kk
2
2
+
S
kP
PS
(G)L
r
fP
PS
(G)gk
2
F
; (4.8)
where the matrix G2C
LQQ
is constructed as
G =
2
6
6
6
6
6
6
6
6
6
6
6
4
FB
1
FB
2
.
.
.
FB
L
3
7
7
7
7
7
7
7
7
7
7
7
5
; (4.9)
and the matrices B
`
, ` = 1; 2;:::;L are QQ diagonal matrices containing the samples of the
sensitivity proles s
`
(x
q
) for each channel. Intuitively, the matrix-vector multiplication G uses
the image and the sensitivity maps to generate a simulated set of Nyquist-sampled multi-channel
k-space data, as required by P-LORAKS to construct the S matrix.
Similar to our previous LORAKS work [3, 5, 6, 70], we solve the nonlinear optimization problem
in Eq. (4.8) using an iterative majorize-minimize algorithm that is guaranteed to monotonically
decrease the cost function. Convergence to the global optimum is not guaranteed because of the
nonconvexity of the cost function, though our previous empirical experience suggests that the
34
algorithm frequently converges to good local optima [3, 5, 6, 70] without requiring a sophisticated
initialization. The detailed steps of our algorithm are described below.
1. Set iteration numberi = 0, and initialize ^
(0)
. For all results shown in this paper, we initialize
with the simple (and likely sub-optimal) SENSE reconstruction obtained by solving Eq. (2.5).
2. Compute S
(i)
=P
PS
(G^
(i)
), and its rank-r approximation L
(i)
=L
r
(S
(i)
).
3. Solve the least squares problem
^
(i+1)
= arg min
2C
Q
kE dk
2
2
+
T
kk
2
2
+
S

P
PS
(G) L
(i)

2
F
=

E
H
E +
T
I +
S
G
H
P
PS
G

1

E
H
d +
S
G
H
P

PS
(L
(i)
)

:
(4.10)
In this expression, I is theQQ identity matrix,P

PS
() is the adjoint of theP
PS
() operator,
and P
PS
is the LQLQ matrix representation of the operatorP

PS
(P
PS
()) :C
LQ
!C
LQ
.
As described in [3, 5, 70], P
PS
is a simple diagonal matrix that is easy to calculate based on
the structure of the S matrix.
Similar to SENSE [22], Eq. (4.10) can be solved iteratively using the conjugate gradient
algorithm, using diagonal matrices, FFTs, and gridding to compute fast matrix-vector mul-
tiplications without directly constructing or inverting large matrices.
4. Set i =i + 1. Iterate steps 2 - 4 until convergence.
4.2.2 Methods
SENSE-LORAKS was implemented as described in the previous section, using a k-space neighbor-
hood radius ofR = 3. As will be discussed later, reconstruction performance was not very sensitive
to the choice of the rank parameter r or the regularization parameters
S
and
T
. As a result, we
used coarsely-tuned sampling-independent (suboptimal) parameters for SENSE-LORAKS, though
performed much more thorough parameter tuning for the alternative reconstruction methods we
compare against.
35
a
b
c
d
e
f
Figure 4.4: Gold standard (a,c,e) magnitude and (b,d,f) phase images used for evaluation. Images
correspond to the (a,b) TSE, (c,d) MPRAGE, and (e,f) EPI datasets.
The proposed SENSE-LORAKS reconstruction was compared against four alternative recon-
struction techniques:
• SENSE [21, 22, 27, 28]. Image reconstruction was performed using Eq. (2.5), with the
Tikhonov regularization parameter
T
optimized for each sampling conguration to achieve
the smallest possible normalized root-mean-squared error (NRMSE).
• Phase-Constrained SENSE [2, 15, 17, 20]. Image reconstruction was performed using
Eq. (2.7). The phase regularization parameter
P
and the Tikhonov regularization parameter

T
were optimized for each dataset and sampling conguration to achieve the smallest possible
NRMSE.
36
• SENSE+TV [81, 82]. Image reconstruction was performed using
^ = arg min
2C
Q
kE dk
2
2
+
TV
kk
TV
; (4.11)
where the total variation (TV) normkk
TV
computes the `
1
-norm of the image gradient (as
estimated using nite dierences). The TV regularization parameter
TV
was optimized for
each dataset and sampling conguration to achieve the smallest possible NRMSE.
• P-LORAKS [6]. Image reconstruction was performed using Eq. (3.11). The rank parameter
r and the LORAKS regularization parameter
S
were optimized for each dataset and sampling
conguration to achieve the smallest possible NRMSE.
Performance of these methods was evaluated on three dierent retrospectively undersampled
datasets. To highlight the generality and
exibility of SENSE-LORAKS, we have chosen a diverse
set of datasets, with variations in the number of receiver channels, acquisition matrix size, image
contrast, and pulse sequences:
• T2-Weighted Turbo Spin Echo (TSE) Data
Fully sampled data was acquired from a healthy subject using a 2D multislice T2-weighted
TSE sequence on a 3T Siemens Tim Trio scanner with a 12 channel headcoil. Imaging pa-
rameters included a 256 mm 187 mm FOV, 256187 Cartesian acquisition grid, 1 mm
slice thickness, TE/TR = 89 ms/13500 ms. For simplicity, we perform subsampling and
reconstruction on a single slice from this dataset. The gold standard SENSE reconstruc-
tion (obtained using fully-sampled data) is shown in Fig. 4.4(a,b). For SENSE-LORAKS
reconstruction with this dataset, we used the coarsely-tuned parameters
S
= 1:9 10
3
,

T
= 10
3
, and r = 40 for all acceleration rates and all undersampling patterns.
• T1-Weighted MPRAGE Data
Fully sampled data was acquired from a stroke patient using a 3D MPRAGE sequence on
a 3T Siemens Tim Trio scanner with a 12 channel headcoil. The headcoil was operated in
combined mode, producing 4 channels of output k-space data. Imaging parameters included
37
a 256 mm 256 208 mm FOV, 256256208 Cartesian acquisition grid,
ip angle =
10

, TI/TE/TR = 800 ms/3.09 ms/2530 ms. A 1D inverse Fourier transform was applied
along the frequency encoding dimension (superior-inferior) to decouple the reconstructions
of the individual 2D slices. For simplicity, we perform subsampling and reconstruction on
a single slice from this dataset. The gold standard SENSE reconstruction (obtained using
fully-sampled data) is shown in Fig. 4.4(c,d). For SENSE-LORAKS reconstruction with this
dataset, we used the coarsely-tuned parameters
S
= 3:4 10
3
,
T
= 10
2
, and r = 55 for
all acceleration rates and all undersampling patterns.
For this dataset, while subsampling along both phase encoding dimensions would be feasible
and improve reconstruction results for all methods [83], we only show results with common
forms of 1D acceleration.
• T2-Weighted EPI data
Fully sampled data was acquired from a healthy subject using a single-shot 2D spin-echo
EPI sequence on the Siemens 3T Connectom scanner with a 64 channel headcoil. Imaging
parameters included a 200 mm 200 mm FOV, 100100 Cartesian acquisition grid, 2 mm
slice thickness, TE/TR = 80 ms/7400 ms. Discrepancies between even and odd lines were
compensated using zero- and rst-order phase corrections, and ramp sampled data was resam-
pled onto the Nyquist grid prior to reconstruction. For simplicity, we perform subsampling
and reconstruction on a single slice from this dataset. The gold standard SENSE recon-
struction (obtained using fully-sampled data) is shown in Fig. 4.4(e,f). For SENSE-LORAKS
reconstruction with this dataset, we used the coarsely-tuned parameters
S
= 4:8 10
3
,

T
= 10
3
, and r = 50 for all acceleration rates and all undersampling patterns.
To preserve the characteristics of this EPI dataset, this dataset was only retrospectively un-
dersampled using uniform undersampling strategies. In addition, retrospective undersampling
was restricted such that the readout gradient polarities alternate between adjacent lines.
Sensitivity maps were computed for each dataset by applying ESPIRiT [26] to a 3232 Nyquist-
sampled grid of calibration data.
38
Our retrospective undersampling experiments explored several sampling schemes that are rep-
resentative of modern sampling design strategies. Specically, we considered the following four
k-space sampling patterns for a range of dierent acceleration factors:
• Uniform sampling (Uniform)
This standard sampling scheme uses evenly-spaced phase encoding lines that are spread across
both sides of k-space. This is the sampling scheme used by standard Cartesian SENSE [21],
and is associated with coherent aliasing.
• Uniformly undersampled partial Fourier (Uniform PF)
Like Uniform sampling, this sampling scheme uses evenly spaced phase encoding lines. How-
ever, instead of sampling both sides of k-space, Uniform PF sampling spreads the phase
encoding lines over 5/8ths of the relevant portion of k-space (encompassing one full half of
k-space plus the low-frequency region from the other side). Measuring 5/8ths of k-space is
a typical strategy for partial Fourier methods [30{32]. For the same total number of phase
encoding lines, Uniform sampling PF will have a smaller sampling interval (higher sampling
density) than Uniform sampling within the measurement region.
• Randomly undersampled partial Fourier (Random PF)
This sampling scheme is identical to Uniform PF, except that the phase encoding lines are
randomly spaced according to a 1D Poisson disk distribution [84]. This sampling pattern has
less coherence than the previous sampling patterns, and as a result, may be better suited to
sparsity-based and low-rank based methods like SENSE+TV and P-LORAKS.
• Autocalibrated partial Fourier (AC PF)
This sampling scheme is similar to Uniform PF, except that the sampling density is adjusted
so that the center of k-space is sampled uniformly and densely, while high-frequency k-space
is sampled uniformly but less densely. This is a conventional sampling approach for phase-
constrained SENSE [2, 15, 17, 20], since the highly-sampled center of k-space can be used to
generate a high-quality phase estimate. For our implementation, we used the central 32 phase
encoding lines as the calibration region. Rather than fully sampling this region (which would
39
.
GoldStandard
. .
SENSE
(+Tikhonov)
.
P-LORAKS
.
SENSE-
LORAKS
.
SENSE-
LORAKS
. .
Sampling
. . .
.
Magnitude
. .
Uniform
.
Uniform
.
Uniform
.
UniformPF
. .
Reconstruction
. . .
.
Phase
.
Error
. . .
. . . .
0.41436
.
0.77902
.
0.22898
.
0.18119
Figure 4.5: Comparison between (a) SENSE with conventional Uniform sampling, (b) P-LORAKS
with conventional Uniform sampling, (c) SENSE-LORAKS with conventional Uniform sampling,
and (d) SENSE-LORAKS with Uniform PF sampling for the TSE data with 5.1 acceleration.
Images for other acceleration factors are shown in Supporting Figs. S1 and S2, while NRMSE values
are plotted in Fig. 4(a). Reconstructed images are shown using a linear grayscale (normalized
so that image intensities are in the range from 0 to 1), while error images are shown using the
indicated colorscale (which ranges from 0 to 0.25 to highlight small errors). NRMSE values are
shown underneath each reconstruction, with the best NRMSE values highlighted with red.
lead to extreme undersampling of the high-frequency content), we uniformly undersampled
the calibration region by a factor of 2 (i.e., a total of 16 phase encoding lines were measured).
The full calibration region was recovered using standard SENSE reconstruction with minimal
loss of accuracy.
For all reconstructions, performance was quantied by computing the NRMSE with respect to
the gold standard fully-sampled reconstruction. We also visualized the reconstructed images and
associated error maps to provide insight into the spatial distribution of error.
40
.
GoldStandard
. .
SENSE
(+Tikhonov)
P-LORAKS
SENSE-
LORAKS
SENSE-
LORAKS
. .
Sampling
.
Magnitude
. .
Uniform Uniform Uniform UniformPF
. .
Reconstruction
.
Phase
.
Error
. . . .
0.39847
.
0.90864
.
0.34155
.
0.23450
Figure 4.6: Comparison between (a) SENSE with conventional Uniform sampling, (b) P-LORAKS
with conventional Uniform sampling, (c) SENSE-LORAKS with conventional Uniform sampling,
and (d) SENSE-LORAKS with Uniform PF sampling for the MPRAGE data with 5.0 acceleration.
Images for other acceleration factors are shown in Supporting Figs. S3 and S4, while NRMSE values
are plotted in Fig. 4(b). Reconstructed images are shown using a linear grayscale (normalized
so that image intensities are in the range from 0 to 1), while error images are shown using the
indicated colorscale (which ranges from 0 to 0.25 to highlight small errors). NRMSE values are
shown underneath each reconstruction, with the best NRMSE values highlighted with red.
4.2.3 Results
Performance Comparisons using TSE and MPRAGE Data Figures 4.5-4.7 and Support-
ing Figs. S1-S4 show comparisons of the proposed SENSE-LORAKS method against various other
reconstruction methods for the TSE data and the MPRAGE data. Using these results to com-
pare SENSE-LORAKS with conventional SENSE reconstruction, we observe, as expected [21],
that conventional SENSE reconstruction with conventional Uniform sampling performs well at low
acceleration factors, though performance degrades rapidly as the acceleration factor increases. As
similarly expected, it is also observed that P-LORAKS does not work very eectively with Uniform
sampling. In comparison, applying SENSE-LORAKS to the same Uniform data yields improved
41
Acceleration Rate
3 3.5 4 4.5 5 5.5 6 6.5 7
NRMSE
0.1
0.2
0.3
0.4
0.5
0.6
0.7
Turbo Spin Echo
SENSE+Tikhonov (Uniform)
P-LORAKS (Uniform)
SENSE-LORAKS (Uniform)
SENSE-LORAKS (Uniform PF)
a
Acceleration Rate
3 3.5 4 4.5 5 5.5 6 6.5 7
NRMSE
0.1
0.2
0.3
0.4
0.5
0.6
0.7
MPRAGE
SENSE+Tikhonov (Uniform)
P-LORAKS (Uniform)
SENSE-LORAKS (Uniform)
SENSE-LORAKS (Uniform PF)
b
Figure 4.7: Plots of the reconstruction NRMSE as a function of the acceleration rate for the (a)
TSE data (corresponding images were shown in Fig. 2 and Supporting Figs. S1 and S2) and (b)
MPRAGE data (corresponding images were shown in Fig. 3 and Supporting Figs. S3 and S4).
reconstruction performance. We believe that this improvement should be expected, since SENSE-
LORAKS is able to incorporate additional phase and support constraints that are not modeled by
SENSE.
Using these same gures to compare SENSE-LORAKS with conventional Uniform sampling
against SENSE-LORAKS with Uniform PF sampling, we observe that conventional Uniform sam-
pling can slightly outperform Uniform PF sampling at low acceleration factors, while Uniform PF
sampling is substantially better than conventional Uniform sampling at high acceleration factors.
We believe that this result should also be expected. Specicially, at low acceleration factors, SENSE
with Uniform sampling is a well-conditioned inverse problem, while partial Fourier sampling has
the potential to incorrectly estimate any high resolution information that does not completely sat-
isfy the k-space symmetry constraints. On the other hand, we know that the reconstruction errors
are dominated by the ill-posedness of the problem at high acceleration rates. In these settings, we
expect that the lower sampling density obtained with conventional Uniform sampling will make the
SENSE problem more ill-posed than it is for Uniform PF sampling with the same acceleration rate.
Figures 4.8-4.10 and Supporting Figs. S5-S8 present a comparison between SENSE-LORAKS
against phase-constrained SENSE, SENSE+TV, and P-LORAKS for the TSE and MPRAGE data.
42
.
Phase-
Constrained
SENSE
SENSE-
LORAKS
Sampling
.
ACPF ACPF
Recon Error
a 0.20515 0.19710
P-LORAKS SENSE-
LORAKS
RandomPF RandomPF
b 0.21510 0.19122
SENSE+TV SENSE- LORAKS
Uniform Uniform
c 0.31753 0.22898
SENSE- LORAKS
UniformPF
d 0.18119
Figure 4.8: Comparison of SENSE-LORAKS against (a) phase-constrained SENSE with AC PF
sampling, (b) P-LORAKS with Random PF sampling, and (c) SENSE+TV with conventional
Uniform sampling for the TSE dataset with 5.1 acceleration. Reconstructions obtained using
SENSE-LORAKS with Uniform PF sampling are shown in (d). Images for other acceleration
factors are shown in Supporting Figs. S5 and S6, while NRMSE values are plotted in Fig. 7(a). The
reconstructed images are displayed using a linear grayscale (normalized so that image intensities
are in the range from 0 to 1). The error images are displayed using the indicated colorscale
(which ranges from 0 to 0.25 to highlight small errors). NRMSE values are shown underneath each
reconstruction, with the best NRMSE values highlighted with bold text in each sampling pattern.
The smallest NRMSE values for a given acceleration rate are indicated in red.
For each comparison, sampling patterns were chosen that were most applicable to the method
being compared against. For the TSE data (Figs. 5 and 7(a) and Supporting Figs. S5 and S6),
we observe that when SENSE-LORAKS is compared against the other methods using the same
sampling pattern, SENSE-LORAKS consistently outperformed phase-constrained SENSE and P-
LORAKS, and outperformed SENSE+TV at all but the lowest acceleration factor. Among dierent
SENSE-LORAKS sampling schemes, Uniform PF sampling always produced the best performance
at high acceleration factors. Consistent with our previous results, conventional Uniform sampling
yielded the best performance at low acceleration factors, which we again hypothesize is related
43
.
Phase-
Constrained
SENSE
SENSE-
LORAKS
Sampling
.
ACPF ACPF
Recon Error
a 0.24953 0.23678
P-LORAKS SENSE-
LORAKS
RandomPF RandomPF
b 0.77941 0.32671
SENSE+TV SENSE- LORAKS
Uniform Uniform
c 0.33305 0.34155
SENSE- LORAKS
UniformPF
d 0.23450
Figure 4.9: Comparison of SENSE-LORAKS against (a) phase-constrained SENSE with AC PF
sampling, (b) P-LORAKS with Random PF sampling, and (c) SENSE+TV with conventional
Uniform sampling for the MPRAGE dataset with 5.0 acceleration. Reconstructions obtained
using SENSE-LORAKS with Uniform PF sampling are shown in (d). Images for other acceleration
factors are shown in Supporting Figs. S7 and S8, while NRMSE values are plotted in Fig. 7(b). The
reconstructed images are displayed using a linear grayscale (normalized so that image intensities
are in the range from 0 to 1). The error images are displayed using the indicated colorscale
(which ranges from 0 to 0.25 to highlight small errors). NRMSE values are shown underneath each
reconstruction, with the best NRMSE values highlighted with bold text in each sampling pattern.
The smallest NRMSE values for a given acceleration rate are indicated in red.
to dierences in the relatively well-posed nature of the SENSE problem for conventional Uniform
sampling at low acceleration factors.
For the MPRAGE data (Figs. 4.9 and 4.10(b) and Supporting Figs. S7 and S8), SENSE-
LORAKS again consistently outperformed phase-constrained SENSE and P-LORAKS. However,
dierent from the TSE case, SENSE-LORAKS was consistently outperformed by SENSE+TV
when applied to conventional Uniform data. Despite this small dierence, the MPRAGE results
are largely consistent with the TSE results from the TSE data. Specically, the best overall
performance was still achieved by SENSE-LORAKS with Uniform PF or AC PF sampling at
the higher acceleration factors, while SENSE+TV combined with Uniform sampling had the best
performance at the lower acceleration factors. It should be noted that, while SENSE-LORAKS
with AC PF sampling very slightly outperforms SENSE-LORAKS with Uniform PF sampling at
44
Acceleration Rate
3 3.5 4 4.5 5 5.5 6
NRMSE
0.1
0.15
0.2
0.25
0.3
0.35
0.4
Turbo Spin Echo
Phase-Constrained SENSE (AC PF)
SENSE-LORAKS (AC PF)
P-LORAKS (Random PF)
SENSE-LORAKS (Random PF)
SENSE+TV (Uniform)
SENSE-LORAKS (Uniform)
SENSE-LORAKS (Uniform PF)
a
Acceleration Rate
3 3.5 4 4.5 5 5.5 6
NRMSE
0.1
0.15
0.2
0.25
0.3
0.35
0.4
MPRAGE
Phase-Constrained SENSE (AC PF)
SENSE-LORAKS (AC PF)
P-LORAKS (Random PF)
SENSE-LORAKS (Random PF)
SENSE+TV (Uniform)
SENSE-LORAKS (Uniform)
SENSE-LORAKS (Uniform PF)
b
Figure 4.10: Plots of the reconstruction NRMSE as a function of the acceleration rate for the (a)
TSE data (corresponding images were shown in Fig. 5 and Supporting Figs. S5 and S6) and (b)
MPRAGE data (corresponding images were shown in Fig. 6 and Supporting Figs. S7 and S8).
an acceleration factor of 6, the Uniform PF sampling scheme may still be preferred in practical
applications because it yields smaller errors within the brain.
Comparing the error images shown in Figs. 4.5, 4.6, 4.8, and 4.9 and Supporting Figs. S2, S4, S6,
and S8, it can also be observed that the majority of SENSE-LORAKS errors are spatially-localized
near the skull in these reconstructions, while errors within the brain parenchyma are generally
much smaller in magnitude. We believe that these errors are related to the relatively fast phase
variations associated with extracranial lipid signal, which violate the smooth phase assumptions
used by SENSE-LORAKS and other phase constrained methods.
Performance Comparisons using EPI Data One benet of the Uniform PF and conven-
tional Uniform sampling strategies is that they have uniformly spaced phase encoding lines, and
are therefore easily used with EPI and balanced SSFP acquisitions. Figure 4.11 shows results based
on undersampled EPI data using these sampling patterns. The results in this case are consistent
with the previous TSE and MPRAGE results. Specically, SENSE-LORAKS outperforms conven-
tional SENSE when both methods are applied to conventional Uniform sampling data. In addition,
Uniform PF sampling can have substantially better performance than conventional Uniform sam-
pling, particularly at high acceleration factors. Notably, SENSE-LORAKS with 7:7-accelerated
45
.
SENSE
Uniform
SENSE-LORAKS
Uniform
SENSE-LORAKS
UniformPF
. .
5.0× 5.0× 4.5× . .
0.15659 0.12210 0.11782
. .
6.7× 6.7× 7.7× . .
0.29720 0.19941 0.16498
Figure 4.11: Comparison between (left) SENSE with conventional Uniform sampling, (middle)
SENSE-LORAKS with conventional Uniform sampling, and (right) SENSE-LORAKS with Uniform
PF sampling for the EPI data. The left columns show reconstructed images using a linear grayscale
(normalized so that image intensities are in the range from 0 to 1), while the right columns show
error images using the indicated colorscale (which ranges from 0 to 0.25 to highlight small errors).
NRMSE values are shown underneath each reconstruction.
Uniform PF sampling is able to achieve similar reconstruction performance to conventional SENSE
with 5-accelerated conventional Uniform sampling.
Parameter Selection Using SENSE-LORAKS requires the selection of three dierent recon-
struction parameters: the low-rank regularization parameter
S
, the rank constraint r, and the
Tikhonov regularization parameter
T
. Many strategies have been previously proposed for au-
tomatically selecting regularization parameters for general regularized reconstruction problems
[28, 47, 85], and while these are applicable to SENSE-LORAKS, they are often computation-
intensive. For practical applications, it is important to know how much ne-tuning of the parame-
ters is necessary to achieve good performance.
Figure 4.12 provides an analysis of reconstruction performance as the parameters
S
,r, and
T
are systematically varied. This evaluation was conducted using the TSE and MPRAGE datasets
46
a
Rank
10 20 30 40 50 60 70 80 90 100
NRMSE
0.1
0.15
0.2
0.25
0.3
0.35
Reconstruction Error versus Rank
TSE
TSE (masked)
MPRAGE
MPRAGE (masked)
b
λ
T
10
-6
10
-4
10
-2
10
0
10
2
NRMSE
0
0.2
0.4
0.6
0.8
1
Reconstruction Error versus λ
T
TSE
TSE (masked)
MPRAGE
MPRAGE (masked)
c
λ
S
10
-6
10
-4
10
-2
10
0
10
2
NRMSE
0.1
0.2
0.3
0.4
0.5
0.6
0.7
Reconstruction Error versus λ
S
TSE
TSE (masked)
MPRAGE
MPRAGE (masked)
Figure 4.12: SENSE-LORAKS reconstruction performance as a function of (a) rankr, (b) Tikhonov
regularization parameter
T
, and (c) LORAKS regularization parameter
S
. Except where param-
eter values are being explicitly changed, the r,
T
, and
S
parameters were set to their default
values as described in the Methods section.
with Uniform PF sampling and 3 acceleration. Performance was quantied using NRMSE. In ad-
dition, we also computed NRMSE after applying a spatial mask that preserves the brain parenchyma
while excluding the extracranial regions. The extracranial regions are typically less interesting
for MRI brain studies, and as described previously, are also the spatial locations where SENSE-
LORAKS tends to have the largest errors.
As seen from Fig. 4.12, reconstruction errors are relatively stable across a wide range of
S
,
r, and
T
values. Rank parameters between 45{60 and
T
and
S
parameters between 10
3
and
10
2
all produced similar NRMSE values in all cases. Therefore, ne-tuning of the reconstruction
47
parameters may not be very important. This is also consistent with our previous observation that
SENSE-LORAKS is very eective compared to other reconstruction methods across a range of dif-
ferent sampling patterns and acceleration rates, even though the SENSE-LORAKS reconstruction
parameters were not individually tailored to each sampling conguration.
While our proposed formulation includes both Tikhonov regularization and LORAKS-based
regularization, it is worth noting that the LORAKS constraint is primarily responsible for the
improvements in image quality. The eect of Tikhonov regularization is to slightly improve the
stability of the SENSE-LORAKS solution by reducing noise sensitivity, which is similar to its
use in conventional SENSE [27, 28]. A comparison of dierent regularization strategies can be
seen in Supporting Fig. S9. While the impact of Tikhonov regularization on SENSE-LORAKS
reconstruction is not very dramatic, we have chosen to include it because it does lead to a small
improvement in image quality, while not having a major negative impact on parameter tuning or
computational complexity.
4.2.4 Discussion
Our results conrmed that SENSE-LORAKS with Uniform PF sampling can have substantial ad-
vantages over previous sampling and reconstruction methods, especially at high acceleration factors
where the reconstruction error becomes dominated by the ill-posedness of the SENSE problem. For
simplicity, our results were based on simple 2D k-space datasets with 1D Cartesian undersampling.
However, it should be noted that the SENSE-LORAKS constraints are easily applied to other
experiment types (including experiments with non-Cartesian sampling [22], experiments with non-
negligible eld inhomogeneity [86], 3D imaging with 2D or 3D undersampling [83], simultaneus
multi-slice imaging [77, 78, 87{89], etc.) by making the appropriate modications to the data
acquisition model matrix E.
The LORAKS regularization constraint is also
exible, and is easily adapted to dynamic imaging
[3, 63], simultaneous multi-slice imaging [9], or scenarios where the image has limited support or
smoothly-varying phase in a known transform domain [5, 63, 64]. In addition, one of the benets
of the LORAKS framework is that the constraints are applied using regularization techniques. As
48
. . .
SENSE+TV SENSE- LORAKS
SENSE- LORAKS
withTV
. .
.
UniformPF
. . .
0.20609 0.18119 0.16962
Figure 4.13: Comparison between SENSE+TV, SENSE-LORAKS, and SENSE-LORAKS with TV
sampling for TSE data with Uniform PF sampling and 5.1 acceleration. Other acceleration factors
are shown in Supporting Fig. S10. The top row shows reconstructed images using a linear grayscale
(normalized so that image intensities are in the range from 0 to 1), while the bottom row shows
error images using the indicated colorscale (which ranges from 0 to 0.25 to highlight small errors).
NRMSE values are shown underneath each reconstruction, with the best NRMSE value highlighted
in red.
a result, it is easy to combine LORAKS constraints with other forms of regularization to achieve
even higher reconstruction performance [3, 6]. As an example of this, it is possible to combine
SENSE-LORAKS with TV regularization by solving
^ = arg min
2C
Q
kE dk
2
2
+
S
kP
PS
(G)L
r
fP
PS
(G)gk
2
F
+
TV
kk
TV
: (4.12)
Figure 4.13 and Supporting Fig. S10 compare reconstruction performance for SENSE-LORAKS
with TV against SENSE-LORAKS and SENSE+TV for Uniform PF sampling. Results indicate
that the combination of SENSE-LORAKS with the TV constraint improves performance, as would
be expected because the dierent constraints use dierent and synergistic prior information about
the structure of typical MRI images.
It is important to keep in mind that, similar to previous LORAKS work [3, 5{7, 9, 70], the
successful application of SENSE-LORAKS will depend on the phase, support, and parallel imaging
49
characteristics of the measured data. Images with looser elds of view, slowly varying phase, and
well-designed array coils will be easier to accelerate than images with tighter elds of view, rapidly
varying phase, or array coils with suboptimal congurations. These dierent factors in
uence the
matrix rank in predictable ways [3, 5], and are important to keep in mind when designing an
accelerated MRI experiment for use with SENSE-LORAKS reconstruction.
It is also worth mentioning that, while our quantitative evaluation of SENSE-LORAKS was
based on NRMSE, NRMSE is a coarse measure of image quality that hides information about the
spatially-varying nature of the reconstruction errors and does not necessarily correlate with the
context-specic tasks that reconstructed images are often used to perform [90] (e.g., pathology de-
tection, parameter quantication, image registration, tissue segmentation/parcellation, morphom-
etry, etc.). Showing the reconstructed images and error maps, as we have done, helps address a
portion of these issues. Nevertheless, as we have previously described [3, 6, 76, 90], we strongly
believe that nonlinear reconstruction methods like SENSE-LORAKS should be tested thoroughly
on context-specic tasks before they are deployed for routine use in practical applications.
As with all advanced constrained reconstruction methods, computation is an important practical
issue for SENSE-LORAKS. In this work, we were primarily interested in investigating the potential
usefulness of SENSE-LORAKS, and have not yet put any eort into optimizing computation speed.
As a result, our preliminary Matlab-based implementation is relatively slow, especially when run
on a simple desktop computer (Intel Xeon E5-1620 3.7 GHz quad core CPU processor and 16 GB
memory). We have observed that reconstruction times depend on various factors, including conver-
gence criteria, the number of channels L, the number of samples M, the size of the neighborhood
size R used by LORAKS, the matrix rank constraint r, and the acceleration factor. For example,
using our current implementation, it took 10-15 minutes to reconstruct the 12-channel TSE data at
low acceleration rates (i.e., 2 or 3), while it could take up to 30 minutes to reconstruct the same
image with highly-accelerated data (i.e., 6 or 7). The computation time also increases with the
number of coils. For example, it took around 4 hours to reconstruct the 64-channel EPI data with
7:7 acceleration. There are many opportunities to improve computation speed dramatically, since
the optimization algorithm has not been designed for fast convergence speed and the computing
50
hardware has not been optimized. In addition, the use of coil compression techniques [91] could
also lead to substantially faster reconstructions. We believe that exploring faster implementations
is a promising area for future development.
One of the key features of the proposed SENSE-LORAKS approach is that it is
exible enough
to enable the use of phase-constrained reconstruction with highly accelerated calibrationless partial
Fourier EPI trajectories. While the partial Fourier acquisition is useful for enhancing reconstruc-
tion performance because it enables increased sampling density relative to conventional uniform
sampling, partial Fourier acquisitions also have other benets in the context of EPI. For example,
for xed resolution and bandwidth, the use of partial Fourier acquisition can also be used to de-
crease the minimum echo time and mitigate the eects of relaxation during the readout [92]. We
believe that such features make SENSE-LORAKS even more attractive for this context.
4.2.5 Conclusion
This work proposed and investigated the SENSE-LORAKS approach for constrained MRI recon-
struction. Compared to existing methods, SENSE-LORAKS enables partial Fourier reconstruction
without prior phase information, and is very
exible with respect to the data sampling scheme.
Specically, SENSE-LORAKS can be used with both calibration-based and calibrationless sampling
patterns, with both uniform and nonuniform sampling patterns, and with both partial Fourier and
full Fourier sampling patterns. This
exibility provides new opportunities for sampling trajectory
optimization, and also means that SENSE-LORAKS can be more compatibile with classical fast
imaging methods that use partial Fourier EPI and balanced SSFP pulse sequences. In addition,
due to the simple regularization-based nature of the SENSE-LORAKS formulation, the approach is
easily combined with other useful constraints like image sparsity. Our results suggest that SENSE-
LORAKS can provide state-of-the-art reconstruction performance, particularly at high acceleration
rates, and we believe that this new approach has the potential to enhance data acquisition and
image reconstruction performance across a wide spectrum of practical application scenarios.
51
4.3 Highly accelerated 3D imaging with Wave-encoding (Wave-
LORAKS)
4.3.1 Theory
3
This section provides a detailed review of the relevant previous literature (to give appropriate
context and rationale for our reconstruction setup), and also describes the implementation choices
we have made for the specic version of Wave-LORAKS investigated in this work.
Wave-CAIPI Wave-CAIPI is a recent data acquisition technique that enables parallel imaging
at high acceleration factors with minimal g-factor penalty [93{96]. Like 2D CAIPIRINHA [83],
Wave-CAIPI for volumetric 3D imaging chooses its phase encoding positions to lie on a 2D lattice
in k-space, while the readout gradient is used to encode the third dimension. Lattice sampling of
k-space leads to periodic aliasing on a reciprocal lattice in the image domain, and careful design
of this aliasing pattern can help to reduce the resulting parallel imaging g-factor in an accelerated
acquisition [83].
However, distinct from 2D CAIPIRINHA (in which the readout follows a straight-line path
through k-space), Wave-CAIPI employs a 3D corkscrew-shaped non-Cartesian readout trajectory.
The choice to use this kind of readout was inspired by the earlier bunched phase encoding approach
[97], and has the eect of spreading aliasing across all 3 spatial dimensions. This makes more
ecient use of the additional spatial encoding provided by 3D coil sensitivity proles, and leads to
even further reduction in the parallel imaging g-factor.
The reduced g-factor means that Wave-CAIPI has major implications for highly-accelerated
MRI. For example, Polak et al. [96] have demonstrated 9 accelerated high-resolution MPRAGE
imaging
4
with an average g-factor of only 1.06 at 3T. This enables an acquisition lasting only 72
seconds, which is a substantial improvement over a fully-sampled acquisition which would take
nearly 11 minutes to acquire.
3
The text and gures in this chapter have been previously published in [67], and are copyright of the John Wiley
& Sons, Inc.
4
The MPRAGE sequence [98] is an inversion-prepared sequence that uses a gradient echo readout train. This
sequence is often used in modern neuroscience studies because it can provide a high-quality high-resolution 3D
T1-weighted image with excellent contrast between gray and white matter in the brain.
52
Similar to traditional SENSE parallel imaging reconstruction [21, 22], most of the previous
Wave-CAIPI implementations [93{96] perform image reconstruction using a least-squares formula-
tion:
^ m = arg min
m2C
N
kEm wk
2
2
; (4.13)
where m2 C
N
is the vector of voxel values for the unknown 3D image to be estimated, E2
C
MN
represents the data acquisition model (including the eects of sensitivity encoding and non-
Cartesian Fourier encoding), and w2C
M
is the vector of measured data samples collected from
all receiver coils. Due to system imperfections in the MRI scanner (e.g., gradient imperfections
that perturb the nominal k-space trajectory), it has proven useful to use a simple autocalibrated
point-spread function model when constructing the E matrix instead of trusting the acquisition to
be faithful to the nominal k-space trajectory [95, 96].
Beyond these least-squares reconstruction approaches, Wave-CAIPI has also previously been
combined with an existing advanced reconstruction approach that uses `
1
-regularization to pro-
mote transform-domain sparsity [99, 100]. Because the use of `
1
-regularization was inspired by
compressed sensing (CS) [38], this approach to Wave reconstruction is known as CS-Wave [99, 100].
The version of CS-Wave we will compare against later in this paper is formulated as [100]
^ m = arg min
m2C
N
kEm wk
2
2
+TV (m); (4.14)
whereTV () represents the standard 3D total variation (TV) regularization penalty that encourages
the reconstructed image to have sparse edges, and is a user-selected regularization parameter.
CS-Wave enables higher acceleration factors than Wave-CAIPI. Instead of using the strict lattice
sampling pattern used in Wave-CAIPI, CS-Wave is generally used with a partially random phase
encoding pattern [99, 100].
Choice of LORAKS Matrix There are several dierent possible ways of choosing the struc-
tured matrix construction operatorP() in Eq. (3.4). For single-channel data, early LORAKS
work described three dierent construction methods, which were called the C, G, and S matrix
constructions [3, 70]. The C matrix is a simple convolution-structured matrix that can be used
53
to impose image-domain support constraints and can be viewed as a single-channel version of the
matrix appearing in PRUNO/SAKE/ESPIRiT [26, 60, 61], while the G and S matrices have more
complicated structure that allows them to impose both support and smooth phase constraints si-
multaneously [3, 70]. In multi-channel datasets, parallel imaging constraints can be additionally
included by stacking the single-coil C, G, or S matrices for each coil next to one another in a larger
matrix [7, 26, 60, 61]. Other matrix constructions are also possible that impose sparsity constraints
[5, 62{64].
In our experience, the parallel imaging version of the S matrix (imposing support, smooth phase,
and parallel imaging constraints simultaneously) consistently leads to the best image reconstruction
performance in most cases [3, 5, 7, 8], and would be a natural choice to use for Wave-CAIPI.
However, a computationally ecient version of S-based LORAKS was not available at the time
we originally performed the research reported in this paper.
5
Since computational eciency is
important for large-scale 3D Wave-LORAKS, we focus in this work on the parallel imaging version
of the C matrix, which is substantially easier to manipulate. While the C matrix is normally
incapable of incorporating smooth phase constraints, it is possible to use phase constraints with
the C matrix by using the concept of virtual conjugate coils [8, 18, 35]. In our experience, combining
the C matrix with virtual conjugate coils does not lead to substantially worse image reconstruction
error values than using the S matrix [8, 11].
Similar to previous work [8, 18, 35], our use of virtual conjugate coils is motivated by the
complex conjugation property of the Fourier transform. Let s
`
(x) denote the spatially-varying coil
sensitivity for the `th channel and m(x) represents the desired image, and let G
`
(k) denote the
Fourier transform of s
`
(x)m(x). The conjugation property of the Fourier transform implies that if
we construct virtual conjugate coil dataD
`
(k) according toD
`
(k),G
`
(k) where the bar denotes
complex conjugation, then this virtual coil data will be equal to the Fourier transform of ~ s
`
(x)m(x),
which can be rewritten in terms of the standard SENSE model as ~ s
`
(x)m(x) for some \virtual"
coil sensitivity prole ~ s
`
(x) dened by
~ s
`
(x) =s
`
(x) exp(i2\m(x)); (4.15)
5
We have subsequently worked out the details of computationally ecientS-based LORAKS [11].
54
where\m(x) is the phase of m(x). Constructing virtual conjugate coils in this way allows us
to double the eective number of channels we've measured data from (increasing the amount of
spatial encoding and the amount of information content that can potentially be extracted from
the data), while also facilitating the use of smooth phase constraints (which classically lead to
linear-prediction relationships between opposite sides of k-space) [8, 18, 35].
Let P
C
() denote the operator that constructs the standard C-matrix from single-channel
Nyquist-sampled k-space data [3]. Also assume that we measure data from L coils, and that
multiplying an image m with the sensitivity map from the `th coil can be represented by the
matrix-vector multiplication R
`
m, where R
`
2 C
NN
is a diagonal matrix with diagonal entries
equal to the sensitivity map values for each voxel. Additionally, let F2C
NN
denote the Carte-
sian Nyquist-sampled Fourier transform operator. Combining parallel imaging and virtual coil
ideas [7, 8, 18, 26, 35, 60, 61], theP() operator we use for Wave-LORAKS in Eq. (3.4) is dened
by concatenating C-matrices corresponding to dierent real and virtual coils according to
P(m) = [C
1
; C
2
;:::; C
L
; D
1
; D
2
;:::; D
L
]; (4.16)
where
C
`
,P
C
(FR
`
m) (4.17)
is the C matrix for the `th real coil, and
D
`
,P
C
(F(R
`
m)) (4.18)
is the C matrix for the corresponding virtual conjugate coil.
Due to its structure, the matrix constructed according to Eq. (4.16) is expected to have nullspace
vectors associated with support constraints, smooth phase constraints, and parallel imaging con-
straints.
55
Use of Autocalibration LORAKS approaches that have been used with calibrationless sampling
have generally used nonlinear/nonconvex penalty functionsJ() [3, 7], since this enables the linear-
prediction relationships to be learned automatically during the optimization procedure. However,
substantial computational accelerations are possible if densely-sampled autocalibration (ACS) data
is available, since this data can be used to pre-learn the linear-prediction relationships for the specic
image of interest [6]. The specic variation of this idea that we describe below for Wave-LORAKS
is a direct adaptation of the previous autocalibrated LORAKS approach [6]. In particular, we
choose J() in Eq. (3.4) to be a simple autocalibrated linear least-squares penalty [6, 60], which
leads to:
^ m = arg min
m2C
N
kEm wk
2
2
+kP(m)Nk
2
F
; (4.19)
wherekk
F
denotes the standard Frobenius norm, and the columns of the matrix N2C
ST
are
estimates of the approximate nullspace vectors of the matrixP(m). By construction, Eq. (4.19)
encourages the rows of the matrixP(m) to be orthogonal to a T -dimensional subspace dened by
N.
If we have a subregion of k-space that is sampled at the Nyquist-rate on a Cartesian grid (i.e.,
ACS data), then we can estimate N by forming a structured LORAKS matrix from the zero-padded
ACS data [6, 26, 60]. In particular, the LORAKS matrix formed from zero-padded ACS data will
contain a fully-sampled submatrix (because the ACS data is fully sampled), and we choose the
columns of N to be an orthonormal basis (obtained using the singular value decomposition) for the
approximate nullspace of this submatrix. Since the submatrix will generally be only approximately
low-rank, the dimension T of the approximate nullspace is a parameter that should be selected by
the user. The matrix N obtained in this way implicitly encodes the linear-prediction relationships
that are observable in the ACS measurements [6].
For Wave-LORAKS, the need to acquire conventional ACS data (i.e., Cartesian data sampled
at the Nyquist rate) would substantially reduce experimental eciency. To avoid this problem, we
instead generate synthetic ACS data by performing an initial unregularized SENSE reconstruction
using Eq. (4.13), and then simulating Cartesian Nyquist-sampled k-space data for each coil using the
SENSE forward model. We estimate coil sensitivity proles for this initial SENSE reconstruction
56
by applying ESPIRiT [26] (with automatic masking of the sensitivity proles based on the image
support) to data measured with a rapid low-resolution prescan that takes about 2 seconds to acquire.
Since the SENSE reconstruction will be most accurate in densely-sampled regions of k-space, we
only generate synthetic ACS data for low-frequency regions of k-space (i.e., the region of k-space
that has the highest SNR and is sampled most densely by the sampling patterns we've considered).
Fast Optimization The optimization problem we wish to solve in Eq. (4.19) has the form of a
simple linear least-squares problem,
6
and we use the conjugate gradient algorithm [101] to solve it.
This is an ecient iterative algorithm that is also commonly used for solving unregularized SENSE
[22] and Wave-CAIPI reconstruction problems [93].
These kinds of iterative algorithms become especially ecient when there are fast algorithms
for implementing matrix-vector multiplication. It has already been established [93] that there are
ecient ways to implement matrix-vector multiplication with the E matrix (based on convolution
with a point-spread function model of the acquisition, which can be implemented using the fast
Fourier transform (FFT) for computational eciency). However, one of the challenges associated
with implementing the LORAKS reconstruction is that the matrixP(m) can be many times larger
than the original image m. This is particularly problematic for large-scale problems like those we
consider in this work. For example, for one of the 3D Wave-CAIPI datasets we consider later in
this paper (i.e., Dataset 1), the vector w occupies around 17 GB (in single precision, and without
coil compression), while the corresponding LORAKS matrix can require up to 1300 GB or more
of memory, which can be dicult for modern computers to accommodate. In addition to these
memory issues, our previous LORAKS implementation [70] of the C matrix operatorP
C
() is
suciently fast for smaller-scale 2D problems, but is relatively slow at building the large matrices
associated with 3D problems.
However, it has recently been observed [102] that the convolutional structure of this kind of
matrix allows computations involvingP
C
() to be performed using simple convolution operations
(which can also be implemented eciently using the FFT), without the need for explicitly forming
6
Note that the complex conjugation operation used in Eq. (4.18) is not linear with respect to complex vectors in
C
N
, but is linear with respect to an equivalent representation that concatenates the real and imaginary parts into a
real vector inR
2N
. Similar to [3], we use this real-valued representation to allow a linear least-squares interpretation.
57
Transaxial Coronal
Sagittal
Transaxial Coronal
Sagittal
Figure 4.14: Gold standard magnitude and phase images for Dataset 1 (top row) and Dataset 2
(bottom row). Images are shown for three representative orthogonal views.
the large-size LORAKS matrix. The structured low-rank matrix used in Ref. [102] has nearly iden-
tical structure to the LORAKS C matrix, with the primary dierence being the shape of the neigh-
borhood system used to form the structured low-rank matrix. In particular, LORAKS generally
uses circular neighborhoods to ensure isotropic resolution characteristics [3], while Ref. [102] uses
rectangular neighborhoods similar to the rectangular kernel shapes used by methods like GRAPPA
[23] and SPIRiT [59]. Since this dierence in the neighborhood system does not change the form of
the computation and because we are using the C matrix in this work, it becomes possible for us to
use the exact same FFT-based approach described previously [102] in our implementation of Wave-
LORAKS. A more detailed description and an example software implementation of this FFT-based
approach is available from Ref. [11]. It should be noted that this FFT-based approach is based on
certain approximations [102]. However, both the results of Ref. [102] and our past experience [11]
suggest that these approximations have a nearly negligible eect on image reconstruction quality
for the type of imaging scenario considered in this paper.
58
4.3.2 Methods
Two in vivo human brain Wave-encoded MPRAGE datasets were acquired using a Siemens 3T
Connectom scanner with a 32 channel headcoil. Both datasets were acquired \unaccelerated," with
phase encoding positions placed on a fully-sampled Cartesian grid at the Nyquist rate. Due to the
use of Wave encoding, these datasets were oversampled along the readout dimension. Dataset 1
was acquired assuming a nominal image matrix size of 240240192 voxels (with 1 mm
3
isotropic
resolution). This data was acquired with 6 oversampling along the readout, corresponding to a
k-space matrix (readout phase encode 1 phase encode 2 coil) size of 1440 240 192 32.
Additional acquisition parameters include: readout duration 5.04 ms, maximum slew rate 180
mT/m/s, maximum gradient amplitude 9.6 mT/m, 15 sinusoidal Wave cycles,
ip angle 9

and
TR/TE/TI = 2500/3.52/1100 ms. Dataset 2 was acquired assuming an image matrix size of
256 256 192 voxels (again with with 1 mm
3
isotropic resolution). This data was acquired
with 3 oversampling along the readout, corresponding to a k-space matrix size of 768 256
192 32. Additional acquisition parameters include: readout duration 5.07 ms, maximum slew
rate 175 mT/m/s, maximum gradient amplitude 9.4 mT/m, 15 sinusoidal Wave cycles,
ip angle
8

, and TR/TE/TI = 2500/3.48/1100 ms. To reduce later computational complexity, the original
32 channels were coil compressed down to 16 virtual channels. Even with this coil compression,
the datasets are still both very large. Specically, the coil-compressed single-precision raw data
for Datasets 1 and 2 respecitively occupies 7.8 GB and 4.2 GB of memory. Basic Wave-CAIPI
reconstructions of the fully-sampled Datasets 1 and 2 are shown in Fig. 4.14.
These two fully-sampled Wave datasets were used to dene gold standard reference images, and
were also retrospectively undersampled to allow evaluation and comparison of dierent accelera-
tion techniques. Retrospective undersampling was only performed along the two phase encoding
dimensions, while the readout dimension was always fully sampled.
One of the potential advantages of Wave-LORAKS is that LORAKS is compatible with a
wide variety of dierent sampling strategies. However, since we do not know what the optimal
undersampling strategy should be in this case, we performed an initial study of dierent k-space
undersampling strategies with 12 acceleration. We specically compared the following:
59
• CAIPI sampling. This is a standard approach in which k-space is sampled on a uniform
lattice, and was used in earlier Wave-CAIPI work [96].
• Variable density (VD) random sampling. K-space was randomly undersampled using
a Poisson disc sampling distribution [84]. The central 72 72 region of Dataset 1 and the
central 74 74 region of Dataset 2 was sampled at a 4 higher sampling density than the
other portions of k-space to account for the fact that low-frequencies generally contain a
substantially higher amount of information content than high frequencies. This is a standard
approach for compressed sensing, and was used in earlier CS-Wave work [99, 100].
• VD CAIPI sampling. This approach is similar to CAIPI, except that the central 72 72
region of Dataset 1 and the central 74 74 region of Dataset 2 was sampled at a 4 higher
sampling density than the other portions of k-space. Both the central and high frequency
regions of k-space were sampled using a uniform lattice (CAIPI) pattern.
• Hybrid sampling. This approach can be viewed as a hybridization of VD random sampling
and VD CAIPI sampling. Specically, we used lattice (CAIPI) sampling for the central 7272
region of Dataset 1 and the central 74 74 region of Dataset 2, and used random Poisson
disc undersampling in high frequency regions.
• Checkerboard sampling. This is an unconventional form of partial Fourier acquisition that
was introduced in Ref. [7]. Like conventional partial Fourier approaches, one side of k-space
is sampled densely while the opposite side of k-space is sampled more sparsely. However,
unlike conventional partial Fourier acquisition, the denser and sparser regions are distributed
on both sides of k-space in an alternating checkerboard pattern, with random Poisson disc
sampling within each checkerboard square. As with VD CAIPI and Hybrid sampling, we used
denser lattice (CAIPI) sampling for the central 72 72 region of Dataset 1 and the central
74 74 region of Dataset 2.
• Partial Fourier sampling. In this case, we combined VD CAIPI with a more conventional
partial Fourier acquisition approach. Specically, we started with a VD CAIPI pattern, and
then removed samples from the edge of one side of k-space. To maintain the same acceleration
60
factor as VD CAIPI, we increased the size of the central densely-sampled region of k-space
to 96 96 for both datasets.
Due to the dierence in matrix size for Datasets 1 and 2, we retrospectively undersampled these
datasets using dierent sampling pattern realizations, as illustrated in Figs. 4.15 and 4.17. For
reference, images showing the 3D aliasing patterns corresponding to each of these dierent sampling
patterns are shown in Figs. 4.16 and 4.18.
For each sampling pattern, reconstructions were performed using traditional Wave-CAIPI re-
construction (Eq. (4.13)), Wave-CS reconstruction using TV regularization (Eq. (4.14)), and our
proposed Wave-LORAKS approach (Eq. (4.19)), with optimization performed in MATLAB on
a desktop computer with an Intel Xeon E5-1620 3.7 GHz quad core CPU processor and 96GB of
RAM. Image quality was judged qualitatively and quantitatively. For quantitative comparisons, we
computed the normalized root-mean-squared error (NRMSE) and the high frequency error norm
(HFEN). The HFEN is based on computing the NRMSE of a high-pass ltered version of the image
[103], and provides more insight than NRMSE into how well the reconstruction has preserved the
high frequency edges and textures of the image. Since we wanted to emphasize errors that occured
within the brain parenchyma and do not care about errors that occur outside the brain, we applied
a brain mask (generated using BrainSuite [104]) to the reconstructed images prior to computing
NRMSE and HFEN.
After this initial comparison of dierent sampling patterns, we also compared Wave-CAIPI, CS-
Wave, and Wave-LORAKS at 16 acceleration. For simplicity, we only performed reconstructions
using the \best" undersampling approach for Wave-LORAKS (as determined based on the results
of the previous comparison at 12 acceleration), while Wave-CAIPI and CS-Wave reconstructions
were performed with the sampling patterns proposed for them in previous literature (i.e., CAIPI
sampling for Wave-CAIPI, and Hybrid sampling for CS-Wave).
To ensure a fair comparison, regularization parameters for CS-Wave were optimized indepen-
dently for each sampling pattern to minimize NRMSE. On the other hand, reconstruction parame-
ters for Wave-LORAKS were set relatively coarsely. Specically, Wave-LORAKS was implemented
with a LORAKS neighborhood radius of 4 and a matrix rank of 2000 for both datasets, and we used
61
12× CAIPI
12× VD
Random
y12× VDy
CAIPI
12× Hybrid
12× Checkerboard
12× Partial
Fourier
Figure 4.15: Dierent 12 accelerated sampling patterns used with Dataset 1. (This gure contains
high resolution detail that may not print clearly on certain printers. Readers may prefer to view
the electronic version of this gure.)
= 1 for Dataset 1 and = 0:5 for Dataset 2. These coarsely-selected Wave-LORAKS parameters
were used uniformly across all sampling patterns, without adaptation to the unique characteristics
of each dataset.
4.3.3 Results
Figure 4.19 shows a zoomed-in version of a representative slice of the 3D reconstructions obtained
from Dataset 1, while Figs. 4.20 and 4.21 respectively show maximum-intensity projection (MIP)
images of the 3D error images for both datasets. The MIP was computed after applying the
previously mentioned brain mask. As can be seen, the Wave-CAIPI reconstructions have the
highest errors with respect to both NRMSE and HFEN, and have a \noisy" appearance as may be
expected from an unregularized reconstruction of highly-undersampled data. On the other hand
CS-Wave and Wave-LORAKS both have substantially lower errors. Interestingly, Wave-LORAKS
outperformed Wave-CAIPI and CS-Wave for almost all undersampling patterns. The one exception
was that CS-Wave had a slightly smaller NRMSE for CAIPI sampling.
When we compare dierent sampling patterns for Wave-CAIPI, we observe that traditional
Wave-CAIPI has the lowest NRMSE with traditional CAIPI undersampling, as might be expected
based on the the excellent g-factor characteristics of CAIPI for traditional reconstruction approaches
[83]. However, surprisingly, we observe that Wave-CAIPI has the lowest HFEN with partial Fourier
sampling. This result is surprising because Wave-CAIPI does not make use of phase constraints,
62
12× CAIPI
12× VD
Random
y12× VDy
CAIPI
12× Hybrid
12× Checkerboard
12× Partial
Fourier
Figure 4.16: Images showing the 3D aliasing patterns corresponding to the sampling patterns from
Fig. 4.15 for Dataset 1. We show (top row) axial, (middle row) coronal, and (bottom row) sagittal
views that are matched to the 3 orthogonal views shown in Fig. 4.14.
and is not expected to be able to extrapolate the missing high-frequency information. Closer
examination of this result suggests that Wave-CAIPI actually does have substantial high-frequency
errors as can be seen in Fig. 4.22. Our surprising HFEN results can be explained by the fact that
the Laplacian of Gaussian lter used in the denition of the HFEN [103] is actually a bandpass lter
that suppresses both low-frequencies and high-frequencies, while emphasing mid-range frequencies.
When we compare dierent sampling patterns for CS-Wave and Wave-LORAKS, we observe
that Partial Fourier sampling yields the smallest NRMSE and HFEN values in both cases. This re-
sult may be expected for Wave-LORAKS, because the good performance of LORAKS with partial
Fourier acquisition is consistent with previous literature [3, 7, 10]. However, this result is surprising
for CS-Wave, which does not impose smooth phase constraints and is not expected to accurately
extrapolate the missing side of k-space. Closer examination suggests that CS-Wave with partial
Fourier sampling has substantial errors on the missing side of k-space (as seen in Fig. 4.22), but
is still able to have lower overall NRMSE and HFEN values than CS-Wave with other sampling
63
12×CAIPI
12×VD
Random
y12×VDy
CAIPI
12×Hybrid
12×
Checkerboard
12×Partial
Fourier
Figure 4.17: Dierent 12 accelerated sampling patterns used with Dataset 2. (This gure contains
high resolution detail that may not print clearly on certain printers. Readers may prefer to view
the electronic version of this gure.)
patterns because of smaller errors in the mid-frequency range. We suspect that CS-Wave is ben-
etting from the fact that Partial Fourier sampling has dense sampling over a larger region of
central k-space than is used by the other sampling schemes. Surprisingly, Hybrid sampling (which
we expected to demonstrate the best performance for CS-Wave based on previous literature) was
slightly outperformed by CAIPI, VD CAIPI, and VD Random sampling with respect to NRMSE,
and was slightly outperformed by VD Random and VD CAIPI sampling with respect to HFEN.
However, in all of these cases, the performance of Hybrid sampling was not substantially worse
than the alternative choices.
Figure 4.23 shows results with 16 acceleration, using CAIPI sampling for Wave-CAIPI, Hy-
brid sampling for CS-Wave, and Partial Fourier sampling for Wave-LORAKS. For reference, we also
show Wave-CAIPI results with 9 accelerated CAIPI sampling, which is the acceleration rate and
sampling strategy considered in previous Wave-CAIPI papers [93, 96]. We observe in this case that
16 accelerated Wave-LORAKS has substantial advantages relative to the other two 16 accel-
erated reconstructions. Perhaps surprisingly, the 16 accelerated Wave-LORAKS reconstruction
even outperforms the 9 accelerated Wave-CAIPI reconstruction with respect to the NRMSE and
HFEN quantitative error metrics. This demonstrates that Wave-LORAKS can enable substantially
more acceleration than Wave-CAIPI without a corresponding loss of image quality. For additional
reference, we also show results obtained using traditional 9 accelerated CAIPI and 16 accel-
erated SENSE-LORAKS without Wave encoding. Since we did not acquire data without Wave
64
12× CAIPI
12× VD
Random
y12× VDy
CAIPI
12× Hybrid
12× Checkerboard
12× Partial
Fourier
Figure 4.18: Images showing the 3D aliasing patterns corresponding to the sampling patterns from
Fig. 4.17 for Dataset 2. We show (top row) axial, (middle row) coronal, and (bottom row) sagittal
views that are matched to the 3 orthogonal views shown in Fig. 4.14.
encoding, this data was simulated based on the fully-sampled gold standard reference image. As
can be seen, results are substantially worse when Wave encoding is not used. This conrms that
Wave encoding is also an important ingredient of the Wave-LORAKS approach.
On our computer, the 16 accelerated reconstructions shown in Fig. 4.23 took about 45 min-
utes for Wave-CAIPI and about 5.5 hours for CS-Wave using an ecient implementation based on
the alternating directions method of multipliers [100]. For comparison, our fast Wave-LORAKS
implementation takes approximately 2.5 hours. Even though our implementation is still at the
proof-of-principle stage and has not been fully optimized, we still observe a major speed advan-
tage relative to CS-Wave. We believe that there are many opportunities for further improving
reconstruction speed by using better hardware, more ecient programming languages, and smarter
algorithms.
65
.
12× CAIPI
12× VD
Random
y12× VDy
CAIPI
12× Hybrid
12× Checkerboard
12× Partial
Fourier
Wave-
CAIPI
.
. NRMSE:0.116
. HFEN: 0.177
. NRMSE:0.148
. HFEN: 0.178
. NRMSE:0.137
. HFEN: 0.166
. NRMSE:0.152
. HFEN: 0.181
. NRMSE:0.177
. HFEN: 0.213
. NRMSE:0.126
. HFEN: 0.122
CS-Wave
.
. NRMSE:0.102
. HFEN: 0.162
. NRMSE:0.111
. HFEN: 0.150
. NRMSE:0.107
. HFEN: 0.141
. NRMSE:0.112
. HFEN: 0.150
. NRMSE:0.125
. HFEN: 0.170
. NRMSE:0.095
. HFEN: 0.104
Wave-
LORAKS
.
. NRMSE:0.103
. HFEN: 0.151
. NRMSE:0.101
. HFEN: 0.139
. NRMSE:0.099
. HFEN: 0.123
. NRMSE:0.100
. HFEN: 0.136
. NRMSE:0.104
. HFEN: 0.150
. NRMSE:0.089
. HFEN: 0.096
Figure 4.19: Reconstructions of a representative sagittal slice from Dataset 1 using dierent recon-
struction techniques and dierent 12 accelerated undersampling patterns. For easier visualization,
we have zoomed-in on a region that shows a variety of important anatomical features that exhibit
structure across a variety of dierent spatial scales, including the brain stem, cerebellum, corpus
callosum, and portions of the occipital and parietal lobes.
4.3.4 Discussion
Wave-LORAKS has a few reconstruction parameters that need to be selected, and it is worthwhile
to understand how sensitive the Wave-LORAKS reconstruction is to the choice of these param-
eters. Figure 4.24 examines how the NRMSE and HFEN error metrics change as a function of
the regularization parameter and the LORAKS matrix rank, in the context of reconstructing
Dataset 1 from 12 accelerated VD CAIPI data. For reference, we also show the NRMSE and
HFEN values obtained from traditional unregularized Wave-CAIPI. These plots demonstrate that
Wave-LORAKS outperforms traditional Wave-CAIPI over a wide range of dierent and matrix
rank values, with the results being slightly more sensitive to than they are to the choice of matrix
rank. As a result, we infer that careful parameter tuning is not essential to the good performance
of Wave-LORAKS, and that Wave-LORAKS is likely to provide benets as long as parameters are
set in a reasonable way. While we selected reconstruction parameters for Wave-LORAKS coarsely
and manually in this work, an automatic data-adaptive parameter selection approach would also be
66
.
12×CAIPI
12×VD
Random
y12×VDy
CAIPI
12×Hybrid
12×
Checkerboard
12×Partial
Fourier
Wave-
CAIPI
.
. NRMSE:0.116
. HFEN: 0.177
. NRMSE:0.148
. HFEN: 0.178
. NRMSE:0.137
. HFEN: 0.166
. NRMSE:0.152
. HFEN: 0.181
. NRMSE:0.177
. HFEN: 0.213
. NRMSE:0.126
. HFEN: 0.122
CS-Wave
.
. NRMSE:0.102
. HFEN: 0.162
. NRMSE:0.111
. HFEN: 0.150
. NRMSE:0.107
. HFEN: 0.141
. NRMSE:0.112
. HFEN: 0.150
. NRMSE:0.125
. HFEN: 0.170
. NRMSE:0.095
. HFEN: 0.104
Wave-
LORAKS
.
. NRMSE:0.103
. HFEN: 0.151
. NRMSE:0.101
. HFEN: 0.139
. NRMSE:0.099
. HFEN: 0.123
. NRMSE:0.100
. HFEN: 0.136
. NRMSE:0.104
. HFEN: 0.150
. NRMSE:0.089
. HFEN: 0.096
Figure 4.20: Maximum intensity projections of the 3D error images associated with reconstructions
of Dataset 1 using dierent reconstruction techniques and dierent 12 accelerated undersampling
patterns. The color scale has been normalized so that a value of 1 corresponds to the maximum
intensity of the image within the brain mask.
a viable and potentially valuable strategy [47, 85], although would be expected to incur substantial
additional computational costs.
All of the results shown so far were based on a LORAKS neighborhood radius of 4. The neigh-
borhood radius is a LORAKS parameter that is analogous to the k-space kernel size in GRAPPA
and ESPIRiT [23, 26]. As has been discussed in previous papers [3, 7], selection of this radius rep-
resents a balance between multiple factors. On the one hand, larger values of the radius mean that
the low-rank model will be more
exible and better able to accommodate rapid spatial variations
in the image support, phase, or parallel imaging constraints. On the other hand, this additional

exibility also means that the LORAKS model will have a larger number of degrees-of-freedom,
and can be more prone to overtting. At the same time, the size of the LORAKS matrices (and
therefore, the amount of memory required if the matrices were formed explicitly) grows in propor-
tion to the square of the neighborhood radius. This last fact may be the most important one to
consider in scenarios where computational resources are limited. Table 4.1 investigates the eects
of dierent neighborhood radius choices for the same context considered in the previous paragraph.
67
.
12× CAIPI
12× VD
Random
y12× VDy
CAIPI
12× Hybrid
12× Checkerboard
12× Partial
Fourier
Wave-
CAIPI
.
. NRMSE:0.117
. HFEN: 0.202
. NRMSE:0.161
. HFEN: 0.223
. NRMSE:0.145
. HFEN: 0.204
. NRMSE:0.165
. HFEN: 0.228
. NRMSE:0.196
. HFEN: 0.264
. NRMSE:0.142
. HFEN: 0.159
CS-Wave
.
. NRMSE:0.102
. HFEN: 0.184
. NRMSE:0.111
. HFEN: 0.178
. NRMSE:0.110
. HFEN: 0.172
. NRMSE:0.112
. HFEN: 0.179
. NRMSE:0.113
. HFEN: 0.186
. NRMSE:0.098
. HFEN: 0.133
Wave-
LORAKS
.
. NRMSE:0.110
. HFEN: 0.171
. NRMSE:0.107
. HFEN: 0.167
. NRMSE:0.106
. HFEN: 0.157
. NRMSE:0.107
. HFEN: 0.165
. NRMSE:0.109
. HFEN: 0.174
. NRMSE:0.095
. HFEN: 0.125
Figure 4.21: Maximum intensity projections of the 3D error images associated with reconstructions
of Dataset 2 using dierent reconstruction techniques and dierent 12 accelerated undersampling
patterns. The color scale has been normalized so that a value of 1 corresponds to the maximum
intensity of the image within the brain mask.
As can be observed, the NRMSE and HFEN both seem to reduce very slightly as we increase the
neighborhood radius from 2 to 4, although this improvement in reconstruction quality is somewhat
oset by very substantial increases in computation time. Interestingly, with a neighborhood radius
of 2, the Wave-LORAKS reconstruction time of 0.99 hours is not too much larger than that of the
much simpler Wave-CAIPI reconstruction (0.75 hours), while the Wave-LORAKS reconstruction
quality is substantially better (i.e., Wave-LORAKS gives an NRMSE of 0.108, while Wave-CAIPI
gives an NRMSE of 0.137 with the same data). Comparing memory usage, we observe that the
use of FFT-based matrix multiplication substantially reduces the amount of memory required for
LORAKS reconstruction compared to what would be required from our original implementation
that explicitly constructs large-scale LORAKS matrices. However, it should also be noted that our
FFT-based Wave-LORAKS implementation still requires more memory than Wave-CAIPI.
We would hypothesize that the trend of reduced NRMSE with larger neighborhood radii will not
continue indenitely, and we would eventually start to see increased NRMSE for larger radius values.
68
Goldstandard Wave-CAIPI CS- Wave Wave-L ORAKS
Figure 4.22: Reconstructed k-space data obtained by applying dierent reconstruction methods
to a subsampled version of Dataset 1, using the 12 partial Fourier undersampling pattern from
Fig. 4.15. Both Wave-CAIPI and CS-Wave demonstrate signicant errors in the high-frequency
region of one side of k-space. This side of k-space was not measured because of partial Fourier
sampling.
However, we have not explored this regime because of the increased computational complexity
associated with larger neighborhood radius values.
The results shown in this paper were based on retrospective undersampling of a \fully sam-
pled" dataset that took roughly 10 minutes to acquire. Specically, Dataset 1 took 10 minutes to
acquire while Dataset 2 took 10.67 minutes to acquire. The fact that we can achieve high quality
reconstruction results with 16 undersampling suggests that a prospective Wave-LORAKS acqui-
sition of this kind of data could be roughly 40 seconds long, which is a substantial improvement
over the previous state-of-the-art. Future prospective implementations of Wave-LORAKS with the
MPRAGE sequence will need to account for T1-blurring eects when designing the phase encode
acquisition order, due to the fact that the MPRAGE sequence uses an echo train. However, ac-
counting for these eects with highly-structured sampling (like the CAIPI-based Partial Fourier
sampling pattern that worked the best with Wave-LORAKS) is more straightforward than it would
be for the VD Random or Hybrid sampling patterns that are better suited for compressed sensing
methods like CS-Wave.
We observed in this work that Partial Fourier sampling appeared to be the best sampling scheme
for Wave-LORAKS in the context of this application. However, it should be noted that we only
considered a restricted class of sampling patterns, and did not perform an exhaustive evaluation of
69
9× Wave-CA IPI
(CAIPIsampling)
16× Wave-CAI PI
(CAIPIsampling)
9× CAIPI
(CAIPIsampling,NoWave)
. NRMSE:0.099
. HFEN: 0.139
. NRMSE:0.154
. HFEN: 0.235
. NRMSE:0.132
. HFEN: 0.204
16× CS-Wave
(Hybridsampling)
16× Wave-L ORAKS
(PartialFouriersampling)
16× SENSE- LORAKS
(PartialFourier,NoWave)
.
. NRMSE:0.116
. HFEN: 0.172
. NRMSE:0.098
. HFEN: 0.138
. NRMSE:0.120
. HFEN: 0.228
Figure 4.23: Maximum intensity projections of the 3D error images associated with reconstructions
of Dataset 1 using 16 accelerated data. The color scale is normalized to match Fig. 4.20. (This
gure contains high resolution detail that may not print clearly on certain printers. Readers may
prefer to view the electronic version of this gure.)
every imaginable sampling strategy. Exploring optimal sampling strategies remains an interesting
topic for future research.
While this work focused on 3D Wave acquisition, we believe that it would be straightforward to
apply the same ideas to simultaneous multislice (SMS) wave data [94]. In particular, there already
exists an SMS formulation for LORAKS [9] that can be easily adapted to this purpose.
Finally, we should mention that while we compared Wave-LORAKS against CS-Wave in this
paper, it would also be possible to use LORAKS synergistically with other regularization constraints
for even further improvements in reconstruction performance, at the cost of increased computational
complexity. Specically, because the LORAKS constraint is just regularization, it is easy to append
additional regularization terms (e.g., TV or wavelet regularization) to the Wave-LORAKS objective
function in Eq. (4.19). Another potentially interesting extension would be the combination of Wave
encoding and LORAKS in the context of multi-contrast imaging, since recent work has shown that
LORAKS constraints can also be very benecial when reconstructing multi-contrast datasets [105].
70
10
−4
10
−3
10
−2
10
−1
10
0
10
1
0.1
0.15
0.2
0.25
0.3
λ
NRMSE
NRMSEversus λ
Wave-LORAKS
Wave-CAIPI
500 1,000 1,500 2,000 2,500 3,000 3,500
0.1
0.15
0.2
0.25
0.3
Rank
NRMSE
NRMSEversusRank
Wave-LORAKS
Wave-CAIPI
10
−4
10
−3
10
−2
10
−1
10
0
10
1
0.1
0.15
0.2
0.25
0.3
λ
HFEN
HFENversus λ
Wave-LORAKS
Wave-CAIPI
500 1,000 1,500 2,000 2,500 3,000 3,500
0.1
0.15
0.2
0.25
0.3
Rank
HFEN
HFENversusRank
Wave-LORAKS
Wave-CAIPI
Figure 4.24: (top) NRMSE and (bottom) HFEN reconstruction error metrics as a function of the
(left) regularization parameter and (right) LORAKS matrix rank, for reconstruction of Dataset
1 with 12 accelerated VD CAIPI sampling.
4.3.5 Conclusion
This work introduced and evaluated a new Wave-LORAKS approach for highly accelerated MRI.
Our experimental results indicate that Wave-LORAKS is quite powerful relative to standard Wave-
CAIPI. Specically, the Wave acquisition and LORAKS reconstruction combine synergistically,
allowing even higher acceleration factors than Wave-CAIPI without a loss of image quality, while
also enabling higher
exibility in the choice of sampling patterns. While the improved performance
of Wave-LORAKS comes at the expense of some additional computational complexity, we believe
that these computational costs can largely be mitigated using appropriate tuning of the LORAKS
neighborhood radius combined with a more ecient numerical implementation.
71
Wave-LORAKS Wave-CAIPI
NeighborhoodRadius 2 3 4
NRMSE 0.108 0.102 0.098 0.137
HFEN 0.133 0.127 0.124 0.166
ComputeTime(hours) 0.99 1.52 2.48 0.75
MemoryUsage(FFT-basedimplementation) 80GB 80GB 80GB 33GB
Matrixsize(originalimplementation) 90GB 320GB 650GB
Table 4.1: The eect of the LORAKS neighborhood radius on NRMSE, memory usage, and com-
putation time for reconstruction of Dataset 1 with 12 accelerated VD CAIPI sampling. For
reference, values corresponding to Wave-CAIPI reconstruction are also shown.
4.4 Summary and Conclusion
This chapter presented accelerated MRI using LORAKS. It introduced SMS-LORAKS for simulta-
neous multislice imaging, SENSE-LORAKS for phase-constrained parallel partial Fourier imaging,
and Wave-LORAKS for highly accelerated 3D imaging. We have demonstrated that LORAKS can
provide many advantages in MRI experiments with improved image qualities and
exibilities in
acquisition settings such as sampling and calibration. Although many promising results have been
shown, we also believe that there would be a lot more research opportunities for improving other
MRI experiments with LORAKS.
72
4.A Supporting Figures for SENSE-LORAKS
73
.
Acceleration
. .
3.0×
.
4.0×
.
5.1×
.
6.0×
SENSE(withTikhonov)withUniformSampling
.
Sampling
. .
Reconstruction
. . . . . .
.
a Uniform
. .
0.17627
.
0.27316
.
0.41436
.
0.50118
P-LORAKSwithUniformSampling
. . . . . .
.
b Uniform
. .
0.43992
.
0.72323
.
0.77902
.
0.87360
SENSE-LORAKSwithUniformSampling
. . . . . .
.
c Uniform
. .
0.11805
.
0.17119
.
0.22898
.
0.28270
SENSE-LORAKSwithUniformPF
. . . . . .
.
d UniformPF
. .
0.13323
.
0.15409
.
0.18119
.
0.20595
Figure S1: Reconstructed TSE images for a range of acceleration factors, presenting a detailed
view of the results summarized in Figs. 2 and 4(a). Comparison between (a) SENSE with con-
ventional Uniform sampling, (b) P-LORAKS with conventional Uniform sampling, (c) SENSE-
LORAKSwithconventionalUniformsampling,and(d)SENSE-LORAKSwithUniformPFsam-
pling. Reconstructedimagesareshownusingalineargrayscale(normalizedsothatimageintensi-
tiesareintherangefrom0to1). NRMSEvaluesareshownunderneatheachreconstruction, with
thebestNRMSEvalueshighlightedwithred. CorrespondingerrorimagesareshowninSupporting
Fig. S2.
74
.
Acceleration
. .
3.0×
.
4.0×
.
5.1×
.
6.0×
SENSE(withTikhonov)withUniformSampling
.
Sampling
. .
ErrorImage
. . . . .
.
a Uniform
. .
0.17627
.
0.27316
.
0.41436
.
0.50118
P-LORAKSwithUniformSampling
. . . . . .
.
b Uniform
. .
0.43992
.
0.72323
.
0.77902
.
0.87360
SENSE-LORAKSwithUniformSampling
. . . . . .
.
c Uniform
. .
0.11805
.
0.17119
.
0.22898
.
0.28270
SENSE-LORAKSwithUniformPF
. . . . . .
.
d UniformPF
. .
0.13323
.
0.15409
.
0.18119
.
0.20595
Figure S2: Error images corresponding to Supporting Fig. S1. Error images are shown using the
indicated colorscale (which ranges from 0 to 0.25 to highlight small errors). NRMSE values are
shownunderneatheachimagewherethesmallestNRMSEvaluesforagivenaccelerationrateare
indicatedinred.
75
.
Acceleration
. .
3.0× 4.0× 5.0× 6.0×
SENSE(withTikhonov)withUniformSampling
.
Sampling
. .
Reconstruction
. . .
.
a Uniform
. .
0.17153 0.28639 0.39847 0.43304
P-LORAKSwithUniformSampling
. . .
.
b Uniform
. .
0.51277 0.70795 0.90864 0.88568
SENSE-LORAKSwithUniformSampling
. . .
.
c Uniform
. .
0.16050 0.24917 0.34155 0.36858
SENSE-LORAKSwithUniformPF
. . .
.
d UniformPF
. .
0.19942 0.21676 0.23450 0.26242
Figure S3: Reconstructed MPRAGE images for a range of acceleration factors, presenting a de-
tailed view of the results summarized in Figs. 3 and 4(b). Comparison between (a) SENSE
with conventional Uniform sampling, (b) P-LORAKS with conventional Uniform sampling, (c)
SENSE-LORAKSwithconventionalUniformsampling,and(d)SENSE-LORAKSwithUniform
PFsampling. Reconstructedimagesareshownusingalineargrayscale(normalizedsothatimage
intensities are in the range from 0 to 1). NRMSE values are shown underneath each reconstruc-
tion,withthebestNRMSEvalueshighlightedwithred. Correspondingerrorimagesareshownin
SupportingFig. S4.
76
.
Acceleration
. .
3.0× 4.0× 5.0× 6.0×
SENSE(withTikhonov)withUniformSampling
.
Sampling
. .
ErrorImage
. .
.
a Uniform
. .
0.17153 0.28639 0.39847 0.43304
P-LORAKSwithUniformSampling
. . .
.
b Uniform
. .
0.51277 0.70795 0.90864 0.88568
SENSE-LORAKSwithUniformSampling
. . .
.
c Uniform
. .
0.16050 0.24917 0.34155 0.36858
SENSE-LORAKSwithUniformPF
. . .
.
d UniformPF
. .
0.19942 0.21676 0.23450 0.26242
Figure S4: Error images corresponding to Supporting Fig. S3. Error images are shown using the
indicated colorscale (which ranges from 0 to 0.25 to highlight small errors). NRMSE values are
shownunderneatheachimagewherethesmallestNRMSEvaluesforagivenaccelerationrateare
indicatedinred.
77
Sampling
Method 3.0× 4.0× 5.1× 6.0×
Phase-
Constrained
SENSE
.
ACPF
.
0.14080 0.16475 0.20515 0.25058
a
SENSE-
LORAKS
. . .
0.13456 0.15629 0.19710 0.22584
P-
LORAKS
.
RandomPF
.
0.14229 0.17652 0.21510 0.45342
b
SENSE-
LORAKS
. . .
0.13801 0.16158 0.19122 0.20879
SENSE+TV
.
Uniform
.
0.10677 0.19796 0.31753 0.41822
c
SENSE-
LORAKS
. . .
0.11805 0.17119 0.22898 0.28270
SENSE-
LORAKS
.
d UniformPF
.
0.13323 0.15409 0.18119 0.20595
Figure S5: Reconstructed TSE images for a range of acceleration factors, presenting a detailed
view of the results summarized in Figs. 5 and 7(a). Comparison of images reconstructed using
SENSE-LORAKS against (a) phase-constrained SENSE with AC PF sampling, (b) P-LORAKS
withRandomPFsampling,(c)SENSE+TVwithconventionalUniformsampling. Reconstructions
obtained using SENSE-LORAKS with Uniform PF sampling are shown in (d). Reconstructed
images are shown using a linear grayscale (normalized so that image intensities are in the range
from 0 to 1). NRMSE values are shown underneath each reconstruction, with the best NRMSE
valueshighlightedwithred. CorrespondingerrorimagesareshowninSupportingFig.S6.
78
Sampling
Method 3.0× 4.0× 5.1× 6.0×
Phase-
Constrained
SENSE
.
ACPF
.
0.14080 0.16475 0.20515 0.25058
a
SENSE-
LORAKS
. . .
0.13456 0.15629 0.19710 0.22584
P-
LORAKS
.
RandomPF
.
0.14229 0.17652 0.21510 0.45342
b
SENSE-
LORAKS
. . .
0.13801 0.16158 0.19122 0.20879
SENSE+TV
.
Uniform
.
0.10677 0.19796 0.31753 0.41822
c
SENSE-
LORAKS
. . .
0.11805 0.17119 0.22898 0.28270
SENSE-
LORAKS
.
d UniformPF
.
0.13323 0.15409 0.18119 0.20595
Figure S6: Error images corresponding to Supporting Fig. S6. Error images are shown using the
indicated colorscale (which ranges from 0 to 0.25 to highlight small errors). NRMSE values are
shownunderneatheachimagewherethesmallestNRMSEvaluesforagivenaccelerationrateare
indicatedinred.
79
Sampling
Method 3.0× 4.0× 5.0× 6.0×
Phase-
Constrained
SENSE
.
ACPF
.
0.20709 0.22236 0.24953 0.27789
a
SENSE-
LORAKS
. . .
0.19786 0.21290 0.23678 0.25735
P-LORAKS
.
RandomPF
.
0.24522 0.28050 0.77941 0.44090
b
SENSE-
LORAKS
. . .
0.20687 0.21947 0.32671 0.26902
SENSE+TV
.
Uniform
.
0.13302 0.20672 0.33305 0.36746
c
SENSE-
LORAKS
. . .
0.16050 0.24917 0.34155 0.36858
SENSE-
LORAKS
.
d UniformPF
.
0.19942 0.21676 0.23450 0.26242
Figure S7: Reconstructed MPRAGE images for a range of acceleration factors, presenting a
detailed view of the results summarized in Figs. 6 and 7(b). Comparison of images recon-
structed using SENSE-LORAKS against (a) phase-constrained SENSE with AC PF sampling, (b)
P-LORAKS with Random PF sampling, (c) SENSE+TV with conventional Uniform sampling.
Reconstructions obtained using SENSE-LORAKS with Uniform PF sampling are shown in (d).
Reconstructedimagesareshownusingalineargrayscale(normalizedsothatimageintensitiesare
in the range from 0 to 1). NRMSE values are shown underneath each reconstruction, with the
best NRMSE values highlighted with red. Corresponding error images are shown in Supporting
Fig.S8.
80
Sampling
Method 3.0× 4.0× 5.0× 6.0×
Phase-
Constrained
SENSE
.
ACPF
.
0.20709 0.22236 0.24953 0.27789
a
SENSE-
LORAKS
. . .
0.19786 0.21290 0.23678 0.25735
P-LORAKS
.
RandomPF
.
0.24522 0.28050 0.77941 0.44090
b
SENSE-
LORAKS
. . .
0.20687 0.21947 0.32671 0.26902
SENSE+TV
.
Uniform
.
0.13302 0.20672 0.33305 0.36746
c
SENSE-
LORAKS
. . .
0.16050 0.24917 0.34155 0.36858
SENSE-
LORAKS
.
d UniformPF
.
0.19942 0.21676 0.23450 0.26242
Figure S8: Error images corresponding to Supporting Fig. S7. Error images are shown using the
indicated colorscale (which ranges from 0 to 0.25 to highlight small errors). NRMSE values are
shownunderneatheachimagewherethesmallestNRMSEvaluesforagivenaccelerationrateare
indicatedinred.
81
4.0×Acceleration
SENSE
.
Sampling
.
WithoutTikhonov
.
WithTikhonov
. . .
.
a
. .
. . .
45.2063
.
0.25489
SENSE-LORAKS
.
Sampling
.
WithoutTikhonov
.
WithTikhonov
. . .
.
b
. .
. . .
0.16089
.
0.15409
Figure S9: Illustration of the effects of Tikhonov regularization on (a) SENSE and (b) SENSE-
LORAKS.
82
.
Sampling
Method 3.0× 4.0× 5.1× 6.0×
.
SENSE+TV
. .
. . .
0.20677 0.19744 0.20609 0.23208
. .
SENSE-
LORAKS
. . .
. . .
0.13323 0.15409 0.18119 0.20595
. .
SENSE-
LORAKS
withTV
. . .
. . .
0.13065 0.14840 0.16962 0.18864
FigureS10: ComparisonbetweenSENSE+TV,SENSE-LORAKS,andSENSE-LORAKSwithTV
for TSE data with Uniform PF sampling. A subset of these results was shown in Fig. 10. The top
rowsshowreconstructedimagesusingalineargrayscale(normalizedsothatimageintensitiesare
in the range from 0 to 1), while the bottom rows show error images using the indicated colorscale
(whichrangesfrom0to0.25tohighlightsmallerrors). NRMSEvaluesareshownunderneatheach
reconstruction,withthebestNRMSEvalueshighlightedinred.
83
This page intentionally left blank
Chapter 5
LORAKI: A Nonlinear Shift-invariant
Autoregressive MRI Reconstruction
through Scan-specic Recurrent
Neural Networks
5.1 Introduction
Slow data acquisition speed has always been one of the biggest impediments to magnetic resonance
imaging (MRI), and developing new methods to improve acquisition speed has been a major research
focus for more than four decades. One way to achieve faster data acquisition is to sample k-space
data below the Nyquist rate, and then use constraints and advanced reconstruction methods to
compensate for the missing information. There is a long history of such methods, ranging from
classical constrained image reconstruction [32] and parallel imaging methods [106], to more recent
sparse [38], low-rank [73, 107, 108], and structured low-rank [3, 4, 61{64] modeling methods, to
very recent machine learning methods [49{52, 54{56, 109{111].
In this work, we propose and evaluate LORAKI, a shift-invariant nonlinear autoregressive auto-
calibrated (AA) deep learning approach for MRI reconstruction that combines features of existing
LORAKS [3, 7] and RAKI [55] methods.
RAKI [55] is a nonlinear deep learning AA method that uses a small amount of autocalibration
(ACS) data to train a scan-specic feedforward convolutional neural network (CNN) that can
perform autoregressive k-space interpolation. RAKI can be viewed as a deep learning version of
85
the widely used non-iterative linear AA method known as GRAPPA [23]. Autocalibrated LORAKS
(AC-LORAKS) [6] is a linear AA method that was not originally developed in the context of neural
networks. Compared to GRAPPA, AC-LORAKS imposes substantially more constraints (i.e.,
support and phase constraints in addition to parallel imaging constraints), has generally better
performance, and can be implemented algorithmically as a series of iterated convolution operations
[11, 102].
Our proposed LORAKI approach combines the nonlinear deep learning approach of RAKI
with the additional constraints and iterative convolutional characteristics of AC-LORAKS. Specif-
ically, LORAKI adopts the general structure of an unfolded iterative AC-LORAKS algorithm and
augments this structure with typical artical neural network characteristics. This results in a con-
volutional recurrent neural network (RNN) architecture for LORAKI, which is dierent from the
feedforward CNN architecture employed by RAKI. We hypothesized that LORAKI would have
advantages over RAKI for the same reasons that AC-LORAKS typically outperforms GRAPPA.
We also hypothesized that LORAKI could outperform AC-LORAKS because nonlinear AA meth-
ods like NL-GRAPPA [65], KerNL [66], and RAKI [55] have already been demonstrated to have
performance advantages over earlier linear AA methods.
Like both AC-LORAKS and RAKI, the proposed LORAKI network is trained to be scan-
specic (i.e., the network is trained specically to reconstruct one specic dataset, and the trained
network should not be expected to generalize to other scans) based on a small amount of ACS data.
This is very dierent from the the large multi-subject databases of training data required by most
other deep learning MRI reconstruction methods that attempt to train generalizable reconstruction
procedures [49{52, 54, 56, 109{111]. This means that \Big Data" is not required, alleviating one
of the main drawbacks of most other deep learning methods. Like AC-LORAKS, LORAKI is also
expected to be compatible with a wide range of autocalibrated Cartesian k-space sampling patterns.
Our results with retrospectively undersampled brain MRI images conrm that LORAKI can
have performance advantages over both AC-LORAKS and RAKI, and that LORAKI can accom-
modate various sampling strategies. A preliminary account of portions of this work was presented
in a recent conference [13].
86
5.2 Background
5.2.1 Autoregression and Autocalibration
Many existing image reconstruction methods assume that k-space data possesses shift-invariant
autoregressive structure, meaning that a missing data sample can be accurately extrapolated or
interpolated based on the values of neighboring samples if autoregression relationships can be
learned. As reviewed in a recent tutorial [4], linear shift-invariant autoregressive structure can
be derived as a natural consequence of limited image support [3], smooth image phase [3, 31],
transform domain sparsity [5, 62{64], inter-image correlations in multi-contrast imaging scenarios
[105], and/or inter-channel correlations in parallel imaging contexts [24]. AA methods are a special
case of autoregressive methods, and their dening characteristic is that they learn the shift-invariant
autoregression relationships from a fully-sampled region of k-space providing ACS data. In a second
step, AA methods then apply this learned information for image reconstruction.
Most existing AA methods have assumed a linear autoregression relationship [4], including
SMASH [24], GRAPPA [23], SPIRiT [59], PRUNO [60], AC-LORAKS [6], and other methods that
haven't been given names or acronyms [31, 62, 64]. Lately, there has also been growing interest
in nonlinear AA methods like NL-GRAPPA [65], KerNL [66], and RAKI [55], where the use of
nonlinearity seems to improve empirical performance.
Since GRAPPA, RAKI, and AC-LORAKS are the methods that are most relevant to our story,
we provide a brief overview of these methods in the sections below, before describing our proposed
LORAKI approach.
In what follows, we assume that k-space is sampled on a Cartesian integer lattice, and use the
integer vector k to denote the coordinates with respect to that lattice. For example, in 2D MRI
with sampling intervals of k
x
and k
y
, the k-space location (mk
x
;nk
y
) would correspond to
the integer vector k = [m;n]
T
. Assuming a multi-channel experiment with a total of L channels,
we will use d
`
[k] to denote the k-space sample at lattice position k from coil `.
87
5.2.1.1 GRAPPA
GRAPPA [23] assumes that there exist shift-invariant linear interpolation relationships in k-space
such that the value of d
`
[k] can be accurately predicted as a linear combination of neighboring
k-space samples according to
d
`
[k]
L
X
c=1
X
m2
k
w
`;m;c
d
c
[k m]: (5.1)
Here,
k
is the set of integer shift vectors that specify the relative positions of the local neighbors
that will be used to interpolate point k, and w
`;m;c
are the GRAPPA kernel weights. Because
GRAPPA assumes this relationship is shift-invariant (i.e., Eq. (5.1) should be valid at every k-
space position with the same kernel weights), it can be represented in convolutional form, and a
small amount of ACS data can be used to train the values of the kernel weights [23]. However,
a dierent set of kernel weights needs to be estimated for each distinct conguration of the local
sampling neighborhood. GRAPPA is usually applied in scenarios with uniform undersampling, in
which case there is a lot of repetition in the local sampling conguration, resulting in only a small
number of distinct neighborhood congurations
k
. We will assume that there are only J distinct
values of
k
, denoted by
j
for j = 1;:::;J.
For the sake of concreteness in what follows (and without loss of generality), we will assume
a 2D imaging scenario with an N
1
N
2
grid of nominal sampling positions and a rectangular
GRAPPA kernel of size R
1
R
2
. We will also assume that non-overlapping binary k-space masks
g
j
2R
N
1
N
2
have been constructed forj = 1;:::;J, such that g
j
[m;n] = 1 if the (m;n)th k-space
position was unsampled and possesses local sampling conguration
j
, and g
j
[m;n] = 0 otherwise.
GRAPPA can be equivalently viewed as a single-layer CNN without bias terms or activation
functions. In the notation of CNNs, GRAPPA is succinctly represented as
[d
rec
]
`
[d
zp
]
`
+
J
X
j=1
g
j
[f
`
(d
zp
)]
j
; (5.2)
88
for` = 1;:::;L. Here, denotes the Hadamard product (elementwise multiplication); for arbitrary
a, [a]
`
denotes the`th channel of a; d
rec
2C
N
1
N
2
L
is the output set of reconstructed k-space data;
d
zp
2C
N
1
N
2
L
is the input set of acquired multi-channel data with unsampled k-space locations
lled with zeros; and each f
`
() : C
N
1
N
2
L
! C
N
1
N
2
J
represents the linear convolution layer
corresponding to ltering the input signal with the J sets of GRAPPA weights w
`;j
2C
R
1
R
2
L
for the `th output channel and jth local sampling conguration. In particular,
[f
`
(d
zp
)]
j
=
L
X
c=1
[w
`;j
]
c

[d
zp
]
c
(5.3)
for j = 1;:::;J, where
denotes convolution. The neural network representation for GRAPPA is
illustrated in Fig. 5.1.
The kernel weights w
`;j
2C
R
1
R
1
L
for` = 1;:::;L andj = 1;:::;J in conventional GRAPPA
are trained with a least-squares loss function [23], where the ACS data is subsampled according
to
j
to generate paired fully-sampled and undersampled training examples. It should be noted
that since the ACS data is usually smaller than N
1
N
2
, the training of the w
`;j
kernels is usually
performed with adjusted input and output variable sizes (i.e., respectively replacing N
1
and N
2
with M
1
and M
2
in the above, assuming the ACS training data has size M
1
M
2
).
5.2.1.2 RAKI
RAKI [55] extends GRAPPA by using multiple convolution layers along with ReLU activation
functions. In particular, the RAKI network can be represented as
[d
rec
]
`
[d
zp
]
`
+
J
X
j=1
g
j
[f
`;3
(relu(f
`;2
(relu(f
`;1
(d
zp
)))))]
j
: (5.4)
for ` = 1;:::;L.
1
In this expression, f
`;1
(), f
`;2
(), and f
`;3
() are each linear convolution layers
without bias terms, and the ReLU activation function relu() is an elementwise operation that
outputs a vector with the same size as the input, with ith element of the output ofrelu(x) dened
1
Note that the original RAKI formulation was described assuming simple uniform 1D undersampling along the
phase encoding dimension, leading to certain dierences from the more general formulation we present here.
89
GRAPPA Local sampling paern 1
Input
Conv
Combina!on
…
Output
Data
Projec!on
Local sampling paern 2
Conv
Local sampling paern J
Conv
AC-LORAKS Landweber Itera!on
Iterate K !mes
Input Output
+
-
Conv Conv
Data
Projec!on
LORAKI
Iterate K !mes
Input Output
+
-
Conv Conv
ReLU
Data
Projec!on
RAKI
Local sampling paern 2
Local sampling paern 1
Input Combina!on
…
Output
Data
Projec!on
Conv Conv Conv
ReLU ReLU
Local sampling paern J
Conv Conv Conv
ReLU ReLU
Conv Conv Conv
ReLU ReLU
Figure 5.1: Neural network representations of GRAPPA, RAKI, AC-LORAKS (with Landweber it-
eration), and LORAKI. It should be noted that neither GRAPPA nor AC-LORAKS were originally
developed in the context of articial neural network models, but they still admit neural network
interpretations.
as max(x
i
; 0). The structure of the RAKI CNN is also shown in Fig. 5.1. The nonlinear ReLU acti-
vation functions are the key features that distinguish RAKI from GRAPPA. In particular, because
of the linearity and associativity properties of convolution, applying a series of convolution layers
without any nonlinearities between them is functionally equivalent to applying a single convolution
layer, which would cause Eq. (5.4) to eectively become an overparameterized version of Eq. (5.2)
if the nonlinearities were removed.
Since the technology for complex-valued neural networks is less developed than for real-valued
neural networks, RAKI was practically implemented by treating the real and imaginary compo-
nents as separate real-valued channels, thereby doubling the eective number of channels [55]. In
90
particular, RAKI uses d
rec
2R
N
1
N
2
2L
and d
zp
2R
N
1
N
2
2L
. Each of the convolution layers has
a similar multi-channel structure to that described above for GRAPPA. In particular, f
`;1
() maps
variables in R
N
1
N
2
2L
to variables in R
N
1
N
2
C
1
by convolving its input with C
1
multi-channel
kernels, where each kernel belongs toR
R
11
R
21
2L
;f
`;2
() maps variables inR
N
1
N
2
C
1
to variables
inR
N
1
N
2
C
2
by convolving its input with C
2
multi-channel kernels, where each kernel belongs to
R
R
12
R
22
C
1
; and f
`;3
() maps variables inR
N
1
N
2
C
2
to variables inR
N
1
N
2
J
by convolving its
input with J multi-channel kernels, where each kernel belongs to R
R
13
R
23
C
2
. The variables C
1
,
C
2
and R
ij
for i = 1; 2 and j = 1; 2; 3 are all user-selected parameters.
RAKI uses the same least-squares loss function and same training data as GRAPPA, although
needs a more complicated training procedure because RAKI is nonlinear, and the simple linear
least-squares techniques used by GRAPPA are not applicable. To overcome this, RAKI can be
trained using backpropagation to minimize the nonlinear least-squares loss function [55]. However,
because RAKI generally has more parameters and is designed capture more complicated autore-
gressive structure than GRAPPA, it generally needs more ACS data than GRAPPA to achieve
good reconstruction results.
5.2.1.3 LORAKS and AC-LORAKS
The LORAKS framework [3, 7] is based on a theoretical association between autoregressive k-space
structure and a variety of classical image reconstruction contraints (including limited support,
smooth phase, sparsity, and parallel imaging constraints) [4]. Specically, LORAKS is based on the
observation that when one or more of these classical constraints are satised by a given image, then
the k-space data will approximately obey at least one (and frequently many more than one) linear
autoregression relationship. The existence of such linear autogression relationships implies that
an appropriately-constructed structured matrix (e.g., convolution-structured Hankel or Toeplitz
matrices) formed from the k-space data will have distinct nullspace vectors associated with each
linear autoregression relationship. This implies that such a matrix will have low-rank structure,
which enables constrained image reconstruction from undersampled k-space data using modern
low-rank matrix recovery methods. A nice feature of this approach is that users of LORAKS
91
do not need to make prior modeling assumptions about the support, phase, or parallel imaging
characteristics of the images { all of this information is implicitly captured by the nullspace of the
structured matrix, which is estimated automatically from the data. This automatic adaptation
means that the LORAKS approach can still be applied in cases where the image may not obey
all of the constraints that motivate the LORAKS framework. Related structured low-rank matrix
modeling approaches that have similar characteristics include Refs. [61{64].
The original implementations of LORAKS were compatible with calibrationless k-space sam-
pling, and reconstructed undersampled k-space data by solving a nonconvex matrix recovery prob-
lem. However, it was later observed that substantial improvements in computational complexity
could be achieved if ACS data were acquired, resulting in a fast linear AA method called AC-
LORAKS [6]. In particular, in the rst step of AC-LORAKS, the ACS data is formed into a
structured \calibration" matrix, and the nullspace of this calibration matrix is estimated. This
nullspace matrix captures linear autoregressive relationships that should hold for all points in k-
space, and the fully sampled k-space data can then be reconstructed by nding values for the
missing data points that are consistent with these linear autoregressive relationships. This can be
done by solving a simple linear least-squares problem:
d
rec
= arg min
d2C
N
1
N
2
L
1
2
kP(d)Nk
2
F
s.t.M(d) = d
zp
: (5.5)
In this expression, the operatorP() :C
N
1
N
2
L
!C
PQ
maps the vector of k-space data into a
structured low rank matrix that is expected to have low-rank; the columns of the matrix N2C
QC
correspond to a collection of C nullspace vectors obtained from the calibration matrix; and the
linear operatorM() : C
N
1
N
2
L
! C
N
1
N
2
L
is a masking operator that sets k-space sample
values equal to zero if they were not measured during data acquisition. While this linear leasts-
squares problem can be solved analytically in principle, the large size of the matrices in Eq. (5.5)
means that practical implementations usually rely on iterative least-squares solvers.
Note that AC-LORAKS is strongly inspired by previous autocalibrated low-rank modeling work
by Liang [62, 107], and can also be viewed as a generalization of both the PRUNO [60] and
SPIRiT [59] reconstruction methods. In particular, AC-LORAKS reduces to PRUNO [60] if the
92
P() operator is designed to construct a structured matrix that only imposes limited support and
parallel imaging constraints (but not the smooth phase constraints that are available through novel
LORAKS matrix constructions), and PRUNO reduces to the previous SPIRiT technique [59] if the
number of nullspace vectors is set to C = 1.
An open source implementation of AC-LORAKS that relies on the iterative conjugate gradient
algorithm is publicly available [11]. However, in this work, we observe that the projected gradient
descent iterative algorithm [112] is easier to adapt to the deep learning formalism used by LORAKI.
For a generic constrained optimization problem of the form
d = arg min
d
f(d) s. t. d2
(5.6)
where f() is the objective function and
is the constraint set, the projected gradient descent
algorithm iterates (starting from some initial guess d
(0)
) according to[112]
d
(i+1)
=C

d
(i)
rf(d
(i)
)

; (5.7)
whereC

() is an operator that projects onto the constraint set
,rf() is the gradient off(), and
is a step size parameter. When applied to least-squares problems like Eq. (5.5), this algorithm
is known as Landweber iteration, and has guaranteed convergence characteristics if the step size is
chosen small enough [113].
When this algorithm is specialized to the AC-LORAKS problem from Eq. (5.5) with f(d) =
1
2
kP(d)Nk
2
F
, the gradient function is given by rf(d) = P

P(d)NN
H

, where the operator
P

() :C
PQ
!C
N
1
N
2
L
is the adjoint ofP(). In addition, the projection onto the constraint
set is obtained asC

(d) =U(d) + d
zp
, where the linear operatorU : C
N
1
N
2
L
! C
N
1
N
2
L
is
dened byU(x), xM(x). The resulting iteration procedure is dened by
d
(i+1)
rec
=U

d
(i)
rec
P

P(d
(i)
rec
)NN
H

+ d
zp
: (5.8)
93
Importantly, because the LORAKS matrices constructed byP() are convolution-structured,
the iteration procedure from Eq. (5.8) takes the form of a two-layer convolutional RNN without
bias terms or activation functions. Specically, each iteration can be written as a convolution layer
g
1
() :C
N
1
N
2
L
!C
N
1
N
2
C
associated with the linear convolutional operatorP()N, a second
convolution layer g
2
() :C
N
1
N
2
C
!C
N
1
N
2
L
associated with the linear convolutional operator
P

(N
H
), and a nal projection onto data consistency implemented using theU operator:
d
(i+1)
rec
=U

d
(i)
rec
g
2
(g
1
(d
(i)
rec
))

+ d
zp
: (5.9)
This RNN structure of Landweber-based AC-LORAKS is also shown in Fig. 5.1.
An important dierence between AC-LORAKS and GRAPPA is that AC-LORAKS applies di-
rectly to the entire undersampled dataset at once, and does not require separate reconstruction of
each channel or enumeration and separate treatment of all of the distinct local sampling congura-
tions. This can simplify the reconstruction procedure, and for example ensures that AC-LORAKS
can be substantially easier to use with non-uniform k-space sampling patterns where the number J
of local sampling congurations may be large. In addition, this can also improve the reconstruction
of missing samples whose closest neighboring acquired samples may be far away with respect to
the size of the convolution kernels used for reconstruction.
While the convolution kernels used in LORAKS could be rectangular like the kernels employed
by most other methods (including RAKI and GRAPPA), LORAKS implementations have classically
always relied on ellipsoidal convolution kernels [3]. AnR
1
R
2
ellipsoidal convolution kernel can be
viewed as a special case of a standardR
1
R
2
rectangular kernel, where the values in the corners of
the rectangle (i.e., the region outside the ellipse inscribed within the rectangle) are forced to be zero.
Ellipsoidal kernels have several advantages, including fewer degrees of freedom to achieve the same
spatial resolution characteristics as rectangular kernels (e.g., in 2D, an ellipse has =4 78:5% of
the area of the rectangle that circumscribes it), and more isotropic resolution characteristics rather
than the anisotropic resolution associated with rectangular kernels. It has recently been shown
that they frequently also provide better empirical reconstruction performance for a wide range of
autoregressive reconstruction methods [114].
94
5.3 Proposed LORAKI Approach
Inspired by RAKI and AC-LORAKS, LORAKI is implemented by simply including nonlinear ReLU
activation functions within the convolutional RNN architecture of Landweber-based AC-LORAKS.
In particular, starting from an initialization of d
(0)
rec
= d
zp
, the LORAKI network iterates the
following equation for a total of K iterations:
d
(i+1)
rec
=U

d
(i)
rec
g
2
(relu(g
1
(d
(i)
rec
)))

+ d
zp
: (5.10)
As before, g
1
() and g
2
() are convolution layers without bias terms. The convolutional RNN
structure of LORAKI is also shown in Fig. 5.1. The number of iterations K is a user-selected
network parameter that is xed prior to training.
Eq. (5.10) has the same basic structure as the projected gradient descent AC-LORAKS algo-
rithm from Eq. (5.8), except that we've replaced the previous gradient termrf(d) =P

P(d)NN
H

with the new gradient termr
~
f(d) =g
2
(relu(g
1
(d))). Importantly, this new gradient term is non-
linear in d, unlike the linear gradient function associated with AC-LORAKS. This dierence means
that LORAKI is implicitly imposing learned shift-invariant nonlinear autoregressive relationships
on the k-space data (similar to NL-GRAPPA [65], KerNL [66], and RAKI [55]), unlike the linear
autoregressive relationships employed by AC-LORAKS. This new gradient term and the overall
projected gradient descent iteration for LORAKI is therefore also implicitly associated with some
new cost function
~
f() that imposes consistency with learned nonlinear autoregressive relationships.
However, we never explicitly calculate the new cost function or the nonlinear autoregressive rela-
tionships, as these are all implicit in the gradient. This can be viewed as similar to the way that the
\kernel trick" from machine learning is used in NL-GRAPPA[65] to simplify the design of nonlinear
autoregressive relationships, although we rely on ReLUs to provide nonlinearity instead of using
kernels.
Similar to RAKI, the nonlinear structure of LORAKI means that this network cannot be trained
with the same relatively simple training procedure used by AC-LORAKS, but is still trainable using
any standard neural network algorithm.
95
5.3.0.1 Training Considerations and Synthetic ACS data
For training, LORAKI uses the same ACS data as used by the other three methods described
above. Similar to GRAPPA and RAKI, this ACS data is subsampled to generate paired fully-
sampled and undersampled training examples. LORAKI is easily compatible with non-uniform
sampling patterns like random sampling or partial Fourier acquisition, and there is no need to
tailor the reconstruction procedure to the specic local sampling congurations that are present
in the acquired data. This means that when constructing paired fully-sampled and undersampled
training examples, the undersampling patterns that are used for training do not need to be a close
match to the real undersampling pattern that will be reconstructed.
Similar to RAKI, LORAKI is more complicated and has more parameters than AC-LORAKS
or GRAPPA. As a result, it should generally be expected that LORAKI will require more ACS
training data than AC-LORAKS does. However, since acquiring a substantial amount ACS data
may reduce experimental eciency, it would be preferable if the dependence on acquired ACS data
could be reduced. Recently, we have explored an approach for generating synthetic ACS data
that worked fairly well in a dierent context [67]. This approach was based on rst performing a
fast initial reconstruction of the data, and then using estimated fully-sampled k-space data from
that initial reconstruction as synthetic ACS data to guide the next stage of LORAKS-based image
reconstruction. In this paper, we observe that if only a small amount of ACS data is acquired,
we can potentially use the full k-space data obtained by a fast initial AC-LORAKS reconstruction
to provide additional synthetic ACS training data to use with LORAKI. Since the potential value
of this synthetic ACS approach is hard to evaluate theoretically, it is evaluated empirically in the
sequel.
5.4 Evaluation Methods
5.4.1 Datasets
We evaluated LORAKI and compared it against other methods by reconstructing retrospectively-
undersampled versions of fully-sampled in vivo human brain datasets from two dierent contexts.
96
In the rst case, T2-weighted images from 5 subjects were acquired with 2D Fourier encoding, using
a 256167 acquisition matrix (readout phase encoding) on a 3T scanner with a 12-channel head
coil. For this data, 1D undersampling simulations were achieved by removing full phase encoding
lines from the fully-sampled data.
In the other case, T1-weighted images from 5 subjects were acquired with 3D Fourier encoding
using an MPRAGE sequence on a 3T scanner with a 12-element multi-channel head coil, which
was coil-compressed down to 4 channels. In four cases, this coil-compression was performed in
hardware by the scanner itself, while we manually applied coil compression in software in the fth
case for uniformity with the other cases. This 3D Fourier data was initially Fourier transformed
along the fully-sampled readout direction, resulting in multiple 2D k-space datasets corresponding
to dierent 2D image slices. For this data, 2D undersampling simulations of each slice were achieved
by removing individual phase encoding positions.
5.4.2 Comparisons and Performance Metrics
LORAKI reconstruction (with either real or synthetic ACS data) was performed with the sub-
sampled data. For reference, the same data was also reconstructed using GRAPPA [23], sRAKI
[115, 116], and AC-LORAKS [6, 11]. sRAKI is a very recent variant of RAKI (indeed, sRAKI
appeared during the preparation of this paper and several months after the rst LORAKI preprint
was made publicly available [14]), and was chosen for comparison because it oers more
exibility
and improved performance compared to the original RAKI. Note that we intentionally did not
make comparisons against large-scale deep learning methods that require large amounts of training
data from multiple subjects, since those methods ll a very dierent niche than autocalibrated
methods do. In particular, unlike autocalibrated methods, such \Big Data" methods would not be
applicable to MRI applications for which vast quantities of existing data do not already exist.
The reconstruction results were evaluated subjectively using visual inspection, as well as quan-
titatively using standard normalized root-mean-squared error (NRMSE) and structural similarity
index (SSIM) error metrics. For NRMSE, smaller numbers are better with a perfect reconstruction
97
corresponding to an NRMSE value of zero. For SSIM, larger numbers are better with a perfect
reconstruction corresponding to an SSIM value of one.
5.4.3 Implementation Details
5.4.3.1 LORAKI
For simplicity and without loss of generality (and similar to RAKI), we have implemented LO-
RAKI using a real-valued deep learning architecture that separates the real and imaginary parts
of the data and doubles the eective number of channels. While there are several formulations
of LORAKS phase constraints[3, 11, 67], we based LORAKI on the use of the virtual conju-
gate coil version of LORAKS [11, 67], which leads to a simplied LORAKI implementation. In
particular, for every original channel, we construct a new virtual channel by applying reversal
and complex conjugation operations to the k-space data [18]. This has the eect of further
doubling the number of channels, eectively leaving us with 4L channels. As a result, we have
d
(i)
rec
2 R
N
1
N
2
4L
, g
1
() : R
N
1
N
2
4L
! R
N
1
N
2
C
, and g
2
() : C
N
1
N
2
C
! C
N
1
N
2
4L
, where
C is a user-selected number of intermediate channels. Unless otherwise noted, all of the results
reported below usedC = 64 withK = 5 iterations. To maintain consistency with LORAKS imple-
mentations and because we have observed (results not shown) that it leads to improved empirical
performance, we choose to use ellipsoidal convolution kernels instead of rectangular convolution ker-
nels in our implementation of LORAKI. Unless otherwise noted, the ellipsoidal convolution kernels
used R
1
=R
2
= 5 for both convolution layers.
Assuming the ACS training data has size M
1
M
2
, training datasets were obtained by pairing
subsampled versions of the ACS data with the fully-sampled ACS data. ACS subsampling masks
were identied by extracting every distinct patch of size M
1
M
2
from the full N
1
N
2
sampling
mask.
LORAKI code was implemented in PyTorch, and optimization was performed using the Adam
optimizer with a learning rate of 10
3
. All experiments were conducted on Google Colab leveraging
an NVidia Tesla K80 GPU.
98
GRAPPA sRAKI AC-LORAKS
LORAKI
(Original)
LORAKI
(Synthetic)
Uniform
Sampling
Gold Standard
NRMSE: 0.148
SSIM: 0.889
NRMSE: 0.0935
SSIM: 0.946
NRMSE: 0.0738
SSIM: 0.958
NRMSE: 0.0645
SSIM: 0.969
NRMSE: 0.0646
SSIM: 0.0967
Figure 5.2: Representative reconstruction results for uniformly-undersampled T2-weighted data.
The top row shows reconstructed images for one slice in a linear grayscale, where the gold standard
image has been normalized to range from 0 (black) to 1 (white). The bottom row shows error
images with the indicated colorscale. NRMSE and SSIM values are also shown below each image,
with the best values highlighted in red.
5.4.3.2 GRAPPA
The implementation of GRAPPA we compared against used a kernel with R
1
= R
2
= A + 1,
where A denotes the acceleration factor of the scan. This choice implies that, for 1D uniform
undersampling patterns, each missing sample is interpolated using the A + 1 nearest samples from
each of the two nearest acquired phase encoding lines.
5.4.3.3 sRAKI
The implementation of sRAKI we compared against used the same choices of network parameters
(including kernel sizes, kernel dilation factors, etc.) as described in Ref. [115].
5.4.3.4 AC-LORAKS
The implementation of AC-LORAKS we compared against is publicly available [11]. We used the
\S"-version of AC-LORAKS (which incorporates support, phase, and parallel imaging constraints),
99
and used ellipsoidal convolution kernels with R
1
= R
2
= 5 (matched to the kernel size used for
LORAKI). TheC parameter for AC-LORAKS was optimized on an image-by-image basis to achieve
the smallest NRMSE. This choice represents a best-case scenario, since the true NRMSE value
would not be available for a prospective acquisition.
5.5 Results
In a rst set of experiments, we performed reconstruction of uniformly-undersampled T2-weighted
datasets. Specically, we simulated an acquisition that measured every fourth line of k-space, while
also fully-acquiring the central 24 phase encoding lines to be used as ACS data. Taken together,
this results in an eective acceleration factor of 2:8. Figure 5.2 shows representative results from
one slice of one subject, while numerical results from a total of 50 slices from 5 dierent subjects
are shown in Fig. 5.3. As can be observed, the proposed LORAKI approach had the best overall
performance, with lower NRMSE values and higher SSIM values compared to GRAPPA, sRAKI,
or AC-LORAKS. Specically, applying a paired t-test to these results suggests that LORAKI
(trained with the original ACS data) had signicantly lower NRMSE values (p = 1:3 10
16
) and
signicantly higher SSIM values (p = 4:010
14
) than AC-LORAKS, the next-best method. There
was not a major dierence between the NRMSE or SSIM values for LORAKI with the original ACS
data versus LORAKI with the synthetic ACS data.
Since neither NRMSE or SSIM provide a complete description of the error characterstics, we
also computed error spectrum plots (ESPs) that decompose error characteristics as a function of
dierent spatial resolution scales [1]. A representative ESP is shown in Fig. 5.4, and demonstrates
that the LORAKI-based approaches had consistently similar or better error characteristics than
the other methods across all spatial frequencies, with the most signicant advantage at high-spatial
frequencies.
In a second set of experiments, we performed reconstruction of randomly-undersampled T1-
weighted datasets.
2
Specically, we simulated an acquisition using a variable-density Poisson disk
2
GRAPPA reconstructions were not performed in this case due to the large number of local sampling congurations
resulting from random sampling.
100

0.05
0.1
0.15
0.2
0.25
NRMSE
T2-weighted
GRAPPA
sRAKI
AC-LORAKS
LORAKI (Original)
LORAKI (Synthetic)

0.08
0.1
0.12
0.14
0.16
0.18
NRMSE
MPRAGE
sRAKI
AC-LORAKS
LORAKI (Original)
LORAKI (Synthetic)

0.7
0.75
0.8
0.85
0.9
0.95
SSIM
T2-weighted
GRAPPA
sRAKI
AC-LORAKS
LORAKI (Original)
LORAKI (Synthetic)

0.88
0.9
0.92
0.94
0.96
0.98
SSIM
MPRAGE
sRAKI
AC-LORAKS
LORAKI (Original)
LORAKI (Synthetic)
Figure 5.3: Boxplots showing performance measures for (left column) T2-weighted data and (right
column) T1-weighted data. The results are compared using NRMSE (top row) and SSIM (bottom
row).
random sampling pattern with an eective acceleration factor of 5:2 (including samples from a
fully-sampled 64 64 ACS region at the center of k-space). Figure 5.5 shows representative results
from one slice of one subject, while numerical results from a total of 50 slices from 5 dierent
subjects were shown in Fig. 5.3 and a representative ESP plot was shown in Fig. 5.4. Similar to
the previous case, LORAKI (trained with the original ACS data) had signicantly lower NRMSE
values (p = 1:8 10
7
) and signicantly higher SSIM values (p = 2:2 10
7
) than AC-LORAKS,
which was again the next-best method. As before, there was not a major dierence between using
the original ACS data versus using synthetic ACS data, which might be explained by the relatively
large size of the ACS region. As before, the ESP plot suggests that this good performance is
consistent across all spatial resolution scales.
101
0 0.1 0.2 0.3 0.4 0.5 0.6
0
0.5
1
1.5
SpatialFrequency (mm)
−1
RelativeError
T2-weightedData
GRAPPA
sRAKI
AC-LORAKS
LORAKI(Original)
LORAKI(Synthetic)
0 0.1 0.2 0.3 0.4 0.5 0.6
0
0.2
0.4
0.6
0.8
1
SpatialFrequency (mm)
−1
RelativeError
MPRAGEData
sRAKI
AC-LORAKS
LORAKI(Original)
LORAKI(Synthetic)
Figure 5.4: Error spectrum plots [1] corresponding to (left) the T2-weighted reconstruction results
shown in Fig. 5.2 and (right) the T1-weighted reconstruction results shown in Fig. 5.5.
The two previous examples used a relatively large amount of acquired ACS data, which is
likely benecial for methods like RAKI and LORAKI, but which might also reduce experimental
eciency. In the next set of experiments, we performed reconstructions with dierent amounts of
ACS data, while holding the eective acceleration factor xed. For the T2-weighted data, we varied
the number of fully-sampled lines at the center of k-space, and performed uniform undersampling
of the remainder of k-space. The sample spacing was adjusted so that the total number of lines
was equal for each case, with an eective acceleration factor of 2.8. For the T1-weighted data, we
varied the size of the fully-sampled region at the center of k-space, and used variable density random
sampling for the remainder of k-space. The total number of samples was held xed in each case,
with an eective acceleration factor of 5.2. Results for T2-weighted and T1-weighted datasets
are shown in Figs. 5.6 and 5.7, respectively. As expected, LORAKI using the original ACS data
works better than AC-LORAKS when the amount of ACS data is large, but AC-LORAKS yields
better results when the amount of ACS data is small. However, these results also demonstrate that
synthetic ACS data is potentially quite valuable when the amount of actual ACS data is relatively
small. In particular, LORAKI with synthetic ACS data appears to consistently outperform AC-
LORAKS across all cases. Notably, we also observe that both LORAKI and AC-LORAKS appear
to consistently outperform sRAKI in all cases.
102
AC-LORAKS
Variable Density
Sampling
NRMSE: 0.0890
SSIM: 0.962
NRMSE: 0.0855
SSIM: 0.971
NRMSE: 0.0851
SSIM: 0.969
Gold Standard
LORAKI
(Original)
LORAKI
(Synthetic)
sRAKI
NRMSE: 0.0959
SSIM: 0.964
Figure 5.5: Representative reconstruction results for randomly-undersampled T1-weighted data.
The top row shows reconstructed images for one slice in a linear grayscale, where the gold standard
image has been normalized to range from 0 (black) to 1 (white). The bottom row shows error
images with the indicated colorscale. NRMSE and SSIM values are also shown below each image,
with the best values highlighted in red.
To evaluate the hypothesis that LORAKI would be compatible with a range of dierent sampling
patterns (a characteristic that it should inherit from AC-LORAKS), we performed reconstruction of
the T2-weighted data from partial Fourier undersampling (5/8ths partial Fourier sampling including
24 fully-sampled lines of central k-space to be used as ACS data, with uniform sampling of the
remaining k-space resulting in an eective acceleration factor of 3:3). LORAKI reconstruction
results for one slice are shown in Fig. 5.8, and AC-LORAKS results are also included for reference.
As can be seen, the advantage of LORAKI over AC-LORAKS is still observed for this sampling
pattern, and there is still not a major dierence between using the original ACS data and synthetic
ACS data.
5.6 Discussion
The results shown in the previous sections demonstrated that LORAKI has potential advantages
compared to existing AA methods when sucient ACS data is available, and also that synthetic
ACS training data is potentially useful for scenarios where it may be impractical to acquire a large
amount of actual ACS data. In practice, there can also be certain scenarios where no ACS training
data is available, where existing calibrationless reconstruction methods like SAKE and LORAKS
103
AC-LORAKS
LORAKI
(Original)
LORAKI
(Synthetic)
sRAKI
8 ACS Lines 16 ACS Lines 24 ACS Lines 32 ACS Lines
GRAPPA
NRMSE: 0.139
SSIM: 0.923
NRMSE: 0.149
SSIM: 0.892
NRMSE: 0.148
SSIM: 0.889
NRMSE: 0.173
SSIM: 0.860
NRMSE: 0.176
SSIM: 0.904
NRMSE: 0.136
SSIM: 0.920
NRMSE: 0.0935
SSIM: 0.946
NRMSE: 0.102
SSIM: 0.943
NRMSE: 0.0952
SSIM: 0.958
NRMSE: 0.0897
SSIM: 0.946
NRMSE: 0.0738
SSIM: 0.958
NRMSE: 0.0992
SSIM: 0.946
NRMSE: 0.115
SSIM: 0.947
NRMSE: 0.0954
SSIM: 0.954
NRMSE: 0.0645
SSIM: 0.969
NRMSE: 0.0829
SSIM: 0.957
NRMSE: 0.0918
SSIM: 0.960
NRMSE: 0.0759
SSIM: 0.962
NRMSE: 0.0646
SSIM: 0.967
NRMSE: 0.0908
SSIM: 0.956
48 ACS Lines
NRMSE: 0.121
SSIM: 0.916
NRMSE: 0.109
SSIM: 0.940
NRMSE: 0.0912
SSIM: 0.962
NRMSE: 0.0813
SSIM: 0.963
NRMSE: 0.0894
SSIM: 0.964
Figure 5.6: Reconstruction results for T2-weighted data with varying amounts of ACS data (with
xed total acceleration rate). Error images are shown using the same colorscale from Fig. 5.2.
(which are also based on linear autoregressive modeling principles) have previously demonstrated
value [3, 7, 61]. While the LORAKI formulation does not directly address calibrationless scenarios,
104
AC-LORAKS
LORAKI
(Original)
LORAKI
(Synthetic)
NRMSE: 0.0966
SSIM: 0.947
16x16 ACS
NRMSE: 0.0845
SSIM: 0.961
NRMSE: 0.0859
SSIM: 0.963
NRMSE: 0.0890
SSIM: 0.962
NRMSE: 0.0995
SSIM: 0.958
NRMSE: 0.123
SSIM: 0.953
NRMSE: 0.167
SSIM: 0.921
NRMSE: 0.121
SSIM: 0.952
NRMSE: 0.0882
SSIM: 0.968
NRMSE: 0.0855
SSIM: 0.971
NRMSE: 0.0916
SSIM: 0.970
NRMSE: 0.116
SSIM: 0.963
NRMSE: 0.0802
SSIM: 0.972
NRMSE: 0.0786
SSIM: 0.974
NRMSE: 0.0824
SSIM: 0.972
NRMSE: 0.0851
SSIM: 0.969
NRMSE: 0.0974
SSIM: 0.968
NRMSE: 0.121
SSIM: 0.959
32x32 ACS 48x48 ACS 64x64 ACS 80x80 ACS 96x96 ACS
NRMSE: 0.186
SSIM: 0.904
NRMSE: 0.124
SSIM: 0.944
NRMSE: 0.102
SSIM: 0.959
NRMSE: 0.0959
SSIM: 0.964
NRMSE: 0.103
SSIM: 0.963
NRMSE: 0.132
SSIM: 0.947
sRAKI
Figure 5.7: Reconstruction results for T1-weighted data with varying amounts of ACS data (with
xed total acceleration rate). Error images are shown using the same colorscale from Fig. 5.5.
it is worth noting that LORAKI could also potentially be applied to such scenarios if synthetic
ACS data can be generated (e.g., by applying a calibrationless reconstruction method as an initial
step). As an initial proof-of-principle for this idea, we performed three dierent calibrationless
simulations, as shown in Fig. 5.9. We simulated calibrationless partial Fourier with an eective
acceleration factor of 3:0. We used the \S"-version of the nonconvex P-LORAKS method [7] (using
publicly available software [11]) to generate an initial reconstruction. This initial reconstruction
was then used as synthetic ACS training data to train LORAKI, and LORAKI reconstruction was
then performed. Reconstruction results are shown in Fig. 5.9, and we also show the P-LORAKS
reconstructions and AC-LORAKS reconstructions (trained using the P-LORAKS reconstruction as
ACS data) for reference. As can be seen, the LORAKI reconstruction has the best performance
105
LORAKI
(Synthetic) AC-LORAKS
LORAKI
(Original)
Gold Standard
NRMSE: 0.0735
SSIM: 0.965
NRMSE: 0.0898
SSIM: 0.944
NRMSE: 0.0727
SSIM: 0.964
Partial Fourier
Sampling
Figure 5.8: AC-LORAKS and LORAKI reconstruction results for T2-weighted data with partial
Fourier sampling patterns. Error images are shown using the same colorscale from Fig. 5.2.
metrics compared to P-LORAKS and AC-LORAKS. These results conrm that LORAKI-type
approaches can still have relevance to calibrationless scenarios.
While our results suggest that synthetic training data can be helpful for LORAKI, it should also
be noted that it can also be used to improve other AA methods like GRAPPA and sRAKI in cases
where the ACS data may be insucient. To test this, we considered the case from Fig. 5.6 with the
least amount of ACS data (8 lines), and derived synthetic ACS data from the AC-LORAKS result
from that gure. This ACS data was then used with GRAPPA, sRAKI, and LORAKI, with results
shown in Fig. 5.10. Results show that using the synthetic data training data does indeed improve
the performance of all three reconstruction methods, although the best overall performance was
still achieved by LORAKI.
The results shown in this paper were all generated using the same set of LORAKI network
parameters. However, dierent choices of these parameters are expected to have an impact on
reconstruction performance. Figure 5.11 illustrates the impact of the parameters C (the number
106
LORAKI
(Synthetic) P-LORAKS
AC-LORAKS
(Synthetic)
Gold Standard
NRMSE: 0.0706
SSIM: 0.973
NRMSE: 0.0793
SSIM: 0.963
NRMSE: 0.0784
SSIM: 0.964
Calibrationless
Partial Fourier
Random
Figure 5.9: Evaluation of calibrationless reconstruction using synthetic ACS data. Error images
are shown using the same colorscale from Fig. 5.2.
of channels in the hidden layer), K (the number of RNN iterations), and R
1
R
2
(the size of the
ellipsoidal convolution kernels) on reconstruction performance for the same scenario considered in
Fig. 5.2. As can be seen, the performance of LORAKI appears to be relatively robust with respect
to variations in these parameters, and maintains an advantage over AC-LORAKS across a wide
range of parameter settings.
Unlike most other deep learning methods, the training procedure for RAKI, sRAKI, and LO-
RAKI is scan-specic and therefore must be performed online. As a result, training time becomes
an important consideration. For the results we've shown, LORAKI training required approximately
20 minutes when using the original ACS data and approximately 1 minute when using the syn-
thetic ACS data, running on a single GPU through Google Colab. Note that the training time for
synthetic ACS data was substantially shorter than for the original ACS data. This occurs because,
for synthetic data, the ACS sizeM
1
M
2
is matched to the reconstruction sizeN
1
N
2
. This leads
to only a single undersampled/fully-sampled training pair, which enables much faster learning. On
107
LORAKI
(Synthetic)
sRAKI
(Synthetic)
Gold Standard
NRMSE: 0.0918
SSIM: 0.960
NRMSE: 0.104
SSIM: 0.954
NRMSE: 0.0958
SSIM: 0.960
Uniform
Sampling
(ACS 8)
GRAPPA
(Synthetic)
Figure 5.10: Comparison of GRAPPA, RAKI, and LORAKI with synthetic data training using
AC-LORAKS reconstruction. Error images are shown using the same colorscale from Fig. 5.2.
the other hand, M
1
M
2
is substantially smaller than N
1
N
2
for the original ACS data, and
multiple training pairs are required to cover the variations in the sampling pattern. This is another
potential advantage for using synthetic ACS training data. While the training time is still relatively
long in all cases, we should note that this implementation was designed for simple proof-of-principle
evaluation, and we did not spend much eort in optimizing training speed. Substantial speedups
may be possible from using better hardware and more ecient training algorithms.
While we've demonstrated that LORAKI improves reconstruction performance over similar
AA methods, we should also mention that the version of LORAKI described in this paper is still
relatively simple, and for example does not consider more advanced deep learning strategies such
as dropout [117] or batch normalization [118]. Combination of LORAKI-type ideas with other
forms of constrained reconstruction can also improve results even further, as illustrated recently in
preliminary extensions of LORAKI [119, 120].
108
20 30 40 50 60 70 80
C
NRMSE
LORAKI
AC-LORAKS
2 4 6 8
R 1
NRMSE
LORAKI
AC-LORAKS
5 10 15 20
K
NRMSE
LORAKI
AC-LORAKS
0.06
0.07
0.08
0.09
0.10
0.06
0.08
0.10
0.12
0.14
0.16
0.06
0.07
0.08
0.09
0.11
0.10
Figure 5.11: Evaluating the eects of dierent LORAKI network parameters on reconstruction
performance. The rst plot shows the eects of varying the number of hidden-channel layers C
while holding the kernel size xed at R
1
= R
2
= 5 and the number of iterations xed at K = 5.
The second plot shows the eects of varying R
1
, while setting R
2
= R
1
and holding the other
parameters xed atC = 64 andK = 5. The nal plot shows the eects of varyingK, while holding
the other parameters xed at C = 64 and R
1
=R
2
= 5. For reference, the NRMSE value for AC-
LORAKS reconstruction with optimized parameters is also shown (the AC-LORAKS parameters
are not varied in this plot).
5.7 Conclusion
This work introduced LORAKI, a novel scan-specic autocalibrated RNN approach for nonlinear
autoregressive MRI reconstruction in k-space that was motivated by ideas from previous RAKI
and AC-LORAKS methods. LORAKI is designed to automatically capture the same support,
phase, and parallel imaging constraints as AC-LORAKS while also maintaining compatibility with
a wide range of k-space sampling patterns, and without requiring a database of preexisting training
data. However, dierent from AC-LORAKS but similar to RAKI, the reconstruction procedure
in LORAKI is nonlinear, and capable of capturing more complicated autoregressive relationships.
Our evaluations with retrospectively undersampled MRI data suggest that LORAKI can outper-
form similar existing reconstruction methods in many situations, and we envision that the further
development of this kind of approach may enable even bigger gains in the future.
109
This page intentionally left blank
Chapter 6
Ecient Iterative Solutions to
Complex-Valued Nonlinear
Least-Squares Problems with Mixed
Linear and Antilinear Operators
6.1 Introduction
Consider a generic complex-valued nite-dimensional inverse problem scenario in which the forward
model is represented as
b =A(x) + n; (6.1)
where b2 C
M
represents the measured data,A() : C
N
! C
M
is the measurement operator,
n2 C
M
represents noise, and x2 C
N
represents the unknown signal that we wish to estimate
based on knowledge of b andA(). A common approach to solving this inverse problem is to nd
a least-squares solution
^ x = arg min
x2C
N
kA(x) bk
2
2
; (6.2)
wherekk
2
denotes the standard `
2
-norm. This choice of formulation can be justied in multiple
ways, and e.g., corresponds to the optimal maximum likelihood estimator when the noise vector n
is independent and identically-distributed (i.i.d.) Gaussian noise [121]. Even for more complicated
noise statistics that follow, e.g., the Poisson, Rician, or non-Central Chi distributions, there exist
111
iterative methods that allow the maximum likelihood estimator to be obtained by iteratively solv-
ing a sequence of least-squares objective functions [122{124]. In addition, another reason for the
popularity of least-squares is that the optimization problem is frequently very easy to solve. For
example, in the case whereA() is a linear operator (i.e.,A() can be represented in an equivalent
matrix form asA(x) = Ax for some matrix A2C
MN
) with a trivial nullspace, the solution to
Eq. (6.2) has the analytic closed-form expression [125]
^ x = (A
H
A)
1
A
H
b; (6.3)
where
H
denotes the conjugate-transpose operation. In large-scale problems where N is very
large, the matrix inversion in Eq. (6.3) may be computationally intractable, although there exist a
variety of simple iterative algorithms that are guaranteed to converge to a globally-optimal solution,
including Landweber iteration [126], the conjugate gradient (CG) algorithm [101], and LSQR [127].
Instead of assuming linearity, we focus in this work on solving least-squares problems in the
scenario whereA() is nonlinear, but can be represented as the summation and/or composition of
some operators that are linear and some operators that are antilinear. Such nonlinear operators
have sometimes been termed as real-linear operators in mathematical physics [128]. Important
common examples of operators that possess this kind of nonlinear structure include the complex-
conjugation operator
A(x) = x; (6.4)
the operator that takes the real part of a complex vector
A(x) = real(x),
1
2
x +
1
2
x; (6.5)
and the operator that takes the imaginary part of a complex vector
A(x) = imag(x),
1
2i
x
1
2i
x: (6.6)
112
Even though the descriptions we present in this paper are generally applicable to arbitrary
real-linear operators, we were initially motivated to consider such operators because of specic
applications in magnetic resonance imaging (MRI) reconstruction. In particular, MRI images are
complex-valued, and real-linear operators have previously been used to incorporate prior informa-
tion about the image phase characteristics into the image reconstruction process, which helps to
regularize/stabilize the solution when the inverse problem is ill posed. For example, there is a line
of research within MRI that poses phase-constrained image reconstruction as [2, 15{17, 19, 20]
^ x = arg min
x2C
N
kAx bk
2
2
+kimag(Bx)k
2
2
= arg min
x2C
N

2
6
6
4
Ax
p
imag(Bx)
3
7
7
5

2
6
6
4
b
0
3
7
7
5

2
2
;
(6.7)
where 2 R is a positive regularization parameter and the matrix B embeds prior information
about the image phase such that the regularization encourages Bx to be real-valued. Another line
of research within MRI instead imposes phase constraints by leveraging linear predictability and
the conjugate-symmetry characteristics of the Fourier transform, leading to an inverse problem
formulation that can take the general form [3, 4, 6, 11]
^ x = arg min
x2C
N
kAx bk
2
2
+kCx D(Ex)k
2
2
= arg min
x2C
N

2
6
6
4
Ax
p
Cx
p
D(Ex)
3
7
7
5

2
6
6
4
b
0
3
7
7
5

2
2
;
(6.8)
for appropriate matrices C, D, and E.
Although these are nonlinear least-squares problems because the operators involved are nonlin-
ear, previous work has benetted from the fact that this kind of inverse problem can be transformed
into an equivalent higher-dimensional real-valued linear least-squares problem [2{4, 6, 11, 15{
17, 19, 20]. Specically, this can be done by replacing all complex-valued quantities with real-valued
quantities, e.g., separating x2C
N
into its real and imaginary components, and treating this as an
113
inverse problem in R
2N
rather than the original space C
N
. While this real-valued transformation
of the problem is eective and enables the use of standard linear least-squares solution methods, it
can also cause computational ineciencies and can sometimes be dicult to implement when the
operators involved have complicated structure.
In this work, we describe theory that enables provably-convergent linear least-squares iterative
algorithms to be applied to this nonlinear least-squares problem setting, without requiring a real-
valued transformation of the original complex-valued vectors and operators. This can enable both
improved computation speed and simplied algorithm implementations.
6.2 Background
6.2.1 Linear, Antilinear, and Real-Linear Operators
In this section, we brie
y summarize some denitions and properties of linear and antilinear op-
erators, with simplications corresponding to our nite-dimensional problem context. Readers
interested in a more detailed and more general treatment are referred to Refs. [128, 129].
Denition 1 (Linear Operator) An operatorF() :C
N
!C
M
is said to be linear (or complex-
linear) if it satises both additivity
F(x + y) =F(x) +F(y) for8x; y2C
N
(6.9)
and homogeneity
F(x) =F(x) for8x2C
N
;82C: (6.10)
Property 1 For any linear operatorF() :C
N
!C
M
, there is a unique matrix F2C
MN
such
thatF(x) = Fx for8x2C
N
.
Denition 2 (Antilinear Operator) An operatorG() :C
N
!C
M
is said to be antilinear (or
conjugate-linear) if it satises both additivity
G(x + y) =G(x) +G(y) for8x; y2C
N
(6.11)
114
and conjugate homogeneity
G(x) =G(x) for8x2C
N
;82C: (6.12)
Property 2 For any antilinear operatorG() : C
N
! C
M
, there is a unique matrix G2 C
MN
such thatG(x) = (Gx) for8x2C
N
.
Note that by taking the matrix G as the identity matrix, we observe that applying complex
conjugation x is an antilinear operation on the vector x.
Denition 3 (Real-Linear Operator) An operatorA() :C
N
!C
M
is said to be real-linear if
it satises both additivity
A(x + y) =A(x) +A(y) for8x; y2C
N
(6.13)
and homogeneity with respect to real-valued scalars
A(x) =A(x) for8x2C
N
;82R: (6.14)
Real-linearity is a generalization of both linearity and antilinearity, as can be seen from the
following property.
Property 3 Every real-linear operatorA() :C
N
!C
M
can be uniquely decomposed as the sum
of a linear operator and an antilinear operator. In particular,A(x) =F(x) + G(x) for8x2C
N
,
whereF() :C
N
!C
M
is the linear operator dened by
F(x),
1
2
A(x)
i
2
A(ix) (6.15)
andG() :C
N
!C
M
is the antilinear operator dened by
G(x),
1
2
A(x) +
i
2
A(ix): (6.16)
115
Property 4 For any real-linear operatorA() : C
N
! C
M
, there are unique matrices F; G2
C
MN
such thatA(x) = Fx + (Gx) for8x2C
N
.
Notably, both the real() and imag() operators from Eqs. (6.5) and (6.6) are observed to have
real-linear form.
Property 5 For any two real-linear operatorsA
1
() :C
N
!C
M
andA
2
() :C
N
!C
M
, their sum
A
1
() +A
2
() is also a real-linear operator.
Property 6 For any two real-linear operatorsA
1
() : C
N
! C
P
andA
2
() : C
P
! C
M
, their
compositionA
2
()A
1
() :C
N
!C
M
,A
2
(A
1
())) is also a real-linear operator.
As can be seen, any operator that can be represented as the summation and/or composition of
some operators that are linear and some operators that are antilinear can be viewed as a real-linear
operator. As a result, the scenarios of interest in this paper all involve real-linear operators, and
the remainder of this paper will assume thatA() obeys real-linearity, and has been decomposed
in matrix form asA(x) = Fx + (Gx).
6.2.2 Real-Valued Transformation of Complex-Valued Least Squares
AssumingA() is real-linear as described in the previous subsection, Eq. (6.2) can be rewritten as
^ x = arg min
x2C
N
kFx + (Gx) bk
2
2
; (6.17)
which is a nonlinear least squares problem. However, as stated in the introduction, previous work
[2{4, 6, 11, 15{17, 19, 20] has transformed this problem into the form of a conventional linear least-
squares problem by treating the variable x as an element ofR
2N
instead ofC
N
. This was achieved
by rewriting x2C
N
as x = x
r
+ix
i
, where the real-valued vectors x
r
; x
i
2R
N
represent the real
116
and imaginary components of x. This allows us to equivalently rewrite the solution to Eq. (6.17)
as ^ x = ^ x
r
+i^ x
i
, with
f^ x
r
; ^ x
i
g = arg min
xr;x
i
2R
N
kFx
r
+iFx
i
+ Gx
r
iGx
i
bk
2
2
= arg min
xr;x
i
2R
N

2
6
6
4
real(Fx
r
+iFx
i
+ Gx
r
iGx
i
b)
imag(Fx
r
+iFx
i
+ Gx
r
iGx
i
b)
3
7
7
5

2
2
= arg min
~ x2R
2N

~
A~ x
~
b

2
2
;
(6.18)
where
~ x,
2
6
6
4
x
r
x
i
3
7
7
5
2R
2N
; (6.19)
~
A,
2
6
6
4
real(F) + real(G) imag(F) imag(G)
imag(F) imag(G) real(F) real(G)
3
7
7
5
2R
2M2N
; (6.20)
and
~
b,
2
6
6
4
real(b)
imag(b)
3
7
7
5
2R
2M
: (6.21)
The nal expression in Eq. (6.18) has the form of a standard real-valued linear least-squares prob-
lem, and therefore can be solved using any of the linear least-squares solution methods described in
the introduction. For example, the Landweber iteration [126] applied to this problem would pro-
ceed as given in Algorithm 1, and with innite numerical precision, ^ x
k
is guaranteed to converge
to a globally optimal solution as k!1 whenever 0<< 2=k
~
Ak
2
2
.
As another example, the CG algorithm [101] applied to this problem would proceed as given
in Algorithm 2, and with innite numerical precision, ^ x
k
would be guaranteed to converge to a
globally optimal solution after at most 2N iterations.
117
Algorithm 1: Landweber Iteration applied to Eq. (6.18)
Inputs:
~
A2R
2M2N
,
~
b2R
2M
, ~ x
0
2R
2N
(initial guess for ~ x), and 2R
(step size parameter)
Initialization:
k = 0;
Iteration:
While stopping conditions are not met:
~ x
k+1
= ~ x
k
+
~
A
H
(
~
b
~
A~ x
k
);
k =k + 1;
Output: Final value of ~ x
k+1
Algorithm 2: Conjugate Gradient Algorithm applied to Eq. (6.18)
Inputs:
~
A2R
2M2N
,
~
b2R
2M
, and ~ x
0
2R
2N
(initial guess for ~ x)
Initialization:
r
0
=
~
A
H
(
~
b
~
A~ x
0
);
p
0
= r
0
;
k = 0;
Iteration:
While stopping conditions are not met:
z
k
=
~
A
H
~
Ap
k
;

k
= (r
H
k
r
k
)=(p
H
k
z
k
);
~ x
k+1
= ~ x
k
+
k
p
k
;
r
k+1
= r
k

k
z
k
;

k
= (r
H
k+1
r
k+1
)=(r
H
k
r
k
);
p
k+1
= r
k+1
+
k
p
k
;
k =k + 1;
Output: Final value of ~ x
k+1
Compared to the analytic linear least-squares solution corresponding to Eq. (6.3), these iter-
ative algorithms are generally useful for larger-scale problems where the matrix
~
A may be too
large to store in memory, and where the matrix has structure so that matrix-vector multiplications
with
~
A and
~
A
H
can be computed quickly using specially-coded function calls rather than work-
ing with actual matrix representations (e.g., if
~
A has convolution structure so that matrix-vector
multiplication can be implemented using the Fast Fourier Transform, if
~
A is sparse, etc.).
Although the problem transformation from Eq. (6.18) has been widely used [2{4, 6, 11, 15{
17, 19, 20], it can also be cumbersome to work with if the operatorA() has more complicated
structure. For example, the optimization problem in Eq. (6.8) involves the composition of linear
and antilinear operators, and the
~
A matrix corresponding to this case has a complicated structure
118
that is laborious to derive. In particular, with much manipulation, the matrix for this case can be
derived to be
~
A =
2
6
6
6
6
6
6
6
6
6
6
6
4
real(A) imag(A)
H
11
H
12
imag(A) real(A)
H
21
H
22
3
7
7
7
7
7
7
7
7
7
7
7
5
; (6.22)
with
H
11
=
p
real(C)
p
real(D)real(E)
p
imag(D)imag(E); (6.23)
H
12
=
p
imag(C) +
p
real(D)imag(E)
p
imag(D)real(E); (6.24)
H
21
=
p
imag(C)
p
imag(D)real(E) +
p
real(D)imag(E); (6.25)
and
H
22
=
p
real(C) +
p
imag(D)imag(E) +
p
real(D)real(E): (6.26)
Of course, Eq. (6.8) relies on a relatively simple mixture of linear and antilinear operators, and
problems involving more complicated mixtures would be even more laborious to derive.
Beyond just the eort required to compute the general form of
~
A, it can also be computationally
expensive to try to use this type of expression in an iterative algorithm, particularly when the
dierent operators have been implemented as specially-coded function calls. For example, if we
were not given the actual matrix representations of A, C, D, and E in Eq. (6.22) and only had
function calls that implemented matrix-vector multiplication with these matrices, then a naive
implementation of matrix multiplication between
~
A and a vector would require 4 calls to the
function that computes multiplication with A (e.g., to compute real(A)r for an arbitrary real-
valued vector r2R
N
, we could instead compute the complex-valued matrix-vector multiplication
function call to obtain s = Ar, and then use real(A)r = real(s), with an analogous approach
119
for computing imag(A)t for an arbitrary real-valued vector t2R
N
), 4 calls to the function that
computes multiplication with C, 8 calls to the function that computes multiplication with D,
and 8 calls to the function that computes multiplication with E. This relatively large number of
function calls represents a substantial increase in computational complexity compared to a standard
evaluation of the complex-valued forward model, which would only require the use of one function
call for each operator. Of course, this number of computations is based on a naive implementation,
and additional careful manipulations could be used to reduce these numbers of function calls by
exploiting redundant computations { however, this would contribute further to the laborious nature
of deriving the form of
~
A.
6.3 Main Results
Our main results are given by the following lemmas, which enable the use of the real-valued linear
least-squares framework from Sec. 6.2.2 while relying entirely on complex-valued representations
and computations.
Lemma 1 Consider a real-linear operatorA() : C
N
! C
M
, with corresponding
~
A matrix as
dened in Eq. (6.20). Also consider arbitrary vectors m2C
N
and n2C
M
, which are decomposed
into their real and imaginary components according to m = m
r
+im
i
and n = n
r
+in
i
, with
m
r
; m
i
2R
N
and n
r
; n
i
2R
M
. Then
~
A
2
6
6
4
m
r
m
i
3
7
7
5
=
2
6
6
4
real(A(m))
imag(A(m))
3
7
7
5
(6.27)
and
~
A
H
2
6
6
4
n
r
n
i
3
7
7
5
=
2
6
6
4
real(A

(n))
imag(A

(n))
3
7
7
5
; (6.28)
withA

() dened below.
120
Denition 4 (A

()) Consider a real-linear operatorA() : C
N
! C
M
, which is represented for
8x2C
N
asA(x) = Fx + (Gx) for some matrices F; G2C
MN
. We deneA

() :C
M
!C
N
as
the mappingA

(n), F
H
n + G
H
n for8n2C
M
.
Note thatA

() is also a real-linear operator, and can be equivalently written in real-linear
form asA

(n), F
H
n + (G
T
n) for8n2C
M
, where
T
denotes the transpose operation (without
conjugation). Interestingly, it can also be shown thatA

() matches the denition of the adjoint
operator ofA() from real-linear operator theory [128].
Lemma 2 Consider a real-linear operatorA() :C
N
!C
M
that can be written as the composition
A() =A
2
()A
1
() of real-linear operatorsA
1
() : C
N
! C
P
andA
2
() : C
P
! C
M
. Then
A

(n) =A

1
(A

2
(n))) for8n2C
M
.
Lemma 3 Consider a real-linear operatorA() :C
N
!C
M
that can be written as the summation
A() =A
1
() +A
2
() of real-linear operatorsA
1
() : C
N
! C
M
andA
2
() : C
N
! C
M
. Then
A

(n) =A

1
(n) +A

2
(n) for8n2C
M
.
The proofs of these three lemmas are straightforward, and are given in the appendices. When
combined together, these three lemmas completely eliminate the need to derive or work with the
real-valued matrix
~
A in the context of iterative algorithms, because the eects of multiplication with
the real-valued matrices
~
A and
~
A
H
can be obtained equivalently using the complex-valued nonlinear
operatorsA() andA

(). This can also lead to computational savings, since e.g., computing
real(A(m)) and imag(A(m)) (as needed for computing multiplication of the matrix
~
A with a
vector using Eq. (6.27)) only requires a single call to the function that computesA(m). Likewise,
computing multiplication of the matrix
~
A
H
with a vector only requires a single call to the function
that computesA

(). And further, ifA() is represented as a complicated summation and/or
composition of real-linear operators, we can rely on Properties 5 and 6 and Lemmas 2 and 3 to
work incrementally with the individual constituent operators, rather than having to work with the
monolithic composite operator in its entirety.
As a consequence of these lemmas, it is, e.g., possible to replace the real-valued Landweber
iteration from Algorithm 1 with the simpler complex-valued iteration given by Algorithm 3.
121
Algorithm 3: Proposed Complex-Valued Landweber Iteration
Inputs:A() :C
N
!C
N
, b2C
M
, x
0
2C
N
(initial guess for x), and 2R
(step size parameter)
Initialization:
k = 0;
Iteration:
While stopping conditions are not met:
x
k+1
= x
k
+A

(bAx
k
);
k =k + 1;
Output: Final value of x
k+1
With innite numerical precision, Algorithm 3 will produce the exact same sequence of iter-
ates as Algorithm 1, and will therefore have the exact same global convergence guarantees stated
previously for Landweber iteration.
We can make similar modications to the CG algorithm from Algorithm 2, although need the
following additional property to be able to correctly handle the inner-products appearing in the
CG algorithm.
Property 7 Consider arbitrary vectors p; q2C
N
, which are decomposed into their real and imag-
inary components according to p = p
r
+ip
i
and q = q
r
+iq
i
, with p
r
; p
i
; q
r
; q
i
2 R
N
. Dene
~ p; ~ q2R
2N
according to
~ p =
2
6
6
4
p
r
p
i
3
7
7
5
and ~ q =
2
6
6
4
q
r
q
i
3
7
7
5
(6.29)
Then ~ p
H
~ q = real(p
H
q).
Combining this property with the previous lemmas leads to the simple complex-valued iteration
for the CG algorithm given by Algorithm 4.
While we have only shown complex-valued adaptations of the Landweber and CG algorithms,
this same approach is easily applied to other related algorithms like LSQR [127].
122
Algorithm 4: Proposed Complex-Valued Conjugate Gradient Algorithm
Inputs:A() :C
N
!C
N
, b2C
M
, and x
0
2C
N
(initial guess for x)
Initialization:
r
0
=A

(bA(x
0
));
p
0
= r
0
;
k = 0;
Iteration:
While stopping conditions are not met:
z
k
=A

(A(p
k
));

k
= (r
H
k
r
k
)=real(p
H
k
z
k
);
x
k+1
= x
k
+
k
p
k
;
r
k+1
= r
k

k
z
k
;

k
= (r
H
k+1
r
k+1
)=(r
H
k
r
k
);
p
k+1
= r
k+1
+
k
p
k
;
k =k + 1;
Output: Final value of x
k+1
6.4 Useful Relations for Common Real-Linear Operators
Before demonstrating the empirical characteristics of our proposed new approach, we believe that
our proposed framework will be easier to use if we enumerated some of the most common real-linear
A() operators and their correspondingA

() operators. Such a list is provided in Table 6.1.
6.5 Numerical Example
To demonstrate the potential benets of our proposed complex-valued approach, we will consider an
instance of the problem described by Eq. (6.8). In this case, the use of complex-valued operations
can lead to both a simpler problem formulation and faster numerical computations.
To address simplicity, we hope that it is obvious by inspection that the process of deriving
~
A for
this case (as given in Eq. (6.22), and needed for the conventional real-valued iterative computations)
was non-trivial and labor-intensive, while the derivation ofA() andA

() (as given in Table 6.1,
and needed for the proposed new complex-valued iterative computations) was comparatively fast
and easy.
To address the computational benets of the proposed approach, we will consider a specic
realization of Eq. (6.8), in which x 2 C
1000
, n 2 C
20000
, A 2 C
200001000
, C 2 C
300001000
,
123
A(x) for
x2C
N
A

(y) for
y2C
M
A

(A(x)) for
x2C
N
Real-linear Fx + (Gx) F
H
y + G
H
y
Conjugation x y x
Real part real(x) real(y) real(x)
Imaginary part imag(x) i real(y) i imag(x)
System from
Eq. (6.7)
"
Ax
p
imag(Bx)
#
A
H
y
1
+
p
iB
H
real(y
2
)
A
H
Ax
+iB
H
imag(Bx)
System from
Eq. (6.8)
"
Ax
p
Cx
p
D(Ex)
#
A
H
y
1
+
p
B

(y
2
)
A
H
Ax
+B

(B(x))
Table 6.1: Table of common real-linearA() operators and correspondingA

() operators. We also
provide expressions forA

(A()) in cases where the combined operator takes a simpler form than
applying each operator sequentially. In the last two rows, it is assumed that the matrix A2C
M
1
N
,
and that the vector y2 C
M
is divided into two components y
1
2 C
M
1
and y
2
2 C
MM
1
with
y =
h
y
T
1
y
T
2
i
T
. In the last row, we takeB(x), Cx D(Ex), with correspondingB

(y) =
C
H
yE
H
(D
H
y). Note that a special case of equivalent complex-valued operators associated with
Eq. (6.7) (with B chosen as the identity matrix) was previously presented by Ref. [2], although
without the more general real-linear mathematical framework developed in this work.
D2C
300002000
, and E2C
20001000
, with the real and imaginary parts of all of these vectors and
matrices drawn at random from the i.i.d. Gaussian distribution. We then took b = Ax+n, and set
= 10
3
. For this random problem instance, we nd the optimal nonlinear least-squares solution
in four distinct ways:
• Conventional Real-Valued Approach with Matrices. We assume that A, C, D, and E
are available to us in matrix form, such that it is straightforward to directly precompute the
real-valued matrix
~
A2R
1000002000
from Eq. (6.22). We then use this precomputed matrix
directly in iterative linear least-squares solution algorithms like Landweber iteration, CG, and
LSQR. Although the form of this
~
A matrix was complicated to derive, multiplications with
the precomputed
~
A and
~
A
H
matrices within each iteration should be very computationally
ecient, particularly since we have taken 4 separate complex-valued matrices A, C, D, and
E that were originally specied by a sum total of 1:1210
8
complex-valued entries (2:2410
8
real numbers), and replaced them with a single real-valued matrix specied by only 2 10
8
real numbers.
124
• Proposed Complex-Valued Approach with Matrices. As in the previous case, we
assume that A, C, D, and E are available to us in matrix form, which allows us to directly
form the F and G matrices corresponding to the complex-valued real-linear formulation of
the problem. Specically, F was formed as
F =
2
6
6
4
A
p
C
3
7
7
5
(6.30)
and G was formed as
G =
2
6
6
4
0

p
DE
3
7
7
5
: (6.31)
We then used these precomputed matrices to evaluateA() andA

() as needed in our pro-
posed complex-valued iterative algorithms.
• Conventional Real-Valued Approach with Function Calls. We assume that we do not
have direct access to the A, C, D, and E matrices, but are only given blackbox functions that
calculate matrix-vector multiplications with these matrices and their conjugate transposes.
As such, we implement matrix-vector multiplication with
~
A (and similarly for
~
A
H
) naively
in each iteration of the conventional iterative linear least-squares solution algorithms, using
multiple calls to each of these functions as described in Section 6.2.2. This approach is not
expected to be computationally ecient given the large number of function calls, although
is simpler to implement than more advanced approaches that might be developed to exploit
redundant computations within Eq. (6.22).
• Proposed Complex-Valued Approach with Function Calls. As in the previous case,
we assume that we do not have direct access to the A, C, D, and E matrices, but are only
given blackbox functions that calculate matrix-vector multiplications with these matrices and
their conjugate transposes. We implement the proposed complex-valued iterative algorithms
125
0 10 20 30 40 50
0
2
4
6
·10
10
Iterations
Multiplications
ProposedwithMatrices
ConventionalwithMatrices
ProposedwithFunctionCalls
ConventionalwithFunctionCalls
0 10 20 30 40 50
0.75
0.8
0.85
0.9
0.95
1
Iterations
NormalizedCostFunctionValue
ProposedwithMatrices
ConventionalwithMatrices
ProposedwithFunctionCalls
ConventionalwithFunctionCalls
0 10 20 30 40 50
0
10
20
30
Iterations
ComputationTime(sec)
ProposedwithMatrices
ConventionalwithMatrices
ProposedwithFunctionCalls
ConventionalwithFunctionCalls
0 10 20 30 40 50
0
1
2
3
·10
−15
Iterations
RelativeDifferenceFrom
Conventional(withMatrices)
ProposedwithMatrices
ProposedwithFunctionCalls
ConventionalwithFunctionCalls
Figure 6.1: Results for Landweber iteration. The plots show the total number of multiplications,
the normalized cost function value (normalized so that the initial value is 1), the computation time
in seconds, and the relative dierence between the solution from the conventional method with
matrices and solutions obtained with other methods.
using the techniques described in Section 6.3, using the expressions forA() andA

() given
in Table 6.1.
For the sake of reproducible research, Matlab code corresponding to this example is included as
supplementary material.
For each case, we ran 50 iterations of Landweber iteration and 15 iterations of CG and LSQR
in MATLAB 2018b, on a system with an Intel Core i7-8700K 3.70 GHz CPU processor. For each
approach, each algorithm, and at each iteration, we computed (1) the total cumulative number
126
0 2 4 6 8 10 12 14
0
0.5
1
1.5
2
·10
10
Iterations
Multiplications
ProposedwithMatrices
ConventionalwithMatrices
ProposedwithFunctionCalls
ConventionalwithFunctionCalls
0 2 4 6 8 10 12 14
0.75
0.8
0.85
0.9
0.95
1
Iterations
NormalizedCostFunctionValue
ProposedwithMatrices
ConventionalwithMatrices
ProposedwithFunctionCalls
ConventionalwithFunctionCalls
0 2 4 6 8 10 12 14
0
2
4
6
8
10
Iterations
ComputationTime(sec)
ProposedwithMatrices
ConventionalwithMatrices
ProposedwithFunctionCalls
ConventionalwithFunctionCalls
0 2 4 6 8 10 12 14
0
1
2
3
4
·10
−15
Iterations
RelativeDifferenceFrom
Conventional(withMatrices)
ProposedwithMatrices
ProposedwithFunctionCalls
ConventionalwithFunctionCalls
Figure 6.2: Results for the conjugate gradient algorithm. The plots show the total number of
multiplications, the normalized cost function value (normalized so that the initial value is 1), the
computation time in seconds, and the relative dierence between the solution from the conventional
method with matrices and solutions obtained with other methods.
of real-valued scalar multiplications (with 1 complex-valued scalar multiplication equal to 4 real-
valued scalar multiplications) used by the algorithm thus far; (2) the cost function value from
Eq. (6.8) using the current estimate (either x
k
or ~ x
k
); (3) the total computation time in seconds;
and (4) the relative `
2
-norm dierence between the x
k
value estimated from the proposed method
with function calls and the other methods, where we dene the relative `
2
-norm dierence between
arbitrary vectors p and q askpqk
2
=k
1
2
p+
1
2
qk
2
. To minimize random
uctuations in computation
speed due to background processing, the computation times we report represent the average of 15
dierent identical trials.
127
0 2 4 6 8 10 12 14
0
0.5
1
1.5
2
·10
10
Iterations
Multiplications
ProposedwithMatrices
ConventionalwithMatrices
ProposedwithFunctionCalls
ConventionalwithFunctionCalls
0 2 4 6 8 10 12 14
0.75
0.8
0.85
0.9
0.95
1
Iterations
NormalizedCostFunctionValue
ProposedwithMatrices
ConventionalwithMatrices
ProposedwithFunctionCalls
ConventionalwithFunctionCalls
0 2 4 6 8 10 12 14
0
2
4
6
8
10
Iterations
ComputationTime(sec)
ProposedwithMatrices
ConventionalwithMatrices
ProposedwithFunctionCalls
ConventionalwithFunctionCalls
0 2 4 6 8 10 12 14
0
1
2
3
4
·10
−15
Iterations
RelativeDifferenceFrom
Conventional(withMatrices)
ProposedwithMatrices
ProposedwithFunctionCalls
ConventionalwithFunctionCalls
Figure 6.3: Results for the LSQR algorithm. The plots show the total number of multiplications,
the normalized cost function value (normalized so that the initial value is 1), the computation time
in seconds, and the relative dierence between the solution from the conventional method with
matrices and solutions obtained with other methods.
Results for Landweber iteration, the CG algorithm, and LSQR are reported in Figs. 6.1-6.3,
respectively. Results conrm that, as should be expected from the theory, all of the dierent
approaches yield virtually identical cost function values and virtually identical solution estimates
x
k
=~ x
k
at each iteration for each of the dierent algorithms. There are some very minor dierences
on the order of 10
15
, which can be attributed to numerical eects resulting from nite-precision
arithmetic. In terms of computational complexity, we observe that the matrix-based approaches are
generally associated with fewer multiplications than the implementations that use function calls,
128
which should be expected because the matrix-based approaches were able to precompute simpler
consolidated matrix representations that were not available to the function call approaches.
The proposed approaches required a moderate number of multiplications, somewhat intermedi-
ate between the conventional approach with matrices (which had the fewest multiplications) and
the conventional approach with function calls (which had the most multiplications). However, in
terms of actual computation time, we observe that the conventional approach with function calls
was much slower than any of the other three methods, while the other three methods were all similar
to one another. It is perhaps surprising that the computation times are not directly proportional
to the number of multiplications, although this discrepancy is likely related to MATLAB's use of
ecient parallelized matrix multiplication libraries. Importantly, we observe that both variations
of the proposed approach are quite fast, and have computation times that are quite similar to the
conventional real-valued approach with matrices (which, as we mentioned, was expected to have
excellent computational eciency). There was negligible dierence between the computation times
assocociated with matrices and function call implementations of the proposed method, which was
denitely not the case for the conventional approaches. And in terms of implementation, the pro-
posed approach with function calls was the easiest to implement, since it didn't require us to derive
the forms of any special matrices like
~
A, F, or G, we could just directly work with the individual
original matrices A, C, D, and E.
6.6 Conclusion
This work proposed a new approach to solving nonlinear least-squares problems involving real-
linear operators. The new approach allows the use of the original complex-valued operators without
transforming them into an unwieldy real-valued form. Theoretically, the approach enables identical
iterative results as the conventional real-valued transformation, but with much simpler implemen-
tation options and potentially much faster computations. We expect the proposed approach to be
valuable for solving general complex-valued nonlinear least-squares problems involving real-linear
129
operators. Note that the proposed complex-valued approach is also an integral but previously-
undescribed component of the most recent version of an open-source MRI reconstruction software
package released by the authors [11].
6.A Proof of Lemma 1
First, note that Eq. (6.27) is a simple consequence of the derivations shown in Eq. (6.18). Thus,
the validity of Eq. (6.28) is the only thing that remains to be proved.
To see that Eq. (6.28) is valid, note that
A

(n) = F
H
n + G
H
n
= F
H
(n
r
+in
i
) + G
H
(n
r
in
i
)
=

real(F
H
) +i imag(F
H
)

(n
r
+in
i
)
+

real(G
H
) +i imag(G
H
)

(n
r
in
i
)
=

real(F
H
)n
r
imag(F
H
)n
i
+ real(G
H
)n
r
+ imag(G
H
)n
i

+i

imag(F
H
)n
r
+ real(F
H
)n
i
+ imag(G
H
)n
r
real(G
H
)n
i

=

real(F)
H
n
r
+ imag(F)
H
n
i
+ real(G)
H
n
r
imag(G)
H
n
i

+i

imag(F)
H
n
r
+ real(F)
H
n
i
imag(G)
H
n
r
real(G)
H
n
i

;
(6.32)
130
where the last line of this expression relies on the fact that imag(B
H
) = imag(B)
H
for an
arbitrary matrix B. Equation (6.32) provides a decomposition ofA

() into its real and imaginary
components, and is equivalent to
2
6
6
4
real(A

(n))
imag(A

(n))
3
7
7
5
=
2
6
6
4
real(F)
H
+ real(G)
H
imag(F)
H
imag(G)
H
imag(F)
H
imag(G)
H
real(F)
H
real(G)
H
3
7
7
5
2
6
6
4
n
r
n
i
3
7
7
5
=
2
6
6
4
real(F) + real(G) imag(F) imag(G)
imag(F) imag(G) real(F) real(G)
3
7
7
5
H2
6
6
4
n
r
n
i
3
7
7
5
=
~
A
H
2
6
6
4
n
r
n
i
3
7
7
5
;
(6.33)
where the last line comes from the denition of
~
A in Eq. (6.20). This proves the validity of
Eq. (6.28).
6.B Proof of Lemma 2
LetA
1
() : C
N
! C
P
be a real-linear operator that is represented for8x 2 C
N
asA
1
(x) =
F
1
x + (G
1
x) for some matrices F
1
; G
1
2C
PN
, and letA
2
() :C
P
!C
M
be a real-linear operator
that is represented for8y2C
P
asA
2
(y) = F
2
y + (G
2
y) for some matrices F
2
; G
2
2C
MP
. Then
the compositionA() =A
2
()A
1
() can be expressed for8x2C
N
as
A(x) =A
2
(A
1
(x))
=A
2

F
1
x + (G
1
x)

= F
2

F
1
x + (G
1
x)

+

G
2

F
1
x + (G
1
x)

= (F
2
F
1
+ G
2
G
1
)x + (F
2
G
1
+ G
2
F
1
)x:
(6.34)
131
ThusA() can be written in the real-linear formA(x) = Fx + (Gx) for8x 2 C
N
with F ,
F
2
F
1
+ G
2
G
1
and G, F
2
G
1
+ G
2
F
1
.
By Denition 4, we also have thatA

(n), F
H
n + G
H
n for8n2C
M
,A

1
(y), F
H
1
y + G
H
1
y
for8y2C
P
, andA

2
(n), F
H
2
n + G
H
2
n for8n2C
M
. Thus, we have for8n2C
M
that
A

1
(A

2
(n)) =A

1

F
H
2
n + G
H
2
n

= F
H
1

F
H
2
n + G
H
2
n

+ G
H
1

F
H
2
n + G
H
2
n

= (F
H
1
F
H
2
+ G
H
1
G
H
2
)n +

F
H
1
G
H
2
+ G
H
1
F
H
2

n
= (F
2
F
1
+ G
2
G
1
)
H
n + (F
2
G
1
+ G
2
F
1
)
H
n
= F
H
n + G
H
n
=A

(n);
(6.35)
which shows thatA

(n) =A

1
(A

2
(n)) for8n2C
M
as desired.
6.C Proof of Lemma 3
LetA
1
() : C
N
! C
M
be a real-linear operator that is represented for8x2 C
N
asA
1
(x) =
F
1
x+(G
1
x) for some matrices F
1
; G
1
2C
MN
, and letA
2
() :C
N
!C
M
be a real-linear operator
that is represented for8x2C
N
asA
2
(x) = F
2
x+(G
2
x) for some matrices F
2
; G
2
2C
MN
. Then
the summationA() =A
1
() +A
2
() can be expressed for8x2C
N
as
A(x) =A
1
(x) +A
2
(x)
= F
1
x + (G
1
x) + F
2
x + (G
2
x)
= (F
1
+ F
2
) x + (G
1
+ G
2
) x:
(6.36)
ThusA() can be written in the real-linear formA(x) = Fx + (Gx) for8x2C
N
with F, F
1
+F
2
and G, G
1
+ G
2
.
132
By Denition 4, we also have thatA

(n), F
H
n + G
H
n for8n2C
M
,A

1
(y), F
H
1
y + G
H
1
y
for8y2C
P
, andA

2
(n), F
H
2
n + G
H
2
n for8n2C
M
. Thus, we have for8n2C
M
that
A

1
(n) +A

2
(n) = F
H
1
n + G
H
1
n + F
H
2
n + G
H
2
n
= (F
H
1
+ F
H
2
)n + (G
H
1
+ G
H
2
)n
= (F
1
+ F
2
)
H
n + (G
1
+ G
2
)
H
n
= F
H
n + G
H
n
=A

(n);
(6.37)
which shows thatA

(n) =A

1
(n) +A

2
(n) for8n2C
M
as desired.
133
This page intentionally left blank
Chapter 7
Conclusion
This dissertation introduced shift-invariant autoregressive reconstruction methods for MRI. For
linear autoregressive methods, we showed improved MRI applications with LORAKS, which is a
model-based method utilizing the structured low-rank matrix modeling. For nonlinear autoregres-
sive techniques, we proposed a scan-specic articial neural network named LORAKI. Lastly, it
introduced iterative solutions to complex-valued least-squares with mixed linear and antilinear op-
erators, which enables convenient and succinct representations of iterative algorithms and provides
eciency in computation for a certain type of nonlinear least-squares problems.
As a future direction, it would be interesting to further delve into machine learning methods for
MRI reconstruction. Recent trends in machine learning for MRI reconstruction is to design large
scale neural networks trained with large set of trainging data. Such approaches can be powerful if
they are properly trained with appropriate set of data, and have been very successful in many other
research area such as in computer vision, speech recognition/synthesis, and machine translation.
However, we should be very cautious when they are applied to the medical imaging where reliability
is uncompromisable. In spite of much eort on explainable articial intelligence, interpretability is
still an ongoing issue for such approaches. Consequently, it is dicult to analyze when they succeed
and when they fail, and also hard to address issues when they fail. Therefore, it is important to
focus on the reliability in reconstruction while taking advantages of them.
One example of such approaches is scan-specic neural networks [14, 55, 115, 116]. They have
small parametric space which can be trained from its own autocalibration signal (ACS) without
135
requiring external dataset, where such ACS training has already been widely exploited and practi-
cally validated [23]. Another approach is plug-and-play (PnP) methods [57, 58], where they take a
multi-step iterative algorithm of a model-based optimization (such as ADMM), and replace some
partial sub-procedures corresponding to the regularization (named "denoiser" step) into convolu-
tional neural networks, while maintaining the other procedures in the optimization algorithms.
This approach can substantially improve the interpretability by constraining the role of the neural
networks to a specic sub-task of the well-established optimization procedures. Finally, there are
also large number of studies in "unrolled neural networks" [50, 52] where they embed the itera-
tive algorithms of model-based methods into articial neural networks, by maintaining the data
consistency steps and replacing the regularization steps with several convolutional layers, which is
intended to learn variability in regularization among the data-consistent solutions. These unrolled
networks are trained end-to-end manner but they can be more reliable over other conventional end-
to-end deep learning methods due to the existence of the data consistency layers and the structural
similarity with model-based optimization methods. Our LORAKI work can be interpreted as a
hybrid of the rst and the last approaches. Combining those or developing novel machine learning
techniques for reliable MRI reconstruction would be an interesting future research topic to explore.
We are living in an era of the articial intelligence (AI) revolution, and medical imaging is no
exception. Currently in hospitals, taking MRI scans, processing data, and diagnosing diseases are
taken as separate steps where each procedure requires sophisticated expert human intervention.
However, in the long run, I believe those entire processes will be combined together and completely
automated by the power of AI. It would raise the eciency and help facilitate patient access
remarkably. Collaborative research eorts would be necessary among research communities, and I
would also like to contribute to the AI revolution in medical imaging in the future.
136
Bibliography
[1] T. H. Kim and J. P. Haldar. The Fourier radial error spectrum plot: A more nuanced
quantitative evaluation of image reconstruction quality. In Proc. IEEE Int. Symp. Biomed.
Imag., pages 61{64, 2018.
[2] M. Bydder and M. D. Robson. Partial Fourier partially parallel imaging. Magn. Reson. Med.,
53:1393{1401, 2005.
[3] J. P. Haldar. Low-rank modeling of local k-space neighborhoods (LORAKS) for constrained
MRI. IEEE Trans. Med. Imag., 33:668{681, 2014.
[4] J. P. Haldar and K. Setsompop. Linear predictability in MRI reconstruction: Leveraging
shift-invariant Fourier structure for faster and better imaging. IEEE Signal Process. Mag.,
37:69{82, 2020.
[5] J. P. Haldar. Low-rank modeling of local k-space neighborhoods: from phase and support
constraints to structured sparsity. In Wavelets and Sparsity XVI, Proc. SPIE 9597, page
959710, 2015.
[6] J. P. Haldar. Autocalibrated LORAKS for fast constrained MRI reconstruction. In Proc.
IEEE Int. Symp. Biomed. Imag., pages 910{913, 2015.
[7] J. P. Haldar and J. Zhuo. P-LORAKS: Low-rank modeling of local k-space neighborhoods
with parallel imaging data. Magn. Reson. Med., 75:1499{1514, 2016.
[8] J. P. Haldar and T. H. Kim. Computational imaging with LORAKS: Reconstructing linearly
predictable signals using low-rank matrix regularization. In Proc. Asilomar Conf. Sig. Sys.
Comp., pages 1870{1874, 2017.
[9] T. H. Kim and J. P. Haldar. SMS-LORAKS: Calibrationless simultaneous multislice MRI
using low-rank matrix modeling. In Proc. IEEE Int. Symp. Biomed. Imag., pages 323{326,
2015.
[10] T. H. Kim, K. Setsompop, and J. P. Haldar. LORAKS makes better SENSE: Phase-
constrained partial Fourier SENSE reconstruction without phase calibration. Magn. Reson.
Med., 77:1021{1035, 2017.
[11] T. H. Kim, B. Bilgic, D. Polak, K. Setsompop, and J. P. Haldar. Wave-LORAKS: Combining
wave encoding with structured low-rank matrix modeling for more highly accelerated 3D
imaging. 2018. Submitted.
137
[12] R. A. Lobos, T. H. Kim, W. S. Hoge, and J. P. Haldar. Navigator-free EPI ghost correction
with structured low-rank matrix models: New theory and methods. IEEE Trans. Med. Imag.,
37:2390{2402, 2018.
[13] T. H. Kim, P. Garg, and J. P. Haldar. LORAKI: Reconstruction of undersampled k-space
data using scan-specic autocalibrated recurrent neural networks. In Proc. Int. Soc. Magn.
Reson. Med., page 4647, 2019.
[14] T. H. Kim, P. Garg, and J. P. Haldar. LORAKI: Autocalibrated recurrent neural networks
for autoregressive reconstruction in k-space. Preprint, 2019. arXiv:1904.09390.
[15] J. D. Willig-Onwuachi, E. N. Yeh, A. K. Grant, M. A. Ohliger, C. A. McKenzie, and D. K.
Sodickson. Phase-constrained parallel MR image reconstruction. J. Magn. Reson., 176:187{
198, 2005.
[16] W. S. Hoge, M. E. Kilmer, C. Zacarias-Almarcha, and D. H. Brooks. Fast regularized recon-
struction of non-uniformly subsampled partial-Fourier parallel MRI data. In Proc. IEEE Int.
Symp. Biomed. Imag., pages 1012{1015, 2007.
[17] C. Lew, A. R. Pineda, D. Clayton, D. Spielman, F. Chan, and R. Bammer. SENSE phase-
constrained magnitude reconstruction with iterative phase renement. Magn. Reson. Med.,
58:910{921, 2007.
[18] M. Blaimer, M. Guterlet, P. Kellman, F. A. Breuer, H. Kostler, and M. A. Griswold. Virtual
coil concept for improved parallel MRI employing conjugate symmetric signals. Magn. Reson.
Med., 61:93{102, 2009.
[19] J. P. Haldar, V. J. Wedeen, M. Nezamzadeh, G. Dai, M. W. Weiner, N. Schu, and Z.-
P. Liang. Improved diusion imaging through SNR-enhancing joint reconstruction. Magn.
Reson. Med., 69:277{289, 2013.
[20] M. Blaimer, M. Heim, D. Neumann, P. M. Jakob, S. Kannengiesser, and F.A. Breuer. Com-
parison of phase-constrained parallel MRI approaches: Analogies and dierences. Magn.
Reson. Med., 75:1086{1099, 2016.
[21] K. P. Pruessmann, M. Weiger, M. B. Scheidegger, and P. Boesiger. SENSE: sensitivity
encoding for fast MRI. Magn. Reson. Med., 42:952{962, 1999.
[22] K. P. Pruessmann, M. Weiger, P. B ornert, and P. Boesiger. Advances in sensitivity encoding
with arbitrary k-space trajectories. Magn. Reson. Med., 46:638{651, 2001.
[23] M. A. Griswold, P. M. Jakob, R. M. Heidemann, M. Nittka, V. Jellus, J. Wang, B. Kiefer,
and A. Haase. Generalized autocalibrating partially parallel acquisitions (GRAPPA). Magn.
Reson. Med., 47:1202{1210, 2002.
[24] D. K. Sodickson and W. J. Manning. Simultaneous acquisition of spatial harmonics (SMASH):
fast imaging with radiofrequency coil arrays. Magn. Reson. Med., 38:591{603, 1997.
[25] M. A. Griswold, P. M. Jakob, M. Nittka, J. W. Goldfarb, and A. Haase. Partially parallel
imaging with localized sensitivities (PILS). Magn. Reson. Med., 44:602{609, 2000.
138
[26] M. Uecker, P. Lai, M. J. Murphy, P. Virtue, M. Elad, J. M. Pauly, S. S. Vasanawala, and
M. Lustig. ESPIRiT{an eigenvalue approach to autocalibrating parallel MRI: Where SENSE
meets GRAPPA. Magn. Reson. Med., 71:990{1001, 2014.
[27] Z.-P. Liang, R. Bammer, J. Ji, N. J. Pelc, and G. H. Glover. Making better SENSE: Wavelet
denoising, Tikhonov regularization, and total least squares. In Proc. Int. Soc. Magn. Reson.
Med., page 2388, 2002.
[28] F. H. Lin, K. K. Kwong, J. W. Belliveau, and L. L. Wald. Parallel imaging reconstruction
using automatic regularization. Magn. Reson. Med., 51:559{567, 2004.
[29] P. Margosian, F. Schmitt, and D. Purdy. Faster MR imaging: imaging with half the data.
Health Care Instrum., 1:195{197, 1986.
[30] D. C. Noll, D. G. Nishimura, and A. Macovski. Homodyne detection in magnetic resonance
imaging. IEEE Trans. Med. Imag., 10:154{163, 1991.
[31] F. Huang, W. Lin, and Y. Li. Partial Fourier reconstruction through data tting and convo-
lution in k-space. Magn. Reson. Med., 62:1261{1269, 2009.
[32] Z. P. Liang, F. Boada, T. Constable, E. M. Haacke, P. C. Lauterbur, and M. R. Smith.
Constrained reconstruction methods in MR imaging. Magn. Reson. Med., 4:67{185, 1992.
[33] A. Samsonov, E. G. Kholmovski, D. L. Parker, and C. R. Johnson. POCSENSE: POCS-based
reconstruction for sensitivity encoded magnetic resonance imaging. Magn. Reson. Med., 52:
1397{1406, 2004.
[34] H.-C. Chang, S. Guhaniyogi, and N.-K. Chen. Interleaved diusion-weighted EPI improved
by adaptive partial-Fourier and multiband multiplexed sensitivity-encoding reconstruction.
Magn. Reson. Med., 73:1872{1884, 2015.
[35] M. Uecker and M. Lustig. Estimating absolute-phase maps using ESPIRiT and virtual con-
jugate coils. Magn. Reson. Med., 77:1201{1207, 2017.
[36] E. J. Candes, J. Romberg, and T. Tao. Robust uncertainty principles: exact signal reconstruc-
tion from highly incomplete frequency information. IEEE Trans. Inf. Theory, 52:489{509,
2006.
[37] D. L. Donoho. Compressed sensing. IEEE Trans. Inf. Theory, 52:1289{1306, 2006.
[38] M. Lustig, D. Donoho, and J. M. Pauly. Sparse MRI: The application of compressed sensing
for rapid MR imaging. Magn. Reson. Med., 58:1182{1195, 2007.
[39] M. Lustig, D. Donoho, J. M. Santos, and J. M. Pauly. Compressed sensing MRI. IEEE Signal
Process. Mag., 25:72{82, 2008.
[40] E. J. Candes and T. Tao. Decoding by linear programming. IEEE Trans. Inf. Theory, 51:
4203{4215, 2005.
[41] E. J. Candes, J. Romberg, and T. Tao. Stable signal recovery from incomplete and inaccurate
measurements. Comm. Pure Appl. Math., 59:1207{1223, 2006.
139
[42] D. L. Donoho and M. Elad. Optimally sparse representation in general (nonorthogonal)
dictionaries via l1 minimization. Proc. Natl. Acad. Sci. USA, 100:2197{2202, 2003.
[43] D. L. Donoho and M. Elad. Stable recovery of sparse overcomplete representations in the
presence of noise. IEEE Trans. Inf. Theory, 52:6{18, 2006.
[44] E. J. Candes and J. Romberg. Sparsity and incoherence in compressive sampling. Inverse
Problems, 23:969{985, 2007.
[45] A. Cohen W. Dahmen and R. DeVore. Compressed sensing and best k-term approximation.
J. Am. Math. Soc., 22:211{231, 2009.
[46] D. L. Donoho and X. Huo. Uncertainty principles and ideal atomic decomposition. IEEE
Trans. Inf. Theory, 47:2845{2862, 2001.
[47] E. J. Candes and C. Fernandez-Granda. Super-resolution from noisy data. Journal of Fourier
Analysis and Applications, 19:1229{1254, 2013.
[48] E. J. Candes and C. Fernandez-Granda. Towards a mathematical theory of super-resolution.
Comm. Pure Appl. Math., 67:906{956, 2013.
[49] S. Wang, Z. Su, L. Ying, X. Peng, S. Zhu, F. Liang, D. Feng, and D. Liang. Accelerating
magnetic resonance imaging via deep learning. In Proc. IEEE Int. Symp. Biomed. Imag.,
pages 514{517, 2016.
[50] K. Hammernik, T. Klatzer, E. Kobler, M. P. Recht, D. K. Sodickson, T. Pock, and F. Knoll.
Learning a variational network for reconstruction of accelerated MRI data. Magn. Reson.
Med., 79:3055{3071, 2018.
[51] J. Schlemper, J. Caballero, J. V. Hajnal, A. N. Price, and D. Rueckert. A deep cascade
of convolutional neural networks for dynamic MR image reconstruction. IEEE Trans. Med.
Imag., 37:491{503, 2018.
[52] H. K. Aggarwal, M. P. Mani, and M. Jacob. MoDL: Model-based deep learning architecture
for inverse problems. IEEE Trans. Med. Imag., 38:394{405, 2019.
[53] J. Y. Cheng, M. Mardani, M. T. Alley, J. M. Pauly, and S. S. Vasanawala. DeepSPIRiT:
Generalized parallel imaging using deep convolutional neural networks. In Proc. Int. Soc.
Magn. Reson. Med., page 570, 2018.
[54] Y. Han, L. Sunwoo, and J. C. Ye. k-space deep learning for accelerated MRI. IEEE Trans.
Med. Imag., 39:377{386, 2020.
[55] M. Akcakaya, S. Moeller, S. Weingartner, and K. Ugurbil. Scan-specic robust articial-
neural-networks for k-space interpolation (RAKI) reconstruction: Database-free deep learning
for fast imaging. Magn. Reson. Med., 81:439{453, 2019.
[56] K. H. Jin, M. T. McCann, E. Froustey, and M. Unser. Deep convolutional neural network
for inverse problems in imaging. IEEE Trans. Image Process., 26:4509{4522, 2017.
140
[57] S. V. Venkatakrishnan, C. A. Bouman, and B. Wohlberg. Plug-and-play priors for model
based reconstruction. In Proc. IEEE Global Conf. Signal Information Processing, pages 945{
948, 2013.
[58] R. Ahmad, C. A. Bouman, G. T. Buzzard, S. Chan, S. Liu, E. T. Reehorst, and P. Schniter.
Plug-and-play methods for magnetic resonance imaging: Using denoisers for image recovery.
IEEE Signal Process. Mag., 37:105{116, 2020.
[59] M. Lustig and J. M. Pauly. SPIRiT: Iterative self-consistent parallel imaging reconstruction
from arbitrary k-space. Magn. Reson. Med., 64:457{471, 2010.
[60] J. Zhang, C. Liu, and M. E. Moseley. Parallel reconstruction using null operations. Magn.
Reson. Med., 66:1241{1253, 2011.
[61] P. J. Shin, P. E. Z. Larson, M. A. Ohliger, M. Elad, J. M. Pauly, D. B. Vigneron, and
M. Lustig. Calibrationless parallel imaging reconstruction based on structured low-rank ma-
trix completion. Magn. Reson. Med., 72:959{970, 2014.
[62] Z. P. Liang, E. M. Haacke, and C. W. Thomas. High-resolution inversion of nite Fourier
transform data through a localised polynomial approximation. Inverse Problems, 5:831{847,
1989.
[63] K. H. Jin, D. Lee, and J. C. Ye. A general framework for compressed sensing and parallel
MRI using annihilating lter based low-rank hankel matrix. IEEE Trans. Comput. Imag., 2:
480{495, 2016.
[64] G. Ongie and M. Jacob. O-the-grid recovery of piecewise constant images from few Fourier
samples. SIAM J. Imag. Sci., 9:1004{1041, 2016.
[65] Y. Chang, D. Liang, and L. Ying. Nonlinear GRAPPA: A kernel approach to parallel MRI
reconstruction. Magn. Reson. Med., 68:730{740, 2011.
[66] J. Lyu, U. Nakarmi, D. Liang, J. Sheng, and L. Ying. KerNL: Kernel-based nonlinear ap-
proach to parallel MRI reconstruction. IEEE Trans. Med. Imag., 38:312{321, 2019.
[67] T. H. Kim, B. Bilgic, D. Polak, K. Setsompop, and J. P. Haldar. Wave-LORAKS: Combining
wave encoding with structured low-rank matrix modeling for more highly accelerated 3d
imaging. Magn. Reson. Med., 81:1620{1633, 2019.
[68] M. R. Smith, S. T. Nichols, R. M. Henkelman, and M. L. Wood. Application of autoregressive
moving average parametric modeling in magnetic resonance image reconstruction. IEEE
Trans. Med. Imag., 5:132{139, 1986.
[69] B. Recht, M. Fazel, and P. A. Parrilo. Guaranteed minimum-rank solutions of linear matrix
equations via nuclear norm minimization. SIAM Rev., 52:471{501, 2010.
[70] J. P. Haldar. Low-rank modeling of local k-space neighborhoods (LORAKS): Implementa-
tion and examples for reproducible research. Technical Report USC-SIPI-414, University of
Southern California, Los Angeles, CA, April 2014.
141
[71] G. Golub and C. van Loan. Matrix Computations. The Johns Hopkins University Press,
London, third edition, 1996.
[72] J. P. Haldar and D. Hernando. Rank-constrained solutions to linear matrix equations using
powerfactorization. IEEE Signal Process. Lett., 16:584{587, 2009.
[73] J. P. Haldar and Z.-P. Liang. Spatiotemporal imaging with partially separable functions: A
matrix recovery approach. In Proc. IEEE Int. Symp. Biomed. Imag., pages 716{719, 2010.
[74] S. P. Souza, J. Szumowski, C. L. Dumoulin, D. P. Plewes, and G. Glover. SIMA: Simultaneous
multislice acquisition of MR images by Hadamard-encoded excitation. J. Comput. Assist.
Tomogr., 12:1026{1030, 1988.
[75] K. Zhu, A Kerr, and J. M. Pauly. Autocalibrating CAIPIRINHA: Reformulating CAIPIR-
INHA as a 3D problem. In Proc. Int. Soc. Magn. Reson. Med., page 518, 2012.
[76] J. P. Haldar, D. Hernando, and Z. Liang. Compressed-sensing MRI with random encoding.
IEEE Trans. Med. Imag., 31:893{903, 2011.
[77] F. A. Breuer, M. Blaimer, R. M. Heidemann, M. F. Mueller, M. A. Griswold, and P. M.
Jakob. Controlled aliasing in parallel imaging results in higher acceleration (CAIPIRINHA)
for multi-slice imaging. Magn. Reson. Med., 53:684{691, 2005.
[78] K. Setsompop, B. A. Gagoski, J. R. Polimeni, T. Witzel, V. J. Wedeen, and L. L. Wald.
Blipped-controlled aliasing in parallel imaging for simultaneous multislice echo planar imaging
with reduced g-factor penalty. Magn. Reson. Med., 67:1210{1224, 2012.
[79] K. Zhu, R. F. Dougherty, J. M. Pauly, and A. B. Kerr. Multislice acquisition with incoherent
aliasing (MICA). In Proc. Int. Soc. Magn. Reson. Med., page 4403, 2014.
[80] C. Chen, Y. Li, and J. Huang. Calibrationless parallel MRI with joint total variation regu-
larization. In Proc. MICCAI, pages 106{114, 2013.
[81] K. T. Block, M. Uecker, and J. Frahm. Undersampled radial MRI with multiple coils. iterative
image reconstruction using a total variation constraint. Magn. Reson. Med., 57:1086{1098,
2007.
[82] D. Liang, B. Liu, J. J. Wang, and L. Ying. Accelerating SENSE using compressed sensing.
Magn. Reson. Med., 62:1574{1584, 2009.
[83] F. A. Breuer, M. Blaimer, M. F. Mueller, N. Seiberlich, R. M. Heidemann, M. A. Griswold,
and P. M. Jakob. Controlled aliasing in volumetric parallel imaging (2D CAIPIRINHA).
Magn. Reson. Med., 55:549{556, 2006.
[84] K. S. Nayak and D. G. Nishimura. Randomized trajectories for reduced aliasing artifact. In
Proc. Int. Soc. Magn. Reson. Med., page 670, 1998.
[85] S. Ramani, Z. Liu, J. Rosen, J.-F. Nielsen, and J. A. Fessler. Regularization parameter
selection for nonlinear iterative image restoration and MRI reconstruction using GCV and
SURE-based methods. IEEE Trans. Image Process., 21:3659{3672, 2012.
142
[86] B. P. Sutton, D. C. Noll, and J. A. Fessler. Fast, iterative image reconstruction for MRI in
the presence of eld inhomogeneities. IEEE Trans. Med. Imag., 22:178{188, 2003.
[87] D. J. Larkman, J. V. Hajnal, A. H. Herlihy, G. A. Coutts, I. R. Young, and G. Ehnholm.
Use of multicoil arrays for separation of signal from multiple slices simultaneously excited. J.
Magn. Reson. Imag., 13:313{317, 2001.
[88] S. Moeller, E. Yacoub, C. A. Olman, E. Auerbach, J. Strupp, N. Harel, and K. Ugurbil. Multi-
band multislice GE-EPI at 7 Tesla, with 16-fold acceleration using partial parallel imaging
with application to high spatial and temporal whole-brain fMRI. Magn. Reson. Med., 63:
1144{1153, 2010.
[89] D. A. Feinberg, S. Moeller, S. M. Smith, E. Auerbach, S. Ramanna, M. F. Glasser, K. L.
Miller, K. Ugurbil, and E. Yacoub. Multiplexed echo planar imaging for sub-second whole
brain fMRI and fast diusion imaging. PLoS One, 5:1{11, 2010.
[90] J. H. Kim, S.-K. Song, and J. P. Haldar. Signal-to-noise ratio-enhancing joint reconstruction
for improved diusion imaging of mouse spinal cord white matter injury. Magn. Reson. Med.,
2015. doi: 10.1002/mrm.25691.
[91] F. Huang, S. Vijayakumar, Y. Li, S. Hertel, and G. R. Duensing. A software channel com-
pression technique for faster reconstruction with many channels. Magn. Reson. Imag., 26:
133{141, 2008.
[92] J. S. Hyde, B. B. Biswal, and A. Jesmanowicz. High-resolution fMRI using multislice partial
k-space GR-EPI with cubic voxels. Magn. Reson. Med., 46:114{125, 2001.
[93] B. Bilgic, B. A. Gagoski, S. F. Cauley, A. P. Fan, J. R. Polimeni, P. E. Grant, L. L. Wald,
and K. Setsompop. Wave-CAIPI for highly accelerated 3D imaging. Magn. Reson. Med., 73:
2152{2162, 2015.
[94] B. A. Gagoski, B. Bilgic, C. Eichner, H. Bhat, P. E. Grant, L. L. Wald, and K. Setsompop.
RARE/turbo spin echo imaging with simultaneous multislice Wave-CAIPI. Magn. Reson.
Med., 73:929{938, 2015.
[95] S. F. Cauley, K. Setsompop, B. Bilgic, H. Bhat, B. Gagoski, and L. L. Wald. Autocalibrated
wave-CAIPI reconstructions; joint optimization of k-space trajectory and parallel imaging
reconstruction. Magn. Reson. Med., 78:1093{1099, 2017.
[96] D. Polak, K. Setsompop, S. F. Cauley, B. A. Gagoski, H. Bhat, F. Maier, P. Bachert, L. L.
Wald, and B. Bilgic. Wave-CAIPI for highly accelerated MP-RAGE imaging. Magn. Reson.
Med., 79:401{406, 2018.
[97] H. Moriguchi and J. L. Duerk. Bunched phase encoding (BPE): A new fast data acquisition
method in MRI. Magn. Reson. Med., 55:633{648, 2006.
[98] J. P. Mugler and J. R. Brookeman. Three-dimensional magnetization-prepared rapid gradient-
echo imaging (3D MP RAGE). Magn. Reson. Med., 15:152{157, 1990.
[99] A. T. Curtis, B. Bilgic, K. Setsompop, R. S. Menon, and C. K. Anand. Wave-CS: Combining
wave encoding and compressed sensing. In Proc. Int. Soc. Magn. Reson. Med., page 82, 2015.
143
[100] B. Bilgic, H. Ye, L. L. Wald, and K. Setsompop. Optimized CS-Wave imaging with tailored
sampling and ecient reconstruction. In Proc. Int. Soc. Magn. Reson. Med., page 612, 2016.
[101] M. R. Hestenes and E. Stiefel. Methods of conjugate gradients for solving linear systems. J.
Res. Natl. Bur. Stand., 49:409{436, 1952.
[102] G. Ongie and M. Jacob. A fast algorithm for convolutional structured low-rank matrix
recovery. IEEE Trans. Comput. Imag., 3:535{550, 2017.
[103] S. Ravishankar and Y. Bresler. MR image reconstruction from highly undersampled k-space
data by dictionary learning. IEEE Trans. Med. Imag., 30:1028{1041, 2011.
[104] D. W. Shattuck and R. M. Leahy. BrainSuite: An automated cortical surface identication
tool. Med. Image Anal., 8:129{142, 2002.
[105] B. Bilgic, T. H. Kim, C. Liao, M. K. Manhard, L. L. Wald, J. P. Haldar, and K. Setsompop.
Improving parallel imaging by jointly reconstructing multi-contrast data. Magn. Reson. Med.,
80:619{632, 2018.
[106] J. Hamilton, D. Franson, and N. Seiberlich. Recent advances in parallel imaging for MRI.
Prog. NMR Spect., 101:71{95, 2017.
[107] Z.-P. Liang. Spatiotemporal imaging with partially separable functions. In Proc. IEEE Int.
Symp. Biomed. Imag., pages 988{991, 2007.
[108] S. G. Lingala, Y. Hu, E. DiBella, and M. Jacob. Accelerated dynamic MRI exploiting sparsity
and low-rank structure: k-t SLR. IEEE Trans. Med. Imag., 30:1042{1054, 2011.
[109] B. Zhu, J. Z. Liu, S. F. Cauley, B. R. Rosen, and M. S. Rosen. Image reconstruction by
domain-transform manifold learning. Nature, 555:487{492, 2018.
[110] Y. Han, J. Yoo, H. H. Kim, H. J. Shin, K. Sung, and J. C. Ye. Deep learning with domain
adaptation for accelerated projection-reconstruction MR. Magn. Reson. Med., 80:1189{1205,
2018.
[111] M. Mardani, E. Gong, J. Y. Cheng, S. S. Vasanawala, G. Zaharchuk, L. Xing, and J. M. Pauly.
Deep generative adversarial neural networks for compressive sensing MRI. IEEE Trans. Med.
Imag., 38:167{179, 2019.
[112] P. L. Combettes and J. C. Pesquet. Proximal splitting methods in signal processing. In
H. H. Bauschke et al., editor, Fixed-Point Algorithms for Inverse Problems in Science and
Engineering, pages 185{212. Springer Science+Business Media, LLC, 2011.
[113] C. R. Vogel. Computational methods for inverse problems. SIAM, 2002.
[114] R. A. Lobos and J. P. Haldar. Improving the performance of accelerated image reconstruction
in k-space: The importance of kernel shape. In Proc. Int. Soc. Magn. Reson. Med., 2019.
[115] Seyed Amir Hossein Hosseini, Chi Zhang, Sebastian Weingartner, Steen Moeller, Matthias
Stuber, Kamil Ugurbil, and Mehmet Akcakaya. Accelerated coronary MRI with sRAKI: A
database-free self-consistent neural network k-space reconstruction for arbitrary undersam-
pling. PLoS One, 2020. arXiv:1907.08137.
144
[116] Seyed Amir Hossein Hosseini, Chi Zhang, Kamil Ugurbil, Steen Moeller, and Mehmet Ak-
cakaya. sRAKI-RNN: accelerated MRI with scan-specic recurrent neural networks using
densely connected blocks. In Wavelets and Sparsity XVIII, Proc. SPIE 11138, page 111381B,
2019.
[117] N. Srivastava, G. Hinton, A. Krizhevsky, I. Sutskever, and R. Salakhutdinov. Dropout: A
simple way to prevent neural networks from overtting. J. Mach. Learn. Res., 15(1929{1958),
2014.
[118] S. Ioe and C. Szegedy. Batch normalization: accelerating deep network training by reducing
internal covariate shift. In Proc. Int. Conf. Mach. Learn., volume 37, pages 448{456, 2015.
[119] T. H. Kim and J. P. Haldar. Learning-based computational mri reconstruction without big
data: From structured low-rank matrices to recurrent neural networks. In Wavelets and
Sparsity XVIII, Proc. SPIE 11138, page 1113817, 2019.
[120] T. H. Kim and J. P. Haldar. Learning how to interpolate fourier data with unknown autore-
gressive structure: An ensemble-based approach. In Proc. Asilomar Conf. Sig. Sys. Comp.,
2019.
[121] S. M. Kay. Fundamentals of Statistical Signal Processing, Volume I: Estimation Theory.
Prentice Hall, Upper Saddle River, 1993.
[122] H. Erdogan and J. A. Fessler. Monotonic algorithms for transmission tomography. IEEE
Trans. Med. Imag., 18:801{814, 1999.
[123] J. A. Fessler and H. Erdogan. A paraboloidal surrogates algorithm for convergent penalized-
likelihood emission image reconstruction. pages 1132{1135, 1998.
[124] D. Varadarajan and J. P. Haldar. A majorize-minimize framework for Rician and non-central
chi MR images. IEEE Trans. Med. Imag., 34:2191{2202, 2015.
[125] D. G. Luenberger. Optimization by Vector Space Methods. Wiley-Interscience, 1969.
[126] L. Landweber. An iteration formula for Fredholm integral equations of the rst kind. Amer.
J. Math., 73:615{624, 1951.
[127] C. C. Paige and M. A. Saunders. LSQR: An algorithm for sparse linear equations and sparse
least squares. ACM Transactions on Mathematical Software, 8:43{71, 1982.
[128] M. Huhtanen and S. Ruotsalainen. Real linear operator theory and its applications. Integral
Equations and Operator Theory, 69:113{132, 2011.
[129] W. Rudin. Functional Analysis. McGraw-Hill Science, second edition, 1991.
[130] Z. Wang and A. C. Bovik. Mean squared error: Love it or leave it? a new look at signal
delity measures. IEEE Signal Process. Mag., 26:98{117, 2009.
[131] Z. Wang, A.C. Bovik, H.R. Sheikh, and E.P. Simoncelli. Image quality assessment: from
error visibility to structural similarity. IEEE Trans. Image Process., 13:600{612, 2004.
145
[132] W. O. Saxton and W. Baumeister. The correlation averaging of a regularly arranged bacterial
cell envelope protein. J. Microsc., 127:127{138, 1982.
[133] M. van Heel, W. Keegstra, W. Schutter, and E. F. J. van Bruggen. The structure and function
of invertebrate respiratory proteins. In EMBO Workshop, pages 69{73, 1982.
[134] G. Harauz and M. van Heel. Exact lters for general geometry three dimensional reconstruc-
tion. Optik, 73:146{158, 1986.
[135] M. van Heel and M. Schatz. Fourier shell correlation threshold criteria. J. Struct. Biol., 151:
250{262, 2005.
[136] C.H. Reinsch. Smoothing by spline functions. Numerische Mathematik, 10:177{183, 1967.
[137] P. Craven and G. Wahba. Smoothing noisy data with spline functions. Numerische Mathe-
matik, 31:377{403, 1979.
[138] J. P. Haldar. Low-rank modeling of local k-space neighborhoods: from phase and support
constraints to structured sparsity. Proc. SPIE, 9597:959710, 2015.
[139] Justin P. Haldar. Calibrationless partial Fourier reconstruction of MR images with slowly-
varying phase: A rank-decient matrix recovery approach. In ISMRM Workshop on Data
Sampling & Image Reconstruction, Sedona, 2013.
[140] R. A. Lobos, T. H. Kim, W. S. Hoge, and J. P. Haldar. Navigator-free EPI ghost correction
with structured low-rank matrix models: New theory and methods. IEEE Trans. Med. Imag.,
2018. Early View.
[141] D. L. Donoho. An invitation to reproducible computational research. Biostat., 11:385{388,
2010.
[142] K. F. Cheung and R. J. Marks II. Imaging sampling below the Nyquist density without
aliasing. J. Opt. Soc. Am. A, 7:92{105, 1990.
[143] M. Lustig. Post-Cartesian calibrationless parallel imaging reconstruction by structured low-
rank matrix completion. In Proc. Int. Soc. Magn. Reson. Med., page 483, 2011.
[144] D. R. Hunter and K. Lange. A tutorial on MM algorithms. Am. Stat., 58:30{37, 2004.
[145] M. Nikolova and M. K. Ng. Analysis of half-quadratic minimization methods for signal and
image recovery. SIAM J. Sci. Comput., 27:937{966, 2005.
146
Appendix A
The Fourier Radial Error Spectrum
Plot: a More nuanced quantitative
evaluation of image reconstruction
quality
A.A Introduction
1
Quantitative image quality assessment is often a key part of the evaluation of image reconstruction
methods for biomedical imaging. In the presence of a (vectorized) gold-standard reference image
p

2C
N
, it is common in the MRI literature to evaluate the quality of an image estimate ^ p2C
N
using quantitative error metrics such as the normalized root mean-squared error (NRMSE) [130]
NRMSE =
kp

^ pk
`
2
kp

k
`
2
; (A.1)
the perception-inspired structural similarity index (SSIM) [131], and more specialized measures
such as the high-frequency error norm (HFEN) [103] which tries to quantify the quality of edges
and other ne image features by computing the NRMSE after high-pass ltering both p

and ^ p.
While each of these error measures can provide useful insight into dierent aspects of image
quality, there are many situations in which a single scalar-valued error metric is insucient to
1
The text and gures in this chapter have been previously published in [1], and are copyright of the IEEE.
147
(a) Gold Standard (b) Blurred
(c) Noisy (d) Ringing
Figure A.1: A gold-standard reference image (obtained from http://mr.usc.edu/download/data/)
along with three versions of this image that each present very dierent kinds of errors, yet all share
the same NRMSE value of 0.231 with respect to the gold standard.
describe the nuanced dierences between dierent methods that can often appear in image recon-
struction. A toy illustration of this is presented in Fig. A.1, which shows three images that each
have very dierent characteristics but all have the same NRMSE value.
In this work, we are inspired by the concepts of Fourier Ring Correlation (FRC) [132, 133] and
Fourier Shell Correlation (FSC) [134] to develop a higher-dimensional quantitative assessment of
error that provides deeper insight into the relative strengths and weaknesses of dierent biomedical
image reconstruction approaches. FRC and FSC are techniques that were introduced in the electron
microscopy literature to gauge the spatial resolution of an imaging system. These techniques operate
by computing the Fourier-domain statistical correlation between two reconstructions of independent
averages of the same image. These correlations are computed as a function of radius in the Fourier
148
domain (i.e., the Fourier domain is partitioned into rings about the origin in 2D or shells about
the origin in 3D, and correlation is computed separately for each ring/shell), and the results are
plotted as a function of spatial frequency radius. The spatial frequency at which these correlation
curves drops below a certain threshold can be used to dene a statistical notion of spatial resolution
[135]. However, one important limitation of FRC and FSC is that they use normalized correlation
and are therefore invariant to radially-symmetric scaling errors. For example, applying a radially-
symmetric Gaussian blur to an image can substantially change the quality of the image, but will
not change the FRC or FSC curves.
This paper proposes the Fourier radial Error Spectrum Plot (ESP) for image quality assessment.
Unlike common scalar error measures like NRMSE, SSIM, and HFEN but similar to FRC and FSC,
our new ESP approach provides an extra dimension of insight that reveals frequency-dependent
image quality variations. And unlike FRC and FSC, our new ESP approach is rooted in NRMSE
rather than normalized correlation, and is therefore sensitive to important scaling dierences that
may exist between the gold standard and the reconstructed image.
For illustration of the potential insight provided by the ESP approach, ESPs corresponding to
the three images from Fig. A.1 are shown in Fig. A.2. These ESPs reveal that even though the three
images all have the same NRMSE value, they are denitely not equivalent to one another and each
have their own strengths and weaknesses. In particular, the \ringing" image has the lowest relative
error at low-frequencies, the \noisy" image has the lowest relative error in the mid-frequency range,
and the \blurred" image has the lowest relative error at high-frequencies. This kind of insight is
potentially quite valuable when choosing between dierent image reconstruction methods.
The following sections describe the creation of the ESP and provide illustrative applications in
MRI reconstruction.
A.B Creating the Fourier Radial Error Spectrum Plot
There are several steps to computing our proposed ESP:
149
Fourier Radius (Nyquist units)
020406080 100
Relative Error
0
0.5
1
1.5
Blurred
Noisy
Ringing
Figure A.2: Proposed ESPs corresponding to the \blurred," \noisy," and \ringing" images from
Fig. A.1.
1. First, we compute Fourier representations of the error image and the gold-standard image. In
some MRI applications, images are reconstructed in the Fourier domain and no extra eort is
required. In other cases, it may be necessary to apply a discrete Fourier transform (DFT) to
the image-domain representations, e.g., k

= Fp

and k
err
= F(p

^ p), where F2C
MN
is
an appropriate DFT operator, and k

2C
M
and k
err
2C
M
are respectively Fourier-domain
representations of the gold-standard and the reconstruction error. Frequently, we will choose
Fourier sampling locations such that the number of Fourier samplesM is equal to the number
of voxels N, though other choices are also possible.
2. Next, each of the M Fourier sampling locations is associated with a corresponding Fourier
radius value, leading to sets of paired values of the form (r
m
; [k

]
m
) and (r
m
; [k
err
]
m
) for
m = 1;:::;M, where r
m
is the Fourier radius corresponding to the mth Fourier sample, and
[k

]
m
denotes the mth entry of the vector k

.
3. In general, multiple points in the Fourier domain will share the same Fourier radius, such
that we will frequently observe multiple true signal values and multiple error values for each
distinct radius value. In addition, the radius values we observe will generally be non-uniformly
150
spaced (e.g., if F samples the Fourier domain on a rectilinear Cartesian grid). Morever, the
values of [k

]
m
and [k
err
]
m
will often demonstrate noise-like variations. To overcome these
issues, we use smoothing splines [136, 137] to t smooth continuous 1D functions that t
the squared error samples (r
m
;j[k
err
]
m
j
2
) and the squared true-signal samples (r
m
;j[k

]
m
j
2
)
as closely as possible as a function of radius r. In particular, the smoothing spline for the
squared error samples is obtained by solving
^
f(r) =
arg min
f
M
X
m=1

j[k
err
]
m
j
2
f(r
m
)

2
+
Z
jf
00
(r)j
2
dr;
(A.2)
while the smoothing spline for the squared true signal samples is obtained by solving
^ y(r) =
arg min
y
M
X
m=1

j[k

]
m
j
2
y(r
m
)

2
+
Z
jy
00
(r)j
2
dr;
(A.3)
where is a regularization parameter. It has been proven that the solutions to these two
optimization problems are natural cubic splines, which enables the optimization problem to
be solved using relatively easy nite-dimensional computations.
4. Finally, the Fourier radial error spectrum is obtained as:
Error(r) =
s
^
f(r)
^ y(r)
: (A.4)
Note that this ratio mirrors the same basic form as NRMSE, taking the square-root of the
ratio between some measure of mean-squared error and some measure of the squared value
of the true signal.
151
(a) Magnitude (b) Phase (c) Sampling Mask
Figure A.3: (a) Magnitude and (b) phase images of the gold-standard T2-weighted reference image.
(c) 4.8-accelerated partial Fourier k-space sampling mask used for retrospective undersampling.
A.C Illustrative Application to MRI Reconstruction
To illustrate the new insight provided by ESP, we consider a head-to-head comparison between
two advanced MRI reconstruction methods: SENSE parallel imaging [21] combined with total
variation regularization (SENSE-TV, [81]), versus a combination of SENSE parallel imaging with
low-rank modeling of local k-space neighborhoods (LORAKS) [3, 7, 138] (SENSE-LORAKS, [10]).
These two reconstruction approaches are both applied to retrospectively undersampled 12-channel
k-space data corresponding to the T2-weighted brain image shown in Fig. A.3(a,b). This data was
retrospectively undersampled using the 4.8-accelerated partial Fourier sampling pattern shown in
Fig. A.3(c).
Reconstruction results for these two reconstruction methods are shown in Fig. A.4(a,b). Qualita-
tively, we believe that the SENSE-TV results appear somewhat blurry, while the SENSE-LORAKS
results appear sharp but noisy. The conventional error metrics, which are presented in Table A.1,
seem to suggest that SENSE-LORAKS generally outperforms SENSE-TV with respect to NRMSE
and HFEN for this dataset but that SENSE-TV has a slight advantage with respect to SSIM.
On the other hand, the ESPs shown in Fig. A.5 enable a more nuanced comparison. In partic-
ular, SENSE-TV appears to have the lowest error at both the very lowest and very highest spatial
frequencies, while SENSE-LORAKS has the lowest error across a broad range of middle frequencies.
152
(a) SENSE-TV (b) SENSE-LORAKS (c) Fusion
Figure A.4: (a) SENSE-TV and (b) SENSE-LORAKS reconstruction results from undersampled
data. (c) Fusion of SENSE-TV and SENSE-LORAKS reconstruction results.
SENSE-TV SENSE-LORAKS
NRMSE (lower is better) 0.204 0.178
SSIM (higher is better) 0.802 0.775
HFEN (lower is better) 0.330 0.216
Table A.1: Conventional Error Metrics for SENSE-TV and SENSE-LORAKS.
This observation is surprising and is not obvious from the conventional scalar error metrics, but
becomes easily apparent when looking at the ESPs. These results help to conrm the potential
new value oered by ESPs for comparing dierent image reconstruction methods.
It should also be noted that the insight provided by ESPs can be valuable for the development
of improved image reconstruction methods. For example, our further analysis of SENSE-TV versus
SENSE-LORAKS reconstruction suggests that the ESP characteristics for these two methods are
fairly consistent across dierent slices in a multi-slice acquisition (results not shown due to space
constraints). In particular, SENSE-TV was consistently better than SENSE-LORAKS at low-
frequencies and high-frequencies, while the opposite was consistently true across a broad range of
middle frequencies. This suggests that an optimized fusion of SENSE-TV and SENSE-LORAKS
(which uses SENSE-TV results at low- and high-frequencies and otherwise uses SENSE-LORAKS
results) may potentially outperform either reconstruction approach individually. One such fusion
153
Fourier Radius (Nyquist units)
0102030405060708090 100
Relative Error
0
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
1
SENSE-LORAKS
SENSE-TV
Fusion
Figure A.5: ESPs for the SENSE-TV, SENSE-LORAKS, and their fusion shown in Fig. A.4.
image is shown in Fig. A.4(c), and quantitative analysis conrms that it dominates both SENSE-TV
and SENSE-LORAKS in terms of NRMSE (0.170), SSIM (0.804), and HFEN (0.212).
This example provides a potentially powerful way to use the new information provided by ESPs,
yet only illustrates one such possibility. We expect that the research community will be able to
nd many other potential uses for this new quantitative error analysis tool.
A.D Conclusion
This work proposed a novel error analysis tool for image quality assessment. Rather than the
conventional approach in which error characteristics are summarized by a single scalar value, the
ESP represents the errors as a spectrum that varies as a function of spatial-frequency. Although
there is no universally best error metric (i.e., assessment of image quality is always subjective
and/or context dependent), we believe that the ESP enables more nuanced insights into general
image quality than are available from conventional measures. The proposed approach is expected
to be useful in many dierent image reconstruction scenarios, and as our results demonstrate, it
154
also enables new reconstruction approaches that have the potential to outperform conventional
approaches by taking optimal account of the unique strengths and weaknesses of existing image
reconstruction approaches. It should also be noted that while we only illustrated ESPs in the
context of MRI reconstruction, we believe that the concept will be equally advantageous across a
broad range of other biomedical imaging modalities.
155
This page intentionally left blank
Appendix B
LORAKS Software Version 2.0:
Faster Implementation and Enhanced
Capabilities
B.A Overview
Over the past several years, our research group has been developing a novel structured low-rank
matrix modeling framework for magnetic resonance (MR) image reconstruction that we call LO-
RAKS (LOw-RAnk modeling of local K-Space neighborhoods) [3, 5{11, 70, 105, 139, 140].
1
In the
spirit of reproducible research [141], we had previously released a public open-source software im-
plementation of LORAKS-based image reconstruction in 2014 [70]. In the present technical report
(and supplementary material available for download at http://mr.usc.edu/download/LORAKS2/),
we describe an updated public open-source software release that provides access to many of the
new developments we've made since 2014, including substantially-faster algorithms and a variety
of new formulations of the inverse problem [5{11, 105, 140].
The LORAKS framework is based on the assumption that ideal uniformly-sampled Fourier data
will often possess a multitude of distinct shift-invariant linear predictability relationships. Linear
predictability is a powerful constraint, because if a sample can be predicted accurately from its
neighbors, then there's no need to actually measure that sample. Linear predictability also arises
1
Related methods have also been explored by other groups, e.g., [60{64]. However, for the sake of brevity, we will
restrict our attention to the perspectives and terminology from our previous LORAKS work.
157
naturally as a consequence of several dierent kinds of classical and widely-used image reconstruc-
tion constraints. For example, shift-invariant linear predictability can be proven in scenarios for
which the original image has limited support [142], slowly-varying phase [3, 31], inter-channel corre-
lations in a multi-channel acquisition [23, 24], and/or sparsity in an appropriate transform domain
[62].
An important feature of linear predictability is that high-dimensional convolution-structured
matrices (e.g., Hankel and/or Toeplitz matrices) formed from linearly-predictable data will often
possess low-rank structure [62], which in turn implies that modern low-rank matrix recovery meth-
ods [69] can be used to improve the recovery of linearly-predictable signals from noisy and/or
incomplete data. When applied to MRI data, this type of structured low-rank matrix recov-
ery approach has enabled impressive performance across a range of challenging scenarios [3, 5{
11, 61, 63, 64, 70, 105, 139, 140].
Relative to traditional constrained reconstruction, a unique advantage of structured low-rank
matrix recovery approaches is that they are very
exible and adaptable to dierent situations.
Specically, classical constrained MR approaches often require (i) a substantial amount of prior
information and (ii) a reconstruction procedure that is specially adapted to enforce the existence a
specic kind of image structure [32]. In contrast, LORAKS can be used successfully even when it's
unclear whether a specic constraint will be applicable to a given dataset. Rather than assuming
that certain constraints are applicable in advance, LORAKS uses the subspace structure of the raw
data to automatically identify and enforce any and all linear prediction relationships that may exist
in the data. This approach is agnostic to the original source of the linear prediction relationship, and
there is no need for the user to know in advance what kind of constraint may be applicable to a given
image. This
exibility and generality allows LORAKS to transition seamlessly between dierent k-
space sampling patterns (i.e., both calibrationless and calibration-based sampling; random, variable
density, partial Fourier, and highly-unconventional sampling patterns), imaging geometries (i.e.,
both single-channel and multi-channel), and application contexts.
158
B.B Mathematical Description of Reconstruction Approaches Provided
by the Software
B.B.1 Problem Formulations
B.B.1.1 Original Single-Channel Formulation
Our original description and software implementation of LORAKS [3, 70] gave users the ability to
solve regularized single-channel MRI reconstruction problems using up to three dierent LORAKS
matrix constructions according to:
^
f = arg min
f
kAf dk
2
`
2
+
C
J
r
C
(P
C
(f)) +
G
J
r
G
(P
G
(f)) +
S
J
r
S
(P
S
(f)): (B.1)
In this expression, f is a vector of uniformly-sampled Cartesian k-space data to be estimated, d is the
vector of measured data, and the matrix A is a sampling operator that models the conversion from
full k-space data to subsampled k-space data. In addition,P
C
(),P
G
(), andP
S
() are operators
that construct high-dimensional structured LORAKS matrices (respectively called the C, G, and S
matrices);
C
,
G
, and
S
are user-selected regularization parameters used to adjust the strength
of the regularization penalty applied to each matrix;r
C
,r
G
, andr
S
are user-selected rank estimates
for the C, G, and S matrices, respectively; andJ
r
() is a nonconvex regularization penalty function
that encourages its matrix argument to have rank less than or equal tor. Specically, this function
is equal to the Frobenius norm of the residual obtained after an optimal rank-r approximation of
its matrix argument. In particular, assuming that X2C
PQ
, we have that
J
r
(X) =
X
k>r

2
k
(B.2)
= min
T2C
PQ
kX Tk
2
F
s.t. rank(T)r (B.3)
= min
V2C
Q(Qr)
kXVk
2
F
s.t. V
H
V = I
(Qr)
; (B.4)
where
k
is the kth singular value of the matrix X, and I
(Qr)
is the (Qr) (Qr) identity
matrix.
159
Theoretically, the C matrix is expected to have low-rank whenever the image obeys a sup-
port constraint, the G matrix is expected to have low-rank whenever the image obeys a support
constraint and possesses smoothly-varying phase, and the S matrix is expected to have low-rank
whenever the image obeys a support constraint and/or possesses smoothly-varying phase. See
Ref. [3] for further details. Additional LORAKS matrix constructors, which form a structured
LORAKS matrix after applying a linear transformation to f were introduced in [5], and will have
low-rank whenever the image is sparse in an appropriate transform domain.
The present software implementation still provides the ability to solve Eq. (B.1), but additionally
includes a matrix constructorP
W
that generates what we call the W-matrix [8]. The W matrix
will have low rank if the spatial derivatives of the original image are sparse [5, 8, 62{64].
Note that the present software implementation no longer supports the G matrix, since previous
literature has consistently demonstrated (e.g., [3]) that this LORAKS matrix is less powerful than
the alternatives. In addition, we only provide support to use one matrix at a time rather than
using multiple matrices simultaneously { it is possible to get small performance improvements
by using multiple matrices simultaneously, although this comes at the expense of computational
complexity and the need to tune a larger number of reconstruction parameters. Specically, for the
single-channel case, the present software provides capabilities to solve
^
f = arg min
f
kAf dk
2
`
2
+J
r
(P
X
(f)); (B.5)
with X2fC; S; Wg:
Our software assumes a certain kind of Cartesian sampling by default (meaning that A is formed
by taking a subset of rows from the identity matrix, which implies that the samples observed in d
are obtained at a subset of the same grid of k-space locations to be reconstructed in f) , similar to
our previous software release [70]. This assumption implies that A
H
A has simple structure (i.e.,
it is a diagonal matrix) that enables substantial computational simplications, as will be described
later. Since our code is open-source, it would be straightforward to modify the code to use non-
Cartesian/o-grid sampling if so desired. This would amount to modeling A as a gridding operator
that interpolates from the original Cartesian grid onto the o-grid sample locations [143], and then
160
solving the resulting optimization steps (to be described in the sequel) directly without assuming
that A
H
A is diagonal.
In our current implementation, we also now support the use of virtual conjugate coils [18, 35],
which enables the use of smoothly varying phase constraints with the C matrix [8, 11].
This software is currently implemented for 2D data reconstruction. However, extension to 3D
reconstruction is straightforward [11].
B.B.1.2 Enforcing Exact Data Consistency
A potential disadvantage of Eq. (B.5) is the need for the user to choose the regularization parameter
. The regularization parameter represents a trade-o between reliance on the measured data and
reliance on the constraint, and selection of regularization parameters is an omnipresent problem
whenever dealing with regularized reconstruction problems.
However, in many circumstances with high signal-to-noise ratio, a user may be interested in
choosing such that the reconstruction is as consistent with the measured data as possible, so that
the LORAKS constraints are just used to interpolate missing data without modifying the measured
data samples. In the context of Eq. (B.5), this corresponds to choosing very small values of [3].
However, due to nite precision numerical eects, it is not possible to choose values that are
arbitrarily small. A way of bypassing this issue is to instead solve the following data-consistency-
constrained optimization problem [6], which no longer involves a regularization parameter:
^
f = arg min
f
J
r
(P
X
(f)) s.t. Af = d: (B.6)
The constraint in this problem is easy to enforce when the k-space sampling locations used for d are
a subset of the k-space sampling locations used in f (i.e., the on-grid Cartesian sampling scenario
described in Sec. B.B.1.1). In particular, we can enforce the constraint in this case by only solving
for the entries from f that are not present in the measured data d, and otherwise directly setting
the entries of f equal to the measured data. This can be achieved by solving [6]
^
f =A(d) + M^ z; (B.7)
161
with
^ z = arg min
z
J
r
(P
X
(Mz) (P
X
(A(d)))); (B.8)
whereA(d) is the zero-lled vector that is the same size as f and contains the samples from d in
the appropriate k-space locations, z is the vector of unmeasured data samples, and the matrix M
places the unmeasured data samples in their appropriate locations while zero-lling the locations
corresponding to acquired data samples.
B.B.1.3 Multi-Channel Formulations: P-LORAKS and SENSE-LORAKS
Beyond the single-channel case, subsequent work has developed multi-channel parallel imaging ver-
sions of the C, G, and S matrices, which are obtained by concatenating together the corresponding
LORAKS matrices constructed from dierent channels [7, 10]. These concatenated matrices will
have low-rank in the same scenarios as for the single-channel case (i.e., in the presence of sup-
port, phase constraints, and/or sparsity constraints), but will have additional low-rank structure
due to the correlations that will exist between channels in a multi-channel imaging experiment
[7, 60, 61, 63].
One of these multi-channel LORAKS approaches, called P-LORAKS [7], simply solves Eq. (B.5)
using this generalized concatenation-based denition of the LORAKS matrices. Due to the fact
that the P-LORAKS matrices reduce to the single-channel LORAKS matrices in the special case
of single-channel data, we do not distinguish notation between the single-channel LORAKS and
the P-LORAKS formulations of this problem, and use Eq. (B.5) to represent both variations.
Note that using Eq. (B.5) with the multichannel C matrix is very similar to SAKE [61] { in this
case, there are only minor dierences between SAKE and P-LORAKS related to the shape of the
LORAKS neighborhood system and the choice of cost functional. Note that it is also possible
to use the data-consistency constraint as described in Sec. B.B.1.2 with both single-channel and
multi-channel data.
162
The other approach, called SENSE-LORAKS [10], combines P-LORAKS with SENSE [21, 22].
In this case, instead of solving for the fully-sampled k-space data of all the coils f, we instead use
prior knowledge of the coil sensitivity proles solve for a single image vector according to
^ = arg min

kE dk
2
`
2
+J
r
(P
X
(F)); (B.9)
with X2fC; S; Wg; where the matrix E is the SENSE model for how the measured data is related
to the desired image (which, for each channel, includes weighting of the image by the sensitivity
prole of the coil, followed by Fourier transformation and sampling at the k-space locations corre-
sponding to those in d [22]), and the matrix F converts the image into fully-sampled k-space data
for each coil (which, for each channel, includes weighting of the image by the sensitivity prole of
the coil, followed by Fourier transformation and sampling at the k-space locations corresponding
to those in f) as needed for constructing the multi-channel LORAKS matrices.
B.B.1.4 Autocalibrated LORAKS
The problem formulations listed above are all nonconvex and can be relatively computionally-
demanding to solve. The Autocalibrated LORAKS (AC-LORAKS) approach [6] recognized that
substantial improvements in computational eciency may be possible by using the approximation
J
r
(X)kX
^
Vk
2
F
(B.10)
for an appropriate choice of the matrix
^
V. This approximation is based on the representation of
J
r
() shown in Eq. (B.4), and can be used to convert all of the previously mentioned nonconvex
formulations described by Eqs. (B.5), (B.6), and (B.9) into simple convex linear least-squares
problems that can be solved quickly and eciently.
In particular, the solution to the AC-LORAKS version of Eq. (B.5) is given by
^
f = arg min
f
kAf dk
2
`
2
+kP
X
(f)
^
Vk
2
F
=M
1
1
A
H
d;
(B.11)
163
where
M
1
(f), A
H
Af +P

X
(P
X
(f)
^
V
^
V
H
) (B.12)
and

is used to denote the adjoint; the solution to the AC-LORAKS version of Eq. (B.6) is given
by
^
f =A(d) + M arg min
z
kP
X
(Mz)
^
V (P
X
(A(d))
^
V)k
2
F
=A(d) MM
1
2
M
H
P

X
(P
X
(A(d))
^
V
^
V
H
);
(B.13)
where
M
2
(z), M
H
P

X
(P
X
(Mz)
^
V
^
V
H
); (B.14)
and the solution to the AC-LORAKS version of Eq. (B.9) is given by
^ = arg min

kE dk
2
`
2
+kP
X
(F)
^
Vk
2
F
=M
1
3
E
H
d;
(B.15)
where
M
3
(), E
H
E +F
H
P

X
(P
X
(F)
^
V
^
V
H
): (B.16)
All three of these solutions can be obtained easily using standard iterative linear least-squares solvers
like the conjugate gradient method or LSQR [101, 127]. As will be discussed in Sec. B.B.2.3, some
of the LORAKS-related computations in these expressions can be substantially accelerated using
FFTs.
The AC-LORAKS approach can be viewed as a generalization of PRUNO [60], and is especially
similar to PRUNO when using the multichannel C matrix, in which case there are only minor
dierences between PRUNO and AC-LORAKS related to the shape of the LORAKS neighborhood.
Like PRUNO, we estimate the matrix
^
V using autocalibration (ACS) data. Specically, if a fully-
sampled region of k-space is available (commonly known in the MR literature as ACS data), then
a LORAKS matrix formed from zero-lled data will contain a submatrix with fully-populated
164
rows. Since the nullspace of the full LORAKS matrix should be included in the nullspace of this
submatrix, we choose
^
V to be a basis for the nullspace of the submatrix.
B.B.2 Algorithm Choices
B.B.2.1 Original Additive Half-Quadratic Majorize-Minimize Approach
Our previous algorithm implementation [70] was based on applying a majorize-minimize (MM)
algorithm [144] to Eq. (B.1), as originally described in Ref. [3]. Specically, it is easy to use the
representation ofJ
r
() from Eq. (B.3) to show that, for a generic matrix X constructed by operator
P
X
(), the following function
g
X
(f;
^
f
(i1)
),kP
X
(f)L
r
(P
X
(
^
f
(i1)
))k
2
F
(B.17)
is a majorant of the functionJ
r
(P
X
(f)) at the point
^
f
(i1)
, where the operatorL
r
() computes the
optimal rank-r approximation of its matrix argument (e.g., which is easily done using the singular
value decomposition). We call this an additive half-quadratic majorizer because the majorant has
structural resemblence to previous additive half-quadratic methods [145].
This majorization relationship means that we can monotonically decrease the cost function
value from Eq. (B.5) using an additive half-quadratic MM approach in which we iteratively solve
the following very simple linear least-squares problem from some initialization
^
f
(0)
:
^
f
(i)
= arg min
f
kAf dk
2
`
2
+g
X
(f;
^
f
(i1)
)
=M
1
4

A
H
d +P

X
(L
r
(P
X
(
^
f
(i1)
)))

;
(B.18)
where
M
4
(f), A
H
Af +P

X
(P
X
(f)): (B.19)
In the case of the right kind of Cartesian data (as dened in Sec. B.B.1.1, and as assumed by our
software), this specic problem can even be solved analytically because the operatorM
4
() can
be represented as a diagonal matrix, due to the special structure of theP
X
() operators and the
165
special structure of the A matrix. For users interested in non-Cartesian/o-grid k-space sampling,
this would be possible by nding the line of code in our software that analytically computes
^
f
(i)
based on diagonal matrix structure, and replacing it with an algorithm for solving Eq. (B.18) that
uses standard iterative linear least-squares solvers like the conjugate gradient method or LSQR
[101, 127].
Similar additive half-quadratic MM algorithms are possible for the other formulations that
involve J
r
(), i.e., LORAKS/P-LORAKS with exact data consistency constraints as dened in
Eq. (B.6), and SENSE-LORAKS as dened in Eq. (B.9). Specically, the corresponding additive
half-quadratic MM algorithm for Eq. (B.6) is given by the iteration
^
f
(i)
=A(d) + M arg min
z
g
X
(A(d) + Mz;
^
f
(i1)
)
=A(d) + MM
1
5
M
H

P

X
(L
r
(P
X
(
^
f
(i1)
)))P

X
(P
X
(A(d)))

;
(B.20)
where
M
5
(z) = M
H
P

X
(P
X
(Mz)): (B.21)
In this case, the matrix inverse can also be solved analytically because the operatorM
5
() can also
be represented as a diagonal matrix.
The corresponding additive half-quadratic MM algorithm for Eq. (B.9) is given by
^
(i)
= arg min

kE dk
2
`
2
+g
x
(F; F^
(i1)
)
=M
1
6

E
H
d +F
H
P

X
(L
r
(P
X
(F^
(i1)
))

;
(B.22)
where
M
6
(), E
H
E +F
H
P

X
(P
X
(F)): (B.23)
In this case, the matrix inverse cannot be solved analytically, although fast computations are
possible using algorithms like the conjugate gradient method or LSQR [101, 127] because the
operatorP

X
(P
X
()) can be represented as a diagonal matrix, and multiplications with the F and
E matrices can be computed eciently using FFTs [22].
166
B.B.2.2 Multiplicative Half-Quadratic Majorize-Minimize Approach
A dierent majorizer forJ
r
() can also be easily derived from the representation ofJ
r
() in Eq. (B.4).
Specically, it is easy to show that the following function
h
X
(f;
^
f
(i1)
),kP
X
(f)N
r
(P
X
(
^
f
(i1)
))k
2
F
(B.24)
=kP
X
(f)P
X
(f)R
r
(P
X
(
^
f
(i1)
))R
r
(P
X
(
^
f
(i1)
))
H
k
2
F
; (B.25)
is also a majorizer of the function J
r
(P
X
(f)) at the point
^
f
(i1)
. If we assume that X2 C
PQ
,
then we can deneN
r
(X) as the operator that constructs a Q (Qr) matrix whose columns
are equal to the right singular vectors associated with the (Qr) smallest and/or zero singular
values in the extended singular value decomposition of X. Under the same assumptions on X,
we can deneR
r
(X) as the operator that constructs a Qr matrix whose columns are equal to
the right singular vectors associated with the r largest singular values in the extended singular
value decomposition of X. In other words, the columns ofN
r
(X) form an orthonormal basis
for the (Qr)-dimensional approximate nullspace of X, while the columns ofR
r
(X) form an
orthonormal basis for the r-dimensional approximate rowspace of X. We call Eqs. (B.24) and
(B.25) multiplicative half-quadratic majorizers because the majorant has structural resemblence to
previous multiplicative half-quadratic methods [145].
Both Eqs. (B.24) and (B.25) enable the following multiplicative half-quadratic MM algorithm
for solving Eq. (B.5) that consists of iteratively solving simple least-squares problems:
^
f
(i)
= arg min
f
kAf dk
2
`
2
+h
X
(f;
^
f
(i1)
)
=M
1
7
A
H
d
(B.26)
where
M
7
(f), A
H
Af +P

X
(P
X
(f)N
r
(P
X
(
^
f
(i1)
))N
r
(P
X
(
^
f
(i1)
))
H
):
= A
H
Af +P

X
(P
X
(f))P

X
(P
X
(f)R
r
(P
X
(
^
f
(i1)
))R
r
(P
X
(
^
f
(i1)
))
H
):
(B.27)
167
Note that Eq. (B.24) is extremely similar to the cost function used in AC-LORAKS [6] from
Eq. (B.10), and can be optimized in exactly the same way. We should also note the MM algorithm
described in Eq. (B.26) using the majorant from Eq. (B.24) has strong similarities to an algorithm
described in Ref. [102] for a slightly dierent cost function.
Similar to the case for the additive half-qudaratic MM algorithm, this multiplicative half-
quadratic approach is easy to generalize to the other formulations that involveJ
r
(), i.e., LORAKS/P-
LORAKS with exact data consistency constraints as dened in Eq. (B.6), and SENSE-LORAKS
as dened in Eq. (B.9). Specically, the multiplicative half-quadratic MM algorithm corresponding
to Eq. (B.6) is given by the iteration
^
f
(i)
=A(d) + M arg min
z
h
X
(A(d) + Mz;
^
f
(i1)
)
=A(d) MM
1
8
M
H
P

X
(P
X
(A(d))N
r
(P
X
(
^
f
(i1)
))N
r
(P
X
(
^
f
(i1)
))
H
)
=A(d) + MM
1
8
M
H
P

X
(P
X
(A(d))R
r
(P
X
(
^
f
(i1)
))R
r
(P
X
(
^
f
(i1)
))
H
);
(B.28)
where
M
8
(z), M
H
P

X
(P
X
(Mz)N
r
(P
X
(
^
f
(i1)
))N
r
(P
X
(
^
f
(i1)
))
H
)
= M
H
P

X
(P
X
(Mz)) M
H
P

X
(P
X
(Mz)R
r
(P
X
(
^
f
(i1)
))R
r
(P
X
(
^
f
(i1)
))
H
);
(B.29)
and the corresponding multiplicative half-quadratic MM algorithm for Eq. (B.9) is given by
^
(i)
= arg min

kE dk
2
`
2
+h
x
(F; F^
(i1)
)
=M
1
9
E
H
d;
(B.30)
where
M
9
(), E
H
E +F
H
P

X
(P
X
(F)N
r
(P
X
(
^
f
(i1)
))N
r
(P
X
(
^
f
(i1)
))
H
)
= E
H
E +F
H
P

X
(P
X
(F))F
H
P

X
(P
X
(F)R
r
(P
X
(
^
f
(i1)
))R
r
(P
X
(
^
f
(i1)
))
H
):
(B.31)
168
Our implementations of Eqs. (B.26), (B.28), and (B.30) frequently use the majorant form based
onR
r
() from Eq. (B.25) (with one exception as described in the next subsection). This choice
generally reduces memory requirements and computational complexity, sinceR
r
(P
X
(
^
f
(i1)
)) will
be a smaller matrix thanN
r
(P
X
(
^
f
(i1)
)) wheneverr< (Qr) (which is true for most multi-channel
reconstruction scenarios.).
B.B.2.3 FFT-Based Computations
All of the algorithms described above depend on being able to evaluate the LORAKS matrix
constructorP
X
(f) repeatedly for dierent choices of f. However, it's important to note that con-
structing the LORAKS matrix can be computationally expensive, especially because the LORAKS
matrix often has a substantially higher dimension than the original k-space data. In scenarios
where the data is large (e.g., high-resolution acquisitions, 3D or higher-dimensional acquisitions, or
parallel imaging with a large receiver array), the LORAKS matrices will often occupy a substantial
amount of memory if they are calculated explicitly and stored in their entirety [11].
A key recent observation [102] is that, because the LORAKS matrices are associated with
shift-invariant convolution operations, it is possible to use FFT-based implementations of fast
convolution to rapidly compute matrix-vector multiplications of the form P
X
(f)n for arbitrary
vectors f and n, without the need to explicitly calculate the matrixP
X
(f). This observation is useful
for accelerating computations associated with AC-LORAKS (i.e., Eqs. (B.11), (B.13), and (B.15))
and the multiplicative half-quadratic MM algorithms (i.e., Eqs. (B.26), (B.28), (B.30)), which don't
possess the simple analytic inversion formulae associated with the additive half-quadratic algorithm,
and for which this type of multiplication appears as a component of the operators that need to be
inverted.
We illustrate this FFT-based approach for the case of the single-channel C matrix. Specically,
consider the computation of the matrix-matrix productP
C
(f)
^
V, which is a subcomponent of the
M
1
operator from Eq. (B.12). It is easy to see that the ith column ofP
C
(f)
^
V can be computed
by the operator
L
i
(f),P
C
(f)^ v
i
; (B.32)
169
where ^ v
i
isith column of
^
V. Because of the convolutional structure of the C matrix, this operator
can be computed with FFTs, leveraging the fact that standard convolution can be implemented
using zero-padded circular convolution, and circular convolution can be implemented eciently
using FFTs. Specically, Eq. (B.32) is equivalent to
L
i
(f) = TF
1
(F(Z
2
^ v
i
)F(Z
1
f)); (B.33)
whereF andF
1
are FFT and inverse FFT operators; Z
1
and Z
2
are zero-padding operators,
represents the Hadamard product operation (i.e., element-wise multiplication), and T extracts the
relevant samples (i.e., the samples corresponding to the LORAKS neighborhood centers) from the
convolution output.
The adjoint ofL
i
is also easy to dene using FFTs:
L

i
(y) = Z
H
1
F
1
(F(Z
2
^ v
i
)F(T
H
y)); (B.34)
where x is used to denote the complex conjugate of x.
Using these FFT-based denitions, the computation ofM
1
from Eq. (B.12) can then be calcu-
lated using
M
1
(f), A
H
Af +P

X
(P
X
(f)
^
V
^
V
H
)
= A
H
Af +
X
i
L

i
(L
i
(f))
= A
H
Af +
X
i
Z
H
1
F
1
(F(Z
2
^ v
i
)F(T
H
TF
1
(F(Z
2
^ v
i
)F(Z
1
f)))):
(B.35)
170
Further computational accelerations are possible if we approximate the binary diagonal matrix
T
H
T with an identity matrix [102]. This approximation only eects the behavior at the edges of k-
space and has relatively small impact on the reconstruction result [102]. With this approximation,
an approximation of M
1
can be implemented using
M
1
(f) A
H
Af +
X
i
Z
H
1
F
1
(F(Z
2
^ v
i
)F(Z
2
^ v
i
)F(Z
1
f))
= A
H
Af +Z
H
1
F
1

X
i
jF(Z
2
^ v
i
)j
2
!
F(Z
1
f)
!
= A
H
Af +Z
H
1
F
1
(NF(Z
1
f));
(B.36)
where N =
P
i
jF(Z
2
^ v
i
)j
2
is a diagonal matrix (in the single-channel case). Note that when we
approximate T
H
T with an identity matrix, we observe that using the nullspace-based majorant
from Eq. (B.24) is preferred over the majorant from (B.25) based on the signal subspace. As a
result, our software implementation for this case uses the majorant from Eq. (B.24).
Although we have only showedM
1
computation for the case of the single-channel C matrix,
we can apply similar approaches for the other operatorsM
i
, the other single-channel LORAKS
matrices S and W, and to the multi-channel LORAKS matrices (note that in the multi-channel
case, N is no longer block diagonal, but instead has the form of a Hermitian-symmetric block
matrix, where each block is diagonal).
B.C Software
The supplementary MATLAB code contains three main LORAKS-related functions, P LORAKS.m,
AC LORAKS.m, and SENSE LORAKS.m (which are described below), as well as several example demon-
stration scripts (which are described in Sec. B.D).
171
B.C.1 P LORAKS.m
The P LORAKS.m function provides capabilities to solve the optimization problems from Eqs. (B.5)
or (B.6), and provides a variety of options as described in the MATLAB help documentation (as
reproduced below):
function [recon] = P LORAKS(kData, kMask, rank, R, LORAKS type, lambda, alg, tol, ...
max iter, VCC)
% This function provides capabilities to solve single-channel and multi-channel
% LORAKS reconstruction problems using one of the formulations from either
% Eq. (5) (which uses LORAKS as regularization and does not require strict data-
% consistency) or Eq. (6) (which enforces strict data-consistency) from the
% technical report:
%
% [1] T. H. Kim, J. P. Haldar. LORAKS Software Version 2.0:
% Faster Implementation and Enhanced Capabilities. University of Southern
% California, Los Angeles, CA, Technical Report USC-SIPI-443, May 2018.
%
% The problem formulations implemented by this function were originally reported
% in:
%
% [2] J. P. Haldar. Low-Rank Modeling of Local k-Space Neighborhoods (LORAKS)
% for Constrained MRI. IEEE Transactions on Medical Imaging 33:668-681,
% 2014.
%
% [3] J. P. Haldar, J. Zhuo. P-LORAKS: Low-Rank Modeling of Local k-Space
% Neighborhoods with Parallel Imaging Data. Magnetic Resonance in Medicine
% 75:1499-1514, 2016.
%
%
*********************
% Input Parameters:
%
*********************
%
% kData: A 3D (size N1 x N2 x Nc) array of measured k-space data to be
172
% reconstructed. The first two dimensions correspond to k-space
% positions, while the third dimension corresponds to the channel
% dimension for parallel imaging. Unsampled data samples should be
% zero-filled. The software will use the multi-channel formulation if
% Nc > 1, and will otherwise use the single-channel formulation.
%
% kMask: A binary mask of size N1 x N2 that corresponds to the same k-space
% sampling grid used in kData. Each entry has value 1 if the
% corresponding k-space location was sampled and has value 0 if that
% k-space location was not measured.
%
% rank: The matrix rank value used to define the non-convex regularization
% penalty from Eq. (2) of Ref. [1].
%
% R: The k-space radius used to construct LORAKS neighborhoods. If not
% specified, the software will use R=3 by default.
%
% LORAKS type: A string that specifies the type of LORAKS matrix that will
% be used in reconstruction. Possible options are: 'C', 'S',
% and 'W'. If not specified, the software will use
% LORAKS type='S' by default.
%
% lambda: The regularization parameter from Eq. (5). If lambda=0, the
% software will use the data-consistency constrained formulation from
% Eq. (6) instead. If not specified, the software will use lambda=0
% by default.
%
% alg: A parameter that specifies which algorithm the software should use for
% computation. There are four different options:
% -alg=1: This choice will use the additive half-quadratic algorithm,
% as described in Eq. (18) or (20) of Ref. [1].
% -alg=2: This choice will use the multiplicative half-quadratic
% algorithm, as described in Eq. (26) or (28) of Ref. [1].
% This version does NOT use FFTs.
% -alg=3: This choice will use the multiplicative half-quadratic
173
% algorithm, as described in Eq. (26) or (28) of Ref. [1].
% This version uses FFTs without approximation, as in
% Eq. (35) of Ref. [1].
% -alg=4: This choice will use the multiplicative half-quadratic
% algorithm, as described in Eq. (26) or (28) of Ref. [1].
% This version uses FFTs with approximation, as in Eq. (36)
% of Ref. [1].
% If not specified, the software will use alg=4 by default.
%
% tol: A convergence tolerance. The computation will halt if the relative
% change (measured in the Euclidean norm) between two successive
% iterates is small than tol. If not specified, the software will use
% tol=1e-3 by default.
%
% max iter: The computation will halt if the number of iterations exceeds
% max iter. If not specified, the software will default to using
% max iter=1000 for the additive half-quadratic algorithm (alg=1),
% and will use max iter=50 for the multiplicative half-quadratic
% algorithms (alg=2,3, or 4).
%
% VCC: The software will use virtual conjugate coils if VCC=1, and otherwise
% will not. If not specified, the software will use VCC=0 by default.
%
%
**********************
% Output Parameters:
%
**********************
%
% recon: The array (size N1 x N2 x Nc) of reconstructed k-space data.
B.C.2 AC LORAKS.m
TheAC LORAKS.m function provides capabilities to solve the optimization problems from Eqs. (B.11)
or (B.13), and provides a variety of options as described in the MATLAB help documentation (as
reproduced below):
174
function [recon] = AC LORAKS(kData, kMask, rank, R, LORAKS type, lambda, alg, tol, ...
max iter, VCC)
% This function provides capabilities to solve single-channel and multi-channel
% AC-LORAKS reconstruction problems using one of the formulations from either
% Eq. (11) (which uses LORAKS as regularization and does not require strict data-
% consistency) or Eq. (13) (which enforces strict data-consistency) from the
% technical report:
%
% [1] T. H. Kim, J. P. Haldar. LORAKS Software Version 2.0:
% Faster Implementation and Enhanced Capabilities. University of Southern
% California, Los Angeles, CA, Technical Report USC-SIPI-443, May 2018.
%
% The problem formulations implemented by this function were originally reported
% in:
%
% [2] J. P. Haldar. Autocalibrated LORAKS for Fast Constrained MRI
% Reconstruction. IEEE International Symposium on Biomedical Imaging: From
% Nano to Macro, New York City, 2015, pp. 910-913.
%
%
*********************
% Input Parameters:
%
*********************
%
% kData: A 3D (size N1 x N2 x Nc) array of measured k-space data to be
% reconstructed. The first two dimensions correspond to k-space
% positions, while the third dimension corresponds to the channel
% dimension for parallel imaging. Unsampled data samples should be
% zero-filled. The software will use the multi-channel formulation if
% Nc > 1, and will otherwise use the single-channel formulation.
%
% kMask: A binary mask of size N1 x N2 that corresponds to the same k-space
% sampling grid used in kData. Each entry has value 1 if the
% corresponding k-space location was sampled and has value 0 if that
% k-space location was not measured. It is assumed that kMask will
175
% contain a fully-sampled autocalibration region that is of
% sufficiently-large size that it is possible to estimate the
% nullspace of the LORAKS matrix by looking at the nullspace of a
% fully-sampled submatrix. An error will occur if the software cannot
% find such an autocalibration region.
%
% rank: The matrix rank value used to define the dimension of the V matrix
% in Eq. (10) of Ref. [1].
%
% R: The k-space radius used to construct LORAKS neighborhoods. If not
% specified, the software will use R=3 by default.
%
% LORAKS type: A string that specifies the type of LORAKS matrix that will
% be used in reconstruction. Possible options are: 'C', 'S',
% and 'W'. If not specified, the software will use
% LORAKS type='S' by default.
%
% lambda: The regularization parameter from Eq. (11). If lambda=0, the
% software will use the data-consistency constrained formulation from
% Eq. (13) instead. If not specified, the software will use lambda=0
% by default.
%
% alg: A parameter that specifies which algorithm the software should use for
% computation. There are three different options:
% -alg=2: This choice will use the multiplicative half-quadratic
% algorithm, as described in Eq. (11) or (13) of Ref. [1].
% This version does NOT use FFTs.
% -alg=3: This choice will use the multiplicative half-quadratic
% algorithm, as described in Eq. (11) or (13) of Ref. [1].
% This version uses FFTs without approximation, as in
% Eq. (35) of Ref. [1].
% -alg=4: This choice will use the multiplicative half-quadratic
% algorithm, as described in Eq. (11) or (13) of Ref. [1].
% This version uses FFTs with approximation, as in Eq. (36)
% of Ref. [1].
176
% If not specified, the software will use alg=4 by default.
%
% tol: A convergence tolerance. The computation will halt if the relative
% change (measured in the Euclidean norm) between two successive
% iterates is small than tol. If not specified, the software will use
% tol=1e-3 by default.
%
% max iter: The computation will halt if the number of iterations exceeds
% max iter. If not specified, the software will default to using
% max iter=50.
%
% VCC: The software will use virtual conjugate coils if VCC=1, and otherwise
% will not. If not specified, the software will use VCC=0 by default.
%
%
**********************
% Output Parameters:
%
**********************
%
% recon: The array (size N1 x N2 x Nc) of reconstructed k-space data.
B.C.3 SENSE LORAKS.m
TheSENSE LORAKS.m function provides capabilities to solve the optimization problem from Eq. (B.9),
and provides a variety of options as described in the MATLAB help documentation (as reproduced
below):
function [recon] = SENSE LORAKS(kData, kMask, coil sens, rank, lambda, R, ...
LORAKS type, alg, tol, max iter)
% This function provides capabilities to solve multi-channel SENSE-LORAKS
% reconstruction problems using the formulation from Eq. (9) from the
% technical report:
%
% [1] T. H. Kim, J. P. Haldar. LORAKS Software Version 2.0:
177
% Faster Implementation and Enhanced Capabilities. University of Southern
% California, Los Angeles, CA, Technical Report USC-SIPI-443, May 2018.
%
% The problem formulation implemented by this function was originally reported
% in:
%
% [2] T. H. Kim, J. P. Haldar. LORAKS makes better SENSE: Phase?constrained
% partial fourier SENSE reconstruction without phase calibration. Magnetic
% Resonance in Medicine 77:1021-1035, 2017.
%
%
*********************
% Input Parameters:
%
*********************
%
% kData: A 3D (size N1 x N2 x Nc) array of measured k-space data to be
% reconstructed. The first two dimensions correspond to k-space
% positions, while the third dimension corresponds to the channel
% dimension for parallel imaging. Unsampled data samples should be
% zero-filled.
%
% kMask: A binary mask of size N1 x N2 that corresponds to the same k-space
% sampling grid used in kData. Each entry has value 1 if the
% corresponding k-space location was sampled and has value 0 if that
% k-space location was not measured.
%
% coil sens: A 3D (size N1 x N2 x Nc) array of estimated coil sensitivity
% profiles.
%
% rank: The matrix rank value used to define the non-convex regularization
% penalty from Eq. (2) of Ref. [1].
%
% lambda: The regularization parameter from Eq. (9).
%
% R: The k-space radius used to construct LORAKS neighborhoods. If not
% specified, the software will use R=3 by default.
178
%
% LORAKS type: A string that specifies the type of LORAKS matrix that will
% be used in reconstruction. Possible options are: 'C', 'S',
% and 'W'. If not specified, the software will use
% LORAKS type='S' by default.
%
% alg: A parameter that specifies which algorithm the software should use for
% computation. There are four different options:
% -alg=1: This choice will use the additive half-quadratic algorithm,
% as described in Eq. (22) of Ref. [1].
% -alg=2: This choice will use the multiplicative half-quadratic
% algorithm, as described in Eq. (30) of Ref. [1].
% This version does NOT use FFTs.
% -alg=3: This choice will use the multiplicative half-quadratic
% algorithm, as described in Eq. (30) of Ref. [1].
% This version uses FFTs without approximation, as in
% Eq. (35) of Ref. [1].
% -alg=4: This choice will use the multiplicative half-quadratic
% algorithm, as described in Eq. (30) of Ref. [1].
% This version uses FFTs with approximation, as in Eq. (36)
% of Ref. [1].
% If not specified, the software will use alg=4 by default.
%
% tol: A convergence tolerance. The computation will halt if the relative
% change (measured in the Euclidean norm) between two successive
% iterates is small than tol. If not specified, the software will use
% tol=1e-3 by default.
%
% max iter: The computation will halt if the number of iterations exceeds
% max iter. If not specified, the software will default to using
% max iter=1000 for the additive half-quadratic algorithm (alg=1),
% and will use max iter=50 for the multiplicative half-quadratic
% algorithms (alg=2,3, or 4).
%
%
**********************
179
Figure B.1: Gold standard magnitude (left) and phase (right) images for the single-channel dataset.
% Output Parameters:
%
**********************
%
% recon: The N1 x N2 reconstructed image.
B.D Examples and Usage Recommendations
Our software provides a lot of dierent options, and selecting between dierent alternative ap-
proaches may be a little daunting for users who are new to LORAKS. The following illustrative
examples are provided in the examples subfolder of our software distribution, and are designed to
provide users with some high-level guidance.
B.D.1 Single-channel reconstruction
Our rst example, provided in ex1.m, demonstrates the use of LORAKS and AC-LORAKS with
single-channel data. Gold standard T2-weighted data was fully sampled on a 256 340 k-space
grid with a multi-channel receiver array coil, and was coil compressed down to a single channel.
The magnitude and phase of this gold standard data is shown in Fig. B.1.
To demonstrate the
exibility and characteristics of LORAKS and AC-LORAKS, this dataset is
retrospectively undersampled using the four dierent sampling patterns shown in Fig. B.2. All four
sampling patterns have an acceleration factor of 2 (i.e., retaining only 50% of the original k-space
data), which is somewhat aggressive for single-channel data with one-dimensional undersampling
(i.e., acceleration only along the phase encoding dimension). The retrospectively undersampled data
180
(a) Random Sampling with
Calibration Region
(b) Calibrationless Random
Sampling
(c) Uniform Sampling with
Calibration Region
(d) Partial Fourier Sampling
with Calibration Region
Figure B.2: Sampling patterns used with single-channel data.
is reconstructed using P LORAKS.m and AC LORAKS.m using the default reconstruction parameters
(i.e., using the S matrix with a LORAKS neighborhood radius of 3; using exact data consistency
as in Eqs. (B.6) and (B.13); and using the FFT-based multiplicative half-quadratic algorithm with
approximation of T
H
T as an identity matrix, as described in Sec. B.B.2.3).
Reconstruction results are shown in Fig. B.3, and we report the normalized mean-squared
error (NRMSE) and reconstruction time (measured on one of our standard desktop computers)
in addition to showing qualitative results. As can be seen, all of these reconstruction results
are reasonably successful, despite the variety of dierent sampling patterns (including random
sampling, calibrationless sampling, uniform sampling, and partial Fourier sampling) and despite
the relatively aggressive acceleration factor. Classical image reconstruction techniques will not be
as successful across such a wide range of dierent settings. Although both approaches generally
lead to similar image quality and NRMSE, LORAKS reconstruction using Eq. (B.6) tends to be
much slower than AC-LORAKS reconstruction using Eq. (B.13). On the other hand, the LORAKS
implementation is slightly more generally applicable than the AC-LORAKS implementation, since
it accommodates calibrationless sampling (e.g., Fig. B.2(b)). While AC-LORAKS can be used with
181
(a) Sampling us-
ing Fig. B.2(a),
NRMSE = 0.088,
time = 18.5 sec.
(b) Sampling us-
ing Fig. B.2(b),
NRMSE = 0.087,
time = 21.1 sec.
(c) Sampling us-
ing Fig. B.2(c),
NRMSE = 0.102,
time = 15.6 sec.
(d) Sampling us-
ing Fig. B.2(d),
NRMSE = 0.083,
time = 6.6 sec.
(e) Sampling us-
ing Fig. B.2(a),
NRMSE = 0.082,
time = 3.8 sec.
(f) Sampling us-
ing Fig. B.2(c),
NRMSE = 0.089,
time = 3.9 sec.
(g) Sampling us-
ing Fig. B.2(d),
NRMSE = 0.084,
time = 3.6 sec.
Figure B.3: Reconstruction results from ex1.m. The top row shows results obtained with LO-
RAKS (using P LORAKS.m), while the bottom row shows results obtained with AC-LORAKS (using
AC LORAKS.m).
external calibration data [140], our current software implementation only supports autocalibration.
It would be straightforward to modify AC LORAKS.m to allow external calibration data if so desired,
and the use of external calibration preserves the signicant speed advantages of AC-LORAKS while
also enabling substantial improvements in image quality [140].
We should note that we have not shown results for uniform undersampling without a calibration
region. As described in [140], getting good LORAKS results with calibrationless uniform under-
sampling (e.g., as in standard echo-planar imaging) requires additional prior information and/or
external calibration data.
B.D.2 Multi-channel reconstruction
Our second example, provided in ex2.m, demonstrates the use of P-LORAKS, AC-LORAKS, and
SENSE-LORAKS with multi-channel data. Gold standard MPRAGE data was fully sampled on a
256 256 k-space grid with a multi-channel receiver array coil, and was coil compressed down to
182
(a) (b)
Figure B.4: (a) Gold standard magnitude (top) and phase (bottom) images for each channel of the
multi-channel dataset. (b) The gold standard root-sum-of-squares combination of the channels.
four channels. The magnitude and phase and the root-sum-of-squares coil combination results for
this gold standard data is shown in Fig. B.4.
To demonstrate the
exibility and characteristics of P-LORAKS, AC-LORAKS, and SENSE-
LORAKS, this dataset is retrospectively undersampled using the four dierent sampling patterns
shown in Fig. B.5. All four sampling patterns have an acceleration factor of 7 (i.e., retaining only
14% of the original k-space data), which is very aggressive for four-channel data. Reconstruction
results at this acceleration factor are not necessarily of diagnostic quality, but we've chosen an ag-
gressive acceleration strategy to better highlight the dierences between dierent reconstruction ap-
proaches. The retrospectively undersampled data is reconstructed using P LORAKS.m, AC LORAKS.m,
and SENSE LORAKS.m using the default reconstruction parameters (i.e., using the S matrix with a
LORAKS neighborhood radius of 3; using exact data consistency as in Eqs. (B.6) and (B.13) for
P-LORAKS and AC-LORAKS, while using regularization as in Eq. (B.9) for SENSE-LORAKS;
and using the FFT-based multiplicative half-quadratic algorithm with approximation of T
H
T as
an identity matrix, as described in Sec. B.B.2.3).
Reconstruction results are shown in Fig. B.6. Similar to the single-channel case, all of these
reconstruction results are reasonably successful, despite the variety of dierent sampling patterns
(including random sampling, calibrationless sampling, uniform sampling, and partial Fourier sam-
pling) and despite the relatively aggressive acceleration factor. Although all three approaches gen-
erally lead to roughly similar image quality and NRMSE, LORAKS reconstruction using Eq. (B.6)
tends to be much slower than SENSE-LORAKS reconstruction using Eq. (B.9), which is in turn
183
(a) Random Sampling
with Calibration Region
(b) Calibrationless Ran-
dom Sampling
(c) Uniform Sampling
with Calibration Region
(d) Partial Fourier Sam-
pling with Calibration
Region
Figure B.5: Sampling patterns used with multi-channel data.
much slower than AC-LORAKS reconstruction using Eq. (B.13). The SENSE-LORAKS recon-
struction is the most general in some ways, because can be used with arbitrary sampling patterns
(including calibrationless uniform undersampling [10]), though requires slightly more prior infor-
mation (in the form of coil sensitivity maps) than the other two approaches. Our implementation
of SENSE-LORAKS does not include the additional Tikhonov regularization term described in [10]
for simplicity, although the code is easily modied to include this additional regularization term,
and the additional regularization would be benecial for reducing noise amplication and improving
image quality.
Similar to the single-channel case, the LORAKS implementation is slightly more generally
applicable than the AC-LORAKS implementation, since it accommodates calibrationless sampling
(e.g., Fig. B.5(b)). As before, it would be straightforward to modify AC LORAKS.m to allow external
calibration data if so desired, and the AC-LORAKS approach has substantial speed advantages.
B.D.3 Choice of LORAKS matrix and neighborhood radius
All of the previous results used the S matrix with a LORAKS neighborhood radius of 3, and
without using virtual conjugate coils. Our third example, provided in ex3.m, demonstrates the
behavior when these parameter settings are changed, in the context of the multi-channel data from
Sec. B.D.2 and the random sampling pattern from Fig. B.5(a). Results without and with virtual
conjugate coils are respectively shown in Figs. B.7 and B.8. The results are consistent with our past
184
(a) Sampling us-
ing Fig. B.5(a),
NRMSE = 0.084,
time = 61.5 sec.
(b) Sampling us-
ing Fig. B.5(b),
NRMSE=0.083,
time = 126.6 sec.
(c) Sampling us-
ing Fig. B.5(c),
NRMSE = 0.070,
time = 80.1 sec.
(d) Sampling us-
ing Fig. B.5(d),
NRMSE=0.116,
time = 103.1 sec.
(e) Sampling us-
ing Fig. B.5(a),
NRMSE = 0.096,
time = 7.6 sec.
.................................
(f) Sampling us-
ing Fig. B.5(c),
NRMSE = 0.075,
time = 7.7 sec.
(g) Sampling us-
ing Fig. B.5(d),
NRMSE = 0.107,
time = 7.5 sec.
(h) Sampling us-
ing Fig. B.5(a),
NRMSE = 0.103,
time = 20.2 sec.
(i) Sampling us-
ing Fig. B.5(b),
NRMSE = 0.102,
time = 23.8 sec.
(j) Sampling us-
ing Fig. B.5(c),
NRMSE = 0.085,
time = 16.7 sec.
(k) Sampling us-
ing Fig. B.5(d),
NRMSE = 0.115,
time = 12.9 sec.
Figure B.6: Reconstruction results from ex1.m. The top row shows results obtained with P-
LORAKS (using P LORAKS.m), while the middle row shows results obtained with AC-LORAKS
(using AC LORAKS.m), and the bottom row shows results obtained with SENSE-LORAKS (using
SENSE LORAKS.m).
185
experience [8], and show that the S matrix without virtual coils, the S matrix with virtual coils,
or the C matrix with virtual coils are generally the top-performing LORAKS matrices, and are
relatively similar to each other in reconstructed image quality. On the other hand, the W matrices
tend to have the worst performance. Among the top-performing matrices, we often recommend
using the S matrix without virtual conjugate coils, because using virtual coils increases the memory
requirements and computational complexity without a noticeable improvement in quality for the
S matrix. The use of virtual coils can often lead to substantial quality improvements for the C
matrix and sometimes (but not in this specic example) for the W matrix. Virtual conjugate
coils are generally not as useful for the S matrix because their main purpose is to introduce phase
constraints, while the S matrix already exploits such constraints.
The choice of the neighborhood radius represents a classical trade-o in constrained reconstruc-
tion. Larger neighborhood radii allows the LORAKS model to be more
exible and adaptable,
but also more sensitive to noise. Using larger neighborhood radii is also generally associated with
increased memory requirements, increased computational complexity per iteration, and a larger
number of iterations to reach convergence.
B.D.4 Choice of algorithm
Our software provides access to four dierent algorithms: (alg 1) the original additive half-quadratic
approach (Sec. B.B.2.1), (alg 2) the multiplicative half-quadratic approach (Sec. B.B.2.2), (alg 3)
the multiplicative half-quadratic approach with FFT-based computations (Sec. B.B.2.3), and (alg 4)
the multiplicative half-quadratic approach with FFT-based computations and with approximation
of T
H
T as an identity matrix (Sec. B.B.2.3). These four dierent algorithms are compared in our
fourth example, provided in ex4.m.
This example considers P-LORAKS reconstruction in the context of the multi-channel data
from Sec. B.D.2 and the random sampling pattern from Fig. B.5(a). P-LORAKS reconstruction is
performed using the software default settings, other than varying the choice of algorithm. NRMSE,
iteration, and computation time results are shown below in Table B.1.
186
(a) S, R =2,
NRMSE = 0.101,
time = 4.1 sec.
(b) S, R=3,
NRMSE = 0.092,
time = 4.8 sec.
(c) S, R=4,
NRMSE = 0.092,
time = 5.0 sec.
(d) S, R=5,
NRMSE = 0.109,
time = 6.0 sec.
(e) C, R=2,
NRMSE = 0.108,
time = 1.9 sec.
(f) C, R=3,
NRMSE = 0.098,
time = 2.3 sec.
(g) C, R=4,
NRMSE = .095,
time = 2.5 sec.
(h) C, R=5,
NRMSE = 0.094,
time = 3.2 sec.
(i) W, R=2,
NRMSE = 0.111,
time = 6.2 sec.
(j) W, R=3,
NRMSE = 0.111,
time = 7.3 sec.
(k) W, R=4,
NRMSE = 0.110,
time = 6.7 sec.
(l) W, R=5,
NRMSE = 0.116,
time = 8.1 sec.
Figure B.7: The eects of dierent choices of the LORAKS matrix type (C, S, or W), dierent
choices of the LORAKS neighborhood radius R, and not using virtual conjugate coils.
187
(a) S, R =2,
NRMSE = 0.096,
time = 9.4 sec.
(b) S, R=3,
NRMSE = 0.092,
time = 10.5 sec.
(c) S, R=4,
NRMSE = 0.093,
time = 10.5 sec.
(d) S, R=5,
NRMSE = 0.118,
time = 12.2 sec.
(e) C, R=2,
NRMSE = 0.100,
time = 3.8 sec.
(f) C, R=3,
NRMSE = 0.091,
time = 4.5 sec.
(g) C, R=4,
NRMSE = 0.093,
time = 4.7 sec.
(h) C, R=5,
NRMSE = 0.111,
time = 5.7 sec.
(i) W, R=2,
NRMSE = 0.134,
time = 17.5 sec.
(j) W, R=3,
NRMSE = 0.135,
time = 19.6 sec.
(k) W, R=4,
NRMSE = 0.137,
time = 18.3 sec.
(l) W, R=5,
NRMSE = 0.146,
time = 18.5 sec.
Figure B.8: The eects of dierent choices of the LORAKS matrix type (C, S, or W), dierent
choices of the LORAKS neighborhood radius R, and using virtual conjugate coils.
188
alg 1 alg 2 alg 3 alg 4
NRMSE 0.09538 0.08354 0.08354 0.08353
# of iterations 71 10 10 10
Time (sec) 141.8 1059.7 1253.5 61.4
Table B.1: Algorithm comparison results from ex4.m.
As can be seen, the additive half-quadratic algorithm (alg 1) has dierent characteristics than
the multiplicative half-quadratic algorithms (alg 2, alg 3, and alg 4). In particular, additive half-
quadratic approach is characterized by having a small computational cost per iteration, though
requires a large number of iterations to converge. The multiplicative half-quadratic algorithms
converge much faster, though require more computational eort per iteration. These dierences in
convergence characteristics mean that the step size for each iteration is dierent between the addi-
tive and multiplicative approaches, such that using the same stopping criterion does not guarantee
the same degree of convergence.
As expected, the two unapproximated multiplicative half-quadratic algorithms (alg 2 and alg
3) have identical results, aside from numerical nite precision eects. However, the original version
(alg 2) is much faster than the version using FFTs (alg 3). The approximate multiplicative half-
quadratic algorithm (alg 4) yields very similar results to the unapproximated versions, though is
by far the fastest algorithm. We generally recommend the use of alg 4 due to its computational
eciency.
B.D.5 Choice of regularization parameter and the maximum number of iterations
Our nal examples illustrate the eects of the regularization parameter and the maximum num-
ber of iterations. Both examples consider AC-LORAKS reconstruction in the context of the multi-
channel data from Sec. B.D.2 and the random sampling pattern from Fig. B.5(a). These recon-
structions use default settings, other than varying and the maximum number of iterations.
Dierent choices of are illustrated in ex5.m, and NRMSE values are shown as a function of
in Fig. B.9. As can be seen, the tuning of is nontrivial. When is too small, numerical eects
dominate and LORAKS regularization doesn't have much eect. On the other hand, choosing
189
10
−5
10
−4
10
−3
10
−2
10
−1
10
0
10
1
10
2
0
0.1
0.2
0.3
λ
NRMSE
NRMSEversusλ
Regularization(non-zeroλ)
Exactdataconsistency(λ = 0)
Figure B.9: The eects of the regularization parameter on AC-LORAKS reconstruction in ex5.m.
too large can also lead to signicant image reconstruction biases. However, enforcing exact data
consistency and using LORAKS only to interpolate/extrapolate missing data (by setting = 0)
works well and does not require parameter tuning. It should be noted that our implementation
of SENSE-LORAKS does not currently allow setting = 0. (In the context of SENSE, there are
several dierent ways of trying to incorporate exact data consistency constraints, and each of these
would be straightforward to incorporate by modifying the code we've provided).
It is also worth mentioning that, like many iterative algorithms, the iterative algorithms we've
implemented for LORAKS can demonstrate classical semiconvergence phenomenon, in which trun-
cating the iterative procedure early can have a noise-suppressing regularization eect. This eect
is illustrated in ex6.m, with results shown in Fig. B.10. As a result, it is not always benecial to
iterate the LORAKS algorithms until convergence. On the other hand, rather than relying on semi-
convergence (which can be dicult to characterize), it can also be benecial to include additional
regularization penalties into the LORAKS reconstruction to prevent excessive noise amplication
[3, 7, 10, 11]. For simplicity, our current software does not provide access to additional regular-
ization, although this would be relatively easy to incorporate through simple modications of the
provided code.
190
10 20 30 40 50 60
0.1
0.12
0.14
0.16
Iterations
NRMSE
NRMSEversusIterations
Figure B.10: The eects of the number of iterations on AC-LORAKS reconstruction in ex6.m.
191
This page intentionally left blank

Abstract (if available)

Abstract Magnetic resonance imaging (MRI) is a noninvasive biomedical imaging modality for visualizing tissues inside the human body without ionizing radiation. It is a versatile diagnostic tool for identifying anatomy (static and dynamic), functionality, and physiology of organs. Due to its fundamental physical limitations, however, generating high-quality images is slow and expensive, where researchers have been working for decades to overcome. Although there are many complementary approaches that can help achieve this goal, the main focus of this dissertation will be on computational imaging methods that use mathematical modeling to obtain high-quality images from incomplete data. ❧ While there are many different kinds of computational imaging methods, this dissertation will focus on methods that assume that MRI data is autoregressive and shift-invariant. To be specific, ""autoregressiveness"" implies that an MRI data sample can be synthesized by taking linear/nonlinear combinations of its local neighbors, and ""shift-invariance"" indicates that such predictive relationships are consistent across entire local MRI data regions. These conventional constraints imply redundancies in MRI data and allow accelerated MRI scan by exploiting them. ❧ In this study, we will first present improved MRI reconstruction using LORAKS. LORAKS (Low-rank modeling of local k-space neighborhoods) is a powerful linear shift-invariant autoregressive reconstruction technique that is based on limited image support, slowly-varying phase, sparsity in transform domains, and/or parallel imaging constraints, where such constraints are implemented through structured low-rank matrix modeling. It will show many improved MRI applications using LORAKS, in simultaneous multislice imaging (SMS-LORAKS), sensitivity-encoded imaging (SENSE-LORAKS), and highly-accelerated 3D imaging combined with Wave-CAIPI data acquisition (Wave-LORAKS). ❧ It will then introduce LORAKI, a novel nonlinear shift-invariant autoregressive MRI reconstruction framework that is based on autocalibrated artificial neural networks. The network structure of LORAKI was adopted and motivated from an iterative algorithm solving LORAKS, comprising recurrent neural networks (RNNs). As a result, LORAKI inherits many good features of LORAKS while it also offers improved image qualities from its nonlinearities. It is a scan-specific network that does not require external dataset, which can be useful and reliable in many scenarios where large-scale training data are difficult to acquire. ❧ Lastly, it will propose efficient iterative solutions to complex-valued nonlinear least-squares problems with mixed linear and antilinear operators. This study focuses on complex-valued least-squares problems where the forward operator can be decomposed into linear and antilinear components. While such formulations are nonlinear in its original complex-domain, previous literature addressed them by mapping into equivalent real-valued linear least-squares and applying linear solvers (e.g. Landweber iteration, Conjugate Gradient, LSQR). While this approach is valid, it may introduce additional efforts in reformulation and inefficiencies in computation. This work proposes theory and computational methods that enable such problems to be solved iteratively using standard linear least-squares tools, while retaining all of the complex-valued structure of the original inverse problem. The proposed algorithms can be widely applicable in solving many inverse problem scenarios, including phase-constrained MRI reconstruction methods.

Linked assets

University of Southern California Dissertations and Theses

Conceptually similar

PDF

New theory and methods for accelerated MRI reconstruction

PDF

Improved brain dynamic contrast enhanced MRI using model-based reconstruction

PDF

Optimization methods and algorithms for constrained magnetic resonance imaging

PDF

Correction, coregistration and connectivity analysis of multi-contrast brain MRI

PDF

High-dimensional magnetic resonance imaging of microstructure

PDF

Efficient deep learning for inverse problems in scientific and medical imaging

PDF

Fast flexible dynamic three-dimensional magnetic resonance imaging

PDF

Measuring functional connectivity of the brain

PDF

Model based PET image reconstruction and kinetic parameter estimation

PDF

Emotional speech production: from data to computational models and applications

PDF

Invariant representation learning for robust and fair predictions

PDF

Edge-cloud collaboration for enhanced artificial intelligence

PDF

Functional real-time MRI of the upper airway

PDF

Human appearance analysis and synthesis using deep learning

PDF

Multi-constrained inversion algorithms for microwave imaging

PDF

Seeing sleep: real-time MRI methods for the evaluation of sleep apnea

PDF

Functional connectivity analysis and network identification in the human brain

PDF

Point-based representations for 3D perception and reconstruction

PDF

Fast iterative image reconstruction for 3D PET and its extension to time-of-flight PET

PDF

Deep learning techniques for supervised pedestrian detection and critically-supervised object detection

Asset Metadata

Creator Kim, Tae Hyung (author)

Core Title Shift-invariant autoregressive reconstruction for MRI

School Viterbi School of Engineering

Degree Doctor of Philosophy

Degree Program Electrical Engineering

Publication Date 12/13/2020

Defense Date 05/01/2020

Publisher University of Southern California (original), University of Southern California. Libraries (digital)

Tag artificial neural networks,computational imaging,constrained reconstruction,deep learning,image reconstruction,inverse problems,magnetic resonance imaging,medical imaging,OAI-PMH Harvest,signal processing

Language English

Contributor Electronically uploaded by the author (provenance)

Advisor Haldar, Justin P. (committee chair), Leahy, Richard M. (committee member), Nayak, Krishna S. (committee member), Wood, John C. (committee member)

Creator Email kimth@umich.edu,taehyung@usc.edu

Permanent Link (DOI) https://doi.org/10.25549/usctheses-c89-414371

Unique identifier UC11667834

Identifier etd-KimTaeHyun-9220.pdf (filename),usctheses-c89-414371 (legacy record id)

Legacy Identifier etd-KimTaeHyun-9220.pdf

Dmrecord 414371

Document Type Dissertation

Rights Kim, Tae Hyung

Type texts

Source University of Southern California (contributing entity), University of Southern California Dissertations and Theses (collection)

Access Conditions The author retains rights to his/her dissertation, thesis or other graduate work according to U.S. copyright law. Electronic access is being provided by the USC Libraries in agreement with the a...

Repository Name University of Southern California Digital Library

Repository Location USC Digital Library, University of Southern California, University Park Campus MC 2810, 3434 South Grand Avenue, 2nd Floor, Los Angeles, California 90089-2810, USA