UMN 2024‐019 SYSTEMS^AND^METHODS^FOR^PHYSICS‐DRIVEN^MRI^RECONSTRUCTION^WITHOUT^ ACCESS^TO^RAW^K‐SPACE^DATA^ CROSS-REFERENCE TO RELATED APPLICATIONS [0001] This application is based on, claims priority to, and incorporates herein by reference for all purposes, U.S. Provisional Patent Application No. 63/622,222 filed on January 18, 2024. STATEMENT REGARDING FEDERALLY SPONSORED RESEARCH [0002] This invention was made with government support under HL153146, EB027061, and EB032830 awarded by the National Institutes of Health. The government has certain rights in the invention. BACKGROUND [0003] Deep learning (“DL”) has emerged as a tool for improving image reconstruction. A common strategy among DL methods is the physics-driven approach, where a regularized iterative algorithm alternating between data consistency and a regularizer is unrolled for a finite number of iterations. This unrolled network may be trained in a supervised, unsupervised, or self-supervised way and applied to undersampled data, provided that the raw k-space data is available. This raw k-space data is typically required to enforce data consistency. However, raw k-space data is often unavailable or difficult to obtain, especially in clinical settings. SUMMARY OF THE DISCLOSURE [0004] The present disclosure provides systems and methods for physics-driven MRI reconstruction without access to raw k-space data. [0005] Some aspects of the present disclosure provide a computer-implemented 1 QB\920171.00621\94202173.1
UMN 2024‐019 method for training a nonlinear reconstruction algorithm to reconstruct an image. The method includes accessing magnetic resonance (MR) image data with a computer system. The MR image data include previously reconstructed images that were generating by reconstructing k-space data acquiring with a magnetic resonance imaging (MRI) system. The method further includes using the computer system to train a physics-driven nonlinear reconstruction algorithm having an objective function. The objective function includes a regularization unit and a data consistency unit, which is defined relative to the previously reconstructed images of the MR image data. The method further includes storing the reconstruction algorithm using the computer system. [0006] Other aspects of the present disclosure provide a method for reconstructing an enhanced image from a reconstructed magnetic resonance image. The method includes accessing a pre-trained non-linear reconstruction algorithm with a computer system. The pre-trained reconstruction algorithm has been trained using an objective function comprising a regularization unit and a data consistency unit, which is defined relative to previously reconstructed MR image data reconstructed from subsampled k-space data. The method further includes accessing a reconstructed magnetic resonance image with the computer system. The reconstructed magnetic resonance image was obtained from a subject using an MRI system with k-space undersampling. The method further includes inputting the reconstructed magnetic resonance image to the pre-trained reconstruction algorithm network to generate output as an enhanced reconstructed image that depicts the subject. The method also includes displaying the image to a user using the computer system. [0007] The foregoing and other aspects and advantages of the present disclosure will appear from the following description. In the description, reference is made to the accompanying drawings that form a part hereof, and in which there is shown by way of 2 QB\920171.00621\94202173.1
UMN 2024‐019 illustration a preferred embodiment. This embodiment does not necessarily represent the full scope of the invention, however, and reference is therefore made to the claims and herein for interpreting the scope of the invention. BRIEF DESCRIPTION OF THE DRAWINGS [0008] FIG. 1A shows a schematic example of an iterative scheme of a reconstruction problem. [0009] FIG. 1B shows an unrolled neural network architecture with each step including a regularization (R) and a data consistency (DC) unit. [0010] FIG. 1C shows an example of a ResNet architecture containing convolutional layers and residual blocks (RB) which include two convolutional layers with the first one being followed by a ReLU and the second one being followed by a constant multiplication layer. [0011] FIG. 2 is a flowchart setting forth the steps of an example method for training and implementing a machine learning algorithm, in which reconstructed magnetic resonance imaging (MRI) images are used to enforce data consistency in accordance with some aspects of the present disclosure. [0012] FIG. 3 is a flowchart setting forth the steps of an example method for implementing a pre-trained neural network or other machine learning algorithm to reconstruct images from previously reconstructed medical image data, where the neural network or other machine learning algorithm has been trained in accordance with the methods described in the present disclosure. [0013] FIG.4 is a block diagram of an example magnetic resonance imaging (MRI) system that can implement the methods described in the present disclosure. [0014] FIG.5 is a block diagram of an example training and image reconstruction 3 QB\920171.00621\94202173.1
UMN 2024‐019 system that can implement the methods of the present disclosure. [0015] FIG.6 is a block diagram of example components that can implement the system of FIG.5. DETAILED DESCRIPTION [0016] Described here are systems and methods for training non-linear models or reconstruction algorithms, such as machine learning (ML) algorithms or other nonlinear models (e.g., compressed sensing-based reconstruction algorithms, etc.), for solving inverse problems for image reconstruction, without requiring raw k-space data. As an example, the non-linear reconstruction algorithm may implement a physics-based reconstruction, such as a physics-based ML reconstruction. As another example, the nonlinear algorithm may implement a compressed sensing-based reconstruction, or other inverse problems with transform domain sparsity regularization. The non-linear or ML-based reconstruction algorithms described in the present disclosure can be trained on existing databases of undersampled images or in a scan-specific manner. [0017] In general, the systems and methods described in the present disclosure generate a substitution for data consistency or data fidelity that is based on reconstructed images that can readily be accessed from a standard magnetic resonance imaging (MRI) system or image database. During training, data consistency is enforced based on coil- combined or muti-coil imaging data, such as Digital Imaging and Communications in Medicine (DICOM) images. In this way, ML algorithms or other models can be trained for reconstruction using a physics-driven inverse problem in many settings, including where raw k-space data is not available or difficult to acquire. As noted above, the ML algorithms can also be trained for other inverse problems, such as compressed sensing-based reconstructions or other inverse problems that include transform domain sparsity 4 QB\920171.00621\94202173.1
UMN 2024‐019 regularization. While the methods are described by way of example in the context of ML algorithms herein, the method for data consistency substitution can be similarly applied to other methods (e.g., proximal gradient method, variable splitting, primal-dual algorithms, and other methods) for solving any inverse problem that includes a data consistency unit. [0018] The following is an example of a linear inverse problem: ^^ ൌ ^^^^ ^ ^^, (1) [0019] where ^^ is the measured k-space data and ^^ is measurement noise. ^^ ∈ ℂெൈே is the magnetic resonance (MR) encoding operator, which may include multi-coil information, such as sensitivity profiles. When ^^ is undersampled, ^^ is typically ill- conditioned in some sense (e.g., either ^^ ^ ^^ or the condition number of ^^ is high). [0020] An estimate for the fully-sampled data ^^ can be obtained based on the observations, ^^, and the encoding matrix, ^^, by solving, m௫in dist^^^, ^^^^^ ^ ^^^^^^ (2) [0021] where dist^∙,∙^ is a distance metric based on the distribution of ^^ and ^^^⋅^ is a regularizer. Usually, ^^ is representative of Gaussian noise, so the distance metric can become dist^^^,^^^^^ ൌ ||^^ െ ^^^^|| ଶ ଶ. As a non-limiting example, the regularizer may have a form similar to ^^‖Ψ^^‖^ , where ^^ is a regularization parameter and Ψ is a sparsifying transform, such as a wavelet transform. [0022] A solution for this problem can be obtained using a variety of methods, which typically commonly include the following: 1) a regularize-related operation, such as a gradient descent step or a proximal operator and 2) a data consistency operation. The data consistency often falls into two categories, including a gradient descent-based method (e.g., in proximal gradient descent) and a use of a penalized-ℓଶ loss (e.g., in alternating direction method of multipliers (ADMM)). Using gradient descent, 5 QB\920171.00621\94202173.1
UMN 2024‐019 intermediate estimates, ^^^^^, are updated according to:
where ^^ is a (learnable) step size, and ^⋅^ு denotes Hermitian transpose of a matrix. Using penalized-ℓଶ loss can be represented as:
where ^^ is a penalty scalar, which may be learnable, and ^^ is the identity matrix. [0023] Notably, both approaches require the use of the raw data ^^ through ^^ு^^. This corresponds to a direct reconstruction of the acquired data, which is often referred to as the zero-filled image. However, most MRI systems or scanners do not provide an option to output this image directly, especially without pulse sequence access. Instead, they output a reconstruction, which may be referred to as an online reconstruction. The online reconstruction often uses parallel imaging, such as GRAPPA or SENSE, which calculates another function of ^^ . Furthermore, most users arguably prefer the latter option to generating an unreconstructed or aliased zero-filled image, especially in a clinical setting. [0024] The present disclosure provides an alternative approach to enforce data consistency, which does not rely on access to the raw data, ^^. The described method can use reconstructed imaging data readily available on the scanner, such as multi-coil or coil- combined complex (e.g., real and imaginary parts or magnitude and phase parts) or real reconstructed images. [0025] In some implementations, the reconstructed image, ^^^ூ is a coil-combined, complex image. The online reconstruction often generates ^^^ூ using GRAPPA or SENSE. As a non-limiting example, when ^^ is overdetermined (e.g., more rows than columns), the parallel imaging solution can be written for SENSE as: 6 QB\920171.00621\94202173.1
UMN 2024‐019
Eqn. (5) applies for an image reconstructed using SENSE. However, the reconstructed image may be generated using another approach, such as GRAPPA, compressed sensing, or another linear or non-linear method. In any case, the reconstructed image ^^^ூ can be estimated by Eqn. (5), where ^^^^^ூ corresponds to the projection of ^^ onto the column space of ^^. Using this, the inverse problem in (2) can be decomposed as follows:
[0027] Eqn. (7) can be solved using several methods, such as proximal gradient descent or quadratic relaxation. As a non-limiting example, using quadratic relaxation, Eqn. (7) can be solved according to: arg min ||^^ െ ^^^^^ு^^^ି^ ு ଶ ௭ ^^ ^^ െ ^^^^||ଶ ^ ^^^^^^ ^ ^^^ arg
subject to ^^ ൌ ^^^ ^ ^^, which is relaxed to arg
where ^^ is the learnable relaxation term. This can be solved using an alternating minimization scheme. For example, the first minimization can include the regularization (or the proximal operator) term, which solves arg m௨in
[0028] As a non-limiting example, Eqn. (8) may be solved implicitly using a neural network, other machine learning model-based approach, or a compressed sensing technique, such as soft-thresholding in a wavelet domain. On the other hand, the data consistency term can be recast such that the raw k-space data is not required. The data 7 QB\920171.00621\94202173.1
UMN 2024‐019 consistency minimization becomes:
where the projections cancel and simple data consistency is left. Note ^^^ ൌ ^^^ூ and plugging that in gives a subtitution of data consistency, ^^^^^௨௧^௨௧:
As an example, the data consistency optimization can be performed using conjugate gradient or other optimization methods. [0029] In this way, the data consistency unit can be recast in order to perform consistency on the orthogonal complement of the projection. Thus, only access to ^^^ூ is needed, which is available when magnitude and phase (or real and imaginary) parts of the DICOMs are exported. In other implementations, the online DICOMs do not include the complex (e.g., magnitude and phase or real and imaginary) information. In this case, the phase can be estimated as will be described further below. [0030] In some implementations, the MRI system can provide coil-specific reconstructed images. Thus, the inverse problem of Eqn. (2) can be solved for each coil, where ^^ is undetermined with fewer rows than columns. Thus ^^ has a non-trivial null space. In this case, the previously described method can be extended to use the output of the standard reconstruction on each coil image. In some cases, the individual coil images will be consistent with the acquired lines (e.g., GRAPPA will keep the acquired lines and generate missing lines by interpolating the acquired data). [0031] Here the decomposition is similar, but the starting approximation, ^^^, only needs to satisfy a very simple assumption of data consistency: ^^ ൌ ^^^^^. 8 QB\920171.00621\94202173.1
UMN 2024‐019 [0032] Following the steps as in Equation (6) gives ||^^ െ ^^^^||ଶ ^ ^^^^^^ ൌ ||^^ ଶ ଶ െ ^^^^^^ ^ ^^^||ଶ ^ ^^^^^^ ^ ^^^
[0033] since ^^ ൌ ^^^^^ . Following quadratic relaxation yields the same proximal term as in (8): argm௨in
[0034] and a new data consistency similar to (9):
െ ^^^^, (12) which leads to the same solution (but with an undetermined system). In some implementations, reconstructed multi-channel data (e.g., GRAPPA reconstructed data prior to coil combination) can be used for the starting approximation, ^^^. [0035] As a non-limiting example, solving the previously described iterative alternating minimization can be achieved using an unrolled network, as illustrated in FIGS.1A-1C.1A shows the iterative scheme of a reconstruction problem. FIG.1B shows an unrolled neural network architecture with each step including a regularization (“R”) unit and a data consistency (“DC”) unit. As an example, the regularization unit can be a trainable unit with convolutional neural networks to proxy the regularization update at the sub-problem in Eqn. (8). The standard data consistency unit can be substituted by ^^^^^௨௧^௨௧ , as previously described, to enforce the data consistency by solving the sub- problem in Eqn. (9). [0036] As a non-limiting example, the iterative optimization problem in Eqns. (8) and (9) can be unrolled for a selected number of iterations, such as T ^ 10 iterations. Conjugate gradient descent can be used in the DC units of the unrolled network. Similar 9 QB\920171.00621\94202173.1
UMN 2024‐019 to the examples described above, a ResNet structure can be used for the regularizer in Eqn. (8), and the network parameters can be shared across the unrolled network. [0037] FIG. 1C shows an example of a ResNet architecture containing convolutional layers and residual blocks (“RB”). Each residual block can include two convolutional layers with the first convolutional layer being followed by a rectified linear unit (“ReLU”) or other suitable activation layer, and the second convolutional later being followed by a constant multiplication layer. Although parameter sharing is shown in the network architecture in FIG. 1C, it will be appreciated that in other implementations parameter sharing may not be implemented. [0038] Referring now to FIG.2, a flowchart is illustrated as setting forth the steps of an example process 200 for training and implementing a machine learning algorithm that uses reconstructed images to enforce data consistency. The algorithm can be trained to reconstruct an image, which may be enhanced (e.g., reduced noise and/or artifacts) with respect to the originally reconstructed image (e.g., online reconstruction), which may serve as the input data. In this example implementation, supervised learning is used, in which images reconstructed from fully-sampled data are available. However, in other implementations, unsupervised or self-supervised training may be used. [0039] The method includes accessing reconstructed sub-sampled imaging data with a computer system, as indicated at step 202. Accessing the reconstructed images can include retrieving such data from an MRI system, a database, a memory, or another suitable data storage device or medium. In other instances, accessing the reconstructed images can include controlling an MRI system to acquire and reconstruct undersampled data. The reconstructed images may be generated from undersampled or subsampled k- space data by an online or other reconstruction. For example, the reconstruction may include parallel imaging, such as SENSE, GRAPPA, or a regularized parallel imaging 10 QB\920171.00621\94202173.1
UMN 2024‐019 reconstruction. The reconstruction may also include Fourier transforms, regridding, zero-padding, denoising, windowing, filtering, artifact correction, or other processing steps. [0040] The reconstructed imaging data may be coil-combined or include multi- channel or multi-coil data acquired from an array of receive radiofrequency (RF) coils. The reconstructed imaging data may include complex data, such as real and imaginary parts or magnitude and phase parts, or the imaging data may be real-valued. As one non- limiting example, the imaging data may be in a scanner- or picture archiving and communication system (PACS)-compatible file format, such as DICOM. In another non- limiting example, the data may be stored in a Neuroimaging Informatics Technology Initiative (NIfTI) format, or another file format based on the MRI system or other reconstruction software used. In this way, the process 200 is flexible, circumventing the need for inaccessible raw k-space data. [0041] The subsampled data may be acquired with a variety of undersampling schemes. Including uniform undersampling in one (e.g., Ry = 2, Ry = 3, Ry = 4, and so forth) or two dimensions (e.g., Ry = 2 and Rz = 2, Ry = 2 and Rz = 2, Ry = 3 and Rz = 2, and so forth), non-uniform undersampling, random under-sampling, pseudo-random undersampling (e.g., Poisson disk), variable density undersampling (e.g., variable density Poisson disk), or other sampling patterns. [0042] In some implementations, accessing image data in step 202 may include accessing or generating coil sensitivity maps. Coil sensitivity maps can be generated, for instance, using low-resolution multi-channel images. In some implementations, the coil sensitivity maps can be generated from autocalibration scan (ACS) data or a center of k- space (e.g., 24 x 24 samples), if available. In some implementations, the coil sensitivity maps can be generated using the multi-channel image data accessed in step 202. As a non- 11 QB\920171.00621\94202173.1
UMN 2024‐019 limiting example, coil sensitivity maps can be generated using the multi-channel data using ESPIRiT or other suitable techniques. As another non-limiting example, coil sensitivity maps can be generated when multi-channel data is not available using techniques, such as joint image reconstruction and sensitivity estimation in SENSE (J- SENSE) or regularized nonlinear inversion (NLINV). The coil sensitivity data can be fixed or constant throughout process 200 or may be updated iteratively, as will be described further below. [0043] Coil sensitivity data can optionally be iteratively updated throughout process 200. For example, if coil sensitivities in ^^ are unknown, they can be estimated. In this case:
[0044] where ^^^^^ are unknown coil sensitivities and ^^ஐ is a (known) undersampling Fourier operator sampling the locations specified in the index set Ω. Thus, the inerse problem can be written as: arg ^^^ு^^^ି^^^^ு^^^
[0045] Equation (14) includes additional unknowns for coil sensitivities. Equation (14) can be rewritten in some implementations using quadratic relaxation as
[0046] Equation (15) also includes a regularizer for coil sensitivities. Equation (15) can be solved with alternating updates. For example, ^^^^^ and ^^ can be fixed while ^^ 12 QB\920171.00621\94202173.1
UMN 2024‐019 is upated, ^^ and ^^^^^ can be fixed while ^^ is updated, and ^^ and ^^ can be fixed while ^^^^^ is updated. These alternating updates lead to:
[0047] As previously described, Equation (16) can be implicitly solved using a neural network. Equation (17) can be solved using a linear method, such as conjugate gradient, with learnable parameter ^^ . Equation (18) can be solved by noting that the definition of ^^ in (13) is separable across coils. Thus: ^^ ^^ା^^ ^ ൌ arg m ^in ||^^^ െ ^^ఆ diag^^^^ ^ ^^^^^^||ଶ ଶ ^ ^^^^^^^^^^^ (19) ೖ with ^^^ ൌ ^^^^^^^^^^^^^. Again, the regularization of Equation (19) can be implicitly learned via a neural network, as is similarly done using a J-SENSE approach. [0048] Equation (19) depends on y or yk, however it is desired to describe the system with respect to x0. Thus, noting that ^^^ ≜ ^^^ு^^^ି^^^^ு^^^ and multiplying both sides by ^^^ு^^^ି^^^^ு^ to solve for y leads to: diag^^^^ ^ ^^^^^^; … . ; ^^ఆ diag ^^
[0049] which can similarly be written as: ^^^^^ ൌ argm^in ||^^^^^ െ ^^^ఆ diag^^^^ ^ ^^^^^^; … . ;^^ఆ diag^^^^ ^ ^^^^^ ଶ ^^_^^^^^^||ଶ ^ ^^^^^^^^^^^^^ ೖ [0050] In this way, the current estimate for A can be used to synthesize undersampled data, and the current solution to z can be used to improve the estimate for coils. This is still separable across coils, as: diag^^^^ ^ ^^^^^^||ଶ ^ ^^^^^^^^^
13 QB\920171.00621\94202173.1
UMN 2024‐019 [0051] In some implementations, ^^^ can be randomly initialized. However, it may be preperable to initialize ^^^ to a known estimate. For example, ^^^ can be initialized based on sensitivity maps measured from a different subject using the same coil at a similar location. Other estimates of ^^^ can also be used for initialization. [0052] In some implementations, step 202 also includes generating or estimating phase maps from the reconstructed sub-sampled image data. For example, if complex data is unavailable, then the algorithm may only have access to |^^^ூ|, the magnitude of the reconstruction. In order to utilize Eq. 9, ^^^ூ can be written as ^^|^^^ூ |, where ^^ is a diagonal matrix containing the image phase. Due to the spatially smooth nature of the phase in the image domain, an additional regularizer can be placed on the entries of ^^. [0053] The ^^ matrix can be estimated from a calibration scan and held fixed or constant throughout process 200 or may be updated iteratively, as will be described further. For example, the inverse problem can be written as minimization over ^^, ^^, and ^^. Thus, if ^^ is known, after quadratic relaxation, the inverse problem can be arranged as: argm
[0054] Equation (20) can be solved by keeping ^^ and ^^ fixed while updating ^^ , keeping ^^ and ^^ fixed while updating ^^, and keeping ^^ and ^^ fixed while updating ^^. This leads to:
[0055] As previously described, y can be written as ^^^^^ , where the current 14 QB\920171.00621\94202173.1
UMN 2024‐019 estimate of A, which is estimated based on P(i), can be used to estimate P(i+1). Thus, equation (23) can be written as:
[0056] Equations (21) and (22) can be solved as previously described. Equation (23) can be solved implicitly using a neural network (e.g., to provide phase regularization). Equation (23) can also be solved using a simple regularization, such as Tikhonov regularization with projection onto the space of phase matrices. For example, to provide a projection onto the space of phase matrices, magnitude entries in Equation (23) can be set to equal 1. [0057] In some implementations, Equation (20) can be extended to include coil sensitivity estimations, as in Equation (15), for example. Using quadratic relaxation, the inverse problem can be written as:
^^ ^ ^^^^^^^^^^ ^ ^ୀ^ ^ (24) [0058] Equation (24) can again be solved by keeping three variables fixed while updating the other variables. For example, updating the image regularizer and overall data fidelity can be achieved as described above. Thus, updating the phase and coil sensitivity estimations can be described as: arg min||^^ െ ^^^^|^^ | െ ^^^^^^ା^^||ଶ ^ ^ ^^ା^^ ^^ା^^ ଶ ^,^^ ^ ଶ ^||^^ െ ^^^|^^^| ^ ^^ ^||ଶ ^ ^^^^^^^^^^^ ^ ೖ ^
[0059] Again, alternating minimization can be used to solve Equation (25), which 15 QB\920171.00621\94202173.1
UMN 2024‐019 may include variable splitting. [0060] The process 200 may optionally include accessing ground-truth data, as indicated in step 204. For example, the ground-truth data may include fully-sampled imaging data corresponding to the sub-sampled imaging data. The ground-truth data can optionally be used to train the machine learning algorithm, as indicated in step 206. However, in some implementations, ground truth data may not be necessary, as unsupervised or self-supervised learning may be implemented in step 206, for example. [0061] With or without ground-truth data, the machine learning algorithm is trained on the training data (e.g., reconstructed sub-sampled data or reconstructed sub- sampled data and ground-truth data), as indicated at step 206. Training the machine learning algorithm may include supervised learning, self-supervised learning, unsupervised learning, reinforcement learning, ensemble learning, active learning, transfer learning, or another suitable learning techniques. As one example, training the algorithm may also include transfer learning by incorporating knowledge of a previously trained network and adjusting the trained model for a desired application. [0062] Training the machine learning algorithm can include recasting a data consistency requirement that is based on the reconstructed data, as previously described. As a non-limiting example, data consistency may be enforced by ^^ ு ^^^௨௧^௨௧ ൌ ^^^ ^^ ^
. Training the machine learning algorithm can also include minimizing a regularization term. In this way, the machine learning algorithm utilizes a physics driven objective function that can advantageously be based on reconstructed data, rather than requiring raw data. The machine learning algorithm can then be trained in this way until an error, as calculated in block 208 according to a training loss function, is sufficiently reduced. [0063] As one example, the machine learning algorithm can be an artificial neural 16 QB\920171.00621\94202173.1
UMN 2024‐019 network, such as a convolutional neural network, a residual neural network, and so on. The machine learning algorithm may in some instances be an unrolled machine learning algorithm. As described above, training the machine learning algorithm can include incorporating the forward operator (e.g., the encoding matrix) into the training process. As described above, the machine learning algorithm can be trained on the training data using, in part, an objective function that iteratively calculates an error based on data consistency and a regularization term. [0064] As one example, training a neural network may include initializing the neural network, such as by computing, estimating, or otherwise selecting initial network parameters (e.g., weights, biases, or both). Training data can then be input to the initialized neural network, generating output as output data, which can include one or more reconstructed images. The quality of the output data can then be evaluated, such as by passing the output data to a loss function to compute an error. The current neural network can then be updated based on the calculated error (e.g., using backpropagation methods based on the calculated error). For instance, the current neural network can be updated by updating the network parameters (e.g., weights, biases, or both) in order to minimize the loss or cost according to the loss function. When the error has been minimized (e.g., by determining whether an error threshold or other stopping criterion has been satisfied), the current neural network and its associated network parameters represent the trained neural network. [0065] In some implementations, training the machine learning algorithm also includes iteratively updating the coil sensitivity data incorporated in the encoding matrix, ^^. For example, ^^ can be adjusted to update the objective function with each or some of the training iterations. As one non-limiting example, ^^ can be updated based on optimization methods, such as those used in J-SENSE. In some implementations, the 17 QB\920171.00621\94202173.1
UMN 2024‐019 phase map estimates incorporated in the measured data, ^^^ூ , can also be updated at each iteration or some subset of iterations, as described above. [0066] When training of the machine learning algorithm is completed, as determined at decision block 210, the trained machine learning algorithm is stored for later use, as indicated at step 212. In some instances, training can conclude after a stopping criterion has been satisfied. In some other instances, training can conclude after a preset number of iterations. Storing the neural network(s) may include storing network parameters (e.g., weights, biases, or both), which have been computed or otherwise estimated by training the neural network(s) on the training data. Storing the trained neural network(s) may also include storing the particular neural network architecture to be implemented. For instance, data pertaining to the layers in the neural network architecture (e.g., number of layers, type of layers, ordering of layers, connections between layers, hyperparameters for layers) may be stored. [0067] The trained machine learning algorithm can then be retrieved for use, such as to reconstruct images or in other linear inverse problem or nonlinear inverse problem applications. [0068] Referring now to FIG.3, a flowchart is illustrated as setting forth the steps of an example process 300 for reconstructing an image from reconstructed image data using a suitably trained neural network or other machine learning algorithm. As a non- limiting example, the process 300 an include accessing a pre-trained neural network and a reconstructed magnetic resonance image to reconstruct an enhanced magnetic resonance image. In this way, the resulting image produced by process 300 may have reduced noise, reduced aliasing artifacts, or otherwise improved quality with respect to the original or input reconstructed magnetic resonance image. [0069] The method includes accessing reconstructed image data with a computer 18 QB\920171.00621\94202173.1
UMN 2024‐019 system, as indicated at step 302. The reconstructed image data may include images produced by reconstructing undersampled k-space data. For example, the images may be generated by an MRI system using an online reconstruction. The reconstruction may include parallel imaging (e.g., SENSE or GRAPPA) to produce unaliased images. The reconstruction may further include other data processing steps, such as artifact correction, filtering, denoising, motion correction, windowing, zero-padding, and so on. Advantageously, the image data are undersampled data prior to being reconstructed. In general, undersample data refers to data sampled with a pattern that does not satisfy the Nyquist criterion. [0070] Accessing the image data may include retrieving such data from a memory or other suitable data storage device or medium. Alternatively, accessing the image data may include acquiring such data with a medical imaging system and transferring or otherwise communicating the data to the computer system, which may be a part of the medical imaging system. The image data may be stored in a standard format, such as DICOM, NIfTI, and so on. [0071] In some embodiments, the medical image data are images reconstructed from undersampled k-space data acquired with an MRI system. For instance, the k-space data can be undersampled in one or more dimensions by an acceleration factor of R ^ 2 , R ^ 4 , R ^ 6 , R ^ 8 , or other suitable acceleration factor. The k-space data can be uniformly undersampled, or non-uniformly undersampled. As one non-limiting example, the k-space data can be undersampled with an acceleration factor of R ^ 8 using a sheared uniform k y ^ k z undersampling pattern. Other undersampling patterns may also be used, such as variable density undersampling, Poisson disk undersampling, random undersampling, and so on. [0072] A trained neural network (or other suitable machine learning algorithm) is 19 QB\920171.00621\94202173.1
UMN 2024‐019 then accessed with the computer system, as indicated at step 304. In general, the neural network is trained, or has been trained, using the techniques described above in order to reconstruct images from reconstructed sub-sampling image data. [0073] Accessing the trained neural network may include accessing network parameters (e.g., weights, biases, or both) that have been optimized or otherwise estimated by training the neural network on training data. In some instances, retrieving the neural network can also include retrieving, constructing, or otherwise accessing the particular neural network architecture to be implemented. For instance, data pertaining to the layers in the neural network architecture (e.g., number of layers, type of layers, ordering of layers, connections between layers, hyperparameters for layers) may be retrieved, selected, constructed, or otherwise accessed. [0074] In some implementations, the pre-trained neural network, or other machine learning algorithm, can be fine-tuned in a scan-specific manner using transfer learning. All of the layers in the pre-trained network can be fine-tuned, or alternatively only a subset of the layers can be fine-tuned (e.g., only higher-level portions/earlier layers of the pre-trained network can be fine-tuned). [0075] A determination is thus made at decision block 306 whether the pre- trained neural network or other machine learning algorithm should be fine-tuned. If so, then, as indicated at step 308, the pre-trained neural network or other machine learning algorithm parameters are fine-tuned in a scan-specific (i.e., subject-specific) manner using the reconstructed sub-sampled image data accessed at step 302. Advantageously, fine-tuning the pre-trained network in this manner can further improve reconstruction performance. Based on the techniques described in the present disclosure, a pre-trained network can be fine-tuned by defining an objective function with respect to the reconstructed images. The network parameters can be initialized with the database- 20 QB\920171.00621\94202173.1
UMN 2024‐019 trained network values. These parameters are then fine-tuned, such that the fine tuning of the network is performed on a per-scan or scan-specific basis. [0076] The medical image data are then input to the neural network or other machine learning algorithm, whether fine-tuned or otherwise accessed, generating output as one or more reconstructed images, as indicated at step 310. The image(s) generated by inputting the medical image data to the trained neural network(s) can then be displayed to a user, stored for later use or further processing, or both, as indicated at step 312. [0077] Referring particularly now to Fig.4, an example of an MRI system 400 that can be used to generate or access data in accordance with the present disclosure is illustrated. For example, the MRI system 400 may be configured to acquire undersampled or subsampled imaging data and reconstruct the undersampled data. For example, the reconstruction may include SENSE, GRAPPA, or another linear or non-linear reconstruction. The reconstruction may also include other data processing steps. As a non-limiting example, the MRI system 400 is configured to provide imaging data in the form of DICOM images. [0078] The MRI system 400 includes an operator workstation 402 that may include a display 404, one or more input devices 406 (e.g., a keyboard, a mouse), and a processor 408. The processor 408 may include a commercially available programmable machine running a commercially available operating system. The operator workstation 402 provides an operator interface that facilitates entering scan parameters into the MRI system 400. The operator workstation 402 may be coupled to different servers, including, for example, a pulse sequence server 410, a data acquisition server 412, a data processing server 414, and a data store server 416. The operator workstation 402 and the servers 410, 412, 414, and 416 may be connected via a communication system 440, which may 21 QB\920171.00621\94202173.1
UMN 2024‐019 include wired or wireless network connections. [0079] The MRI system 400 also includes a magnet assembly 424 that includes a polarizing magnet 426, which may be a low-field magnet. The MRI system 400 may optionally include a whole-body RF coil 428 and a gradient system 418 that controls a gradient coil assembly 422. [0080] The pulse sequence server 410 functions in response to instructions provided by the operator workstation 402 to operate a gradient system 418 and a radiofrequency (“RF”) system 420. Gradient waveforms for performing a prescribed scan are produced and applied to the gradient system 418, which then excited gradient coils in an assembly 422 to produce the magnetic field gradients (e.g., ^^௫, ^^௬, and ^^௭) that can be used for spatially encoding magnetic resonance signals. The gradient coil assembly 422 forms part of a magnet assembly 424 that includes a polarizing magnet 426 and a whole-body RF coil 428. [0081] RF waveforms are applied by the RF system 420 to the RF coil 428, or a separate local coil to perform the prescribed magnetic resonance pulse sequence. Responsive magnetic resonance signals detected by the RF coil 428, or a separate local coil, are received by the RF system 420. The responsive magnetic resonance signals may be amplified, demodulated, filtered, and digitized under direction of commands produced by the pulse sequence server 410. The RF system 420 includes an RF transmitter for producing a wide variety of RF pulses used in MRI pulse sequences. The RF transmitter is responsive to the prescribed scan and direction from the pulse sequence server 410 to produce RF pulses of the desired frequency, phase, and pulse amplitude waveform. The generated RF pulses may be applied to the whole-body RF coil 428 or to one or more local coils or coil arrays. [0082] The RF system 420 also includes one or more RF receiver channels. An RF 22 QB\920171.00621\94202173.1
UMN 2024‐019 receiver channel includes an RF preamplifier that amplifies the magnetic resonance signal received by the coil 428 to which it is connected, and a detector that detects and digitizes the ^^ and ^^ quadrature components of the received magnetic resonance signal. The magnitude of the received magnetic resonance signal may, therefore, be determined at a sampled point by the square root of the sum of the squares of the ^^ and ^^ components: ^^ ൌ ^^^^ଶ ^ ^^ଶ^ [0083] and the phase of the received magnetic resonance signal may also be determined according to the following relationship:
[0084] The pulse sequence server 410 may receive patient data from a physiological acquisition controller 430. By way of example, the physiological acquisition controller 430 may receive signals from a number of different sensors connected to the patient, including electrocardiograph (“ECG”) signals from electrodes, or respiratory signals from a respiratory bellows or other respiratory monitoring devices. These signals may be used by the pulse sequence server 410 to synchronize, or “gate,” the performance of the scan with the subject’s heartbeat or respiration. [0085] The pulse sequence server 410 may also connect to a scan room interface circuit 432 that receives signals from various sensors associated with the condition of the patient and the magnet system. Through the scan room interface circuit 432, a patient positioning system 434 can receive commands to move the patient to desired positions during the scan. [0086] The digitized magnetic resonance signal samples produced by the RF system 420 are received by the data acquisition server 412. The data acquisition server 23 QB\920171.00621\94202173.1
UMN 2024‐019 412 operates in response to instructions downloaded from the operator workstation 402 to receive the real-time magnetic resonance data and provide buffer storage, so that data are not lost by data overrun. In some scans, the data acquisition server 412 passes the acquired magnetic resonance data to the data processor server 414. In scans that require information derived from acquired magnetic resonance data to control the further performance of the scan, the data acquisition server 412 may be programmed to produce such information and convey it to the pulse sequence server 410. For example, during pre-scans, magnetic resonance data may be acquired and used to calibrate the pulse sequence performed by the pulse sequence server 410. As another example, navigator signals may be acquired and used to adjust the operating parameters of the RF system 420 or the gradient system 418, or to control the view order in which k-space is sampled. In still another example, the data acquisition server 412 may also process magnetic resonance signals used to detect the arrival of a contrast agent in a magnetic resonance angiography (“MRA”) scan. For example, the data acquisition server 412 may acquire magnetic resonance data and processes it in real-time to produce information that is used to control the scan. [0087] The data processing server 414 receives magnetic resonance data from the data acquisition server 412 and processes the magnetic resonance data in accordance with instructions provided by the operator workstation 402. Such processing may include, for example, reconstructing two-dimensional or three-dimensional images by performing a Fourier transformation of raw k-space data, performing parallel imaging reconstruction, performing other image reconstruction algorithms (e.g., iterative or backprojection reconstruction algorithms), applying filters to raw k-space data or to reconstructed images, generating functional magnetic resonance images, or calculating motion or flow images. 24 QB\920171.00621\94202173.1
UMN 2024‐019 [0088] Images reconstructed by the data processing server 414 are conveyed back to the operator workstation 402 for storage. Real-time images may be stored in a data base memory cache, from which they may be output to operator display 402 or a display 436. Batch mode images or selected real time images may be stored in a host database on disc storage 438. When such images have been reconstructed and transferred to storage, the data processing server 414 may notify the data store server 416 on the operator workstation 402. The operator workstation 402 may be used by an operator to archive the images, produce films, or send the images via a network to other facilities. [0089] The MRI system 400 may also include one or more networked workstations 442. For example, a networked workstation 442 may include a display 444, one or more input devices 446 (e.g., a keyboard, a mouse), and a processor 448. The networked workstation 442 may be located within the same facility as the operator workstation 402, or in a different facility, such as a different healthcare institution or clinic. [0090] The networked workstation 442 may gain remote access to the data processing server 414 or data store server 416 via the communication system 440. Accordingly, multiple networked workstations 442 may have access to the data processing server 414 and the data store server 416. In this manner, magnetic resonance data, reconstructed images, or other data may be exchanged between the data processing server 414 or the data store server 416 and the networked workstations 442, such that the data or images may be remotely processed by a networked workstation 442. [0091] Referring now to FIG. 5, an example of a system 500 for training and implementing a machine learning algorithm to reconstruct an image in accordance with some embodiments of the systems and methods described in the present disclosure is shown. As shown in FIG.5, a computing device 550 can receive one or more types of data 25 QB\920171.00621\94202173.1
UMN 2024‐019 (e.g., reconstructed image data, ground truth data, k-space data, other image data, or sub- sampled data) from data source 502, which may be a medical image data source. In some embodiments, computing device 550 can execute at least a portion of a training and image reconstruction system 504 to train and implement a machine learning algorithm to reconstruct an image from data received from the data source 502. [0092] Additionally or alternatively, in some embodiments, the computing device 550 can communicate information about data received from the data source 502 to a server 552 over a communication network 554, which can execute at least a portion of the training and image reconstruction system 504. In such embodiments, the server 552 can return information to the computing device 550 (and/or any other suitable computing device) indicative of an output of the training and image reconstruction system 504. [0093] In some embodiments, computing device 550 and/or server 552 can be any suitable computing device or combination of devices, such as a desktop computer, a laptop computer, a smartphone, a tablet computer, a wearable computer, a server computer, a virtual machine being executed by a physical computing device, and so on. The computing device 550 and/or server 552 can also reconstruct images from the data. [0094] In some embodiments, data source 502 can be any suitable source of image data (e.g., measurement data, images reconstructed from measurement data), such as an MRI system, PACS system, stored image database, another computing device (e.g., a server storing image data), and so on. In some embodiments, data source 502 can be local to computing device 550. For example, data source 502 can be incorporated with computing device 550 (e.g., computing device 550 can be configured as part of a device for capturing, scanning, and/or storing images). As another example, data source 502 can be connected to computing device 550 by a cable, a direct wireless link, and so on. 26 QB\920171.00621\94202173.1
UMN 2024‐019 Additionally or alternatively, in some embodiments, data source 502 can be located locally and/or remotely from computing device 550 and can communicate data to computing device 550 (and/or server 552) via a communication network (e.g., communication network 554). [0095] In some embodiments, communication network 554 can be any suitable communication network or combination of communication networks. For example, communication network 554 can include a Wi-Fi network (which can include one or more wireless routers, one or more switches, etc.), a peer-to-peer network (e.g., a Bluetooth network), a cellular network (e.g., a 3G network, a 4G network, etc., complying with any suitable standard, such as CDMA, GSM, LTE, LTE Advanced, WiMAX, etc.), a wired network, and so on. In some embodiments, communication network 554 can be a local area network, a wide area network, a public network (e.g., the Internet), a private or semi- private network (e.g., a corporate or university intranet), any other suitable type of network, or any suitable combination of networks. Communications links shown in FIG. 5 can each be any suitable communications link or combination of communications links, such as wired links, fiber optic links, Wi-Fi links, Bluetooth links, cellular links, and so on. [0096] Referring now to FIG.6, an example of hardware 600 that can be used to implement data source 502, computing device 550, and server 552 in accordance with some embodiments of the systems and methods described in the present disclosure is shown. As shown in FIG. 6, in some embodiments, computing device 550 can include a processor 602, a display 604, one or more inputs 606, one or more communication systems 608, and/or memory 610. In some embodiments, processor 602 can be any suitable hardware processor or combination of processors, such as a central processing unit (“CPU”), a graphics processing unit (“GPU”), and so on. In some embodiments, display 604 can include any suitable display devices, such as a computer monitor, a touchscreen, 27 QB\920171.00621\94202173.1
UMN 2024‐019 a television, and so on. In some embodiments, inputs 606 can include any suitable input devices and/or sensors that can be used to receive user input, such as a keyboard, a mouse, a touchscreen, a microphone, and so on. [0097] In some embodiments, communications systems 608 can include any suitable hardware, firmware, and/or software for communicating information over communication network 554 and/or any other suitable communication networks. For example, communications systems 608 can include one or more transceivers, one or more communication chips and/or chip sets, and so on. In a more particular example, communications systems 608 can include hardware, firmware and/or software that can be used to establish a Wi-Fi connection, a Bluetooth connection, a cellular connection, an Ethernet connection, and so on. [0098] In some embodiments, memory 610 can include any suitable storage device or devices that can be used to store instructions, values, data, or the like, that can be used, for example, by processor 602 to present content using display 604, to communicate with server 552 via communications system(s) 608, and so on. Memory 610 can include any suitable volatile memory, non-volatile memory, storage, or any suitable combination thereof. For example, memory 610 can include RAM, ROM, EEPROM, one or more flash drives, one or more hard disks, one or more solid state drives, one or more optical drives, and so on. In some embodiments, memory 610 can have encoded thereon, or otherwise stored therein, a computer program for controlling operation of computing device 550. In such embodiments, processor 602 can execute at least a portion of the computer program to present content (e.g., images, user interfaces, graphics, tables), receive content from server 552, transmit information to server 552, and so on. [0099] In some embodiments, server 552 can include a processor 612, a display 28 QB\920171.00621\94202173.1
UMN 2024‐019 614, one or more inputs 616, one or more communications systems 618, and/or memory 620. In some embodiments, processor 612 can be any suitable hardware processor or combination of processors, such as a CPU, a GPU, and so on. In some embodiments, display 614 can include any suitable display devices, such as a computer monitor, a touchscreen, a television, and so on. In some embodiments, inputs 616 can include any suitable input devices and/or sensors that can be used to receive user input, such as a keyboard, a mouse, a touchscreen, a microphone, and so on. [00100] In some embodiments, communications systems 618 can include any suitable hardware, firmware, and/or software for communicating information over communication network 554 and/or any other suitable communication networks. For example, communications systems 618 can include one or more transceivers, one or more communication chips and/or chip sets, and so on. In a more particular example, communications systems 618 can include hardware, firmware and/or software that can be used to establish a Wi-Fi connection, a Bluetooth connection, a cellular connection, an Ethernet connection, and so on. [00101] In some embodiments, memory 620 can include any suitable storage device or devices that can be used to store instructions, values, data, or the like, that can be used, for example, by processor 612 to present content using display 614, to communicate with one or more computing devices 550, and so on. Memory 620 can include any suitable volatile memory, non-volatile memory, storage, or any suitable combination thereof. For example, memory 620 can include RAM, ROM, EEPROM, one or more flash drives, one or more hard disks, one or more solid state drives, one or more optical drives, and so on. In some embodiments, memory 620 can have encoded thereon a server program for controlling operation of server 552. In such embodiments, processor 612 can execute at least a portion of the server program to transmit 29 QB\920171.00621\94202173.1
UMN 2024‐019 information and/or content (e.g., data, images, a user interface) to one or more computing devices 550, receive information and/or content from one or more computing devices 550, receive instructions from one or more devices (e.g., a personal computer, a laptop computer, a tablet computer, a smartphone), and so on. [00102] In some embodiments, data source 502 can include a processor 622, one or more data acquisition systems 624, one or more communications systems 626, and/or memory 628. In some embodiments, processor 622 can be any suitable hardware processor or combination of processors, such as a CPU, a GPU, and so on. In some embodiments, the one or more data acquisition systems 624 are generally configured to acquire data, images, or both, and can include an MRI system, another medical imaging system, and so on. Additionally or alternatively, in some embodiments, one or more data acquisition systems 624 can include any suitable hardware, firmware, and/or software for coupling to and/or controlling operations of an MRI system, another medical imaging system, or so on. In some embodiments, one or more portions of the one or more data acquisition systems 624 can be removable and/or replaceable. [00103] Note that, although not shown, data source 502 can include any suitable inputs and/or outputs. For example, data source 502 can include input devices and/or sensors that can be used to receive user input, such as a keyboard, a mouse, a touchscreen, a microphone, a trackpad, a trackball, and so on. As another example, data source 502 can include any suitable display devices, such as a computer monitor, a touchscreen, a television, etc., one or more speakers, and so on. [00104] In some embodiments, communications systems 626 can include any suitable hardware, firmware, and/or software for communicating information to computing device 550 (and, in some embodiments, over communication network 554 and/or any other suitable communication networks). For example, communications 30 QB\920171.00621\94202173.1
UMN 2024‐019 systems 626 can include one or more transceivers, one or more communication chips and/or chip sets, and so on. In a more particular example, communications systems 626 can include hardware, firmware and/or software that can be used to establish a wired connection using any suitable port and/or communication standard (e.g., VGA, DVI video, USB, RS-232, etc.), Wi-Fi connection, a Bluetooth connection, a cellular connection, an Ethernet connection, and so on. [00105] In some embodiments, memory 628 can include any suitable storage device or devices that can be used to store instructions, values, data, or the like, that can be used, for example, by processor 622 to control the one or more data acquisition systems 624, and/or receive data from the one or more data acquisition systems 624; to images from data; present content (e.g., images, a user interface) using a display; communicate with one or more computing devices 550; and so on. Memory 628 can include any suitable volatile memory, non-volatile memory, storage, or any suitable combination thereof. For example, memory 628 can include RAM, ROM, EEPROM, one or more flash drives, one or more hard disks, one or more solid state drives, one or more optical drives, and so on. In some embodiments, memory 628 can have encoded thereon, or otherwise stored therein, a program for controlling operation of data source 502. In such embodiments, processor 622 can execute at least a portion of the program to generate images, transmit information and/or content (e.g., data, images) to one or more computing devices 550, receive information and/or content from one or more computing devices 550, receive instructions from one or more devices (e.g., a personal computer, a laptop computer, a tablet computer, a smartphone, etc.), and so on. [00106] In some embodiments, any suitable computer readable media can be used for storing instructions for performing the functions and/or processes described herein. For example, in some embodiments, computer readable media can be transitory or non- 31 QB\920171.00621\94202173.1
UMN 2024‐019 transitory. For example, non-transitory computer readable media can include media such as magnetic media (e.g., hard disks, floppy disks), optical media (e.g., compact discs, digital video discs, Blu-ray discs), semiconductor media (e.g., random access memory (“RAM”), flash memory, electrically programmable read only memory (“EPROM”), electrically erasable programmable read only memory (“EEPROM”)), any suitable media that is not fleeting or devoid of any semblance of permanence during transmission, and/or any suitable tangible media. As another example, transitory computer readable media can include signals on networks, in wires, conductors, optical fibers, circuits, or any suitable media that is fleeting and devoid of any semblance of permanence during transmission, and/or any suitable intangible media. [00107] The present disclosure has described one or more preferred embodiments, and it should be appreciated that many equivalents, alternatives, variations, and modifications, aside from those expressly stated, are possible and within the scope of the invention. 32 QB\920171.00621\94202173.1