WO2022075954A1

WO2022075954A1 - A calibration method having a super-resolution for magnetic particle imaging

Info

Publication number: WO2022075954A1
Application number: PCT/TR2021/051016
Authority: WO
Inventors: Alper GÜNGÖR
Original assignee: Aselsan Elektroni̇k Sanayi̇ Ve Ti̇caret Anoni̇m Şi̇rketi̇
Priority date: 2020-10-06
Filing date: 2021-10-05
Publication date: 2022-04-14
Also published as: TR202015881A2

Abstract

The invention relates to a calibration method having a super-resolution that allows imaging with a size that is smaller than a size of the magnetic nanoparticle by using calibration measurements along with the signal processing techniques for magnetic particle imaging. In the proposed method, a large MNP sample is used. However, the mentioned sample is moved at smaller steps compared to the size of the MNP (for example, at half its size). Then, the measurements from the half-distance moved samples are deconvolved to reconstruct a smaller sized system matrix. Hence, using the proposed technique, it is possible to calibrate to smaller sizes compared to the size of the point magnetic nanoparticle.

Description

A CALIBRATION METHOD HAVING A SUPER-RESOLUTION FOR MAGNETIC PARTICLE IMAGING

Technical Field

The invention is related to a calibration method having a super-resolution that allows imaging with a dimension which is smaller than the dimension of a magnetic particle for magnetic particle imaging..

State of the Art

Two different methods are used as standard for image reconstruction in the magnetic particle imaging method. The first one is the system calibration method, in which a small volume nanoparticle sample is scanned mechanically at the desired system resolution steps in the field of view to obtain the calibration data of the system (US8355771 B2, J. Weizenecker et al. 2009 Phys. Med. Biol. 54 L1 ). Images are generated using this calibration data (which is also called the system matrix).

In the mentioned standard system calibration method, the calibration measurements last very long since the sample nanoparticle must be mechanically scanned and measured at every grid point in the field of view (US 8355771 B2, J. Weizenecker et al. 2009 Phys. Med. Biol. 54 L1 ). The mechanical scanning time from one point to the other and acquisition of the measurement data takes about 1.3 sec (A von Gladiss et aL, 2017 Phys. Med. Biol., Vol.62 pp.3392-3406). For a small field of view having 30 x 30 x 30 grid points, the calibration time lasts 9.75 hours. In clinical practice, the calibration of a larger imaging volume may last for months. There is a need to calibrate the system frequently, since nanoparticle characteristics are known to vary from batch to batch and is also affected by the imaging sequence. For this reason, standard system calibration method cannot be practically adopted for systems with large field of view. In addition, since the nanoparticle to be scanned must be smaller than the voxel size of the image, the number of nanoparticles in the scanned sample is limited and the signal-to-noise ratio is small. A method to increase the signal-to-noise ratio is multiple data acquisition at the same position, and averaging. For this reason, the mechanical motion cannot be continuous, and the scanner should be stopped at every grid point, and moved to the next point after taking enough measurements to reach to the desired signal noise level. This limits the speed of the calibration measurements.

A calibration method has been proposed, in which the nanoparticle sample is scanned at random positions much fewer than the total number of voxels in the field of view. This is possible since the system matrix is sparse in certain transform domains (discrete Fourier, cosine, or Chebychev) (US2015221 103A1 ). It has been shown that this method can reduce the number of scanned points by 80-90%. In that technique, instead of taking measurements from all of the voxels (N) in the field of view, system calibration can also be done by making reduced number of measurements at random M (<N) voxel positions using compressed sensing techniques. Since it is not possible to calculate how small M should be analytically, the M / N ratio should be chosen according to the image quality. Experimental images were obtained in the above mentioned reference. While image quality was acceptable for M / N = 0.1 , it was significantly degraded for lower M / N ratios. With this method, the calibration time can be reduced by a factor of 10, but very long calibration times are still needed since the sample is mechanical scanned. For example, a measurement area of 200 x 200 x 200 points will take longer than 10 days to measure.

As in EP314399A1 , the second reconstruction method is the X-space approach. In this method, there is no calibration step; images are generated using the signal equation model for the magnetic particles imaging. Image reconstruction is done in the time domain by using the MPI signal equation. In this method, deviations from the ideal of MPI hardware are not taken into account and the resolution is lower than the system calibration method.

Apart from these methods, there is also a hybrid method, in which the system matrix is generated by measuring the nanoparticle in a separate calibration unit outside the system (A von Gladiss et al., 2017 Phys. Med. Biol., Vol.62 pp.3392-3406). In this method, the magnetic field to be applied to the nanoparticle is electronically generated in a separate hardware. The system calibration matrix is obtained by relating the measurements made with the actual system with the calibration unit measurements. Thus, electronic scanning can be performed instead of mechanical scanning, and calibration can be performed in a shorter time. They propose a fast calibration procedure. However, the use of a separate calibration unit is required, and the magnetic field distribution of the MPI system must be separately measured by scanning a magnetic field sensor at each voxel in the imaging field.

Finally, recently our group have proposed coded calibration scenes (llbey, et. aL, 2019 IEEE Trans, on Med. Im.). We proposed employing compressive sensing for reconstruction of the underlying system matrix. Instead of a single point source being moved around the scene, we proposed using calibration scenes that consist of a filled shape, moving / rotating it and scanning data. Then, we proposed solving a compressive sensing reconstruction problem using the known MNP distribution corresponding to each measurement. They use heavily undersampled data, and use prior information for reconstruction. Since the underlying MNP distribution is known, it is up to the user to adjust Field of View / resolution. However, in that method, selection of Field of View (FoV) and resolution is a big issue. Because, if too large FoV is selected, then the requirements for compressive sensing (i.e. such as restricted isometry property) will not be satisfied, and the reconstruction will fail. Moreover, if a smaller FoV is selected, then MNPs outside the FoV will degrade sensitivity reconstruction. A similar problem arises in the resolution selection. If resolution is chosen as too small, then the coherence between neighboring elements will be too high, and compressive sensing reconstruction will fail. The achievable FoV / resolution is currently unknown for that method. Moreover, for large images, the method requires solving equations involving very large matrices.

In conclusion, due to downsides of each technique for the problem at hand, a new technique is proposed.

Brief Description of the Invention

The present invention proposes a calibration method for an magnetic particle imaging (MPI) in which signal processing techniques is used along with calibration measurements. In the proposed method, a large point source is used for calibration. However, said point source is moved by a size smaller than that of the point source, for example moved by a half point size. Then, the results received from the measurements performed by moving with a half point size are passed through a deconvolution step and a small-sized system matrix reconstruction can be performed. By this way, it would be possible to perform calibration with a size much more smaller than the point MNPs which are used for calibration measurements. The proposed method is a resolution development method for the system calibration matrix.

Moreover, in the methods used in the state of the art it is required that calibration particle must be kept stationary during the calibration process. However, the present invention allows performing calibration in a moving system. Furthermore, it is possible to perform reconstruction even if some measurements in some points are missing.

In previously proposed calibration methods (US8355771 B2, J. Weizenecker et al. 2009 Phys. Med. Biol. 54 L1 , US patent no. 2015/0221 103 A1 ), the nanoparticle sample is usually mechanically scanned, and for this reason the resolution of the system calibration matrix is limited to the size of the nanoparticle. However, with the proposed method, one can scan the scene electronically or mechanically with sub-point-source shifts (for example half of the nanoparticle specimen or quarter of the nanoparticle specimen) and gather information about high resolution components. Thus, it is possible to gather information from the system calibration matrix in the lower order of nanoparticle dimension.

In the state of the art, US patent no. 2015/0221 103 A1 document, a small single nanoparticle sample is mechanically scanned for all the voxels in the field of view one by one for MPI system calibration. This proposition is simply a data reduction module for the previous invention (US 8355771 B2, J. Weizenecker et al. 2009 Phys. Med. Biol. 54 L1 , US patent no. 2015/0221 103 A1 ). It still suffers from the same resolution limiting factors of the previous method and the highest achievable resolution is the size of the point-source-size. Moreover, they propose solving a compressive sensing based reconstruction problem for the reconstruction of the system matrix. In the present invention, it is proposed to solve a deconvolution-like reconstruction problem for system matrix reconstruction with high resolution features.

In the proposed method there are plurality of nanoparticle samples. For this reason, received signal level is higher in proportion to the number of employed nanoparticle samples. As a result of this it is enough to take a single measurement at a point. This allows performing measurements while the scanning scene is continuously moving. The continuity of the mechanical motion increases the speed of the calibration process. In addition to this, since the measurements are performed at the different points simultaneously the information content has increased, thus it is possible to obtain system calibration matrix with a small number of measurements. This provides a great advantage for a large sized imaging fields. The US patent no. US2015221103A1 discloses the scanning of an non-homogenous sample whereas said non-homogenous sample is scanned in a same manner with the homogenous one.

In the method proposed by A von Gladiss et al. 2017 Phys. Med. Biol. vol.62 pp.3392- 3406, a separate calibration unit is used for nanoparticle characterization and magnetic field in the field of views should be measured separately. However in the present invention, all the effects related to magnetic field (magnetic field inhomogeneities, nanoparticle response) are taken into account in a single calibration scan.

In the method disclosed in llbey, et. al., 2019 IEEE Trans, on Med. Im., selection of resolution and Field of View is left to the user. However, in the present invention, thanks to the vast literature in the deconvolution area, achievable resolution / field of view is predetermined. Moreover, since deconvolution is used, a large number of algorithms that exploit fast computation methods for reconstruction can be employed. Furthermore, better algorithms which can be used for deconvolution problems but not suitable for compressive sensing can be employed in the presente invention to better reconstruct system matrices.

Structural and characteristic features of the invention and all its advantages can be clearly understood by the help of the attached figures and following detailed description in which the figures are referenced.

Description of the Figures

Figure 1 shows distribution of magnetic nanoparticles in a volume filled with magnetic nanoparticles using a dual-zone non-homogeneous first magnetic field.

Figure 2 shows a filled magnetic nanoparticle sample in a hypothetically gridded field of view.

Figure 3 shows Fourier domain representations for various microscanning superresolution factors.

Figures need not be in the proper scale and unnecessary details may be omitted to help better understanding of the invention. Description of the References

1 . Magnetic particle imaging system

2. First magnetic field

3. First zone

4. Second zone

5. Second magnetic field

6. Field of view

7. Grid

8. Magnetic nanoparticle filled sample

9. Mechanical means

Detailed Description of the Invention

In this detailed description, preferred embodiments of the invention are given only to help better explanation of the invention and does not put any extra restrictions on the invention.

Magnetic nanoparticles can be used for various purposes in medical field such as angiography, stem cell tracking, imaging of cancerous cells by using the cancerous- focusedly developed nanoparticles and also these magnetic nanoparticles can be imaged externally by using the Non-lnvasive Magnetic Particle Imaging (MPI) method.

In an magnetic particle imaging system (1 ), the distribution of magnetic nanoparticles in a volume filled with magnetic nanoparticles is imaged by using a non-homogeneous first magnetic field (2) having two zones (Gleich B, Weizenecker J. ; 435 (7046)). A first zone (3) has a very low magnetic field intensity and is called a magnetic field free region (FFR). The magnetic nanoparticles in the FFR can be magnetized in the direction of a second magnetic field (5) which is externally applied. In a second zone (4), the magnetic field intensity is high. Since the magnetic nanoparticles in this second zone (4) are in saturation, they do not respond to the second magnetic field (5) or respond at the lowest level. The second magnetic field (5) is applied to the entire field of view (6) as a time varying magnetic field. The time-dependent magnetization of the magnetic nanoparticles in the FFR is measured by a receiving coil(s). The amplitude of the measured signal is directly proportional to the number of nanoparticles in the FFR. The FFR is scanned electronically or mechanically throughout the field of view (6) to obtain the nanoparticle distribution in the field of view (6). Since the magnetic nanoparticles have a non-linear magnetization curve, the received signal from the particles in the FFR contains the harmonics of the frequency of the transmitted signal. The received signal properties depend on both the nanoparticle characteristics (size, shape, material, etc.) and solution properties (viscosity, temperature), and the magnetic field properties of the imaging system (1 ). In MPI, best image quality is achieved with the image reconstruction method based on the system calibration method, which takes all these effects into account (Knopp T, Rahmer J, Sattel TF, et aL, Weighted iterative reconstruction for magnetic particle imaging. 2010; 55: 1577-1589).

In the system calibration-based image reconstruction, firstly the entire field of view (6) is hypothetically divided into small grids (7) by certain intervals. A system matrix is formed by using a sample filled with a magnetic nanoparticle (8) having an equal or lower size of the grid (7). For this, the sample containing the nanoparticles (8) is scanned to every grid position by means of a mechanical means (9) such as a scanner. The sample containing nanoparticles (8) is stopped at every grid position. Secondary magnetic field signal is applied, and the nanoparticle signal received by the receiving coils is stored in a digital storage unit (e.g. hard disk) In practice, the measurement data are acquired multiple times at the same grid point, and the signal to noise ratio is increased by averaging the measurements data. The measured signal at a single grid position is converted to the frequency domain by using the Fourier transform, forming a column of the system matrix (A). The whole system matrix is generated by taking measurements at all grid positions. This process is called the calibration step.

For imaging, the signal received from the object to be imaged is reconstructed by using the system matrix and converted into an image. For this, a linear equation set Ax = b is solved. In this equation set, A is the system matrix, b is the measurement taken from the object, and x is the nanoparticle distribution inside the object. The major disadvantage of the system matrix calibration method is its long duration (-the number of grid positions times 1.3 seconds) (A von Gladiss et aL, 2017 Phys. Med. 62 pp.3392-3406) due to the requirement of mechanical positioning of the sample containing nanoparticles (8) for every grid position. In addition, since the sample size of the nanoparticle is very small, the signal level is low and it is necessary to increase the signal-to-noise ratio by taking multiple measurements. This prevents continuous mechanical scanning, leading to the prolongation of the measurement period.

Two problems of the prior art: low signal to noise ratio (SNR) and low resolution of single small-voxel measurements. To solve these problems of the prior art, the present invention proposes the use of microscanning calibration. Microscanning calibration is the use of sub-pixel shifts with a deconvolution procedure in order to obtain calibration matrices having high resolution & SNR. In this method, samples containing large-sized magnetic nanoparticles (8) are used and these samples (8) are moved with sizes smaller than that of the sample (8) size either mechanically or electronically. This achieves multiple times more SNR with respect to using small-sized magnetic nanoparticles. However, these measurements have lower resolution when conventional methods are used. However, instead of using conventional procedures and techniques, it is possible to obtain high SNR and resolution by using some reconstruction methods which can also be used in microscanning imaging techniques.

Microscanning imaging has previously been studied for optical imaging to visualize images having resolution higher than that of detector. However, it has never been previously established in the context of magnetic particle imaging for such a calibration procedure. In the context of the microscanning imaging, a normal image is firstly taken. Then, the detector has been shifted sub-pixelly in both x and y directions and new measurements are taken. These measurements are then stitched together and combined. After that, an optional deconvolution step has been performed and images having high resolution have been formed. However, most of the time the underlying structure of the image is unknown, and deconvolution mostly results in noise amplification in the image. Moreover, each of these snapshots are taken in a consecutive manner, and the object of interest may be moving in time domain. Hence, these snapshots may not match well together. Moreover, some of the measurements can be skipped in order to reduce the calibration time.. However, the skipped measurement data has a clear structure, which is not very suitable for compressed sensing type approaches. Microscanning calibration differs from microscanning imaging and compressed sensing techniques in this manner. Problem model, used reconstruction techniques and undersampling procedures will be discussed in detail in the next paragraphs. An exemplary embodiment of the present invention is as follows: a calibration sample of size “D” is first filled with magnetic nanoparticles. Then, a scan is performed. Next, a robot (mechanical) or electronic scanner moves the particle to next sub-pixel shift (say with a distance of D/2, D/3, D/4 ... etc). The scanning operation continues until the entire field of view has been scanned. Finally, a deconvolution problem is solved in order to remove the low-resolution effect originating from the large pixel size. Let us call superresolution factor in x-direction s_x, y-direction s_y and z-direction s_z. Let h denotes a matrix filled with “1 ” with a size of “s_x x s_y x s_z". One can extend or decrease the number of dimensions arbitrarily in the context of present invention. For the first measurement, we write “h” to the beginning of the image matrix “x”, and denote as “x_h”. Then, the first measurement can be expressed as: y = Ax_h + n, where y is the measurement vector, A is the calibration matrix (or system matrix) and n is the noise vector. Then, we move h inside “x”. This operation can be expressed with a convolution operator over each column of the matrix, A. Hence, for each column i in the calibration matrix expressed as A_t, we get the following equality: yi = (Ai * /i) + n_z.

This is a convolutional forward model. Deconvolution is a well-studied method for lots of different problems. Here, numerous number of deconvolution methods can be used for reconstruction of A_t, from the measurements y_t. Let us discuss a brief model for deconvolution as an example. The convolution operation with h can be expressed in Fourier transformation domain with multiplication. The overall forward model would become: y_t = SU^HHUAi + nt, where H represents a Fourier transform of the filter h, U represents a multi-dimensional Fourier transform and S represents a masking operator to avoid the effect of circular convolution that comes with circular Fourier transform [1], Then, one can solve an optimization problem:

Here, 1/1/ denotes a sparsity transformation that transforms the system matrix into a sparse domain. y_t is a measurement converted to Fourier space for each measurement position, £_p represents an error parameter depends on the error caused by the system noise originating from measurement. Different algorithms in the literature can be used to solve Eq. [1] (e.g. Fast Iterative Shrinkage Thresholding Algorithm (FISTA), Alternating Direction Method of Multipliers (ADMM) [2, 3, 4]). In addition to this, one may also use dictionary learning based reconstruction to learn from data and reconstruct simultaneously [3]. Moreover, one may also use deep learning based deconvolution methods [5].

Here, the Fourier transform of the filter is the main limiting factor of the achievable resolution. Fourier domain representations of the various super-resolution factors are given in Fig. 3 for microscanning. White color represents high sensitivity in Fourier domain, and black represents low sensitivity. As can be seen, 2x2 super-resolution results in almost no-zeros in Fourier domain, which corresponds to no loss of information having high-resolution. Hence, it follows that one can immediately achieve the resolution of a calibration sample of D/2 size by a calibration sample of D size. For a higher superresolution imaging such as 3x3 or 4x4, it may be required to use above-mentioned nonlinear deconvolution methods. In addition to that, if a different and known shape is used instead of a point-like MNP filled object, (such as L, O or U shaped point-shaped objects) the method still works and more advantageous super-resolution factors compared to Figure 3 can be achieved. In this case, the above-mentioned filter “H”, should be updated to reflect the shape of the MNP sample.

It can also be shown that, continuous movement does not change the convolutional model. The methods existing in the prior art cannot gather calibration data continuously while moving the calibration system. Let us now revisit the problem model. Here the calculation model is linear and measurement is gathered in time-domain. Let us denote the received in time-domain with A_t, and received data with y. Let us also set x_ftjl to the received data at first position. Also, let us assume that there is a continuous and linear motion while receiving data. In this case, there occurs a motion blur which can be expressed as a convolutional operation on each of the received data. . Let us denote this motion filter with m. Although not necessary, for the sake of simplicity let us assume that the point-source is moved from point x₀ to x₀ + D/2 in a time period of a single scan. Here, the first row of the calibration matrix A would be multiplied by the original signal, x_hil to generate the first time domain measurement, y . The second row would be multiplied by a bit shifted version of the original signal x_{h l}, which can be represented with the motion filter. Hence, for re-visiting the original system model, the first acquisition would become: y = Ax_{h l} + n,

Fourier domain and multiply A with 3D spatial Fourier transformation matrix. Then,

where A_{t k l} represents the spatial Fourier transformation of each row of A_t, and M represents the diagonal matrix with entries of the Fourier transformation of the motion filter. Then, the overall model for the first continuous set of data becomes:

Y = A[^xh,i - ^,N] + /V.

Here, we constructed x_hii = xt * h, where x represents mathematically the high- resolution point-source at position i (i.e. the first calibration object is a large filter with the first position as the large voxel). During the calibration process, we are trying to figure out the overall response of the system for each high-resolution point source, Xj.

Separating x_h>i, we observe that:

These equation set shows that calibration can be performed by solving the deconvolution problem under the continuous motion. Here, one can use two different approaches. Either overall system matrix can be used independently in time-domain or first the effect of the motion is corrected and then reconstruction can be performed in frequencydomain. Let us first assume that time-domain approach is used:

Here, if full data is sampled, concatenated x vectors constitute identity matrix. For a single time-point row, the overall model becomes:

If some of the measurement points are skipped, then concatenated x vectors constitute a row selection type sampling matrix, S. Again, this is still a deconvolution problem that can be solved using compressive sensing, deep learning or deconvolution type approaches. One can solve the deconvolution with the filter (7i * m * ••• * m), for each step. Here, let us note that even if motion is not linear, i.e. filter associated with each time point i does not correspond to M^1-1, one can still solve the same problem with separate filters

Hence, it is shown that system matrix reconstruction problem, i.e. the calibration process can be cast as a deconvolution problem. This allows employing a vast number of algorithms for system matrix reconstruction. Moreover, achievable resolution can be determined according to the highest frequency in Fourier space. These advantages separate the proposed method from the other methods existing in the prior art. Moreover, one can achieve super-resolution (i.e. resolution beyond the point-source) using this method.

Alternatively, one can use the frequency domain based approach. For this purpose, it is assumed that there is no continuous motion in the image, i.e. M is the identity matrix.

Taking Fourier transformation of each side in time domain, we get:

Then, a calibration problem such as the following one may be solved in the frequency domain:

In this document, a microscanning calibration technique for magnetic particle imaging has been discussed. The key advantages of microscanning calibration, compared to other calibration methods currently present for MPI have been shown. Next, the key differences of microscanning calibration from microscanning imaging have been discussed. Finally, the applications of the microscanning calibration in time and frequency domains have been explained in detail. It is also explained that how the timedomain approach may be used with motion. Furthermore, it is explained that microscanning calibration may be used in conjunction with compressed sensing or deep learning based approaches when some of the measurement points are skipped. References

[1 ] A. Matakos, S. Ramani, and J. A. Fessler, “Accelerated Edge-Preserving Image Restoration Without Boundary Artifacts,” IEEE Trans, on Image Proc., vol. 22, no. 5, pp. 2019 - 2029, 2013. [2] A. Gungor and H. E. Guven, “Feature-enhanced computational infrared imaging,” in

Computational Imaging II. International Society for Optics and Photonics, 2017, vol. 10222, p. 1022204.

[3] A. Gungor and O. F. Kar, “A Transform Learning Based Deconvolution Technique with Super-Resolution and Microscanning Applications,” in IEEE International Conf. On Image Processing (ICIP), 2019, Taiwan.

[4] L. B. Lucy, “An iterative technique for the rectification of observed distributions,” Astronomical Journal. 79 (6): p. 745 - 754.

[5] L. Xu, J. S. J. Ren, C. Liu, J. Jia, “Deep convolutional neural network for image deconvolution,” Proceedings of the 27th International Conference on Neural Information Processing Systems, Dec. 2014, p. 1790 - 1798.

Claims

CLAIMS A calibration method that allows imaging with a lower resolution than a size of a magnetic particle sample employed in calibration for magnetic particle imaging wherein, the method comprises the steps of;

• receiving images of the multiple magnetic nanoparticle samples within the field of view,

• shifting the magnetic nanoparticle samples to sub-pixel positions by moving them by a distance that is smaller than the size of said magnetic nanoparticles and remeasuring samples,

• Interlacing the shifted measurements by placing each measurement side-by- side after repeated shifting process. The calibration method according to claim 1 wherein, it comprises a step of forming images having a high resolution by performing deconvolution operation on the interlaced measurements in order to eliminate the low resolution effect caused by the size of the magnetic particle samples employed in the calibration. The calibration method according to claim 1 wherein, said shifting process can be done by using a mechanical or electronic scanner. The calibration method according to claim 1 wherein, the signal level of the received measurements can be improved by increasing the number of magnetic nanoparticle sample. The calibration method according to claim 1 wherein, the measurements can be taken during the movement of the calibration sample.