JP2021041065A - Mri device, image processing device and image processing method - Google Patents

Abstract

To provide an image processing method and the like which can create a tomographic image having the excellent image quality on the basis of partial k space data collected for a partial space of a k space in an MRI device.SOLUTION: In a CNN processing step S12, a first tomographic image x1 created in a first tomographic image creation step S11 on the basis of partial k space data y0 collected for a partial space of a k space or a second tomographic image x2 created in a second tomographic image creation step S15 is input to a CNN, and a CNN output tomographic image xCNN is output from the CNN. The processing in the CNN processing step S12 is performed more than two times. In the first processing of the CNN processing step S12, the first tomographic image x1 is input to the CNN. In the second or subsequent processing of the CNN processing step S12, the second tomographic image x2 created on the basis of the CNN output tomographic image xCNN by the previous processing is input to the CNN.SELECTED DRAWING: Figure 2

Description

The present invention relates to an MRI apparatus, an image processing apparatus and an image processing method.

An MRI (magnetic resonance imaging) device can image information inside a subject by utilizing a nuclear magnetic resonance phenomenon. The MRI apparatus causes a resonance phenomenon in hydrogen atoms in the subject by applying a high-frequency magnetic field, collects radio waves generated by the resonance phenomenon with a receiving coil, and based on the collected data, a tomographic image of the subject. To create. The MRI apparatus is suitably used for diagnosing a part such as a brain or a blood vessel having a large amount of water.

The MRI apparatus encodes the spin phase as a signal source in the gradient direction of the gradient magnetic field applied to the subject. The amount of the gradient magnetic field applied in each direction in the real space is directly the position in the k-space, which is the space of the spatial frequency of the tomographic image. Therefore, a tomographic image of the subject can be obtained by performing an inverse Fourier transform on the k-space data collected by the receiving coil by applying a gradient magnetic field. The Fourier transform and the inverse Fourier transform are substantially the same in performing numerical operations.

In the MRI apparatus, it is required to shorten the time required for collecting k-space data in order to reduce the burden on the subject and improve the inspection throughput. Compressed sensing is attracting attention as one of the technologies for speeding up k-space data collection. Compressed sensing utilizes the fact that observation data has the property of being sparse in a certain expression space, and based on observation data that is less than the amount of data originally required, an object is targeted under certain conditions. How to restore. Non-Patent Documents 1 to 3 describe a method of applying compressed sensing to an MRI apparatus to create a tomographic image based on k-space data. Among them, Non-Patent Document 3 proposes a method for improving the image quality of a tomographic image.

In the method described in Non-Patent Document 3, a first tomographic image is created by inverse Fourier transforming the partial k-space data collected for a part of the k-space, and the first tomographic image has been learned. It is input to a convolutional neural network (CNN), and a CNN output tomographic image is output from the CNN. This CNN output tomographic image is Fourier transformed to create the first k spatial data. The second k-space data is created based on the partial k-space data for the partial space and the k-space data for the space other than the partial space among the first k-space data. Then, the second k spatial data is inverse Fourier transformed to create a second tomographic image.

In this method, since the k-space data needs to be collected only for a part of the k-space, not for the entire k-space, the time required for collecting the k-space data can be shortened. Further, the second tomographic image finally obtained is said to have better image quality than the first tomographic image and the CNN output tomographic image.

Daiki Tamada, "Implementation of Compressed Sensing in MRI", Journal of Japanese Society of Magnetic Resonance Medicine 38.3 (2018): 76-86. Yoshio Machida, Issei Mori, "Progress of MRI High-Speed Imaging-From Imaging Principles to Compressed Sensing-", Journal of the Medical Imaging Information Society 30.1 (2013): 7-11. Hyun, Chang Min, et al. "Deep learning for undersampled MRI reconstruction," Physics inMedicine & Biology 63.13 (2018): 135007. Biomedical Image Analysis Group, ImperialCollege London, “IXI Dataset”, [online], [Search on August 20, 1st year of Reiwa], Internet <URL: http://brain-development.org/ixi-dataset/> Ronneberger O, Fischer P and BroxT 2015 U-net: convolutional networks for biomedical image segmentation (arXiv: 1505.04597).

However, periodic artifacts can be seen in the tomographic image finally obtained by the method described in Non-Patent Document 3. Further improvement in image quality is desired in the method of applying compressed sensing to an MRI apparatus to create a tomographic image based on partial k-space data collected for a part of k-space.

The present invention has been made to solve the above problems, and the image quality is further improved based on the partial k-space data collected for a part of the k-space by applying compressed sensing to the MRI apparatus. It is an object of the present invention to provide an image processing apparatus and an image processing method capable of producing a tomographic image having the above. Another object of the present invention is to provide an MRI apparatus including such an image processing apparatus.

The image processing apparatus of the present invention includes (1) a first tomographic image creating unit that creates a first tomographic image based on partial k-space data collected for a part of the k-space in the MRI apparatus, and (2). ) CNN processing unit that inputs a tomographic image to the convolutional neural network and outputs a CNN output tomographic image from the convolutional neural network, and (3) 1k spatial data creation unit that creates 1k spatial data based on the CNN output tomographic image. And (4) the second k space for creating the second k space data based on the partial k space data for the partial space and the k space data for the space other than the partial space among the first k space data. It includes a data creation unit and (5) a second tomographic image creation unit that creates a second tomographic image based on the second k spatial data. Then, the CNN processing unit performs the processing by the convolutional neural network twice or more, in the first processing, the first tomographic image is input to the convolutional neural network, and in the second and subsequent processing, the previous processing is performed. The second tomographic image created based on the CNN output tomographic image is input to the convolutional neural network.

When the CNN processing unit trains the convolutional neural network, (a) the first tomographic image or the second tomographic image of the subject created based on the partial k-space data collected for the partial space in the k-space. The image is used as an input image to the convolutional neural network, and (b) a tomographic image of the subject created based on the k-space data collected for a space wider than the partial space in the k-space is used as a teacher image. Is preferable. Further, it is more preferable that the CNN processing unit uses a tomographic image of the subject created based on the k-space data collected for the entire k-space as a teacher image when training the convolutional neural network.

When training the convolutional neural network, the CNN processing unit preferably uses a function including the difference between the teacher image and the CNN output tomographic image as a loss function. Further, when the convolutional neural network is trained, the CNN processing unit Fourier transforms the difference between the teacher image and the CNN output tomographic image, and the k-space data obtained by Fourier transforming the teacher image and the CNN output tomographic image. It is more preferable to use a function including the difference from the obtained k-space data as a loss function.

Alternatively, the image processing apparatus of the present invention includes (1) a first tomographic image creating unit that creates a first tomographic image based on partial k-space data collected for a part of the k-space in the MRI apparatus. (2) CNN processing unit that inputs the first tomographic image to the convolutional neural network and outputs the CNN output tomographic image from the convolutional neural network, and (3) the first k that creates the first k spatial data based on the CNN output tomographic image. The second k space data is created based on the spatial data creation unit, (4) the partial k space data for the partial space, and the k space data for the space other than the partial space among the first k space data. It is provided with a second k spatial data creation unit for creating a second tomographic image, and (5) a second tomographic image creation unit for creating a second tomographic image based on the second k spatial data. Then, when the CNN processing unit trains the convolutional neural network, (a) the first tomographic image or the first tomographic image of the subject created based on the partial k-space data collected for the partial space in the k-space. 2 The tomographic image is used as an input image to the convolutional neural network, and (b) the tomographic image of the subject created based on the k-space data collected for a space wider than the partial space in the k-space is used as the teacher image. Including (c) the difference between the teacher image and the CNN output tomographic image, and the difference between the k-spatial data obtained by Fourier transforming the teacher image and the k-spatial data obtained by Fourier transforming the CNN output tomographic image. Use the function as a loss function.

The MRI apparatus of the present invention includes a k-space data collecting unit that collects k-space data, and the above-mentioned image processing apparatus of the present invention that creates a tomographic image based on the k-space data collected by the k-space data collecting unit. To be equipped.

The image processing method of the present invention includes (1) a first tomographic image creation step of creating a first tomographic image based on partial k-space data collected for a part of k-space in an MRI apparatus, and (2). ) CNN processing step to input tomographic image to convolutional neural network and output CNN output tomographic image from convolutional neural network, and (3) 1k spatial data creation step to create 1k spatial data based on CNN output tomographic image And (4) the second k space for creating the second k space data based on the partial k space data for the partial space and the k space data for the space other than the partial space among the first k space data. It includes a data creation step and (5) a second tomographic image creation step of creating a second tomographic image based on the second k spatial data. Then, the processing of the CNN processing step is performed twice or more, and in the first processing, the first tomographic image is input to the convolutional neural network, and in the second and subsequent processing, the CNN output tomographic image obtained by the previous processing is used. The second tomographic image created based on this is input to the convolutional neural network.

In the CNN processing step, when training the convolutional neural network, (a) a first tomographic image or a second tomographic image of the subject created based on the partial k-space data collected for the partial k-space. The image is used as an input image to the convolutional neural network, and (b) a tomographic image of the subject created based on the k-space data collected for a space wider than the partial space in the k-space is used as a teacher image. Is preferable. Further, in the CNN processing step, it is more preferable to use a tomographic image of the subject created based on the k-space data collected for the entire k-space as a teacher image when training the convolutional neural network.

In the CNN processing step, it is preferable to use a function including the difference between the teacher image and the CNN output tomographic image as a loss function when training the convolutional neural network. Further, in the CNN processing step, when the convolutional neural network is trained, the difference between the teacher image and the CNN output tomographic image, and the k-space data obtained by Fourier transforming the teacher image and the CNN output tomographic image are Fourier transformed. It is more preferable to use a function including the difference from the obtained k-space data as a loss function.

Alternatively, the image processing method of the present invention includes (1) a first tomographic image creation step of creating a first tomographic image based on partial k-space data collected for a part of the k-space in the MRI apparatus. (2) CNN processing step of inputting the first tomographic image to the convolutional neural network and outputting the CNN output tomographic image from the convolutional neural network, and (3) the first k to create the first k spatial data based on the CNN output tomographic image. The second k space data is created based on the spatial data creation step, (4) the partial k space data for the partial space, and the k space data for the space other than the partial space among the first k space data. It includes a second k space data creation step for creating a second tomographic image, and (5) a second tomographic image creation step for creating a second tomographic image based on the second k spatial data. Then, in the CNN processing step, when the convolutional neural network is trained, (a) the first tomographic image or the first tomographic image of the subject created based on the partial k-space data collected for the partial space in the k-space. 2 The tomographic image is used as an input image to the convolutional neural network, and (b) the tomographic image of the subject created based on the k-space data collected for a space wider than the partial space in the k-space is used as the teacher image. Including (c) the difference between the teacher image and the CNN output tomographic image, and the difference between the k-spatial data obtained by Fourier transforming the teacher image and the k-spatial data obtained by Fourier transforming the CNN output tomographic image. Use the function as a loss function.

According to the present invention, compressed sensing can be applied to an MRI apparatus to create a tomographic image having better image quality based on partial k-space data collected for a part of k-space.

FIG. 1 is a block diagram showing the configuration of the MRI apparatus 1. FIG. 2 is a flowchart illustrating an image processing method. FIG. 3 is a diagram illustrating the processing of the second k space data creation step S14 performed by the second k space data creation unit 14. FIG. 4 is an example of a correct image. FIG. 5 is a diagram showing an undersampling pattern. Figure 6 is a diagram showing a first example of a tomographic image x ₁ generated based on partial k-space data y ₀ in the first tomographic image producing step S11. FIG. 7 is a diagram showing an example of _{a CNN output tomographic image x CNN} obtained by performing the processing of the CNN processing step S12 only once. FIG. 8 is a diagram showing an example of _{the second tomographic image x 2} obtained by performing the processing of the CNN processing step S12 only once. FIG. 9 is a diagram showing an example of _{the second tomographic image x 2} obtained by performing the processing of the CNN processing step S12 10 times. FIG. 10 is a diagram showing an example of _{a CNN output tomographic image x CNN} obtained by performing the processing of the CNN processing step S12 only once. FIG. 11 is a diagram showing an example of _{the second tomographic image x 2} obtained by performing the processing of the CNN processing step S12 only once. FIG. 12 is a diagram showing an example of _{the second tomographic image x 2} obtained by performing the processing of the CNN processing step S12 10 times. FIG. 13 is a diagram showing regions of interest R1 and R2 in the correct image (FIG. 4). FIG. 14 is a diagram showing regions of interest R1 and R2 in the CNN output tomographic image x _{CNN (FIG. 7).} FIG. 15 is a diagram showing regions of interest R1 and R2 in the second tomographic image x _{2 (FIG. 8).} FIG. 16 is a diagram showing regions of interest R1 and R2 in the second tomographic image x _{2 (FIG. 9).} FIG. 17 is a diagram showing regions of interest R1 and R2 in the CNN output tomographic image x _{CNN (FIG. 10).} FIG. 18 is a diagram showing regions of interest R1 and R2 in the second tomographic image x _{2 (FIG. 11).} FIG. 19 is a diagram showing regions of interest R1 and R2 in the second tomographic image x _{2 (FIG. 12).} Figure 20, _L A as a loss function when CNN learning, the case of using the respective _{L B,} a graph showing a first tomographic image _{x 1} and the second tomographic image _{x 2} each PSNR. 21, _L A as a loss function when CNN learning, the case of using the respective _{L B,} a graph showing a first tomographic image _{x 1} and the second tomographic image _{x 2,} respectively SSIM.

Hereinafter, embodiments for carrying out the present invention will be described in detail with reference to the accompanying drawings. In the description of the drawings, the same elements are designated by the same reference numerals, and duplicate description will be omitted. The present invention is not limited to these examples, and is indicated by the scope of claims, and is intended to include all modifications within the meaning and scope equivalent to the scope of claims.

FIG. 1 is a block diagram showing the configuration of the MRI apparatus 1. The MRI apparatus 1 includes a k-space data collection unit 10 and an image processing apparatus 2.

The k-space data collection unit 10 applies a gradient magnetic field to the subject and receives radio waves generated by the resonance phenomenon of hydrogen atoms in the subject by the receiving coil to collect k-space data. The k-space data collection unit 10 can collect k-space data (full sampling) for the entire k-space, and can collect k-space data (undersampling) for a part of the k-space. it can. As a pattern of a part of the k-space at the time of undersampling, a lattice pattern (cartesian), a radial pattern, a spiral pattern, and the like are known (see Non-Patent Documents 1 and 2).

The image processing device 2 creates a tomographic image of the subject based on the k-space data collected by the k-space data collection unit 10. The image processing device 2 includes a first tomographic image creation unit 11, a CNN processing unit 12, a first k spatial data creation unit 13, a second k spatial data creation unit 14, a second tomographic image creation unit 15, a control unit 16, and a storage unit 17. And a display unit 18.

The first tomographic image creating unit 11 creates a tomographic image of the subject based on the k-space data collected (full sampling or undersampling) by the k-space data collecting unit 10. Specifically, the first tomographic image creating unit 11 creates a tomographic image by performing an inverse Fourier transform on the k-space data. a tomographic image generated by the first tomographic image creating part 11 and the first tomographic image x ₁ based on the partial k-space data y ₀ collected for a part of the area of the k-space.

_{The CNN processing unit 12 convolves the first tomographic image x 1} created by the first tomographic image creation unit 11 _{or the second tomographic image x 2} created by the second tomographic image creation unit 15 into a convolutional neural network (CNN). ). As a result, the CNN processing unit 12 outputs the CNN output tomographic image x _CNN from the CNN.

The first k spatial data creation unit 13 creates the first k spatial data y _{1 based on the CNN output tomographic image x CNN} . Specifically, the first k spatial data creation unit 13 creates the first k spatial data y ₁ by _{Fourier transforming the CNN output tomographic image x CNN.}

The second k space data creation unit 14 is based on the partial k space data y ₀ for the partial space and the k space data for the space other than the partial space in the _first k space data y 1, and the second k. Spatial data y ₂ is created.

The second tomographic image creation unit 15 creates a second tomographic image x _{2 based on the second k spatial data y 2} . Specifically, the second tomographic image creating unit 15 creates the second tomographic image x ₂ by _{performing an inverse Fourier transform on the second k spatial data y 2.}

The control unit 16 controls the overall operation of the image processing device 2, and in particular, the first tomographic image creation unit 11, the CNN processing unit 12, the first k space data creation unit 13, and the second k space data creation unit 14. And the processing of each of the second tomographic image creating unit 15 is controlled. The storage unit 17 stores partial k spatial data y ₀ , first tomographic image x ₁ , CNN output tomographic image x _CNN , first k spatial data y ₁ , second k spatial data y _2, second tomographic image x _{2, and the} like. .. The display unit 18 displays partial k spatial data y ₀ , first tomographic image x ₁ , CNN output tomographic image x _CNN , first k spatial data y ₁ , second k spatial data y _2, and second tomographic image x ₂ on the screen. Display on.

The image processing device 2 includes a CPU (central processing unit), a DSP (digital signal processor), a ROM (read only memory), a GPU (graphics processing unit), a RAM (random access memory), a hard disk drive, a display, a keyboard, a mouse, and the like. It can be configured by a computer having. The first tomographic image creation unit 11, the first k spatial data creation unit 13, the second k spatial data creation unit 14, and the second tomographic image creation unit 15 may be configured to include a CPU or a DSP. The CNN processing unit 12 may be configured to include a CPU, DSP or GPU. The control unit 16 may be configured to include a CPU. The storage unit 17 may include a ROM, a RAM, and a hard disk drive. The display unit 18 may be configured to include a display.

The CNN processing unit 12 performs the processing by CNN twice or more. Them in Part 1 of the process, CNN processing unit 12 to input the first tomographic image _{x 1} to CNN. In the second and subsequent processes, the CNN processing unit 12 causes the CNN to input the second tomographic image x _{2 created based on the CNN output tomographic image x CNN obtained by the previous process.}

FIG. 2 is a flowchart illustrating an image processing method. This figure also shows a k-space data collection step S10 in which the k-space data collection unit 10 collects partial k-space data y _0. The image processing method includes a first tomographic image creation step S11, a CNN processing step S12, a first k spatial data creation step S13, a second k spatial data creation step S14, and a second tomographic image creation step S15.

The first tomographic image creation step S11 is a process performed by the first tomographic image creation unit 11. In the first tomographic image creation step S11, the first tomographic image x ₁ is created _{based on the partial k-space data y 0} collected for a part of the k-space in the k-space data collection step S10.

The CNN processing step S12 is a process performed by the CNN processing unit 12. In the CNN processing step S12, the CNN is made to input the first tomographic image x _{1 created in the first tomographic image creation step S11 or the second tomographic image x 2} created in the second tomographic image creation step S15. As a result, in the CNN processing step S12, the CNN output tomographic image x _CNN is output from the CNN.

The first k space data creation step S13 is a process performed by the first k space data creation unit 13. In the first k spatial data creation step S13, the first k spatial data y ₁ is created _{based on the CNN output tomographic image x CNN.}

The second k space data creation step S14 is a process performed by the second k space data creation unit 14. In the second k space data creation step S14, the second k is based on the partial k space data y ₀ for the partial space and the k space data for the space other than the partial space in the _first k space data y 1. Spatial data y ₂ is created.

The second tomographic image creation step S15 is a process performed by the second tomographic image creation unit 15. In the second tomographic image creation step S15, the second tomographic image x ₂ is created _{based on the second k spatial data y 2.}

The process of CNN process step S12 is performed twice or more. Them in Part 1 of the process, CNN processing step S12, to input the first tomographic image _{x 1} to CNN. In the second and subsequent processes, the CNN process step S12 causes the CNN to input the second tomographic image x _{2 created based on the CNN output tomographic image x CNN obtained by the previous process.}

FIG. 3 is a diagram illustrating the processing of the second k space data creation step S14 performed by the second k space data creation unit 14. In this figure, the partial k-space data y ₀ , the first k-space data y ₁ and the second k-space data y ₂ are schematically shown. Partial k-space data y ₀ shown in FIG. 3 (a), has been collected (undersampling) for some of the k-space-space (the hatched region in FIG. 3 (a)), that part of the area The data in the space other than the space (white region in FIG. 3A) has a value of 0.

The 1k spatial data y ₁ shown in FIG. 3 (c), has been prepared by Fourier transforming CNN output tomographic image x _CNN, it may take a value other than 0 in the entire k-space. _{The second k-space data y 2} shown in FIG. 3 (b) is created by the second k-space data creation step S14, and has the same value as the _{partial k-space data y 0 for the partial space.} Spaces other than the above partial space have the same values as the _{first k-space data y1.}

The image processing apparatus and the image processing method of the present embodiment are different from the methods described in Non-Patent Document 3 in the following points. The method described in Non-Patent Document 3 includes 1 for each of the first tomographic image creation step S11, the CNN processing step S12, the first k spatial data creation step S13, the second k spatial data creation step S14, and the second tomographic image creation step S15. It is performed only once, and the CNN processing step S12 is not repeated. On the other hand, in the present embodiment described so far, the CNN processing step S12 by the CNN processing unit 12 is performed twice or more. Thus, in this embodiment, it is possible to create a tomographic image having a good image quality based on the partial k-space data y ₀ collected for a part of the area of the k-space.

Next, learning of CNN will be described. _{When training the CNN, the CNN processing unit 12 may use the first tomographic image x 1} or the first tomographic image x 1 of the subject created based on _{the partial k-space data y 0} collected (undersampled) for a partial space in the k-space. The second tomographic image x ₂ is used as an input image to the CNN. Further, the CNN processing unit 12 uses a tomographic image of the subject created based on the k-space data collected for a space wider than the partial space as the teacher image x _true . Preferably, the CNN processing unit 12 uses a tomographic image of the subject created based on the k-space data collected (full sampling) for the entire k-space as the teacher image x _true .

The loss function used in learning the CNN may be a function including the difference between _{the teacher image x true} and the CNN output tomographic image x _{CNN (Equation (1) below).} The equation _{L A} represents the L2 norm of the teacher image x _true and CNN output tomographic image x _CNN. CNN training is performed so that the loss function becomes small.

The loss function is _{the difference between the teacher image x true} and the CNN output tomographic image x _CNN, and the k spatial data F (x _true ) obtained by Fourier transforming the teacher image x _true and the CNN output tomographic image x _CNN . It may be a function (Equation (2) below) including a difference from the k-space data F (x _{CNN) obtained by Fourier transform.} The equation L _B multiplies the L2 norm of the teacher image x _true and CNN output tomographic image x _CNN, the L2 norm of the k-space data F (x _true) and k-space data F (x _CNN) a weighting factor λ Represents the sum of data and. The value of the weighting coefficient λ is preferably selected in the range of more than 0 and not more than 1.

If caused to learn CNN with loss function _L A ((1) type), the effect of image quality improvement is obtained by the number of repetitions of the CNN processing step S12 by the CNN processing unit 12 twice or more. The maximum number of repetitions of the CNN processing step S12 by the CNN processing unit 12 is about 10 times. The final tomographic image with improved image quality may be either _{the CNN output tomographic image x CNN} or the second tomographic image x ₂ after the CNN processing step S12 by the CNN processing unit 12 is performed twice or more.

If caused to learn CNN with loss function _L B ((2) type), not only when the CNN processing step S12 by the CNN processing unit 12 performs two or more times, the CNN processing step S12 by the CNN processing unit 12 1 It is possible to create a _{second tomographic image x 2} having good image quality based on _{the partial k-space data y 0} collected for a part of the k-space even when it is performed only once.

Next, the simulation results will be described. Among the tomographic images of the brain by MRI accumulated in Non-Patent Document 4, T2-weighted images were used. CNN was trained using 7427 tomographic images for 57 people. Using 7460 pieces of tomographic images of 57 persons, we have created a partial k-space data y ₀ by the under subsampling. The pattern of undersampling was a grid pattern (cartesian), and the space was set to 20% of the total k-space at the time of full sampling. As CNN, U-net described in Non-Patent Document 5 was used. CNN was used during learning in the loss function _{L A} (equation (1)) and _{L B} (above (2)). When using the L _B as a loss function, and the weighting factor λ is 0.1.

FIG. 4 is an example of a correct image. The rectangular areas R1 and R2 in the figure are areas of interest. FIG. 5 is a diagram showing an undersampling pattern. In the figure, the white region is an undersampled partial space of the k space, and the data of the space of the black region other than the partial space has a value of 0. _{Partial k-space data y 0} is created by extracting only the data in the area corresponding to the white area of the undersampling pattern (Fig. 5) from the k-space data obtained by Fourier transforming the correct image (Fig. 4). did.

Figure 6 is a diagram showing a first example of a tomographic image x ₁ generated based on partial k-space data y ₀ in the first tomographic image producing step S11. FIG. 6 (a) shows the first tomographic image x ₁ , and FIG. 6 (b) _{shows the difference between the first tomographic image x 1} and the correct image (FIG. 4).

7 to 9 are diagrams showing an example of a tomographic image created when _LA is used as a loss function during CNN learning. FIG. 7 is a diagram showing an example of _{a CNN output tomographic image x CNN} obtained by performing the processing of the CNN processing step S12 only once. 7 (a) shows the CNN output tomographic image x _CNN, FIG. 7 (b) shows the difference between the CNN output tomographic image x _CNN and correct image (FIG. 4). FIG. 8 is a diagram showing an example of _{the second tomographic image x 2} obtained by performing the processing of the CNN processing step S12 only once. FIG. 8 (a) shows the second tomographic image x ₂ , and FIG. 8 (b) _{shows the difference between the second tomographic image x 2} and the correct image (FIG. 4). FIG. 9 is a diagram showing an example of _{the second tomographic image x 2} obtained by performing the processing of the CNN processing step S12 10 times. 9 (a) shows the second tomographic image x ₂ , and FIG. 9 (b) _{shows the difference between the second tomographic image x 2} and the correct image (FIG. 4).

10 to 12 are diagrams showing an example of a tomographic image created when _LB is used as a loss function during CNN learning. FIG. 10 is a diagram showing an example of _{a CNN output tomographic image x CNN} obtained by performing the processing of the CNN processing step S12 only once. 10 (a) shows the CNN output tomographic image x _CNN, FIG. 10 (b) shows the difference between the CNN output tomographic image x _CNN and correct image (FIG. 4). FIG. 11 is a diagram showing an example of _{the second tomographic image x 2} obtained by performing the processing of the CNN processing step S12 only once. FIG. 11 (a) shows the second tomographic image x ₂ , and FIG. 11 (b) _{shows the difference between the second tomographic image x 2} and the correct image (FIG. 4). FIG. 12 is a diagram showing an example of _{the second tomographic image x 2} obtained by performing the processing of the CNN processing step S12 10 times. FIG. 12 (a) shows the second tomographic image x ₂ , and FIG. 12 (b) _{shows the difference between the second tomographic image x 2} and the correct image (FIG. 4).

13 to 19 are enlarged views showing the regions of interest R1 and R2 in each tomographic image. Each figure (a) shows the area of interest R1 and each figure (b) shows the area of interest R2. FIG. 13 is a diagram showing regions of interest R1 and R2 in the correct image (FIG. 4).

FIG. 14 is a diagram showing regions of interest R1 and R2 _{in the CNN output tomographic image x CNN} (FIG. 7) obtained by performing the processing of the CNN processing step S12 only once. FIG. 15 is a diagram showing regions of interest R1 and R2 in _{the second tomographic image x 2} (FIG. 8) obtained by performing the processing of the CNN processing step S12 only once. FIG. 16 is a diagram showing regions of interest R1 and R2 in _{the second tomographic image x 2} (FIG. 9) obtained by performing the processing of the CNN processing step S12 10 times. 14 to 16 show the case where _LA is used as a loss function during CNN learning.

FIG. 17 is a diagram showing regions of interest R1 and R2 _{in the CNN output tomographic image x CNN} (FIG. 10) obtained by performing the processing of the CNN processing step S12 only once. FIG. 18 is a diagram showing regions of interest R1 and R2 in _{the second tomographic image x 2} (FIG. 11) obtained by performing the processing of the CNN processing step S12 only once. FIG. 19 is a diagram showing regions of interest R1 and R2 in _{the second tomographic image x 2} (FIG. 12) obtained by performing the processing of the CNN processing step S12 10 times. These views 17 through 19 are those in the case of using _{L B} as a loss function when CNN learning.

In the region of interest R1 of the correct image (FIGS. 4 and 13 (a)), lacunar infarction is observed in the region close to white indicated by the solid arrow. Lacunar infarction is a cerebral infarction caused by clogging of small blood vessels flowing deep in the brain among cerebral thrombosis. _L A as a loss function when CNN learning, in the case of using any of _{L B} may, in comparison with CNN output tomographic image x _CNN obtained by performing the processing of the CNN processing step S12 only once, the CNN processing step S12 second the tomographic image x ₂ lacunar infarction were observed clearly obtained processing performed only once, the second tomographic images x ₂ in lacunar infarction further obtained by performing 10 times the processing of the CNN processing step S12 It is clearly recognized.

L as a loss function when CNN learning _A, even when using any of L _B, in a region where the broken line arrows in the region of interest R1 is indicated, CNN output tomographic obtained by performing the processing of the CNN processing step S12 only once Compared with the image x _{CNN , the second tomographic image x 2} obtained by performing the processing of the CNN processing step S12 only once has reduced artifacts, and was obtained by performing the processing of the CNN processing step S12 10 times. In the second tomographic image x ₂ , the artifacts are further reduced.

In attention area R2 of the correct images (FIG. 4, FIG. 13 (b)), when focusing on the region where the solid arrow is pointing, _L A as a loss function when CNN learning, even when using any of _{L B,} CNN processing steps Compared with the CNN output tomographic image x _CNN obtained by performing the processing of S12 only once, the brain structure is clearly restored in _{the second tomographic image x 2} obtained by performing the processing of the CNN processing step S12 only once. _{In the second tomographic image x 2} obtained by performing the processing of the CNN processing step S12 10 times, the brain structure is more clearly restored.

In attention area R2 of the correct images (FIG. 4, FIG. 13 (b)), when focusing on the area indicated by the dashed line marked, _L A as a loss function when CNN learning, even when using any of _{L B,} CNN processing steps Compared with the CNN output tomographic image x _CNN obtained by performing the processing of S12 only once, the artifacts are reduced in _{the second tomographic image x 2} obtained by performing the processing of the CNN processing step S12 only once. _{, The second tomographic image x 2} obtained by performing the processing of the CNN processing step S12 10 times further reduces the artifacts. Particularly, in the case of using L _B as a loss function when CNN learning, second tomographic image x ₂ in artifact obtained by performing 10 times the processing of the CNN processing step S12 is hardly recognized.

Figure 20, _L as a loss function when CNN learning A, the case of using the respective _{L B,} the number of repetitions itr the first tomographic image _{x 1} and the second tomographic image _x 2 (CNN processing steps S12 1, 2, 5 , 10) It is a graph which shows each PSNR. PSNR (Peak Signal to Noise Ratio) expresses the quality of an image in decibels (dB), and the higher the value, the better the image quality.

21 1, 2 and 5, _L as a loss function when CNN learning A, the case of using the respective _{L B,} the number of repetitions itr the first tomographic image _{x 1} and the second tomographic image _x 2 (CNN processing step S12 , 10) It is a graph which shows each SSIM. The SSIM (Structural Similarity Index) is an index for quantifying changes in image brightness, contrast, and structure, and the higher the value, the better the image quality.

Both the PSNR and SSIM indexes show that the image quality of the second tomographic image x ₂ is better than that of the first tomographic image x ₁ , and the second isr as the number of repetitions of the CNN processing step S12 is increased. It shows that the image quality of the tomographic image x _{2 is improved.} Also, any indication of PSNR and SSIM also towards the case of using _{L B} from _{L A} as a loss function when CNN learning, indicating that the image quality is good.

1 ... MRI device, 2 ... image processing device, 10 ... k space data collection unit, 11 ... first tomographic image creation unit, 12 ... CNN processing unit, 13 ... 1st k space data creation unit, 14 ... second k space data creation Unit, 15 ... 2nd tomographic image creation unit, 16 ... control unit, 17 ... storage unit, 18 ... display unit, x ₁ ... 1st tomographic image, x ₂ ... 2nd tomographic image, x _CNN ... CNN output tomographic image, y ₀ ... partial k space data, y ₁ ... first k space data, y ₂ ... second k space data.

Claims (13)

A first tomographic image creation unit that creates a first tomographic image based on partial k-space data collected for a part of k-space in an MRI apparatus.
A CNN processing unit that inputs a tomographic image to a convolutional neural network and outputs a CNN output tomographic image from the convolutional neural network.
A first k spatial data creation unit that creates first k spatial data based on the CNN output tomographic image, and
A second k-space data creation unit that creates second k-space data based on the partial k-space data for the partial space and the k-space data for a space other than the partial space among the first k-space data. When,
A second tomographic image creation unit that creates a second tomographic image based on the 2k spatial data,
With
The CNN processing unit performs the processing by the convolutional neural network twice or more, in the first processing, the first tomographic image is input to the convolutional neural network, and in the second and subsequent processing, the previous processing is performed. The second tomographic image created based on the CNN output tomographic image by processing is input to the convolutional neural network.
Image processing device. When the CNN processing unit trains the convolutional neural network, the first tomographic image of the subject or the first tomographic image of the subject created based on the partial k-space data collected for the partial space in the k-space. 2 The tomographic image is used as an input image to the convolutional neural network, and the tomographic image of the subject created based on the k-space data collected for a space wider than the partial space in the k-space is used as a teacher image. ,
The image processing apparatus according to claim 1. When training the convolutional neural network, the CNN processing unit uses a tomographic image of the subject created based on k-space data collected for the entire k-space as a teacher image.
The image processing apparatus according to claim 2. When training the convolutional neural network, the CNN processing unit uses a function including the difference between the teacher image and the CNN output tomographic image as a loss function.
The image processing apparatus according to claim 2 or 3. When the convolutional neural network is trained, the CNN processing unit determines the difference between the teacher image and the CNN output tomographic image, and the k-space data obtained by Fourier transforming the teacher image and the CNN output tomographic image. The function including the difference from the k-space data obtained by Fourier transforming is used as the loss function.
The image processing apparatus according to claim 2 or 3. A first tomographic image creation unit that creates a first tomographic image based on partial k-space data collected for a part of k-space in an MRI apparatus.
A CNN processing unit that inputs the first tomographic image to the convolutional neural network and outputs a CNN output tomographic image from the convolutional neural network.
A first k spatial data creation unit that creates first k spatial data based on the CNN output tomographic image, and
A second k-space data creation unit that creates second k-space data based on the partial k-space data for the partial space and the k-space data for a space other than the partial space among the first k-space data. When,
A second tomographic image creation unit that creates a second tomographic image based on the 2k spatial data,
With
When the CNN processing unit trains the convolutional neural network,
The first tomographic image or the second tomographic image of the subject created based on the partial k-space data collected for the partial k-space is used as an input image to the convolutional neural network.
A tomographic image of the subject created based on k-space data collected for a space wider than the partial space in the k-space was used as a teacher image.
A function including the difference between the teacher image and the CNN output tomographic image, and the difference between the k-space data obtained by Fourier transforming the teacher image and the k-space data obtained by Fourier transforming the CNN output tomographic image. As a loss function,
Image processing device. The k-space data collection unit that collects k-space data,
The image processing apparatus according to any one of claims 1 to 6, which creates a tomographic image based on the k-space data collected by the k-space data collection unit.
An MRI apparatus comprising. The first tomographic image creation step of creating a first tomographic image based on the partial k-space data collected for a part of the k-space in the MRI apparatus,
A CNN processing step in which a tomographic image is input to a convolutional neural network and a CNN output tomographic image is output from the convolutional neural network.
The first k spatial data creation step for creating the first k spatial data based on the CNN output tomographic image, and
A second k-space data creation step for creating a second k-space data based on the partial k-space data for the partial space and the k-space data for a space other than the partial space among the first k-space data. When,
A second tomographic image creation step for creating a second tomographic image based on the 2k spatial data, and
With
The processing of the CNN processing step is performed twice or more, in the first processing, the first tomographic image is input to the convolutional neural network, and in the second and subsequent processing, the CNN output fault obtained by the previous processing is performed. The second tomographic image created based on the image is input to the convolutional neural network.
Image processing method. In the CNN processing step, when the convolutional neural network is trained, the first tomographic image of the subject or the first tomographic image of the subject created based on the partial k-space data collected for the partial space in the k-space. 2 The tomographic image is used as an input image to the convolutional neural network, and the tomographic image of the subject created based on the k-space data collected for a space wider than the partial space in the k-space is used as a teacher image. ,
The image processing method according to claim 8. In the CNN processing step, when training the convolutional neural network, a tomographic image of the subject created based on k-space data collected for the entire k-space is used as a teacher image.
The image processing method according to claim 9. In the CNN processing step, when training the convolutional neural network, a function including the difference between the teacher image and the CNN output tomographic image is used as a loss function.
The image processing method according to claim 9 or 10. In the CNN processing step, when the convolutional neural network is trained, the difference between the teacher image and the CNN output tomographic image, and the k-space data obtained by Fourier transforming the teacher image and the CNN output tomographic image. Is used as a loss function by using a function including the difference from the k-space data obtained by Fourier transforming.
The image processing method according to claim 9 or 10. The first tomographic image creation step of creating a first tomographic image based on the partial k-space data collected for a part of the k-space in the MRI apparatus,
A CNN processing step in which the first tomographic image is input to the convolutional neural network and a CNN output tomographic image is output from the convolutional neural network.
The first k spatial data creation step for creating the first k spatial data based on the CNN output tomographic image, and
A second k-space data creation step for creating a second k-space data based on the partial k-space data for the partial space and the k-space data for a space other than the partial space among the first k-space data. When,
A second tomographic image creation step for creating a second tomographic image based on the 2k spatial data, and
With
In the CNN processing step, when training the convolutional neural network,
The first tomographic image or the second tomographic image of the subject created based on the partial k-space data collected for the partial k-space is used as an input image to the convolutional neural network.
A tomographic image of the subject created based on k-space data collected for a space wider than the partial space in the k-space was used as a teacher image.
A function including the difference between the teacher image and the CNN output tomographic image, and the difference between the k-space data obtained by Fourier transforming the teacher image and the k-space data obtained by Fourier transforming the CNN output tomographic image. As a loss function,
Image processing method.

Images