TW202307863A - Brain extraction and coil shimming correction model establishment method for brain image capable of significantly reducing manpower and time costs with high accuracy - Google Patents

Abstract

A brain extraction and coil shimming correction model establishment method for brain images is provided. A computer device builds a generator model and a discriminator model. For each training data, the discriminator model generates a first prediction result and a second prediction result according to a three-dimensional brain image before image processing and a three-dimensional brain image after image processing of the training data, and a first brain extraction and correction image generated by the generator model. The discriminator model is adjusted based on the first prediction result and the second prediction result. The adjusted discriminator model generates a third prediction result according to the three-dimensional brain image before image processing and the first brain extraction and correction image. Finally, the adjusted generator model is obtained according to the third prediction result.

Description

Brain Extraction and Coil Shimming Correction Model Establishment Method for Brain Image

The present invention relates to a method for establishing a model, in particular to a method for establishing a brain image extraction and coil shimming correction model.

MRI (Magnetic Resonance Imaging) has been widely used to obtain structural and functional information of the brain. By constructing brain atlases and templates, these informative multimodal data from different subjects at different times can be image-fused into a common reference space for voxel-wise and regional analysis to understand aging, disease, and treatment Impact. For accurate image fusion, images need to be corrected for head motion, geometric distortion, and coil B1 field non-uniformity. Since poor inhomogeneity correction and brain extraction/skull stripping can lead to misalignment or bias, manual correction and manual extraction of brain parts is required.

Although there are various methods for coil non-uniformity correction, these methods still need to be specific for different image contrast (for example, spin-echo (spin-echo) vs. gradient-echo (gradient-echo)), coil design and head. Adjust the position manually. Likewise, brain extraction also requires manual editing to obtain accurate regions of interest for brain images.

However, manual adjustment and editing still consume a lot of time and related human resources.

For example, if the entire batch of unprocessed data sets has 400 3D images (image size is 64×64×64), taking the task of selecting the region of interest in the brain as an example, assuming that it takes about For one minute, it takes about 64 minutes for images of 64 slices, and it takes about 426.67 hours to process one data set.

Therefore, the object of the present invention is to provide a brain extraction and coil shimming correction model establishment method that can automatically extract the region of interest and perform non-uniform correction of the brain image.

Therefore, the method for establishing the brain extraction and coil shimming correction model of the brain image of the present invention is performed by: a computer device, the computer device stores a plurality of training data, and each training data includes a three-dimensional image of the brain before image processing And a three-dimensional image of the brain after image processing after non-uniform correction, the three-dimensional image of the brain after image processing includes a region of interest, the method includes a step (A), a step (B), a step (C), a Step (D), a step (E), a step (F), a step (G), a step (H), and a step (I).

In the step (A), the computer device establishes a generator model for generating an image after brain extraction and coil shimming correction, the generator model includes a generating input port, at least one generating encoder, a The convolutional layer, at least one generating decoder respectively corresponding to the at least one encoder, and a generating output port.

In the step (B), the computer device builds a discriminator model for predicting the authenticity of two input images, and the discriminator model includes two discriminative input ports, at least one discriminative decoder, and a discriminative output port.

In the step (C), for each training data, the computer device inputs the three-dimensional brain image of the training data before image processing to the generator model to generate a first brain extraction and correction image.

In the step (D), for each training data, the computer device inputs the three-dimensional images of the brain before image processing and the three-dimensional images of the brain after image processing of the training data into the discriminator model to generate a correlation A first prediction result of whether the three-dimensional brain image before the image processing and the three-dimensional brain image after the image processing of the training data are true.

In the step (E), the computer device adjusts the discriminator model according to the first prediction results corresponding to the training data.

In the step (F), for each training data, the computer device inputs the three-dimensional image of the brain before the image processing of the training data and the first extracted and corrected image of the brain corresponding to the training data to the discriminator model , so as to generate a second prediction result related to whether the pre-processing three-dimensional brain image of the training data and the first brain extraction and correction image corresponding to the training data are true.

In the step (G), the computer device adjusts the discriminator model according to the second prediction result corresponding to the training data.

In the step (H), for each training data, the computer device inputs the three-dimensional image of the brain before the image processing of the training data and the first extracted and corrected image of the brain corresponding to the training data into the adjusted A discriminator model for generating a third prediction result related to whether the pre-image-processed three-dimensional brain image of the training data and the first brain extraction and correction image corresponding to the training data are true.

In the step (I), the computer device processes the three-dimensional images of the brain according to the third prediction results corresponding to the training data, the images of the training data, and the first brain extraction and correction corresponding to the training data The generator model is image adjusted to obtain an adjusted generator model.

The effect of the present invention is that the adjusted generator model can output a non-uniformly corrected image with brain regions of interest according to any three-dimensional image of the brain before processing, greatly reducing the manpower and time costs of relevant professionals and having accurate Spend.

Before the present invention is described in detail, it should be noted that in the following description, similar elements are denoted by the same numerals.

Referring to Fig. 1, it illustrates a computer device 1 of an embodiment for implementing the brain extraction of the brain image of the present invention and the establishment method of the coil shimming correction model, the computer system 1 includes a storage unit 11 and an electrical connection to the The processing unit 12 of the storage unit 11 . In this embodiment, the implementation of the computer system 1 is, for example, a personal computer, a server or a cloud host, but it is not limited thereto.

The storage unit 11 stores multiple pieces of training data and multiple pieces of verification data. Each piece of training data or verification data includes a three-dimensional image of the brain before image processing and a three-dimensional image of the brain after image processing after non-uniform correction. The 3D image includes a region of interest.

It is worth noting that the three-dimensional images of the brain before and after image processing of the training data or verification data are (Magnetic Resonance Imaging, MRI), and the training data or verification data can all be rats ( rat) brain images, or all mouse (mouse) brain images, or mixed, but not limited thereto.

Referring to FIG. 1 and FIG. 2 , the steps included in this embodiment of the method for brain extraction and coil shimming correction model establishment of brain images of the present invention will be described below.

In step 201, the processing unit 12 establishes a generator model for generating an image after brain extraction and coil shimming correction, the generator model includes a generator input port, at least one generator encoder, and a convolutional layer , at least one generating decoder respectively corresponding to the at least one encoder, and a generating output port.

In step 202, the processing unit 12 establishes a discriminator model for predicting the authenticity of two input images, the discriminator model includes two discriminative input ports and at least one discriminative decoder.

It is worth noting that in this embodiment, Keras is used to establish the generator model and the discriminator model of the pix2pix architecture against the generation network (generative adversarial network, GAN). The generator model is a u-net structure, It includes 6 generation encoders and 6 generation decoders, and the convolution kernel size (kernel size) is a three-dimensional size of 4×4×4, and the size of the generation input port and the generation output port is 64×64×64; the The discriminator model is a patchGAN structure, including 6 discriminative decoders, and the size of the convolution kernel is 4×4×4, the size of the discriminator input port is 64×64×64, and the discriminator output port size is 1× 1×1, but not limited to this. The prediction result of the discriminator model is between zero and one, representing the probability that all the input images are 'true'. If the prediction result is one, it means that the judgment is 'true', that is, the input images are all from the training data or the verification data; otherwise, if the prediction result is zero, it means that the judgment is 'false', that is, the input One of the images of is not from the training data or the verification data.

It should be especially noted that the existing confrontation network is based on the model structure of 2D images. Although a 3D image can be disassembled into several 2D images and processed separately, this means that each 2D slice is independent. image, so two adjacent two-dimensional slices in the original three-dimensional image may be visually discontinuous; and in this embodiment, the size of the convolution kernel is three-dimensional, which is different from decomposing a three-dimensional image into several two-dimensional slices. Therefore, the problem of discontinuity between two adjacent two-dimensional slices can be improved.

In step 203, for each training data, the processing unit 12 inputs the pre-processing three-dimensional brain image of the training data into the generator model to generate a first brain extraction and correction image.

In step 204, for each training data, the processing unit 12 inputs the 3D brain image before image processing and the 3D brain image after image processing of the training data into the discriminator model to generate a correlation with the The first prediction result of whether the three-dimensional brain image of the training data before the image processing and the three-dimensional brain image after the image processing is true.

In step 205, the processing unit 12 adjusts the discriminator model according to the first prediction result corresponding to the training data.

It should be noted that in this embodiment, since the images input to the discriminator model in step 204 are all from the training data, in step 205 it is desired to enhance the ability of the discriminator model to identify 'true', Therefore, it is expected that the first prediction results are one, so the processing unit 12 uses a first loss function to calculate the distance between the first prediction results and "one" according to the first prediction results, so as to The gap adjusts the discriminator model, and the first loss function is, for example, binary crossentropy (binary crossentropy), but not limited thereto.

In step 206, for each training data, the processing unit 12 inputs the pre-processing three-dimensional brain image of the training data and the first extracted and corrected brain image corresponding to the training data to the discriminator model, so as to A second prediction result is generated as to whether the 3D brain image before image processing of the training data and the first brain extraction and correction image corresponding to the training data are true.

In step 207, the processing unit 12 adjusts the discriminator model according to the second prediction result corresponding to the training data.

It should be noted that, in this embodiment, since the images input to the discriminator model in step 206 are not all from the training data or the verification data, it is hoped to enhance the discriminator model in step 207 to identify Therefore, it is expected that the second prediction results are zero, so the processing unit 12 uses the first loss function to calculate the distance between the second prediction results and "zero" according to the second prediction results to adjust the discriminator model according to its gap.

In step 208, for each training data, the processing unit 12 inputs the pre-processed 3D brain image of the training data and the first extracted and corrected brain image corresponding to the training data to the adjusted discriminator A model for generating a third prediction result related to the difference between the pre-processing three-dimensional brain image of the training data and the first brain extraction and correction image corresponding to the training data.

It should be noted that, in this embodiment, in steps 204, 206, and 208, the discriminator model first combines the two input images, and then generates the first prediction result, the second prediction result, and the third prediction result, but not limited thereto.

In step 209, the processing unit 12 adjusts according to the third prediction result corresponding to the training data, the image-processed three-dimensional brain image of the training data, and the first brain extraction and correction image corresponding to the training data. the generator model to obtain an adjusted generator model.

It should be noted that in this embodiment, in step 209, it is hoped that the first brain extraction and correction image generated by the generator model can be close to the images of the training data, so that the discrimination model can make prediction errors. Therefore, it is desired The third prediction results are one, so the processing unit 12 uses the first loss function to calculate the distance between the third prediction results and "one" according to the third prediction results, and according to the training After the image processing of the data, the three-dimensional brain image and the corresponding first brain extraction and correction image are used to calculate the gap between the three-dimensional brain image after image processing and the first brain extraction and correction image by a second loss function, The generator model can be adjusted according to the second difference. The second loss function is, for example, mean absolute error (MAE), but not limited thereto.

It should be noted again that when the generator model is tuned, the weights of the discriminator model will be fixed, whereas when the discriminator model is tuned, the weights of the generator model will be fixed.

1。當判定出已獲得K+1個調整後生成器模型時，流程進行步驟211；而當定出未獲得K+1個調整後生成器模型時，流程進行重複步驟203~209。 In step 210, the processing unit 12 determines whether K+1 adjusted generator models have been obtained, K

1. When it is determined that K+1 adjusted generator models have been obtained, the process proceeds to step 211 ; and when it is determined that K+1 adjusted generator models have not been obtained, the process proceeds to repeat steps 203 - 209 .

In step 211, for each verification data and each adjusted generator model, the processing unit 12 inputs the pre-image-processed three-dimensional brain image of the verification data into the adjusted generator model to generate a second Brain extraction and rectification of images.

In step 212, for each verification data and each adjusted generator model, the processing unit 12 processes the three-dimensional brain image and the second brain extraction and correction image corresponding to the verification data according to the verification data, A similarity index is calculated.

It is worth noting that the similarity index is, for example, cosine angle distance (cosine angle distance, CAD), Euclidean distance (Euclidean distance, L2 norm), mean square error (mean square error), peak signal-to-noise ratio (peak signal-to-noise ratio) to-noise ratio) and mean structural similarity (MSSIM), among which Euclidean distance, mean square error, and peak signal-to-noise ratio are just linear combinations of each other, so only the Euclidean distance is exemplified below.

，

， 其中， A為該驗證資料的影像處理後腦部三維影像， B為該驗證資料對應的第二腦部提取及校正影像， a ₁, a ₂,…, a _N 為該驗證資料的影像處理後腦部三維影像的N個體素， b ₁, b ₂,…, b _N 為該驗證資料對應的第二腦部提取及校正影像的N個體素，該相似度指標介於-1至1之間。 If the similarity index is cosine angular distance, then the similarity index CAD is expressed by the following formula:

,

, wherein, A is the three-dimensional image of the brain after image processing of the verification data, B is the second brain extraction and correction image corresponding to the verification data, a ₁ , a ₂ ,..., a _N are the image processing of the verification data N voxels of the three-dimensional image of the hindbrain, b ₁ , b ₂ ,…, b _N are the N voxels of the second brain extraction and correction image corresponding to the verification data, and the similarity index is between -1 and 1 between.

， 其中，

為該驗證資料的影像處理後腦部三維影像，

為該驗證資料對應的第二腦部提取及校正影像， A為

的局部窗口， B為

的局部窗口，

和

分別位於

和

內，SSIM包括亮度 l、對比度 c和結構 s， w _i 為圓對稱高斯加權函數，

1，

為可變動權重，

10 ^-3的純量， M為影像的窗口總數，該相似度指標介於-1至1之間。 If the similarity index is the average structural similarity, then the similarity index MSSIM is expressed by the following formula:

, in,

3D images of the brain after image processing for the validation data,

is the extracted and corrected image of the second brain corresponding to the verification data, A is

The local window of , B is

the local window of

and

respectively located in

and

Inside, SSIM includes brightness l , contrast c and structure s , w _i is a circularly symmetric Gaussian weighting function,

1,

is the variable weight,

10 ^-3 scalar, M is the total number of image windows, and the similarity index is between -1 and 1.

為1，

10 ^-4、9×10 ^-4、4.5×10 ^-4，但不以此為限。 It is worth noting that in this example,

is 1,

10 ^-4 , 9×10 ^-4 , 4.5×10 ^-4 , but not limited thereto.

，

， 其中， A為該驗證資料的影像處理後腦部三維影像， B為該驗證資料對應的第二腦部提取及校正影像， a ₁, a ₂,…, a _N 為該驗證資料的影像處理後腦部三維影像的N個體素， b ₁, b ₂,…, b _N 為該驗證資料對應的第二腦部提取及校正影像的N個體素，該相似度指標最小值為0。 If the similarity index is Euclidean distance, then the similarity index L2 is expressed by the following formula:

,

, wherein, A is the three-dimensional image of the brain after image processing of the verification data, B is the second brain extraction and correction image corresponding to the verification data, a ₁ , a ₂ ,..., a _N are the image processing of the verification data The N voxels of the three-dimensional image of the hindbrain, b ₁ , b ₂ ,..., b _N are the N voxels of the second brain extraction and correction image corresponding to the verification data, and the minimum value of the similarity index is 0.

In step 213, the processing unit 12 selects an optimal adjusted generator model from the K+1 adjusted generator models according to the similarity indicators corresponding to the K+1 adjusted generator models.

It is worth noting that if the similarity index is cosine angular distance or average structural similarity, the average of the similarity index corresponding to the optimally adjusted generator model is relatively highest; if the similarity index is Euclidean distance, Then the average of the similarity indicators corresponding to the optimally adjusted generator model is relatively the lowest.

It should be noted that in other embodiments, only steps 201 to 209 may be performed, and the adjusted generator model may be directly used as the brain extraction and coil shimming correction model of the brain image.

In summary, the method for establishing the brain extraction and coil shimming correction model of the brain image of the present invention uses the processing unit 12 to establish the generator model and the discriminator model. For each training data, the discriminator The model generates the first prediction result and the second prediction result according to the three-dimensional brain image before the image processing, the three-dimensional brain image after the image processing, and the first brain extraction and correction image generated by the generation model, and according to The first prediction result and the second prediction result adjust the discriminator model, and the adjusted discriminator model generates the third prediction result according to the three-dimensional image of the brain before the image processing and the first extracted and corrected image of the brain , and finally adjust the generator model according to the third prediction result to obtain the adjusted generator model, then repeat steps 203-209 to obtain multiple adjusted generator models, and finally according to the similarity index corresponding to the verification data Select the best adjusted generator model, the best adjusted generator model can output images with brain regions of interest and non-uniform correction according to the three-dimensional images of the brain before any image processing, greatly reducing the manpower and labor of relevant professionals Time cost and accuracy, so can really achieve the purpose of the present invention.

But the above-mentioned ones are only embodiments of the present invention, and should not limit the scope of the present invention. All simple equivalent changes and modifications made according to the patent scope of the present invention and the content of the patent specification are still within the scope of the present invention. Within the scope covered by the patent of the present invention.

1: computer device 11: storage unit 12: Processing unit 201~213: Steps

Other features and effects of the present invention will be clearly presented in the implementation manner with reference to the drawings, wherein: Fig. 1 is a block diagram illustrating a computer device of an embodiment of the method for implementing the brain extraction and coil shimming correction model of the brain image of the present invention; and FIG. 2 is a flowchart illustrating the embodiment of the method for brain extraction and coil shimming correction model establishment of brain images of the present invention.

201~213: Steps

Claims (9)

A method for establishing a brain extraction and coil shimming correction model of a brain image, executed by a computer device, the computer device stores multiple training data, each training data includes a three-dimensional image of the brain before image processing and a non-uniform The corrected image-processed three-dimensional image of the brain, the image-processed three-dimensional image of the brain includes a region of interest, the method includes the following steps: (A) Establish a generator model for generating an image after brain extraction and coil shimming correction, the generator model includes a generator input port, at least one generator encoder, a convolutional layer, and at least one corresponding to at least one encoder generating decoder, and a generating output port; (B) establishing a discriminator model for predicting the authenticity of two input images, the discriminator model comprising two discriminative input ports, at least one discriminative decoder, and a discriminative output port; (C) for each training data, input the pre-image processed three-dimensional brain image of the training data into the generator model to generate a first brain extraction and correction image; (D) For each training data, input the three-dimensional image of the brain before the image processing and the three-dimensional image after the image processing of the training data into the discriminator model to generate a corresponding image processing of the training data The first prediction result of whether the three-dimensional image of the forebrain and the three-dimensional image of the brain after processing the image is true; (E) adjusting the discriminator model according to the first prediction result corresponding to the training data; (F) For each training data, input the three-dimensional image of the brain before the image processing of the training data and the first extracted and corrected image of the brain corresponding to the training data into the discriminator model to generate a Whether the three-dimensional image of the brain before the image processing of the data and the first brain extraction and correction image corresponding to the training data is true or not is the second prediction result; (G) adjusting the discriminator model according to the second prediction result corresponding to the training data; (H) For each training data, input the pre-processing three-dimensional brain image of the training data and the first extracted and corrected brain image corresponding to the training data into the adjusted discriminator model to generate a correlation a third prediction result of whether the three-dimensional brain image before the image processing of the training data and the first brain extraction and correction image corresponding to the training data are true; and (1) adjust the generator model according to the third prediction result corresponding to the training data, the three-dimensional image of the brain after image processing of the training data, and the first brain extraction and correction image corresponding to the training data, so as to Obtain an adjusted generator model. The method for establishing a brain extraction and coil shimming correction model of a brain image as described in Claim 1, wherein, in step (D), for each training data, the discriminator model will be The 3D brain image before image processing and the 3D brain image after image processing of the training data are merged, and then the first prediction result is generated by the at least one discriminant decoder. According to the method of brain extraction and coil shimming correction model of brain image described in claim 1, the computer device also stores multiple pieces of verification data, each piece of verification data includes a three-dimensional image of the brain before image processing and a non-invasive The three-dimensional image of the brain after image processing of uniform correction, the three-dimensional image of the brain after image processing includes a region of interest, and the following steps are also included after step (I): (J) Repeat steps (C) to (I) K times , to obtain K+1 adjusted generator models, K

1; (K) for each validation data and each adjusted generator model, input the pre-image-processed 3D brain image of the validation data into the adjusted generator model to generate a second brain extraction and Correcting the image; (L) For each verification data and each adjusted generator model, calculate a Similarity index; (M) Select an optimal adjusted generator model from the K+1 adjusted generator models according to the similarity indexes corresponding to the K+1 adjusted generator models. The method for establishing the brain extraction and coil shimming correction model of the brain image as described in claim 3, wherein, in step (L), the similarity index CAD is represented by the following formula:

,

, wherein, A is the three-dimensional image of the brain after image processing of the verification data, B is the second brain extraction and correction image corresponding to the verification data, a ₁ , a ₂ ,..., a _N are the image processing of the verification data N voxels of the three-dimensional image of the hindbrain, b ₁ , b ₂ ,..., b _N are N voxels of the second brain extraction and correction image corresponding to the verification data. The method for establishing the brain extraction and coil shimming correction model of the brain image described in claim 3, wherein, in step (L), the similarity index MSSIM is represented by the following formula:

, in,

3D images of the brain after image processing for the validation data,

is the extracted and corrected image of the second brain corresponding to the verification data, A is

The local window of , B is

the local window of

and

respectively located in

and

Inside, SSIM includes brightness l , contrast c and structure s , w _i is a circularly symmetric Gaussian weighting function,

1,

is the variable weight,

The scalar of ^-3 , M is the total number of windows of the image. The method for establishing the brain extraction and coil shimming correction model of the brain image described in claim 4 or 5, wherein, in step (M), the average of the similarity indicators corresponding to the optimally adjusted generator model is relatively highest. The method for establishing the brain extraction and coil shimming correction model of the brain image described in claim 3, wherein, in step (L), the similarity index L2 is represented by the following formula:

,

, wherein, A is the three-dimensional image of the brain after image processing of the verification data, B is the second brain extraction and correction image corresponding to the verification data, a ₁ , a ₂ ,..., a _N are the image processing of the verification data N voxels of the three-dimensional image of the hindbrain, b ₁ , b ₂ ,..., b _N are N voxels of the second brain extraction and correction image corresponding to the verification data. The method for establishing the brain extraction and coil shimming correction model of the brain image described in claim 7, wherein, in step (M), the average of the similarity indicators corresponding to the optimally adjusted generator model is relatively the lowest . The method for establishing the brain extraction and coil shimming correction model of the brain image as described in claim 1, wherein, in step (A), the convolution kernel size of the generator model is three-dimensional, and in step (B) In , the convolution kernel size of the discriminator model is three-dimensional.

Images