KR102655827B1 - Brain tissue segmentation System and method from the brain MRI data using deep-learning - Google Patents

Abstract

The present invention relates to a brain region segmentation system and method from brain MRI data using deep learning, and to segment the brain regions of premature infants, especially infants and toddlers aged 3 months to 3 years old, with high accuracy, which are difficult to distinguish from brain MRI data by brightness values alone. It is about technology that can segment and measure its volume.

Description

Brain tissue segmentation system and method from the brain MRI data using deep-learning}

The present invention relates to a brain region segmentation system and method from brain MRI data using deep learning, and more specifically, to segmenting brain regions (GM, WM, and CSF) from brain MR image data of premature infants using deep learning. It relates to a brain region segmentation system and method from brain MRI data using deep learning that can quantitatively track brain development patterns by measuring the volume of the segmented region.

MRI (Magnetic Resonance Imaging) is an imaging technique that can obtain arbitrary tomographic images of the living body using magnetic fields generated by magnetism. Depending on the image emphasis, even for the same patient, T1-weighted images, T2-weighted images, etc. It is divided into emphasis video, FLAIR video, etc. Typically, multiple weighted images are used simultaneously to diagnose the condition.

In T1-weighted images, water appears black and fat appears white, and anatomical structures can be best identified, while T2-weighted images, contrary to T1-weighted images, make water appear white and fat appear black, making it easy to detect lesions.

The brain area consists of the cerebral cortex, called gray matter (GM), and the white matter (WM) and cerebrospinal fluid (CSF) that form the inside. Morphologically, brain regions are divided and utilized to determine infant brain development patterns and neurodevelopmental disorders.

As the novel coronavirus infection (COVID-19) continues to spread widely since 2020, research results have been published showing that it inhibits the brain development of premature babies or infants. According to results announced by Brown University in the United States, the brain development scores of infants and toddlers aged 3 months to 3 years have fallen sharply compared to before in 2020, when COVID-19 began to spread. This is predicted to be due to the suppression of face-to-face activities and the inability to see the facial expressions of others due to wearing masks at a time when social skills should be learned, preventing proper social skills learning from taking place. In addition, pregnant women's stress due to COVID-19 is also being pointed out as a factor in hindering the child's brain development.

However, in the case of MRI data that can determine the brain development patterns of premature infants or infants within 6 months of age, there is little difference in brightness values between GM and WM in both T1-weighted images and T2-weighted images, so they can be distinguished with the naked eye. Therefore, it is very difficult to determine brain development patterns or neurodevelopmental disorders.

Accordingly, recently, technology has been developed to perform brain region segmentation by analyzing MRI of premature or 6-month-old infants by applying deep learning methods that are effective in distinguishing various characteristics as well as brightness values that are difficult for humans to perceive. It is being developed.

In this regard, in Korea Registered Patent No. 10-1718130 (“Brain region segmentation method and system using magnetic resonance imaging”), the initial lateral ventricle region is created using a standard brain map image and the lateral ventricle region is divided based on the brightness value threshold. is updating and is launching a technology that can improve the accuracy and efficiency of brain region segmentation.

Domestic registered patent No. 10-1718130 (registration date 2017.03.14.)

The present invention was created to solve the problems of the prior art as described above. The purpose of the present invention is to detect GM (gray matter) considering brightness values through brain MR image data of premature infants around 6 months old. Since it is difficult to distinguish between WM and WM (white matter), it relates to a brain region segmentation system and method from brain MRI data using deep learning that can effectively separate elements that are difficult to distinguish using deep learning. .

In a brain region segmentation system from brain MRI data using deep learning according to an embodiment of the present invention, a first MRI data set for deep learning learning processing and second MRI data in which brain regions to be segmented are labeled A data input unit 100 that receives the first MRI data set and the second MRI data set, performs image processing to correspond to each preset direction, and produces a first MRI data set and a second MRI data set for each direction. The data preprocessor 200, which generates an MRI data set, receives the first MRI data set and the second MRI data set for each direction, generates them as learning data, and performs learning processing using a pre-stored deep learning network, A deep learning learning unit 300 that generates a learning model for each direction, an external data input unit 400 that receives a third MRI data set to segment brain regions, and a preset angle input unit that receives the third MRI data set. An external data preprocessor 500 performs image processing to correspond to the direction and generates a third MRI data set for each direction, receives the third MRI data set for each direction, and inputs each into a learning model for each direction. Thus, a deep learning inference unit 600 that outputs prediction data for each direction, an inference fusion unit 700 that derives a brain region corresponding to each pixel of the third MRI data set by integrating the prediction data in each direction. ) and an analysis unit 800 that performs segmentation processing of each brain region derived using each pixel information, wherein the first MRI data set is a T1-weighted MRI data set and a T2-weighted MRI data obtained by imaging the same region. It is desirable to include three.

Furthermore, the data preprocessor 200 and the external data preprocessor 500 generate three-dimensional data for each transmitted MRI data set, and for each three-dimensional data, the horizontal plane (Axial) direction and the coronal plane (Coronal plane) are generated. ) direction and sagittal direction, and generate MRI data sets for each direction.

Alternatively, the data pre-processing unit 200 and the external data pre-processing unit 500 analyze the information included in each transmitted MRI data set and determine the corresponding horizontal (Axial) direction, Coronal direction (Coronal direction), and Sagittal direction. It is desirable to classify by (sagittal) direction and generate MRI data sets for each direction.

Furthermore, the deep learning learning unit 300 receives prediction data for each direction by the deep learning inference unit 600, and calculates a preset loss function using the prediction data for each direction and the second MRI data set. It is desirable to update the learning model for each direction to minimize the calculation of and generate the final learning model for each direction.

Furthermore, the deep learning learning unit 300 performs learning processing by applying a softmax function as an activation function in the deep learning network, and the deep learning inference unit 600 performs a third It is desirable to output the prediction data in the form of a probability map corresponding to each brain region for each pixel of the MRI data set.

Furthermore, the inference fusion unit 700 uses the prediction data for each direction to calculate the average of the probability that each pixel of the third MRI data set corresponds to each brain region in each direction. , it is desirable to derive the final brain region of the corresponding pixel as the brain region corresponding to the highest average value.

Furthermore, the analysis unit 800 uses each pixel information to perform segmentation processing on each brain region, and the number of pixels included in each segmented brain region. It is desirable to include a volume calculation unit 820 that calculates the volume of the brain region.

Furthermore, the volume calculation unit 820 reflects the actual pixel size information for each direction included in the third MRI data set in the number of pixels included in each segmented brain region to determine the actual scale of each brain region. It is desirable to calculate the volume of .

Brain regions from brain MRI data using deep learning using a brain region segmentation system from brain MRI data using deep learning, each step of which is performed by computational processing means including a computer according to another embodiment of the present invention As a segmentation method, a first input step (S100) of receiving a first MRI data set for deep learning learning processing and a second MRI data set in which brain regions to be segmented are labeled, the first input step (S100) A first pre-processing step (S200) of receiving the MRI data set and performing image processing to correspond to each preset direction to generate a first MRI data set and a second MRI data set for each direction (S200), pre-stored deep Deep learning that receives the first MRI data set and the second MRI data set for each direction in the first preprocessing step (S200) as learning data in the learning network, performs learning processing, and generates a learning model for each direction. A learning step (S300), a second input step (S400) of receiving a third MRI data set for dividing brain regions, and receiving the MRI data set by the second input step (S400), in each preset direction. A second pre-processing step (S500) of performing image processing to generate a third MRI data set for each direction, using the third MRI data set for each direction by the second pre-processing step (S500), A deep learning inference step (S600) of inputting the learning model for each direction in the deep learning learning step (S300) and outputting predicted data for each direction, and using the integrated prediction data for each direction, the second input step. An inference fusion step (S700) of deriving a brain region corresponding to each pixel of the third MRI data set in (S400), and using each pixel information derived by the inference fusion step (S700), each brain region A segmentation step (S800) of performing segmentation processing and a volume calculation step of calculating the volume of each brain region using the number of pixels included in each brain region segmented by the segmentation step (S800) ( S900), wherein the first MRI data set preferably includes a T1-weighted MRI data set and a T2-weighted MRI data set obtained by imaging the same region.

Furthermore, the first preprocessing step (S200) and the second preprocessing step (S500) generate three-dimensional data for each transmitted MRI data set, and for each three-dimensional data, the horizontal plane (Axial) direction and the coronal plane ( It is desirable to perform slicing in the coronal and sagittal directions to generate MRI data sets for each direction.

Alternatively, the first preprocessing step (S200) and the second preprocessing step (S500) analyze the information included in each transmitted MRI data set and determine the corresponding horizontal (Axial) direction, Coronal direction (Coronal direction), and Sagittal direction. It is desirable to classify by sagittal direction and generate MRI data sets for each direction.

Furthermore, the deep learning learning step (S300) uses the prediction data for each direction by the deep learning inference step (S600) and the second MRI data set for each direction by the first preprocessing step (S200). It is desirable to perform the learning process repeatedly to minimize the calculation of the loss function, create a learning model for each direction, and perform the learning process by applying a softmax function as an activation function.

Furthermore, the deep learning inference step (S600) preferably outputs the prediction data in the form of a probability map corresponding to each brain region for each pixel of the third MRI data set for each direction.

Furthermore, the inference fusion step (S700) uses prediction data for each direction to calculate the average of the probability that each pixel of the third MRI data set corresponds to each brain region for each direction. , it is desirable to derive the final brain region of the corresponding pixel as the brain region corresponding to the highest average value.

The brain region segmentation system and method from brain MRI data using deep learning of the present invention according to the above configuration are difficult to distinguish between GM and WM only by brightness values, so it is difficult to determine the brain development pattern with the naked eye (premature infants or For brain MR images of 6-month-old infants), GM, WM, and CSF can be segmented more quickly and accurately using a deep learning method, and the volume of each segmented region can be measured to morphologically track brain development patterns. It has the advantage of being able to present quantitative diagnostic factors.

In addition, considering computational efficiency for model learning and prediction, usability and computational efficiency can be increased by using a 2D-based method instead of a 3D-based model, and input MR images can be divided into axial, sagittal, and coronal directions. By analyzing and fusing the results of each learned model, there is an advantage in improving segmentation accuracy, which can be low due to 2D-based models.

Through this, the brain volume of premature infants or 6-month-old infants can be calculated from brain MR images, and brain development patterns can be quantitatively tracked over time, which has the advantage of diagnosing and treating brain development inhibition at an early stage. There is.

In addition, because the entire process is performed automatically without the intervention of users (medical staff, analysts, engineers, etc.), the clinical usability value is expected to be even higher.

The global medical image analysis software market is expected to grow at an average annual rate of 8.2% from KRW 3.1 trillion in 2017 to reach KRW 4.6 trillion in 2022, and the global market for artificial intelligence/big data software medical devices using medical images is expected to reach KRW 1,100 in 2018. It is expected to grow at an average annual rate of 42.20% from KRW 100 million to reach KRW 620 billion in 2023. Brain region segmentation system and brain region segmentation system from brain MRI data using deep learning of the present invention, which must precede most artificial intelligence software utilizing medical images, Demand for medical image analysis such as that method is expected to naturally increase.

Figure 1 is an exemplary configuration diagram showing a brain region segmentation system from brain MRI data using deep learning according to an embodiment of the present invention.
Figure 2 is an exemplary diagram showing the brain cross-sectional direction used in the brain region segmentation system and method from brain MRI data using deep learning according to an embodiment of the present invention.
Figure 3 is an example of generating learning data for each direction by a brain region segmentation system and method from brain MRI data using deep learning according to an embodiment of the present invention.
Figure 4 is an example diagram of generating a learning model for each direction by a brain region segmentation system and method from brain MRI data using deep learning according to an embodiment of the present invention.
Figure 5 is an example diagram of deriving a brain region corresponding to each pixel by a brain region segmentation system and method from brain MRI data using deep learning according to an embodiment of the present invention.
Figure 6 is an example of brain region segmentation and volume calculation by a brain region segmentation system and method from brain MRI data using deep learning according to an embodiment of the present invention.
Figure 7 is a performance analysis table of a brain region segmentation system and method from brain MRI data using deep learning according to an embodiment of the present invention.
Figure 8 is a flowchart illustrating a method for segmenting brain regions from brain MRI data using deep learning according to an embodiment of the present invention.

Hereinafter, a brain region segmentation system and method from brain MRI data using deep learning of the present invention will be described in detail with reference to the attached drawings. The drawings introduced below are provided as examples so that the idea of the present invention can be sufficiently conveyed to those skilled in the art. Accordingly, the present invention is not limited to the drawings presented below and may be embodied in other forms. Additionally, like reference numerals refer to like elements throughout the specification.

At this time, if there is no other definition in the technical and scientific terms used, they have the meaning commonly understood by those skilled in the art to which this invention pertains, and the gist of the present invention is summarized in the following description and attached drawings. Descriptions of known functions and configurations that may be unnecessarily obscure are omitted.

In addition, a system refers to a set of components including devices, mechanisms, and means that are organized and interact regularly to perform necessary functions.

A brain region segmentation system and method from brain MRI data using deep learning according to an embodiment of the present invention divides GM, WM, and CSF from brain MR images of premature infants or 6-month-old infants, respectively, and This relates to a technology that can provide quantitative diagnostic factors that can morphologically track brain development patterns by measuring the volume of an area. In particular, considering that it is difficult to distinguish between GM and WM in brain MR images of 6-month-old infants or premature infants based on brightness values alone, adopting a deep learning method has the advantage of effectively separating various elements that are difficult to distinguish with the naked eye. there is.

In general, 3D-based models have higher segmentation performance than 2D-based models, but their application may not be possible depending on the computing power (memory size, etc.) of the calculation equipment, so there are clear limitations in terms of usability. To solve this problem, by considering computational efficiency for model learning and prediction, a 2D-based method is used instead of a 3D-based model to increase usability and computational efficiency, but to change the input MR image in the axial direction. , by analyzing the sagittal and coronal directions and fusing the results of each learned model, there is an advantage in improving segmentation accuracy, which can be low due to a 2D-based model.

Figure 1 is an example configuration diagram showing a brain region segmentation system from brain MRI data using deep learning according to an embodiment of the present invention. With reference to Figure 1, a system using deep learning according to an embodiment of the present invention is shown. The brain region segmentation system from brain MRI data is described in detail.

As shown in FIG. 1, the brain region segmentation system from brain MRI data using deep learning according to an embodiment of the present invention includes a data input unit 100, a data preprocessing unit 200, and a deep learning learning unit 300. , it is preferable to include an external data input unit 400, an external data pre-processing unit 500, a deep learning inference unit 600, an inference fusion unit 700, and an analysis unit 800, and each component uses a computer. It is preferable that it be included in one calculation processing means or in a plurality of calculation processing means.

To learn more about each configuration,

The data input unit 100 receives a first MRI data set for deep learning processing and a second MRI data set in which brain regions to be segmented are labeled.

The first MRI data set and the second MRI data set receive a public brain MRI data set, and T1-weighted image data, T2-weighted image data, and labeled image data are input for each patient.

In detail, the first MRI data set is two-channel MRI data, and a T1-weighted image and a T2-weighted image are input as a pair. That is, the first MRI data set includes a T1-weighted MRI set and a T2-weighted MRI set obtained by imaging the same brain region of the same patient. The second MRI data set is a data set in which the background area, GM area, WM area, and CSF area are labeled, and is preferably labeled data obtained from images of the same brain area of the same patient as the first MRI data set.

The data pre-processing unit 200 receives the first MRI data set and the second MRI data set from the data input unit 100, performs image processing to correspond to each preset direction, and produces a first MRI data set for each direction. It is desirable to generate a data set and a second MRI data set.

At this time, each preset direction means the horizontal direction (axial), coronal direction (coronal), and sagittal direction (sagittal) depending on which direction the body is divided, and each direction is as shown in Figure 2. .

The data pre-processing unit 200 performs image processing on each of the first MRI data set (T1-weighted image - T2-weighted image) and the second MRI data set (label data) received from the data input unit 100. do.

At this time, the data pre-processing unit 200 processes each transmitted MRI data set (T1-weighted MRI set, T2-weighted MRI set), as shown in FIG. 3, because the generally published brain MRI data is horizontal MRI data. Third, for the second MRI data set), 2-dimensional data are stacked to create each 3-dimensional data, and slicing is performed on each 3-dimensional data in the horizontal, coronal, and sagittal directions, and An MRI data set is created. At this time, if MRI data in the horizontal direction is input, it can be used as is to reduce the calculation process.

Through this, the preprocessed first MRI data set is two-dimensional for each of the T1-W horizontal plane, T1-W coronal plane, T1-W sagittal plane, T2-W horizontal plane, T2-W coronal plane, and T2-W sagittal plane. MRI image data can be acquired, and the preprocessed second MRI data set also acquires two-dimensional MRI image data for each of the horizontal, coronal, and sagittal directions.

In this way, the reason why coronal MRI image data and sagittal MRI image data are generated through image processing is because, as described above, generally disclosed brain MRI data is horizontal MRI image data.

However, depending on external environmental conditions such as the specifications of the imaging device, the published brain MRI data may include not only the horizontal direction, but also the sagittal and coronal planes.

In this case, the data pre-processing unit 200 analyzes the information included in each transmitted MRI data set (T1-weighted MRI set, T2-weighted MRI set, and second MRI data set), classifies it in the corresponding direction, and classifies each MRI data set. MRI data sets for each direction are generated.

For this purpose, the data input unit 100 uses the T1-W horizontal plane, T1-W coronal plane, T1-W sagittal plane, T2-W horizontal plane, T2-W coronal plane, and T2-W sagittal plane for the same brain region of the same patient. Two-dimensional MRI image data for each plane direction. It is also desirable for label data to be input as two-dimensional MRI image data for each of the horizontal, coronal, and sagittal directions.

That is, through the data input unit 100, the T1-W horizontal plane, T1-W coronal plane, T1-W sagittal plane, T2-W horizontal plane, T2-W coronal plane, and T2-W sagittal plane for pixels having the same coordinates. MRI image data for the label plane, label horizontal plane, label coronal plane, and label sagittal plane directions are all input, and the data preprocessor 200 simply classifies them.

For example, the A coordinate is input to MRI data in the T1-W horizontal plane direction, the B coordinate is input to MRI data in the labeled coronal plane direction, and the C coordinate is to input MRI data in the T2-W sagittal direction. In this case, if it is simply classified, it is insufficient to use it as data for deep learning learning.

Considering this, the data preprocessor 200 analyzes the information primarily included in each transmitted MRI data set (T1-weighted MRI set, T2-weighted MRI set, and second MRI data set) and , If MRI data for all desired directions are included, simple classification is performed and preprocessing is performed. If MRI data in any direction is missing, the 2-dimensional data are stacked to create each 3-dimensional data. , slicing is performed on each 3D data to generate data for each direction.

The deep learning learning unit 300 generates a first MRI data set for each direction (T1-W horizontal plane, T1-W coronal plane, T1-W sagittal plane, T2-W horizontal plane, T2-W coronal plane, and T2-W sagittal plane. 2-dimensional MRI image data for each plane direction) and a second MRI data set (2-dimensional MRI image data for each horizontal, coronal, and sagittal direction) are input and generated as learning data, and a pre-stored deep learning network is used. Learning processing for each direction is performed using this method.

In detail, as shown in FIG. 4, the deep learning learning unit 300 receives the first MRI data set and the second MRI data set for each direction from the data preprocessor 200 and Classify learning data (T1-W, T2-W in the horizontal direction, labeled MRI image data/T1-W, T2-W in the coronal direction, labeled MRI image data/T1-W, T2-W in the sagittal direction, label MRI image data) is generated.

Afterwards, the deep learning learning unit 300 performs learning processing in each direction to generate at least three learning models (horizontal plane learning model, coronal plane learning model, and sagittal plane learning model). At this time, it is preferable that the deep learning learning unit 300 applies a 2D U-Net-based learning network.

In addition, the learning model by the deep learning learning unit 300 performs learning processing by applying the softmax function as the final activation function for prediction, so that each class (background area) is generated for each pixel of the input image data. , the resulting data (predicted data) is derived in the form of a probabilistic map to be included in the GM area, WM area, and CSF area.

At this time, the deep learning learning unit 300 learns the model in a way to minimize the loss function between the result (prediction) and the correct answer.

In detail, the deep learning learning unit 300 performs learning processing for each direction through the learning data to generate at least three learning models, and then adds a second corresponding to each direction to the generated learning model for each direction. 1 By inputting the MRI image data set, result data (prediction data) is derived in the form of a probabilistic map. Afterwards, the learning model is updated to minimize the calculation of a preset loss function using the result data (prediction data) and the second MRI data set corresponding to each direction. Through this, the final learning model for each direction is created.

At this time, a hybrid function combining FL (Focal Loss) and GDL (Generalized Dice Loss) with the same weight is set as the loss function, as shown in Equation 1 below.

Loss function = FL + GDL

As shown in Figure 7, this loss function was selected as the most appropriate loss function based on the results of comparing the segmentation performance as a result of learning various loss functions.

In this way, after the final learning model for each direction is generated through the deep learning learning unit 300, the third MRI data set for dividing brain regions is input through the external data input unit 400. . As the third MRI data set, it is most desirable to receive a pair of T1-weighted MRI data sets and T2-weighted MRI data sets obtained by imaging the same area of the same patient. However, for only one of these, T1-weighted MRI data It is okay to input only three or T2-weighted MRI data sets.

Like the data preprocessor 200, the external data preprocessor 500 receives the third MRI data set through the external data input unit 400 and performs image processing to correspond to each preset direction. , a third MRI data set for each direction is generated.

In detail, considering that brain MRI data is usually horizontal MRI data, the external data preprocessor 500 stacks the third MRI data set transmitted through the external data input unit 400 to create a three-dimensional Data is generated, and slicing is performed on the 3D data in the horizontal, coronal, and sagittal directions to generate MRI data sets for each direction. At this time, if MRI data in the horizontal direction is input, it can be used as is to reduce the calculation process.

Through this, the preprocessed third MRI data set is, when both the T1-weighted MRI data set and the T2-weighted MRI data set are input, the T1-W horizontal plane, T1-W coronal plane, T1-W sagittal plane, T2-W horizontal plane, Two-dimensional MRI image data can be acquired for each of the T2-W coronal and T2-W sagittal directions, and when only the T1-weighted MRI data set or the T2-weighted MRI data set is input, the horizontal plane of the corresponding weighted image data, Two-dimensional MRI image data is acquired for each of the coronal and sagittal directions.

Of course, depending on external environmental conditions such as specifications of the imaging device, the transmitted third MRI data set may include both the sagittal and coronal planes as well as the horizontal plane direction. In this case, the external data preprocessor 500 analyzes the information included in the transmitted third MRI data set, classifies it into the corresponding direction, and generates a third MRI data set for each direction.

In this way, in order to perform a simple classification process, all MRI image data for the horizontal, sagittal, and coronal directions for pixels with the same coordinates must be input through the external data input unit 400.

The deep learning inference unit 600 receives the preprocessed third MRI data set for each direction, inputs it into a learning model for each direction (horizontal plane, sagittal plane, and coronal plane), and generates prediction data (results) for each direction. data) is output.

At this time, as described above, the deep learning inference unit 600 classifies each class (background area) for each pixel included in the third MRI data set for each direction preprocessed due to the softmax final activation function applied to the learning model. , GM area, WM area, and CSF area), in other words, the resulting data is output in the form of a probability map corresponding to each brain area.

In addition, as a result of comparing the segmentation performance by performing various learning processes in Figure 7, it was found that performing learning in all three directions and utilizing the results in a comprehensive manner was more effective than learning in any one direction alone. It can be seen that it shows performance.

For this purpose, the inference fusion unit 700 uses the integrated prediction data for each direction by the deep learning inference unit 600 to derive a brain region corresponding to each Paxil in the third MRI data set.

In detail, the inference fusion unit 700 averages the predicted data for each direction by the deep learning inference unit 600 for each pixel at the same location to select the class showing the highest probability for each pixel, that is, the most The final segmentation result for the corresponding pixel is derived as a brain region with a high probability.

That is, using the prediction data for each direction, for each pixel of the third MRI data set, the average of the probability corresponding to each brain region for each direction is calculated, and the brain region corresponding to the highest average value is calculated. The final brain area of the corresponding pixel is derived.

For example, as shown in FIG. 5, for the A pixel of the preprocessed third MRI data set, a first learning model (horizontal plane learning model), a second learning model (sagittal plane learning model), and a third learning model Predicted data is output from (coronal plane learning model), and the probability map values corresponding to each brain region included in each predicted data are averaged, that is, the predicted data of the first learning model (A pixel on the horizontal plane is Probability of corresponding to the background area, probability of corresponding to the GM area, probability of corresponding to the WM area, probability of corresponding to the CSF area), predicted data of the second learning model (probability of A pixel corresponding to the background area based on the sagittal plane) , probability that the pixel corresponds to the GM area, probability that it corresponds to the WM area, probability that it corresponds to the CSF area), predicted data of the third learning model (probability that the A pixel corresponds to the background area based on the coronal plane, probability that it corresponds to the GM area Probability, probability of corresponding to WM area, probability of corresponding to CSF area) are averaged for each area (probability that A pixel corresponds to background area (based on horizontal plane/sagittal plane/coronal plane), A pixel corresponds to GM region Probability of corresponding (horizontal/sagittal/coronal), probability that A pixel corresponds to WM area (horizontal/sagittal/coronal), probability that A pixel corresponds to CSF area (horizontal) /sagittal plane standard/coronal plane standard)) is calculated, and the final brain region of the pixel corresponding to the brain region corresponding to the highest average value is derived.

At this time, the inference fusion unit 700 performs an average calculation for each region under the condition that no error occurs in the prediction data for each direction by the deep learning inference unit 600.

However, because it cannot be 100% certain, the inference fusion unit 700 calculates the average of the remaining two probability map values, excluding the lowest probability map value for each pixel of each prediction data, if the difference is more than a predetermined value from the overall average. Calculate the average of the remaining two probability map values excluding the high probability map value, or the average of the remaining two probability map values excluding the low probability map value so that it differs by a predetermined value from the overall average, and calculate the brain corresponding to the highest average value. The final brain area of the pixel corresponding to the area is derived.

The analysis unit 800 performs segmentation processing for each brain region using the final brain region derived for each pixel by the inference fusion unit 700.

As shown in FIG. 1, the analysis unit 800 includes a segmentation processing unit 810 and a volume calculation unit 820.

The segmentation processing unit 810 performs segmentation processing on each brain region using the final brain region derived for each pixel by the inference fusion unit 700, and divides each brain region into become separated. This is equivalent to Equation 2 below.

The volume calculation unit 820 uses the number of pixels included in each segmented brain region to calculate the volume of each brain region, like the volume data in FIG. 6.

In detail, the volume calculation unit 820 performs a voxel conversion operation to convert the width, length, and height of the unit voxels of the 3D array of the third MRI data set on which the segmentation was performed to the same size, and then performs a 3D The volume ratio occupied by each brain region in the cast is calculated.

The volume ratio is the number of nodes corresponding to each area compared to the total number of nodes in the 3D array, and finally, the actual volume is calculated by multiplying the actual volume of the entire 3D array by the volume ratio of each area.

At this time, since the medical image basically includes voxel size information of the captured image, voxel conversion is performed using this.

That is, after deriving the number of pixels occupying each segmented brain region (GM, WM, CSF) as shown in Equation 3 below, it is basically included in the third MRI data set as shown in Equation 4 below. The actual scale volume is calculated by multiplying the actual pixel size information (dx, dy, dz) in the x, y, and z axes by the number of pixels in each brain region.

FIG. 8 is an example diagram showing a method for segmenting brain regions from brain MRI data using deep learning according to an embodiment of the present invention. Referring to FIG. 8, a method for segmenting brain regions from brain MRI data using deep learning is shown. is explained in detail.

The brain region segmentation method from brain MRI data using deep learning according to an embodiment of the present invention is a brain region segmentation system from brain MRI data using deep learning, in which each step is performed by computational processing means including a computer. This is a brain region segmentation method from brain MRI data using deep learning.

As shown in FIG. 8, the brain region segmentation method using deep learning according to an embodiment of the present invention includes a first input step (S100), a first preprocessing step (S200), and deep learning learning. It includes a step (S300), a second input step (S400), a second preprocessing step (S500), a deep learning inference step (S600), an inference fusion step (S700), a segmentation step (S800), and a volume calculation step (S900). It is composed in this way.

To learn more about each step,

In the first input step (S100), the data input unit 100 receives a first MRI data set for deep learning learning processing and a second MRI data set in which brain regions to be segmented are labeled.

The first MRI data set and the second MRI data set receive a public brain MRI data set, and T1-weighted image data, T2-weighted image data, and labeled image data are input for each patient.

In detail, the first MRI data set is two-channel MRI data, and a T1-weighted image and a T2-weighted image are input as a pair. That is, the first MRI data set includes a T1-weighted MRI set and a T2-weighted MRI set obtained by imaging the same brain region of the same patient. The second MRI data set is a data set in which the background area, GM area, WM area, and CSF area are labeled, and is preferably labeled data obtained from images of the same brain area of the same patient as the first MRI data set.

The first pre-processing step (S200) receives the first MRI data set and the second MRI data set from the first input step (S100) from the data pre-processing unit 200, and generates data corresponding to each preset direction. Image processing is performed to generate a first MRI data set and a second MRI data set for each direction.

In the first preprocessing step (S200), since the generally published brain MRI data is MRI data in the horizontal plane direction, as shown in FIG. 3, each transmitted MRI data set (T1-weighted MRI set, T2-weighted MRI set, For the second MRI data set), 2-dimensional data are stacked to create 3-dimensional data, and each 3-dimensional data is sliced in the horizontal, coronal, and sagittal directions to produce MRI data for each direction. This creates three. At this time, if MRI data in the horizontal direction is input, it can be used as is to reduce the calculation process.

Through this, the preprocessed first MRI data set is two-dimensional for each of the T1-W horizontal plane, T1-W coronal plane, T1-W sagittal plane, T2-W horizontal plane, T2-W coronal plane, and T2-W sagittal plane. MRI image data can be acquired, and the preprocessed second MRI data set also acquires two-dimensional MRI image data for each of the horizontal, coronal, and sagittal directions.

In this way, the reason why coronal MRI image data and sagittal MRI image data are generated through image processing is because, as described above, generally disclosed brain MRI data is horizontal MRI image data.

However, depending on external environmental conditions such as the specifications of the imaging device, the published brain MRI data may include not only the horizontal direction, but also the sagittal and coronal planes.

In this case, the first preprocessing step (S200) analyzes the information included in each transmitted MRI data set (T1-weighted MRI set, T2-weighted MRI set, and second MRI data set) and classifies it in the corresponding direction. MRI data sets for each direction are generated.

That is, through the first input step (S100), the T1-W horizontal plane, T1-W coronal plane, T1-W sagittal plane, T2-W horizontal plane, T2-W coronal plane, and T2-W for pixels having the same coordinates. When MRI image data for the W sagittal plane, label horizontal plane, label coronal plane, and label sagittal plane direction are all input, the first preprocessing step (S200) simply classifies them.

For example, the A coordinate is input to MRI data in the T1-W horizontal plane direction, the B coordinate is input to MRI data in the labeled coronal plane direction, and the C coordinate is to input MRI data in the T2-W sagittal direction. In this case, if it is simply classified, it is insufficient to use it as data for deep learning learning.

Taking this into consideration, the information primarily included in each delivered MRI data set (T1-weighted MRI set, T2-weighted MRI set, and second MRI data set) is analyzed to obtain MRI data in all desired directions. If included, simple classification is performed and preprocessing is performed. If MRI data is missing in any direction, the 2-dimensional data are stacked to create 3-dimensional data, and slicing is performed on each 3-dimensional data. is performed to generate data for each direction.

In the deep learning learning step (S300), the deep learning learning unit 300 generates a first MRI data set for each direction (T1-W horizontal plane, T1-W coronal plane, T1-W sagittal plane, T2-W horizontal plane, Learning data by receiving input of the second MRI data set (two-dimensional MRI image data for each of the T2-W coronal and T2-W sagittal directions) and the second MRI data set (two-dimensional MRI image data for each of the horizontal, coronal, and sagittal directions) It is created and learning processing for each direction is performed using a pre-stored deep learning network.

In detail, as shown in FIG. 4, the first MRI data set and the second MRI data set for each direction are input, classified by direction, and training data (T1-W, T2-W in the horizontal direction, label MRI Image data/T1-W, T2-W in the coronal direction, label MRI image data/T1-W, T2-W, label MRI image data in the sagittal direction) are generated.

Afterwards, learning processing is performed for each direction to generate at least three learning models (horizontal plane learning model, coronal plane learning model, and sagittal plane learning model). At this time, it is preferable that the deep learning learning unit 300 applies a 2D U-Net-based learning network.

At this time, by performing learning processing by applying the softmax function as the final activation function for prediction, the learning model creates each class (background area, GM area, WM area, and CSF area) for each pixel of the input image data. ), the result data (predicted data) is derived in the form of a probabilistic map.

In the deep learning learning step (S300), the model is learned to minimize the loss function between the result (prediction) and the correct answer.

In detail, at least three learning models are generated by performing learning processing for each direction through the learning data, and then inputting the first MRI image data set corresponding to each direction into the generated learning model for each direction to determine the probability. Result data (predicted data) is derived in the form of a map (probabilistic map). Afterwards, the learning model is updated to minimize the calculation of a preset loss function using the result data (prediction data) and the second MRI data set corresponding to each direction. Through this, the final learning model for each direction is created.

At this time, a hybrid function combining FL (Focal Loss) and GDL (Generalized Dice Loss) with the same weight is set as the loss function, as shown in Equation 1 above.

In this way, after the final learning model for each direction is generated, the second input step (S400) receives the third MRI data set for dividing brain regions from the external data input unit 400. As the third MRI data set, it is most desirable to receive a pair of T1-weighted MRI data sets and T2-weighted MRI data sets obtained by imaging the same area of the same patient. However, for only one of these, T1-weighted MRI data It is okay to input only three or T2-weighted MRI data sets.

The second pre-processing step (S500) receives the third MRI data set from the second input step (S400) from the external data pre-processing unit 500 and performs image processing to correspond to each preset direction. Thus, a third MRI data set for each direction is generated.

In detail, considering that brain MRI data is usually MRI data in the horizontal direction, the transmitted third MRI data set is stacked to create 3-dimensional data, and the 3-dimensional data is divided into horizontal, coronal, and sagittal directions. By performing slicing in the plane direction, MRI data sets for each direction are created. At this time, if MRI data in the horizontal direction is input, it can be used as is to reduce the calculation process.

Through this, the preprocessed third MRI data set is, when both the T1-weighted MRI data set and the T2-weighted MRI data set are input, the T1-W horizontal plane, T1-W coronal plane, T1-W sagittal plane, T2-W horizontal plane, Two-dimensional MRI image data can be acquired for each of the T2-W coronal and T2-W sagittal directions, and when only the T1-weighted MRI data set or the T2-weighted MRI data set is input, the horizontal plane of the corresponding weighted image data, Two-dimensional MRI image data is acquired for each of the coronal and sagittal directions.

Of course, depending on external environmental conditions such as specifications of the imaging device, the transmitted third MRI data set may include both the sagittal and coronal planes as well as the horizontal plane direction. In this case, the information included in the transmitted third MRI data set is analyzed and classified into corresponding directions to generate a third MRI data set for each direction.

In this way, in order to perform a simple classification process, all MRI image data for the horizontal, sagittal, and coronal directions for pixels with the same coordinates must be input through the second input step (S400).

The deep learning inference step (S600) receives the third MRI data set for each direction preprocessed by the second preprocessing step (S500) and applies each to a learning model for each direction (horizontal, sagittal, and coronal planes). By inputting, predicted data (result data) for each direction is output.

At this time, the deep learning inference step (S600) is performed for each pixel included in the third MRI data set for each direction preprocessed due to the softmax final activation function applied to the learning model. area and CSF area), in other words, the resulting data is output in the form of a probability map corresponding to each brain region.

In the inference fusion step (S800), the inference fusion unit 700 integrates and uses the prediction data for each direction by the deep learning inference step (S600) to generate the third input signal by the second input step (S400). The brain region corresponding to each Paxil in the MRI data set is derived.

In detail, the inference fusion step (S800) averages the prediction data for each direction for each pixel at the same location, and selects the pixel as the class showing the highest probability for each pixel, that is, the brain region with the highest probability. The final division result is derived.

That is, using the prediction data for each direction, for each pixel of the third MRI data set, the average of the probability corresponding to each brain region for each direction is calculated, and the brain region corresponding to the highest average value is calculated. The final brain area of the corresponding pixel is derived.

For example, as shown in FIG. 5, for the A pixel of the preprocessed third MRI data set, a first learning model (horizontal plane learning model), a second learning model (sagittal plane learning model), and a third learning model Predicted data is output from (coronal plane learning model), and the probability map values corresponding to each brain region included in each predicted data are averaged, that is, the predicted data of the first learning model (A pixel on the horizontal plane is Probability of corresponding to the background area, probability of corresponding to the GM area, probability of corresponding to the WM area, probability of corresponding to the CSF area), predicted data of the second learning model (probability of A pixel corresponding to the background area based on the sagittal plane) , probability that the pixel corresponds to the GM area, probability that it corresponds to the WM area, probability that it corresponds to the CSF area), predicted data of the third learning model (probability that the A pixel corresponds to the background area based on the coronal plane, probability that it corresponds to the GM area Probability, probability of corresponding to WM area, probability of corresponding to CSF area) are averaged for each area (probability that A pixel corresponds to background area (based on horizontal plane/sagittal plane/coronal plane), A pixel corresponds to GM region Probability of corresponding (horizontal/sagittal/coronal), probability that A pixel corresponds to WM area (horizontal/sagittal/coronal), probability that A pixel corresponds to CSF area (horizontal) /sagittal plane standard/coronal plane standard)) is calculated, and the final brain region of the pixel is derived as the brain region corresponding to the highest average value.

At this time, the inference fusion step (S800) performs an average calculation for each region under the condition that no error occurs in the prediction data for each direction by the deep learning inference step (S600).

However, because it cannot be 100% certain, the inference fusion step (S800) is performed by excluding the lowest probability map value for each pixel from each prediction data according to the selection of an external user (manager, engineer, etc.). The average of the probability map values, the average of the remaining two probability map values excluding the high probability map value that differs by a predetermined value from the overall average, or the remaining two probability map values excluding the low probability map value that differs by a predetermined value from the overall average. By calculating the average, the final brain region of the pixel corresponding to the brain region corresponding to the highest average value is derived.

In the segmentation step (S800), the analysis unit 800 performs segmentation processing of each brain region using the final brain region derived for each pixel through the inference fusion step (S700). That is, segmentation processing is performed on each brain region using the final brain region derived for each pixel, and each brain region is separated, as shown in the prediction data in FIG. 6. This is the same as Equation 2 above.

The volume calculation step (S900) uses the number of pixels included in each brain region divided by the segmentation step (S800) in the analysis unit 800, as in the volume data of FIG. 6, for each brain region. The volume of the brain region is calculated.

In detail, the volume calculation step (S900) performs a voxel conversion operation to convert the width, length, and height of the unit voxels forming the 3D array of the third MRI data set on which the segmentation was performed to the same size, The volume ratio occupied by each brain region in the 3D model is calculated.

The volume ratio is the number of nodes corresponding to each area compared to the total number of nodes in the 3D array, and finally, the actual volume is calculated by multiplying the actual volume of the entire 3D array by the volume ratio of each area.

At this time, since the medical image basically includes voxel size information of the captured image, voxel conversion is performed using this.

That is, after deriving the number of pixels occupying each divided brain region (GM, WM, CSF) as in Equation 3 above, it is basically included in the third MRI data set as in Equation 4 above. The actual scale volume is calculated by multiplying the actual pixel size information (dx, dy, dz) in the x, y, and z axes by the number of pixels in each brain region.

As described above, the present invention has been described with reference to specific details such as specific components and limited embodiment drawings, but this is only provided to facilitate a more general understanding of the present invention, and the present invention is not limited to the above-mentioned embodiment. No, those skilled in the art can make various modifications and variations from this description.

Therefore, the spirit of the present invention should not be limited to the described embodiments, and all matters that are equivalent or equivalent to the claims of this patent as well as the claims described below shall fall within the scope of the spirit of the present invention. .

100: data input unit
200: data preprocessing unit
300: Deep learning learning department
400: External data input unit
500: External data preprocessing unit
600: Deep learning inference unit
700: Inference fusion unit
800: analysis department
810: Division processing unit 820: Volume calculation unit

Claims (14)

A data input unit 100 that receives a first MRI data set for deep learning learning processing and a second MRI data set in which brain regions to be segmented are labeled;
A data preprocessor that receives the first MRI data set and the second MRI data set, performs image processing to correspond to each preset direction, and generates a first MRI data set and a second MRI data set for each direction. 200);
A deep learning learning unit (which receives the first MRI data set and the second MRI data set for each direction and generates them as learning data, performs learning processing using a pre-stored deep learning network, and generates a learning model for each direction) 300);
An external data input unit 400 that receives a third MRI data set for dividing brain regions;
an external data preprocessor 500 that receives the third MRI data set, performs image processing to correspond to each preset direction, and generates a third MRI data set for each direction;
A deep learning inference unit 600 that receives the third MRI data set for each direction, inputs it into a learning model for each direction, and outputs prediction data for each direction;
an inference fusion unit 700 that derives a brain region corresponding to each pixel of the third MRI data set by integrating prediction data for each direction; and
An analysis unit 800 that performs segmentation processing on each brain region derived using each pixel information;
Including,
The first MRI data set is
A brain region segmentation system from brain MRI data using deep learning, including a T1-weighted MRI data set and a T2-weighted MRI data set of the same region.
According to clause 1,
The data pre-processing unit 200 and the external data pre-processing unit 500
3D data is generated for each transmitted MRI data set, and slicing is performed for each 3D data in the horizontal, coronal, and sagittal directions to produce MRI data for each direction. A brain region segmentation system from brain MRI data using deep learning that generates a set.
According to clause 1,
The data pre-processing unit 200 and the external data pre-processing unit 500
The information contained in each transmitted MRI data set is analyzed and classified into the corresponding horizontal, coronal, and sagittal directions to generate MRI data sets for each direction. Brain region segmentation system from brain MRI data using deep learning.
According to clause 1,
The deep learning learning unit 300
Updating the learning model for each direction to minimize the calculation of prediction data for each direction using the first MRI data set and a preset loss function using the second MRI data set, and generating a final learning model for each direction, Brain region segmentation system from brain MRI data using deep learning.
According to clause 4,
The deep learning learning unit 300
In the deep learning network, learning processing is performed by applying the softmax function as an activation function,
The deep learning inference unit 600
A brain region segmentation system from brain MRI data using deep learning that outputs the predicted data in the form of a probability map corresponding to each brain region for each pixel of the third MRI data set in each direction.
According to clause 5,
The inference fusion unit 700 is
Using predicted data for each direction,
For each pixel of the third MRI data set, calculating the average of the probability of corresponding to each brain region in each direction, and deriving the final brain region of the pixel corresponding to the brain region corresponding to the highest average value. , Brain region segmentation system from brain MRI data using deep learning.
According to clause 5,
The analysis unit 800 is
A segmentation processing unit 810 that performs segmentation processing of each brain region using each pixel information; and
a volume calculation unit 820 that calculates the volume of each brain region using the number of pixels included in each segmented brain region;
A brain region segmentation system from brain MRI data using deep learning, including.
According to clause 7,
The volume calculation unit 820 is
Deep learning is used to calculate the actual scale volume of each brain region by reflecting the actual pixel size information for each direction included in the third MRI data set in the number of pixels included in each segmented brain region. A brain region segmentation system from one brain MRI data.
A brain region segmentation method from brain MRI data using deep learning using a brain region segmentation system from brain MRI data using deep learning in which each step is performed by computational processing means including a computer, comprising:
A first input step (S100) of receiving a first MRI data set for deep learning learning processing and a second MRI data set in which brain regions to be segmented are labeled;
First preprocessing to receive the MRI data set through the first input step (S100), perform image processing to correspond to each preset direction, and generate a first MRI data set and a second MRI data set for each direction. Step (S200);
In the pre-stored deep learning network, the first MRI data set and the second MRI data set for each direction in the first preprocessing step (S200) are input as learning data, and learning processing is performed to generate a learning model for each direction. deep learning learning step (S300);
A second input step (S400) of receiving a third MRI data set to be divided into brain regions;
A second preprocessing step (S500) of receiving the MRI data set from the second input step (S400), performing image processing to correspond to each preset direction, and generating a third MRI data set for each direction;
Using the third MRI data set for each direction in the second preprocessing step (S500), inputting it into the learning model for each direction in the deep learning learning step (S300) to output prediction data for each direction Deep learning inference step (S600);
An inference fusion step (S700) of deriving a brain region corresponding to each pixel of the third MRI data set from the second input step (S400) by integrating the prediction data for each direction;
A segmentation step (S800) of performing segmentation processing on each brain region using each pixel information derived from the inference fusion step (S700); and
A volume calculation step (S900) of calculating the volume of each brain region using the number of pixels included in each brain region divided by the division step (S800);
Including,
A brain region segmentation method from brain MRI data using deep learning, wherein the first MRI data set includes a T1-weighted MRI data set and a T2-weighted MRI data set obtained by imaging the same region.
According to clause 9,
The first preprocessing step (S200) and the second preprocessing step (S500) are
3D data is generated for each transmitted MRI data set, and slicing is performed for each 3D data in the horizontal, coronal, and sagittal directions to produce MRI data for each direction. Brain region segmentation method from brain MRI data using deep learning to generate sets.
According to clause 9,
The first preprocessing step (S200) and the second preprocessing step (S500) are
The information contained in each transmitted MRI data set is analyzed and classified into the corresponding horizontal, coronal, and sagittal directions to generate MRI data sets for each direction. Brain region segmentation method from brain MRI data using deep learning.
According to clause 9,
The deep learning learning step (S300) is
Learning processing is performed repeatedly to minimize the calculation of the prediction data for each direction using the first MRI data set and the preset loss function using the second MRI data set for each direction in the first preprocessing step (S200). , create a learning model for each direction,
A brain region segmentation method from brain MRI data using deep learning, which performs learning processing by applying the softmax function as an activation function.
According to clause 12,
The deep learning inference step (S600) is
A brain region segmentation method from brain MRI data using deep learning, which outputs the predicted data in the form of a probability map corresponding to each brain region for each pixel of the third MRI data set for each direction.
According to clause 13,
The inference fusion step (S700) is
Using predicted data for each direction,
For each pixel of the third MRI data set, calculating the average of the probability of corresponding to each brain region in each direction, and deriving the final brain region of the pixel corresponding to the brain region corresponding to the highest average value. , Brain region segmentation method from brain MRI data using deep learning.