CN110853048A - MRI image segmentation method, device and storage medium based on rough training and fine training - Google Patents

MRI image segmentation method, device and storage medium based on rough training and fine training Download PDF

Info

Publication number
CN110853048A
CN110853048A CN201910973756.7A CN201910973756A CN110853048A CN 110853048 A CN110853048 A CN 110853048A CN 201910973756 A CN201910973756 A CN 201910973756A CN 110853048 A CN110853048 A CN 110853048A
Authority
CN
China
Prior art keywords
training
segmentation
pixel
module
neural network
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
CN201910973756.7A
Other languages
Chinese (zh)
Inventor
郭宸芸
朱程
董家鸿
葛均波
赵宏
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Beijing Jinrong Medical Technology Co ltd
Original Assignee
Beijing Jinrong Medical Technology Co ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Beijing Jinrong Medical Technology Co ltd filed Critical Beijing Jinrong Medical Technology Co ltd
Priority to CN201910973756.7A priority Critical patent/CN110853048A/en
Publication of CN110853048A publication Critical patent/CN110853048A/en
Pending legal-status Critical Current

Links

Images

Classifications

    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06TIMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00Image analysis
    • G06T7/10Segmentation; Edge detection
    • G06T7/11Region-based segmentation
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06TIMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00Image analysis
    • G06T7/10Segmentation; Edge detection
    • G06T7/181Segmentation; Edge detection involving edge growing; involving edge linking
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06TIMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00Image analysis
    • G06T7/60Analysis of geometric attributes
    • G06T7/62Analysis of geometric attributes of area, perimeter, diameter or volume
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06TIMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00Indexing scheme for image analysis or image enhancement
    • G06T2207/10Image acquisition modality
    • G06T2207/10072Tomographic images
    • G06T2207/10088Magnetic resonance imaging [MRI]
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06TIMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00Indexing scheme for image analysis or image enhancement
    • G06T2207/20Special algorithmic details
    • G06T2207/20081Training; Learning
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06TIMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00Indexing scheme for image analysis or image enhancement
    • G06T2207/20Special algorithmic details
    • G06T2207/20084Artificial neural networks [ANN]
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06TIMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00Indexing scheme for image analysis or image enhancement
    • G06T2207/30Subject of image; Context of image processing
    • G06T2207/30004Biomedical image processing
    • G06T2207/30016Brain

Landscapes

  • Engineering & Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • General Physics & Mathematics (AREA)
  • Theoretical Computer Science (AREA)
  • Geometry (AREA)
  • Image Analysis (AREA)
  • Magnetic Resonance Imaging Apparatus (AREA)

Abstract

The invention discloses an MRI image segmentation method based on coarse and fine training, which comprises the steps of segmenting an MRI image by segmentation software, taking a segmentation result as auxiliary label data, and performing coarse training on a full convolution neural network by adopting a large number of samples with auxiliary labels; carrying out fine training on the full convolution neural network by adopting a small data sample with an artificial labeling label; in the course of rough training and fine training, adding weight in logic loss terms of the loss function; and the trained neural network acts on the MRI image needing to be segmented to realize image segmentation. The invention also provides an MRI image segmentation device based on coarse and fine training and a computer readable storage medium. By using the method and the device, the segmentation precision can be improved under the condition that training data manually marked are limited.

Description

MRI image segmentation method, device and storage medium based on rough training and fine training
Technical Field
The invention relates to the technical field of image segmentation and deep learning of computer vision, in particular to an MRI image segmentation method and device based on coarse and fine training and a computer readable storage medium.
Background
Magnetic Resonance Imaging (MRI) can provide detailed in vivo information for the study of human brain morphology, which is crucial for the study of development, aging and disease. In order to assess the volume, thickness and shape of a certain measurement structure, neuroanatomy requires segmentation of the original image, which is very time consuming for manual segmentation. Based on the development of computers, people generally adopt a method of mapping a target to be scanned after manual segmentation so as to realize automatic segmentation. However, such a method has two significant disadvantages: (i) estimating the 3D field vectors used for mapping would be very computer-intensive, (ii) the lack of homology would lead to errors for cortical segmentation. Based on this, the existing mapping method not only requires a very long processing time, but also cannot obtain an optimal solution. This results in the annotation of the image being available long after data acquisition, which limits the development of the overall morphological analysis.
Deep learning has had unprecedented success over the past few years, but significant training requires large amounts of annotated data. In the field of computer vision, image semantic segmentation is always dominated by F-CNN (full convolutional neural network). A major challenge in extending the F-CNN model to the field of complex human tissue segmentation, such as brain images, comes from the limited nature of the available manually labeled training data.
Disclosure of Invention
In view of the above, the invention provides a brain MRI image segmentation method based on rough mental training through network training, which can improve the segmentation accuracy under the condition that training data manually labeled is limited.
In order to solve the technical problem, the invention is realized as follows:
an MRI image segmentation method comprising:
segmenting the tissue type of the MRI image by using segmentation software, taking a segmentation result as auxiliary label data, and performing coarse training on the full convolution neural network by using a sample with an auxiliary label;
using a rough training network as initialization, and adopting a sample with an artificial labeling label to carry out fine training on the full convolution neural network;
and the trained full convolution neural network acts on the MRI image needing to be segmented to realize image segmentation.
Preferably, in the course of coarse training and fine training, a weight ω (x) is added to the logic loss term of the loss function; the weight ω (x) of the pixel x is related to the volume of the tissue class in which the pixel x is located, the larger the volume, the lower the weight; ω (x) is also related to whether pixel x is located at a class edge, where the pixel weight is greater than the pixel weight that is not located at the edge.
Preferably, the weight ω (x) is obtained by:
calculating the number of pixels of each tissue class aiming at a single volume pixel as a sample, obtaining the median med of the number of pixels of all the tissue classes, and then obtaining the first part weight omega of the pixel x1(x) Dividing the median med of the pixel number by the pixel number count _ i of the tissue class i in which x is located;
for the single body pixel, calculating an edge position through a gradient; for the pixel x at the edge position, the first partial weight value ω is weighted1(x) Multiplying by a set multiple to obtain the weight omega (x) of the pixel.
Preferably, the Loss function is a joint Loss function including a logic Loss Logistic Loss and a similarity coefficient Loss Dice Loss.
Preferably, the segmentation software employs FreeSprofer software.
Preferably, the MRI image is segmented into three-dimensional views along the coronal, axial and sagittal directions, respectively, each dimension is segmented by a set of fully convolutional neural networks, and finally the segmentation results of the three dimensions are summarized to obtain the image segmentation result.
Preferably, the segmentation results of the three dimensions are collected by a weighted sum method.
Preferably, the sagittal and coronal weights are set to be less than the axial weights.
Preferably, the full convolution neural network adopts a U-net structure, and the encoder and the decoder both adopt multilayer dense connection modules; the pooling layer is followed after the dense connection module of the encoder, and the upper sampling layer is connected before the dense connection module of the decoder; the dense connection module comprises three convolution layers c, wherein each convolution layer is preceded by an example normalization layer n; both of the first two convolutional layers are followed by a connection layer for connecting the input signature of the current convolutional layer with the input signatures of all preceding stages of convolutional layers.
Preferably, the MRI image is a brain MRI image.
The invention also provides an MRI image segmentation device, which comprises an automatic segmentation module, a full convolution neural network module and a training module;
an automatic segmentation module configured to segment the tissue type of the MRI image using segmentation software, the segmentation result being used as auxiliary tag data;
the training module is configured for carrying out coarse training on the full convolution neural network by adopting a sample with an auxiliary label; using a rough training network as initialization, and adopting a sample with an artificial labeling label to carry out fine training on the full convolution neural network;
and after the training of the full convolution neural network in the full convolution neural network module is finished, the full convolution neural network acts on the MRI image to be segmented to realize image segmentation.
Preferably, the training module adds a weight ω (x) to the logic loss term of the loss function during the course of the coarse training and the fine training; the weight ω (x) of the pixel x is related to the volume of the tissue class in which the pixel x is located, the larger the volume, the lower the weight; ω (x) is also related to whether pixel x is located at a class edge, where the pixel weight is greater than the pixel weight that is not located at the edge.
Preferably, the Loss function adopted by the training module in the training is a joint Loss function including a logic Loss logistic Loss and a similarity coefficient Loss Dice Loss.
Preferably, the automatic segmentation module adopts the segmentation software of FreeSprofer software.
Preferably, the full convolution neural network module comprises a three-dimensional segmentation module, a summarization module and three sets of the same full convolution neural network;
the three-dimensional segmentation module segments the MRI image into three-dimensional views along the coronal, axial and sagittal directions respectively; and each dimension is segmented by utilizing a set of full-convolution neural network, and finally, the segmentation results of the three dimensions are summarized by the summarizing module to obtain an image segmentation result.
Preferably, the summarizing module performs weighting and calculation on the segmentation results of the three dimensions, and the sagittal weight and the coronal weight are set to be smaller than the axial weight.
Preferably, the full convolution neural network adopts a U-net structure, and the encoder and the decoder both adopt multilayer dense connection modules; the pooling layer is followed after the dense connection module of the encoder, and the upper sampling layer is connected before the dense connection module of the decoder; the dense connection module comprises three convolution layers c, wherein each convolution layer is preceded by an example normalization layer n; both of the first two convolutional layers are followed by a connection layer for connecting the input signature of the current convolutional layer with the input signatures of all preceding stages of convolutional layers.
The present invention also provides a computer-readable storage medium having stored thereon a computer program which, when executed by a processor, implements the aforementioned MRI image segmentation method.
Has the advantages that:
(1) the invention adopts a rough and fine training method to train the neural network, and rough pre-training can provide good prior initialization for the neural network, so that the scarce manual label can be better utilized to obtain high-precision segmentation.
(2) The invention also improves the loss function used in coarse and fine network training, and adds weight omega (x) in the logic loss term to balance relative importance between pixels. The problem of class imbalance is compensated by increasing the proportion of classes in the image, and meanwhile, the weight of the anatomical edge region is increased to ensure that the contour can be correctly segmented. In a preferred embodiment, the ratio weight and the edge weight are multiplied and superimposed, so that the superimposed result of the weights does not exceed 1, and the superimposed result does not need to be further processed.
(3) The present invention does not divide the image into patches (patch), but rather divides the MRI image into three dimensions of 2D views along the coronal, axial, and sagittal directions, respectively, for separate segmentation and then re-aggregation. The speed of processing aiming at the 2D view is faster, different weights can be set aiming at each direction, and the segmentation accuracy is improved.
(4) For brain MRI, tissues can be better represented in the axial direction, namely, the quantity and the form distribution of each tissue in certain planes are uniform, so the sagittal weight and the coronal weight are set to be smaller than the axial weight, and the information of axial pictures is highlighted.
(5) The F-CNN of the invention adopts dense connection modules, and the dense connection can improve the gradient trend in the training process and promote the reusability of the characteristics in different convolution stages. In addition, the dense connection can also enable the characteristics of the same module to obtain different results when being learned by different convolutional layers, and the performance is better.
(6) Each densely connected module of the decoder is preceded by an upsampling layer, which has the advantage over the convolutional conversion used in U-net that it does not require any learned parameters.
Drawings
FIG. 1 is a flow chart of a brain MRI image segmentation method according to an embodiment;
FIG. 2 is a schematic diagram of segmenting an MRI image into three-dimensional 2D views according to a second embodiment;
FIG. 3 shows the result of the classification of MRI images in one direction;
FIG. 4 is a schematic diagram of an embodiment of a three-full convolution neural network;
FIG. 5 is a schematic view of the densely-connected module of FIG. 4;
FIG. 6 is a schematic diagram of a four-MRI image segmentation apparatus according to an embodiment;
fig. 7 is a block diagram of the full convolutional neural network module in fig. 6.
Detailed Description
The invention is described in detail below by way of example with reference to the accompanying drawings. The following embodiments take a brain MRI image as a segmentation object, and the invention is also suitable for the segmentation of other tissues with complicated outlines of human bodies.
Example one
For a brain scan image I of an MRI, it is desired to obtain its segmentation map S, which contains n cortical and subcutaneous tissues, each segmentation map corresponding to a tissue class. The invention relates to a method for obtaining a group of scanning images I ═ { I1, I2, … …, In } and corresponding segmentation images S ═ { S1, S2 … …, Sn }, and then learning to obtain a mapping relation: i → S. The invention adopts a full convolution neural network (F-CNN) model to approximate the mapping relation. Therefore, the input of the F-CNN is an MRI image, the output is a classification result, and each pixel in the output result is the probability of whether the pixel belongs to a certain class.
The brain MRI image segmentation method provided in this embodiment, as shown in fig. 1, includes the following steps:
firstly, the MRI image is segmented by segmentation software, the segmentation result is used as auxiliary label data, and a large number of samples with auxiliary labels are adopted to carry out coarse training on the full convolution neural network.
In this step, the MRI image is segmented without annotation using currently existing segmentation software, such as freesrush, which performs image preprocessing by referring to the automatically generated segmentation data as auxiliary label data, which normalizes the data and completes the segmentation in about 1 s.
The neural network is pre-trained with data with auxiliary labels. The auxiliary labels may not be as accurate as the expert labels, but rather allow supervised neural network training with large amounts of initially unlabeled data, and the auxiliary labels also make the neural network suitable for a large number of morphological changes of different human brain tissues.
And (II) utilizing the rough training network as initialization, and adopting a small amount of samples with manual labeling labels to carry out fine training on the full convolution neural network. The small amounts here are large amounts compared to step one.
The pre-trained neural network obtained in the first step is finely adjusted (continuously trained) by using a relatively small amount of manually-identified data. The more accurate the manual identification, the better the fine training effect of the second step.
The rough and fine training method can provide good prior initialization for the neural network, so that scarce manual labels can be better utilized to obtain high-precision segmentation.
The invention also improves the loss function used in the coarse and fine network training, and adds the weight omega (x) in the logic loss term; the weight ω (x) of the pixel x is related to the volume of the tissue class in which the pixel x is located, the larger the volume, the lower the weight; ω (x) is also related to whether pixel x is located at a class edge, where the pixel weight is greater than the pixel weight that is not located at the edge.
In this embodiment, when training the full convolution network proposed by the present invention, the Loss function is composed of two terms, which are a logic Loss term Logistic Loss and a similarity coefficient Loss term Dice Loss, respectively.
With the probability pl (x) that the pixel x belongs to the class l and the class gl (x) where its true belongs, the penalty function can be expressed as:
Figure BDA0002232956800000071
in the above equation, the first term is the Logistic Loss, which provides a probability value of the similarity of the estimated label and the manually labeled label at the pixel level. The second term is the multiple-layer Dice loss. Dice loss is derived from the Dice coincidence and also represents the similarity of the estimate to the manually labeled label. Dice loss was originally introduced for two classes and will be applied to multiple classes in the present invention.
The invention improves the logic Loss term Logistic Loss, and adds the weight omega (x) to balance the relative importance among pixels. The weighting terms are introduced primarily for the following two challenges:
(a) imbalance between classes: therefore, the weight ω (x) of the pixel x is related to the volume of the tissue class in which the pixel x is located, and the larger the volume is, the more representative samples are, the easier it is to train relatively, and therefore the weight is lower; compensating the class imbalance problem by increasing the few classes in the image;
(b) errors in anatomical edge segmentation: therefore, ω (x) is also related to whether the pixel x is located at a class edge, where the pixel weight is greater than the pixel weight that is not located at the edge. By increasing the weight of the anatomical edge region, it is ensured that the contour can be correctly segmented.
In this embodiment, a median frequency balancing (median frequency balancing) is used to obtain ω (x), and the flow is as follows:
(1) inputting single volume pixel data, wherein the volume pixel refers to MRI data;
(2) calculating the number of pixels for each class label;
(3) finding the median med of the pixel number in all the category labels;
(4) first partial weight ω for each class label1(x) Namely: dividing the median value of all the categories by the number count _ i occupied by the category i, namely med/count _ i; then for the pixel x of the ith class, the weight thereof adopts med/count _ i;
(5) for the corresponding pixel of the image to be segmented, using med/count _ i to replace a category label, namely a first part of weight, and obtaining a preliminary weight map;
(6) then, the gradient of the input volume pixel data is calculated;
(7) utilization (grad (h)2+grad(v)2)>0 pixel position, finding the tissue edge; grad (h) represents the horizontal gradient of the pixel, and grad (v) represents the vertical gradient of the pixel. For digitized images specifically: the gradient in a certain direction can be obtained by a differential method, namely, grad (h) -w (h, v) -w (h-1, v); grad (v) ═ w (h, v) -w (h, v-1). w (h, v) represents the pixel of the h row and v column in the preliminary weighted image.
(8) And (4) indexing the preliminary weight map obtained in the step (5) by utilizing the edge position found in the step (7), wherein the weight value of the edge position is correspondingly increased by n times, and n is 1.5 times after the experiment.
(9) The final result w (x) is the weight map.
And (III) the trained neural network acts on the MRI image needing to be segmented to realize image segmentation.
The rapid neuroanatomy segmentation method based on the depth and complete convolution neural network utilizes the GPU, can shorten the segmentation time from a plurality of hours required by the current mapping-based method to a plurality of seconds, achieves the improvement of a plurality of orders of magnitude on the speed, and can bring wide influence on neuroimages. When processing larger data, the processing can be completed by using a single GPU workstation without a cluster; the numerous morphological measurements obtained from one scan take only a few seconds. Further, such a fast speed allows a variety of segmentations to be done in a limited time and thus can be used as an estimate of the segmentation uncertainty in the automatic quality control. Besides the speed, the method provided by the invention can also achieve the purpose of accurate segmentation by analyzing data of different age groups, different magnetic field strengths and different causes. Finally, the multiple measurement accuracy exhibited by the method proposed by the present invention makes it particularly beneficial for radial analysis.
Example two
In this embodiment, the image is not divided into small blocks (patch), but as shown in fig. 2, the MRI image is divided into three-dimensional 2D views along the coronal, axial and sagittal directions, each dimension corresponds to a set of full convolution neural networks, and in the multi-view fusion stage, the three-dimensional segmentation results are merged together to complete the final segmentation.
The segmentation results of the three dimensions can be summarized in a weighted sum mode. For a pixel at the same position in each dimension, such as a small square in fig. 2, the probability that the pixel belongs to each class can be obtained by image segmentation of the full convolution neural network. Fig. 3 shows the classification result in one direction, with different gray levels representing different tissue classes. And weighting and calculating the pixel segmentation results of the three dimensions, wherein the classification corresponding to the maximum probability is the classification of the pixel.
The final probability vector of pixel x is represented by p (x), and is specifically calculated as follows:
P(x)=λ1×pAx(x)+λ2×pCor(x)+λ3×pSag(x)
wherein, pAx (x), pCor (x) and pSag (x) are probability vectors respectively estimated under axial, coronal and sagittal views, lambda1、λ2And λ3For their respective weights. Integrating all estimates for pixel x is equivalent to providing a regularization effect for the tag estimates, reducing false estimates. P (x) the class corresponding to the maximum value in the vector is the class of the pixel x.
Due to the symmetry of the brain, it is not possible to distinguish whether a layer in the sagittal view comes from the left hemisphere or the posterior hemisphere, which makes the segmentation of the sagittal view difficult. In contrast, for brain MRI, the axial direction is relatively better to represent the tissues, i.e. the number and shape distribution of each tissue in some planes is uniform, so the present invention sets the sagittal and coronal weights to be less than the axial weights. In this embodiment, λ is set1、λ2And λ3Set to 0.4, 0.3 and 0.3, respectively, to highlight the information of the axial picture.
For example, assuming that there are 3 types of classifications, if the three-dimensional segmentation results for the same pixel x are dimension 1 ═ {0.1,0.2,0.7}, dimension 2 ═ 0.5,0.1,0.4}, and dimension 3 ═ 0.2,0.2,0.6}, respectively, the weighting weights for the three dimensions are set to dimension 1 ═ 0.4, dimension 2 ═ 0.3, and dimension 3 ═ 0.3, respectively, and the weighting and thickness computation segmentation results are: {0.25,0.17,0.58}, the maximum probability of 0.58 corresponds to the third class, and the pixel x can be considered to belong to the third class.
The three F-CNNs may adopt the same architecture or different architectures, but the training methods are all the coarse and fine training methods described in the first embodiment.
EXAMPLE III
The present embodiment provides an F-CNN applied to the above-described first and second embodiments. The three full convolution neural networks of the second embodiment may all be the F-CNN architecture of this embodiment.
As shown in fig. 4, the overall F-CNN architecture is based on U-net, which utilizes a skip connection (skip connection) encoder and decoder. Each encoder and decoder employs dense connection blocks (dense connections blocks) to ensure the direction of the gradient and improve feature reusability. The pooling layer is followed after the dense connection module of the encoder, and the upper sampling layer is connected before the dense connection module of the decoder for reinforcement, so that the excitation distribution can be well ensured to obtain correct spatial mapping in up-sampling, and the segmentation precision can be improved in turn, especially for small subcutaneous tissues. The upsampling layer may employ general upsampling, hole convolution, or transposed convolution.
As shown in fig. 5, the dense connection module includes three convolutional layers c, each of which is preceded by an instance normalization layer n (instance normalization); the activation function in the convolutional layer c adopts a linear modified activation function (ReLU); both of the first two convolutional layers are followed by a connecting layer (+) for connecting the input profile of the current convolutional layer with the input profiles of all preceding stages of convolutional layers. These dense connections can improve the gradient trend during training and promote the reusability of features at different convolution stages. In addition, the dense connection can also enable the characteristics of the same module to obtain different results when being learned by different convolutional layers, and the performance is better. To limit the number of parameters, the convolution kernels for these two convolution layers are set to be relatively small, such as: 3x 3.
The encoding process of the above-described U-net structure comprises four densely-connected modules, each densely-connected module followed by a very large pooling layer (max-pooling block) of 2 × 2. The max-pooling layer halves the spatial dimension at each stage. During the down-sampling process of the max-pooling layer, the index corresponding to the maximum activation value is recorded and passed to the decoding module for up-sampling.
The decoding process also includes four densely connected modules. Each densely connected module is preceded by an up-sampling (un-firing) layer. The upsampling layer may recover the true spatial location of the largest activation value lost in the very large pooling layer of the encoding process and place it in the correct location during the upsampling process. This procedure is very meaningful for segmenting relatively small subcutaneous tissue. Another advantage of upsampling over the convolution conversion used in U-net is that it does not require any learned parameters. The upsampling is followed by a dense connection that connects the upsampled feature distribution with the input feature distribution in the corresponding code before the maximum pooling. To facilitate segmentation, skip-concatenations add coding features to the decoding process to ensure that gradients can flow from regions of deep features to regions of shallow layers. On the basis of the same architecture, the well-connected feature distribution will be passed to the next densely-connected module.
The output feature distribution of the last decoder is passed to the classification module fc. The classification module is essentially a convolution layer with a convolution kernel size of 1x1 that maps the input to an N-layer feature distribution, where N is the number of classes. Next follows a softmax layer, not shown, that maps activation values to probability values. The softmax layer outputs the probability that each pixel belongs to the respective class.
The full convolution neural network of the invention can be trained by adopting a momentum random gradient descent method, the learning rate is selected to ensure that the verification data can be properly converged, such as 0.1, and the learning rate is properly reduced every ten rounds (epochs) in the whole pre-training stage. Training is continued until the validation loss reaches a plateau. In this process, the weighted decay constant is set to 0.0001 and the amount of data per session (Batch size) is set to 24 (this value is limited by the computer's memory). The momentum is set to 0.95 to compensate for the noise of the gradient due to the relatively small Batch size.
Example four
In order to implement the above solution, the present embodiment provides an MRI image segmentation apparatus 600, as shown in fig. 6, including an automatic segmentation module, a full convolution neural network module, and a training module.
An automatic segmentation module configured to segment the tissue type of the MRI image using segmentation software, the segmentation result being used as auxiliary tag data;
the training module is configured for carrying out coarse training on the full convolution neural network by adopting a large number of samples with auxiliary labels; carrying out fine training on the full convolution neural network by adopting a small data sample with an artificial labeling label; in the course of rough training and fine training, adding weight omega (x) in the logic loss term of the loss function; the weight ω (x) of the pixel x is related to the volume of the tissue type in which the pixel x is located, and the larger the volume is, the higher the weight is; ω (x) is also related to whether the location of pixel x is a category edge, where the weight of pixels at the edge is greater than the weight of pixels not at the edge;
and after the training of the full convolution neural network in the full convolution neural network module is finished, the full convolution neural network acts on the MRI image to be segmented to realize image segmentation.
FIG. 7 is a block diagram of a full convolution neural network module, which includes a three-dimensional segmentation module, three sets of identical full convolution neural networks, and a summarization module; the three-dimensional segmentation module segments the MRI image into three-dimensional views along the coronal, axial and sagittal directions respectively; and each dimension is segmented by utilizing a set of full-convolution neural network, and finally, the segmentation results of the three dimensions are summarized by the summarizing module to obtain an image segmentation result.
The summarizing module is used for weighting and calculating the segmentation results of the three dimensions, and the sagittal weight and the coronal weight are set to be smaller than the axial weight.
The full convolution neural network adopts a U-net structure, and the encoder and the decoder both adopt multilayer dense connection modules; the pooling layer is followed after the dense connection module of the encoder, and the upper sampling layer is connected before the dense connection module of the decoder; the dense connection module comprises three convolution layers c, wherein each convolution layer is preceded by an example normalization layer n; both of the first two convolutional layers are followed by a connection layer for connecting the input signature of the current convolutional layer with the input signatures of all preceding stages of convolutional layers.
It should be understood that the units or modules described in the apparatus 600 shown in fig. 6 correspond to the steps in the methods described in the first to fourth embodiments, and the details of the implementation thereof are not repeated herein.
The units or modules described in the embodiments of the present application may be implemented by software or hardware. The units or modules described may also be provided in a processor, and the names of the units or modules do not in some cases constitute a limitation of the units or modules themselves.
The present application also provides a computer-readable storage medium, which may be the computer-readable storage medium contained in the apparatus described in the above embodiments; or it may be a separate computer readable storage medium not incorporated into the device. The computer readable storage medium stores one or more programs for use by one or more processors in performing the segmentation methods described herein.
In summary, the above description is only a preferred embodiment of the present invention, and is not intended to limit the scope of the present invention. Any modification, equivalent replacement, or improvement made within the spirit and principle of the present invention should be included in the protection scope of the present invention.

Claims (18)

1. An MRI image segmentation method, comprising:
segmenting the tissue type of the MRI image by using segmentation software, taking a segmentation result as auxiliary label data, and performing coarse training on the full convolution neural network by using a sample with an auxiliary label;
using a rough training network as initialization, and adopting a sample with an artificial labeling label to carry out fine training on the full convolution neural network;
and the trained full convolution neural network acts on the MRI image needing to be segmented to realize image segmentation.
2. The method of claim 1, wherein during the course training and the fine training, a weight ω (x) is added to a logistic loss term of the loss function; the weight ω (x) of the pixel x is related to the volume of the tissue class in which the pixel x is located, the larger the volume, the lower the weight; ω (x) is also related to whether pixel x is located at a class edge, where the pixel weight is greater than the pixel weight that is not located at the edge.
3. The method of claim 2, wherein the weight ω (x) is obtained by:
calculating the number of pixels of each tissue class aiming at a single volume pixel as a sample, obtaining the median med of the number of pixels of all the tissue classes, and then obtaining the first part weight omega of the pixel x1(x) Dividing the median med of the pixel number by the pixel number count _ i of the tissue class i in which x is located;
for the single body pixel, calculating an edge position through a gradient; for the pixel x at the edge position, the first partial weight value ω is weighted1(x) Multiplying by a set multiple to obtain the weight omega (x) of the pixel.
4. The method of claim 2, wherein the penalty function is a joint penalty function including a logic penalty Logistic Loss and a similarity coefficient penalty Dice Loss.
5. The method of claim 1, wherein the segmentation software employs FreeSprofer software.
6. The method of claim 1, wherein the MRI image is segmented into three-dimensional views along the coronal, axial and sagittal directions, respectively, each dimension is segmented using a set of fully convolutional neural networks, and the segmentation results for the three dimensions are finally summed to obtain the image segmentation results.
7. The method of claim 6, wherein the aggregating the segmentation results of the three dimensions is in a weighted sum manner.
8. The method of claim 7, wherein the sagittal and coronal weights are set to be less than the axial weights.
9. The method of claim 1, wherein the full convolutional neural network employs a U-net structure, and the encoder and decoder each employ multiple layers of densely connected modules; the pooling layer is followed after the dense connection module of the encoder, and the upper sampling layer is connected before the dense connection module of the decoder; the dense connection module comprises three convolution layers c, wherein each convolution layer is preceded by an example normalization layer n; both of the first two convolutional layers are followed by a connection layer for connecting the input signature of the current convolutional layer with the input signatures of all preceding stages of convolutional layers.
10. The method of claim 1, wherein the MRI image is a brain MRI image.
11. An MRI image segmentation device is characterized by comprising an automatic segmentation module, a full convolution neural network module and a training module;
an automatic segmentation module configured to segment the tissue type of the MRI image using segmentation software, the segmentation result being used as auxiliary tag data;
the training module is configured for carrying out coarse training on the full convolution neural network by adopting a sample with an auxiliary label; using a rough training network as initialization, and adopting a sample with an artificial labeling label to carry out fine training on the full convolution neural network;
and after the training of the full convolution neural network in the full convolution neural network module is finished, the full convolution neural network acts on the MRI image to be segmented to realize image segmentation.
12. The apparatus of claim 11, wherein the training module adds a weight ω (x) in a logistic-loss term of the loss function during coarse training and fine training; the weight ω (x) of the pixel x is related to the volume of the tissue class in which the pixel x is located, the larger the volume, the lower the weight; ω (x) is also related to whether pixel x is located at a class edge, where the pixel weight is greater than the pixel weight that is not located at the edge.
13. The apparatus of claim 12, wherein the penalty function employed by the training module in training is a joint penalty function including a logic penalty Logistic Loss and a similarity coefficient penalty Dice Loss.
14. The apparatus of claim 11, wherein the segmentation software employed by the automatic segmentation module is freesrush software.
15. The apparatus of claim 11, wherein the full convolutional neural network module comprises a three-dimensional segmentation module, a summarization module, and three sets of identical full convolutional neural networks;
the three-dimensional segmentation module segments the MRI image into three-dimensional views along the coronal, axial and sagittal directions respectively; and each dimension is segmented by utilizing a set of full-convolution neural network, and finally, the segmentation results of the three dimensions are summarized by the summarizing module to obtain an image segmentation result.
16. The apparatus of claim 15, wherein the summarization module weights and calculates the segmentation results for three dimensions, with sagittal and coronal weights set to less than axial weights.
17. The apparatus of claim 11 or 15, wherein the full convolutional neural network employs a U-net structure, and the encoder and the decoder each employ multiple layers of densely connected modules; the pooling layer is followed after the dense connection module of the encoder, and the upper sampling layer is connected before the dense connection module of the decoder; the dense connection module comprises three convolution layers c, wherein each convolution layer is preceded by an example normalization layer n; both of the first two convolutional layers are followed by a connection layer for connecting the input signature of the current convolutional layer with the input signatures of all preceding stages of convolutional layers.
18. A computer-readable storage medium, on which a computer program is stored, which, when being executed by a processor, carries out the method of any one of claims 1 to 10.
CN201910973756.7A 2019-10-14 2019-10-14 MRI image segmentation method, device and storage medium based on rough training and fine training Pending CN110853048A (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN201910973756.7A CN110853048A (en) 2019-10-14 2019-10-14 MRI image segmentation method, device and storage medium based on rough training and fine training

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN201910973756.7A CN110853048A (en) 2019-10-14 2019-10-14 MRI image segmentation method, device and storage medium based on rough training and fine training

Publications (1)

Publication Number Publication Date
CN110853048A true CN110853048A (en) 2020-02-28

Family

ID=69596381

Family Applications (1)

Application Number Title Priority Date Filing Date
CN201910973756.7A Pending CN110853048A (en) 2019-10-14 2019-10-14 MRI image segmentation method, device and storage medium based on rough training and fine training

Country Status (1)

Country Link
CN (1) CN110853048A (en)

Cited By (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN111667488A (en) * 2020-04-20 2020-09-15 浙江工业大学 Medical image segmentation method based on multi-angle U-Net
CN113269764A (en) * 2021-06-04 2021-08-17 重庆大学 Automatic segmentation method and system for intracranial aneurysm, sample processing method and model training method
CN113379757A (en) * 2021-05-01 2021-09-10 首都医科大学宣武医院 Method for training brain image segmentation model and brain image segmentation method
CN113920314A (en) * 2021-09-30 2022-01-11 北京百度网讯科技有限公司 Semantic segmentation and model training method, device, equipment and storage medium

Citations (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN107220980A (en) * 2017-05-25 2017-09-29 重庆理工大学 A kind of MRI image brain tumor automatic division method based on full convolutional network
US10304193B1 (en) * 2018-08-17 2019-05-28 12 Sigma Technologies Image segmentation and object detection using fully convolutional neural network

Patent Citations (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN107220980A (en) * 2017-05-25 2017-09-29 重庆理工大学 A kind of MRI image brain tumor automatic division method based on full convolutional network
US10304193B1 (en) * 2018-08-17 2019-05-28 12 Sigma Technologies Image segmentation and object detection using fully convolutional neural network

Non-Patent Citations (1)

* Cited by examiner, † Cited by third party
Title
ABHIJIT GUHA ROY 等: "QuickNAT: A fully convolutional network for quick and accurate segmentation of neuroanatomy", 《NEUROIMAGE》 *

Cited By (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN111667488A (en) * 2020-04-20 2020-09-15 浙江工业大学 Medical image segmentation method based on multi-angle U-Net
CN111667488B (en) * 2020-04-20 2023-07-28 浙江工业大学 Medical image segmentation method based on multi-angle U-Net
CN113379757A (en) * 2021-05-01 2021-09-10 首都医科大学宣武医院 Method for training brain image segmentation model and brain image segmentation method
CN113379757B (en) * 2021-05-01 2024-04-12 首都医科大学宣武医院 Method for training brain image segmentation model and brain image segmentation method
CN113269764A (en) * 2021-06-04 2021-08-17 重庆大学 Automatic segmentation method and system for intracranial aneurysm, sample processing method and model training method
CN113920314A (en) * 2021-09-30 2022-01-11 北京百度网讯科技有限公司 Semantic segmentation and model training method, device, equipment and storage medium
CN113920314B (en) * 2021-09-30 2022-09-02 北京百度网讯科技有限公司 Semantic segmentation and model training method, device, equipment and storage medium

Similar Documents

Publication Publication Date Title
CN109035263B (en) Automatic brain tumor image segmentation method based on convolutional neural network
CN110930416B (en) MRI image prostate segmentation method based on U-shaped network
CN110853048A (en) MRI image segmentation method, device and storage medium based on rough training and fine training
CN111488914B (en) Alzheimer disease classification and prediction system based on multitask learning
CN110070935B (en) Medical image synthesis method, classification method and device based on antagonistic neural network
CN109242860B (en) Brain tumor image segmentation method based on deep learning and weight space integration
CN112767417B (en) Multi-modal image segmentation method based on cascaded U-Net network
WO2022121100A1 (en) Darts network-based multi-modal medical image fusion method
CN113989551B (en) Alzheimer's disease classification method based on improved ResNet network
CN107845065A (en) Super-resolution image reconstruction method and device
CN112785603A (en) Brain tissue segmentation method based on Unet and superpixel
CN115147600A (en) GBM multi-mode MR image segmentation method based on classifier weight converter
CN115375711A (en) Image segmentation method of global context attention network based on multi-scale fusion
CN114596317A (en) CT image whole heart segmentation method based on deep learning
CN111507184B (en) Human body posture detection method based on parallel cavity convolution and body structure constraint
CN111598964A (en) Quantitative magnetic susceptibility image reconstruction method based on space adaptive network
CN115496720A (en) Gastrointestinal cancer pathological image segmentation method based on ViT mechanism model and related equipment
CN116579975A (en) Brain age prediction method and system of convolutional neural network
Arega et al. Leveraging uncertainty estimates to improve segmentation performance in cardiac MR
CN112489048B (en) Automatic optic nerve segmentation method based on depth network
CN112750131B (en) Pelvis nuclear magnetic resonance image musculoskeletal segmentation method based on scale and sequence relation
Alsenan et al. A Deep Learning Model based on MobileNetV3 and UNet for Spinal Cord Gray Matter Segmentation
CN115409843B (en) Brain nerve image feature extraction method based on scale equalization coupling convolution architecture
CN116883341A (en) Liver tumor CT image automatic segmentation method based on deep learning
CN113538348A (en) Processing method of encephalic magnetic resonance diffusion weighted image and related product

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination