CN113902674A - Medical image segmentation method and electronic equipment - Google Patents
Medical image segmentation method and electronic equipment Download PDFInfo
- Publication number
- CN113902674A CN113902674A CN202111035523.6A CN202111035523A CN113902674A CN 113902674 A CN113902674 A CN 113902674A CN 202111035523 A CN202111035523 A CN 202111035523A CN 113902674 A CN113902674 A CN 113902674A
- Authority
- CN
- China
- Prior art keywords
- medical image
- image data
- training
- pyramid
- scale
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
- 238000000034 method Methods 0.000 title claims abstract description 57
- 238000003709 image segmentation Methods 0.000 title claims abstract description 53
- 230000011218 segmentation Effects 0.000 claims abstract description 51
- 238000007781 pre-processing Methods 0.000 claims abstract description 5
- 238000012549 training Methods 0.000 claims description 96
- 230000006870 function Effects 0.000 claims description 40
- 230000008569 process Effects 0.000 claims description 13
- 238000000605 extraction Methods 0.000 claims description 10
- 230000004927 fusion Effects 0.000 claims description 10
- 238000010606 normalization Methods 0.000 claims description 9
- 238000012545 processing Methods 0.000 claims description 9
- 230000004913 activation Effects 0.000 claims description 6
- 238000011176 pooling Methods 0.000 claims description 6
- 238000004590 computer program Methods 0.000 claims description 4
- 238000002372 labelling Methods 0.000 claims description 4
- NTHWMYGWWRZVTN-UHFFFAOYSA-N sodium silicate Chemical compound [Na+].[Na+].[O-][Si]([O-])=O NTHWMYGWWRZVTN-UHFFFAOYSA-N 0.000 claims description 2
- 210000000683 abdominal cavity Anatomy 0.000 description 12
- 238000004364 calculation method Methods 0.000 description 10
- 238000004891 communication Methods 0.000 description 10
- 210000004534 cecum Anatomy 0.000 description 6
- 210000002429 large intestine Anatomy 0.000 description 6
- 210000000813 small intestine Anatomy 0.000 description 6
- 238000010586 diagram Methods 0.000 description 5
- 238000005516 engineering process Methods 0.000 description 4
- 210000000056 organ Anatomy 0.000 description 4
- 238000001514 detection method Methods 0.000 description 3
- 238000007499 fusion processing Methods 0.000 description 3
- 238000012986 modification Methods 0.000 description 3
- 230000004048 modification Effects 0.000 description 3
- 230000008901 benefit Effects 0.000 description 2
- 238000013527 convolutional neural network Methods 0.000 description 2
- 238000013135 deep learning Methods 0.000 description 2
- 238000012805 post-processing Methods 0.000 description 2
- 230000009471 action Effects 0.000 description 1
- 230000003044 adaptive effect Effects 0.000 description 1
- 230000009286 beneficial effect Effects 0.000 description 1
- 230000008859 change Effects 0.000 description 1
- 239000003086 colorant Substances 0.000 description 1
- 230000001186 cumulative effect Effects 0.000 description 1
- 230000001419 dependent effect Effects 0.000 description 1
- 238000011161 development Methods 0.000 description 1
- 230000018109 developmental process Effects 0.000 description 1
- 230000000694 effects Effects 0.000 description 1
- 230000002708 enhancing effect Effects 0.000 description 1
- 239000000284 extract Substances 0.000 description 1
- 230000006872 improvement Effects 0.000 description 1
- 230000003993 interaction Effects 0.000 description 1
- 230000005012 migration Effects 0.000 description 1
- 238000013508 migration Methods 0.000 description 1
- 230000001737 promoting effect Effects 0.000 description 1
- 230000000644 propagated effect Effects 0.000 description 1
- 230000009467 reduction Effects 0.000 description 1
- 230000003068 static effect Effects 0.000 description 1
- 230000002194 synthesizing effect Effects 0.000 description 1
- 230000009466 transformation Effects 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/0002—Inspection of images, e.g. flaw detection
- G06T7/0012—Biomedical image inspection
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/045—Combinations of networks
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/048—Activation functions
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/08—Learning methods
- G06N3/084—Backpropagation, e.g. using gradient descent
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/10—Segmentation; Edge detection
- G06T7/11—Region-based segmentation
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/10—Segmentation; Edge detection
- G06T7/136—Segmentation; Edge detection involving thresholding
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/10—Image acquisition modality
- G06T2207/10072—Tomographic images
- G06T2207/10081—Computed x-ray tomography [CT]
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20081—Training; Learning
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- General Health & Medical Sciences (AREA)
- Health & Medical Sciences (AREA)
- Computing Systems (AREA)
- Mathematical Physics (AREA)
- Data Mining & Analysis (AREA)
- Evolutionary Computation (AREA)
- Biophysics (AREA)
- Molecular Biology (AREA)
- Biomedical Technology (AREA)
- General Engineering & Computer Science (AREA)
- Artificial Intelligence (AREA)
- Computational Linguistics (AREA)
- Software Systems (AREA)
- Life Sciences & Earth Sciences (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Medical Informatics (AREA)
- Nuclear Medicine, Radiotherapy & Molecular Imaging (AREA)
- Radiology & Medical Imaging (AREA)
- Quality & Reliability (AREA)
- Image Analysis (AREA)
Abstract
The present disclosure provides a medical image segmentation method and an electronic device. The method comprises the following steps: acquiring medical image data, and preprocessing the medical image data to obtain processed medical image data; inputting the processed medical image data into a medical image segmentation model; obtaining pyramid bottom characteristics f of the medical image according to the processed medical image data in the medical image segmentation model1(ii) a According to the pyramid bottom characteristic f of the medical image1Obtaining a prediction score p; and obtaining a segmentation result P of the medical image according to the prediction fraction P. This is thatIn the embodiment of the specification, the medical image segmentation result P is finally obtained through medical image data and a pre-trained medical image segmentation model.
Description
Technical Field
The disclosure relates to the technical field of image semantic segmentation of computer vision, in particular to a medical image segmentation method.
Background
With the development of science and technology, medical image segmentation technology is gradually replaced by deep learning methods with better performance and stronger generalization capability from the previous Otsu threshold method, watershed method, Graphcut and some methods based on active contour. However, the current medical image segmentation field still has some difficulties, firstly, the data volume of the medical image is insufficient when the medical image segmentation model is trained; secondly, the difference of the target dimension to be segmented in the medical image is large, the model is difficult to accurately identify, and finally, most target organs in the existing medical image are fuzzy and easy to be confused with organs in other adjacent parts, and the model is difficult to identify.
Disclosure of Invention
In view of the above, an object of the present disclosure is to provide a medical image segmentation method and an electronic device.
In view of the above, the present disclosure provides a medical image segmentation method, which includes:
acquiring medical image data;
preprocessing the medical image data to obtain processed medical image data;
inputting the processed medical image data into a medical image segmentation model;
obtaining pyramid bottom characteristics f of the medical image according to the processed medical image data in the medical image segmentation model1(ii) a According to the pyramid bottom characteristic f of the medical image1Obtaining a prediction score p;
and obtaining a segmentation result P of the medical image according to the prediction fraction P.
Optionally, the training process of the medical image segmentation model includes:
acquiring medical image data for training;
acquiring the number c of classes to be segmented of medical image data for training;
acquiring a real Mask corresponding to medical image data for training, wherein the real Mask corresponding to the medical image data for training is as follows: pre-labeling the medical image data for training with a good result pixel by pixel;
according to the medical image data for training, carrying out thresholding, normalization and data enhancement processing on the medical image data for training to obtain the processed medical image data for training;
performing initial multi-scale feature extraction on the processed medical image data for training to obtain a multi-scale feature pyramid of the processed medical image data for training;
performing multi-scale feature fusion on the multi-scale feature pyramid of the processed medical image data for training to obtain an enhanced feature pyramid for training;
obtaining the bottom feature f of the pyramid for training according to the enhanced feature pyramid for training1′；
Extracting weights of the training enhanced feature pyramid aiming at the number c of all the classes to be segmented to obtain scale self-adaptive attention class weights;
obtaining a multi-scale self-adaptive segmentation loss function according to the scale self-adaptive attention category weight;
according to the pyramid tower bottom characteristic f for training1', obtaining a class segmentation score p' and a target feature;
obtaining an intra-class consistency constraint loss function and an inter-class difference constraint loss function according to the target characteristics;
obtaining a segmentation loss function according to the class segmentation fraction p' and a real Mask corresponding to the medical image data for training;
and training to obtain the medical image segmentation model according to the multi-scale self-adaptive segmentation loss function, the intra-class consistency constraint loss function, the inter-class difference constraint loss function and the segmentation loss function.
Optionally, obtaining the scale-adaptive attention category weight includes:
according to a certain characteristic f in the enhanced characteristic pyramid for trainingi' after convolution operation and average pooling, the data is inputted to a first full-connection network FC1Activated using the ReLU activation function and re-input into the second fully connected network FC2Obtaining the scale self-adaptive attention category weight w by using Sigmoid activation function activationi。
Optionally, obtaining the target feature includes:
according to the pyramid tower bottom characteristic f for training1' after 2d convolution operation and self-adaptive pooling are carried out on the target feature f, flattening operation and dimension replacement operation are carried out, and then 1d convolution operation is carried out to obtain the target feature f1 c′。
Optionally, the processed medical image data is obtained by thresholding and normalizing the medical image data.
Optionally, obtaining pyramid bottom feature f of the medical image1The method specifically comprises the following steps:
extracting initial multi-scale features of the processed medical image data to obtain a multi-scale feature pyramid;
performing multi-scale feature fusion on the multi-scale feature pyramid to obtain an enhanced feature pyramid;
obtaining pyramid bottom characteristics f of the medical image according to the enhanced characteristic pyramid1。
Optionally, the prediction score p is obtained by performing 2d convolution operation on the pyramid tower bottom features of the medical image.
Optionally, the size of the prediction score p is c × H × W, and c is the number c of categories to be segmented of the medical image data for training; h is the length of the prediction fraction p, and W is the width of the prediction fraction p.
Optionally, the final segmentation result P of the medical image is passed
Calculating to obtain;
the above-mentioned
In, plij,l∈[1,c],i∈[1,H],j∈[1,W]The value of the prediction score of the ith class to be segmented in the prediction score vector of each pixel position (i, j) in the prediction score p;
l∈[1,c]to predict a certain pixel position in the fraction p(i, j) prediction score vector pijC is the number c of classes to be segmented of the medical image data for training.
Based on the same inventive concept, the present disclosure also provides an electronic device, which includes a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the medical image segmentation method according to any one of the above aspects.
As can be seen from the foregoing, according to the medical image segmentation method provided by the present disclosure, after medical image data is acquired and preprocessed, a pre-trained medical image segmentation model is input to obtain a prediction score P, and a segmentation result P is obtained according to the prediction score P. According to the method, the medical image data is additionally subjected to data enhancement preprocessing during model training, so that the data volume entering the model is greatly improved, and the model learning is facilitated; when the model is trained, initial multi-scale feature extraction and multi-scale feature fusion are carried out on medical image data, so that the segmentation capability of the trained model on the target to be segmented under different scales is well improved; when the model is trained, the medical image data is subjected to target feature enhancement processing, and the processed medical image data has the characteristics of small intra-class difference and large inter-class difference, so that the model learning is facilitated; the training means enables the data volume to be sufficient during model training, and the trained model has the characteristics of strong segmentation capability and strong classification performance.
Drawings
In order to more clearly illustrate the technical solutions in the present disclosure or related technologies, the drawings needed to be used in the description of the embodiments or related technologies are briefly introduced below, and it is obvious that the drawings in the following description are only embodiments of the present disclosure, and for those skilled in the art, other drawings can be obtained according to these drawings without creative efforts.
Fig. 1 is a flowchart of a medical image segmentation method according to an embodiment of the present disclosure;
fig. 2 is a schematic diagram illustrating a training process of a medical image segmentation model according to an embodiment of the disclosure;
fig. 3 is an electronic device structure according to an embodiment of the disclosure.
Detailed Description
For the purpose of promoting a better understanding of the objects, aspects and advantages of the present disclosure, reference is made to the following detailed description taken in conjunction with the accompanying drawings.
It is to be noted that technical terms or scientific terms used in the embodiments of the present disclosure should have a general meaning as understood by those having ordinary skill in the art to which the present disclosure belongs, unless otherwise defined. The use of "first," "second," and similar terms in the embodiments of the disclosure is not intended to indicate any order, quantity, or importance, but rather to distinguish one element from another. The word "comprising" or "comprises", and the like, means that the element or item listed before the word covers the element or item listed after the word and its equivalents, but does not exclude other elements or items. The terms "connected" or "coupled" and the like are not restricted to physical or mechanical connections, but may include electrical connections, whether direct or indirect. "upper", "lower", "left", "right", and the like are used merely to indicate relative positional relationships, and when the absolute position of the object being described is changed, the relative positional relationships may also be changed accordingly.
As described in the background section, the medical image segmentation method in the related art has the disadvantages of insufficient data amount during training of the medical image segmentation model, large scale difference of the target to be segmented, and easy confusion between the target organ and the nearby organ, and brings great inconvenience to the segmentation of the medical image, and the segmentation result obtained after the medical image is segmented is not ideal.
In view of the above, the embodiment of the present disclosure provides a medical image segmentation method, which obtains a segmentation result P of a medical image based on a pre-trained medical image segmentation model and medical data for training, so that the segmentation result of the medical image is more accurate.
Hereinafter, the technical means of the embodiments of the present disclosure will be described in detail by specific examples.
Referring to fig. 1, a medical image segmentation method according to an embodiment of the present disclosure includes the following steps:
step S101, acquiring medical image data.
In this step, the medical image data is in a currently common format such as DICOM, NIFFT, etc., and these data include patient information, slice information, image information, etc. Therefore, the first step of the medical image segmentation method is to acquire medical image data, i.e. to read the image data information required by the embodiment from the medical image. The data in the above format can be read by corresponding python library functions, such as nibabel, pydicom, etc.
Step S103, preprocessing the medical image data to obtain processed medical image data.
In this step, the range of the numerical value of the medical image data acquired in step S101 is wide, and if the medical image data is directly normalized, the contrast between the targets to be segmented in the subsequent segmentation process may be insufficient, or even the targets cannot be identified, so that the segmentation result has a problem. The threshold in thresholding is typically set by the physician based on clinical experience. If the medical image data acquired in step S101 is recorded as O, the thresholded lower bound is min, the thresholded upper bound is max, and the thresholded data is C, the thresholding may be represented as:
after thresholding the medical image data, because the difference between the medical image data is large, the data ranges corresponding to different targets may have large difference after thresholding, and are not easy to process, and in order to process the acquired medical image data more uniformly, the embodiment performs normalization processing on the medical image data, so that the medical image data is easy to convert into common image formats, such as PNG, JPEG, and the like. In this embodiment, a normalization scheme is adopted to normalize the maximum and minimum values, and C is recordedminIs the minimum value of thresholded data, CmaxFor the maximum value of the thresholded data, the normalized data is N, then the normalization can be expressed as:
and performing thresholding and normalization processing on the medical image data to obtain processed medical image data.
Step S105, inputting the processed medical image data into a medical image segmentation model.
In this step, the medical image segmentation model is trained in advance, and a specific training process is described in detail in the embodiments described later.
Step S107, in the medical image segmentation model, obtaining pyramid bottom characteristics f of the medical image according to the processed medical image data1(ii) a According to the pyramid bottom characteristic f of the medical image1And obtaining a prediction score p. In this step, the processed medical image data is subjected to initial multi-scale feature extraction in a medical image segmentation model to obtain a multi-scale feature pyramid, multi-scale feature fusion is performed on the multi-scale feature pyramid to obtain an enhanced feature pyramid, and a pyramid bottom feature f is obtained according to the enhanced feature pyramid1According to the pyramid bottom feature f1And obtaining a prediction score p.
In this embodiment, the pyramid bottom feature f1Obtaining a prediction score graph which is obtained after 2d convolution and is the prediction score p, wherein the scale size is c H W, and c is the number of the categories to be segmented of the medical image data for training; h is the length of the prediction fraction p, and W is the width of the prediction fraction p.
And step S109, obtaining a segmentation result P of the medical image according to the prediction fraction P.
In the present embodiment, the input medical image segmentation model is the processed medical image data, and the output model is the corresponding pre-image dataAnd measuring the fraction P, wherein the segmentation result P of the medical image is obtained by calculation outside the medical image segmentation model. The calculation formula is as follows:
in the formula plij,l∈[1,c],i∈[1,H],j∈[1,W]The value of the prediction score of the ith class to be segmented in the prediction score vector of each pixel position (i, j) in the prediction score p;
l∈[1,c]for a vector p of prediction scores for a certain pixel position (i, j) in the prediction score pijC is the number c of classes to be segmented of the medical image data for training.
Taking abdominal cavity CT as an example, the large intestine, small intestine and cecum are divided. Reading medical image data in abdominal cavity CT by pydicom, carrying out thresholding and normalization processing on the medical image data to obtain processed abdominal cavity CT medical image data, inputting the processed abdominal cavity CT medical image data into a pre-trained medical image segmentation model, carrying out initial multi-scale feature extraction on the processed abdominal cavity CT medical image data in the model to obtain a multi-scale feature pyramid of abdominal cavity CT, carrying out multi-scale feature fusion on the multi-scale feature pyramid of abdominal cavity CT to obtain an enhanced feature pyramid of abdominal cavity CT, taking the pyramid bottom feature of the pyramid, carrying out 2d convolution on the pyramid bottom feature, outputting prediction scores of large intestine, small intestine and cecum in abdominal cavity CT by the model, calculating the prediction scores by the calculation formula in step S109 to obtain the segmentation results of the large intestine, small intestine and cecum in abdominal cavity CT, wherein the segmentation results are index values of the large intestine, the small intestine and the cecum in the abdominal cavity CT, in order to more intuitively recognize the ranges of the large intestine, the small intestine and the cecum, RGB color values are generally given to the index values, and the effect of the range of the large intestine, the small intestine and the cecum appearing in the abdominal cavity CT in different colors on one graph is finally presented.
In some embodiments, the medical image segmentation model is obtained by pre-training, and the present embodiment will describe the training process of the medical image segmentation model in detail. As shown in fig. 2, the training process of the model includes the following steps:
in step S201, medical image data for training is acquired.
In this step, the medical image data is in a currently common format such as DICOM, NIFFT, etc., and these data include patient information, slice information, image information, etc. Therefore, the first step of the medical image segmentation method is to acquire medical image data, i.e. to read the image data information required by the embodiment from the medical image. The data in the above format can be read by corresponding python library functions, such as nibabel, pydicom, etc. During training, massive training data is input, the massive training data is divided into a plurality of batches, the size of each batch is recorded as B, and data in one batch are forwarded and error is reversely propagated each time.
Step S203, acquiring the number c of classes to be segmented of the medical image data for training.
The number c of the classes to be segmented of the medical image data for training is the number of the classes to be segmented of the medical image data to be inferred, if the number c of the classes to be segmented is required to be changed, retraining of the whole model is not required, and the method only needs to extract the weight of the pyramid according to the enhanced feature for training and aiming at the number c of the classes to be segmented, so that the migration training is started again at the step of obtaining the scale self-adaptive attention class weight.
Step S205, acquiring a real Mask corresponding to the medical image data for training, where the real Mask corresponding to the medical image data for training is: and pre-labeling the medical image data for training pixel by pixel to obtain a good result.
In this embodiment, the real Mask corresponding to the medical image data may be a result labeled in advance on professional labeling software by a doctor in the field; if it is from an open source data set, it is the data set publisher responsible for collation and labeling.
And step S207, performing thresholding, normalization and data enhancement processing on the medical image data for training according to the medical image data for training to obtain the processed medical image data for training.
In this embodiment, the steps of thresholding and normalizing the medical image data for training are the same as those of thresholding and normalizing the medical image data in the inference process, and are not described herein again. The data quantity and the data diversity of the medical image cannot meet the requirement of deep learning easily, so that the data quantity and the data diversity need to be expanded, the precision of the model is improved, data enhancement operations such as random scale transformation, random cutting, random horizontal inversion and normalization are adopted in the embodiment, and one or more operations can be selected in actual operation.
Step S209, performing initial multi-scale feature extraction on the processed medical image data for training to obtain a multi-scale feature pyramid of the processed medical image data for training.
In the step, the initial multi-scale feature extraction is to improve the segmentation capability of the model on the target to be segmented with different scales, and the method adopts the classical convolutional neural network to complete the step, wherein the classical convolutional neural network gradually extracts high-level features by stacking convolutional modules, and in the process, feature graphs with different scales are formed to obtain a multi-scale feature pyramid structure.
In this embodiment, a High-Resolution Network (HRNet) is used to perform initial multi-scale feature extraction, in this embodiment, medical image data is subjected to initial multi-scale feature extraction to obtain a multi-scale feature pyramid of the medical image data for post-processing training, and a feature map from bottom to top in the obtained multi-scale feature pyramid of the medical image data for post-processing training is denoted as s1,s2,s3,s4The dimensions of the four multi-scale features are 1/4, 1/8, 1/16 and 1/32 of the original image in sequence, and the four multi-scale features form a multi-scale feature pyramid. At this time, the information of the multi-scale feature pyramid is insufficient for the medical image segmentation task of the disclosure, the features of different scales have respective advantages and disadvantages, the position information of the high-resolution feature map is more accurate and rich, but the semantic information is lower; since the semantic information of the low-resolution feature map is higher, but the position information loss is more serious, in this embodiment, the multi-scale feature pyramid is subjected to the multi-scale feature fusion processing, so that the feature maps of the respective scales are mutually lengthenedAnd (5) making up for the shortness.
Step S211 performs multi-scale feature fusion on the multi-scale feature pyramid of the processed medical image data for training to obtain an enhanced feature pyramid for training.
In this embodiment, a Feature Pyramid Network (FPN) structure is used to perform multi-scale Feature fusion, and the multi-scale features after FPN fusion are respectively denoted as f1′,f2′,f3′,f4' which in turn are all 1/4 of the original drawing. The multi-scale feature pyramid is subjected to multi-scale feature fusion processing to obtain an enhanced feature pyramid, each level of the feature pyramid has good classification capability in the aspect of uniformity, and certain position information is reserved or recovered.
Step S213, obtaining the bottom feature f of the training pyramid according to the enhanced feature pyramid for training1′。
In this embodiment, the enhanced bottom feature of the feature pyramid has the best positioning capability, and the pyramid bottom feature f has good classification capability because of being subjected to multi-scale fusion processing, so that this embodiment uses the pyramid bottom feature f1' as the final segmentation feature.
Step S215, extracting the weight of the enhanced feature pyramid for training aiming at the number c of all the categories to be segmented to obtain the scale self-adaptive attention category weight.
Considering that the target to be segmented has large change in scale, the difference of scales of different targets on the same slice is large, and the scales of the same target on different slices can also be greatly changed; meanwhile, in consideration of the fact that the detection capabilities of different scales of features on different scales of targets are different, the loss of position information of the feature map with a larger scale is less, the detection capability on a small target is stronger, and the pertinence of the feature map with a smaller scale on a large target is stronger, so that the feature maps with different scales of targets are required to be adopted for detection and segmentation. Therefore, the method adopts a light-weight small network to extract the weight of each feature map with different scales aiming at each object to be segmented, the feature map with each scale can generate a weight suitable for the scale in a self-adaptive manner, the weight can be weighted with each object to be segmented in the feature map with the corresponding scale, so that the different-scale feature maps can pay attention to certain categories to be segmented aiming at the self-adaptive emphasis, and the different-scale feature maps can play their own roles, thereby improving the segmentation precision of the network.
In the present embodiment, the weight generation employs an operation based on the Squeeze-and-Excitation in SEnet. Since the feature maps of different scales are generated by adaptive weight independently, but the flows are the same, the present embodiment will be described in detail only for the weight generation of a feature map of a certain scale:
for a feature of a certain scale fiI ∈ {1,2,3,4}, with a size ciH w, wherein ciIs fiH is fiW is fiC, the weight w of the scale self-adaptive attention classiMay be expressed as:
according to a certain characteristic f in the enhanced characteristic pyramid for trainingi' after convolution operation and average pooling, the data is inputted to a first full-connection network FC1Activated using the ReLU activation function and re-input into the second fully connected network FC2Obtaining the scale self-adaptive attention category weight w by using Sigmoid activation function activationi。
And S217, obtaining a multi-scale self-adaptive segmentation loss function according to the scale self-adaptive attention category weight.
The convolution network corresponding to the feature map of each scale is optimized independently, so that the classification capability of the medical image segmentation model for segmentation targets of different scale bands is improved, and finally, each scale feature map has a corresponding convolution network suitable for the scale, and scale self-adaptive segmentation can be completed.
In this embodiment, the feature map f of each scaleiAfter convolution, a feature map with the channel number c, namely a feature score, recorded as o, is obtainediC is the number of classes to be dividedTo achieve the purpose. And the characteristic diagram, i.e. fi' segmentation scores of all corresponding target classes to be segmented are calculated according to the actual Mask calculation error L corresponding to the medical image data for training input into the modeliThe calculation formula is as follows:
where c is the object class to be segmented, oijIs oiSegmentation score, M, on object class j to be segmentedjA real Mask corresponding to the medical image data for training corresponding to the target class j to be segmented, wherein l (x, y) represents a loss function between x and y, a cross entropy loss function is adopted in the embodiment, which is o in the embodimentijAnd MjA loss function in between.
On the basis, the self-adaptive attention class weight w corresponding to each scale feature map obtained in the previous step needs to be considerediThen in this case, for each scale feature map, the error LiThe calculation formula of (2) is as follows:
wherein, wijIs corresponding to fi' weight of the scale on the object class j to be segmented.
Finally, obtaining a multi-scale self-adaptive segmentation loss function Li。
Step S219, according to the pyramid tower bottom characteristic f for training1', get the class segmentation score p' and the target feature.
In the present embodiment, the pyramid bottom feature f for training1', the feature map obtained after the 2d convolution is referred to as a class division score p'. The process of target feature extraction is as follows: according to the pyramid tower bottom characteristic f for training1' after 2d convolution operation and self-adaptive pooling, carrying out flattening operation and dimension replacement operation, and then carrying out 1d convolution operation to obtainThe target feature
And step S221, obtaining an intra-class consistency constraint loss function and an inter-class difference constraint loss function according to the target characteristics.
The embodiment of the disclosure expects that when the medical image segmentation model is trained, the features in the same category have smaller differences among the medical image data entering the model in the same batch, and the features in different categories have larger differences, so that the medical image segmentation model can be conveniently learned. In order to reduce the difference of the features in the same category, it may be considered to measure the difference between the features in the same category by calculating the similarity, that is, by calculating the difference between the similarity of the features in the same category and 1, an unsupervised loss function is introduced, so as to introduce a new regularization term, which will make the consistency between the features in the same category as large as possible. Similar to the reduction of the difference of the internal features of the same category, in order to achieve the purpose of reducing the consistency of the features between different categories, a new regularization term is introduced by calculating the difference between the similarity and 0 between the features of different categories, and the regularization term enables the difference between the internal features of different categories to be as large as possible.
In this embodiment, different medical images are different medical image pictures in the same batch. For a certain target class k to be segmented (k belongs to { 1.,. c }), c is the target class to be segmented, the quantity of medical image data of each group in the medical image segmentation model with training is B', and each medical image is BqWherein q ∈ { 1., B' }, then the medical image BqThe extracted characteristics of a certain target class k to be segmented are
The similarity between features is measured by a cosine similarity calculation method, the cosine similarity is recorded as S (x, y), and the cosine similarity is calculated in the following manner:
in this example, x is
y is
Then
Should be close to 1, m, n e {1,. and B' }. l (x, y) represents a function that measures the dissimilarity between x and y, where x is in this embodiment
y is 1, the absolute value loss is adopted in calculation, and the formula is as follows:
then the intra-class feature consistency constraint penalty LcomThe calculation formula is as follows:
calculating to obtain the intra-class feature consistency constraint loss L according to a formulacom。
Inter-class feature diversity constraint loss LdivConstraint loss L of consistency of computing mode and intra-class featurescomSimilarly, but it differs from intra-class feature consistency constraint loss computation in that the similarity of features between different classes should be as close to 0 as possible, and therefore, the cumulative feature difference constraint loss LdivCan be expressed as:
calculating according to a formula to obtain a classInter-feature dissimilarity constraint loss Lcom。
And step S223, obtaining a segmentation loss function according to the class segmentation fraction p' and the real Mask corresponding to the medical image data for training.
Due to pyramid bottom feature f1The method has the best positioning capability and improved classification capability, so that the method disclosed by the invention can be used for independently calculating a segmentation loss by using a real Mask corresponding to training medical image data corresponding to the real Mask, and enhancing the precision of a medical image segmentation model.
In this embodiment, in step S219, the pyramid bottom feature f for training is used1'obtaining a class segmentation score p' which generates a segmentation loss L with respect to a true Mask corresponding to the medical image data for training corresponding to the class segmentation score psegHere, the real Mask corresponding to the medical image data for training is denoted as M. In the present embodiment, the division loss LsegA simple cross entropy loss function is used for the calculation.
And step S225, training to obtain the medical image segmentation model according to the multi-scale self-adaptive segmentation loss function, the intra-class consistency constraint loss function, the inter-class difference constraint loss function and the segmentation loss function.
In this embodiment, the formula for calculating the total loss L is:
wherein s is the number of the multi-scale feature maps, and is 4 in the embodiment;
represents the weight lost by the scale-adaptive segmentation produced by the ith scale,
representing the intra-class feature consistency constraint penalty L generated by the ith scalecomThe weight of (a) is determined,
representing the constrained loss L of the inter-class feature diversity generated by the ith scaledivThe weight of (a) is determined,
represents the final segmentation loss L generated by the ith scalesegThe weight of (c).
And finally, synthesizing the four loss functions and training the network at the same time to obtain a medical image segmentation model.
It should be noted that the method of the embodiments of the present disclosure may be executed by a single device, such as a computer or a server. The method of the embodiment can also be applied to a distributed scene and completed by the mutual cooperation of a plurality of devices. In such a distributed scenario, one of the devices may only perform one or more steps of the method of the embodiments of the present disclosure, and the devices may interact with each other to complete the method.
It should be noted that the above describes some embodiments of the disclosure. Other embodiments are within the scope of the following claims. In some cases, the actions or steps recited in the claims may be performed in a different order than in the embodiments described above and still achieve desirable results. In addition, the processes depicted in the accompanying figures do not necessarily require the particular order shown, or sequential order, to achieve desirable results. In some embodiments, multitasking and parallel processing may also be possible or may be advantageous.
Based on the same inventive concept, corresponding to any of the above embodiments, the present disclosure further provides an electronic device, which includes a memory, a processor, and a computer program stored in the memory and executable on the processor, and when the processor executes the program, the medical image segmentation method according to any of the above embodiments is implemented.
Fig. 3 is a schematic diagram illustrating a more specific hardware structure of an electronic device according to this embodiment, where the electronic device may include: a processor 1010, a memory 1020, an input/output interface 1030, a communication interface 1040, and a bus 1050. Wherein the processor 1010, memory 1020, input/output interface 1030, and communication interface 1040 are communicatively coupled to each other within the device via bus 1050.
The processor 1010 may be implemented by a general-purpose CPU (Central Processing Unit), a microprocessor, an Application Specific Integrated Circuit (ASIC), or one or more Integrated circuits, and is configured to execute related programs to implement the technical solutions provided in the embodiments of the present disclosure.
The Memory 1020 may be implemented in the form of a ROM (Read Only Memory), a RAM (Random Access Memory), a static storage device, a dynamic storage device, or the like. The memory 1020 may store an operating system and other application programs, and when the technical solution provided by the embodiments of the present specification is implemented by software or firmware, the relevant program codes are stored in the memory 1020 and called to be executed by the processor 1010.
The input/output interface 1030 is used for connecting an input/output module to input and output information. The i/o module may be configured as a component in a device (not shown) or may be external to the device to provide a corresponding function. The input devices may include a keyboard, a mouse, a touch screen, a microphone, various sensors, etc., and the output devices may include a display, a speaker, a vibrator, an indicator light, etc.
The communication interface 1040 is used for connecting a communication module (not shown in the drawings) to implement communication interaction between the present apparatus and other apparatuses. The communication module can realize communication in a wired mode (such as USB, network cable and the like) and also can realize communication in a wireless mode (such as mobile network, WIFI, Bluetooth and the like).
It should be noted that although the above-mentioned device only shows the processor 1010, the memory 1020, the input/output interface 1030, the communication interface 1040 and the bus 1050, in a specific implementation, the device may also include other components necessary for normal operation. In addition, those skilled in the art will appreciate that the above-described apparatus may also include only those components necessary to implement the embodiments of the present description, and not necessarily all of the components shown in the figures.
The electronic device of the above embodiment is used to implement the corresponding medical image segmentation method in any of the foregoing embodiments, and has the beneficial effects of the corresponding method embodiment, which are not described herein again.
Those of ordinary skill in the art will understand that: the discussion of any embodiment above is meant to be exemplary only, and is not intended to intimate that the scope of the disclosure, including the claims, is limited to these examples; within the idea of the present disclosure, also technical features in the above embodiments or in different embodiments may be combined, steps may be implemented in any order, and there are many other variations of the different aspects of the embodiments of the present disclosure as described above, which are not provided in detail for the sake of brevity.
In addition, well-known power/ground connections to Integrated Circuit (IC) chips and other components may or may not be shown in the provided figures for simplicity of illustration and discussion, and so as not to obscure the embodiments of the disclosure. Furthermore, devices may be shown in block diagram form in order to avoid obscuring embodiments of the present disclosure, and this also takes into account the fact that specifics with respect to implementation of such block diagram devices are highly dependent upon the platform within which the embodiments of the present disclosure are to be implemented (i.e., specifics should be well within purview of one skilled in the art). Where specific details (e.g., circuits) are set forth in order to describe example embodiments of the disclosure, it should be apparent to one skilled in the art that the embodiments of the disclosure can be practiced without, or with variation of, these specific details. Accordingly, the description is to be regarded as illustrative instead of restrictive.
While the present disclosure has been described in conjunction with specific embodiments thereof, many alternatives, modifications, and variations of these embodiments will be apparent to those of ordinary skill in the art in light of the foregoing description. For example, other memory architectures (e.g., dynamic ram (dram)) may use the discussed embodiments.
The disclosed embodiments are intended to embrace all such alternatives, modifications and variances which fall within the broad scope of the appended claims. Therefore, any omissions, modifications, equivalents, improvements, and the like that may be made within the spirit and principles of the embodiments of the disclosure are intended to be included within the scope of the disclosure.
Claims (10)
1. A medical image segmentation method comprises the following steps:
acquiring medical image data;
preprocessing the medical image data to obtain processed medical image data;
inputting the processed medical image data into a medical image segmentation model;
obtaining pyramid bottom characteristics f of the medical image according to the processed medical image data in the medical image segmentation model1(ii) a According to the pyramid bottom characteristic f of the medical image1Obtaining a prediction score p;
and obtaining a segmentation result P of the medical image according to the prediction fraction P.
2. The method of claim 1, wherein the training process of the medical image segmentation model comprises:
acquiring medical image data for training;
acquiring the number c of classes to be segmented of medical image data for training;
acquiring a real Mask corresponding to medical image data for training, wherein the real Mask corresponding to the medical image data for training is as follows: pre-labeling the medical image data for training with a good result pixel by pixel;
according to the medical image data for training, carrying out thresholding, normalization and data enhancement processing on the medical image data for training to obtain the processed medical image data for training;
performing initial multi-scale feature extraction on the processed medical image data for training to obtain a multi-scale feature pyramid of the processed medical image data for training;
performing multi-scale feature fusion on the multi-scale feature pyramid of the processed medical image data for training to obtain an enhanced feature pyramid for training;
obtaining the bottom feature f of the pyramid for training according to the enhanced feature pyramid for training1′；
Extracting weights of the training enhanced feature pyramid aiming at the number c of all the classes to be segmented to obtain scale self-adaptive attention class weights;
obtaining a multi-scale self-adaptive segmentation loss function according to the scale self-adaptive attention category weight;
according to the pyramid tower bottom characteristic f for training1', obtaining a class segmentation score p' and a target feature;
obtaining an intra-class consistency constraint loss function and an inter-class difference constraint loss function according to the target characteristics;
obtaining a segmentation loss function according to the class segmentation fraction p' and a real Mask corresponding to the medical image data for training;
and training to obtain the medical image segmentation model according to the multi-scale self-adaptive segmentation loss function, the intra-class consistency constraint loss function, the inter-class difference constraint loss function and the segmentation loss function.
3. The method of claim 2, wherein deriving the scale-adaptive attention class weight comprises:
according to a certain characteristic f in the enhanced characteristic pyramid for trainingi' after convolution operation and average pooling, the data is inputted to a first full-connection network FC1Activated using the ReLU activation function and re-input into the second fully connected network FC2Using Sigmoid activation functionActivating to obtain the scale self-adaptive attention class weight wi。
4. The method of claim 2, wherein obtaining the target feature comprises:
according to the pyramid tower bottom characteristic f for training1' after 2d convolution operation and self-adaptive pooling are carried out on the target feature f, flattening operation and dimension replacement operation are carried out, and then 1d convolution operation is carried out to obtain the target feature f1 c′。
5. The method of claim 1, wherein the processed medical image data is obtained by thresholding and normalizing the medical image data.
6. The method of claim 1, wherein the pyramid bottom feature f of the medical image is obtained1The method specifically comprises the following steps:
extracting initial multi-scale features of the processed medical image data to obtain a multi-scale feature pyramid;
performing multi-scale feature fusion on the multi-scale feature pyramid to obtain an enhanced feature pyramid;
obtaining pyramid bottom characteristics f of the medical image according to the enhanced characteristic pyramid1。
7. The method according to claim 1, wherein the prediction score p is obtained by 2d convolution operation for pyramid bottom features of the medical image.
8. The method of claim 7, wherein the predictive score, p, scale is of size c H W; c is the number c of classes to be segmented of the medical image data for training; h is the length of the prediction fraction p, and W is the width of the prediction fraction p.
9. The method of claim 1, wherein the medical image is a top-most image of a patientFinal segmentation result P is passed
Calculating to obtain;
the above-mentioned
In, plij,l∈[1,c],i∈[1,H],j∈[1,W]The value of the prediction score of the ith class to be segmented in the prediction score vector of each pixel position (i, j) in the prediction score p;
l∈[1,c]for a vector p of prediction scores for a certain pixel position (i, j) in the prediction score pijC is the number c of classes to be segmented of the medical image data for training.
10. An electronic device comprising a memory, a processor and a computer program stored on the memory and executable on the processor, the processor implementing the method of any one of claims 1 to 8 when executing the program.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202111035523.6A CN113902674B (en) | 2021-09-02 | 2021-09-02 | Medical image segmentation method and electronic equipment |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202111035523.6A CN113902674B (en) | 2021-09-02 | 2021-09-02 | Medical image segmentation method and electronic equipment |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN113902674A true CN113902674A (en) | 2022-01-07 |
| CN113902674B CN113902674B (en) | 2024-05-24 |
Family
ID=79188485
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202111035523.6A Active CN113902674B (en) | 2021-09-02 | 2021-09-02 | Medical image segmentation method and electronic equipment |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN113902674B (en) |
Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN105069790A (en) * | 2015-08-06 | 2015-11-18 | 潍坊学院 | Rapid imaging detection method for gear appearance defect |
| US20200184278A1 (en) * | 2014-03-18 | 2020-06-11 | Z Advanced Computing, Inc. | System and Method for Extremely Efficient Image and Pattern Recognition and Artificial Intelligence Platform |
| CN112085744A (en) * | 2020-09-04 | 2020-12-15 | 厦门大学 | Renal tumor segmentation method based on multi-scale feature learning |
| WO2021017372A1 (en) * | 2019-08-01 | 2021-02-04 | 中国科学院深圳先进技术研究院 | Medical image segmentation method and system based on generative adversarial network, and electronic equipment |
| CN113205523A (en) * | 2021-04-29 | 2021-08-03 | 浙江大学 | Medical image segmentation and identification system, terminal and storage medium with multi-scale representation optimization |
-
2021
- 2021-09-02 CN CN202111035523.6A patent/CN113902674B/en active Active
Patent Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20200184278A1 (en) * | 2014-03-18 | 2020-06-11 | Z Advanced Computing, Inc. | System and Method for Extremely Efficient Image and Pattern Recognition and Artificial Intelligence Platform |
| CN105069790A (en) * | 2015-08-06 | 2015-11-18 | 潍坊学院 | Rapid imaging detection method for gear appearance defect |
| WO2021017372A1 (en) * | 2019-08-01 | 2021-02-04 | 中国科学院深圳先进技术研究院 | Medical image segmentation method and system based on generative adversarial network, and electronic equipment |
| CN112085744A (en) * | 2020-09-04 | 2020-12-15 | 厦门大学 | Renal tumor segmentation method based on multi-scale feature learning |
| CN113205523A (en) * | 2021-04-29 | 2021-08-03 | 浙江大学 | Medical image segmentation and identification system, terminal and storage medium with multi-scale representation optimization |
Non-Patent Citations (3)
| Title |
|---|
| 和超;张印辉;何自芬;: "多尺度特征融合工件目标语义分割", 中国图象图形学报, no. 03, 16 March 2020 (2020-03-16) * |
| 胡锦美;李佐勇: "基于改进的数学形态学医学影像分割算法", 计算机仿真, vol. 28, no. 5, 15 May 2011 (2011-05-15) * |
| 陈思伃;许文俊;田宝玉;贺志强: "一种基于神经网络的传播模型自适应分类校正算法", 中国通信学会通信理论与信号处理专业委员会2010年通信理论与信号处理学术年会, 31 December 2010 (2010-12-31) * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN113902674B (en) | 2024-05-24 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| Yap et al. | Deep learning in diabetic foot ulcers detection: A comprehensive evaluation | |
| US10482603B1 (en) | Medical image segmentation using an integrated edge guidance module and object segmentation network | |
| Liu et al. | A deep spatial contextual long-term recurrent convolutional network for saliency detection | |
| KR20210048523A (en) | Image processing method, apparatus, electronic device and computer-readable storage medium | |
| CN111448581A (en) | System and method for image processing using deep neural networks | |
| George et al. | Computer assisted recognition of breast cancer in biopsy images via fusion of nucleus-guided deep convolutional features | |
| CN112488923A (en) | Image super-resolution reconstruction method and device, storage medium and electronic equipment | |
| Chen et al. | Multi-SVM based Dempster–Shafer theory for gesture intention understanding using sparse coding feature | |
| Sharma et al. | Li-SegPNet: Encoder-decoder mode lightweight segmentation network for colorectal polyps analysis | |
| CN111694954B (en) | Image classification method and device and electronic equipment | |
| CN111709481A (en) | Tobacco disease identification method, system, platform and storage medium | |
| CN114548213A (en) | Model training method, image recognition method, terminal device, and computer medium | |
| CN113850796A (en) | Lung disease identification method and device based on CT data, medium and electronic equipment | |
| Yadav et al. | End-to-end bare-hand localization system for human–computer interaction: a comprehensive analysis and viable solution | |
| CN111414823B (en) | Human body characteristic point detection method and device, electronic equipment and storage medium | |
| CN113781387A (en) | Model training method, image processing method, device, equipment and storage medium | |
| Aboulmira et al. | Ensemble learning methods for deep learning: Application to skin lesions classification | |
| CN116168439A (en) | Lightweight lip language identification method and related equipment | |
| CN113902674B (en) | Medical image segmentation method and electronic equipment | |
| CN113111879B (en) | Cell detection method and system | |
| Saleh et al. | Identification of facial skin diseases from face phenotypes using FSDNet in uncontrolled environment | |
| CN111598144B (en) | Training method and device for image recognition model | |
| CN114332463A (en) | MR brain tumor image example segmentation method, device, equipment and storage medium | |
| Liu et al. | Histopathologic cancer detection by dense-attention network with incorporation of prior knowledge | |
| Johnson et al. | A study on eye fixation prediction and salient object detection in supervised saliency |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |