CN112837325A - Medical image processing method, device, electronic equipment and medium - Google Patents
Medical image processing method, device, electronic equipment and medium Download PDFInfo
- Publication number
- CN112837325A CN112837325A CN202110105819.4A CN202110105819A CN112837325A CN 112837325 A CN112837325 A CN 112837325A CN 202110105819 A CN202110105819 A CN 202110105819A CN 112837325 A CN112837325 A CN 112837325A
- Authority
- CN
- China
- Prior art keywords
- lesion
- image
- segmentation
- model
- candidate
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/10—Segmentation; Edge detection
- G06T7/11—Region-based segmentation
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/21—Design or setup of recognition systems or techniques; Extraction of features in feature space; Blind source separation
- G06F18/214—Generating training patterns; Bootstrap methods, e.g. bagging or boosting
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/10—Segmentation; Edge detection
- G06T7/143—Segmentation; Edge detection involving probabilistic approaches, e.g. Markov random field [MRF] modelling
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/40—Extraction of image or video features
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/10—Image acquisition modality
- G06T2207/10004—Still image; Photographic image
Abstract
The embodiment of the application discloses a medical image processing method, a medical image processing device, electronic equipment and a medium, wherein the method comprises the steps of inputting a medical image to be detected into a pre-trained feature extraction model to extract a multi-dimensional feature image; inputting the multi-dimensional feature image into a pre-trained candidate frame generation model to generate a plurality of candidate frame coordinates and a first prediction probability corresponding to each candidate frame; intercepting a corresponding segmentation candidate image in the medical image to be detected according to each candidate frame; inputting each segmentation candidate image into a lesion segmentation model trained in advance, and generating a predicted lesion boundary and a corresponding second prediction probability corresponding to each segmentation candidate image; and generating a target lesion boundary in the medical image to be detected based on the corresponding predicted lesion boundary, the corresponding first prediction probability and the second prediction probability of each segmentation candidate image. The method and the device improve the accuracy of early canceration boundary prediction.
Description
Technical Field
The present application relates to the field of artificial intelligence technologies, and in particular, to a method and an apparatus for processing a medical image, an electronic device, and a medium.
Background
The human body may have malignant cancer, such as gastric cancer and cancer of the middle-sized intestine, due to genetics, genetic variation, long-term eating habits, and work and rest habits. The cancerous tissues can be divided into early stage, advanced stage and late stage according to the pathological changes, and different treatment methods are applied to different stages. Where the difficulty of treatment for finding cancer at an early stage is minimal and the chances of patient recovery are greatest. In the prior art, early canceration is usually detected through medical image images, but because the morphological and color characteristics of early canceration cells and non-pathological cells are not obvious, the diagnosis of early cancer belongs to the problem of fine image detection, doctors are difficult to find through medical image observation, and in addition, after the early cancer is diagnosed, the cancerous part is usually required to be accurately resected through a physical resection mode, so how to accurately predict the boundary of pathological change and normal cells is an extremely important link.
With the development of artificial intelligence, in the prior art, the shape boundary of early cancer is predicted by training an image segmentation model based on medical image images, in the technology, the early cancer boundary is used as an image segmentation task, the image segmentation model has a very accurate segmentation effect on common targets such as vehicles, pedestrians, airplanes, flowers and plants, but the difference between a large number of lesion parts and normal cells is very slight, so that the segmentation effect on early cancer is poor, the prediction accuracy on the early cancer boundary is low, and a large amount of over-detection is generated. Therefore, how to provide a medical image processing technology and improve the accuracy of the early canceration boundary prediction is an urgent technical problem to be solved.
Disclosure of Invention
Embodiments of the present application provide a method, an apparatus, an electronic device, and a medium for processing a medical image, so as to solve the technical problems mentioned in the background art, or at least partially solve the technical problems mentioned in the background art.
In a first aspect, an embodiment of the present application provides a medical image processing method, including:
inputting a medical image to be detected into a pre-trained feature extraction model to extract a multi-dimensional feature image;
inputting the multi-dimensional feature image into a pre-trained candidate frame generation model to generate a plurality of candidate frame coordinates and a first prediction probability corresponding to each candidate frame;
intercepting a corresponding segmentation candidate image in the medical image to be detected according to each candidate frame;
inputting each segmentation candidate image into a lesion segmentation model trained in advance, and generating a predicted lesion boundary and a corresponding second prediction probability corresponding to each segmentation candidate image;
and generating a target lesion boundary in the medical image to be detected based on the corresponding predicted lesion boundary, the corresponding first prediction probability and the second prediction probability of each segmentation candidate image.
In a second aspect, an embodiment of the present application provides a medical image processing apparatus, including:
the characteristic extraction module is used for inputting the medical image to be detected into a pre-trained characteristic extraction model to extract multi-dimensional characteristic images;
the candidate frame generation module is used for inputting the multi-dimensional feature images into a pre-trained candidate frame generation model to generate a plurality of candidate frame coordinates and a first prediction probability corresponding to each candidate frame;
the image extraction module is used for intercepting corresponding segmentation candidate images in the medical image to be detected according to each candidate frame;
the lesion segmentation module is used for inputting each segmentation candidate image into a lesion segmentation model trained in advance and generating a predicted lesion boundary and a second predicted probability corresponding to each segmentation candidate image;
and the target generation module is used for generating a target lesion boundary in the medical image to be detected based on the corresponding predicted lesion boundary, the corresponding first prediction probability and the second prediction probability of each segmentation candidate image.
In a third aspect, an embodiment of the present application provides an electronic device, including: at least one processor; and a memory communicatively coupled to the at least one processor; wherein the memory stores instructions executable by the at least one processor, the instructions being arranged to perform the method of the first aspect of the application.
In a fourth aspect, an embodiment of the present application provides a computer-readable storage medium, where the computer instructions are used to execute the method described in the first aspect of the present application.
According to the technical scheme, feature maps with different dimensions are extracted from a medical image to be detected, a candidate frame of a local possible lesion area is determined based on the feature maps with different dimensions, the local area of the possible lesion is taken from the original medical image to be detected, namely the candidate image is segmented, and then the local area of the possible lesion is analyzed respectively to predict a lesion boundary; and comprehensively judging whether the segmentation candidate image is a lesion area according to the first prediction probability and the second prediction probability, and finally generating a target lesion boundary in the medical image to be detected. According to the method and the device, the lesion boundary is predicted through the local possible lesion area and is finally displayed on the whole image to be detected instead of directly performing prediction analysis on the whole image to be detected, so that the characteristics of the lesion area are more concerned in the whole boundary prediction process, the classification influence of normal cells around the lesion on lesion cells is reduced, and the accuracy of early canceration boundary prediction is improved.
Drawings
The above and other objects, features and advantages of the present application will become more apparent by describing in more detail embodiments of the present application with reference to the attached drawings. The accompanying drawings are included to provide a further understanding of the embodiments of the application and are incorporated in and constitute a part of this specification, illustrate embodiments of the application and together with the description serve to explain the principles of the application.
Fig. 1 is a flowchart illustrating a medical image processing method according to an embodiment of the present application;
FIG. 2 is a flow chart of training of the feature extraction model and candidate box generation model described in an embodiment of the present application;
FIG. 3 is a flowchart illustrating a lesion segmentation model training process according to an embodiment of the present application;
fig. 4 is a schematic diagram of a medical image processing apparatus according to an embodiment of the present application.
Detailed Description
In order to make the objects, technical solutions and advantages of the embodiments of the present application clearer, the technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the drawings in the embodiments of the present application, and it is obvious that the described embodiments are some embodiments of the present application, but not all embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present application.
Aiming at the problem of early cancer boundary prediction under the condition of small sample number, the method provides a technical logic of 'positioning and segmentation', combines a target detection technology and an image segmentation technology according to the processing logic of the method for medical image images, preferentially solves the problem of missed detection, improves the accuracy rate of model detection, and avoids excessive false detection.
An embodiment of the present application provides a medical image processing method, as shown in fig. 1, including the following steps:
s1, inputting the medical image to be detected into a pre-trained feature extraction model to extract a multi-dimensional feature image;
step S2, inputting the multi-dimensional feature image into a pre-trained candidate frame generation model to generate a plurality of candidate frame coordinates and a first prediction probability corresponding to each candidate frame;
step S3, intercepting corresponding segmentation candidate images in the medical image to be detected according to each candidate frame;
step S4, inputting each segmentation candidate image into a lesion segmentation model trained in advance, and generating a predicted lesion boundary and a second predicted probability corresponding to each segmentation candidate image;
and step S5, generating a target lesion boundary in the medical image to be detected based on the corresponding predicted lesion boundary, the corresponding first predicted probability and the second predicted probability of each segmentation candidate image.
The medical image to be measured may be a medical image for detecting cancer in the stomach, a medical image for detecting middle-intestine cancer, or the like. The medical image for detecting gastric cancer may be a white light image of the stomach acquired by a gastroscopic system. It can be understood that the medical image to be measured is not limited to the medical image of the stomach and the gastrointestinal cancer, but may be a medical image of other human body parts; the image style is not limited to a white light image, but may also be a CT image, etc., and the types of images acquired by different sensors are different, which is not limited in the present application. The multi-dimensional feature image is an image extracted based on a medical image to be detected through different dimensions, and specific image features can include shapes, colors, textures and the like. Since the medical image acquisition process is usually a dynamic process, and the imaging size, angle, illumination condition and the like of the same lesion are changed at any time along with the movement of the camera, the missed detection is easily generated, so that the problem can be effectively solved through dimension extraction, and the accuracy of lesion boundary prediction is improved.
The method comprises the steps of extracting feature maps with different dimensions from a medical image to be detected, determining a candidate frame of a local possible lesion area based on the feature maps with different dimensions, taking the local area of the possible lesion from the original medical image to be detected, namely segmenting the candidate image, analyzing the local area of the possible lesion respectively, predicting the lesion boundary, reducing the omission ratio, ensuring that the omission ratio is not detected as much as possible, comprehensively judging whether the segmented candidate image is the lesion area according to a first prediction probability and a second prediction probability on the basis, further improving the prediction accuracy, reducing the omission ratio, and finally generating the target lesion boundary on the medical image to be detected. According to the method and the device, the lesion boundary is predicted through the local possible lesion area and is finally displayed on the whole image to be detected instead of directly performing prediction analysis on the whole image to be detected, so that the characteristics of the lesion area are more concerned in the whole boundary prediction process, the classification influence of normal cells around the lesion on lesion cells is reduced, and the accuracy of early cancer boundary prediction is improved.
It should be understood that the various steps recited in the method embodiments of the present application may be performed in a different order and/or in parallel. Moreover, method embodiments may include additional steps and/or omit performing the illustrated steps. The scope of the present application is not limited in this respect.
As an embodiment, the method further includes step S10, training to obtain the feature extraction model and the candidate box generation model, as shown in fig. 2, and specifically includes:
s101, setting initial parameters of the feature extraction model and the candidate frame generation model;
step S102, inputting a plurality of medical image sample images marked with real lesion rectangular frames into the feature extraction model as supervision data to generate a multi-dimensional feature image corresponding to each sample image;
specifically, a backbone network of CNN (convolutional neural network) may be used to extract features of the input image.
Step S103, inputting the multi-dimensional feature image corresponding to each sample image into the candidate frame generation model to generate a plurality of candidate frame coordinates corresponding to each sample image and a first prediction probability corresponding to each candidate frame;
specifically, a one-stage or two-stage target detection algorithm may be used to generate a candidate frame that may include a lesion.
Step S104, acquiring a first loss function based on a plurality of candidate frame coordinates corresponding to each sample image and a first prediction probability, a lesion area true value coordinate and a lesion real probability value corresponding to each candidate frame;
the first loss function may be set to:
L(l,l*,p,p*)＝CE(p,p*)+SmoothL1(l,l*)
wherein l, l*,p,p*Respectively representing the coordinate of a candidate frame predicted by the model, the true value coordinate of a lesion area, a first prediction probability and the true probability value of the lesion, wherein CE represents the cross entropy SmoothL1Indicating a loss of smoothness L1.
And S105, adjusting parameters of the feature extraction model and the candidate frame generation model based on the first loss function, returning to the step S102 for the next round of training until the first loss function meets a preset first state.
It can be understood that the condition that the first loss function meets the preset first state is that the feature extraction model and the candidate frame generation model are ended, specifically, a first loss function threshold may be set, and when values of M consecutive first loss functions are smaller than the first loss function threshold, it is determined that the first loss function meets the preset first state. The first state may also be set as a first loss function convergence, and when the first loss function converges, it is determined that the first loss function conforms to a preset first state. With continuous training of the candidate frame generation model, the candidate frame generation model can gradually obtain lesion candidate frames with continuously improved quality, wherein the real lesion areas can be recalled completely, meanwhile, a certain amount of over-detection can be generated, and the occurrence of missing detection is avoided.
According to the embodiment of the application, a plurality of medical image sample images marked with the real lesion rectangular frames are used as supervision data to jointly train the feature extraction model and the candidate frame generation model, a plurality of segmentation candidate areas corresponding to the medical image images to be detected can be generated based on the training feature extraction model and the candidate frame generation model, the real lesions can be guaranteed to be recalled in a high proportion by reasonably setting the training parameters and the threshold, and the problem of the most serious missed detection in early cancer detection is solved.
As an embodiment, the method further includes step S20, training to obtain the lesion segmentation model, as shown in fig. 3, specifically including:
step S201, setting initial model parameters of the lesion segmentation model;
step S202, intercepting a lesion area image corresponding to a lesion boundary from a plurality of medical image images for marking a real lesion boundary as a positive sample, randomly intercepting a normal cell area image as a negative sample, and inputting the lesion segmentation model;
step S203, the lesion segmentation model segments each sample area image into a plurality of sub-areas, traverses each sub-area to obtain a third prediction probability of each sub-area, predicts the sub-area with the prediction probability being greater than or equal to a preset third threshold as a lesion sub-area, and predicts the sub-area with the prediction probability being smaller than the third threshold as a normal sub-area;
it is understood that the third threshold is set according to parameters such as the accuracy of the lesion segmentation model, the prediction accuracy, and the like, for example, the third threshold may be set to 0.7, sub-regions with a third prediction probability exceeding 0.7 are all predicted to be cancerous, the sub-regions may include one or more pixels, and the smaller the sub-region includes pixels, the higher the prediction accuracy.
Step S204, generating predicted lesion boundary coordinates and a second predicted probability corresponding to the sample region image based on the predicted lesion sub-region, normal sub-region and the third predicted probability of each sub-region of each sample region image;
it should be noted that the second prediction probability corresponding to the sample region image may be a mean value of the third prediction probabilities of all the sub-regions corresponding to the sample region image, or a minimum value of the third prediction probabilities of all the sub-regions.
Step S205, obtaining a second loss function according to the real lesion boundary coordinate, the predicted lesion boundary coordinate, the real lesion probability and the second predicted probability corresponding to the sample region image;
and S206, adjusting parameters of the lesion segmentation model based on the second loss function, returning to the step S202 for the next round of training until the second loss function conforms to a preset second state.
It can be understood that the second loss function meets the condition that the preset second state is the end of the lesion segmentation model, and specifically, a second loss function threshold may be set, and when values of N consecutive second loss functions are smaller than the second loss function threshold, it is determined that the second loss function meets the preset second state. The second state may also be set to a second loss function convergence, and when the second loss function converges, it is determined that the second loss function conforms to the preset second state. As an example, the lesion segmentation model may be a CNN image segmentation model for obtaining an accurate contour of a lesion region in all segmentation candidate images. With the training, the lesion segmentation model can gradually and accurately distinguish real lesion regions from non-lesion regions.
According to the embodiment of the application, the lesion area image corresponding to the lesion boundary is intercepted from the medical image for marking the real lesion boundary and used as the positive sample, and the normal cell area image is randomly intercepted and used as the negative sample to train the lesion segmentation model, so that a large number of positive and negative samples can be generated, and the problem of small data volume in the deep learning medical project is solved. In addition, a series of segmentation candidate graphs obtained from the original medical image extraction module 3 to be detected are generated based on the candidate frame to carry out lesion segmentation analysis, so that the lesion segmentation model is more concerned with the characteristics of the lesion in the training and using processes, the classification influence of surrounding normal cells on lesion cells is reduced, the false detection rate of the model is greatly reduced, and the accuracy of early-stage cancer boundary prediction is improved.
After the feature extraction model, the candidate frame generation model and the lesion segmentation model are trained respectively, the feature extraction model, the candidate frame generation model and the lesion segmentation model are combined according to the processing logic of the medical image to be detected for subsequent use.
In the process of training or using the feature extraction model, as an embodiment, the method further includes step S11, scaling the medical image to be detected to a preset first size, and then inputting the scaled medical image to the pre-trained feature extraction model to extract the multi-dimensional feature image. And in the process of training and using the lesion segmentation model, the method further includes step S12, scaling each segmentation candidate image to a preset second size, and inputting the scaled segmentation candidate image into the lesion segmentation model trained in advance, wherein the first size and the second size are the same or different, and generally the first size and the second size are positively correlated with the accuracy of the calculation result and the calculation amount, so that the first size and the second size can be specifically set according to the accuracy requirement of the calculation result and the calculation amount that can be adapted by the current calculation resource.
Based on the steps S11 and S12, as an embodiment, the step S5 may include:
step S51, multiplying the first prediction probability and the second prediction probability corresponding to each segmentation candidate image to obtain a corresponding fourth prediction probability;
step S52, comparing a fourth prediction probability corresponding to each segmentation candidate image with a preset fourth probability threshold, and if the fourth prediction probability is greater than or equal to the fourth probability threshold, determining the segmentation candidate image as a target lesion area;
step S53, determining the corresponding position of each target lesion area on the detected medical image according to the coordinates and the zoom factor of the target lesion area, so as to display the predicted lesion boundary corresponding to the target lesion area on the detected medical image.
And each segmentation candidate image comprehensively obtains a fourth prediction probability based on the first prediction probability and the second prediction probability, and the lesion prediction is carried out based on the image of the local possible lesion, and the lesion boundary is predicted based on the region of the local possible lesion, so that the accuracy of the prediction of the early canceration boundary is improved. And finally, a predicted lesion boundary corresponding to the target lesion area can be correspondingly presented on the original image, so that the user experience is improved.
The following are embodiments of the apparatus of the present application that may be used to perform embodiments of the method of the present application. For details which are not disclosed in the embodiments of the apparatus of the present application, reference is made to the embodiments of the method of the present application. A medical image processing device is shown in FIG. 4, and is described by taking a gastroscope image as an example of a medical image to be detected in FIG. 4, the device comprises a feature extraction module 1, a candidate frame generation module 2, an image extraction module 3, a lesion segmentation module 4 and a target generation module 5, wherein the feature extraction module 1 is used for inputting the medical image to be detected into a pre-trained feature extraction model to extract a multi-dimensional feature image; the candidate frame generation module 2 is configured to input the multidimensional feature image into a candidate frame generation model trained in advance, and generate a plurality of candidate frame coordinates and a first prediction probability corresponding to each candidate frame; the image extraction module 3 is used for intercepting a corresponding segmentation candidate image in the medical image to be detected according to each candidate frame; the lesion segmentation module 4 is configured to input each segmentation candidate image into a lesion segmentation model trained in advance, and generate a predicted lesion boundary and a second predicted probability corresponding to each segmentation candidate image; the target generation module 5 is configured to generate a target lesion boundary in the medical image to be detected based on the predicted lesion boundary corresponding to each segmented candidate image, the corresponding first prediction probability, and the second prediction probability.
As an embodiment, the apparatus further includes a first model training module, configured to train to obtain the feature extraction model and the candidate box generation model, and specifically includes: setting initial parameters of the feature extraction model and the candidate frame generation model; inputting a plurality of medical image sample images marked with real lesion rectangular frames into the feature extraction model as supervision data to generate a multi-dimensional feature image corresponding to each sample image; inputting the multi-dimensional characteristic image corresponding to each sample image into the candidate frame generation model to generate a plurality of candidate frame coordinates corresponding to each sample image and a first prediction probability corresponding to each candidate frame; acquiring a first loss function based on a plurality of candidate frame coordinates corresponding to each sample image and a first prediction probability, a lesion region truth value coordinate and a lesion real probability value corresponding to each candidate frame; and adjusting parameters of the feature extraction model and the candidate frame generation model based on the first loss function, and performing next round of training until the first loss function conforms to a preset first state.
As an embodiment, the apparatus further includes a second model training module, configured to train to obtain the lesion segmentation model, which specifically includes: setting initial model parameters of the lesion segmentation model; intercepting lesion area images corresponding to lesion boundaries from a plurality of medical image images for marking real lesion boundaries to serve as positive samples, randomly intercepting normal cell area images to serve as negative samples, and inputting the lesion segmentation models; the lesion segmentation model segments each sample area image into a plurality of sub-areas, traverses each sub-area to obtain a third prediction probability of each sub-area, predicts the sub-area with the prediction probability being greater than or equal to a preset third threshold as a lesion sub-area, and predicts the sub-area with the prediction probability being smaller than the third threshold as a normal sub-area; generating a predicted lesion boundary coordinate and a second predicted probability corresponding to each sample region image based on the predicted lesion sub-region, normal sub-region and the third predicted probability of each sub-region of each sample region image; acquiring a second loss function according to the real lesion boundary coordinate, the predicted lesion boundary coordinate, the real lesion probability and the second predicted probability corresponding to the sample region image; and adjusting parameters of the lesion segmentation model based on the second loss function, and performing next round of training until the second loss function conforms to a preset second state.
As an embodiment, the feature extraction module 1 is further configured to scale the medical image to be detected to a preset first size, and then input the scaled medical image to a pre-trained feature extraction model to extract a multi-dimensional feature image. And the lesion segmentation module 4 is further configured to scale each of the segmentation candidate images to a preset second size and then input the scaled segmentation candidate images into a lesion segmentation model trained in advance, where the first size and the second size are the same or different.
As an embodiment, the target generating module 5 includes a fourth probability predicting unit, a target lesion area determining module, and a target lesion boundary generating module, where the fourth probability predicting unit is configured to multiply the first prediction probability and the second prediction probability corresponding to each segmentation candidate image to obtain a corresponding fourth prediction probability; the target lesion area determining module is used for comparing a fourth prediction probability corresponding to each segmentation candidate image with a preset fourth probability threshold, and if the fourth prediction probability is greater than or equal to the fourth probability threshold, determining the segmentation candidate image as a target lesion area; and the target lesion boundary generating module is used for determining the corresponding position of each target lesion area on the detected medical image according to the coordinate and the scaling factor of the target lesion area, so that the predicted lesion boundary corresponding to the target lesion area is displayed on the detected medical image.
The device extracts feature maps with different dimensions from a medical image to be detected, determines a candidate frame of a local possible lesion area based on the feature maps with different dimensions, extracts the local area of the possible lesion from the original medical image to be detected, namely, segments the candidate image, analyzes the local area of the possible lesion respectively, predicts the lesion boundary, reduces the omission ratio, ensures that the omission ratio is not detected as much as possible, comprehensively judges whether the segmented candidate image is the lesion area according to the first prediction probability and the second prediction probability on the basis, further improves the prediction accuracy, reduces the omission ratio, and finally generates a target lesion boundary on the medical image to be detected. According to the method and the device, the lesion boundary is predicted through the local possible lesion area and is finally displayed on the whole image to be detected instead of directly performing prediction analysis on the whole image to be detected, so that the characteristics of the lesion area are more concerned in the whole boundary prediction process, the classification influence of normal cells around the lesion on lesion cells is reduced, and the accuracy of early cancer boundary prediction is improved.
An embodiment of the present application provides an electronic device, including: at least one processor; and a memory communicatively coupled to the at least one processor; wherein the memory stores instructions executable by the at least one processor, the instructions configured to perform a method according to embodiments of the present application.
The embodiment of the present application further provides a computer-readable storage medium, where the computer instructions are used to execute the method in the embodiment of the present application.
The above description is only a preferred embodiment of the application and is illustrative of the principles of the technology employed. It will be appreciated by those skilled in the art that the scope of the disclosure herein is not limited to the particular combination of features described above, but also encompasses other arrangements formed by any combination of the above features or their equivalents without departing from the spirit of the disclosure. For example, the above features may be replaced with (but not limited to) features having similar functions disclosed in the present application.
Claims (10)
1. A medical image processing method is characterized by comprising the following steps:
inputting a medical image to be detected into a pre-trained feature extraction model to extract a multi-dimensional feature image;
inputting the multi-dimensional feature image into a pre-trained candidate frame generation model to generate a plurality of candidate frame coordinates and a first prediction probability corresponding to each candidate frame;
intercepting a corresponding segmentation candidate image in the medical image to be detected according to each candidate frame;
inputting each segmentation candidate image into a lesion segmentation model trained in advance, and generating a predicted lesion boundary and a corresponding second prediction probability corresponding to each segmentation candidate image;
and generating a target lesion boundary in the medical image to be detected based on the corresponding predicted lesion boundary, the corresponding first prediction probability and the second prediction probability of each segmentation candidate image.
2. The method of claim 1,
further comprising:
training to obtain the feature extraction model and the candidate frame generation model, and specifically comprises the following steps:
setting initial parameters of the feature extraction model and the candidate frame generation model;
inputting a plurality of medical image sample images marked with real lesion rectangular frames into the feature extraction model as supervision data to generate a multi-dimensional feature image corresponding to each sample image;
inputting the multi-dimensional characteristic image corresponding to each sample image into the candidate frame generation model to generate a plurality of candidate frame coordinates corresponding to each sample image and a first prediction probability corresponding to each candidate frame;
acquiring a first loss function based on a plurality of candidate frame coordinates corresponding to each sample image and a first prediction probability, a lesion region truth value coordinate and a lesion real probability value corresponding to each candidate frame;
and adjusting parameters of the feature extraction model and the candidate frame generation model based on the first loss function, and performing next round of training until the first loss function conforms to a preset first state.
3. The method of claim 1,
the method further comprises the following steps:
training to obtain the lesion segmentation model specifically comprises:
setting initial model parameters of the lesion segmentation model;
intercepting lesion area images corresponding to lesion boundaries from a plurality of medical image images for marking real lesion boundaries to serve as positive samples, randomly intercepting normal cell area images to serve as negative samples, and inputting the lesion segmentation models;
the lesion segmentation model segments each sample area image into a plurality of sub-areas, traverses each sub-area to obtain a third prediction probability of each sub-area, predicts the sub-area with the prediction probability being greater than or equal to a preset third threshold as a lesion sub-area, and predicts the sub-area with the prediction probability being smaller than the third threshold as a normal sub-area;
generating a predicted lesion boundary coordinate and a second predicted probability corresponding to each sample region image based on the predicted lesion sub-region, normal sub-region and the third predicted probability of each sub-region of each sample region image;
acquiring a second loss function according to the real lesion boundary coordinate, the predicted lesion boundary coordinate, the real lesion probability and the second predicted probability corresponding to the sample region image;
and adjusting parameters of the lesion segmentation model based on the second loss function, and performing next round of training until the second loss function conforms to a preset second state.
4. The method of claim 1,
the method further comprises the following steps:
zooming the medical image to be detected to a preset first size, and inputting the zoomed medical image to a pre-trained feature extraction model to extract a multi-dimensional feature image;
and the number of the first and second groups,
and scaling each segmentation candidate image to a preset second size and then inputting the scaled segmentation candidate image into a lesion segmentation model trained in advance, wherein the first size and the second size are the same or different.
5. The method of claim 4,
the generating of the target lesion boundary in the medical image to be detected based on the predicted lesion boundary corresponding to each segmentation candidate image, the corresponding first prediction probability and the second prediction probability includes:
multiplying the first prediction probability and the second prediction probability corresponding to each segmentation candidate image to obtain a corresponding fourth prediction probability;
comparing a fourth prediction probability corresponding to each segmentation candidate image with a preset fourth probability threshold, and if the fourth prediction probability is greater than or equal to the fourth probability threshold, determining the segmentation candidate image as a target lesion area;
and determining the corresponding position of each target lesion area on the detected medical image according to the coordinate and the scaling factor of the target lesion area, so as to display the predicted lesion boundary corresponding to the target lesion area on the detected medical image.
6. A medical image processing apparatus, comprising:
the characteristic extraction module is used for inputting the medical image to be detected into a pre-trained characteristic extraction model to extract multi-dimensional characteristic images;
the candidate frame generation module is used for inputting the multi-dimensional feature images into a pre-trained candidate frame generation model to generate a plurality of candidate frame coordinates and a first prediction probability corresponding to each candidate frame;
the image extraction module is used for intercepting corresponding segmentation candidate images in the medical image to be detected according to each candidate frame;
the lesion segmentation module is used for inputting each segmentation candidate image into a lesion segmentation model trained in advance and generating a predicted lesion boundary and a second predicted probability corresponding to each segmentation candidate image;
and the target generation module is used for generating a target lesion boundary in the medical image to be detected based on the corresponding predicted lesion boundary, the corresponding first prediction probability and the second prediction probability of each segmentation candidate image.
7. The apparatus of claim 6,
the method further comprises a first model training module, which is used for training to obtain the feature extraction model and the candidate box generation model, and specifically comprises the following steps:
setting initial parameters of the feature extraction model and the candidate frame generation model;
inputting a plurality of medical image sample images marked with real lesion rectangular frames into the feature extraction model as supervision data to generate a multi-dimensional feature image corresponding to each sample image;
inputting the multi-dimensional characteristic image corresponding to each sample image into the candidate frame generation model to generate a plurality of candidate frame coordinates corresponding to each sample image and a first prediction probability corresponding to each candidate frame;
acquiring a first loss function based on a plurality of candidate frame coordinates corresponding to each sample image and a first prediction probability, a lesion region truth value coordinate and a lesion real probability value corresponding to each candidate frame;
and adjusting parameters of the feature extraction model and the candidate frame generation model based on the first loss function, and performing next round of training until the first loss function conforms to a preset first state.
8. The apparatus of claim 6,
the device further comprises a second model training module, which is used for training to obtain the lesion segmentation model, and specifically comprises:
setting initial model parameters of the lesion segmentation model;
intercepting lesion area images corresponding to lesion boundaries from a plurality of medical image images for marking real lesion boundaries to serve as positive samples, randomly intercepting normal cell area images to serve as negative samples, and inputting the lesion segmentation models;
the lesion segmentation model segments each sample area image into a plurality of sub-areas, traverses each sub-area to obtain a third prediction probability of each sub-area, predicts the sub-area with the prediction probability being greater than or equal to a preset third threshold as a lesion sub-area, and predicts the sub-area with the prediction probability being smaller than the third threshold as a normal sub-area;
generating a predicted lesion boundary coordinate and a second predicted probability corresponding to each sample region image based on the predicted lesion sub-region, normal sub-region and the third predicted probability of each sub-region of each sample region image;
acquiring a second loss function according to the real lesion boundary coordinate, the predicted lesion boundary coordinate, the real lesion probability and the second predicted probability corresponding to the sample region image;
and adjusting parameters of the lesion segmentation model based on the second loss function, and performing next round of training until the second loss function conforms to a preset second state.
9. An electronic device, comprising:
at least one processor;
and a memory communicatively coupled to the at least one processor;
wherein the memory stores instructions executable by the at least one processor, the instructions being arranged to perform the method of any of the preceding claims 1-5.
10. A computer-readable storage medium having stored thereon computer-executable instructions for performing the method of any of the preceding claims 1-5.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110105819.4A CN112837325A (en) | 2021-01-26 | 2021-01-26 | Medical image processing method, device, electronic equipment and medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110105819.4A CN112837325A (en) | 2021-01-26 | 2021-01-26 | Medical image processing method, device, electronic equipment and medium |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN112837325A true CN112837325A (en) | 2021-05-25 |
Family
ID=75931740
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202110105819.4A Pending CN112837325A (en) | 2021-01-26 | 2021-01-26 | Medical image processing method, device, electronic equipment and medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN112837325A (en) |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113633306A (en) * | 2021-08-31 | 2021-11-12 | 上海商汤智能科技有限公司 | Image processing method and related device, electronic equipment and storage medium |
| WO2023103467A1 (en) * | 2021-12-09 | 2023-06-15 | 杭州海康慧影科技有限公司 | Image processing method, apparatus and device |
-
2021
- 2021-01-26 CN CN202110105819.4A patent/CN112837325A/en active Pending
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113633306A (en) * | 2021-08-31 | 2021-11-12 | 上海商汤智能科技有限公司 | Image processing method and related device, electronic equipment and storage medium |
| WO2023103467A1 (en) * | 2021-12-09 | 2023-06-15 | 杭州海康慧影科技有限公司 | Image processing method, apparatus and device |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN111325739B (en) | Method and device for detecting lung focus and training method of image detection model | |
| CN109741346B (en) | Region-of-interest extraction method, device, equipment and storage medium | |
| CN111524137B (en) | Cell identification counting method and device based on image identification and computer equipment | |
| CN111951221B (en) | Glomerular cell image recognition method based on deep neural network | |
| CN109389129B (en) | Image processing method, electronic device and storage medium | |
| CN111445478B (en) | Automatic intracranial aneurysm region detection system and detection method for CTA image | |
| JP6710373B2 (en) | Ultrasound image diagnosis support method and system | |
| CN110021425B (en) | Comparison detector, construction method thereof and cervical cancer cell detection method | |
| CN108830149B (en) | Target bacterium detection method and terminal equipment | |
| CN110929728B (en) | Image region-of-interest dividing method, image segmentation method and device | |
| CN110570350A (en) | two-dimensional follicle detection method and device, ultrasonic equipment and readable storage medium | |
| JP2013051988A (en) | Device, method and program for image processing | |
| Wittenberg et al. | Computer aided detection of polyps in whitelight-colonoscopy images using deep neural networks | |
| CN111145200B (en) | Blood vessel center line tracking method combining convolutional neural network and cyclic neural network | |
| CN112001895B (en) | Thyroid calcification detection device | |
| CN112837325A (en) | Medical image processing method, device, electronic equipment and medium | |
| CN110880177A (en) | Image identification method and device | |
| CN111916206A (en) | CT image auxiliary diagnosis system based on cascade connection | |
| Li et al. | Sublingual vein extraction algorithm based on hyperspectral tongue imaging technology | |
| CN111401102B (en) | Deep learning model training method and device, electronic equipment and storage medium | |
| Ratheesh et al. | Advanced algorithm for polyp detection using depth segmentation in colon endoscopy | |
| CN114549462A (en) | Focus detection method, device, equipment and medium based on visual angle decoupling Transformer model | |
| CN112861942A (en) | Medical image processing method, device, electronic equipment and medium | |
| KR101991452B1 (en) | Method for detecting nipple location, method for displaying breast image and apparatus for detecting nipple location | |
| CN113963427B (en) | Method and system for rapid in-vivo detection |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |