WO2022088665A1 - 病灶分割方法、装置及存储介质 - Google Patents
病灶分割方法、装置及存储介质 Download PDFInfo
- Publication number
- WO2022088665A1 WO2022088665A1 PCT/CN2021/096395 CN2021096395W WO2022088665A1 WO 2022088665 A1 WO2022088665 A1 WO 2022088665A1 CN 2021096395 W CN2021096395 W CN 2021096395W WO 2022088665 A1 WO2022088665 A1 WO 2022088665A1
- Authority
- WO
- WIPO (PCT)
- Prior art keywords
- image
- map
- category
- block
- feature
- Prior art date
Links
- 230000003902 lesion Effects 0.000 title claims abstract description 106
- 230000011218 segmentation Effects 0.000 title claims abstract description 72
- 238000000034 method Methods 0.000 title claims abstract description 53
- 238000000605 extraction Methods 0.000 claims abstract description 17
- 238000013528 artificial neural network Methods 0.000 claims description 42
- 238000012545 processing Methods 0.000 claims description 33
- 238000002372 labelling Methods 0.000 claims description 21
- 238000003709 image segmentation Methods 0.000 claims description 13
- 238000004590 computer program Methods 0.000 claims description 10
- 238000004891 communication Methods 0.000 claims description 5
- 238000005070 sampling Methods 0.000 claims description 5
- 238000005516 engineering process Methods 0.000 abstract description 4
- 230000009286 beneficial effect Effects 0.000 abstract 1
- 238000010586 diagram Methods 0.000 description 6
- 238000012549 training Methods 0.000 description 5
- 238000001514 detection method Methods 0.000 description 4
- 238000003745 diagnosis Methods 0.000 description 4
- 230000008878 coupling Effects 0.000 description 3
- 238000010168 coupling process Methods 0.000 description 3
- 238000005859 coupling reaction Methods 0.000 description 3
- 238000003384 imaging method Methods 0.000 description 2
- 230000003287 optical effect Effects 0.000 description 2
- 238000003672 processing method Methods 0.000 description 2
- 208000010412 Glaucoma Diseases 0.000 description 1
- 206010025421 Macule Diseases 0.000 description 1
- 208000017442 Retinal disease Diseases 0.000 description 1
- 230000002159 abnormal effect Effects 0.000 description 1
- 238000013473 artificial intelligence Methods 0.000 description 1
- 210000004204 blood vessel Anatomy 0.000 description 1
- 210000003161 choroid Anatomy 0.000 description 1
- 238000011161 development Methods 0.000 description 1
- 230000018109 developmental process Effects 0.000 description 1
- 238000002059 diagnostic imaging Methods 0.000 description 1
- 239000003814 drug Substances 0.000 description 1
- 230000006870 function Effects 0.000 description 1
- 238000011478 gradient descent method Methods 0.000 description 1
- 230000008595 infiltration Effects 0.000 description 1
- 238000001764 infiltration Methods 0.000 description 1
- 230000010354 integration Effects 0.000 description 1
- 238000011084 recovery Methods 0.000 description 1
- 238000011160 research Methods 0.000 description 1
- 210000001525 retina Anatomy 0.000 description 1
- 230000016776 visual perception Effects 0.000 description 1
- XLYOFNOQVPJJNP-UHFFFAOYSA-N water Substances O XLYOFNOQVPJJNP-UHFFFAOYSA-N 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/0002—Inspection of images, e.g. flaw detection
- G06T7/0012—Biomedical image inspection
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/24—Classification techniques
- G06F18/241—Classification techniques relating to the classification model, e.g. parametric or non-parametric approaches
- G06F18/2415—Classification techniques relating to the classification model, e.g. parametric or non-parametric approaches based on parametric or probabilistic models, e.g. based on likelihood ratio or false acceptance rate versus a false rejection rate
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/047—Probabilistic or stochastic networks
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/08—Learning methods
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/10—Segmentation; Edge detection
- G06T7/11—Region-based segmentation
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/10—Segmentation; Edge detection
- G06T7/136—Segmentation; Edge detection involving thresholding
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/10—Image acquisition modality
- G06T2207/10024—Color image
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20081—Training; Learning
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20084—Artificial neural networks [ANN]
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/30—Subject of image; Context of image processing
- G06T2207/30004—Biomedical image processing
- G06T2207/30041—Eye; Retina; Ophthalmic
Definitions
- the present application relates to the technical field of image recognition, and in particular, to a lesion segmentation method, device and storage medium.
- Fundus color photography is a way to check the fundus, which can be used to see the tissue structure of the fundus, analyze the normal and abnormal structure of the fundus, and determine whether there is a problem with the optic disc, blood vessels, retina or choroid of the fundus.
- image segmentation technology provides rich visual perception information for medical imaging and other applications, it can be applied to the segmentation of retinal disease-related lesions on fundus color photos.
- the inventor found that there is a big difference between the lesion segmentation of fundus images and the segmentation of natural images. Affected by the shooting light and imaging quality, the contrast of the edge of the lesion is not as clear as that of the natural image, resulting in the segmentation of fundus lesions has always been a difficult and complex challenge. .
- Embodiments of the present application provide a lesion segmentation method, device, and storage medium. Segmenting the fundus color photos through multiple dimensions can improve the segmentation accuracy of the lesion contour information.
- the embodiments of the present application provide a lesion segmentation method, including:
- the first feature map A a first category and a first mask map corresponding to each image block in the multiple image blocks corresponding to the first feature map A in the color fundus image are determined, and the first feature map Figure A is any one of the first feature maps in the plurality of first feature maps;
- each image block and the first mask map determine the category corresponding to each pixel in the color fundus image
- Segmentation of lesions is performed on the color fundus image according to the category of each pixel in the color fundus image.
- a lesion segmentation device including:
- a processing unit configured to perform feature extraction on the color fundus image to obtain a plurality of first feature maps, wherein the dimensions between any two first feature maps in the plurality of first feature maps are different;
- the processing unit is further configured to determine, according to the first feature map A, a first category and a first mask corresponding to each image block in the multiple image blocks corresponding to the first feature map A in the color fundus image.
- a film map the first feature map A is any one of the first feature maps of the plurality of first feature maps;
- the processing unit is further configured to determine the category corresponding to each pixel in the color fundus image according to the first category of each image block and the first mask map;
- the processing unit is further configured to perform lesion segmentation on the color fundus image according to the category of each pixel in the color fundus image.
- embodiments of the present application provide an electronic device, including a processor, a memory, a communication interface, and one or more programs, wherein the one or more programs are stored in the memory and configured by The processor executes to implement the following methods:
- the first feature map A a first category and a first mask map corresponding to each image block in the multiple image blocks corresponding to the first feature map A in the color fundus image are determined, and the first feature map Figure A is any one of the first feature maps in the plurality of first feature maps;
- each image block and the first mask map determine the category corresponding to each pixel in the color fundus image
- Segmentation of lesions is performed on the color fundus image according to the category of each pixel in the color fundus image.
- an embodiment of the present application provides a computer-readable storage medium, where the computer-readable storage medium stores a computer program, and the computer program causes a computer to execute the following method:
- the first feature map A a first category and a first mask map corresponding to each image block in the multiple image blocks corresponding to the first feature map A in the color fundus image are determined, and the first feature map Figure A is any one of the first feature maps in the plurality of first feature maps;
- each image block and the first mask map determine the category corresponding to each pixel in the color fundus image
- Segmentation of lesions is performed on the color fundus image according to the category of each pixel in the color fundus image.
- the color fundus image is divided into grids to perform lesion segmentation.
- the present application uses the entire fundus color photo for segmentation, and more lesion areas can be used for segmentation. Therefore, more lesion edge contour information can be used to make the segmented lesion edge contour more refined, thereby making the lesion segmentation result in the fundus color photo image more accurate, and improving the doctor's diagnosis accuracy.
- FIG. 1 is a schematic flowchart of a lesion segmentation method provided by an embodiment of the present application
- FIG. 2 is a schematic structural diagram of a neural network provided by an embodiment of the present application.
- FIG. 3 is a schematic diagram of dividing a color photograph of a fundus into blocks according to an embodiment of the present application
- FIG. 4 is a schematic flowchart of a neural network training method provided by an embodiment of the present application.
- FIG. 5 is a schematic structural diagram of a lesion segmentation device provided by an embodiment of the present application.
- FIG. 6 is a block diagram of functional units of a lesion segmentation apparatus provided in an embodiment of the present application.
- the technical solutions of the present application may relate to the technical field of artificial intelligence and/or big data, for example, may specifically relate to neural network technology, so as to realize image-based lesion segmentation.
- the data involved in this application such as various images and/or lesion segmentation results, may be stored in a database, or may be stored in a blockchain, which is not limited in this application.
- FIG. 1 is a schematic flowchart of a lesion segmentation method provided by an embodiment of the present application. The method is applied to a lesion segmentation device. The method includes the following steps:
- the lesion segmentation device acquires a color fundus image.
- the fundus color photograph image is generated by fundus color Doppler imaging, and will not be described again.
- the lesion segmentation device performs feature extraction on the color fundus image to obtain a plurality of first feature maps, wherein the dimensions of any two first feature maps in the plurality of first feature maps are different.
- Exemplarily extracting features of the color fundus image at different depths to obtain a plurality of first feature maps. Due to the different depths, the dimensions between any two first feature maps in the plurality of first feature maps are different.
- the feature extraction of the color fundus image can be realized by a neural network, which is pre-trained.
- the training process of the neural network will be described in detail later, and will not be described here. . Since each first feature map is extracted by a neural network, the first feature map is obtained by splicing multiple first sub-feature maps obtained from multiple channels of the neural network, wherein each channel corresponds to a first feature map.
- feature extraction is performed on the color fundus image through feature pyramid networks (FPN), and multiple first feature maps are obtained in multiple network layers with different depths, as shown in FIG. 2 .
- the first feature map corresponding to each dimension is only the first sub-feature map on one channel.
- the first feature map of the bottom layer is obtained by feature extraction, and the first feature map of other layers is obtained by superimposing the first feature map of the previous layer and the feature map obtained by this layer.
- each first feature map is obtained by feature extraction on the color fundus image
- one pixel of each first feature map contains information about an area in the color fundus image
- each The first feature map can be regarded as dividing the color fundus image into multiple grids, that is, multiple image blocks, and the first feature maps of different dimensions have different numbers and dimensions of the image blocks divided by the color fundus image.
- the dimension of the color fundus image is 320*320. If the dimension of a first feature map is 64*64*10, where 10 represents the number of channels, the first feature map will be The color fundus image is divided into 5*5 image blocks, and the dimension corresponding to each image block is 5*5.
- the lesion segmentation device determines, according to the first feature map A, a first category and a first mask map corresponding to each image block in the multiple image blocks corresponding to the first feature map A in the color fundus image,
- the first feature map A is any one of the plurality of first feature maps.
- the first mask map corresponding to each image block is used to represent the probability that each pixel in the image block belongs to the first category.
- the first category corresponding to each image block may be a background or a lesion, where the lesion includes at least one of the following: macula, glaucoma, water infiltration, and the like.
- the present application takes the first feature map A as an example to illustrate the method of image segmentation and image classification for the image blocks in the color fundus image
- the first feature map A is any one of the plurality of first feature maps.
- the processing method of the other first feature maps in the plurality of first feature maps is similar to the processing method of the first feature map A, and will not be described again.
- image classification is performed according to the first feature map A. That is, according to the first feature map A, a feature vector corresponding to each image block in the multiple image blocks corresponding to the first feature map A is obtained; first category.
- first feature map A a feature vector corresponding to each image block in the multiple image blocks corresponding to the first feature map A is obtained; first category.
- the features on each channel since the first feature map is obtained by splicing feature maps on multiple channels, the features on each channel The first pixel in the figure represents the feature information of the first image block. Therefore, the gray value of the first pixel in the feature map on each channel can be formed into a feature vector, and the The feature vector is used as the feature vector of the first image block, and the feature vector is input to the fully connected layer and the softmax layer to obtain the first category of the first image block.
- the first feature map A is obtained by splicing feature maps on three channels. If the gray value of the first pixel in the feature map on the first channel is 23, the gray value of the first pixel in the feature map on the second channel is 36, and the gray value of the first pixel in the feature map on the second channel is 36. The gray value of the first pixel in the feature map is 54, then the feature vector of the first image block can be obtained as [23, 36, 54].
- image segmentation can be performed on multiple image blocks corresponding to the first feature map A to obtain a first mask map corresponding to each image block, and the first mask map is used to represent The probability that each pixel in the image block belongs to the first category corresponding to the image block, the image segmentation of the first feature map A is similar to the image segmentation of the feature map through a fully convolutional network, and will not be described again.
- the feature information corresponding to each image block on the feature map on each channel can be formed into a feature map of the image block, and image segmentation is performed according to the feature map to obtain the first mask map corresponding to the image block.
- the neural network may be an FPN-based neural network.
- two branches can be connected, one for image classification and one for image segmentation.
- the branch used for image classification can be implemented by a fully connected layer, and each first feature map is input to the fully connected layer to obtain the first category of the image block corresponding to the first feature in the fundus color photo image; used for image segmentation
- the branch can be implemented through a fully convolutional network.
- each image block can be segmented through a convolution kernel with a convolution kernel of 1*1, and each pixel in each image block belongs to the first image block corresponding to the image block. probability of a class.
- the lesion segmentation device determines, according to the first category of each image block and the first mask map, a category corresponding to each pixel in the color fundus image.
- the category of each pixel point in the color fundus image is obtained, and the category of each pixel point includes the background or the lesion.
- recovery processing is performed on the first mask map of each image block corresponding to the first feature map A, to obtain a second mask map corresponding to each image block, wherein the corresponding The dimension of the second mask image is the same as the dimension of the color fundus image. Therefore, the second mask map corresponding to each image block is used to represent the probability that each pixel in the color fundus image belongs to the first category corresponding to each image block; according to the second mask map corresponding to each image block and the first category, to determine the category corresponding to each pixel in the color fundus image.
- the first mask map corresponding to each image block may be up-sampled by a bilinear interpolation method, so as to obtain a second mask map corresponding to each image block, wherein, by bilinear interpolation
- the method is the prior art and will not be described again.
- the first category corresponding to each image block is clustered to obtain at least one first category, for example, the first feature map A corresponds to 4 image blocks, wherein the first category of the first image block is is category A, the first category of the second image block is category B, the first category of the third image block is category A, and the first category of the fourth image block is category C. Therefore, by clustering the four first categories corresponding to the four image blocks, three first categories can be obtained, and the first image blocks corresponding to category A include the first image block and the third first image block.
- the second mask maps of all image blocks corresponding to each first category are superimposed and normalized to obtain a target mask map corresponding to each first category, and the target mask map is used to represent the fundus The probability that each pixel in the color photo image belongs to the first category.
- the first category of the plurality of image blocks corresponding to each of the first feature maps in the plurality of first feature maps is clustered to obtain the at least one category;
- the categories are superimposed and normalized on all the second mask maps corresponding to the plurality of first feature maps to obtain a target mask map corresponding to each first category.
- the target mask map corresponding to each first category the probability that each pixel in the color fundus image belongs to each first category can be determined; finally, according to the color fundus image, each pixel belongs to each The probability of the first category is obtained, and the category corresponding to each pixel in the color fundus image is obtained.
- the first category with the largest probability value is taken as the category of each pixel point.
- the target mask of the first category A indicates that the probability that the first pixel in the color fundus image belongs to the first category A is 0.5
- the target mask of the first category B indicates that the color fundus image has a probability of 0.5.
- the probability that the first pixel belongs to the first category B is 0.4, and the target mask map of the first category C indicates that the probability that the first pixel in the color fundus image belongs to the first category C is 0.2, which can be determined
- the category of the first pixel in the color fundus image is the first category A.
- the lesion segmentation device performs lesion segmentation on the color fundus image according to the category of each pixel in the color fundus image.
- each pixel point in the color fundus image perform lesion segmentation on the fundus image, that is, segment the region of the pixel points belonging to the same lesion to obtain the lesion region.
- the color fundus image is divided into grids to perform lesion segmentation.
- the present application uses the entire fundus color photo for segmentation, and can use more
- the lesion area can be segmented by using more information on the edge contour of the lesion, so that the edge contour of the segmented lesion is more refined, so that the segmentation result of the lesion in the color fundus image is more accurate, and the diagnosis accuracy of the doctor is improved.
- the lesion segmentation method of the present application can also be applied to the field of smart medicine. For example, by segmenting the color fundus image through the lesion segmentation method, more detailed lesion contour information can be segmented, so as to provide doctors with more refined quantitative indicators, improve the diagnosis accuracy of doctors, and promote the development of medical technology.
- FIG. 4 is a schematic flowchart of a neural network training method provided by the present application. The method includes the following steps:
- the manner of obtaining multiple second feature maps is similar to the aforementioned manner of performing feature extraction on a color fundus image to obtain multiple first feature maps, and will not be described again.
- each second feature map divides the image sample into multiple image sample blocks, that is, multiple grids
- the method of dividing the multiple grids is the same as the above-mentioned first feature map to divide the fundus color photograph into multiple grids.
- the method is similar and will not be described again.
- acquiring the third mask map and the second category corresponding to each image sample block is similar to the above-mentioned manner of acquiring the first mask map and the first category corresponding to each image block, and will not be described again.
- the third mask map corresponding to each image sample block is used to represent the predicted probability that each pixel in each image sample block belongs to the second category corresponding to the image sample block. Then, according to the third mask map and the second category corresponding to each image sample block, determine the proportion of the number of pixels belonging to the lesion in each image sample, that is, in the case that the second category is a lesion, according to The third mask map determines the pixels belonging to the second category in each image block, that is, the pixels belonging to the lesions, for example, the probability is greater than the threshold; then, the number of pixels belonging to the lesions is determined relative to the image The ratio of the number of all pixels in the sample block to get the ratio. It should be understood that in the case that the second category is not a lesion, that is, the background, the pixel points belonging to the lesion in the sample image block are 0.
- the first threshold may be 0.2 or other values
- the second threshold may be 1 or other values.
- the image sample block can be used as a separate The training sample is obtained, that is, the labeling result corresponding to the image sample block is obtained, the labeling result is pre-labeled, and the labeling result is the true probability that each pixel in the image sample block belongs to the lesion; then, according to the image sample block and the labeling result of the sample block of the image block to obtain the first loss;
- the third mask map of the image sample block determine the predicted probability that each pixel in the image sample block belongs to the lesion; according to the predicted probability that each pixel in the image sample block belongs to the lesion and each pixel in the image sample block
- the real probability that a pixel belongs to a lesion is calculated by cross-entropy loss to obtain the first loss. Therefore, this first loss can be expressed by formula (1):
- y is the y-th pixel in the image sample
- P(y) is the true probability that the y-th pixel belongs to the lesion
- M is the total number of pixels in the image sample block.
- the third mask corresponding to each image sample block is used.
- the image is restored to obtain a fourth mask map corresponding to each image sample block, and the second loss is determined according to the labeling result of the image sample and the fourth mask map corresponding to each image sample block, wherein the labeling result is It is pre-marked and is used to indicate the true probability that each pixel in the image sample belongs to the lesion.
- the restoration processing for the third mask image is similar to the above-mentioned restoration processing for the first mask image, and will not be described again.
- the predicted probability that each pixel in the image sample belongs to the lesion is obtained according to the fourth mask map of the image sample block; the cross is performed according to the predicted probability and the real probability that each pixel in the image sample block belongs to the lesion.
- the entropy loss is calculated to obtain the second loss. Therefore, this second loss can be expressed by formula (2):
- x is the xth pixel in the image sample
- P(x) is the true probability that the xth pixel belongs to the lesion
- N is the total number of pixels in the image sample.
- the target loss is determined, that is, the first loss and the second loss are weighted to obtain the target loss, and according to the target loss and the gradient descent method, the network of the neural network is The parameters are adjusted until the neural network converges, and the training of the neural network is completed.
- FIG. 5 is a schematic structural diagram of a lesion segmentation device provided by an embodiment of the present application.
- the lesion segmentation device 500 includes a processor, a memory, a communication interface, and one or more programs, wherein the one or more programs are stored in the memory and configured to be executed by the processor, and the above-mentioned one or more programs are configured to be executed by the above-mentioned processor.
- the program includes instructions for performing the following steps:
- the first feature map A a first category and a first mask map corresponding to each image block in the multiple image blocks corresponding to the first feature map A in the color fundus image are determined, and the first feature map Figure A is any one of the first feature maps in the plurality of first feature maps;
- each image block and the first mask map determine the category corresponding to each pixel in the color fundus image
- Segmentation of lesions is performed on the color fundus image according to the category of each pixel in the color fundus image.
- the first category and the first category corresponding to each image block in the multiple image blocks corresponding to the first feature map A in the color fundus image are determined.
- the above program is specifically used to execute the instructions of the following steps:
- a first category of each image block is determined according to the feature vector of each image block.
- the above program in terms of determining the category corresponding to each pixel in the color fundus image according to the first category of each image block and the first mask map, the above program is specifically configured to perform the following steps command:
- the category corresponding to each pixel in the color fundus image image is determined.
- the above program in terms of determining the category corresponding to each pixel in the color fundus image according to the second mask map and the first category corresponding to each image block, the above program is specifically used to execute Instructions for the following steps:
- the category corresponding to each pixel in the color fundus image is determined according to the probability that each pixel in the color fundus image belongs to each of the first categories.
- the above program in terms of restoring the first mask map corresponding to each image block to obtain the second mask map corresponding to each image block, the above program is specifically used to execute the following Instructions for steps:
- up-sampling processing is performed on the first mask map corresponding to each image block to obtain a second mask map corresponding to each image block.
- the above program is also used to execute the instructions of the following steps:
- the third mask map and the second category corresponding to each image sample block in the plurality of image sample blocks corresponding to the second feature map B are determined, and the second feature map B is the any one of the second feature maps in the plurality of second feature maps;
- the network parameters of the neural network are adjusted according to the third mask map and the second category corresponding to each image sample block.
- the above program is specifically used to execute the instructions of the following steps:
- the third mask map and the second category corresponding to each image sample block determine the proportion of the number of pixels belonging to the lesion in each image sample block
- the labeling result corresponding to each image sample block is acquired, and according to the first threshold of each second feature map Three masks and the labeling result of each image sample block to obtain the first loss;
- the target loss is obtained
- the network parameters of the neural network are adjusted according to the target loss.
- FIG. 6 is a block diagram of functional units of a lesion segmentation device provided by an embodiment of the present application.
- the lesion segmentation device 600 includes: an acquisition unit 601 and a processing unit 602, wherein:
- an acquisition unit 601 configured to acquire a color fundus image
- a processing unit 602 configured to perform feature extraction on the color fundus image to obtain a plurality of first feature maps, wherein the dimensions between any two first feature maps in the plurality of first feature maps are different;
- the processing unit 602 is further configured to determine, according to the first feature map A, a first category and a first mask corresponding to each image block in the multiple image blocks corresponding to the first feature map A in the color fundus image Figure, the first feature map A is any one of the first feature maps of the plurality of first feature maps;
- the processing unit 602 is further configured to determine the category corresponding to each pixel in the color fundus image according to the first category of each image block and the first mask map;
- the processing unit 602 is further configured to perform lesion segmentation on the color fundus image according to the category of each pixel in the color fundus image.
- the processing unit 602 is specifically used for:
- a first category of each image block is determined according to the feature vector of each image block.
- the processing unit 602 is specifically configured to:
- the class corresponding to each pixel in the color fundus image image is determined.
- the processing unit 602 in determining the category corresponding to each pixel in the color fundus image according to the second mask map and the first category corresponding to each image block, the processing unit 602 specifically uses At:
- the category corresponding to each pixel in the color fundus image is determined according to the probability that each pixel in the color fundus image belongs to each of the first categories.
- the processing unit 602 is specifically configured to: :
- up-sampling processing is performed on the first mask map corresponding to each image block to obtain a second mask map corresponding to each image block.
- the acquiring unit 601 is further configured to acquire image samples
- the processing unit 602 is further configured to input the image sample into the neural network to obtain multiple second feature maps, and the dimensions between any two second feature maps in the multiple second feature maps are different;
- the third mask map and the second category corresponding to each image sample block in the plurality of image sample blocks corresponding to the second feature map B are determined, and the second feature map B is the any one of the second feature maps in the plurality of second feature maps;
- the network parameters of the neural network are adjusted according to the third mask map and the second category corresponding to each image sample block.
- the processing unit 602 is specifically configured to:
- the third mask map and the second category corresponding to each image sample block determine the proportion of the number of pixels belonging to the lesion in each image sample block
- the labeling result corresponding to each image sample block is acquired, and according to the first threshold of each second feature map Three masks and the labeling result of each image sample block to obtain the first loss;
- the target loss is obtained
- the network parameters of the neural network are adjusted according to the target loss.
- Embodiments of the present application further provide a computer storage medium, such as a computer-readable storage medium, where a computer program is stored in the computer-readable storage medium, and the computer program is executed by a processor to implement any one of the foregoing method embodiments. Some or all of the steps of a lesion segmentation method.
- the storage medium involved in this application such as a computer-readable storage medium, may be non-volatile or volatile.
- the embodiments of the present application further provide a computer program product, the computer program product includes a non-transitory computer-readable storage medium storing a computer program, and the computer program is operable to cause a computer to execute the methods described in the foregoing method embodiments. Some or all of the steps of any lesion segmentation method.
- the lesion segmentation device in this application may include smart phones (such as Android mobile phones, iOS mobile phones, Windows Phone mobile phones, etc.), tablet computers, handheld computers, notebook computers, and MID (Mobile Internet Devices, referred to as: MID) or wearable devices, etc.
- smart phones such as Android mobile phones, iOS mobile phones, Windows Phone mobile phones, etc.
- tablet computers such as Samsung mobile phones, iOS mobile phones, Windows Phone mobile phones, etc.
- MID Mobile Internet Devices, referred to as: MID
- wearable devices etc.
- the above-mentioned lesion segmentation device is only an example, not exhaustive, including but not limited to the above-mentioned lesion segmentation device.
- the above-mentioned lesion segmentation apparatus may further include: an intelligent vehicle-mounted terminal, a computer device, and the like.
- the disclosed apparatus may be implemented in other manners.
- the apparatus embodiments described above are only illustrative, for example, the division of the units is only a logical function division, and there may be other division methods in actual implementation, for example, multiple units or components may be combined or Integration into another system, or some features can be ignored, or not implemented.
- the shown or discussed mutual coupling or direct coupling or communication connection may be through some interfaces, indirect coupling or communication connection of devices or units, and may be in electrical or other forms.
- the units described as separate components may or may not be physically separated, and components displayed as units may or may not be physical units, that is, may be located in one place, or may be distributed to multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution in this embodiment.
- each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically alone, or two or more units may be integrated into one unit.
- the above-mentioned integrated units can be implemented in the form of hardware, and can also be implemented in the form of software program modules.
- the integrated unit if implemented in the form of a software program module and sold or used as a stand-alone product, may be stored in a computer readable memory.
- the technical solution of the present application can be embodied in the form of a software product in essence, or the part that contributes to the prior art, or all or part of the technical solution, and the computer software product is stored in a memory.
- a computer device which may be a personal computer, a server, or a network device, etc.
- the aforementioned memory includes: U disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), mobile hard disk, magnetic disk or optical disk and other media that can store program codes.
Landscapes
- Engineering & Computer Science (AREA)
- Physics & Mathematics (AREA)
- Theoretical Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Data Mining & Analysis (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Life Sciences & Earth Sciences (AREA)
- Artificial Intelligence (AREA)
- General Health & Medical Sciences (AREA)
- Health & Medical Sciences (AREA)
- Evolutionary Computation (AREA)
- General Engineering & Computer Science (AREA)
- Mathematical Physics (AREA)
- Probability & Statistics with Applications (AREA)
- Software Systems (AREA)
- Biomedical Technology (AREA)
- Biophysics (AREA)
- Computational Linguistics (AREA)
- Computing Systems (AREA)
- Molecular Biology (AREA)
- Bioinformatics & Cheminformatics (AREA)
- Evolutionary Biology (AREA)
- Bioinformatics & Computational Biology (AREA)
- Medical Informatics (AREA)
- Nuclear Medicine, Radiotherapy & Molecular Imaging (AREA)
- Radiology & Medical Imaging (AREA)
- Quality & Reliability (AREA)
- Eye Examination Apparatus (AREA)
- Image Analysis (AREA)
- Image Processing (AREA)
Abstract
本申请涉及医疗科技技术领域,具体公开了一种病灶分割方法、装置及存储介质。该方法包括:获取眼底彩照图像;对所述眼底彩照图像进行特征提取,得到多个第一特征图,其中,所述多个第一特征图中任意两个第一特征图之间的维度不同;根据第一特征图A,确定所述眼底彩照图像中与所述第一特征图A对应的多个图像块中每个图像块对应的第一类别以及第一掩膜图,所述第一特征图A为所述多个第一特征图中的任意一个第一特征图;根据所述每个图像块的第一类别以及第一掩膜图,确定所述眼底彩照图像中各个像素点对应的类别;根据所述眼底彩照图像中各个像素点的类别,对所述眼底彩照图像进行病灶分割。本申请有利于提高病灶分割精度。
Description
本申请要求于2020年10月30日提交中国专利局、申请号为202011187336.5,发明名称为“病灶分割方法、装置及存储介质”的中国专利申请的优先权,其全部内容通过引用结合在本申请中。
本申请涉及图像识别技术领域,具体涉及一种病灶分割方法、装置及存储介质。
眼底彩照是检查眼底的一种方式,能利用其看清眼底的组织结构,分析眼底结构的正常与异常,判断是否眼底的视盘、血管、视网膜或脉络膜等出现问题。由于图像分割技术为医学影像等应用提供了丰富的视觉感知信息,可应用于眼底彩照上与视网膜疾病相关的病灶分割。发明人研究过程中发现,眼底图像的病灶分割和自然图像的分割有较大的差异,受拍摄光线和成像质量影响,病灶边缘对比度没有自然图像清晰,导致眼底病灶分割一直是一个困难复杂的挑战。
发明人意识到,目前一般用于病灶分割的方法通常先对病灶进行检测,得到病灶区域的检测框;然后,在检测框内对病灶进行单独分割。由于病灶框内只能用很小的特征图大小进行分割,对于病灶边缘的分割质量不佳,导致对病灶分割的精度低,影响医生的诊断。
发明内容
本申请实施例提供了一种病灶分割方法、装置及存储介质。通过多个维度对眼底彩照进行分割,可提高对病灶轮廓信息的分割精度。
第一方面,本申请实施例提供一种病灶分割方法,包括:
获取眼底彩照图像;
对所述眼底彩照图像进行特征提取,得到多个第一特征图,其中,所述多个第一特征图中任意两个第一特征图之间的维度不同;
根据第一特征图A,确定所述眼底彩照图像中与所述第一特征图A对应的多个图像块中每个图像块对应的第一类别以及第一掩膜图,所述第一特征图A为所述多个第一特征图中的任意一个第一特征图;
根据所述每个图像块的第一类别以及第一掩膜图,确定所述眼底彩照图像中各个像素点对应的类别;
根据所述眼底彩照图像中各个像素点的类别,对所述眼底彩照图像进行病灶分割。
第二方面,本申请实施例提供一病灶分割装置,包括:
获取单元,用于获取眼底彩照图像;
处理单元,用于对所述眼底彩照图像进行特征提取,得到多个第一特征图,其中,所述多个第一特征图中任意两个第一特征图之间的维度不同;
所述处理单元,还用于根据第一特征图A,确定所述眼底彩照图像中与所述第一特征图A对应的多个图像块中每个图像块对应的第一类别以及第一掩膜图,所述第一特征图A为所述多个第一特征图中的任意一个第一特征图;
所述处理单元,还用于根据所述每个图像块的第一类别以及第一掩膜图,确定所述眼底彩照图像中各个像素点对应的类别;
所述处理单元,还用于根据所述眼底彩照图像中各个像素点的类别,对所述眼底彩照图像进行病灶分割。
第三方面,本申请实施例提供一种电子设备,包括处理器、存储器、通信接口以及一个或多个程序,其中,所述一个或多个程序被存储在所述存储器中,并且被配置由所述处理器执行以实现以下方法:
获取眼底彩照图像;
对所述眼底彩照图像进行特征提取,得到多个第一特征图,其中,所述多个第一特征图中任意两个第一特征图之间的维度不同;
根据第一特征图A,确定所述眼底彩照图像中与所述第一特征图A对应的多个图像块中每个图像块对应的第一类别以及第一掩膜图,所述第一特征图A为所述多个第一特征图中的任意一个第一特征图;
根据所述每个图像块的第一类别以及第一掩膜图,确定所述眼底彩照图像中各个像素点对应的类别;
根据所述眼底彩照图像中各个像素点的类别,对所述眼底彩照图像进行病灶分割。
第四方面,本申请实施例提供一种计算机可读存储介质,所述计算机可读存储介质存储有计算机程序,所述计算机程序使得计算机执行以下方法:
获取眼底彩照图像;
对所述眼底彩照图像进行特征提取,得到多个第一特征图,其中,所述多个第一特征图中任意两个第一特征图之间的维度不同;
根据第一特征图A,确定所述眼底彩照图像中与所述第一特征图A对应的多个图像块中每个图像块对应的第一类别以及第一掩膜图,所述第一特征图A为所述多个第一特征图中的任意一个第一特征图;
根据所述每个图像块的第一类别以及第一掩膜图,确定所述眼底彩照图像中各个像素点对应的类别;
根据所述眼底彩照图像中各个像素点的类别,对所述眼底彩照图像进行病灶分割。
在本申请实施例中,对眼底彩照图像通过划分格子的方式进行病灶分割,相比对病灶检测框进行单独分割,本申请利用了整张眼底彩照图进行分割,可以利用更多的病灶区域进行分割,从而可以利用到更多的病灶边缘轮廓信息,使分割出的病灶边缘轮廓更加精细,进而使眼底彩照图像中的病灶分割结果更加准确,提高医生诊断精度。
为了更清楚地说明本申请实施例中的技术方案,下面将对实施例描述中所需要使用的附图作简单地介绍,显而易见地,下面描述中的附图是本申请的一些实施例,对于本领域普通技术人员来讲,在不付出创造性劳动的前提下,还可以根据这些附图获得其他的附图。
图1为本申请实施例提供的一种病灶分割方法的流程示意图;
图2为本申请实施例提供的一种神经网络的结构示意图;
图3为本申请实施例提供的一种对眼底彩照图进行分块的示意图;
图4为本申请实施例提供的一种神经网络训练方法的流程示意图;
图5为本申请实施例提供的一种病灶分割装置的结构示意图;
图6为本申请实施例提供的一种病灶分割装置的功能单元组成框图。
下面将结合本申请实施例中的附图,对本申请实施例中的技术方案进行清楚、完整地描述,显然,所描述的实施例是本申请一部分实施例,而不是全部的实施例。基于本申请中的实施例,本领域普通技术人员在没有作出创造性劳动前提下所获得的所有其他实施例,都属于本申请保护的范围。
本申请的说明书和权利要求书及所述附图中的术语“第一”、“第二”、“第三”和“第四”等是用于区别不同对象,而不是用于描述特定顺序。此外,术语“包括”和“具有”以及它们任何变形,意图在于覆盖不排他的包含。例如包含了一系列步骤或单元的过程、方法、系统、产品或设备没有限定于已列出的步骤或单元,而是可选地还包括没有列出的步骤或单元,或可选地还包括对于这些过程、方法、产品或设备固有的其它步骤或单元。
在本文中提及“实施例”意味着,结合实施例描述的特定特征、结果或特性可以包含 在本申请的至少一个实施例中。在说明书中的各个位置出现该短语并不一定均是指相同的实施例,也不是与其它实施例互斥的独立的或备选的实施例。本领域技术人员显式地和隐式地理解的是,本文所描述的实施例可以与其它实施例相结合。
本申请的技术方案可涉及人工智能和/或大数据技术领域,如可具体涉及神经网络技术,以实现基于图像的病灶分割。可选的,本申请涉及的数据如各种图像和/或病灶分割结果等可存储于数据库中,或者可以存储于区块链中,本申请不做限定。
参阅图1,图1为本申请实施例提供的一种病灶分割方法的流程示意图。该方法应用于病灶分割装置。该方法包括以下步骤:
101:病灶分割装置获取眼底彩照图像。
其中,该眼底彩照图像是通过眼底彩超成像生成的,不再叙述。
102:病灶分割装置对所述眼底彩照图像进行特征提取,得到多个第一特征图,其中,所述多个第一特征图中任意两个第一特征图之间的维度不同。
示例性的,在不同的深度对该眼底彩照图像特征提取,得多个第一特征图。由于深度不同,该多个第一特征图中任意两个第一特征图之间的维度不同。
在本申请的一个实施方式中,对该眼底彩照图像进行特征提取可通过神经网络实现,该神经网络是预先训练好的,后续详细叙述对该神经网络的训练过程,在此不做过多描述。由于每个第一特征图是通过神经网络提取得到的,所以,该第一特征图是对该神经网络的多个通道得到的多个第一子特征图拼接得到的,其中,每个通道对应一个第一特征图。
举例来说,如图2所示,通过特征图金字塔网络(feature pyramid networks,FPN)对眼底彩照图像进行特征提取,在多个不同深度的网络层得到多个第一特征图,图2中示出的每个维度对应的第一特征图仅是一个通道上的第一子特征图。最底层的第一特征图是通过特征提取得到的,其他层的第一特征图是将上一层的第一特征图和本层得到的特征图进行叠加得到的。
示例性的,由于每个第一特征图是对该眼底彩照图像进行特征提取得到,所以,每个第一特征图的一个像素点包含有该眼底彩照图像中的一个区域的信息,因此,每个第一特征图可以看做将眼底彩照图像划分成多个格子,即多个图像块,而且,不同维度的第一特征图,将该眼底彩照图像划分出的图像块的数量和维度不同。
举例来说,如图3所示,眼底彩照图像的维度为320*320,若某个第一特征图的维度为64*64*10,其中,10表示通道数,则该第一特征图将眼底彩照图像图划分成了5*5个图像块,且每个图像块对应的维度为5*5。
103:病灶分割装置根据第一特征图A,确定所述眼底彩照图像中与所述第一特征图A对应的多个图像块中每个图像块对应的第一类别以及第一掩膜图,所述第一特征图A为所述多个第一特征图中的任意一个第一特征图。
其中,每个图像块对应的第一掩膜图用于表示该图像块中各个像素点属于该第一类别的概率。每个图像块对应的第一类别可以为背景或者病灶,其中,该病灶包括以下至少一种:黄斑、青光眼、渗水,等等。
应理解,本申请以第一特征图A为例说明对眼底彩照图像中的图像块进行图像分割以及图像分类的方式,该第一特征图A为该多个第一特征图中的任意一个第一特征图,对该多个第一特征图中的其他第一特征图处理方式与对该第一特征图A的处理方式类似,不再叙述。
示例性的,根据该第一特征图A进行图像分类。即根据该第一特征图A,得到与该第一特征图A对应的多个图像块中每个图像块对应的特征向量;根据每个图像块对应的特征向量,确定每个图像块对应的第一类别。举例来说,在使用该第一特征图A对第一个图像块进行分类的情况下,由于该第一特征图是由多个通道上的特征图拼接得到的,则每个通 道上的特征图中的第一个像素点表征了该第一个图像块的特征信息,因此,可将每个通道上的特征图中的第一个像素点的灰度值组成一个特征向量,可将该特征向量作为该第一个图像块的特征向量,并对该特征向量输入到全连接层以及softmax层,得到该第一个图像块的第一类别。
举例来说,该第一特征图A是由三个通道上的特征图拼接得到。假如第一个通道上的特征图中的第一个像素点的灰度值为23,第二个通道上的特征图中的第一个像素点的灰度值为36,第三个通道上的特征图中的第一个像素点的灰度值为54,则可得到第一个图像块的特征向量为[23,36,54]。
进一步地,还可根据第一特征图A,对第一特征图A对应的多个图像块进行图像分割,得到每个图像块对应的第一掩膜图,该第一掩膜图用于表示该图像块中各个像素点属于该图像块对应的第一类别的概率,对第一特征图A进行图像分割与通过全卷积网络对特征图进行图像分割类似,不再叙述。比如,可以将每个通道上的特征图上与每个图像块对应的特征信息组成与该图像块的特征图,根据该特征图进行图像分割,得到该图像块对应的第一掩膜图。
在本申请的一个实施方式中,该神经网络可以是以FPN为基础的神经网络。如图2所示,可以在FPN网络输出的第一特征图之后,连接两个分支,一个分支用于图像分类,一个分支用于图像分割。用于图像分类的分支可以通过全连接层实现,将每个第一特征图输入到全连接层,得到该眼底彩照图像中与该第一特征对应的图像块的第一类别;用于图像分割的分支可以通过全卷积网络实现,比如,可以通过卷积核为1*1的卷积核对每个图像块进行图像分割,得到每个图像块中每个像素点属于该图像块对应的第一类别的概率。
104:病灶分割装置根据所述每个图像块的第一类别以及第一掩膜图,确定所述眼底彩照图像中各个像素点对应的类别。
示例性的,根据每个图像块的第一类别以及第一掩膜图,得到该眼底彩照图像中各个像素点的类别,各个像素点的类别包括背景或者病灶。
示例性的,对与该第一特征图A对应的每个图像块的第一掩膜图进行恢复处理,得到与每个图像块对应的第二掩膜图,其中,每个图像块对应的第二掩膜图的维度与该眼底彩照图像的维度相同。因此,每个图像块对应的第二掩膜图用于表示该眼底彩照图像图中各个像素点属于每个图像块对应的第一类别的概率;根据每个图像块对应的第二掩膜图以及第一类别,确定该眼底彩照图像中个像素点对应的类别。
示例性的,可通过双线性插值法,对每个图像块对应的第一掩膜图进行上采样处理,得到与每个图像块对应的第二掩膜图,其中,通过双线性插值法为现有技术,不再叙述。
示例性的,对每个图像块对应的第一类别进行聚类,得到至少一个第一类别,比如,该第一特征图A对应4个图像块,其中,第一个图像块的第一类别为类别A,第二个图像块的第一类别为类别B,第三个图像块的第一类别为类别A,第四个图像块的第一类别为类别C。因此,将该四个图像块对应的四个第一类别进行聚类可得到三个第一类别,且类别A对应的第一图像块有第一个图像块和第三个第一图像块。然后,将每个第一类别对应的所有的图像块的第二掩膜图进行叠加以及归一化,得到每个第一类别对应的目标掩膜图,该目标掩膜图用于表示该眼底彩照图像中各个像素点属于该第一类别的概率。
应理解,在实际应用中是将该多个第一特征图中每个第一特征图对应的多个图像块的第一类别进行聚类,得到该至少一个类别;然后,将每个第一类别在该多个第一特征图对应的所有第二掩码图进行叠加以及归一化,得到每个第一类别对应的目标掩膜图。
因此,根据每个第一类别对应的目标掩膜图,可以确定出该眼底彩照图像图中各个像素点属于每个第一类别的概率;最后,根据该眼底彩照图像图中各个像素点属于每个第一类别的概率,得到该眼底彩照图像图中各个像素点对应的类别。即将概率值最大的第一类 别作为各个像素点的类别。比如,第一类别A的目标掩码图表示该眼底彩照图像中的第一个像素点属于该第一类别A的概率为0.5,第一类别B的目标掩码图表示该眼底彩照图像中的第一个像素点属于该第一类别B的概率为0.4,第一类别C的目标掩码图表示该眼底彩照图像中的第一个像素点属于该第一类别C的概率为0.2,可确定该眼底彩照图像中的第一个像素点的类别该第一类别A。
105:病灶分割装置根据所述眼底彩照图像中各个像素点的类别,对所述眼底彩照图像进行病灶分割。
示例性的,根据该眼底彩照图像中各个像素点的类别,对该眼底图像进行病灶分割,即将属于同一病灶的像素点的区域分割出来,可得到该病灶区域。
可以看出,在本申请实施例中,对眼底彩照图像通过划分格子的方式进行病灶分割,相比对病灶检测框进行单独分割,本申请利用了整张眼底彩照图进行分割,可以利用更多的病灶区域进行分割,从而可以利用到更多的病灶边缘轮廓信息,使分割出的病灶边缘轮廓更加精细,进而使眼底彩照图像中的病灶分割结果更加准确,提高医生诊断精度。
在本申请的一个实施方式中,本申请的病灶分割方法还可以应用到智慧医疗领域。比如,通过该病灶分割方法对眼底彩照图像进行分割,可以分割出更细节的病灶轮廓信息,从而为医生提供更加精细的量化指标,提高医生的诊断精度,推动医疗科技的发展。
参阅图4,图4为本申请提供的一种神经网络训练方法的流程示意图。该方法包括以下步骤:
401:获取图像样本。
402:将所述图像样本输入到所述神经网络,得到多个第二特征图,所述多个第二特征图中的任意两个第二特征图之间的维度不同。
示例性的,获取多个第二特征图的方式,与上述对眼底彩照图进行特征提取,得到多个第一特征图的方式类似,不再叙述。
403:根据第二特征图B,确定与所述第二特征图B对应的多个图像样本块中每个图像样本块对应的第三掩膜图以及第二类别,其中,所述第二特征图B为所述多个第二特征图中的任意一个第二特征图。
示例性的,每个第二特征图会将该图像样本划分为多个图像样本块,即多个格子,且划分多个格子的方式与上述第一特征图将眼底彩照图划分为多个格子的方式类似,不再叙述。此外,获取每个图像样本块对应的第三掩膜图与第二类别与上述获取每个图像块对应的第一掩膜图和第一类别的方式类似,也不再叙述。
404:根据所述每个图像样本块对应的第三掩膜图以及第二类别,调整所述神经网络的网络参数。
示例性的,每个图像样本块对应的第三掩膜图用于表示每个图像样本块中各个像素点属于该图像样本块对应的第二类别的预测概率。然后,根据每个图像样本块对应的第三掩膜图以及第二类别,确定每个图像样本中属于病灶的像素点的数量的占比,即在该第二类别为病灶的情况下,根据该第三掩膜图确定每个图像块中属于第二类别的像素点,即为属于病灶的像素点,比如,将概率大于阈值的;然后,确定属于病灶的像素点的数量相对于该图像样本块中所有像素点的数量的比值,得到该占比。应理解,在该第二类别不是病灶的,即是背景的情况下,该样本图像块中属于病灶的像素点的为0。
其中,该第一阈值可以为0.2或者其他值,第二阈值可以为1或者其他值。
进一步地,在该占比大于第一阈值,且小于第二阈值的情况下,确定该图像样本块处于病灶的边缘,为了提高神经网络对边缘分割的精度,可将该图像样本块作为一个单独的训练样本,即获取与该图像样本块对应的标注结果,该标注结果是预先标注好的,该标注结果为该图像样本块中各个像素点属于病灶的真实概率;然后,根据该图像样本块的第三 掩膜图以及该图像块样本块的标注结果,得到第一损失;
示例性的,根据该图像样本块的第三掩码图,确定该图像样本块中每个像素点属于病灶的预测概率;根据该图像样本块中每个像素点属于病灶的预测概率以及每个像素点属于病灶的真实概率,进行交叉熵损失计算,得到该第一损失。因此,该第一损失可通过公式(1)表示:
进一步地,在每个图像样本块对应的占比大于或等于该第二阈值的情况下,也就是说该图像样本块完全位于病灶区域内,则将每个图像样本块对应的第三掩膜图进行恢复处理,得到每个图像样本块对应的第四掩膜图,根据图像样本的标注结果以及每个图像样本块对应的第四掩膜图,确定第二损失,其中,该标注结果为预先标注好的,用于表示该图像样本中每个像素点属于病灶的真实概率,对第三掩膜图进行恢复处理与上述对第一掩膜图进行恢复处理的方式类似,不再叙述。
示例性的,根据该图像样本块的第四掩膜图得到该图像样本中每个像素点属于病灶的预测概率;根据该图像样本块中每个像素点属于病灶的预测概率以及真实概率进行交叉熵损失计算,得到该第二损失。因此,该第二损失可以通过公式(2)表示:
最后,根据该第一损失和第二损失,确定目标损失,即将该第一损失和第二损失进行加权处理,得到该目标损失,并根据该目标损失以及梯度下降法,对该神经网络的网络参数进行调整,直至该神经网络收敛,完成对该神经网络的训练。
参阅图5,图5为本申请实施例提供的一种病灶分割装置的结构示意图。如图5所示,病灶分割装置500包括处理器、存储器、通信接口以及一个或多个程序,其中,上述一个或多个程序被存储在上述存储器中,并且被配置由上述处理器执行,上述程序包括用于执行以下步骤的指令:
获取眼底彩照图像;
对所述眼底彩照图像进行特征提取,得到多个第一特征图,其中,所述多个第一特征图中任意两个第一特征图之间的维度不同;
根据第一特征图A,确定所述眼底彩照图像中与所述第一特征图A对应的多个图像块中每个图像块对应的第一类别以及第一掩膜图,所述第一特征图A为所述多个第一特征图中的任意一个第一特征图;
根据所述每个图像块的第一类别以及第一掩膜图,确定所述眼底彩照图像中各个像素 点对应的类别;
根据所述眼底彩照图像中各个像素点的类别,对所述眼底彩照图像进行病灶分割。
在一些可能的实施方式中,在根据第一特征图A,确定所述眼底彩照图像中与所述第一特征图A对应的多个图像块中每个图像块对应的第一类别以及第一掩膜图方面,上述程序具体用于执行以下步骤的指令:
根据所述第一特征图A进行图像分割,确定所述眼底彩照图像图中与所述第一特征图A对应的多个图像块中每个图像块的第一掩膜图;
根据所述第一特征图A,确定所述每个图像块的特征向量;
根据所述每个图像块的特征向量,确定所述每个图像块的第一类别。
在一些可能的实施方式中,在根据所述每个图像块的第一类别以及第一掩膜图,确定所述眼底彩照图像中各个像素点对应的类别方面,上述程序具体用于执行以下步骤的指令:
对所述每个图像块对应的第一掩膜图进行恢复处理,得到所述每个图像块对应的第二掩膜图,所述第二掩膜图用于表示所述眼底彩照图像图中各个像素点属于所述每个图像块对应的第一类别的概率;
根据所述每个图像块对应的第二掩膜图以及第一类别,确定所述眼底彩照图像图中各个像素点对应的类别。
在一些可能的实施方式中,在根据所述每个图像块对应的第二掩膜图以及第一类别,确定所述眼底彩照图像图中各个像素点对应的类别方面,上述程序具体用于执行以下步骤的指令:
对所述每个图像块对应的第一类别进行聚类,得到至少一个第一类别;
将所述至少一个第一类别中的每个第一类别对应的所有图像块的第二掩膜图进行叠加以及归一化,得到所述每个第一类别对应的目标掩膜图;
根据所述每个第一类别对应的目标掩膜图,确定所述眼底彩照图像图中各个像素点属于所述每个第一类别的概率;
根据所述眼底彩照图像图中各个像素点属于所述每个第一类别的概率,确定所述眼底彩照图像图中各个像素点对应的类别。
在一些可能的实施方式中,在对所述每个图像块对应的第一掩膜图进行恢复处理,得到所述每个图像块对应的第二掩膜图方面,上述程序具体用于执行以下步骤的指令:
通过双线性插值法,对所述每个图像块对应的第一掩膜图进行上采样处理,得到与所述每个图像块对应的第二掩膜图。
在一些可能的实施方式中,上述程序还用于执行以下步骤的指令::
获取图像样本;
将所述图像样本输入到所述神经网络,得到多个第二特征图,所述多个第二特征图中的任意两个第二特征图之间的维度不同;
根据第二特征图B,确定与所述第二特征图B对应的多个图像样本块中每个图像样本块对应的第三掩膜图以及第二类别,所述第二特征图B为所述多个第二特征图中的任意一个第二特征图;
根据所述每个图像样本块对应的第三掩膜图以及第二类别,调整所述神经网络的网络参数。
在一些可能的实施方式中,在根据所述每个图像样本块对应的第三掩膜图以及第二类别,调整所述神经网络的网络参数方面,上述程序具体用于执行以下步骤的指令:
根据所述每个图像样本块对应的第三掩膜图以及第二类别,确定所述每个图像样本块中属于病灶的像素点的数量的占比;
在所述每个图像样本块对应的占比大于第一阈值,且小于第二阈值的情况下,获取所 述每个图像样本块对应的标注结果,根据每个所述第二特征图的第三掩膜图以及所述每个图像样本块的标注结果,得到第一损失;
在所述每个图像样本块对应的占比大于或等于所述第二阈值的情况下,对所述每个图像样本块对应的第三掩膜图进行恢复处理,得到所述每个图像样本块对应的第四掩膜图,根据所述图像样本的标注结果以及所述每个图像样本块对应的第四掩膜图,确定第二损失;
根据所述第一损失和所述第二损失,得到目标损失;
根据所述目标损失调整所述神经网络的网络参数。
参阅图6,图6本申请实施例提供的一种病灶分割装置的功能单元组成框图。病灶分割装置600包括:获取单元601和处理单元602,其中:
获取单元601,用于获取眼底彩照图像;
处理单元602,用于对所述眼底彩照图像进行特征提取,得到多个第一特征图,其中,所述多个第一特征图中任意两个第一特征图之间的维度不同;
处理单元602,还用于根据第一特征图A,确定所述眼底彩照图像中与所述第一特征图A对应的多个图像块中每个图像块对应的第一类别以及第一掩膜图,所述第一特征图A为所述多个第一特征图中的任意一个第一特征图;
处理单元602,还用于根据所述每个图像块的第一类别以及第一掩膜图,确定所述眼底彩照图像中各个像素点对应的类别;
处理单元602,还用于根据所述眼底彩照图像中各个像素点的类别,对所述眼底彩照图像进行病灶分割。
在一些可能的实施方式中,在根据第一特征图A,确定所述眼底彩照图像中与所述第一特征图A对应的多个图像块中每个图像块对应的第一类别以及第一掩膜图方面,处理单元602,具体用于:
根据所述第一特征图A进行图像分割,确定所述眼底彩照图像图中与所述第一特征图A对应的多个图像块中每个图像块的第一掩膜图;
根据所述第一特征图A,确定所述每个图像块的特征向量;
根据所述每个图像块的特征向量,确定所述每个图像块的第一类别。
在一些可能的实施方式中,在根据所述每个图像块的第一类别以及第一掩膜图,确定所述眼底彩照图像中各个像素点对应的类别方面,处理单元602,具体用于:
对所述每个图像块对应的第一掩膜图进行恢复处理,得到所述每个图像块对应的第二掩膜图,所述第二掩膜图用于表示所述眼底彩照图像图中各个像素点属于所述每个图像块对应的第一类别的概率;
根据所述每个图像块对应的第二掩膜图以及第一类别,确定所述眼底彩照图像图中各个像素点对应的类别。
在一些可能的实施方式中,在根据所述每个图像块对应的第二掩膜图以及第一类别,确定所述眼底彩照图像图中各个像素点对应的类别方面,处理单元602,具体用于:
对所述每个图像块对应的第一类别进行聚类,得到至少一个第一类别;
将所述至少一个第一类别中的每个第一类别对应的所有图像块的第二掩膜图进行叠加以及归一化,得到所述每个第一类别对应的目标掩膜图;
根据所述每个第一类别对应的目标掩膜图,确定所述眼底彩照图像图中各个像素点属于所述每个第一类别的概率;
根据所述眼底彩照图像图中各个像素点属于所述每个第一类别的概率,确定所述眼底彩照图像图中各个像素点对应的类别。
在一些可能的实施方式中,在对所述每个图像块对应的第一掩膜图进行恢复处理,得到所述每个图像块对应的第二掩膜图方面,处理单元602,具体用于:
通过双线性插值法,对所述每个图像块对应的第一掩膜图进行上采样处理,得到与所述每个图像块对应的第二掩膜图。
在一些可能的实施方式中,获取单元601还用于获取图像样本;
处理单元602,还用于将所述图像样本输入到所述神经网络,得到多个第二特征图,所述多个第二特征图中的任意两个第二特征图之间的维度不同;
根据第二特征图B,确定与所述第二特征图B对应的多个图像样本块中每个图像样本块对应的第三掩膜图以及第二类别,所述第二特征图B为所述多个第二特征图中的任意一个第二特征图;
根据所述每个图像样本块对应的第三掩膜图以及第二类别,调整所述神经网络的网络参数。
在一些可能的实施方式中,在根据所述每个图像样本块对应的第三掩膜图以及第二类别,调整所述神经网络的网络参数方面,处理单元602,具体用于:
根据所述每个图像样本块对应的第三掩膜图以及第二类别,确定所述每个图像样本块中属于病灶的像素点的数量的占比;
在所述每个图像样本块对应的占比大于第一阈值,且小于第二阈值的情况下,获取所述每个图像样本块对应的标注结果,根据每个所述第二特征图的第三掩膜图以及所述每个图像样本块的标注结果,得到第一损失;
在所述每个图像样本块对应的占比大于或等于所述第二阈值的情况下,对所述每个图像样本块对应的第三掩膜图进行恢复处理,得到所述每个图像样本块对应的第四掩膜图,根据所述图像样本的标注结果以及所述每个图像样本块对应的第四掩膜图,确定第二损失;
根据所述第一损失和所述第二损失,得到目标损失;
根据所述目标损失调整所述神经网络的网络参数。
本申请实施例还提供一种计算机存储介质如计算机可读存储介质,所述计算机可读存储介质存储有计算机程序,所述计算机程序被处理器执行以实现如上述方法实施例中记载的任何一种病灶分割方法的部分或全部步骤。
可选的,本申请涉及的存储介质如计算机可读存储介质可以是非易失性的,也可以是易失性的。
本申请实施例还提供一种计算机程序产品,所述计算机程序产品包括存储了计算机程序的非瞬时性计算机可读存储介质,所述计算机程序可操作来使计算机执行如上述方法实施例中记载的任何一种病灶分割方法的部分或全部步骤。
应理解,本申请中的病灶分割装置可以包括智能手机(如Android手机、iOS手机、Windows Phone手机等)、平板电脑、掌上电脑、笔记本电脑、移动互联网设备MID(Mobile Internet Devices,简称:MID)或穿戴式设备等。上述病灶分割装置仅是举例,而非穷举,包含但不限于上述病灶分割装置。在实际应用中,上述病灶分割装置还可以包括:智能车载终端、计算机设备等等。
需要说明的是,对于前述的各方法实施例,为了简单描述,故将其都表述为一系列的动作组合,但是本领域技术人员应该知悉,本申请并不受所描述的动作顺序的限制,因为依据本申请,某些步骤可以采用其他顺序或者同时进行。其次,本领域技术人员也应该知悉,说明书中所描述的实施例均属于可选实施例,所涉及的动作和模块并不一定是本申请所必须的。
在上述实施例中,对各个实施例的描述都各有侧重,某个实施例中没有详述的部分,可以参见其他实施例的相关描述。
在本申请所提供的几个实施例中,应该理解到,所揭露的装置,可通过其它的方式实现。例如,以上所描述的装置实施例仅仅是示意性的,例如所述单元的划分,仅仅为一种 逻辑功能划分,实际实现时可以有另外的划分方式,例如多个单元或组件可以结合或者可以集成到另一个系统,或一些特征可以忽略,或不执行。另一点,所显示或讨论的相互之间的耦合或直接耦合或通信连接可以是通过一些接口,装置或单元的间接耦合或通信连接,可以是电性或其它的形式。
所述作为分离部件说明的单元可以是或者也可以不是物理上分开的,作为单元显示的部件可以是或者也可以不是物理单元,即可以位于一个地方,或者也可以分布到多个网络单元上。可以根据实际的需要选择其中的部分或者全部单元来实现本实施例方案的目的。
另外,在本申请各个实施例中的各功能单元可以集成在一个处理单元中,也可以是各个单元单独物理存在,也可以两个或两个以上单元集成在一个单元中。上述集成的单元既可以采用硬件的形式实现,也可以采用软件程序模块的形式实现。
所述集成的单元如果以软件程序模块的形式实现并作为独立的产品销售或使用时,可以存储在一个计算机可读取存储器中。基于这样的理解,本申请的技术方案本质上或者说对现有技术做出贡献的部分或者该技术方案的全部或部分可以以软件产品的形式体现出来,该计算机软件产品存储在一个存储器中,包括若干指令用以使得一台计算机设备(可为个人计算机、服务器或者网络设备等)执行本申请各个实施例所述方法的全部或部分步骤。而前述的存储器包括:U盘、只读存储器(ROM,Read-Only Memory)、随机存取存储器(RAM,Random Access Memory)、移动硬盘、磁碟或者光盘等各种可以存储程序代码的介质。
本领域普通技术人员可以理解上述实施例的各种方法中的全部或部分步骤是可以通过程序来指令相关的硬件来完成,该程序可以存储于一计算机可读存储器中,存储器可以包括:闪存盘、只读存储器(英文:Read-Only Memory,简称:ROM)、随机存取器(英文:Random Access Memory,简称:RAM)、磁盘或光盘等。
以上对本申请实施例进行了详细介绍,本文中应用了具体个例对本申请的原理及实施方式进行了阐述,以上实施例的说明只是用于帮助理解本申请的方法及其核心思想;同时,对于本领域的一般技术人员,依据本申请的思想,在具体实施方式及应用范围上均会有改变之处,综上所述,本说明书内容不应理解为对本申请的限制。
Claims (20)
- 一种病灶分割方法,包括:获取眼底彩照图像;对所述眼底彩照图像进行特征提取,得到多个第一特征图,其中,所述多个第一特征图中任意两个第一特征图之间的维度不同;根据第一特征图A,确定所述眼底彩照图像中与所述第一特征图A对应的多个图像块中每个图像块对应的第一类别以及第一掩膜图,所述第一特征图A为所述多个第一特征图中的任意一个第一特征图;根据所述每个图像块的第一类别以及第一掩膜图,确定所述眼底彩照图像中各个像素点对应的类别;根据所述眼底彩照图像中各个像素点的类别,对所述眼底彩照图像进行病灶分割。
- 根据权利要求1所述的方法,其中,所述根据第一特征图A,确定所述眼底彩照图像中与所述第一特征图A对应的多个图像块中每个图像块对应的第一类别以及第一掩膜图,包括:根据所述第一特征图A进行图像分割,确定所述眼底彩照图像图中与所述第一特征图A对应的多个图像块中每个图像块的第一掩膜图;根据所述第一特征图A,确定所述每个图像块的特征向量;根据所述每个图像块的特征向量,确定所述每个图像块的第一类别。
- 根据权利要求1或2所述的方法,其中,所述根据所述每个图像块的第一类别以及第一掩膜图,确定所述眼底彩照图像中各个像素点对应的类别,包括:对所述每个图像块对应的第一掩膜图进行恢复处理,得到所述每个图像块对应的第二掩膜图,所述第二掩膜图用于表示所述眼底彩照图像图中各个像素点属于所述每个图像块对应的第一类别的概率;根据所述每个图像块对应的第二掩膜图以及第一类别,确定所述眼底彩照图像图中各个像素点对应的类别。
- 根据权利要求3所述的方法,其中,所述根据所述每个图像块对应的第二掩膜图以及第一类别,确定所述眼底彩照图像图中各个像素点对应的类别,包括:对所述每个图像块对应的第一类别进行聚类,得到至少一个第一类别;将所述至少一个第一类别中的每个第一类别对应的所有图像块的第二掩膜图进行叠加以及归一化,得到所述每个第一类别对应的目标掩膜图;根据所述每个第一类别对应的目标掩膜图,确定所述眼底彩照图像图中各个像素点属于所述每个第一类别的概率;根据所述眼底彩照图像图中各个像素点属于所述每个第一类别的概率,确定所述眼底彩照图像图中各个像素点对应的类别。
- 根据权利要求4所述的方法,其中,所述对所述每个图像块对应的第一掩膜图进行恢复处理,得到所述每个图像块对应的第二掩膜图,包括:通过双线性插值法,对所述每个图像块对应的第一掩膜图进行上采样处理,得到与所述每个图像块对应的第二掩膜图。
- 根据权利要求5所述的方法,其中,得到所述多个第一特征图以及确定所述第一特征图A对应的第一类别以及第一掩膜图是通过神经网络执行的,所述神经网络通过以下步骤训练得到:获取图像样本;将所述图像样本输入到所述神经网络,得到多个第二特征图,所述多个第二特征图中的任意两个第二特征图之间的维度不同;根据第二特征图B,确定与所述第二特征图B对应的多个图像样本块中每个图像样本块对应的第三掩膜图以及第二类别,所述第二特征图B为所述多个第二特征图中的任意一个第二特征图;根据所述每个图像样本块对应的第三掩膜图以及第二类别,调整所述神经网络的网络参数。
- 根据权利要求6所述的方法,其中,所述根据所述每个图像样本块对应的第三掩膜图以及第二类别,调整所述神经网络的网络参数,包括:根据所述每个图像样本块对应的第三掩膜图以及第二类别,确定所述每个图像样本块中属于病灶的像素点的数量的占比;在所述每个图像样本块对应的占比大于第一阈值,且小于第二阈值的情况下,获取所述每个图像样本块对应的标注结果,根据每个所述第二特征图的第三掩膜图以及所述每个图像样本块的标注结果,得到第一损失;在所述每个图像样本块对应的占比大于或等于所述第二阈值的情况下,对所述每个图像样本块对应的第三掩膜图进行恢复处理,得到所述每个图像样本块对应的第四掩膜图,根据所述图像样本的标注结果以及所述每个图像样本块对应的第四掩膜图,确定第二损失;根据所述第一损失和所述第二损失,得到目标损失;根据所述目标损失调整所述神经网络的网络参数。
- 一种病灶分割装置,包括:获取单元,用于获取眼底彩照图像;处理单元,用于对所述眼底彩照图像进行特征提取,得到多个第一特征图,其中,所述多个第一特征图中任意两个第一特征图之间的维度不同;所述处理单元,还用于根据第一特征图A,确定所述眼底彩照图像中与所述第一特征图A对应的多个图像块中每个图像块对应的第一类别以及第一掩膜图,所述第一特征图A为所述多个第一特征图中的任意一个第一特征图;所述处理单元,还用于根据所述每个图像块的第一类别以及第一掩膜图,确定所述眼底彩照图像中各个像素点对应的类别;所述处理单元,还用于根据所述眼底彩照图像中各个像素点的类别,对所述眼底彩照图像进行病灶分割。
- 一种病灶分割装置,包括处理器、存储器、通信接口以及一个或多个程序,其中,所述一个或多个程序被存储在所述存储器中,并且被配置由所述处理器执行以实现以下方法:获取眼底彩照图像;对所述眼底彩照图像进行特征提取,得到多个第一特征图,其中,所述多个第一特征图中任意两个第一特征图之间的维度不同;根据第一特征图A,确定所述眼底彩照图像中与所述第一特征图A对应的多个图像块中每个图像块对应的第一类别以及第一掩膜图,所述第一特征图A为所述多个第一特征图中的任意一个第一特征图;根据所述每个图像块的第一类别以及第一掩膜图,确定所述眼底彩照图像中各个像素点对应的类别;根据所述眼底彩照图像中各个像素点的类别,对所述眼底彩照图像进行病灶分割。
- 根据权利要求9所述的病灶分割装置,其中,执行所述根据第一特征图A,确定所述眼底彩照图像中与所述第一特征图A对应的多个图像块中每个图像块对应的第一类别以及第一掩膜图,包括:根据所述第一特征图A进行图像分割,确定所述眼底彩照图像图中与所述第一特征图 A对应的多个图像块中每个图像块的第一掩膜图;根据所述第一特征图A,确定所述每个图像块的特征向量;根据所述每个图像块的特征向量,确定所述每个图像块的第一类别。
- 根据权利要求9或10所述的病灶分割装置,其中,执行所述根据所述每个图像块的第一类别以及第一掩膜图,确定所述眼底彩照图像中各个像素点对应的类别,包括:对所述每个图像块对应的第一掩膜图进行恢复处理,得到所述每个图像块对应的第二掩膜图,所述第二掩膜图用于表示所述眼底彩照图像图中各个像素点属于所述每个图像块对应的第一类别的概率;根据所述每个图像块对应的第二掩膜图以及第一类别,确定所述眼底彩照图像图中各个像素点对应的类别。
- 根据权利要求11所述的病灶分割装置,其中,执行所述根据所述每个图像块对应的第二掩膜图以及第一类别,确定所述眼底彩照图像图中各个像素点对应的类别,包括:对所述每个图像块对应的第一类别进行聚类,得到至少一个第一类别;将所述至少一个第一类别中的每个第一类别对应的所有图像块的第二掩膜图进行叠加以及归一化,得到所述每个第一类别对应的目标掩膜图;根据所述每个第一类别对应的目标掩膜图,确定所述眼底彩照图像图中各个像素点属于所述每个第一类别的概率;根据所述眼底彩照图像图中各个像素点属于所述每个第一类别的概率,确定所述眼底彩照图像图中各个像素点对应的类别;执行所述对所述每个图像块对应的第一掩膜图进行恢复处理,得到所述每个图像块对应的第二掩膜图,包括:通过双线性插值法,对所述每个图像块对应的第一掩膜图进行上采样处理,得到与所述每个图像块对应的第二掩膜图。
- 根据权利要求12所述的病灶分割装置,其中,得到所述多个第一特征图以及确定所述第一特征图A对应的第一类别以及第一掩膜图是通过神经网络执行的,所述神经网络通过以下步骤训练得到:获取图像样本;将所述图像样本输入到所述神经网络,得到多个第二特征图,所述多个第二特征图中的任意两个第二特征图之间的维度不同;根据第二特征图B,确定与所述第二特征图B对应的多个图像样本块中每个图像样本块对应的第三掩膜图以及第二类别,所述第二特征图B为所述多个第二特征图中的任意一个第二特征图;根据所述每个图像样本块对应的第三掩膜图以及第二类别,调整所述神经网络的网络参数。
- 根据权利要求13所述的病灶分割装置,其中,执行所述根据所述每个图像样本块对应的第三掩膜图以及第二类别,调整所述神经网络的网络参数,包括:根据所述每个图像样本块对应的第三掩膜图以及第二类别,确定所述每个图像样本块中属于病灶的像素点的数量的占比;在所述每个图像样本块对应的占比大于第一阈值,且小于第二阈值的情况下,获取所述每个图像样本块对应的标注结果,根据每个所述第二特征图的第三掩膜图以及所述每个图像样本块的标注结果,得到第一损失;在所述每个图像样本块对应的占比大于或等于所述第二阈值的情况下,对所述每个图像样本块对应的第三掩膜图进行恢复处理,得到所述每个图像样本块对应的第四掩膜图,根据所述图像样本的标注结果以及所述每个图像样本块对应的第四掩膜图,确定第二损失;根据所述第一损失和所述第二损失,得到目标损失;根据所述目标损失调整所述神经网络的网络参数。
- 一种计算机可读存储介质,其中,所述计算机可读存储介质存储有计算机程序,所述计算机程序被处理器执行以实现以下方法:获取眼底彩照图像;对所述眼底彩照图像进行特征提取,得到多个第一特征图,其中,所述多个第一特征图中任意两个第一特征图之间的维度不同;根据第一特征图A,确定所述眼底彩照图像中与所述第一特征图A对应的多个图像块中每个图像块对应的第一类别以及第一掩膜图,所述第一特征图A为所述多个第一特征图中的任意一个第一特征图;根据所述每个图像块的第一类别以及第一掩膜图,确定所述眼底彩照图像中各个像素点对应的类别;根据所述眼底彩照图像中各个像素点的类别,对所述眼底彩照图像进行病灶分割。
- 根据权利要求15所述的计算机可读存储介质,其中,执行所述根据第一特征图A,确定所述眼底彩照图像中与所述第一特征图A对应的多个图像块中每个图像块对应的第一类别以及第一掩膜图,包括:根据所述第一特征图A进行图像分割,确定所述眼底彩照图像图中与所述第一特征图A对应的多个图像块中每个图像块的第一掩膜图;根据所述第一特征图A,确定所述每个图像块的特征向量;根据所述每个图像块的特征向量,确定所述每个图像块的第一类别。
- 根据权利要求15或16所述的计算机可读存储介质,其中,执行所述根据所述每个图像块的第一类别以及第一掩膜图,确定所述眼底彩照图像中各个像素点对应的类别,包括:对所述每个图像块对应的第一掩膜图进行恢复处理,得到所述每个图像块对应的第二掩膜图,所述第二掩膜图用于表示所述眼底彩照图像图中各个像素点属于所述每个图像块对应的第一类别的概率;根据所述每个图像块对应的第二掩膜图以及第一类别,确定所述眼底彩照图像图中各个像素点对应的类别。
- 根据权利要求17所述的计算机可读存储介质,其中,执行所述根据所述每个图像块对应的第二掩膜图以及第一类别,确定所述眼底彩照图像图中各个像素点对应的类别,包括:对所述每个图像块对应的第一类别进行聚类,得到至少一个第一类别;将所述至少一个第一类别中的每个第一类别对应的所有图像块的第二掩膜图进行叠加以及归一化,得到所述每个第一类别对应的目标掩膜图;根据所述每个第一类别对应的目标掩膜图,确定所述眼底彩照图像图中各个像素点属于所述每个第一类别的概率;根据所述眼底彩照图像图中各个像素点属于所述每个第一类别的概率,确定所述眼底彩照图像图中各个像素点对应的类别;执行所述对所述每个图像块对应的第一掩膜图进行恢复处理,得到所述每个图像块对应的第二掩膜图,包括:通过双线性插值法,对所述每个图像块对应的第一掩膜图进行上采样处理,得到与所述每个图像块对应的第二掩膜图。
- 根据权利要求18所述的计算机可读存储介质,其中,得到所述多个第一特征图以及确定所述第一特征图A对应的第一类别以及第一掩膜图是通过神经网络执行的,所述神 经网络通过以下步骤训练得到:获取图像样本;将所述图像样本输入到所述神经网络,得到多个第二特征图,所述多个第二特征图中的任意两个第二特征图之间的维度不同;根据第二特征图B,确定与所述第二特征图B对应的多个图像样本块中每个图像样本块对应的第三掩膜图以及第二类别,所述第二特征图B为所述多个第二特征图中的任意一个第二特征图;根据所述每个图像样本块对应的第三掩膜图以及第二类别,调整所述神经网络的网络参数。
- 根据权利要求19所述的计算机可读存储介质,其中,执行所述根据所述每个图像样本块对应的第三掩膜图以及第二类别,调整所述神经网络的网络参数,包括:根据所述每个图像样本块对应的第三掩膜图以及第二类别,确定所述每个图像样本块中属于病灶的像素点的数量的占比;在所述每个图像样本块对应的占比大于第一阈值,且小于第二阈值的情况下,获取所述每个图像样本块对应的标注结果,根据每个所述第二特征图的第三掩膜图以及所述每个图像样本块的标注结果,得到第一损失;在所述每个图像样本块对应的占比大于或等于所述第二阈值的情况下,对所述每个图像样本块对应的第三掩膜图进行恢复处理,得到所述每个图像样本块对应的第四掩膜图,根据所述图像样本的标注结果以及所述每个图像样本块对应的第四掩膜图,确定第二损失;根据所述第一损失和所述第二损失,得到目标损失;根据所述目标损失调整所述神经网络的网络参数。
Applications Claiming Priority (2)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
CN202011187336.5 | 2020-10-30 | ||
CN202011187336.5A CN112017185B (zh) | 2020-10-30 | 2020-10-30 | 病灶分割方法、装置及存储介质 |
Publications (1)
Publication Number | Publication Date |
---|---|
WO2022088665A1 true WO2022088665A1 (zh) | 2022-05-05 |
Family
ID=73527471
Family Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
PCT/CN2021/096395 WO2022088665A1 (zh) | 2020-10-30 | 2021-05-27 | 病灶分割方法、装置及存储介质 |
Country Status (2)
Country | Link |
---|---|
CN (1) | CN112017185B (zh) |
WO (1) | WO2022088665A1 (zh) |
Cited By (2)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN116385812A (zh) * | 2023-06-06 | 2023-07-04 | 依未科技(北京)有限公司 | 图像分类方法及装置、电子设备及存储介质 |
CN116797611A (zh) * | 2023-08-17 | 2023-09-22 | 深圳市资福医疗技术有限公司 | 一种息肉病灶分割方法、设备及存储介质 |
Families Citing this family (5)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN112017185B (zh) * | 2020-10-30 | 2021-02-05 | 平安科技(深圳)有限公司 | 病灶分割方法、装置及存储介质 |
CN113425248B (zh) * | 2021-06-24 | 2024-03-08 | 平安科技(深圳)有限公司 | 医疗影像评估方法、装置、设备及计算机存储介质 |
CN113838028A (zh) * | 2021-09-24 | 2021-12-24 | 无锡祥生医疗科技股份有限公司 | 一种颈动脉超声自动多普勒方法、超声设备及存储介质 |
CN113749690B (zh) * | 2021-09-24 | 2024-01-30 | 无锡祥生医疗科技股份有限公司 | 血管的血流量测量方法、装置及存储介质 |
CN115187577B (zh) * | 2022-08-05 | 2023-05-09 | 北京大学第三医院(北京大学第三临床医学院) | 一种基于深度学习的乳腺癌临床靶区自动勾画方法和系统 |
Citations (4)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN109872306A (zh) * | 2019-01-28 | 2019-06-11 | 腾讯科技(深圳)有限公司 | 医学图像分割方法、装置和存储介质 |
CN111192285A (zh) * | 2018-07-25 | 2020-05-22 | 腾讯医疗健康(深圳)有限公司 | 图像分割方法、装置、存储介质和计算机设备 |
CN111292301A (zh) * | 2018-12-07 | 2020-06-16 | 北京市商汤科技开发有限公司 | 一种病灶检测方法、装置、设备及存储介质 |
CN112017185A (zh) * | 2020-10-30 | 2020-12-01 | 平安科技(深圳)有限公司 | 病灶分割方法、装置及存储介质 |
Family Cites Families (7)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
JP2634126B2 (ja) * | 1992-07-27 | 1997-07-23 | インターナショナル・ビジネス・マシーンズ・コーポレイション | グラフィックス表示方法および装置 |
CN103345643B (zh) * | 2013-06-13 | 2016-08-24 | 南京信息工程大学 | 一种遥感图像分类方法 |
CN108537197B (zh) * | 2018-04-18 | 2021-04-16 | 吉林大学 | 一种基于深度学习的车道线检测预警装置及预警方法 |
CN108710919A (zh) * | 2018-05-25 | 2018-10-26 | 东南大学 | 一种基于多尺度特征融合深度学习的裂缝自动化勾画方法 |
US10643092B2 (en) * | 2018-06-21 | 2020-05-05 | International Business Machines Corporation | Segmenting irregular shapes in images using deep region growing with an image pyramid |
CN111047591A (zh) * | 2020-03-13 | 2020-04-21 | 北京深睿博联科技有限责任公司 | 基于深度学习的病灶体积测量方法、系统、终端及存储介质 |
CN111768392B (zh) * | 2020-06-30 | 2022-10-14 | 创新奇智(广州)科技有限公司 | 目标检测方法及装置、电子设备、存储介质 |
-
2020
- 2020-10-30 CN CN202011187336.5A patent/CN112017185B/zh active Active
-
2021
- 2021-05-27 WO PCT/CN2021/096395 patent/WO2022088665A1/zh active Application Filing
Patent Citations (4)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN111192285A (zh) * | 2018-07-25 | 2020-05-22 | 腾讯医疗健康(深圳)有限公司 | 图像分割方法、装置、存储介质和计算机设备 |
CN111292301A (zh) * | 2018-12-07 | 2020-06-16 | 北京市商汤科技开发有限公司 | 一种病灶检测方法、装置、设备及存储介质 |
CN109872306A (zh) * | 2019-01-28 | 2019-06-11 | 腾讯科技(深圳)有限公司 | 医学图像分割方法、装置和存储介质 |
CN112017185A (zh) * | 2020-10-30 | 2020-12-01 | 平安科技(深圳)有限公司 | 病灶分割方法、装置及存储介质 |
Cited By (4)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN116385812A (zh) * | 2023-06-06 | 2023-07-04 | 依未科技(北京)有限公司 | 图像分类方法及装置、电子设备及存储介质 |
CN116385812B (zh) * | 2023-06-06 | 2023-08-25 | 依未科技(北京)有限公司 | 图像分类方法及装置、电子设备及存储介质 |
CN116797611A (zh) * | 2023-08-17 | 2023-09-22 | 深圳市资福医疗技术有限公司 | 一种息肉病灶分割方法、设备及存储介质 |
CN116797611B (zh) * | 2023-08-17 | 2024-04-30 | 深圳市资福医疗技术有限公司 | 一种息肉病灶分割方法、设备及存储介质 |
Also Published As
Publication number | Publication date |
---|---|
CN112017185B (zh) | 2021-02-05 |
CN112017185A (zh) | 2020-12-01 |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
WO2022088665A1 (zh) | 病灶分割方法、装置及存储介质 | |
CN108021916B (zh) | 基于注意力机制的深度学习糖尿病视网膜病变分类方法 | |
CN110662484B (zh) | 用于全身测量结果提取的系统和方法 | |
CN110033456B (zh) | 一种医疗影像的处理方法、装置、设备和系统 | |
CN109753978B (zh) | 图像分类方法、装置以及计算机可读存储介质 | |
CN108510482B (zh) | 一种基于阴道镜图像的宫颈癌检测装置 | |
WO2021082691A1 (zh) | 眼部oct图像病灶区域的分割方法、装置及终端设备 | |
WO2020151307A1 (zh) | 病灶自动识别方法、装置及计算机可读存储介质 | |
WO2020215672A1 (zh) | 医学图像病灶检测定位方法、装置、设备及存储介质 | |
WO2020140370A1 (zh) | 眼底出血点的自动检测方法、装置及计算机可读存储介质 | |
Sinthanayothin | Image analysis for automatic diagnosis of diabetic retinopathy | |
CN108615236A (zh) | 一种图像处理方法及电子设备 | |
CN110263755B (zh) | 眼底图像识别模型训练方法、眼底图像识别方法和设备 | |
CN110276408B (zh) | 3d图像的分类方法、装置、设备及存储介质 | |
WO2021159811A1 (zh) | 青光眼辅助诊断装置、方法及存储介质 | |
US20230080098A1 (en) | Object recognition using spatial and timing information of object images at diferent times | |
CN111899247B (zh) | 脉络膜血管的管腔区域识别方法、装置、设备及介质 | |
CN110232318A (zh) | 穴位识别方法、装置、电子设备及存储介质 | |
CN112686855B (zh) | 一种眼象与症状信息的信息关联方法 | |
CN113658165B (zh) | 杯盘比确定方法、装置、设备及存储介质 | |
CN113012093B (zh) | 青光眼图像特征提取的训练方法及训练系统 | |
CN113313680A (zh) | 一种结直肠癌病理图像预后辅助预测方法及系统 | |
CN117038088B (zh) | 糖尿病视网膜病变的发病确定方法、装置、设备及介质 | |
CN112288697B (zh) | 用于量化异常程度的方法、装置、电子设备及可读存储介质 | |
Giancardo et al. | Quality assessment of retinal fundus images using elliptical local vessel density |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
121 | Ep: the epo has been informed by wipo that ep was designated in this application |
Ref document number: 21884408 Country of ref document: EP Kind code of ref document: A1 |
|
NENP | Non-entry into the national phase |
Ref country code: DE |
|
122 | Ep: pct application non-entry in european phase |
Ref document number: 21884408 Country of ref document: EP Kind code of ref document: A1 |