CN114170167B - Polyp segmentation method and computer device based on attention-guided context correction - Google Patents
Polyp segmentation method and computer device based on attention-guided context correction Download PDFInfo
- Publication number
- CN114170167B CN114170167B CN202111434451.2A CN202111434451A CN114170167B CN 114170167 B CN114170167 B CN 114170167B CN 202111434451 A CN202111434451 A CN 202111434451A CN 114170167 B CN114170167 B CN 114170167B
- Authority
- CN
- China
- Prior art keywords
- feature
- semantic information
- polyp
- information image
- channel
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/0002—Inspection of images, e.g. flaw detection
- G06T7/0012—Biomedical image inspection
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/25—Fusion techniques
- G06F18/253—Fusion techniques of extracted features
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/045—Combinations of networks
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/048—Activation functions
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/08—Learning methods
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/10—Segmentation; Edge detection
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/10—Image acquisition modality
- G06T2207/10068—Endoscopic image
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/30—Subject of image; Context of image processing
- G06T2207/30004—Biomedical image processing
- G06T2207/30028—Colon; Small intestine
- G06T2207/30032—Colon polyp
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Data Mining & Analysis (AREA)
- Life Sciences & Earth Sciences (AREA)
- Artificial Intelligence (AREA)
- General Health & Medical Sciences (AREA)
- Health & Medical Sciences (AREA)
- Evolutionary Computation (AREA)
- General Engineering & Computer Science (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Molecular Biology (AREA)
- Software Systems (AREA)
- Biomedical Technology (AREA)
- Biophysics (AREA)
- Computational Linguistics (AREA)
- Mathematical Physics (AREA)
- Computing Systems (AREA)
- Evolutionary Biology (AREA)
- Bioinformatics & Cheminformatics (AREA)
- Bioinformatics & Computational Biology (AREA)
- Medical Informatics (AREA)
- Nuclear Medicine, Radiotherapy & Molecular Imaging (AREA)
- Radiology & Medical Imaging (AREA)
- Quality & Reliability (AREA)
- Image Analysis (AREA)
Abstract
The invention provides a polyp segmentation method based on attention-guided context correction, a computer-readable storage medium and a computer device, comprising: inputting a polyp picture to be segmented into an enhanced context correction model for training, then performing down-sampling, then performing enhanced context correction model training, then performing down-sampling, and repeatedly obtaining a final semantic information image; inputting the final semantic information image into a progressive context fusion model for training, and outputting a semantic information image with fused features; performing upsampling on the semantic information image after feature fusion, performing enhanced context correction model training to obtain a feature mapping image, performing upsampling, performing enhanced context correction model training, and repeatedly obtaining a final feature mapping image with the same size as a polyp image channel to be segmented; and inputting the final feature mapping image into a multi-level feature fusion model for training and outputting a polyp segmentation image. Thereby enabling the present invention to more accurately identify polyps.
Description
Technical Field
The present invention belongs to the medical field, and in particular, to a polyp segmentation method based on attention-guided context correction, a computer-readable storage medium, and a computer device.
Background
Colorectal cancer is a colorectal cancer which is caused by long-term and many reasons when polyps (raised masses in gastrointestinal channels) formed in intestinal tracts are developed at the earliest stage, and if the polyps can be detected and cut off by intervention at the early stage and the colorectal cancer can be prevented, the most effective method for screening and diagnosing the colorectal cancer is colorectal endoscopy which is the most mainstream method for diagnosing the colorectal cancer at present.
However, although the current diagnosis method is advanced and accurate, some problems still exist, according to some professional research reports, in the process of endoscopy, every four polyps are missed to cause incomplete resection, and hidden troubles are left, in addition, the shapes of the polyps are different and changeable, each fine judgment is difficult to be carried out through naked eyes, and especially under the condition that some polyps are not different from the gastrointestinal channel background, the last is that the rapid identification can not be carried out only by people, a lot of time and energy are needed, and the judgment needs to be carried out by adding a lot of workload of gastroenterology doctors under the current medical system.
Disclosure of Invention
The invention aims to provide a polyp segmentation method based on attention-guided context correction, a computer-readable storage medium and a computer device, and aims to solve the problem that hidden dangers are left due to incomplete resection caused by omission of endoscopic polyps.
In a first aspect, the present invention provides a polyp segmentation method based on attention-guided context correction, comprising:
acquiring a polyp picture to be segmented;
inputting a polyp picture to be segmented into an enhanced context correction model for training to obtain a semantic information image, then performing down-sampling on the semantic information image to obtain a down-sampled semantic information image, inputting the down-sampled semantic information image into the enhanced context correction model again for training and then performing down-sampling, and repeating for multiple times to obtain a final semantic information image; the enhanced context correction model is characterized in that an input polyp picture to be segmented is divided into two feature maps with equal channel quantity by channel dimension, one feature map is subjected to attention mechanism to obtain a first feature map, the other feature map is subjected to feature extraction by depth separable convolution to obtain a second feature map, the first feature map and the second feature map are spliced, and then residual errors are connected and fused to output a semantic information image;
inputting the final semantic information image into a progressive context fusion model for training, and outputting a semantic information image with fused features; the progressive context fusion model is used for respectively performing feature extraction on a final semantic information image through a hole convolution and a conventional convolution to obtain two feature maps, splicing the two feature maps and inputting the spliced feature maps into a channel attention mechanism of context modeling to obtain a channel weight, performing feature fusion on the channel weight and the final semantic information image according to channel dimensions, and outputting a semantic information image after feature fusion;
performing up-sampling on the semantic information image after feature fusion to obtain an up-sampled semantic information image, performing enhanced context correction model training on the up-sampled semantic information image to obtain a feature mapping image, performing up-sampling on the feature mapping image again, performing enhanced context correction model training, and repeating for multiple times to obtain a final feature mapping image with the same channel size as the polyp image to be segmented;
inputting the final feature mapping image into a multi-level feature fusion model for training, and outputting a polyp segmentation image; the multi-level feature fusion model is characterized in that the final feature mapping image is up-sampled to the same resolution ratio one by one, the up-sampled feature mapping images are spliced and input into a channel attention mechanism to obtain channel weight, and the channel weight and the final feature mapping output image are modeled by pixel-level multiplication to obtain a polyp segmentation image.
Further, after the upsampling is performed on the semantic information image after the feature fusion, the method further includes: and adding a jump link structure during the training of the enhanced context correction model, and complementing the spatial fine granularity of deep semantic information of a decoding layer by using the representation information of a shallow coding layer.
Further, the inputting of the polyp picture to be segmented into the enhanced context correction model training specifically includes:
defining a polyp picture X to be segmented in Is X in ∈R C×H×W Extracting the characteristics of the polyp picture to be segmented by convolution of 1 multiplied by 1, and outputting two characteristic graphs X with the same channel number 1 And X 2 ，
And
and two feature maps X 1 And X 2 Obtaining a first profile X by means of an attention mechanism and a depth-separable convolution, respectively att And a second feature map X' 2 Namely:
the first characteristic diagram X att And a second characteristic diagram X 2 ' splicing, and then obtaining semantic information image X by residual connection and fusion out Comprises the following steps:
wherein the content of the first and second substances,
is X 1 Sending the feature graph obtained by 1 × 1 convolution through a batch regularization algorithm and a RELU nonlinear activation function; r represents a three-dimensional array image, C is the number of channels, H is the length, W is the width, sigma and
respectively referred to as sigmoid activation function and pixel level summation,
for pixel level multiplication, up is conventional bilinear interpolation upsampling, down is downsampling, cat represents splicing in channel dimension, and X out To output a characteristic map, F 3×3 Representing a 3 x 3 convolution batch normalization and nonlinear activation function.
Further, the training of the progressive context fusion model specifically includes:
defining the final semantic information image X 'as X' belonged to R C×H×W Extracting features by convolution of 1 × 1, and extracting two feature maps X according to conventional convolution and cavity convolution respectively s And X l Namely:
two kinds of feature maps X s And X l Obtaining a spliced characteristic diagram X after splicing cat The method comprises the following steps:
X cat ＝Cat(X l ,X s )；
the spliced feature map X cat Inputting the channel attention mechanism of global context modeling to obtain channel weight, performing feature fusion on the channel weight and the final semantic information image according to channel dimension, and outputting semantic information image y after feature fusion i The method comprises the following steps:
wherein, T p = H · W denotes X cat J represents the lower bound of the summation symbol, β j Representing global attention pooling weights for context modeling, P = S 3 RELU(LN(S 2 (. -)) refers to a bottleneck layer for capturing the dependency between channels, RELU stands for a non-linear activation function, S 2 And S 3 Representing the information interaction between the channel dimensions, the LN is used as an optimizer for LayerNorm.
Further, the step of inputting the final feature map into the multi-level feature fusion model for training specifically comprises:
definition of FinalIs L ∈ R C×H×W The final feature maps are up-sampled to the same resolution one by one and then spliced, and the spliced feature maps are sent into a convolution W of 1 multiplied by 1 1 Extracting the features to obtain an extracted feature map G:
G＝W 1 (Cat(B(l 1 ,l 2 ,l 3 ,l 4 ),l 0 ))；
inputting G into a channel attention mechanism to obtain channel weight, modeling the channel weight and the final feature mapping graph by pixel level multiplication to obtain a polyp segmentation picture Y, wherein the obtained polyp segmentation picture Y comprises the following steps:
wherein, L = (L) 0 ,l 1 ,l 2 ,l 3 ,l 4 )，l 0 -l 4 Respectively represent decoded layer feature maps from large resolution to small resolution, B (l) 1 ,l 2 ,l 3 ,l 4 ),l 0 Represents l 1 -l 4 Upsampling to 0 Splicing is carried out after the resolution is the same, ξ is a correlation coefficient related to G, G is global average pooling, and delta represents an activation function.
In a second aspect, the invention provides a computer-readable storage medium having a computer program stored thereon, wherein the computer program, when being executed by a processor, performs the steps of the method for polyp segmentation based on attention-guided context correction according to the first aspect.
In a third aspect, the invention provides a computer apparatus comprising: one or more processors, a memory, and one or more computer programs, said processors and said memory being connected by a bus, wherein said one or more computer programs are stored in said memory and configured to be executed by said one or more processors, characterized in that said processor, when executing said computer programs, implements the steps of the polyp segmentation method based on attention-directed context correction as described in the first aspect.
In the invention, the operation of downsampling is carried out after the enhanced context correction model is repeatedly trained for many times, deeper semantic information can be obtained, and the interference of background noise can be effectively inhibited; the problem of large-scale polyps in the polyp recognition process is solved through the progressive context fusion model training; through multi-level feature fusion model training, accurate segmentation results can be effectively obtained, and the accuracy of polyp recognition results is improved.
Drawings
Fig. 1 is a flowchart of a polyp segmentation method based on attention-guided context correction according to an embodiment of the present invention.
Fig. 2 is a flowchart of another polyp segmentation method based on attention-guided context correction according to an embodiment of the present invention.
FIG. 3 is a flowchart of training an enhanced context correction model according to an embodiment of the present invention.
FIG. 4 is a flowchart of progressive context fusion model training according to an embodiment of the present invention.
Fig. 5 is a flowchart of multi-level feature fusion model training according to an embodiment of the present invention.
Fig. 6 is a block diagram of a specific structure of a computer device according to an embodiment of the present invention.
Detailed Description
In order to make the objects, technical solutions and advantages of the present invention more clearly apparent, the present invention is described in further detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative of the invention and do not limit the invention.
In order to explain the technical means of the present invention, the following description will be given by way of specific examples.
Referring to fig. 1, a polyp segmentation method based on attention-guided context correction according to an embodiment of the present invention includes the following steps: it should be noted that the polyp segmentation method based on attention-directed context correction according to the present invention is not limited to the flow sequence shown in fig. 1, if the results are substantially the same.
S1, obtaining a polyp picture to be segmented;
s2, inputting a polyp picture to be segmented into an enhanced context correction model for training to obtain a semantic information image, then performing down-sampling on the semantic information image to obtain a down-sampled semantic information image, inputting the down-sampled semantic information image into the enhanced context correction model again for training and then performing down-sampling, and repeating for multiple times to obtain a final semantic information image; the enhanced context correction model is characterized in that an input polyp picture to be segmented is divided into two feature maps with equal channel quantity by channel dimension, one feature map is subjected to attention mechanism to obtain a first feature map, the other feature map is subjected to feature extraction by depth separable convolution to obtain a second feature map, the first feature map and the second feature map are spliced, and then residual errors are connected and fused to output a semantic information image;
s3, inputting the final semantic information image into a progressive context fusion model for training, and outputting a semantic information image with fused features; the progressive context fusion model is used for respectively extracting the characteristics of the final semantic information image through a cavity convolution and a conventional convolution to obtain two characteristic graphs, splicing the two characteristic graphs and inputting the spliced characteristic graphs into a channel attention mechanism of context modeling to obtain channel weights, performing characteristic fusion on the channel weights and the final semantic information image according to channel dimensions, and outputting a semantic information image after the characteristic fusion;
s4, performing up-sampling on the semantic information image subjected to feature fusion to obtain an up-sampled semantic information image, performing enhanced context correction model training on the up-sampled semantic information image to obtain a feature mapping image, performing up-sampling on the feature mapping image again, performing enhanced context correction model training, and repeating for multiple times to obtain a final feature mapping image with the same channel size as the polyp image to be segmented;
s5, inputting the final feature mapping image into a multi-level feature fusion model for training, and outputting a polyp segmentation image; the multi-level feature fusion model is characterized in that the final feature mapping images are up-sampled one by one to the same resolution, the up-sampled feature mapping images are spliced and input into a channel attention mechanism to obtain channel weights, and the channel weights and the final feature mapping output images are modeled by pixel-level multiplication to obtain polyp segmentation images.
Fig. 2 is a flowchart illustrating a polyp segmentation method based on attention-guided context correction according to an embodiment of the present invention, where Input is an Input picture to be segmented, ECC represents an enhanced context correction model, down × 2 represents a downsampling operation (with a convolution kernel size of 2), PCF progressive context fusion model, up × 2 represents an upsampling operation (with a convolution kernel size of 2), MPA is a multi-level feature fusion model, output is an Output segmentation result, and skip connection is residual connection.
In an embodiment of the present invention, after performing upsampling on the semantic information image after the feature fusion, the method further includes: and adding a jump link structure during the training of the enhanced context correction model, and complementing the spatial fine granularity of deep semantic information of a decoding layer by using the representation information of a shallow coding layer.
In an embodiment of the present invention, the inputting of the polyp picture to be segmented into the enhanced context correction model training specifically includes:
defining a polyp picture X to be segmented in Is X in ∈R C×H×W Extracting features from the polyp picture to be segmented by 1X 1 convolution, and outputting two feature maps X with equal channel number 1 And X 2 ，
And
and two feature maps X 1 And X 2 Obtaining a first profile X by means of an attention mechanism and a depth-separable convolution, respectively att And a second characteristic diagram X 2 ', i.e.:
the first characteristic diagram X att And a second characteristic diagram X 2 ' splicing, and then obtaining semantic information image X by residual connection and fusion out Comprises the following steps:
wherein the content of the first and second substances,
is X 1 Sending the feature graph obtained by 1 × 1 convolution through a batch regularization algorithm and a RELU nonlinear activation function; r represents a three-dimensional array image, C is the number of channels, H is the length, W is the width, sigma and
referred to as sigmoid activation function and pixel level summation respectively,
for pixel-level multiplication, up is conventional bilinear interpolation upsampling, down is downsampling, cat represents concatenation in channel dimension, and X out For outputting a characteristic map, F 3×3 Representing a 3 x 3 convolution batch normalization and a nonlinear activation function.
Fig. 3 is a flow chart of enhanced context correction model training, wherein,
a pixel level summation, a pixel level multiplication operation,
the function is activated for the sigmoid and,
in order to perform the down-sampling,
in order to splice in the channel dimension,
is a conventional bilinear interpolated upsampling.
In an embodiment of the present invention, referring to fig. 4, the training of the progressive context fusion model specifically includes:
defining the final semantic information image X 'as X' belonging to R C×H×W Extracting features by convolution of 1 × 1, and extracting two feature maps X according to conventional convolution and cavity convolution respectively s And X l Namely:
two kinds of feature maps X s And X l Obtaining a spliced characteristic diagram X after splicing cat The method comprises the following steps:
X cat ＝Cat(X l ,X s )；
the spliced characteristic diagram X cat Inputting the channel attention mechanism of global context modeling to obtain channel weight, performing feature fusion on the channel weight and the final semantic information image according to channel dimension, and outputting semantic information image y after feature fusion i The method comprises the following steps:
wherein, T p = H · W denotes X cat J represents the lower bound of the summation symbol, β j Representing global attention pooling weights for context modeling, P = S 3 RELU(LN(S 2 (.))) refers to a bottleneck layer for capturing the dependencies between channels, RELU stands for a non-linear activation function, S 2 And S 3 Representing the information interaction between the channel dimensions, LN is used as optimizer for LayerNorm.
In an embodiment of the present invention, the step of inputting the final feature map into the multi-level feature fusion model for training specifically includes:
defining the final characteristic mapping chart L as L epsilon R C×H×W The final feature maps are up-sampled to the same resolution one by one and then spliced, and the spliced feature maps are sent into a convolution W of 1 multiplied by 1 1 Extracting the features to obtain an extracted feature map G:
G＝W 1 (Cat(B(l 1 ,l 2 ,l 3 ,l 4 ),l 0 ))；
inputting G into a channel attention mechanism to obtain channel weight, modeling the channel weight and the final feature mapping graph by pixel level multiplication to obtain a polyp segmentation picture Y, wherein the obtained polyp segmentation picture Y comprises the following steps:
wherein, L = (L) 0 ,l 1 ,l 2 ,l 3 ,l 4 )，l 0 -l 4 Respectively representing decoded layer feature maps from large to small resolution, B (l) 1 ,l 2 ,l 3 ,l 4 ),l 0 Represents l 1 -l 4 Upsampling to 0 Splicing is carried out after the resolution is the same, xi is a correlation coefficient related to G, G is global average pooling, and delta represents an activation function.
Fig. 5 is a flow chart of multi-level fusion model training, wherein,
for the purpose of bi-linear interpolation up-sampling,
is a pixel level multiplication operation.
An embodiment of the present invention provides a computer-readable storage medium storing a computer program which, when executed by a processor, implements the steps of a polyp segmentation method based on attention-guided context correction as provided by an embodiment of the present invention.
Fig. 6 shows a specific block diagram of a computer device according to an embodiment of the present invention, where the computer device 100 includes: one or more processors 101, a memory 102, and one or more computer programs, wherein the processors 101 and the memory 102 are connected by a bus, the one or more computer programs being stored in the memory 102 and configured to be executed by the one or more processors 101, the processor 101 implementing the steps of the attention-directed context correction based polyp segmentation method as provided by an embodiment of the invention when executing the computer programs.
The computer equipment comprises a server, a terminal and the like. The computer device may be a desktop computer, a mobile terminal or a vehicle-mounted device, and the mobile terminal includes at least one of a mobile phone, a tablet computer, a personal digital assistant or a wearable device.
In the embodiment of the invention, the operation of downsampling is carried out after the enhanced context correction model is repeatedly trained for many times, deeper semantic information can be obtained, and the interference of background noise can be effectively inhibited; the problem of large-scale polyps in the polyp recognition process is solved through the progressive context fusion model training; through multi-level feature fusion model training, accurate segmentation results can be effectively obtained, and the accuracy of polyp recognition results is improved.
Those skilled in the art will appreciate that all or part of the steps in the methods of the above embodiments may be implemented by hardware related to instructions of a program, and the program may be stored in a computer-readable storage medium, and the storage medium may include: read Only Memory (ROM), random Access Memory (RAM), magnetic or optical disks, and the like.
The above description is only for the purpose of illustrating the preferred embodiments of the present invention and is not to be construed as limiting the invention, and any modifications, equivalents and improvements made within the spirit and principle of the present invention are intended to be included within the scope of the present invention.
Claims (7)
1. A polyp segmentation method based on attention-directed context correction, comprising:
acquiring a polyp picture to be segmented;
inputting a polyp picture to be segmented into an enhanced context correction model for training to obtain a semantic information image, then performing down-sampling on the semantic information image to obtain a down-sampled semantic information image, inputting the down-sampled semantic information image into the enhanced context correction model again for training and then performing down-sampling, and repeating for multiple times to obtain a final semantic information image; the enhanced context correction model is characterized in that an input polyp picture to be segmented is divided into two feature maps with equal channel quantity according to channel dimensions, one feature map is subjected to attention mechanism to obtain a first feature map, the other feature map is subjected to depth separable convolution to extract features to obtain a second feature map, the first feature map and the second feature map are spliced to obtain a spliced feature map, and then the spliced feature map and pixel features of the polyp picture to be segmented are fused by residual connection and a semantic information image is output;
inputting the final semantic information image into a progressive context fusion model for training, and outputting a semantic information image with fused features; the progressive context fusion model is used for respectively extracting the characteristics of the final semantic information image through a cavity convolution and a conventional convolution to obtain two characteristic graphs, splicing the two characteristic graphs and inputting the spliced characteristic graphs into a channel attention mechanism of context modeling to obtain channel weights, performing characteristic fusion on the channel weights and the final semantic information image according to channel dimensions, and outputting a semantic information image after the characteristic fusion;
performing up-sampling on the semantic information image after feature fusion to obtain an up-sampled semantic information image, performing enhanced context correction model training on the up-sampled semantic information image to obtain a feature mapping image, performing up-sampling on the feature mapping image again, performing enhanced context correction model training, and repeating for multiple times to obtain a final feature mapping image with the same channel size as the polyp image to be segmented;
inputting the final feature mapping image into a multi-level feature fusion model for training, and outputting a polyp segmentation image; the multi-level feature fusion model is characterized in that final feature maps with the same channel size are up-sampled one by one to the same resolution size as a polyp picture to be segmented, the up-sampled feature maps with the same resolution size are spliced and then feature extraction is carried out to obtain extracted feature maps, the extracted feature maps are input into a channel attention mechanism to obtain channel weights, and the channel weights and the extracted feature maps are modeled by pixel-level multiplication to obtain the polyp segmentation picture.
2. The polyp segmentation method according to claim 1, wherein after the upsampling of the feature-fused semantic information image, further comprising: and adding a jump link structure during the training of the enhanced context correction model, and complementing the spatial fine granularity of deep semantic information of a decoding layer by using the representation information of a shallow coding layer.
3. The polyp segmentation method as set forth in claim 1, wherein the inputting of the polyp picture to be segmented into the enhanced context correction model training is specifically:
defining a polyp picture X to be segmented in Is X in ∈R C×H×W Extracting features from the polyp picture to be segmented by 1X 1 convolution, and outputting two feature maps X with equal channel number 1 And X 2 ，
And
and two feature maps X 1 And X 2 Obtaining a first profile X by means of an attention mechanism and a depth-separable convolution, respectively att And a second characteristic diagram X 2 ', i.e.:
the first characteristic diagram X att And a second feature map X' 2 Splicing, and obtaining semantic information image X by residual error connection and fusion out Comprises the following steps:
wherein the content of the first and second substances,
is X 1 Sending a feature graph obtained by 1 × 1 convolution through a batch regularization algorithm and a RELU nonlinear activation function; r represents a three-dimensional array image, C is the number of channels, H is the length, W is the width, sigma and
referred to as sigmoid activation function and pixel level summation respectively,
for pixel-level multiplication, up is conventional bilinearInterpolation upsampling, down downsampling, cat representing the concatenation in channel dimension, X out For outputting a characteristic map, F 3×3 Representing a 3 x 3 convolution batch normalization and a nonlinear activation function.
4. The polyp segmentation method of claim 1, wherein the progressive context fusion model is trained to specifically:
defining the final semantic information image X 'as X' belonging to R C×H×W Extracting features by convolution of 1 × 1, and extracting two feature maps X according to conventional convolution and cavity convolution respectively s And X l Namely:
two feature maps X s And X l Obtaining a spliced characteristic diagram X after splicing cat The method comprises the following steps:
X cat ＝Cat(X l ,X s )；
the spliced characteristic diagram X cat Inputting the channel attention mechanism of global context modeling to obtain channel weight, performing feature fusion on the channel weight and the final semantic information image according to channel dimensions, and outputting a semantic information image y after feature fusion i The method comprises the following steps:
wherein, T p = H · W denotes X cat J represents the lower bound of the summation symbol, β j Representing global attention pooling weights for context modeling, P = S 3 RELU(LN(S 2 (·)) to refer to a bottleneck layerDependent relationships between capture channels, RELU stands for nonlinear activation function, S 2 And S 3 Representing the information interaction between the channel dimensions, LN is used as optimizer for LayerNorm.
5. The polyp segmentation method as set forth in claim 1, wherein the inputting the final feature map into the multi-level feature fusion model for training is specifically:
defining the final characteristic mapping chart L as L epsilon R C×H×W The final feature maps are up-sampled to the same resolution one by one and then spliced, and the spliced feature maps are sent into a convolution W of 1 multiplied by 1 1 Extracting the features to obtain an extracted feature map G:
G＝W 1 (Cat(B(l 1 ,l 2 ,l 3 ,l 4 ),l 0 ))；
inputting G into a channel attention mechanism to obtain channel weight, modeling the channel weight and the final feature mapping graph by pixel level multiplication, and obtaining a polyp segmentation picture Y as follows:
wherein, L = (L) 0 ,l 1 ,l 2 ,l 3 ,l 4 )，l 0 -l 4 Respectively represent decoded layer feature maps from large resolution to small resolution, B (l) 1 ,l 2 ,l 3 ,l 4 ),l 0 Represents l 1 -l 4 Upsampling to 0 Splicing is carried out after the resolution is the same, xi is a correlation coefficient related to G, G is global average pooling, and delta represents an activation function.
6. A computer-readable storage medium, in which a computer program is stored which, when being executed by a processor, carries out the steps of a polyp segmentation method based on attention-guided context correction according to any one of claims 1 to 5.
7. A computer device, comprising: one or more processors, a memory, and one or more computer programs, the processors and the memory being connected by a bus, wherein the one or more computer programs are stored in the memory and configured to be executed by the one or more processors, characterized in that the steps of the attention-directed context correction based polyp segmentation method according to any one of claims 1 to 5 are implemented when the computer programs are executed by the processors.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202111434451.2A CN114170167B (en) | 2021-11-29 | 2021-11-29 | Polyp segmentation method and computer device based on attention-guided context correction |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202111434451.2A CN114170167B (en) | 2021-11-29 | 2021-11-29 | Polyp segmentation method and computer device based on attention-guided context correction |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN114170167A CN114170167A (en) | 2022-03-11 |
| CN114170167B true CN114170167B (en) | 2022-11-18 |
Family
ID=80481501
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202111434451.2A Active CN114170167B (en) | 2021-11-29 | 2021-11-29 | Polyp segmentation method and computer device based on attention-guided context correction |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN114170167B (en) |
Families Citing this family (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN114913325B (en) * | 2022-03-24 | 2024-05-10 | 北京百度网讯科技有限公司 | Semantic segmentation method, semantic segmentation device and computer program product |
| CN114742848B (en) * | 2022-05-20 | 2022-11-29 | 深圳大学 | Polyp image segmentation method, device, equipment and medium based on residual double attention |
| CN115578341B (en) * | 2022-09-30 | 2023-05-12 | 深圳大学 | Method for segmenting large intestine polyps based on attention-directed pyramid context network |
| CN115439470B (en) * | 2022-10-14 | 2023-05-26 | 深圳职业技术学院 | Polyp image segmentation method, computer readable storage medium and computer device |
Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110197182A (en) * | 2019-06-11 | 2019-09-03 | 中国电子科技集团公司第五十四研究所 | Remote sensing image semantic segmentation method based on contextual information and attention mechanism |
| CN111462126A (en) * | 2020-04-08 | 2020-07-28 | 武汉大学 | Semantic image segmentation method and system based on edge enhancement |
| CN112927255A (en) * | 2021-02-22 | 2021-06-08 | 武汉科技大学 | Three-dimensional liver image semantic segmentation method based on context attention strategy |
Family Cites Families (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2021031066A1 (en) * | 2019-08-19 | 2021-02-25 | 中国科学院深圳先进技术研究院 | Cartilage image segmentation method and apparatus, readable storage medium, and terminal device |
| CN112541503B (en) * | 2020-12-11 | 2022-08-26 | 南京邮电大学 | Real-time semantic segmentation method based on context attention mechanism and information fusion |
| CN113486890A (en) * | 2021-06-16 | 2021-10-08 | 湖北工业大学 | Text detection method based on attention feature fusion and cavity residual error feature enhancement |
| CN113298818B (en) * | 2021-07-09 | 2023-08-18 | 大连大学 | Remote sensing image building segmentation method based on attention mechanism and multi-scale features |
| CN113538313B (en) * | 2021-07-22 | 2022-03-25 | 深圳大学 | Polyp segmentation method and device, computer equipment and storage medium |
-
2021
- 2021-11-29 CN CN202111434451.2A patent/CN114170167B/en active Active
Patent Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110197182A (en) * | 2019-06-11 | 2019-09-03 | 中国电子科技集团公司第五十四研究所 | Remote sensing image semantic segmentation method based on contextual information and attention mechanism |
| CN111462126A (en) * | 2020-04-08 | 2020-07-28 | 武汉大学 | Semantic image segmentation method and system based on edge enhancement |
| CN112927255A (en) * | 2021-02-22 | 2021-06-08 | 武汉科技大学 | Three-dimensional liver image semantic segmentation method based on context attention strategy |
Also Published As
| Publication number | Publication date |
|---|---|
| CN114170167A (en) | 2022-03-11 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN114170167B (en) | Polyp segmentation method and computer device based on attention-guided context correction | |
| US11328430B2 (en) | Methods, systems, and media for segmenting images | |
| CN113240580B (en) | Lightweight image super-resolution reconstruction method based on multi-dimensional knowledge distillation | |
| CN109949309B (en) | Liver CT image segmentation method based on deep learning | |
| CN110211045B (en) | Super-resolution face image reconstruction method based on SRGAN network | |
| CN111898701B (en) | Model training, frame image generation and frame insertion methods, devices, equipment and media | |
| CN111369440B (en) | Model training and image super-resolution processing method, device, terminal and storage medium | |
| CN111476719A (en) | Image processing method, image processing device, computer equipment and storage medium | |
| KR101977067B1 (en) | Method for reconstructing diagnosis map by deep neural network-based feature extraction and apparatus using the same | |
| CN115439470B (en) | Polyp image segmentation method, computer readable storage medium and computer device | |
| CN112488923A (en) | Image super-resolution reconstruction method and device, storage medium and electronic equipment | |
| CN112700460A (en) | Image segmentation method and system | |
| CN115953303A (en) | Multi-scale image compressed sensing reconstruction method and system combining channel attention | |
| CN113838047A (en) | Large intestine polyp segmentation method and system based on endoscope image and related components | |
| CN112633260B (en) | Video motion classification method and device, readable storage medium and equipment | |
| CN115293966A (en) | Face image reconstruction method and device and storage medium | |
| CN114926734A (en) | Solid waste detection device and method based on feature aggregation and attention fusion | |
| CN114418987A (en) | Retinal vessel segmentation method and system based on multi-stage feature fusion | |
| CN114332574A (en) | Image processing method, device, equipment and storage medium | |
| CN112053287B (en) | Image super-resolution method, device and equipment | |
| CN117522896A (en) | Self-attention-based image segmentation method and computer equipment | |
| CN117252890A (en) | Carotid plaque segmentation method, device, equipment and medium | |
| Lai et al. | Generative focused feedback residual networks for image steganalysis and hidden information reconstruction | |
| CN116630152A (en) | Image resolution reconstruction method and device, storage medium and electronic equipment | |
| CN116681888A (en) | Intelligent image segmentation method and system |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| TA01 | Transfer of patent application right |
Effective date of registration: 20220429 Address after: 518000 Guangdong city of Shenzhen province Nanshan District Xili Lake Applicant after: SHENZHEN POLYTECHNIC Address before: 518000 Guangdong city of Shenzhen province Nanshan District Xili Lake Applicant before: SHENZHEN POLYTECHNIC Applicant before: University of Science and Technology Liaoning |
|
| TA01 | Transfer of patent application right | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |