CN117830751B - DenseNet-based intestinal polyp LST morphology identification method and DenseNet-based intestinal polyp LST morphology identification device - Google Patents
DenseNet-based intestinal polyp LST morphology identification method and DenseNet-based intestinal polyp LST morphology identification device Download PDFInfo
- Publication number
- CN117830751B CN117830751B CN202410251590.9A CN202410251590A CN117830751B CN 117830751 B CN117830751 B CN 117830751B CN 202410251590 A CN202410251590 A CN 202410251590A CN 117830751 B CN117830751 B CN 117830751B
- Authority
- CN
- China
- Prior art keywords
- layer
- convolution
- image
- image data
- focus
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 208000032177 Intestinal Polyps Diseases 0.000 title claims abstract description 44
- 238000000034 method Methods 0.000 title claims abstract description 24
- 230000000877 morphologic effect Effects 0.000 claims abstract description 45
- 238000013145 classification model Methods 0.000 claims abstract description 42
- 238000001514 detection method Methods 0.000 claims abstract description 31
- 210000001035 gastrointestinal tract Anatomy 0.000 claims abstract description 27
- 238000007781 pre-processing Methods 0.000 claims abstract description 16
- 230000000968 intestinal effect Effects 0.000 claims abstract description 12
- 238000007789 sealing Methods 0.000 claims description 99
- 238000011176 pooling Methods 0.000 claims description 41
- 230000004913 activation Effects 0.000 claims description 31
- 230000003902 lesion Effects 0.000 claims description 19
- 208000037062 Polyps Diseases 0.000 claims description 8
- 238000013135 deep learning Methods 0.000 claims description 8
- 230000003213 activating effect Effects 0.000 claims description 7
- 238000000605 extraction Methods 0.000 claims description 7
- 238000012545 processing Methods 0.000 claims description 7
- 238000013507 mapping Methods 0.000 claims description 6
- 238000010606 normalization Methods 0.000 claims description 6
- 238000012549 training Methods 0.000 claims description 6
- 210000003750 lower gastrointestinal tract Anatomy 0.000 claims 1
- 238000010586 diagram Methods 0.000 description 5
- 230000008878 coupling Effects 0.000 description 3
- 238000010168 coupling process Methods 0.000 description 3
- 238000005859 coupling reaction Methods 0.000 description 3
- 239000008187 granular material Substances 0.000 description 3
- 230000008569 process Effects 0.000 description 3
- 238000004891 communication Methods 0.000 description 2
- 230000006870 function Effects 0.000 description 2
- 230000008595 infiltration Effects 0.000 description 2
- 238000001764 infiltration Methods 0.000 description 2
- 238000012986 modification Methods 0.000 description 2
- 230000004048 modification Effects 0.000 description 2
- 238000012935 Averaging Methods 0.000 description 1
- 206010016654 Fibrosis Diseases 0.000 description 1
- 238000013473 artificial intelligence Methods 0.000 description 1
- 238000013461 design Methods 0.000 description 1
- 238000011161 development Methods 0.000 description 1
- 238000003745 diagnosis Methods 0.000 description 1
- 238000005516 engineering process Methods 0.000 description 1
- 230000004761 fibrosis Effects 0.000 description 1
- 238000002347 injection Methods 0.000 description 1
- 239000007924 injection Substances 0.000 description 1
- 239000000243 solution Substances 0.000 description 1
- 238000006467 substitution reaction Methods 0.000 description 1
- 208000024891 symptom Diseases 0.000 description 1
Landscapes
- Image Analysis (AREA)
Abstract
The application relates to the technical field of computer vision and medical image recognition, in particular to a method and a device for recognizing intestinal polyp LST (least squares) form based on DenseNet, which are used for acquiring intestinal endoscopic image data to be recognized and preprocessing the intestinal endoscopic image data to be recognized; inputting the preprocessed enteroscope image data to be identified into a lower digestive tract detection model, and judging whether a focus exists in the enteroscope image data to be identified; when a focus exists in the enteroscope image data to be identified, cutting the enteroscope image data to be identified according to the position of the focus to obtain focus image data; and inputting the focus image data into a morphological classification model to obtain intestinal polyp LST morphological recognition results of the focus image data. The application realizes the auxiliary operation of recognizing the intestinal endoscope by classifying the LST morphology, and effectively improves the detection rate and accuracy of the focus.
Description
Technical Field
The application relates to the technical field of computer vision and medical image recognition, in particular to a method and a device for recognizing intestinal polyp LST (least squares) morphology based on DenseNet.
Background
With the development of artificial intelligence, deep learning is widely applied to the recognition direction of medical images. Enteroscopy is currently the main technical means of examination of the intestinal region, through which the surface morphology of polyps can be observed. The method for typing the intestinal polyp according to the surface morphology is divided into two types, namely a granule type (LST-G) and a non-granule type (LST-NG), according to the morphology characteristics of the intestinal polyp, and the two types are divided into a granule uniform type and a nodular mixed type according to the existence of large nodules of the symptoms; the latter is classified into flat bump type and pseudo-bump type according to the presence or absence of a concave lesion.
However, there are often cases where SM infiltration occurs in multiple places in the lesion, and when there is a lesion accompanied with fibrosis, local injection is mostly not obvious in the gap, and it is difficult to determine the SM infiltration. Meanwhile, huge workload also affects the decision of doctors, and the dense connection network (DenseNet) is widely applied to computer vision tasks, so that the design of an auxiliary diagnosis technology based on DenseNet deep learning is not slow.
Disclosure of Invention
The application mainly aims to provide a method and a device for identifying intestinal polyp LST (least squares) morphology based on DenseNet, which can realize auxiliary operation of intestinal endoscope identification for doctors by classifying LST morphology, and effectively improve the detection rate and accuracy of lesions.
In order to achieve the above purpose, the present application provides the following technical solutions:
according to a first aspect of the present invention, the present invention claims a method for identifying intestinal polyp LST morphology based on DenseNet, comprising:
Acquiring enteroscope image data to be identified, and preprocessing the enteroscope image data to be identified;
Inputting the preprocessed enteroscope image data to be identified into a lower digestive tract detection model, and judging whether a focus exists in the enteroscope image data to be identified;
When a focus exists in the enteroscope image data to be identified, cutting the enteroscope image data to be identified according to the position of the focus to obtain focus image data;
And inputting the focus image data into a morphological classification model to obtain an intestinal polyp LST morphological recognition result of the focus image data.
Further, the obtaining the enteroscope image data to be identified, and performing a preprocessing operation on the enteroscope image data to be identified, further includes:
and acquiring the enteroscope image data to be identified, and cutting out irrelevant areas of the enteroscope image data to be identified.
Further, the inputting the preprocessed enteroscope image data to be identified into a lower digestive tract detection model, judging whether there is a focus in the enteroscope image data to be identified, and further comprising:
inputting the preprocessed enteroscope image data to be identified into a lower digestive tract detection model, detecting whether a focus exists in the enteroscope image data to be identified through the lower digestive tract model, directly outputting the whole enteroscope image data to be identified if the focus does not exist, and outputting the position of the focus if the focus exists.
Further, when there is a focus in the enteroscope image data to be identified, cutting the enteroscope image data to be identified according to the position of the focus to obtain focus image data, and further including:
cutting the focus on the enteron image data to be identified according to the coordinates of the focus position output by the lower digestive tract detection model, and cutting the focus out;
the coordinates of the lesion location include coordinates of an upper left corner and a lower right corner of the lesion.
Further, the inputting the focus image data into a morphological classification model to obtain an intestinal polyp LST morphological recognition result of the focus image data, further includes:
performing size standardization processing on the focus image data, inputting a morphological classification model, outputting a corresponding classification result, and displaying the result on the enteroscope image data to be identified;
The morphological classification model adopts a classification model based on deep learning, training data is a cut polyp image, a network used by the morphological classification model is DenseNet, and four sealing blocks and three connecting layers are used in the network;
The network structure of the morphological classification model comprises 16 layers, which are sequentially a first BN layer, a first RELU activation layer, a convolution layer, a maximum pooling layer, a first sealing block layer, a first connecting layer, a second sealing block layer, a second connecting layer, a third sealing block layer, a third connecting layer, a fourth sealing block layer, a second BN layer, a second RELU activation layer, a global average pooling layer, a unidimensional layer and a full connecting layer;
Inputting the focus image data into the first BN layer for normalization, and then inputting a first RELU activation layer for mapping to a nonlinear feature space;
the convolution layer carries out the first feature extraction through two-dimensional convolution, and outputs a first focus image;
Inputting the first focus image into the maximum pooling layer, and obtaining an initial image by selecting the maximum value in each small area block in the first focus image as output;
Four convolution blocks are contained in the first, second, third and fourth sealing block layers, each convolution block contains a BN layer, RELU activation layer and a layer of convolution operation, and the convolution kernels of each convolution layer have the same size and number;
Inputting the initial image into a first convolution block of a first sealing block layer to obtain a first convolution image, merging the first convolution image with the initial image, and then inputting the first convolution image with the initial image into a second convolution block to obtain a second convolution image;
Fusing the second convolution image, the first convolution image and the initial image, and then inputting a third convolution block to obtain a third convolution image;
fusing the third convolution image, the second convolution image, the first convolution image and the initial image, and then inputting a fourth convolution block to obtain a fourth convolution image;
fusing the fourth convolution image, the third convolution image, the second convolution image, the first convolution image and the initial image to obtain a first sealing block output image, and inputting the first sealing block output image into a first connecting layer;
the first connecting layer, the second connecting layer and the third connecting layer comprise a BN layer, a RELU activating layer, a convolution layer and an average pooling layer;
After the output image of the first close-joint block is input into the convolution layer of the first connecting layer, the feature quantity is compressed by half, and then the size of the feature map is reduced through the average pooling layer;
the connecting layer integrates the characteristics obtained by the last sealing block layer, and reduces the number, width and height of the output characteristic graphs of the last sealing block layer by half;
The deep feature images which are deeply fused are obtained after the deep feature images pass through the first sealing block layer, the first connecting layer, the second sealing block layer, the second connecting layer, the third sealing block layer, the third connecting layer and the fourth sealing block layer in sequence;
inputting the deep feature image into a second BN layer to normalize the fused features, and enabling a second RELU activation layer to enhance the nonlinearity of the features and enhance the expression capacity of the model;
The input global average pooling layer adds all pixel values of the feature image to obtain an average value, so that the number of parameters is reduced and overfitting is prevented;
Finally, inputting a unidimensional layer and a full-connection layer to obtain a predicted value of the intestinal polyp LST morphological recognition result of the focus image data.
According to a second aspect of the present invention, the present invention claims a DenseNet-based intestinal polyp LST morphology recognition device, comprising:
the preprocessing module is used for acquiring the enteroscope image data to be identified and preprocessing the enteroscope image data to be identified;
The focus detection module inputs the preprocessed enteroscope image data to be identified into a lower digestive tract detection model, and judges whether a focus exists in the enteroscope image data to be identified;
the focus cutting module cuts the enteroscope image data to be identified according to the position of the focus when the focus exists in the enteroscope image data to be identified, so as to obtain focus image data;
And the morphology classification module inputs the focus image data into a morphology classification model to obtain an intestinal polyp LST morphology recognition result of the focus image data.
Further, the preprocessing module further includes:
and acquiring the enteroscope image data to be identified, and cutting out irrelevant areas of the enteroscope image data to be identified.
Further, the focus detection module further includes:
inputting the preprocessed enteroscope image data to be identified into a lower digestive tract detection model, detecting whether a focus exists in the enteroscope image data to be identified through the lower digestive tract model, directly outputting the whole enteroscope image data to be identified if the focus does not exist, and outputting the position of the focus if the focus exists.
Further, the focus clipping module further includes:
cutting the focus on the enteron image data to be identified according to the coordinates of the focus position output by the lower digestive tract detection model, and cutting the focus out;
the coordinates of the lesion location include coordinates of an upper left corner and a lower right corner of the lesion.
Further, the morphology classification module further includes:
performing size standardization processing on the focus image data, inputting a morphological classification model, outputting a corresponding classification result, and displaying the result on the enteroscope image data to be identified;
The morphological classification model adopts a classification model based on deep learning, training data is a cut polyp image, a network used by the morphological classification model is DenseNet, and four sealing blocks and three connecting layers are used in the network;
The network structure of the morphological classification model comprises 16 layers, which are sequentially a first BN layer, a first RELU activation layer, a convolution layer, a maximum pooling layer, a first sealing block layer, a first connecting layer, a second sealing block layer, a second connecting layer, a third sealing block layer, a third connecting layer, a fourth sealing block layer, a second BN layer, a second RELU activation layer, a global average pooling layer, a unidimensional layer and a full connecting layer;
Inputting the focus image data into the first BN layer for normalization, and then inputting a first RELU activation layer for mapping to a nonlinear feature space;
the convolution layer carries out the first feature extraction through two-dimensional convolution, and outputs a first focus image;
Inputting the first focus image into the maximum pooling layer, and obtaining an initial image by selecting the maximum value in each small area block in the first focus image as output;
Four convolution blocks are contained in the first, second, third and fourth sealing block layers, each convolution block contains a BN layer, RELU activation layer and a layer of convolution operation, and the convolution kernels of each convolution layer have the same size and number;
Inputting the initial image into a first convolution block of a first sealing block layer to obtain a first convolution image, merging the first convolution image with the initial image, and then inputting the first convolution image with the initial image into a second convolution block to obtain a second convolution image;
Fusing the second convolution image, the first convolution image and the initial image, and then inputting a third convolution block to obtain a third convolution image;
fusing the third convolution image, the second convolution image, the first convolution image and the initial image, and then inputting a fourth convolution block to obtain a fourth convolution image;
fusing the fourth convolution image, the third convolution image, the second convolution image, the first convolution image and the initial image to obtain a first sealing block output image, and inputting the first sealing block output image into a first connecting layer;
the first connecting layer, the second connecting layer and the third connecting layer comprise a BN layer, a RELU activating layer, a convolution layer and an average pooling layer;
After the output image of the first close-joint block is input into the convolution layer of the first connecting layer, the feature quantity is compressed by half, and then the size of the feature map is reduced through the average pooling layer;
the connecting layer integrates the characteristics obtained by the last sealing block layer, and reduces the number, width and height of the output characteristic graphs of the last sealing block layer by half;
The deep feature images which are deeply fused are obtained after the deep feature images pass through the first sealing block layer, the first connecting layer, the second sealing block layer, the second connecting layer, the third sealing block layer, the third connecting layer and the fourth sealing block layer in sequence;
inputting the deep feature image into a second BN layer to normalize the fused features, and enabling a second RELU activation layer to enhance the nonlinearity of the features and enhance the expression capacity of the model;
The input global average pooling layer adds all pixel values of the feature image to obtain an average value, so that the number of parameters is reduced and overfitting is prevented;
Finally, inputting a unidimensional layer and a full-connection layer to obtain a predicted value of the intestinal polyp LST morphological recognition result of the focus image data.
The application relates to the technical field of computer vision and medical image recognition, in particular to a method and a device for recognizing intestinal polyp LST (least squares) form based on DenseNet, which are used for acquiring intestinal endoscopic image data to be recognized and preprocessing the intestinal endoscopic image data to be recognized; inputting the preprocessed enteroscope image data to be identified into a lower digestive tract detection model, and judging whether a focus exists in the enteroscope image data to be identified; when a focus exists in the enteroscope image data to be identified, cutting the enteroscope image data to be identified according to the position of the focus to obtain focus image data; and inputting the focus image data into a morphological classification model to obtain intestinal polyp LST morphological recognition results of the focus image data. The application realizes the auxiliary operation of recognizing the intestinal endoscope by classifying the LST morphology, and effectively improves the detection rate and accuracy of the focus.
Drawings
FIG. 1 is a workflow diagram of a method for identifying intestinal polyp LST morphology based on DenseNet in accordance with an embodiment of the present application;
FIG. 2 is a schematic view of image-independent region clipping for a DenseNet-based intestinal polyp LST morphology recognition method in accordance with an embodiment of the present application;
FIG. 3 is a schematic view illustrating focus position clipping for a DenseNet-based intestinal polyp LST morphology identification method according to an example of the present application;
FIG. 4 is a schematic view of a morphology classification model junction block layer process for a DenseNet-based intestinal polyp LST morphology recognition method according to an example of the present application;
fig. 5 is a block diagram of a intestinal polyp LST morphology recognition device based on DenseNet according to an embodiment of the present application.
Detailed Description
The following description of the embodiments of the present application will be made clearly and fully with reference to the accompanying drawings, in which it is evident that the embodiments described are only some, but not all embodiments of the application. All other embodiments, which can be made by those skilled in the art based on the embodiments of the application without making any inventive effort, are intended to be within the scope of the application.
The terms "first," "second," "third," and the like in this disclosure are used for descriptive purposes only and are not to be construed as indicating or implying relative importance or implicitly indicating the number of technical features indicated. Thus, a feature defining "first," "second," and "third" may explicitly or implicitly include at least one such feature. In the description of the present application, the meaning of "plurality" means at least two, for example, two, three, etc., unless specifically defined otherwise. All directional indications (such as up, down, left, right, front, back … …) in the embodiments of the present application are merely used to explain the relative positional relationship, movement, etc. between the components in a particular gesture (as shown in the drawings), and if the particular gesture changes, the directional indication changes accordingly. Furthermore, the terms "comprise" and "have," as well as any variations thereof, are intended to cover a non-exclusive inclusion. For example, a process, method, system, article, or apparatus that comprises a list of steps or elements is not limited to only those listed steps or elements but may include other steps or elements not listed or inherent to such process, method, article, or apparatus.
Reference herein to "an embodiment" means that a particular feature, structure, or characteristic described in connection with the embodiment may be included in at least one embodiment of the application. The appearances of such phrases in various places in the specification are not necessarily all referring to the same embodiment, nor are separate or alternative embodiments mutually exclusive of other embodiments. Those of skill in the art will explicitly and implicitly appreciate that the embodiments described herein may be combined with other embodiments.
According to a first embodiment of the present invention, the present invention claims a method for identifying intestinal polyp LST morphology based on DenseNet, referring to fig. 1, comprising:
step 1, obtaining enteroscope image data to be identified, and preprocessing the enteroscope image data to be identified;
Step 2, inputting the preprocessed enteroscope image data to be identified into a lower digestive tract detection model, and judging whether a focus exists in the enteroscope image data to be identified;
Step 3, when a focus exists in the enteroscope image data to be identified, cutting the enteroscope image data to be identified according to the position of the focus to obtain focus image data;
and 4, inputting focus image data into a morphological classification model to obtain an intestinal polyp LST morphological recognition result of the focus image data.
Further, step 1 further includes:
and acquiring the enteroscope image data to be identified, and cutting out irrelevant areas of the enteroscope image data to be identified.
In this embodiment, referring to fig. 2, the irrelevant area is a black area of the image, and clipping coordinates [ x1, y1, x2, y2] = [697,35,1856,1042] are set according to the model of the endoscope, and the area outside this area of the image is clipped.
Further, step 2 further includes:
Inputting the preprocessed enteroscope image data to be identified into a lower digestive tract detection model, detecting whether a focus exists in the enteroscope image data to be identified through the lower digestive tract model, directly outputting the whole enteroscope image data to be identified if the focus does not exist, and outputting the position of the focus if the focus exists.
Further, step3 further includes:
Cutting the focus on the enteron image data to be identified according to the coordinates of the focus position output by the lower digestive tract detection model, and cutting the focus out;
the coordinates of the lesion location include coordinates of the upper left and lower right corners of the lesion.
Wherein, referring to fig. 3, in this embodiment, intestinal polyps are cut out according to the outputted focus position coordinates when the focus is cut out;
when a focus is cut, the polyp region is cut off completely in a rectangular mode;
The object to be cut is to remove the image of the irrelevant area by directly obtaining according to the outputted focus position.
Further, step 4 further includes:
Performing size standardization processing on focus image data, inputting a morphological classification model, outputting a corresponding classification result, and displaying the result on the enteroscope image data to be identified;
the morphological classification model adopts a classification model based on deep learning, training data is a polyp image which is cut off, a network used by the morphological classification model is DenseNet, and four sealing blocks and three connecting layers are used in the network;
The network structure of the morphological classification model comprises 16 layers, which are sequentially a first BN layer, a first RELU activation layer, a convolution layer, a maximum pooling layer, a first sealing block layer, a first connecting layer, a second sealing block layer, a second connecting layer, a third sealing block layer, a third connecting layer, a fourth sealing block layer, a second BN layer, a second RELU activation layer, a global average pooling layer, a unidimensional layer and a full connecting layer;
Inputting focus image data into a first BN layer for normalization, and then inputting a first RELU activation layer for mapping to a nonlinear feature space;
the convolution layer carries out the first feature extraction through two-dimensional convolution, and outputs a first focus image;
Inputting the first focus image into a maximum pooling layer, and obtaining an initial image by selecting the maximum value in each small area block in the first focus image as output;
Four convolution blocks are contained in the first, second, third and fourth sealing block layers, each convolution block contains a BN layer, RELU activation layer and a layer of convolution operation, and the convolution kernels of each convolution layer have the same size and number;
Referring to fig. 4, an initial image is input into a first convolution block of a first junction block layer to obtain a first convolution image, and the first convolution image and the initial image are fused and then input into a second convolution block to obtain a second convolution image;
Fusing the second convolution image, the first convolution image and the initial image, and then inputting a third convolution block to obtain a third convolution image;
fusing the third convolution image, the second convolution image, the first convolution image and the initial image, and then inputting a fourth convolution block to obtain a fourth convolution image;
Fusing the fourth convolution image, the third convolution image, the second convolution image, the first convolution image and the initial image to obtain a first sealing block output image, and inputting the first sealing block output image into the first connecting layer;
the first connecting layer, the second connecting layer and the third connecting layer comprise a BN layer, a RELU activating layer, a convolution layer and an average pooling layer;
After the output image of the first close-joint block is input into the convolution layer of the first connecting layer, the feature quantity is compressed by half, and the size of the feature map is reduced through the average pooling layer;
the connecting layer integrates the characteristics obtained by the last sealing block layer, and reduces the number, width and height of the output characteristic graphs of the last sealing block layer by half;
The deep feature images which are deeply fused are obtained after the deep feature images pass through the first sealing block layer, the first connecting layer, the second sealing block layer, the second connecting layer, the third sealing block layer, the third connecting layer and the fourth sealing block layer in sequence;
Inputting the deep feature image into a second BN layer to normalize the fused features, and enabling the second RELU to activate the nonlinearity of the layer enhanced features and enhance the expression capacity of the model;
The input global average pooling layer adds all pixel values of the feature image to obtain an average value, so that the number of parameters is reduced and overfitting is prevented;
finally, inputting a unidimensional layer and a full-connection layer to obtain a predicted value of the intestinal polyp LST morphological recognition result of the focus image data.
In this embodiment, the shape of the input data is 1×181×181, the data is normalized by layer 1, the data is mapped to the nonlinear feature space by a second RELU activation layer, the convolution kernel of the third layer is 64, the size is 7×7, the step size is 2, and the zero padding is 3. The first feature extraction is performed by two-dimensional convolution, and the output image size is 64×91×91. The features extracted for the first time are input into a maximum pooling layer, the convolution kernel of the maximum pooling layer is 3 multiplied by 3, the step length is 2, zero padding is 1, and the size of the feature map is reduced by selecting the maximum value of each small area (3 multiplied by 3 small block) in the feature map as output, and simultaneously, the number of model parameters is reduced.
The method is input to a fifth layer, namely a first sealing block layer 1, each sealing block comprises four convolution blocks, each convolution block comprises a BN layer, RELU activation layers and a layer of convolution operation, the convolution kernels of each convolution layer are the same in size and number, 32 convolution kernels with the size of 3 multiplied by 3 are adopted, zero padding is adopted, the size of the convolution kernels is equal to 1, the zero padding operation is controlled to be unchanged in the characteristic size of the whole sealing block, and the output number of each convolution layer is the same. After the first convolution block, an output of the 32×46×46 feature map is obtained, and the output feature is fused with the initial input feature, i.e. the input of the next convolution block is 96×46×46 by adding the 32 46×46 feature maps to the 64 inputs.
The obtained fused characteristic is input into a second convolution block, the same convolution layer structure enables the fused characteristic map to output 32 46×46 characteristic maps through the second convolution block, the original input 64 characteristic maps and the first convolution block output 32 characteristic maps are added to obtain the fused 128 46×46 characteristic maps, the next convolution block is input to obtain 32 46×46 characteristic maps again, the original input and the upper two layers output are added to obtain the fused 160 46×46 characteristic maps, the last convolution block is processed to obtain 192 46×46 characteristic maps, namely, after one convolution block is processed, 128 characteristic maps are added, and then the characteristic maps are input into a connecting layer. In the close-coupled block layer, the input of each layer is the output characteristics of the previous layer plus the initial input characteristics, and the information of each layer is saved, used in each subsequent convolution operation, and added to the final output, which fully utilizes the data information.
The connecting layer comprises a BN layer, RELU activating layers, 1 multiplied by 1 convolution of the output quantity of the sealing block layer and AvgPool average pooling layers, the quantity of the characteristics after convolution is compressed by half, the size of the compressed characteristic diagram is also halved through the average pooling layers, and the size of the characteristic diagram output through the connecting layer 1 is 96 multiplied by 23. The function of the connecting layer is to integrate the characteristics obtained by the last sealing block layer and halve the number, width and height of the output characteristic diagrams of the last sealing block layer.
After passing through the sealing block layer 1, the sealing block layer 2, the sealing block layer 3, the sealing block layer 4, the connecting layer 1, the connecting layer 2 and the connecting layer 3, deep fused deep features 248×5×5 are obtained, the fused features are standardized through BN of a twelfth layer, nonlinearity of the features is enhanced through an activation function layer, the expression capacity of a model is enhanced, all pixel values of the feature images are added and averaged through a global averaging pooling layer, a data value is output, 248 data points are output from 248 feature images, and the purposes of reducing the number of parameters and preventing overfitting are achieved. The points are combined into a 248 multiplied by 1 eigenvector which is fully connected through FCNs to obtain a predicted value.
TABLE 1 morphological classification model layer structure table
According to a second embodiment of the present invention, the present invention claims a DenseNet-based intestinal polyp LST morphology recognition device, referring to fig. 5, comprising:
The preprocessing module is used for acquiring the enteroscope image data to be identified and preprocessing the enteroscope image data to be identified;
The focus detection module inputs the preprocessed enteroscope image data to be identified into a lower digestive tract detection model, and judges whether a focus exists in the enteroscope image data to be identified;
the focus cutting module cuts the enteroscope image data to be identified according to the position of the focus when the focus exists in the enteroscope image data to be identified, so as to obtain focus image data;
and the morphology classification module inputs the focus image data into a morphology classification model to obtain an intestinal polyp LST morphology recognition result of the focus image data.
Further, the preprocessing module further comprises:
and acquiring the enteroscope image data to be identified, and cutting out irrelevant areas of the enteroscope image data to be identified.
Further, the focus detection module further includes:
Inputting the preprocessed enteroscope image data to be identified into a lower digestive tract detection model, detecting whether a focus exists in the enteroscope image data to be identified through the lower digestive tract model, directly outputting the whole enteroscope image data to be identified if the focus does not exist, and outputting the position of the focus if the focus exists.
Further, the focus tailors the module, still include:
Cutting the focus on the enteron image data to be identified according to the coordinates of the focus position output by the lower digestive tract detection model, and cutting the focus out;
the coordinates of the lesion location include coordinates of the upper left and lower right corners of the lesion.
Further, the morphology classification module further includes:
Performing size standardization processing on focus image data, inputting a morphological classification model, outputting a corresponding classification result, and displaying the result on the enteroscope image data to be identified;
the morphological classification model adopts a classification model based on deep learning, training data is a polyp image which is cut off, a network used by the morphological classification model is DenseNet, and four sealing blocks and three connecting layers are used in the network;
The network structure of the morphological classification model comprises 16 layers, which are sequentially a first BN layer, a first RELU activation layer, a convolution layer, a maximum pooling layer, a first sealing block layer, a first connecting layer, a second sealing block layer, a second connecting layer, a third sealing block layer, a third connecting layer, a fourth sealing block layer, a second BN layer, a second RELU activation layer, a global average pooling layer, a unidimensional layer and a full connecting layer;
Inputting focus image data into a first BN layer for normalization, and then inputting a first RELU activation layer for mapping to a nonlinear feature space;
the convolution layer carries out the first feature extraction through two-dimensional convolution, and outputs a first focus image;
Inputting the first focus image into a maximum pooling layer, and obtaining an initial image by selecting the maximum value in each small area block in the first focus image as output;
Four convolution blocks are contained in the first, second, third and fourth sealing block layers, each convolution block contains a BN layer, RELU activation layer and a layer of convolution operation, and the convolution kernels of each convolution layer have the same size and number;
inputting the initial image into a first convolution block of a first sealing block layer to obtain a first convolution image, merging the first convolution image and the initial image, and then inputting the first convolution image and the initial image into a second convolution block to obtain a second convolution image;
Fusing the second convolution image, the first convolution image and the initial image, and then inputting a third convolution block to obtain a third convolution image;
fusing the third convolution image, the second convolution image, the first convolution image and the initial image, and then inputting a fourth convolution block to obtain a fourth convolution image;
Fusing the fourth convolution image, the third convolution image, the second convolution image, the first convolution image and the initial image to obtain a first sealing block output image, and inputting the first sealing block output image into the first connecting layer;
the first connecting layer, the second connecting layer and the third connecting layer comprise a BN layer, a RELU activating layer, a convolution layer and an average pooling layer;
After the output image of the first close-joint block is input into the convolution layer of the first connecting layer, the feature quantity is compressed by half, and the size of the feature map is reduced through the average pooling layer;
the connecting layer integrates the characteristics obtained by the last sealing block layer, and reduces the number, width and height of the output characteristic graphs of the last sealing block layer by half;
The deep feature images which are deeply fused are obtained after the deep feature images pass through the first sealing block layer, the first connecting layer, the second sealing block layer, the second connecting layer, the third sealing block layer, the third connecting layer and the fourth sealing block layer in sequence;
Inputting the deep feature image into a second BN layer to normalize the fused features, and enabling the second RELU to activate the nonlinearity of the layer enhanced features and enhance the expression capacity of the model;
The input global average pooling layer adds all pixel values of the feature image to obtain an average value, so that the number of parameters is reduced and overfitting is prevented;
finally, inputting a unidimensional layer and a full-connection layer to obtain a predicted value of the intestinal polyp LST morphological recognition result of the focus image data.
In the several embodiments provided in the present application, it should be understood that the disclosed system, apparatus and method may be implemented in other manners. For example, the apparatus embodiments described above are merely illustrative, e.g., the division of elements is merely a logical functional division, and there may be additional divisions of actual implementation, e.g., multiple elements or components may be combined or integrated into another system, or some features may be omitted, or not performed. Alternatively, the coupling or direct coupling or communication connection shown or discussed with each other may be an indirect coupling or communication connection via some interfaces, devices or units, which may be in electrical, mechanical or other forms.
In addition, each functional unit in the embodiments of the present application may be integrated in one processing unit, or each unit may exist alone physically, or two or more units may be integrated in one unit. The integrated units may be implemented in hardware or in software functional units. The foregoing is only the embodiments of the present application, and the patent scope of the application is not limited thereto, but is also covered by the patent protection scope of the application, as long as the equivalent structure or equivalent flow changes made by the description and the drawings of the application or the direct or indirect application in other related technical fields are adopted.
The embodiments of the application have been described in detail above, but they are merely examples, and the application is not limited to the above-described embodiments. It will be apparent to those skilled in the art that any equivalent modifications or substitutions to this application are within the scope of the application, and therefore, all equivalent changes and modifications, improvements, etc. that do not depart from the spirit and scope of the principles of the application are intended to be covered by this application.
Claims (8)
1. A method for identifying intestinal polyp LST morphology based on DenseNet comprising:
Acquiring enteroscope image data to be identified, and preprocessing the enteroscope image data to be identified;
Inputting the preprocessed enteroscope image data to be identified into a lower digestive tract detection model, and judging whether a focus exists in the enteroscope image data to be identified;
When a focus exists in the enteroscope image data to be identified, cutting the enteroscope image data to be identified according to the position of the focus to obtain focus image data;
inputting the focus image data into a morphological classification model to obtain an intestinal polyp LST morphological recognition result of the focus image data;
inputting the focus image data into a morphological classification model to obtain an intestinal polyp LST morphological recognition result of the focus image data, and further comprising:
performing size standardization processing on the focus image data, inputting a morphological classification model, outputting a corresponding classification result, and displaying the result on the enteroscope image data to be identified;
The morphological classification model adopts a classification model based on deep learning, training data is a cut polyp image, a network used by the morphological classification model is DenseNet, and four sealing blocks and three connecting layers are used in the network;
The network structure of the morphological classification model comprises 16 layers, which are sequentially a first BN layer, a first RELU activation layer, a convolution layer, a maximum pooling layer, a first sealing block layer, a first connecting layer, a second sealing block layer, a second connecting layer, a third sealing block layer, a third connecting layer, a fourth sealing block layer, a second BN layer, a second RELU activation layer, a global average pooling layer, a unidimensional layer and a full connecting layer;
Inputting the focus image data into the first BN layer for normalization, and then inputting a first RELU activation layer for mapping to a nonlinear feature space;
the convolution layer carries out the first feature extraction through two-dimensional convolution, and outputs a first focus image;
Inputting the first focus image into the maximum pooling layer, and obtaining an initial image by selecting the maximum value in each small area block in the first focus image as output;
Four convolution blocks are contained in the first, second, third and fourth sealing block layers, each convolution block contains a BN layer, RELU activation layer and a layer of convolution operation, and the convolution kernels of each convolution layer have the same size and number;
Inputting the initial image into a first convolution block of a first sealing block layer to obtain a first convolution image, merging the first convolution image with the initial image, and then inputting the first convolution image with the initial image into a second convolution block to obtain a second convolution image;
Fusing the second convolution image, the first convolution image and the initial image, and then inputting a third convolution block to obtain a third convolution image;
fusing the third convolution image, the second convolution image, the first convolution image and the initial image, and then inputting a fourth convolution block to obtain a fourth convolution image;
fusing the fourth convolution image, the third convolution image, the second convolution image, the first convolution image and the initial image to obtain a first sealing block output image, and inputting the first sealing block output image into a first connecting layer;
the first connecting layer, the second connecting layer and the third connecting layer comprise a BN layer, a RELU activating layer, a convolution layer and an average pooling layer;
After the output image of the first close-joint block is input into the convolution layer of the first connecting layer, the feature quantity is compressed by half, and then the size of the feature map is reduced through the average pooling layer;
the connecting layer integrates the characteristics obtained by the last sealing block layer, and reduces the number, width and height of the output characteristic graphs of the last sealing block layer by half;
The deep feature images which are deeply fused are obtained after the deep feature images pass through the first sealing block layer, the first connecting layer, the second sealing block layer, the second connecting layer, the third sealing block layer, the third connecting layer and the fourth sealing block layer in sequence;
inputting the deep feature image into a second BN layer to normalize the fused features, and enabling a second RELU activation layer to enhance the nonlinearity of the features and enhance the expression capacity of the model;
The input global average pooling layer adds all pixel values of the feature image to obtain an average value, so that the number of parameters is reduced and overfitting is prevented;
Finally, inputting a unidimensional layer and a full-connection layer to obtain a predicted value of the intestinal polyp LST morphological recognition result of the focus image data.
2. The method for identifying intestinal polyp LST morphology based on DenseNet of claim 1, wherein said obtaining intestinal endoscopic image data to be identified, performing a preprocessing operation on said intestinal endoscopic image data to be identified, further comprises:
and acquiring the enteroscope image data to be identified, and cutting out irrelevant areas of the enteroscope image data to be identified.
3. The method for identifying intestinal polyp LST morphology based on DenseNet of claim 1, wherein said inputting the preprocessed enteron image data to be identified into a lower gastrointestinal tract detection model, determining whether there is a lesion in the enteron image data to be identified, further comprises:
inputting the preprocessed enteroscope image data to be identified into a lower digestive tract detection model, detecting whether a focus exists in the enteroscope image data to be identified through the lower digestive tract model, directly outputting the whole enteroscope image data to be identified if the focus does not exist, and outputting the position of the focus if the focus exists.
4. The method for identifying intestinal polyp LST morphology based on DenseNet of claim 1, wherein when there is a focus in the intestinal endoscopic image data to be identified, cutting the intestinal endoscopic image data to be identified according to the position of the focus to obtain focus image data, further comprising:
cutting the focus on the enteron image data to be identified according to the coordinates of the focus position output by the lower digestive tract detection model, and cutting the focus out;
the coordinates of the lesion location include coordinates of an upper left corner and a lower right corner of the lesion.
5. A DenseNet-based intestinal polyp LST morphology identification device comprising:
the preprocessing module is used for acquiring the enteroscope image data to be identified and preprocessing the enteroscope image data to be identified;
The focus detection module inputs the preprocessed enteroscope image data to be identified into a lower digestive tract detection model, and judges whether a focus exists in the enteroscope image data to be identified;
the focus cutting module cuts the enteroscope image data to be identified according to the position of the focus when the focus exists in the enteroscope image data to be identified, so as to obtain focus image data;
the morphology classification module inputs the focus image data into a morphology classification model to obtain intestinal polyp LST morphology recognition results of the focus image data;
the morphology classification module further comprises:
performing size standardization processing on the focus image data, inputting a morphological classification model, outputting a corresponding classification result, and displaying the result on the enteroscope image data to be identified;
The morphological classification model adopts a classification model based on deep learning, training data is a cut polyp image, a network used by the morphological classification model is DenseNet, and four sealing blocks and three connecting layers are used in the network;
The network structure of the morphological classification model comprises 16 layers, which are sequentially a first BN layer, a first RELU activation layer, a convolution layer, a maximum pooling layer, a first sealing block layer, a first connecting layer, a second sealing block layer, a second connecting layer, a third sealing block layer, a third connecting layer, a fourth sealing block layer, a second BN layer, a second RELU activation layer, a global average pooling layer, a unidimensional layer and a full connecting layer;
Inputting the focus image data into the first BN layer for normalization, and then inputting a first RELU activation layer for mapping to a nonlinear feature space;
the convolution layer carries out the first feature extraction through two-dimensional convolution, and outputs a first focus image;
Inputting the first focus image into the maximum pooling layer, and obtaining an initial image by selecting the maximum value in each small area block in the first focus image as output;
Four convolution blocks are contained in the first, second, third and fourth sealing block layers, each convolution block contains a BN layer, RELU activation layer and a layer of convolution operation, and the convolution kernels of each convolution layer have the same size and number;
Inputting the initial image into a first convolution block of a first sealing block layer to obtain a first convolution image, merging the first convolution image with the initial image, and then inputting the first convolution image with the initial image into a second convolution block to obtain a second convolution image;
Fusing the second convolution image, the first convolution image and the initial image, and then inputting a third convolution block to obtain a third convolution image;
fusing the third convolution image, the second convolution image, the first convolution image and the initial image, and then inputting a fourth convolution block to obtain a fourth convolution image;
fusing the fourth convolution image, the third convolution image, the second convolution image, the first convolution image and the initial image to obtain a first sealing block output image, and inputting the first sealing block output image into a first connecting layer;
the first connecting layer, the second connecting layer and the third connecting layer comprise a BN layer, a RELU activating layer, a convolution layer and an average pooling layer;
After the output image of the first close-joint block is input into the convolution layer of the first connecting layer, the feature quantity is compressed by half, and then the size of the feature map is reduced through the average pooling layer;
the connecting layer integrates the characteristics obtained by the last sealing block layer, and reduces the number, width and height of the output characteristic graphs of the last sealing block layer by half;
The deep feature images which are deeply fused are obtained after the deep feature images pass through the first sealing block layer, the first connecting layer, the second sealing block layer, the second connecting layer, the third sealing block layer, the third connecting layer and the fourth sealing block layer in sequence;
inputting the deep feature image into a second BN layer to normalize the fused features, and enabling a second RELU activation layer to enhance the nonlinearity of the features and enhance the expression capacity of the model;
The input global average pooling layer adds all pixel values of the feature image to obtain an average value, so that the number of parameters is reduced and overfitting is prevented;
Finally, inputting a unidimensional layer and a full-connection layer to obtain a predicted value of the intestinal polyp LST morphological recognition result of the focus image data.
6. The intestinal polyp LST modality identification device based on DenseNet, wherein the preprocessing module further includes:
and acquiring the enteroscope image data to be identified, and cutting out irrelevant areas of the enteroscope image data to be identified.
7. The intestinal polyp LST modality identification device based on DenseNet of claim 6, wherein said lesion detection module further comprises:
inputting the preprocessed enteroscope image data to be identified into a lower digestive tract detection model, detecting whether a focus exists in the enteroscope image data to be identified through the lower digestive tract model, directly outputting the whole enteroscope image data to be identified if the focus does not exist, and outputting the position of the focus if the focus exists.
8. The intestinal polyp LST modality identification device based on DenseNet of claim 7, wherein said lesion clipping module further comprises:
cutting the focus on the enteron image data to be identified according to the coordinates of the focus position output by the lower digestive tract detection model, and cutting the focus out;
the coordinates of the lesion location include coordinates of an upper left corner and a lower right corner of the lesion.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202410251590.9A CN117830751B (en) | 2024-03-06 | 2024-03-06 | DenseNet-based intestinal polyp LST morphology identification method and DenseNet-based intestinal polyp LST morphology identification device |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202410251590.9A CN117830751B (en) | 2024-03-06 | 2024-03-06 | DenseNet-based intestinal polyp LST morphology identification method and DenseNet-based intestinal polyp LST morphology identification device |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN117830751A CN117830751A (en) | 2024-04-05 |
| CN117830751B true CN117830751B (en) | 2024-05-07 |
Family
ID=90506162
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202410251590.9A Active CN117830751B (en) | 2024-03-06 | 2024-03-06 | DenseNet-based intestinal polyp LST morphology identification method and DenseNet-based intestinal polyp LST morphology identification device |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN117830751B (en) |
Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2020077962A1 (en) * | 2018-10-16 | 2020-04-23 | 杭州依图医疗技术有限公司 | Method and device for breast image recognition |
| CN114511502A (en) * | 2021-12-30 | 2022-05-17 | 浙江大学 | Gastrointestinal endoscope image polyp detection system based on artificial intelligence, terminal and storage medium |
| CN116993703A (en) * | 2023-08-11 | 2023-11-03 | 烟台毓璜顶医院(青岛大学附属烟台毓璜顶医院) | Breast CEM image focus recognition system and equipment based on deep learning |
-
2024
- 2024-03-06 CN CN202410251590.9A patent/CN117830751B/en active Active
Patent Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2020077962A1 (en) * | 2018-10-16 | 2020-04-23 | 杭州依图医疗技术有限公司 | Method and device for breast image recognition |
| CN114511502A (en) * | 2021-12-30 | 2022-05-17 | 浙江大学 | Gastrointestinal endoscope image polyp detection system based on artificial intelligence, terminal and storage medium |
| CN116993703A (en) * | 2023-08-11 | 2023-11-03 | 烟台毓璜顶医院(青岛大学附属烟台毓璜顶医院) | Breast CEM image focus recognition system and equipment based on deep learning |
Non-Patent Citations (1)
| Title |
|---|
| 基于深度学习人工智能在结肠镜检查中应用研究;阿依木克地斯・亚力孔;庄惠军;蔡世伦;牛雪静;谭伟敏;颜波;姚礼庆;周平红;钟芸诗;;中国实用外科杂志;20200301(03);第118-122页 * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN117830751A (en) | 2024-04-05 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| Wang et al. | Smartphone-based wound assessment system for patients with diabetes | |
| TWI777092B (en) | Image processing method, electronic device, and storage medium | |
| CN109635783B (en) | Video monitoring method, device, terminal and medium | |
| CN112528831B (en) | Multi-target attitude estimation method, multi-target attitude estimation device and terminal equipment | |
| CN107958453B (en) | Method and device for detecting lesion region of mammary gland image and computer storage medium | |
| US20230052133A1 (en) | Medical image processing method and apparatus, device, storage medium, and product | |
| WO2011028023A2 (en) | Apparatus and method for detecting eye state | |
| CN113222038B (en) | Breast lesion classification and positioning method and device based on nuclear magnetic image | |
| CN111275002A (en) | Image processing method and device and electronic equipment | |
| CN111325266B (en) | Detection method and device for microcalcification clusters in breast molybdenum target image and electronic equipment | |
| CN111191582A (en) | Three-dimensional target detection method, detection device, terminal device and computer-readable storage medium | |
| CN111415373A (en) | Target tracking and segmenting method, system and medium based on twin convolutional network | |
| CN112884702B (en) | Polyp identification system and method based on endoscope image | |
| US11776132B2 (en) | Multi-phase object contour refinement | |
| US20220284569A1 (en) | Multi-phase object contour refinement | |
| CN117830751B (en) | DenseNet-based intestinal polyp LST morphology identification method and DenseNet-based intestinal polyp LST morphology identification device | |
| US20220358650A1 (en) | Systems and methods to process electronic images to provide localized semantic analysis of whole slide images | |
| CN107038705B (en) | Retinal image bleeding area segmentation method and device and computing equipment | |
| WO2022205841A1 (en) | Robot navigation method and apparatus, and terminal device and computer-readable storage medium | |
| CN115223173A (en) | Object identification method and device, electronic equipment and storage medium | |
| CN115497092A (en) | Image processing method, device and equipment | |
| CN114758124A (en) | Occlusion detection method, device, equipment and computer readable medium for target object | |
| CN115115699A (en) | Attitude estimation method and device, related equipment and computer product | |
| CN112288683A (en) | Pulmonary tuberculosis judgment device and method based on multi-mode fusion | |
| Chu et al. | Graph-based region growing for mass-segmentation in digital mammography |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |