CN113570573A - Pulmonary nodule false positive eliminating method, system and equipment based on mixed attention mechanism - Google Patents

Pulmonary nodule false positive eliminating method, system and equipment based on mixed attention mechanism Download PDF

Info

Publication number
CN113570573A
CN113570573A CN202110855736.7A CN202110855736A CN113570573A CN 113570573 A CN113570573 A CN 113570573A CN 202110855736 A CN202110855736 A CN 202110855736A CN 113570573 A CN113570573 A CN 113570573A
Authority
CN
China
Prior art keywords
nodule
false positive
lung
pulmonary
attention mechanism
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
CN202110855736.7A
Other languages
Chinese (zh)
Inventor
李良才
王骏
王艳芳
马力
陈庆武
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Zhongshan Yangshi Technology Co ltd
Original Assignee
Zhongshan Yangshi Technology Co ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Zhongshan Yangshi Technology Co ltd filed Critical Zhongshan Yangshi Technology Co ltd
Priority to CN202110855736.7A priority Critical patent/CN113570573A/en
Publication of CN113570573A publication Critical patent/CN113570573A/en
Pending legal-status Critical Current

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/0002—Inspection of images, e.g. flaw detection
    • G06T7/0012—Biomedical image inspection
    • A—HUMAN NECESSITIES
    • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B6/00—Apparatus for radiation diagnosis, e.g. combined with radiation therapy equipment
    • A61B6/02—Devices for diagnosis sequentially in different planes; Stereoscopic radiation diagnosis
    • A61B6/03—Computerised tomographs
    • A—HUMAN NECESSITIES
    • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B6/00—Apparatus for radiation diagnosis, e.g. combined with radiation therapy equipment
    • A61B6/52—Devices using data or image processing specially adapted for radiation diagnosis
    • A61B6/5211—Devices using data or image processing specially adapted for radiation diagnosis involving processing of medical diagnostic data
    • A61B6/5217—Devices using data or image processing specially adapted for radiation diagnosis involving processing of medical diagnostic data extracting a diagnostic or physiological parameter from medical diagnostic data
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00—Pattern recognition
    • G06F18/20—Analysing
    • G06F18/21—Design or setup of recognition systems or techniques; Extraction of features in feature space; Blind source separation
    • G06F18/214—Generating training patterns; Bootstrap methods, e.g. bagging or boosting
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G06N3/045—Combinations of networks
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/10—Image acquisition modality
    • G06T2207/10072—Tomographic images
    • G06T2207/10081—Computed x-ray tomography [CT]
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/30—Subject of image; Context of image processing
    • G06T2207/30004—Biomedical image processing
    • G06T2207/30061—Lung
    • G06T2207/30064—Lung nodule

Abstract

The invention discloses a pulmonary nodule false positive elimination method, a system, equipment and a medium based on a mixed attention mechanism. The invention relates to a lung nodule false positive eliminating method based on a mixed attention mechanism, which comprises the following steps: obtaining positive pulmonary nodule three-dimensional training data and false positive pulmonary nodule three-dimensional training data; building a TSN candidate pulmonary nodule classification network based on a mixed attention mechanism; inputting training data into a TSN candidate pulmonary nodule classification network based on a mixed attention mechanism for training to obtain a pulmonary nodule false positive exclusion model; obtaining a three-dimensional lung CT image to be excluded, and cutting the three-dimensional lung CT image to obtain candidate lung nodule slice data; and inputting the candidate lung nodule slice data into a lung nodule false positive elimination model, and obtaining a lung nodule false positive elimination result. The lung nodule false positive elimination method based on the mixed attention mechanism has the advantages of appropriate parameter quantity, light and simple calculation amount, high model convergence speed and high lung nodule false positive elimination result precision.

Description

Pulmonary nodule false positive eliminating method, system and equipment based on mixed attention mechanism
Technical Field
The invention relates to the technical field of image processing, in particular to a pulmonary nodule false positive elimination method, a system, equipment and a computer readable storage medium based on a mixed attention mechanism.
Background
Lung cancer is the cancer with the highest morbidity and mortality in the world. Early lung cancer exists in the form of lung nodules, and early diagnosis and treatment of lung nodules can improve patient survival. Computed Tomography (CT) is currently the most commonly used image for lung nodule detection. The existing pulmonary nodule computer-aided detection system mainly comprises two stages of nodule candidate detection and false positive screening. The target of the candidate nodule detection stage is to detect all suspected nodules in the CT image as much as possible so as to improve the lung nodule detection sensitivity; the false positive screening stage aims to classify the detected candidate nodules into true positive nodules and false positive nodules to eliminate the false positive nodules in the candidate nodules, so that the accuracy of lung nodule detection is improved.
In the conventional pulmonary nodule computer-aided detection system, in order to obtain high pulmonary nodule detection sensitivity in a nodule candidate detection stage, a large number of false positive pulmonary nodules exist in detected nodule candidates, so that the false positive pulmonary nodules need to be screened. In conventional false positive lung nodule screening, a 3D CNN model is generally used to perform feature extraction on candidate lung nodules, and then two classifications are used to classify the features.
However, the existing false positive pulmonary nodule screening technology has the following defects: the 3D CNN model has the defects of extremely large parameter quantity, huge and complicated calculated quantity, extremely low model convergence speed and easiness in overfitting.
Disclosure of Invention
Accordingly, an object of the present invention is to provide a method, a system, a device and a computer readable storage medium for eliminating false positive of pulmonary nodule based on a hybrid attention mechanism, which have suitable parameter amount, light and simple calculation amount, fast model convergence rate and high accuracy of the result of eliminating false positive of pulmonary nodule.
The purpose of the invention is realized by the following technical scheme:
in a first aspect, a method for eliminating false positive of pulmonary nodule based on mixed attention mechanism includes the following steps:
obtaining positive pulmonary nodule three-dimensional training data and false positive pulmonary nodule three-dimensional training data;
building a TSN candidate pulmonary nodule classification network based on a mixed attention mechanism;
inputting the three-dimensional training data of the positive pulmonary nodules and the three-dimensional training data of the false positive pulmonary nodules into a TSN candidate pulmonary nodule classification network based on a mixed attention mechanism for training to obtain a pulmonary nodule false positive exclusion model;
obtaining a three-dimensional lung CT image to be excluded, and cutting the three-dimensional lung nodule three-dimensional training data and the three-dimensional lung nodule three-dimensional training data to obtain candidate lung nodule slice data by taking the coordinate positions of the positive lung nodule three-dimensional training data and the false positive lung nodule three-dimensional training data as centers;
and inputting the candidate lung nodule slice data into a lung nodule false positive elimination model, and obtaining a lung nodule false positive elimination result.
The invention discloses a lung nodule false positive elimination method based on a mixed attention mechanism, which comprises the following steps:
(1) regarding the three-dimensional slice data as a two-dimensional slice sequence form in the slice dimension, modeling the sequence by using a TSN network model, and performing overall time sequence learning on the slice sequence.
(2) And improving a Residual block (Residual block) of a backbone network 2D Resnet-18 network used for feature extraction: the hybrid Attention Module (Attention Module) combining Motion Attention (ME), Coordinate Attention (CA), compression-and-Excitation (SE) learns the temporal variation characteristics, spatial position characteristics, and channel importance of the candidate nodules.
(3) The lung nodule false positive elimination method based on the mixed attention mechanism is light-weight in network and convenient to deploy. Compared with a 3D CNN structure model, the method can effectively reduce the parameter quantity and the calculated quantity, still improve the overall accuracy while saving the operation resources, and effectively solve the problems of extremely large parameter quantity, huge and complicated calculated quantity, extremely low model convergence speed and easiness in overfitting in the 3D CNN model.
Further preferably, the constructing of the TSN candidate pulmonary nodule classification network based on the hybrid attention mechanism specifically includes: and a 2D Resnet 18 backbone network is adopted as a TSN feature extraction network, a residual module in the TSN feature extraction network is improved, a mixed attention module combining motion excitation, coordinate attention, compression and excitation is added, and the TSN candidate pulmonary nodule classification network based on the mixed attention mechanism is obtained.
Further preferably, the inputting of the three-dimensional training data of the positive pulmonary nodules and the three-dimensional training data of the false positive pulmonary nodules into a TSN candidate pulmonary nodule classification network based on a mixed attention mechanism for training specifically includes: inputting the three-dimensional training data of the positive pulmonary nodules and the three-dimensional training data of the false positive pulmonary nodules into a TSN candidate pulmonary nodule classification network based on a mixed attention mechanism, and training the TSN candidate pulmonary nodules by using a binary cross entropy loss function to obtain a pulmonary nodule false positive exclusion model; wherein the formula of the binary cross entropy loss function is as follows:
Figure BDA0003184102780000031
wherein y isiAs a true tag, piFalse positive lung nodule prediction.
Further preferably, the method for eliminating false positive of pulmonary nodule based on mixed attention mechanism further comprises the following steps: and preprocessing the three-dimensional lung CT image to be excluded.
In a second aspect, a lung nodule false positive exclusion system based on a mixed attention mechanism includes:
the training data acquisition module is used for acquiring positive pulmonary nodule three-dimensional training data and false positive pulmonary nodule three-dimensional training data;
the classification network building module is used for building a TSN candidate pulmonary nodule classification network based on a mixed attention mechanism;
the model training module is used for inputting the three-dimensional training data of the positive pulmonary nodules and the three-dimensional training data of the false positive pulmonary nodules into a TSN candidate pulmonary nodule classification network based on a mixed attention mechanism for training to obtain a pulmonary nodule false positive exclusion model;
the data to be excluded acquisition module is used for acquiring a three-dimensional lung CT image to be excluded and cutting the three-dimensional lung CT image to obtain candidate lung nodule slice data by taking the coordinate positions of the positive lung nodule three-dimensional training data and the false positive lung nodule three-dimensional training data as centers;
and the result output module is used for inputting the lung nodule candidate slice data into the lung nodule false positive elimination model and obtaining a lung nodule false positive elimination result.
Further preferably, the building of the classification network building module based on the TSN candidate pulmonary nodule classification network of the hybrid attention mechanism specifically includes: and a 2D Resnet 18 backbone network is adopted as a TSN feature extraction network, a residual module in the TSN feature extraction network is improved, a mixed attention module combining motion excitation, coordinate attention, compression and excitation is added, and the TSN candidate pulmonary nodule classification network based on the mixed attention mechanism is obtained.
Further preferably, the model training module inputs the three-dimensional training data of the positive pulmonary nodules and the three-dimensional training data of the false positive pulmonary nodules into a TSN candidate pulmonary nodule classification network based on a mixed attention mechanism for training, and specifically includes: inputting the three-dimensional training data of the positive pulmonary nodules and the three-dimensional training data of the false positive pulmonary nodules into a TSN candidate pulmonary nodule classification network based on a mixed attention mechanism, and training the TSN candidate pulmonary nodules by using a binary cross entropy loss function to obtain a pulmonary nodule false positive exclusion model; wherein the formula of the binary cross entropy loss function is as follows:
Figure BDA0003184102780000041
wherein y isiAs a true tag, piFalse positive lung nodule prediction.
Further preferably, the system for eliminating false positive of pulmonary nodule based on mixed attention mechanism further comprises a preprocessing module for preprocessing the three-dimensional pulmonary CT image to be eliminated.
In a third aspect, a lung nodule false positive exclusion apparatus based on a mixed attention mechanism includes a storage device for storing one or more programs and a processor;
when the one or more programs are executed by the processor, the processor implements a mixed attention mechanism-based lung nodule false positive exclusion method as any of the above.
In a fourth aspect, a computer readable storage medium stores at least one program which, when executed by a processor, implements a mixed attention mechanism-based lung nodule false positive exclusion method as in any one of the above.
Compared with the prior art, the lung nodule false positive elimination method, the system, the equipment and the computer readable storage medium based on the mixed attention mechanism have the advantages of appropriate parameter quantity, light and simple calculation amount, high model convergence speed and high lung nodule false positive elimination result precision.
For a better understanding and practice, the invention is described in detail below with reference to the accompanying drawings.
Drawings
FIG. 1 is a flow chart of a method of lung nodule false positive exclusion based on a mixed attention mechanism of the present invention.
FIG. 2 is a modeling schematic of a compression and excitation module in a hybrid attention machine.
FIG. 3 is a schematic diagram of modeling a motion excitation module in a hybrid attention mechanism.
FIG. 4 is a modeling diagram of a coordinate attention module in a hybrid attention mechanism.
Fig. 5 shows the overall structure of the network of the lung nodule false positive exclusion model in the present invention.
Figure 6 shows the sequence of true positive lung nodule slices.
Fig. 7 shows a sequence of false positive lung nodule slices.
FIG. 8 is a block diagram of a lung nodule false positive exclusion system based on a mixed attention mechanism of the present invention.
Detailed Description
The terms of orientation of up, down, left, right, front, back, top, bottom, and the like, referred to or may be referred to in this specification, are defined relative to their configuration, and are relative concepts. Therefore, it may be changed according to different positions and different use states. Therefore, these and other directional terms should not be construed as limiting terms.
The implementations described in the exemplary embodiments below are not intended to represent all implementations consistent with the present disclosure. Rather, they are merely examples of methods consistent with certain aspects of the present disclosure, as detailed in the appended claims.
The terminology used in the present disclosure is for the purpose of describing particular embodiments only and is not intended to be limiting of the disclosure. As used in this disclosure and the appended claims, the singular forms "a," "an," and "the" are intended to include the plural forms as well, unless the context clearly indicates otherwise. It should also be understood that the term "and/or" as used herein refers to and encompasses any and all possible combinations of one or more of the associated listed items.
A lung nodule false positive elimination method based on a mixed attention mechanism is shown in figure 1 and comprises the following steps:
and S1, obtaining positive lung nodule three-dimensional training data and false positive lung nodule three-dimensional training data.
Specifically, the three-dimensional training data of the positive pulmonary nodule and the three-dimensional training data of the false positive pulmonary nodule may be obtained from a Lung CT public data set LUNA2016(Lung nodeleanalysis 16), which is obtained by combining the positive pulmonary nodule and the false positive pulmonary nodule detected by a plurality of pulmonary nodule detection systems, respectively, and corresponding labeling information is stored in a candidates _ v2.csv file, where the labeling information includes seriesuid of a CT to which the pulmonary nodule belongs, world coordinates (x, y, z) of the pulmonary nodule, and a category of the pulmonary nodule (the true positive pulmonary nodule is 1, and the false positive pulmonary nodule is 0), the world coordinates of the pulmonary nodule are converted into corresponding preprocessed pixel coordinates, and a positive sample (the true positive pulmonary nodule) and a negative sample (the false positive pulmonary nodule) with a size of 42 × 42 pixels are cut out by using the pixel coordinates as a center, so as to construct a training set, a verification set, and a test set.
In this embodiment, the three-dimensional training data of the positive lung nodule and the three-dimensional training data of the false positive lung nodule may be training data including several pieces of slice data obtained from a three-dimensional CT image of the lung nodule.
And S2, constructing a TSN candidate lung nodule classification network based on the mixed attention mechanism.
Specifically, a 2D Resnet 18 backbone network is adopted as a TSN feature extraction network, a residual module in the TSN feature extraction network is improved, a mixed Attention module combining Motion Excitation (ME), Coordinate Attention (CA), compression and Excitation (SE) is added, and the TSN candidate pulmonary nodule classification network based on the mixed Attention mechanism is obtained.
Time series segmentation network (TSN), the first framework to use 2D CNN for video behavior recognition. The TSN extracts short-term motion characteristics of images at different time sequence positions, and finally, a plurality of extracted characteristics are fused to realize long-term motion characteristic learning. In particular, given a video V, it is divided into K parts of the same duration S1,S2,…,SKFrom the corresponding fragment SkIn randomly extracting a frame TkTSN frame sequence (T)1,T2,…,TK) Modeling is performed as follows:
TSN(T1,T2,…,TK)＝H(G(F(T1,W),F(T2,W),…,F(TK,W))) (1)
wherein F (T)KW) is a function representing a convolutional network with parameter W, video frame TKAnd obtaining a C-dimensional vector after the convolution network, wherein C represents the number of the categories. The function G (-) is a segment consensus function, which can beAnd fusing the prediction vectors of the K segments, wherein the function H (-) is a Softmax function and is used for predicting the category of the video.
The model algorithm is based on a TSN model thought, a lung nodule three-dimensional CT image is regarded as a combination in the slicing direction (the cross section and the Z-axis direction), TSN is used for segmenting the slices, feature vectors of all segments are respectively provided through a convolution network, finally the feature vectors of all segments are fused, and Softmax is used for outputting class probability (true positive lung nodule probability or false positive lung nodule probability) to the fused feature vectors.
The feature extraction network uses a Deep Residual network (Deep Residual Networks), and a 2D ResNet-18 backbone network is selected as a TSN feature extraction network G (-) in consideration of the problems of large volume and overfitting of pulmonary nodule training data.
The 2D ResNet-18 backbone network is used as a TSN feature extraction network G (-) and the specific feature extraction is described as follows:
converting the shapes [ L, H and W ] of the lung nodules in the positive lung nodule three-dimensional training data and the false positive lung nodule three-dimensional training data into the forms of [ T, C, H and W ], wherein L is the lung nodule slice number of the positive lung nodule three-dimensional training data and the false positive lung nodule three-dimensional training data, H, W corresponds to the width and the height of a section, T represents the number of segments, and C represents the number of channels. Considering that the number L of pulmonary nodule slices is small, after a nodule is divided into T segments, each segment is not randomly selected, but the slice allocated to each segment is treated as a channel, that is, L is T × C. The T segment data are each input into 2D ResNet-18 to extract segmented features F ═ F1, F2, …, FT ], where F is the representation of the features in the hidden layer with dimensions [ T, C ', H', W '], considering the case of Batch Size, F has dimensions [ N, T, C', H ', W' ], where N is the Size of Batch Size.
In other embodiments, the feature extraction network may also employ other backbone networks, including but not limited to ResNet, ResNext, densnet, MobileNet, inclepionnet, and other feature extraction networks.
The mixed Attention module comprises three parts, namely a compression-and-Excitation (SE) module, a Motion Excitation (ME) module and a Coordinate Attention (CA) module, and is used for respectively learning the channel importance degree, the adjacent segment Motion information and the spatial position information of the lung nodule characteristics.
It should be noted that, the tensor outside the hybrid attention module is 4D, i.e., [ N × T, C, H, W ], where N represents the back Size, T represents the number of segments, C represents the number of color channels, H represents the height, and W represents the width, when the hybrid attention module is input in the TSN model, the input 4D tensor should be firstly reshaped into 5D tensor [ N, T, C, H, W ], so that the operation can be performed on a specific dimension inside the hybrid attention module. The 5D tensor output by the hybrid attention module is then reshaped into a 4D tensor, which is provided for subsequent use by 2D convolution.
The compression and excitation module is a channel attention mechanism, mainly from the aspect of characteristic channel correlation. The directivity of the convolutional layer extracted features is enhanced by adaptively recalibrating the feature response of the channels by explicitly modeling the interdependencies between the channels. The core of the compression and excitation module used in the present invention is divided into two parts, compression and excitation, as shown in fig. 2.
Compression given input characteristics
Figure BDA0003184102780000091
X represents the characteristics of the three-dimensional training data of the positive pulmonary nodules or the characteristics of the three-dimensional training data of the false positive pulmonary nodules, and global average pooling is performed and can be represented as:
Figure BDA0003184102780000092
wherein
Figure BDA0003184102780000093
Excitation of feature F after extrusionsBy giving a scaling r, the features are obtained by 1 x 1 two-dimensional convolution
Figure BDA0003184102780000094
To FrAfter ReLU nonlinear transformation, the features are restored by 1 × 1 two-dimensional convolution
Figure BDA0003184102780000095
Will be characterized by FtempInputting the signal into a Sigmoid activation function to obtain an inverted channel mask,
Figure BDA0003184102780000096
Figure BDA0003184102780000097
the output characteristics of the final compression and excitation module can be expressed as:
Fo＝X+X⊙Fm(3) wherein, in the step (A),
Figure BDA0003184102780000098
the purpose of the motion excitation module is to model the motion information of neighboring segments, as shown in figure 3,
to reduce the amount of computation, the input feature X is first squeezed using a 1 × 1 two-dimensional convolution, the feature after squeezing being
Figure BDA0003184102780000099
The modeling operation on the motion characteristics can then be expressed as:
Fme＝K*Fr[:,t+1,:,:,:]－Fr[:,t,:,:,:] (4)
wherein K is a 3X 3 2D convolutional layer,
Figure BDA00031841027800000910
the motion features are then concatenated according to the time dimension, with 0 filling the last element, i.e., FME＝[Fme(1),...,Fme(t-1),0]Wherein
Figure BDA00031841027800000911
Then F is paired with the above formula (2)MEPerforming spatial average pooling, and restoring the processed characteristic channel by 1 × 1 two-dimensional convolution to obtain characteristics
Figure BDA00031841027800000912
Will be characterized by FtempInputting the information into a Sigmoid activation function to obtain a corresponding mask,
Figure BDA00031841027800000913
Figure BDA00031841027800000914
finally, the output of the ME module is obtained according to the formula (3)
Figure BDA00031841027800000915
The compression and excitation modules (channel attention) usually ignore the position information, which is very important for generating a spatially selective attention map. The coordinate attention module embeds position information into channel attention, unlike channel attention which converts feature tensors into a single feature vector through two-dimensional global pooling, the coordinate attention module is an encoding process that decomposes channel attention into two one-dimensional features, aggregating the features along two spatial directions, respectively. Therefore, the remote dependency relationship in one spatial direction can be obtained, and the accurate position information in the other spatial direction can be reserved. The generated feature maps are then encoded into a pair of direction-aware and position-sensitive attention maps, respectively, which can be applied complementarily to the input feature maps to increase the representation of the object of interest. The details of the coordinate attention module used in the present invention are shown in fig. 4, and are specifically divided into two parts of coordinate information embedding and coordinate attention generation.
Coordinate information embedding: the input feature X is decomposed into a pair of one-dimensional feature encoding operations using global pooling equation (2). Specifically, for input feature X, each channel is first encoded along the horizontal and vertical coordinate directions using pooling kernels of sizes (H, 1) and (1, W). For the C channel, the output in the H direction can be expressed by equation (5):
Figure BDA0003184102780000101
i represents a value between 0 (inclusive of 0) and W (exclusive of W).
Similarly, for the C-th channel, the output in the W direction can be expressed by equation (6):
Figure BDA0003184102780000102
j represents a value between 0 (inclusive of 0) and H (exclusive of H).
The above two transformations perform feature aggregation along two spatial directions, returning a pair of direction-aware attention maps.
Coordinate attention generation: for the aggregate feature maps generated by equations (5) and (6), they are first stitched together and then used with a shared 1 × 1 convolution transform function F1To obtain
f＝S(F1([zh,zw])) (7)
Wherein [ Z ] ish，ZW]Representing the stitching operation along the spatial dimension, delta is a non-linear activation function,
Figure BDA0003184102780000111
Figure BDA0003184102780000112
intermediate feature maps obtained by encoding spatial information from the horizontal direction and the vertical direction. F is then decomposed into two separate tensors along the spatial dimension
Figure BDA0003184102780000113
And
Figure BDA0003184102780000114
Figure BDA0003184102780000115
reusing two 1 x 1 convolution transform functions FhAnd FwRespectively converting the tension back to the same number of channels as the input characteristic X to obtain ghAnd gwRespectively expressed as:
gh＝δ(Fh(fh)) (8)
gw＝S(Fw(fw)) (9)
g is prepared fromh、gwAs attention weights, the final attention module output is:
Fo＝X+X×gh×gw(10) wherein
Figure BDA0003184102780000116
The feature fusion strategy in the invention is as follows: and directly fusing the extracted T feature vectors by using an average pooling (avgPooling) mode. And outputting the classification probability (the probability of the true positive pulmonary nodule or the probability of the false positive pulmonary nodule) by using Softmax for the fused feature vector.
S3, inputting the three-dimensional training data of the positive pulmonary nodules and the three-dimensional training data of the false positive pulmonary nodules into a TSN candidate pulmonary nodule classification network based on a mixed attention mechanism, and training the TSN candidate pulmonary nodule classification network by utilizing a binary cross entropy loss function to obtain a pulmonary nodule false positive exclusion model; wherein the formula of the binary cross entropy loss function is as follows:
Figure BDA0003184102780000117
wherein y isiAs a true tag, piFalse positive lung nodule prediction.
Specifically, 42 × 42 pixels of positive lung nodule three-dimensional training data and false positive lung nodule three-dimensional training data are subjected to data enhancement (x-axis, y-axis and z-axis are randomly inverted, 36 × 36 size regions are randomly cut, and x-axis and y-axis are randomly rotated by 90, 180 and 270 degrees). The enhanced data reshape is input into a TSN classification model in a form of 12 × 3 × 36, 12 time sequence feature vectors are extracted through a backbone network, the 12 time sequence feature vectors are subjected to average pooling to obtain final fusion features, and the final fusion features are subjected to binary classification. The overall structure of the network of the lung nodule false positive exclusion model is shown in fig. 5.
And S4, obtaining a three-dimensional lung CT image to be excluded, and cutting the three-dimensional lung nodule three-dimensional training data and the three-dimensional training data of the false positive lung nodule by taking the coordinate position of the three-dimensional training data of the positive lung nodule and the three-dimensional training data of the false positive lung nodule as a center to obtain candidate lung nodule slice data. Referring to fig. 6-7, fig. 6 is a sequence of slices of a true positive lung nodule, fig. 7 is a sequence of slices of a false positive lung nodule, and one row lists the partial slice sequences of one lung nodule.
This step further includes step S41: and preprocessing the three-dimensional lung CT image to be excluded. Specifically, the spacing uniform sampling of the three-dimensional lung CT image data is adjusted to be 1mm × 1mm × 1mm pixel interval, and the HU value of the CT image is intercepted to the interval [ -1200,600], and is converted into the gray value between [0, 255 ].
And S5, inputting the lung nodule candidate slice data into a lung nodule false positive elimination model, and obtaining a lung nodule false positive elimination result.
When the lung nodule false positive exclusion model is used for obtaining the false positive lung nodule predicted value of the lung nodule candidate slice data, the probability that the lung nodule candidate slice data is a false positive lung nodule can be obtained, so that whether the lung nodule candidate represented by the lung nodule candidate slice data is excluded or not is judged, and when the predicted value is larger than a preset threshold value, the lung nodule candidate represented by the lung nodule candidate slice data is considered as a false positive lung nodule, namely, the lung nodule candidate represented by the lung nodule candidate slice data is excluded.
The invention discloses a lung nodule false positive elimination method based on a mixed attention mechanism, which comprises the following steps:
(1) regarding the three-dimensional slice data as a two-dimensional slice sequence form in the slice dimension, modeling the sequence by using a TSN network model, and performing overall time sequence learning on the slice sequence.
(2) And improving a Residual block (Residual block) of a backbone network 2D Resnet-18 network used for feature extraction: the hybrid Attention Module (Attention Module) combining Motion Attention (ME), Coordinate Attention (CA), compression-and-Excitation (SE) learns the temporal variation characteristics, spatial position characteristics, and channel importance of the candidate nodules.
(3) The lung nodule false positive elimination method based on the mixed attention mechanism is light-weight in network and convenient to deploy. Compared with a 3D CNN structure model, the method can effectively reduce the parameter quantity and the calculated quantity, and the overall accuracy is still improved while the operation resources are saved. See table below:
Figure BDA0003184102780000131
compared with a model without a mixed attention module, the TSN candidate lung nodule classification network model based on the mixed attention mechanism and constructed by the lung nodule false positive elimination method based on the mixed attention mechanism increases about 22 thousands of parameters under the condition of using the mixed attention module, and the accuracy of predicting true positive lung nodules and false positive lung nodules is improved by 0.64%.
Compared with the 3D Resnet-18, the TSN candidate pulmonary nodule classification network model based on the mixed attention mechanism and built by the pulmonary nodule false positive elimination method based on the mixed attention mechanism has the advantages that the parameter quantity is about one third, and meanwhile, the accuracy rate of predicting the true positive pulmonary nodules and the false positive pulmonary nodules is improved by 0.28%.
Therefore, the lung nodule false positive elimination method based on the mixed attention mechanism can effectively solve the problems of extremely large parameter quantity, huge and complicated calculated quantity, extremely low model convergence speed and easiness in overfitting in a 3D CNN model.
In addition, it should be noted that the invention is applicable to the second-stage false positive lung nodule screening task of the lung nodule computer-aided detection rule system, and does not relate to the design of the detection network of the first-stage candidate lung nodule. Any nodule candidate detection network capable of predicting the coordinate position of a nodule candidate can be applied to the method provided by the invention to carry out false positive screening processing.
Furthermore, the classification algorithm used by the invention can classify three-dimensional targets of other CT classes, and is also applicable.
The invention also discloses a lung nodule false positive elimination system based on a mixed attention mechanism, as shown in fig. 8, comprising:
the training data acquisition module is used for acquiring positive pulmonary nodule three-dimensional training data and false positive pulmonary nodule three-dimensional training data;
the classification network building module is used for building a TSN candidate pulmonary nodule classification network based on a mixed attention mechanism;
the model training module is used for inputting the three-dimensional training data of the positive pulmonary nodules and the three-dimensional training data of the false positive pulmonary nodules into a TSN candidate pulmonary nodule classification network based on a mixed attention mechanism for training to obtain a pulmonary nodule false positive exclusion model;
the data to be excluded acquisition module is used for acquiring a three-dimensional lung CT image to be excluded and cutting the three-dimensional lung CT image to obtain candidate lung nodule slice data by taking the coordinate positions of the positive lung nodule three-dimensional training data and the false positive lung nodule three-dimensional training data as centers;
and the result output module is used for inputting the lung nodule candidate slice data into the lung nodule false positive elimination model and obtaining a lung nodule false positive elimination result.
Further preferably, the building of the classification network building module based on the TSN candidate pulmonary nodule classification network of the hybrid attention mechanism specifically includes: and a 2D Resnet 18 backbone network is adopted as a TSN feature extraction network, a residual module in the TSN feature extraction network is improved, a mixed attention module combining motion excitation, coordinate attention, compression and excitation is added, and the TSN candidate pulmonary nodule classification network based on the mixed attention mechanism is obtained.
Further preferably, the model training module inputs the three-dimensional training data of the positive pulmonary nodules and the three-dimensional training data of the false positive pulmonary nodules into a TSN candidate pulmonary nodule classification network based on a mixed attention mechanism for training, and specifically includes: inputting the three-dimensional training data of the positive pulmonary nodules and the three-dimensional training data of the false positive pulmonary nodules into a TSN candidate pulmonary nodule classification network based on a mixed attention mechanism, and training the TSN candidate pulmonary nodules by using a binary cross entropy loss function to obtain a pulmonary nodule false positive exclusion model; wherein the formula of the binary cross entropy loss function is as follows:
Figure BDA0003184102780000151
wherein y isiAs a true tag, piFalse positive lung nodule prediction.
Further preferably, the system for eliminating false positive of pulmonary nodule based on mixed attention mechanism further comprises a preprocessing module for preprocessing the three-dimensional pulmonary CT image to be eliminated.
The invention also discloses a pulmonary nodule false positive elimination device based on the mixed attention mechanism, which comprises a storage device and a processor, wherein the storage device is used for storing one or more programs;
when the one or more programs are executed by the processor, the processor implements a mixed attention mechanism-based lung nodule false positive exclusion method as described above.
The device may also preferably include a communication interface for communicating with external devices and for interactive transmission of data.
It should be noted that the memory may include a high-speed RAM memory, and may also include a nonvolatile memory (nonvolatile memory), such as at least one disk memory.
In a specific implementation, if the memory, the processor and the communication interface are integrated on a chip, the memory, the processor and the communication interface can complete mutual communication through the internal interface. If the memory, the processor and the communication interface are implemented independently, the memory, the processor and the communication interface may be connected to each other through a bus and perform communication with each other.
The present invention also discloses a computer-readable storage medium storing at least one program which, when executed by a processor, implements a method for mixed attention mechanism-based lung nodule false positive exclusion as described above.
It should be appreciated that the computer-readable storage medium is any data storage device that can store data or programs which can thereafter be read by a computer system. Examples of computer-readable storage media include: read-only memory, random access memory, CD-ROM, HDD, DVD, magnetic tape, optical data storage devices, and the like.
The computer readable storage medium can also be distributed over network coupled computer systems so that the computer readable code is stored and executed in a distributed fashion.
Program code embodied on a computer readable storage medium may be transmitted using any appropriate medium, including but not limited to: wireless, wire, fiber optic cable, Radio Frequency (RF), etc., or any suitable combination of the foregoing.
In some embodiments, the computer-readable storage medium may also be non-transitory.
Compared with the prior art, the lung nodule false positive elimination method, the system, the equipment and the computer readable storage medium based on the mixed attention mechanism have the advantages of appropriate parameter quantity, light and simple calculation amount, high model convergence speed and high lung nodule false positive elimination structure precision.
The above-mentioned embodiments only express several embodiments of the present invention, and the description thereof is more specific and detailed, but not construed as limiting the scope of the invention. It should be noted that, for a person skilled in the art, several variations and modifications can be made without departing from the inventive concept, which falls within the scope of the present invention.

Claims (10)

1. A pulmonary nodule false positive elimination method based on a mixed attention mechanism is characterized by comprising the following steps:
obtaining positive pulmonary nodule three-dimensional training data and false positive pulmonary nodule three-dimensional training data;
building a TSN candidate pulmonary nodule classification network based on a mixed attention mechanism;
inputting the three-dimensional training data of the positive pulmonary nodules and the three-dimensional training data of the false positive pulmonary nodules into a TSN candidate pulmonary nodule classification network based on a mixed attention mechanism for training to obtain a pulmonary nodule false positive exclusion model;
obtaining a three-dimensional lung CT image to be excluded, and cutting the three-dimensional lung nodule three-dimensional training data and the three-dimensional lung nodule three-dimensional training data to obtain candidate lung nodule slice data by taking the coordinate positions of the positive lung nodule three-dimensional training data and the false positive lung nodule three-dimensional training data as centers;
and inputting the candidate lung nodule slice data into a lung nodule false positive elimination model, and obtaining a lung nodule false positive elimination result.
2. The method for eliminating the false positive of the pulmonary nodule based on the hybrid attention mechanism according to claim 1, wherein the constructing of the TSN candidate pulmonary nodule classification network based on the hybrid attention mechanism is specifically as follows: and a 2D Resnet 18 backbone network is adopted as a TSN feature extraction network, a residual module in the TSN feature extraction network is improved, a mixed attention module combining motion excitation, coordinate attention, compression and excitation is added, and the TSN candidate pulmonary nodule classification network based on the mixed attention mechanism is obtained.
3. The method for eliminating the false positive of the lung nodule based on the hybrid attention mechanism according to claim 1, wherein the three-dimensional training data of the positive lung nodule and the three-dimensional training data of the false positive lung nodule are input into a TSN candidate lung nodule classification network based on the hybrid attention mechanism for training, specifically: inputting the three-dimensional training data of the positive pulmonary nodules and the three-dimensional training data of the false positive pulmonary nodules into a TSN candidate pulmonary nodule classification network based on a mixed attention mechanism, and training the TSN candidate pulmonary nodules by using a binary cross entropy loss function to obtain a pulmonary nodule false positive exclusion model; wherein the formula of the binary cross entropy loss function is as follows:
Figure FDA0003184102770000011
wherein y isiAs a true tag, piFalse positive lung nodule prediction.
4. The method for mixed attention mechanism-based lung nodule false positive exclusion as claimed in claim 1, wherein the method for mixed attention mechanism-based lung nodule false positive exclusion further comprises the steps of: and preprocessing the three-dimensional lung CT image to be excluded.
5. A pulmonary nodule false positive exclusion system based on a mixed attention mechanism, comprising:
the training data acquisition module is used for acquiring positive pulmonary nodule three-dimensional training data and false positive pulmonary nodule three-dimensional training data;
the classification network building module is used for building a TSN candidate pulmonary nodule classification network based on a mixed attention mechanism;
the model training module is used for inputting the three-dimensional training data of the positive pulmonary nodules and the three-dimensional training data of the false positive pulmonary nodules into a TSN candidate pulmonary nodule classification network based on a mixed attention mechanism for training to obtain a pulmonary nodule false positive exclusion model;
the data to be excluded acquisition module is used for acquiring a three-dimensional lung CT image to be excluded and cutting the three-dimensional lung CT image to obtain candidate lung nodule slice data by taking the coordinate positions of the positive lung nodule three-dimensional training data and the false positive lung nodule three-dimensional training data as centers;
and the result output module is used for inputting the lung nodule candidate slice data into the lung nodule false positive elimination model and obtaining a lung nodule false positive elimination result.
6. The system for eliminating the false positive of the pulmonary nodule based on the hybrid attention mechanism according to claim 5, wherein the building of the classification network building module is specifically as follows: and a 2D Resnet 18 backbone network is adopted as a TSN feature extraction network, a residual module in the TSN feature extraction network is improved, a mixed attention module combining motion excitation, coordinate attention, compression and excitation is added, and the TSN candidate pulmonary nodule classification network based on the mixed attention mechanism is obtained.
7. The system according to claim 5, wherein the model training module inputs the three-dimensional training data of positive lung nodules and the three-dimensional training data of false positive lung nodules into a TSN candidate lung nodule classification network based on the mixed attention mechanism for training, specifically: inputting the three-dimensional training data of the positive pulmonary nodules and the three-dimensional training data of the false positive pulmonary nodules into a TSN candidate pulmonary nodule classification network based on a mixed attention mechanism, and training the TSN candidate pulmonary nodules by using a binary cross entropy loss function to obtain a pulmonary nodule false positive exclusion model; wherein the formula of the binary cross entropy loss function is as follows:
Figure FDA0003184102770000031
wherein y isiAs a true tag, piFalse positive lung nodule prediction.
8. The mixed attention mechanism-based lung nodule false positive exclusion system of claim 5, further comprising a pre-processing module for pre-processing the three-dimensional lung CT image to be excluded.
9. A mixed attention mechanism-based lung nodule false positive exclusion apparatus, comprising a storage device for storing one or more programs and a processor;
the one or more programs, when executed by the processor, implement the mixed attention mechanism-based lung nodule false positive exclusion method of any of claims 1-4.
10. A computer readable storage medium storing at least one program which, when executed by a processor, implements a method for lung nodule false positive exclusion based on a mixed attention mechanism according to any one of claims 1-4.
CN202110855736.7A 2021-07-28 2021-07-28 Pulmonary nodule false positive eliminating method, system and equipment based on mixed attention mechanism Pending CN113570573A (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202110855736.7A CN113570573A (en) 2021-07-28 2021-07-28 Pulmonary nodule false positive eliminating method, system and equipment based on mixed attention mechanism

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202110855736.7A CN113570573A (en) 2021-07-28 2021-07-28 Pulmonary nodule false positive eliminating method, system and equipment based on mixed attention mechanism

Publications (1)

Publication Number Publication Date
CN113570573A true CN113570573A (en) 2021-10-29

Family

ID=78168443

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202110855736.7A Pending CN113570573A (en) 2021-07-28 2021-07-28 Pulmonary nodule false positive eliminating method, system and equipment based on mixed attention mechanism

Country Status (1)

Country Link
CN (1) CN113570573A (en)

Cited By (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN115830020A (en) * 2023-02-14 2023-03-21 成都泰莱生物科技有限公司 Pulmonary nodule feature extraction method, classification method, device and medium
CN116843956A (en) * 2023-06-14 2023-10-03 长江大学 Cervical pathology image abnormal cell identification method, system and storage medium

Citations (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN111539491A (en) * 2020-07-07 2020-08-14 点内（上海）生物科技有限公司 System and method for classifying multiple nodules based on deep learning and attention mechanism
CN112465830A (en) * 2020-11-11 2021-03-09 上海健康医学院 Automatic segmentation method for grinded glass-like pulmonary nodules and computer equipment

Patent Citations (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN111539491A (en) * 2020-07-07 2020-08-14 点内（上海）生物科技有限公司 System and method for classifying multiple nodules based on deep learning and attention mechanism
CN112465830A (en) * 2020-11-11 2021-03-09 上海健康医学院 Automatic segmentation method for grinded glass-like pulmonary nodules and computer equipment

Non-Patent Citations (4)

* Cited by examiner, † Cited by third party
Title
PENG WAN等: "Hierarchical Temporal Attention Network for Thyroid Nodule Recognition Using Dynamic CEUS Imaging", 《IEEE TRANSACTIONS ON MEDICAL IMAGING》 *
QIBIN HOU等: "Coordinate Attention for Efficient Mobile Network Design", 《IEEE/CVF CONFERENCE ON COMPUTER VISION AND PATTERN RECOGNITION, CVPR 2021》, pages 3 *
ZHAN WU等: "MD-NDNet: a multi-dimensional convolutional neural network for false-positive reduction in pulmonary nodule detection", 《PHYSICS IN MEDICINE & BIOLOGY》, pages 2 *
ZHENGWEI WANG等: "ACTION-Net: Multipath Excitation for Action Recognition", 《2021 IEEE/CVF CONFERENCE ON COMPUTER VISION AND PATTERN RECOGNITION (CVPR)》, pages 3 *

Cited By (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN115830020A (en) * 2023-02-14 2023-03-21 成都泰莱生物科技有限公司 Pulmonary nodule feature extraction method, classification method, device and medium
CN115830020B (en) * 2023-02-14 2023-04-28 成都泰莱生物科技有限公司 Lung nodule feature extraction method, classification method, device and medium
CN116843956A (en) * 2023-06-14 2023-10-03 长江大学 Cervical pathology image abnormal cell identification method, system and storage medium

Similar Documents

Publication Publication Date Title
CN108520535B (en) Object classification method based on depth recovery information
CN111415316B (en) Defect data synthesis method based on generation countermeasure network
CN110428428B (en) Image semantic segmentation method, electronic equipment and readable storage medium
CN108629367B (en) Method for enhancing garment attribute identification precision based on deep network
Pang et al. Visual haze removal by a unified generative adversarial network
CN112488210A (en) Three-dimensional point cloud automatic classification method based on graph convolution neural network
WO2018052587A1 (en) Method and system for cell image segmentation using multi-stage convolutional neural networks
CN110210431B (en) Point cloud semantic labeling and optimization-based point cloud classification method
CN111597920B (en) Full convolution single-stage human body example segmentation method in natural scene
CN113762138B (en) Identification method, device, computer equipment and storage medium for fake face pictures
CN112528961B (en) Video analysis method based on Jetson Nano
CN113570573A (en) Pulmonary nodule false positive eliminating method, system and equipment based on mixed attention mechanism
WO2023030182A1 (en) Image generation method and apparatus
CN110532959B (en) Real-time violent behavior detection system based on two-channel three-dimensional convolutional neural network
CN113988147B (en) Multi-label classification method and device for remote sensing image scene based on graph network, and multi-label retrieval method and device
CN109671055B (en) Pulmonary nodule detection method and device
CN114897728A (en) Image enhancement method and device, terminal equipment and storage medium
CN112132207A (en) Target detection neural network construction method based on multi-branch feature mapping
CN114663315B (en) Image bit enhancement method and device for generating countermeasure network based on semantic fusion
CN114842238A (en) Embedded mammary gland ultrasonic image identification method
CN113837062A (en) Classification method and device, storage medium and electronic equipment
CN114764746A (en) Super-resolution method and device for laser radar, electronic device and storage medium
CN111860668A (en) Point cloud identification method of deep convolution network for original 3D point cloud processing
CN112652059B (en) Mesh R-CNN model-based improved target detection and three-dimensional reconstruction method
CN113642429B (en) Marine fish identification method based on TPP-TCCNN

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination