CN115147636A - Lung disease identification and classification method based on chest X-ray image - Google Patents
Lung disease identification and classification method based on chest X-ray image Download PDFInfo
- Publication number
- CN115147636A CN115147636A CN202210171528.XA CN202210171528A CN115147636A CN 115147636 A CN115147636 A CN 115147636A CN 202210171528 A CN202210171528 A CN 202210171528A CN 115147636 A CN115147636 A CN 115147636A
- Authority
- CN
- China
- Prior art keywords
- image
- network
- chest
- classification
- ray image
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 208000019693 Lung disease Diseases 0.000 title claims abstract description 34
- 238000011976 chest X-ray Methods 0.000 title claims abstract description 29
- 238000000034 method Methods 0.000 title claims abstract description 29
- 230000007246 mechanism Effects 0.000 claims abstract description 21
- 238000007781 pre-processing Methods 0.000 claims abstract description 11
- 238000000605 extraction Methods 0.000 claims abstract description 9
- 238000013145 classification model Methods 0.000 claims abstract description 4
- 230000006870 function Effects 0.000 claims description 15
- 238000011160 research Methods 0.000 claims description 12
- 230000002159 abnormal effect Effects 0.000 claims description 10
- 238000013461 design Methods 0.000 claims description 7
- 239000013598 vector Substances 0.000 claims description 7
- 206010035664 Pneumonia Diseases 0.000 claims description 6
- 238000013528 artificial neural network Methods 0.000 claims description 5
- 230000006872 improvement Effects 0.000 claims description 4
- 238000010606 normalization Methods 0.000 claims description 4
- 230000008901 benefit Effects 0.000 claims description 3
- 238000013135 deep learning Methods 0.000 claims description 2
- 238000002372 labelling Methods 0.000 claims description 2
- 230000005012 migration Effects 0.000 claims 2
- 238000013508 migration Methods 0.000 claims 2
- 238000013527 convolutional neural network Methods 0.000 abstract description 10
- 238000005516 engineering process Methods 0.000 abstract description 9
- 238000001228 spectrum Methods 0.000 abstract description 7
- 238000005520 cutting process Methods 0.000 abstract description 3
- 230000003935 attention Effects 0.000 description 26
- 238000010586 diagram Methods 0.000 description 11
- 238000012545 processing Methods 0.000 description 8
- 230000000694 effects Effects 0.000 description 7
- 238000003745 diagnosis Methods 0.000 description 6
- 201000010099 disease Diseases 0.000 description 6
- 208000037265 diseases, disorders, signs and symptoms Diseases 0.000 description 6
- 210000004072 lung Anatomy 0.000 description 5
- 238000004195 computer-aided diagnosis Methods 0.000 description 4
- 238000009826 distribution Methods 0.000 description 4
- 238000012546 transfer Methods 0.000 description 4
- 230000009466 transformation Effects 0.000 description 4
- 206010035737 Pneumonia viral Diseases 0.000 description 3
- 239000002131 composite material Substances 0.000 description 3
- 238000002474 experimental method Methods 0.000 description 3
- 238000013507 mapping Methods 0.000 description 3
- 238000011176 pooling Methods 0.000 description 3
- 208000009421 viral pneumonia Diseases 0.000 description 3
- 201000001178 Bacterial Pneumonia Diseases 0.000 description 2
- 230000008859 change Effects 0.000 description 2
- 238000002059 diagnostic imaging Methods 0.000 description 2
- 230000002708 enhancing effect Effects 0.000 description 2
- 230000005284 excitation Effects 0.000 description 2
- 238000010801 machine learning Methods 0.000 description 2
- 230000004048 modification Effects 0.000 description 2
- 238000012986 modification Methods 0.000 description 2
- 230000008569 process Effects 0.000 description 2
- 230000004083 survival effect Effects 0.000 description 2
- 230000003612 virological effect Effects 0.000 description 2
- 241000287196 Asthenes Species 0.000 description 1
- 238000012935 Averaging Methods 0.000 description 1
- 208000035473 Communicable disease Diseases 0.000 description 1
- 101100481876 Danio rerio pbk gene Proteins 0.000 description 1
- 241000282326 Felis catus Species 0.000 description 1
- 208000032376 Lung infection Diseases 0.000 description 1
- 101100481878 Mus musculus Pbk gene Proteins 0.000 description 1
- 206010056342 Pulmonary mass Diseases 0.000 description 1
- 230000002776 aggregation Effects 0.000 description 1
- 238000004220 aggregation Methods 0.000 description 1
- 238000004458 analytical method Methods 0.000 description 1
- 238000013473 artificial intelligence Methods 0.000 description 1
- 230000001580 bacterial effect Effects 0.000 description 1
- 230000009286 beneficial effect Effects 0.000 description 1
- 238000004364 calculation method Methods 0.000 description 1
- 230000006835 compression Effects 0.000 description 1
- 238000007906 compression Methods 0.000 description 1
- 230000001186 cumulative effect Effects 0.000 description 1
- 238000002405 diagnostic procedure Methods 0.000 description 1
- 239000003814 drug Substances 0.000 description 1
- 230000003203 everyday effect Effects 0.000 description 1
- 239000000284 extract Substances 0.000 description 1
- 239000012530 fluid Substances 0.000 description 1
- 230000002538 fungal effect Effects 0.000 description 1
- 230000036541 health Effects 0.000 description 1
- 238000003709 image segmentation Methods 0.000 description 1
- 230000036039 immunity Effects 0.000 description 1
- 208000015181 infectious disease Diseases 0.000 description 1
- 230000002458 infectious effect Effects 0.000 description 1
- 230000003902 lesion Effects 0.000 description 1
- 231100000915 pathological change Toxicity 0.000 description 1
- 230000036285 pathological change Effects 0.000 description 1
- 230000007170 pathology Effects 0.000 description 1
- 230000004044 response Effects 0.000 description 1
- 230000010332 selective attention Effects 0.000 description 1
- 238000005728 strengthening Methods 0.000 description 1
- 238000012549 training Methods 0.000 description 1
- 230000000007 visual effect Effects 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/08—Learning methods
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T5/00—Image enhancement or restoration
- G06T5/40—Image enhancement or restoration using histogram techniques
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/0002—Inspection of images, e.g. flaw detection
- G06T7/0012—Biomedical image inspection
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/10—Image acquisition modality
- G06T2207/10072—Tomographic images
- G06T2207/10081—Computed x-ray tomography [CT]
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20048—Transform domain processing
- G06T2207/20052—Discrete cosine transform [DCT]
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/30—Subject of image; Context of image processing
- G06T2207/30004—Biomedical image processing
- G06T2207/30061—Lung
- G06T2207/30064—Lung nodule
Landscapes
- Engineering & Computer Science (AREA)
- Physics & Mathematics (AREA)
- Theoretical Computer Science (AREA)
- General Physics & Mathematics (AREA)
- General Health & Medical Sciences (AREA)
- Health & Medical Sciences (AREA)
- Artificial Intelligence (AREA)
- Computational Linguistics (AREA)
- Quality & Reliability (AREA)
- Radiology & Medical Imaging (AREA)
- Nuclear Medicine, Radiotherapy & Molecular Imaging (AREA)
- Life Sciences & Earth Sciences (AREA)
- Medical Informatics (AREA)
- Biomedical Technology (AREA)
- Biophysics (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Data Mining & Analysis (AREA)
- Evolutionary Computation (AREA)
- Molecular Biology (AREA)
- Computing Systems (AREA)
- General Engineering & Computer Science (AREA)
- Mathematical Physics (AREA)
- Software Systems (AREA)
- Image Analysis (AREA)
Abstract
The invention discloses a lung disease identification and classification method based on chest X-ray images, which comprises the following steps: 1) Preprocessing a medical image: changing the gray value range of the original X-ray image from 0-255 to 0-1, balancing to enable the probability density to be 1, performing histogram subdivision according to the brightness value, the tie value and the exposure threshold value, cutting, and further subdividing the lower sub-histogram based on the sub-histogram; 2) Establishing a lung disease identification and classification model of the chest X-ray image: a. adding an FCAnet multi-spectrum channel attention mechanism into a backbone network of the DPN network; b. providing a multi-spectral channel attention mechanism convolution network layer based on a DPN network structure, which is used for solving the problem of information loss of classified objects in an image space; c. the convolutional neural network is designed to be combined with a backhaul network of an FCAnet multi-spectral channel attention mechanism, so that a feature extraction network which is effective for classification targets can be obtained. The invention realizes the remote classification and identification of the chest X-ray image, and improves the efficiency through an improved histogram equalization technology and a multi-spectral channel attention mechanism network, thereby improving the accuracy.
Description
Technical Field
The invention belongs to the technical field of medical image identification, and particularly relates to a lung disease identification and classification method based on a chest X-ray image.
Background
Pneumonia is one of the most common infectious diseases in clinical medicine, and children and old people with short disease period, complex cause and relatively low immunity are more susceptible people. Pneumonia is mainly caused by bacterial, fungal or viral lung infection, and alveoli in infected lungs can be filled with tissue fluid, so that patients feel difficult to breathe and even die. Viral and bacterial pneumonia are infectious, and in particular viral pneumonia is prone to cause regional and global and pandemic epidemics.
Increasingly refined medical images provide a great deal of useful information and play a crucial role in assisting doctors in making accurate diagnoses. However, manual reading is time-consuming and labor-consuming, and the condition of missed examination and misjudgment is often caused by the wide difference of pathology, the potential fatigue of doctors and the deviation of subjective consciousness of different doctors. Massive image data are produced every day, and the trend that a machine reads the images to help the manual work to roughly screen the focus positions and assist in diagnosing diseases is formed. Under the additional hold of an artificial intelligence technology, the machine film reading has the advantages of high speed, high accuracy, high concurrency and the like. Therefore, the lung nodules are positioned by using a Computer Aided Diagnosis (CAD) technology, so that doctors can be better assisted in diagnosing diseases, the survival rate of patients is improved, and the life quality is improved.
Since the research of lung disease identification and classification by using chest X-ray images has important application value, scholars at home and abroad carry out a great deal of research on the problem. Classification of lung diseases in medical imaging is an important area of computer-aided diagnosis. With the wide application of machine learning in bioinformatics, the application of machine learning methods to the diagnosis of lung diseases in medical imaging is the direction in which researchers have been working on research. Pulmonary disease is a serious threat to human health, and it would greatly improve survival rates if patients were diagnosed and treated in a timely manner during critical periods of the disease. Medical images can provide a large amount of useful information data as a broad diagnostic method. However, the ever-increasing amount of image data also presents significant challenges for manual reading. Deviations in subjective awareness from personal experience for different physicians often lead to inefficiencies and even false positives. Therefore, information extraction and processing analysis of medical images has become an important research field in the field of computer-aided diagnosis in recent years. The types of common lung diseases are more and the pathological changes are complex, so that the auxiliary diagnosis of only a single disease type on the chest X-ray image has certain limitation on efficiency. Aiming at the problems, the invention designs a lung disease identification and classification method of a chest X-ray image, which comprises the following specific contents:
1. the system carries out image preprocessing and lung disease prediction through a cloud server. The complete model is stored at the server end to classify the input images. In practical application, a primary hospital uploads an image to be predicted at a terminal, and a server calls a model to process and predict the image.
2. And the cloud server receives the file transmitted by the terminal and transmits the file to the database. And the database receives and stores the image information, and the image is put into the convolutional neural network model after being preprocessed. The model predicts the image, and the prediction result is packaged into a data stream and transmitted back to the terminal.
3. And (3) carrying out lung multiple disease diagnosis based on the convolutional neural network. The system extracts pictures from the database and locates the lesions in the pictures through a pre-trained model. And after the suspected target is found, marking the target on the original image and feeding back the target to the terminal. And after receiving the information, the terminal outputs the prediction graph.
The research of lung disease identification and classification method of chest X-ray image has been greatly advanced, but with the continuous and deep research, some new challenges emerge, specifically as follows:
1. considering the numerous lung diseases, the first challenge is how to design algorithms to enhance image quality to improve the impact on the classification results.
2. Considering that channel attention (e.g., SE attention, ECA attention) has a significant effect on boosting model performance, but they typically ignore position information, which is important for generating spatially selective attention maps, it is important how to design a combination of channel and spatial attention mechanisms.
3. Considering that DPN-SEnet may discard other frequency components containing a large amount of useful information on the feature channel, a third challenge is how to properly design and improve the DPN-SEnet network structure to utilize information on other frequency bands.
Disclosure of Invention
In view of this, in order to solve the above problems in the prior art, the present invention provides a method for identifying and classifying a lung disease by using a chest X-ray image, which can accurately complete auxiliary diagnosis of a lung image to be classified, and has the beneficial effects of strong adaptability and high classification accuracy.
The invention provides a lung disease identification and classification method by using chest X-ray images, which comprises the following steps:
1) Preprocessing a medical image: changing the gray value range of an original X-ray image from 0-255 to 0-1, equalizing to enable the probability density to be 1, then inputting the X-ray image, calculating a histogram of the input image, segmenting the image according to the brightness value, segmenting the segmented image into two sub-histograms according to the average intensity value, wherein the two sub-histograms are respectively a lower histogram and an upper histogram, calculating an exposure threshold value of the lower histogram, and dividing the exposure threshold value into the two sub-histograms according to the exposure threshold value to respectively perform histogram equalization technology processing on the two sub-histograms and the upper histogram to obtain an enhanced image.
2) Establishing a chest X-ray image lung disease identification and classification model:
a. a DCT module is added in a backbone network of the DPN-SEnet, the DCT module is used for carrying out discrete cosine transform processing on the characteristic diagram, a large number of useful frequency components in the characteristic channel are obtained through the characteristic diagram after DCT, and the classification of small sample pneumonia images in the later period is facilitated;
b. a convolution neural network structure layer based on a DPN-FCAnet network structure is provided, and is used for solving the problem of small samples caused by lack of labeling information in a medical image during deep learning, an improved histogram equalization technology is used for enhancing an input image, and a DCT (discrete cosine transform function) is added for improving the network;
c. and a channel attention mechanism is designed to be combined with the DPN network, so that a feature extraction network which is more effective for classifying the small sample medical images can be obtained. The composite network structure is used as a shared feature extraction network of the DPN network. Optionally, based on the chest X-ray image lung disease identification and classification platform, the chest X-ray image lung disease identification and classification platform includes a cloud server, a plurality of edge servers and X-ray instruments, the cloud server is connected to each edge server, and the edge servers are connected to terminal X-ray instruments in various regions.
Compared with the prior art, the invention has the following advantages: the chest X-ray image lung disease identification and classification platform is based on, the chest X-ray image remote classification identification is realized, the image is enhanced by utilizing an improved histogram equalization technology through image preprocessing, and the accuracy is improved through a channel attention mechanism.
Drawings
FIG. 1 is a diagram of the SEnet architecture;
FIG. 2 is a diagram of the structure of FCAnet;
FIG. 3 is a diagram of a DPN network architecture;
fig. 4 is a diagram of a DPN network architecture;
fig. 5 is an overall frame diagram.
Detailed Description
Preferred embodiments of the present invention will be described in detail below with reference to the accompanying drawings, but the present invention is not limited to only these embodiments. The invention is intended to cover alternatives, modifications, equivalents and alternatives which may be included within the spirit and scope of the invention.
In the following description of the preferred embodiments of the present invention, specific details are set forth in order to provide a thorough understanding of the present invention, and it will be apparent to those skilled in the art that the present invention may be practiced without these specific details.
The invention discloses a chest X-ray image lung disease identification and classification method, which is characterized by comprising the following steps: the method comprises the following steps:
1) Preprocessing a medical image: changing the gray value range of an original X-ray image from 0-255 to 0-1, equalizing to enable the probability density to be 1, then inputting the X-ray image, calculating a histogram of the input image, segmenting the image according to the brightness value, segmenting the segmented image into two sub-histograms according to the average intensity value, wherein the two sub-histograms are respectively a lower histogram and an upper histogram, calculating an exposure threshold value of the lower histogram, and dividing the exposure threshold value into the two sub-histograms according to the exposure threshold value to respectively perform histogram equalization technology processing on the two sub-histograms and the upper histogram to obtain an enhanced image.
2) DPN-FCAnet: a DCT module is added in a backbone network of the DPN-SEnet, the DCT module is used for carrying out discrete cosine transform processing on the characteristic diagram, and a large number of useful frequency components in a characteristic channel are obtained through the characteristic diagram after DCT, so that the classification of the small sample pneumonia images in the later period is facilitated;
3) The proposed FCAnet multi-spectrum attention mechanism aims to improve the accuracy by improving the attention mechanism network layer under the condition that the width, the depth and the space of a network model are simultaneously considered under the condition that the network calculation burden is not increased.
The invention applies the DPN-FCAnet network to the lung disease classification of medical images so as to improve the medical image diagnosis efficiency of doctors. Aiming at the problem that a DPNnet network is composed of a large number of convolution layers, and a convolution kernel is used as a core of a convolution neural network, feature mapping can be generally obtained only from a local receptive field, but features of a global receptive field are lacked, a multi-spectral channel attention neural network is provided. The DPN-SEnet network and the cosine discrete transformation function are combined to be used as a backbone network of the DPN network.
Based on chest X-ray image lung disease discernment and classification platform, chest X-ray image lung disease discernment and classification platform include high in the clouds server, a plurality of edge server and X-ray instrument, every edge server is connected to high in the clouds server, edge server is connected with the terminal X-ray instrument in each district.
Inputting an X-ray image to be classified to a cloud server, and performing feature extraction by the cloud server through the model; the cloud server compares the extracted features with the existing feature set; if the abnormal case is judged to be an abnormal case through comparison, the abnormal features and the abnormal sites are transmitted back to the edge server, the cloud server records the abnormal case and analyzes the case features, and the initial feature set is updated accordingly to achieve improvement of the identification precision of the X-ray image.
In step 1), a modified histogram equalization technique is employed to enhance the image to increase the contrast of the image. The gray value of each image is changed from 0-255 to 0-1, and the probability density is set to 1 by using equalization, so that the gray value of each image can be concentrated in a narrower interval, and the contrast map can be adjusted by using histogram equalization, so that the image details are clearer. The traditional histogram equalization may cause a series of problems such as information loss, excessive image enhancement, or unnatural artifacts, which cause the contour of the medical lung image to be blurred and difficult to distinguish. The image segmentation according to the brightness can effectively avoid the problems. The contrast of the gray level image is improved through an improved histogram equalization technology, and compared with the nodules of the original image, the gray level image is clearer and is convenient for later-stage classification.
In step 2), since the DPNnet network is composed of a large number of convolutional layers, and the convolutional kernel is used as the core of the convolutional neural network, the feature map can be generally obtained only from the local perceptual field, but the feature of the global perceptual field is lacking, and a multi-spectral channel attention neural network is provided. The DPN-SEnet network and the cosine discrete transformation function are combined to serve as a backbone network of the DPN network, so that the effect of a global receptive field can be fully achieved, and frequency components containing a large amount of useful information on a characteristic channel are utilized, so that the accuracy of the network is improved.
More specific methods and procedures of the present invention are further described below:
preprocessing a medical image:
the standard file format of the image and its related information, such file includes many raw data information such as image resolution, age and sex of the patient, etc. in addition to the X-ray image. In order to make the image data better applied in the subsequent steps, the image preprocessing part of the system mainly works as follows: changing the gray value range of the original X-ray image from 0-255 to 0-1, equalizing to enable the probability density to be 1, carrying out histogram subdivision according to the tie value and the exposure threshold value, cutting, and further subdividing the lower sub-histogram based on the sub-histogram.
(1) Converting the gray value range of the original X-ray image from 0-255 to 0-1
The histogram equalization is to make the distribution on the 0-255 gray scale more balanced by stretching the distribution range of the pixel intensity, improve the contrast of the image and achieve the purpose of improving the subjective visual effect of the image. Images with low contrast are suitable for enhancing image details using histogram equalization methods. Then inputting an original X-ray image, calculating a histogram H of the input X-ray image according to a formula, carrying out histogram normalization, wherein the sum of group distances of the histogram is 255, and calculating a histogram integral: h' (i) = ∑ E 0≤j≤i H (j), using H' as a lookup table to perform image transformation: dst (x, y) = H' (src (x, y)), each gray level of the histogram is normalized, the cumulative distribution of each gray level is obtained, a mapped gray mapping table is obtained, and then each pixel in the original image is corrected according to the corresponding gray value.
(2) By cutting
In order to eliminate the problem of excessive enhancement of histogram equalization and to control the enhancement rate, a clipping technique is used. The technique is used to clip the original histogram h (k) and form a new histogram h c (k) In that respect In this technique, a formula is used
The new cut histogram is then represented as
Then using function to calculate the gray level average value X of X-ray image m 。
(3) Dividing the clipped histogram into two sub-histograms according to the average intensity value Xm
According to the mean value X of the gray levels m The histogram is decomposed into two sub-histograms. Where the lower sub-histogram contains intensity values, the highest average intensity and upper sub-histogram with intensity values again uses the exposure threshold X between the average and maximum gray levels e The lower sub-histogram is split into two sub-histograms. Thus, two kinds of sub-histograms, an extremely low exposure sub-histogram and a low exposure sub-histogram, are formed. Finally, these histograms are subdivided into three sub-images, sub-image-1, sub-image-2 and sub-image-3, and then the probability density functions of sub-image 1, sub-image 2 and sub-image 3 are calculated using equations (4), (5) and (6), respectively
Variable N Ll 、N Lu And N U Representing the number of pixels in sub-image-1, sub-image-2, and sub-image-3, respectively.
The transfer function (mapping function) of the individual sub-images is determined with (7-9). Transfer function of subimage-1:
transfer function of subimage-2:
transfer function of subimage-3: f u ＝(X m +1)+(l-(X m +1))C U
These three sub-images are then integrated to obtain the complete image: y (s, t) = F Ll ∪F Lu ∪F U
Establishing a lung disease identification and classification model of the chest X-ray image:
the research work applies the DPN network to the lung disease classification of medical images to improve the medical image diagnosis efficiency of doctors. Aiming at the phenomenon that a DPN network lacks the characteristics of a global receptive field, a multi-spectral-channel attention mechanism is provided. The proposed multi-spectral channel attention mechanism is combined with a DPN network to serve as a backhaul network of the DPN-FCAnet network. Experiments show that the improved DPN-FCAnet improves the classification effect of small samples and improves the overall classification precision.
(1) Multi-frequency spectrum channel attention mechanism
DPN-92 is composed of a large number of convolutional layers, and the convolutional kernel, which is the core of the convolutional neural network, can generally only obtain feature maps from local perceptual fields, lacks the features of global perceptual fields, and thus, the attention mechanism of Senet is added, and Senet only uses GAP global mean pooling, which results in discarding other frequency components containing a large amount of useful information on feature channels. The learning of the weight of the channel attention is shown in the following formula, which represents that the input is learned by the full connection layer after GAP processing and is activated by Sigmoid to obtain a weighted mask. att = sigmoid (fc (gap (X))). Then, the mask and the original input are multiplied channel by the following formula to obtain the output after attention operation. X to: i,: ,: = atti X: i,: ,: s.t.i. epsilon {0,1, \8230;, C-1}
Discrete cosine change: the basic form of DCT is as follows, f ∈ R L For the frequency spectrum of DCT, x ∈ R L For input, L is inputLength.
s.t.k.e {0,1. Further, we generalize to get a two-dimensional DCT, f 2d ∈R H×W Is the frequency spectrum of a two-dimensional DCT, x 2d ∈R H×W Is the input, and H and W are the height and width of the input.
s.t.w∈{0，1.........W-1}
s.t.h∈{0，1.........H-1}
Also, the formula of the inverse DCT transform is as follows.
In the above two equations, some constant normalization constraint factors are removed for simplicity. DCT transform belongs to the knowledge in the field of signal processing, is a core algorithm of JPEG image compression, and is equivalent to the aggregation of important information. It can be seen from this that the DCT transform is also a weighted sum of the inputs, and the cosine component in the equation is the weight. Therefore, GAP, the averaging operation, can be considered as the simplest spectrum of input, which is obviously informative and therefore draws attention to the following multi-spectral channel. It is first demonstrated here that GAP is actually a special case of two-dimensional DCT, and the result is proportional to the lowest component of the two-dimensional DCT. This proof is obtained by letting h and w both be 0, where
The two-dimensional DCT lowest frequency component is represented, and it is apparent that it is directly proportional to GAP as a result.
From the above conclusions, it is naturally conceivable to introduce other components into the channel attention, first, for descriptive convenience, two-dimensionalThe basic function of DCT is described as
The inverse two-dimensional DCT transform is then rewritten as follows:
it is not difficult to find from this equation that the previous channel attention only applies the lowest frequency component part of the first term, and the following other parts expressed by the following formula are not used, and all the information is ignored.
U, v in the above formula]Denotes the component index of 2DDCT, which uses different frequency components for each block, so that the final output Freq ∈ R is obtained by the following formula c The multispectral vector is obtained, and then the vector is sent to a full connection layer commonly used by the attention of the channel for learning to obtain the attention map. Freq = cat ([ Freq: [) 0 ，Freq 1 ，...，Freq n-1 ]) ms_att＝sigmoid([fc(Freq)]). How to select [ u, v ]]A heuristic two-step rule is designed to select the frequency components of the multispectral attention module, and the main idea is thatThe effect of first obtaining the importance of each frequency component and then determining the different number of frequency components is obtained. Specifically, the results of using each frequency component in the channel attention are calculated respectively, and then, the component with the best topk performance is selected according to the results.
In the experiment, a multi-spectral channel attention mechanism is added into a backbone network of DPN-92, an improved histogram equalization technology is used for inputting images for enhancement, the feature information of the small sample classifier is enhanced through the enhanced feature map, and the later-stage small sample classification is facilitated.
(3) Senet network structure and FCAnet network structure
The convolution kernel, which is the core of the CNN, generally aggregates spatial (spatial) information and feature-dimension (channel-wise) information on a local receptive field to obtain global information. The convolutional neural network is composed of a series of convolutional layers, nonlinear layers and downsampling layers, so that they can capture the characteristics of an image from a global receptive field for image description, however, it is quite difficult to learn a network with very strong performance, and the core building block of the Convolutional Neural Network (CNN) is a convolution operator, which enables the network to build information characteristics by fusing spatial and channel information in each layer of local receptive field. Starting from the relationships between feature channels, the interdependencies between feature channels are explicitly modeled. Specifically, the importance degree of each feature channel is automatically acquired in a learning mode, then useful features are enhanced according to the importance degree, and features which are not useful for the current task are suppressed. Fig. 1 shows a SE block, which contains two parts, namely, squeeze and Excitation, and then, the Figure is combined with a formula to explain fig. 1, and a feature graph with height H, width W and channel number C is firstly subjected to GAP global average pooling operation, wherein the formula is as follows:
squeeze turns each two-dimensional feature channel into a real number, whichThe real numbers have a somewhat global receptive field that characterizes the global distribution of responses over the feature channels. The specification operation follows, as shown in equation 3: s = F cx (z，W)＝σ(g(z，W))＝σ(W 2 σ(W 1 Z)), where Z is multiplied by W1, i.e. a fully connected layer operation, W1 has the dimension C/r C, which is a scaling parameter, here 16, and this parameter is used to reduce the number of channels and thus the amount of computation. Then passing through a ReLU layer; then multiplying with W2, wherein the multiplying with W2 is a process of fully connecting layers, the dimension of W2 is C x C/r, and therefore the dimension of output is 11C; and finally obtaining s through a sigmoid function. Finally, we regard the weight of the output of the Excitation as the importance of each feature channel, and then weight the previous feature channel by channel through multiplication to complete the recalibration of the original feature in the channel dimension, as shown in the public: x' c ＝F scalc (u c ，s c )＝s c ×u c ，
As shown in fig. 2, the FCAnet network structure used this time is obtained by first performing split operation on a feature map with a height H, a width W and a channel number C, dividing the feature map into n blocks, in this experiment n =16, selecting a corresponding frequency component for each feature map, performing DCT cosine discrete change processing on each feature map from the lowest frequency to the highest frequency, adding together each processed feature map, and then performing scale operation to obtain a brand new feature map.
The research work provides an inclusion structure-based transposed convolution network layer, which is used for solving the problem of information loss of small sample classifiers in an image space and restoring the feature information of small samples under the condition of enlarging the size of a feature map.
(4) DPN network architecture
The research work mainly designs a DPN network structure, and achieves the effect of reserving and strengthening small sample characteristic information. According to the fcantet structure, a DPN network structure (DPN-FCAnet for short) with a spectrum channel attention mechanism is designed, and the structure diagram is shown in fig. 3.
In fig. 3, DPN is actually the union of resenxt and DenseNet, and in fig. 3, left is DenseNet and right is rennext (appearing more like ResNet, but actually operated through group). In fig. 3, the left and right sides are added, 3 × 3 convolution is performed, 1 × 1 dimensional transformation is performed, channel splitting is performed, the left side is combined with the left original input, and the right side is added with the right original input, so that a block is formed. (the original input may be the input that initially entered the network, or the input of the previous stage)
The right graph 4 shows DPN-92, which is first convolved by 7 × 7, then maximally pooled by 3 × 3, and then entered into the left graph, where [ ] represents a block, where × 3 represents the block of 3 such patterns in a loop, G represents how many paths (i.e., the number of groups) are divided in a block of rennext, and +16 represents the number of channels per increment in a block of densnet (i.e., the width of each increment as the left vertical bar in the left graph goes down). And performing multi-classification through conv3, conv4, conv5 and finally softmax.
The research works in a DPN network-based structure, and some improvements are made to improve the characteristic restoration effect of small sample classifiers in a convolution output characteristic diagram and improve the convergence speed and the network precision of training.
The FCAnet designed by the research work is combined with a backhaul network of a DPN network, so that a feature extraction network which is more effective for classifying small samples can be obtained. The composite network structure is used as a shared feature extraction network of the DPN network, and the precision and the small sample classification are improved to a certain extent.
(2) Improved DPN network
The present work is for DPN-FCAnet networks that combine the proposed FCAnet network with DPN networks.
The backhaul network in the DPN-FCAnet network is formed by combining the FCAnet and the DPN network, and the network is used for extracting the characteristics of the image, so that the semantic characteristics of the image can be obtained, and the position information of the small sample classifier in the image space can be reserved. The composite backhaul network can improve the classification precision and the small sample classification effect under the condition of smaller parameter quantity.
As shown in fig. 5, the original X-ray image is first enhanced by the improved histogram equalization technique and then input into the DPN-fcantet network, then passes through the CNN convolutional layer, the posing pooling layer, the FC full link layer, and finally is divided into four categories by the softmax function, namely new coronary pneumonia, normal lung, bacterial pneumonia, and viral pneumonia.
The above-described embodiments do not limit the scope of the present invention. Any modification, equivalent replacement, and improvement made within the spirit and principle of the above-described embodiments should be included in the protection scope of the technical solution.
Claims (5)
1. A lung disease identification and classification method based on chest X-ray images is characterized in that: the method comprises the following steps:
1) Preprocessing a medical image: in data preprocessing, we use clipping, rotation, flipping, and normalization techniques. First, we modify the image size to (224 × 224), rotate randomly selected images, flip the images horizontally, and normalize the values for each dimension in the data to the same range.
2) Establishing a chest X-ray image lung disease identification and classification model:
a. adding ESCA (efficient spatial channel attention mechanism) into a backbone network of ResNet-101, carrying out emphasis extraction on a feature map by using an ESCA module, extracting important features, obtaining a large amount of useful frequency and position component information in the feature channel through the feature map after ESCA, and facilitating the classification of small sample pneumonia images in the later stage;
b. a migration learning network structure based on an efficient spatial channel attention mechanism is provided, and is used for solving the problem of small samples caused by lack of labeling information in medical images in deep learning. Then, we connect the feature vectors extracted from each of the pre-trained models-ResNet 152, denseNet121, and ResNet 18-to the same dimensional vector. Then, the feature vectors extracted from each model are combined and connected into the same dimensional vector. We can use the advantages of each model as a feature extractor; then, the ESCA attention mechanism module is applied. Note that the output vector of the operation is passed to the classifier (last fully connected layer).
c. A novel network Attention mechanism is provided by fusing a space Channel Attention mechanism with an ECAnet network, and is called as 'ESCA-Net' effective Spatial Channel Attention for Deep conditional Neural Networks "
d. And the LOSS LOSS function is designed by the network structure, so that the classification precision of the network is greatly improved.
2. The chest X-ray image based lung disease identification and classification method of claim 1 wherein: based on chest X-ray image lung disease discernment and classification platform, chest X-ray image lung disease discernment and classification platform include high in the clouds server, a plurality of edge server and X-ray instrument, every edge server is connected to high in the clouds server, edge server is connected with the terminal X-ray instrument in each district.
3. The chest X-ray image based lung disease identification and classification method according to claim 1 or 2, wherein: inputting an X-ray enhanced image to be classified to a cloud server, and performing feature extraction by the cloud server through the model of the invention; the cloud server compares the extracted features with the existing feature set; if the abnormal case is judged to be an abnormal case through comparison, the abnormal features and the abnormal sites are transmitted back to the edge server, the cloud server records the abnormal case, the case features are analyzed, and the initial feature set is updated accordingly to achieve improvement of the recognition accuracy of the X-ray image.
4. The chest X-ray image based lung disease identification and classification method of claim 1 wherein: in step 1), the standard file format of the image and its related information, such file includes many raw data information such as image resolution, age and sex of the patient, etc. in addition to the X-ray image. In order to make the image data better applied in the following steps, the image preprocessing part of the system mainly works as follows: in data preprocessing, we use clipping, rotation, flipping, and normalization techniques. First, we modify the image size to (224 × 224), rotate randomly selected images, flip the images horizontally, and normalize the values for each dimension in the data to the same range.
5. The chest X-ray image based lung disease identification and classification method of claim 1 wherein: in step 2), the research works design a migration learning network structure based on an efficient spatial channel attention mechanism, and design an LOSS LOSS function, so that the precision and the classification of small sample images are improved to a certain extent.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210171528.XA CN115147636A (en) | 2022-02-24 | 2022-02-24 | Lung disease identification and classification method based on chest X-ray image |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210171528.XA CN115147636A (en) | 2022-02-24 | 2022-02-24 | Lung disease identification and classification method based on chest X-ray image |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN115147636A true CN115147636A (en) | 2022-10-04 |
Family
ID=83405547
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202210171528.XA Pending CN115147636A (en) | 2022-02-24 | 2022-02-24 | Lung disease identification and classification method based on chest X-ray image |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN115147636A (en) |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN116563848A (en) * | 2023-07-12 | 2023-08-08 | 北京大学 | Abnormal cell identification method, device, equipment and storage medium |
| CN117974465A (en) * | 2024-03-31 | 2024-05-03 | 天津市胸科医院 | Chest CT image processing method and system |
-
2022
- 2022-02-24 CN CN202210171528.XA patent/CN115147636A/en active Pending
Cited By (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN116563848A (en) * | 2023-07-12 | 2023-08-08 | 北京大学 | Abnormal cell identification method, device, equipment and storage medium |
| CN116563848B (en) * | 2023-07-12 | 2023-11-10 | 北京大学 | Abnormal cell identification method, device, equipment and storage medium |
| CN117974465A (en) * | 2024-03-31 | 2024-05-03 | 天津市胸科医院 | Chest CT image processing method and system |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN110599448B (en) | Migratory learning lung lesion tissue detection system based on MaskScoring R-CNN network | |
| CN110930416B (en) | MRI image prostate segmentation method based on U-shaped network | |
| CN111429421B (en) | Model generation method, medical image segmentation method, device, equipment and medium | |
| CN111882560B (en) | Lung parenchyma CT image segmentation method based on weighted full convolution neural network | |
| CN115147636A (en) | Lung disease identification and classification method based on chest X-ray image | |
| CN111583285A (en) | Liver image semantic segmentation method based on edge attention strategy | |
| CN115082743B (en) | Full-field digital pathological image classification system considering tumor microenvironment and construction method | |
| CN114897914A (en) | Semi-supervised CT image segmentation method based on confrontation training | |
| Li et al. | Natural tongue physique identification using hybrid deep learning methods | |
| Prusty et al. | A novel transfer learning technique for detecting breast cancer mammograms using VGG16 bottleneck feature | |
| CN113782184A (en) | Cerebral apoplexy auxiliary evaluation system based on facial key point and feature pre-learning | |
| CN117975170B (en) | Medical information processing method and system based on big data | |
| CN114359637A (en) | Brain medical image classification method and device | |
| CN112419396A (en) | Thyroid ultrasonic video automatic analysis method and system | |
| CN112036298A (en) | Cell detection method based on double-segment block convolutional neural network | |
| Rad et al. | Dental x-ray image segmentation and multiple feature extraction | |
| CN114565786A (en) | Tomography image classification device and method based on channel attention mechanism | |
| Chen et al. | Detection of various dental conditions on dental panoramic radiography using Faster R-CNN | |
| CN113643297A (en) | Computer-aided age analysis method based on neural network | |
| CN111667469B (en) | Lung disease classification method, device and equipment | |
| CN112967254A (en) | Lung disease identification and detection method based on chest CT image | |
| Pavithra et al. | An Overview of Convolutional Neural Network Architecture and Its Variants in Medical Diagnostics of Cancer and Covid-19 | |
| CN116630964A (en) | Food image segmentation method based on discrete wavelet attention network | |
| CN115761226A (en) | Oral cavity image segmentation identification method and device, electronic equipment and storage medium | |
| CN114359308A (en) | Aortic dissection method based on edge response and nonlinear loss |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication |