CN111110269A - Infant pneumonia rapid intelligent diagnosis method based on mixed deep learning model - Google Patents

Abstract

The invention discloses a rapid and intelligent infant pneumonia diagnosis method based on a mixed deep learning model, which solves the problem that the body of an infant is damaged due to overlong infant pneumonia diagnosis time or misdiagnosis at present. The method comprises the following steps: the method comprises the following steps: measuring and collecting various physiological index data of the infants, and marking the data according to the fact whether the infants suffer from pneumonia or not; step two: performing data cleaning and abnormal data elimination on the non-respiratory audio data to construct a data set for training a pneumonia diagnosis model; step three: constructing a data set for training a spiral voice recognition model; step four: training a long-time memory neural network LSTM; step five: and (4) carrying out deep neural network DNN training for interpreting various physiological index data of the patient so as to judge whether the patient suffers from pneumonia. The invention can effectively improve the diagnosis speed and the correct rate of infant pneumonia, and avoid serious injury to the infant body caused by overlong diagnosis waiting time or misdiagnosis.

Description

Infant pneumonia rapid intelligent diagnosis method based on mixed deep learning model

Technical Field

The invention belongs to the field of intelligent medical treatment, and relates to a rapid intelligent diagnosis method for infant pneumonia based on a mixed deep learning model.

Background

Pneumonia refers to inflammation of the terminal airways, alveoli and lung interstitium, and may be caused by disease microorganisms, physicochemical factors, immune injury, allergy and drugs. Bacterial pneumonia is the most common pneumonia and is also one of the most common infectious diseases. Pneumonia in daily life is mainly pneumonia caused by bacterial infection. Before the application of antibiotics, bacterial pneumonia threatens human health greatly, and the occurrence and development of antibiotics obviously reduce the death rate of pneumonia. In daily life, pneumonia is a common disease of infants, and is high in the first place among hospitalized infants, and is also one of diseases causing death of infants. The symptoms of pneumonia in infants can easily be mixed with colds. Therefore, the rapid intelligent diagnosis technology has a vital role in finding the state of an illness and treating the illness according to the illness. Avoiding the serious damage to the body of the infant caused by the overlong time for waiting for diagnosis or misdiagnosis when going to a hospital for a doctor.

Disclosure of Invention

In addition, the infant is ill and cannot be expressed by the speech of the infant, the body is damaged by the pneumonia, misdiagnosis is easy to occur, and therefore rapid diagnosis is needed. The invention provides a rapid and intelligent infant pneumonia diagnosis method based on a mixed deep learning model.

In order to achieve the purpose, the technical scheme adopted by the invention is as follows:

a rapid intelligent diagnosis method for infant pneumonia based on a hybrid deep learning model comprises the following steps:

the method comprises the following steps: measuring and collecting various physiological indexes of the infant, including respiratory audio data, body temperature, heart rate and respiratory frequency;

step two: cleaning the obtained infant non-audio physiological data, removing abnormal data, marking the data, and constructing an infant pneumonia diagnosis physiological index data set, wherein the data set comprises a training set and a testing set and is used for training a Deep Neural Network (DNN);

step three: analyzing the respiratory audio data by utilizing Fourier transform, extracting corresponding characteristic information, and constructing a data set about the spiral sound identification, wherein the data set also comprises a training set and a testing set and is used for training a long-time memory network LSTM;

step four: training a long-time memory neural network LSTM by using the frequency domain characteristics of the respiratory audio and the labels thereof, and judging the spiroid of the respiratory audio data;

step five: and establishing and training a deep neural network DNN for identifying pneumonia by using a spiral voice identification result of the breathing audio data and physiological indexes of body temperature, heart rate and breathing frequency, and interpreting new input data so as to judge whether the patient has pneumonia.

Further, the first step specifically comprises: according to local conditions, various physiological indexes of infant pneumonia patients and normal infants are collected, and the method comprises the following steps: heart rate, body temperature, respiratory rate, respiratory audio data, and then labeling and storing the data by the category of whether or not pneumonia is present.

Further, the second step is specifically: processing data except respiratory audio by using filtering, linear transformation and standardization technologies, and eliminating data groups corresponding to abnormal data and invalid data for deep neural network DNN training;

assuming that there are N detection samples and M characteristic parameters, a matrix from which the original data can be obtained is:

after standardization treatment, each index is obtained:

in the formula (I), the compound is shown in the specification,

and σ_jMean and standard deviation of the jth characteristic parameter:

further, analyzing the breathing audio data in the third step and extracting corresponding characteristic information specifically include:

for respiratory audio data, feature Frequency domain feature extraction (using Mel-Frequency Cepstrum coefficients) is performed using fourier transform, and in the field of sound processing, Mel-Frequency Cepstrum (Mel-Frequency Cepstrum) is a linear transform of a nonlinear Mel scale (Mel scale) logarithmic energy spectrum based on sound Frequency. The Mel-frequency cepstral Coefficient (MFCC) is a Coefficient constituting the Mel-frequency cepstral. For training of long and short term memory networks LSTM, the solution of MFCC typically comprises the following steps:

a. decomposing a voice signal into a plurality of frames;

b. pre-emphasizing the voice signal, and passing the voice signal through a high-pass filter;

c. performing fourier transform to transform the signal into the frequency domain, namely:

d. passing the obtained frequency spectrum of each frame through a Mel filter (triangular overlapping window) to obtain Mel scale;

e. extracting logarithmic energy on each mel scale;

f. the result obtained above is subjected to inverse discrete fourier transform and transformed into the cepstrum domain, that is:

MFCC is the magnitude (amplitudes) of this cepstral plot. Typically 12 coefficients are used, superimposed with the frame energy to obtain 13-dimensional coefficients.

Further, the fourth step is specifically:

training a long-time memory neural network LSTM by using the frequency domain characteristics of the respiratory audio and the labels thereof to obtain a first part of a model, namely a part for recognizing the spiral sound;

the LSTM has three gate structures, namely an input gate, an output gate and a forgetting gate;

(4) forget the door:

let the network "forget" the information that was not useful in the previous step, it will depend on the current input x^(t)State of the last moment c^(t-1)Output h of the previous moment^(t-1)The input part of the forgetting gate mainly comprises a hidden state h at the previous moment^(t-1)And input data x of the current time^(t)It uses an activation function sigma, generally a sigmoid function, to ensure the output of the forgetting gate is [0,1 ] due to the characteristics of the sigmoid function]Output f^(t)Comprises the following steps:

f^(t)＝σ(W_fh^(t-1)+U_fx^(t)+b_f) (6)

wherein W_f,U_f,b_fRespectively, the cyclic weight, the input weight and the offset of the forgetting gate;

(5) an input gate:

after the neural network 'forgets' the information of the previous state, supplementing new memory; according toCurrent input x^(t)State of the last moment c^(t-1)Output h of the previous moment^(t-1)Jointly determining useful information into a new state c^(t),And stored in memory cells; the output of which comprises two parts, i^(t)And a^(t)：

i^(t)＝σ(W_ih^(t-1)+U_ix^(t)+b_i) (7)

a^(t)＝tanh(W_ah^(t-1)+U_ax^(t)+b_a) (8)

Wherein, W_i,W_aFor different connection weights, U, of the previous state_i，U_aFor different connection weights of the inputs, i_b，a_bRespectively, a respective bias portion.

(6) An output gate:

using status cells c^(t)Determining how much information is output to hidden layer state unit h^(t)When the user passes through the forgetting gate and the input gate, the results of the two gates act on the state unit c^(t)，c^(t)The mathematical expression implemented is:

c^(t)＝i^(t)Θa^(t)+f_tΘc^(t)(9)

wherein i^(t)Θa^(t)For remaining in state unit c after passing through input gate^(t)Information of (1), f_tΘc^(t-1)To remain at c after passing through a forgetting door^(t)The information in (1). At the same time, the layer state h is hidden^(t)Updated mainly by two parts, x^(t)、h^(t-1)Result of the combined action o^(t)And status unit c^(t)To determine o^(t)And h^(t)The formula implemented is:

o^(t)＝σ(W_oh^(t-1)+U_ox^(t)+b_o) (10)

h^(t)＝o^(t)Θtanh(c^(t)) (11)

wherein U is_o，W_o，b_oRespectively inputting x for the current time^(t)Output gate weight, last hidden state unit h^(t-1)Round robin weight and output gate offset;

and (3) processing the respiratory audio signal according to the flow, inputting the respiratory audio signal into an LSTM network, if the output result is 1, judging that the respiratory audio contains spiral sound, and if the output result is 0, judging that the respiratory audio does not contain spiral sound.

Further, the fifth step is specifically:

inputting data including body temperature, heart rate, respiratory rate and whether the respiratory audio data contain spiral sound (1 or 0) into a deep neural network DNN for training to obtain a second part of the model, namely a pneumonia diagnosis part;

(1) initialization

The convergence of the network is greatly influenced by the setting of the initial value of the connection weight (including a threshold value); the net input value of each neuron is small due to the expected initial value so as to ensure that the neuron works in an activation region, namely a region with sensitive change, and the initial value of each connection weight is a random number;

(2) forward pass process computation

(a) Input learning sample pair (X)_q,T_q) Wherein Q is 1,2, … Q;

(b) for each input sample, sequentially calculating the output value of each neuron according to the following formula;

in the formula:

output of each hidden layer of DNN, w_ji，w_kj，w_lkThe weight coefficients of each hidden layer are respectively, and because the signals in the neural network are transmitted layer by layer, the output of the previous hidden layer in the neural network is used as the input of the next hidden layer, so that

Is the output of the first hidden layer, is also the input of the second hidden layer,

is the output of the second hidden layer and is also the input of the third hidden layer;

(3) error back propagation

Calculating equivalent errors of neurons of each layer from the output layer to the input layer, then returning to the step (2), and performing forward propagation calculation and error backward propagation on other learning sample pairs until all α learning sample pairs are similarly calculated;

(4) adjusting connection weight between layers

And modifying the connection weight of each layer according to the formula (13), namely:

in the formula: w is a_ik(n+1)，w_kj(n+1)，w_ji(n +1) are the weight coefficients of the hidden layers after the back propagation update by the gradient method, w_ik(n)，w_kj(n)，w_ji(n) the current weight coefficients of the hidden layers, η represents the step size of the gradient method,

respectively representing gradient information of each hidden layer after being propagated through the neural network;

(5) returning to the step (2), performing forward calculation according to the new connection weight, and if each learning sample pair meets the precision requirement, that is:

wherein, p is 1,2, … …, α, l is 1,2, … …, m, (α, m respectively represents the number of the activation units of each hidden layer and the number of the hidden layers of the neuron), epsilon is a certain value given according to the precision requirement, when the above formula is satisfied, the learning of the network is completed, otherwise, the calculation is repeated until the requirement is satisfied.

The calculation process of the Deep Neural Network (DNN) is actually an updating process of the weight and the threshold of the network, and the weight and the threshold which enable the overall error to be minimum are found;

the final diagnostic model (classification recognition model) is:

in the formula: f. of₁，f₂，f₃Respectively representing the activation functions of all the hidden layers, and other symbols are described in the description;

inputting the output result (1 or 0) of the LSTM network and physiological indexes of body temperature, heart rate and respiratory rate into the deep neural network, if the output is 1, judging that the patient has pneumonia, and if the output is 0, judging that the patient does not have pneumonia.

The working principle of the invention is as follows:

pulmonary rale (whorl) refers to auscultation of breath sounds, when air passes through the trachea containing secretions, or the bronchi narrowed due to spasm or swelling, an additional breath mure, i.e., rale, is heard on the basis of the breath sounds. According to the nature of the pitch, the pitch can be classified into dry pitch and wet pitch. Dry rale is due to the narrowing or incomplete blockage of the trachea, bronchi or bronchioles, and turbulence in the airflow during inhalation or exhalation. The pathological basis is inflammation-induced mucosal congestion edema, secretion increase and bronchial smooth muscle spasm; intraluminal foreign matter, tumor obstruction, and swollen lymph or mediastinum compress the airways. Localized moist rale in the lung only suggests local lesions in the lung, such as pneumonia, pulmonary tuberculosis or bronchiectasis; moist and crackle at both sides are mostly seen in blood stasis of lung and bronchopneumonia caused by heart failure; for example, the wet-swollen and rale-swollen lung fields are often seen in acute pulmonary edema or severe bronchopneumonia. Therefore, pulmonary rale is an important index in the process of pneumonia diagnosis.

In order to integrate the typical indexes of rale and rale, a plurality of physiological indexes of body temperature, heart rate, respiratory rate and the like when the infant suffers from pneumonia, a classification recognition model which is established and trained by the body temperature, heart rate and respiratory rate physiological indexes including whether the infant contains spiral sound or not is established by adopting a deep neural network, data is interpreted, and whether the patient suffers from pneumonia or not is judged.

The deep neural network is a machine learning technology used for establishing and simulating a neural network for analyzing and learning of the human brain and simulating the mechanism of the human brain to explain data. Is part of the Artificial Intelligence (AI) framework. By means of the strong calculation and analysis capability of the deep neural network DNN, the spiral sound can be rapidly and effectively intelligently identified.

DNN implements the concept of multi-layer network learning. When an input mode of the network is given, the input mode is transmitted to the hidden layer unit from the input layer unit, processed layer by the hidden layer unit and then transmitted to the output layer unit, and an output mode is generated after the output layer unit processes the hidden layer unit and is a layer-by-layer state updating process called forward propagation. If the output response has errors with the expected output mode and does not meet the requirements, the error is transferred to the error back propagation, the error values are transmitted layer by layer along the connecting passage, and the connecting weight values of each layer are corrected. For a given set of training patterns, the network is continuously trained by using the training patterns, the forward propagation and error back propagation processes are repeated, and when the requirements of all the training patterns are met, the DNN network is considered to be trained. There are actually two types of signals flowing in a BP network:

(1) the working signal propagates in the forward direction. The input signal passes from the input layer through the hidden layer to the output layer, producing an output signal at the output. The weight value of the network is fixed and invariable in the forward transmission process of the signal, and the state of each layer of neuron only affects the state of the next layer of neuron. If the desired output is not available at the output layer, the input error signal back-propagation process is entered.

(2) The error signal propagates in reverse. The difference between the actual output and the expected output of the network is an error signal, and the error signal is propagated from the output end layer by layer. In the process of error signal back propagation, the weight of the network is adjusted by error feedback. The actual output of the network is closer to the expected output through continuous modification of the weight values.

The invention has the beneficial effects that:

the invention combines a long-time memory network LSTM and a deep neural network DNN (hybrid deep learning model) to carry out intelligent diagnosis on infant pneumonia. According to the existing medical experience, the existence of spiral sound in the breath sound is an important basis for judging that the infant has pneumonia, and the spiral sound is an abnormal sound caused by lung lesion contained in the breath sound and is generally judged by an experienced doctor with a stethoscope. The automatic identification of the spiral sound is similar to voice identification, so the method adopts the most common long-time memory network LSTM in the voice identification to identify the spiral sound of the breathing voice frequency. Meanwhile, the diagnosis of pneumonia is based on several other physiological indicators, including: in order to realize intelligent diagnosis of pneumonia, the body temperature, the heart rate and the respiratory rate need to judge other physiological indexes besides the existence of the spiral sound, therefore, the invention adopts a deep neural network to establish a classification recognition model for the spiral sound recognition result (the existence is 1, and the nonexistence is 0) and other physiological indexes, thereby finally realizing intelligent diagnosis of pneumonia. In summary, the combination of LSTM and DNN for intelligent pneumonia diagnosis comes from understanding the actual pneumonia diagnosis process, and makes use of the advantages of LSTM (LSTM excels in classifying time series signals, such as respiratory audio) and DNN (DNN excels in classifying static data, such as digital identification). Therefore, the invention is an innovation combining actual diagnosis and treatment knowledge and artificial intelligence technology. The invention can effectively improve the diagnosis speed and the correct rate of infant pneumonia, and avoid serious injury to the infant body caused by overlong diagnosis waiting time or misdiagnosis.

Drawings

FIG. 1 is a flow chart of the operation of the present invention;

FIG. 2 is a flow chart of a hybrid deep learning model pneumonia diagnosis of the present invention;

fig. 3 is a schematic diagram of a Deep Neural Network (DNN) architecture.

Detailed Description

In order to make the objects, technical solutions and advantages of the present invention more apparent, the present invention is described in further detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative of the invention and are not intended to limit the invention. In addition, the ordinal terms such as step one, step two, etc. in this document are established based on the step relationships shown in the flowcharts.

In the present invention, the terms such as "fourier transform", "long-short term memory network", etc. conform to the meanings defined in the relevant technical standards of the terms, and it is obvious to those skilled in the art that the specific meanings of the above terms in the present invention can be understood according to specific situations.

Example 1

Referring to fig. 1-2, a method for rapidly and intelligently diagnosing infant pneumonia based on a hybrid deep learning model includes the following steps:

infants refer to children born to the age of 1 year old, and young children refer to children 1-3 years old. Infants have the following physiological characteristics:

firstly, the growth and development period is rapid, the metabolism is vigorous, but the development of each organ is incomplete, and the function is immature.

Secondly, the immune function is low, the defense mechanism is poor, so the medicine is easy to be infected by acute infectious diseases and easy to cause food allergy, poisoning and infection.

The normal physiological index of human body is an important standard for judging whether the human body is healthy or not. The contents include body temperature, heart rate, blood pressure, etc. For example, the normal body temperature is 36-37 deg.C, the normal heart rate is 110-.

The physiological indexes of the infant mainly refer to respiratory audio data, body temperature, heart rate and respiratory frequency. The special data acquisition instrument mainly comprises a respiratory audio detector, a body temperature measurer, a heart rate detector and a respiratory frequency sensor.

The method comprises the following steps: according to local conditions, 300 cases of physiological indexes of infant pneumonia patients and normal infants are collected by visiting local large hospitals (data collection of 2 months in local hospitals) and the method comprises the following steps: the method comprises the steps of obtaining heart rate, body temperature, respiratory rate and respiratory audio data, marking the data according to the type of pneumonia, and dividing the data into a training set (comprising physiological index data of 260 pneumonia patients and physiological index data of 260 normal infants) and a testing set (comprising physiological index data of 40 pneumonia patients and physiological index data of 40 normal infants); because of the enormous amount of data, some experimental data are provided as shown in table 1:

table 1: part of the collected data for experiments

Heart rate	Body temperature (°)	Breathing frequency (times/minute)	Whether there is any spiral sound	Whether or not to suffer from pneumonia
					135	36.5	55	0	0
137	36.6	56	0	0
					140	36.4	50	0	0
100	37.8	65	1	1
					150	38.5	70	1	1
155	38.6	72	1	1
					…	…	…	…	…

Step two: cleaning the obtained infant non-audio physiological data, removing abnormal data, marking the data, and constructing an infant pneumonia physiological index data set which comprises a training set and a testing set and is used for training a deep neural network DNN;

processing data except respiratory audio by using filtering, linear transformation and standardization technologies, and eliminating data groups corresponding to abnormal data and invalid data for deep neural network DNN training; the method specifically comprises the following steps:

assuming that there are N detection samples and M characteristic parameters, a matrix from which the original data can be obtained is:

after standardization treatment, each index is obtained:

in the formula (I), the compound is shown in the specification,

and σ_jMean and standard deviation of the jth characteristic parameter:

step three: analyzing the breathing audio data in the data by utilizing Fourier transform, extracting corresponding characteristic information, and constructing a data set about the spiral sound, wherein the data set also comprises a training set and a test set;

the method comprises the following steps of analyzing the respiratory audio data and extracting corresponding characteristic information:

for respiratory audio data, feature Frequency domain feature extraction (using Mel-Frequency cepstral coefficients) is performed using fourier transform, and in the field of sound processing, Mel-Frequency Cepstrum (Mel-Frequency Cepstrum) is a linear transform of the log energy spectrum based on the nonlinear Mel scale (Mel scale) of sound frequencies. The Mel-Frequency Cepstral Coefficient (MFCC) is a Coefficient constituting the Mel-Frequency Cepstral. For training of long and short term memory networks LSTM, the solution of MFCC typically comprises the following steps:

a. decomposing a voice signal into a plurality of frames;

b. pre-emphasizing the voice signal, and passing the voice signal through a high-pass filter;

c. performing fourier transform to transform the signal into the frequency domain, namely:

d. passing the obtained frequency spectrum of each frame through a Mel filter (triangular overlapping window) to obtain Mel scale;

e. extracting logarithmic energy on each mel scale;

f. the result obtained above is subjected to inverse discrete fourier transform and transformed into the cepstrum domain, that is:

MFCC is the magnitude (amplitudes) of this cepstral plot. Typically 12 coefficients are used, superimposed with the frame energy to obtain 13-dimensional coefficients.

Step four: training a long-time memory neural network LSTM by using the frequency domain characteristics of the respiratory audio and the labels thereof, and carrying out classification and identification on the respiratory audio data; the method specifically comprises the following steps:

training a long-time memory neural network LSTM by using the frequency domain characteristics of the respiratory audio and the labels thereof to obtain a first part of a model and a spiral sound identification part;

the LSTM has three "gate" structures, namely an input gate, an output gate, and a forgetting gate.

(1) Forget the door:

let the network "forget" the information that was not useful in the previous step, it will depend on the current input x^(t)State of the last moment c^(t-1)Output h of the previous moment^(t-1)The input part of the forgetting gate mainly comprises a hidden state h at the previous moment^(t-1)And input data x of the current time^(t)It passes through an activation function sigma, generally a sigmoid function, which, due to its characteristics, ensures that the output of the forgetting gate is at[0,1]Output f^(t)Comprises the following steps:

f^(t)＝σ(W_fh^(t-1)+U_fx^(t)+b_f) (6)

wherein W_f,U_f,b_fRespectively, the cyclic weight, the input weight and the offset of the forgetting gate;

(2) an input gate:

after the neural network 'forgets' the information of the previous state, supplementing new memory; according to the current input x^(t)State of the last moment c^(t-1)Output h of the previous moment^(t-1)Jointly determining useful information into a new state c^(t)And stored in memory cells; the output of which comprises two parts, i^(t)And a^(t)：

i^(t)＝σ(W_ih^(t-1)+U_ix^(t)+b_i) (7)

a^(t)＝tanh(W_ah^(t-1)+U_ax^(t)+b_a) (8)

Wherein, W_i,W_aFor different connection weights, U, of the previous state_i，U_aFor different connection weights of the inputs, i_b，a_bRespectively, a respective bias portion.

(3) An output gate:

using status cells c^(t)Determining how much information is output to hidden layer state unit h^(t)When the user passes through the forgetting gate and the input gate, the results of the two gates act on the state unit c^(t)，c^(t)The mathematical expression implemented is:

c^(t)＝i^(t)Θa^(t)+f_tΘc^(t)(9)

wherein i^(t)Θa^(t)For remaining in state unit c after passing through input gate^(t)Information of (1), f_tΘc^(t-1)To remain at c after passing through a forgetting door^(t)The information in (1). At the same time, the layer state h is hidden^(t)Updated mainly by two parts, x^(t)、h^(t-1)Result of the combined action o^(t)And status unit c^(t)To determine o^(t)And h^(t)The formula implemented is:

o^(t)＝σ(W_oh^(t-1)+U_ox^(t)+b_o) (10)

h^(t)＝o^(t)Θtanh(c^(t)) (11)

wherein U is_o，W_o，b_oRespectively inputting x for the current time^(t)Output gate weight, last hidden state unit h^(t-1)Round robin weight and output gate offset;

and (3) processing the respiratory audio signal according to the flow, inputting the respiratory audio signal into an LSTM network, if the output result is 1, judging that the respiratory audio contains spiral sound, and if the output result is 0, judging that the respiratory audio does not contain spiral sound.

The LSTM back propagation algorithm is slightly different from the traditional neural network back propagation algorithm, for the recurrent neural network, because 'cycles' exist in the network, some weight matrixes are calculated and shared in multiple layers of the network, and each time node depends on previous node information, namely historical information, so that the common BP algorithm cannot be directly used in a recurrent neural network model training stage. The back propagation algorithm utilized by the LSTM is bptt (backpropagation threouthtime), which is to calculate partial derivatives of parameters on a time sequence and calculate gradients of loss functions layer by layer so as to update the parameters.

The collected respiratory audio data of 260 infants with pneumonia and 260 normal infants are used for carrying out LSTM training to obtain a recognition model for spiral sound, and the respiratory audio data of the rest 40 groups of infants with pneumonia and 40 normal infants are used for carrying out model verification, so that the accuracy rate of spiral sound recognition of the model is 85%.

Step five: the method comprises the following steps of establishing and training a deep neural network DNN classification recognition model for pneumonia diagnosis by utilizing a spirophone recognition result, body temperature, heart rate and respiratory frequency physiological indexes, judging and reading various winning data of a patient, and further judging whether the patient suffers from pneumonia, wherein the method specifically comprises the following steps:

inputting data including body temperature, heart rate, respiratory rate and whether the respiratory audio data contains spiral sound (1 or 0) into a deep neural network DNN for training (the structure of the deep neural network DNN refers to FIG. 3), and obtaining a second part of the model, namely a pneumonia diagnosis part;

(1) initialization

The convergence of the network is greatly influenced by the setting of the initial value of the connection weight (including a threshold value); the net input value of each neuron is small due to the expected initial value so as to ensure that the neuron works in an activation region, namely a region with sensitive change, and the initial value of each connection weight is a random number;

(2) forward pass process computation

(a) Input learning sample pair (X)_q,T_q) Wherein Q is 1,2, … Q;

(b) for each input sample, sequentially calculating the output value of each neuron according to the following formula;

in the formula:

output of each hidden layer of DNN, w_ji，w_kj，w_lkThe weight coefficients of each hidden layer are respectively, and because the signals in the neural network are transmitted layer by layer, the output of the previous hidden layer in the neural network is used as the input of the next hidden layer, so that

Is the output of the first hidden layer, is also the input of the second hidden layer,

is the output of the second hidden layer and is also the input of the third hidden layer;

(3) error back propagation

Calculating equivalent errors of neurons of each layer from the output layer to the input layer, then returning to the step (2), and performing forward propagation calculation and error backward propagation on other learning sample pairs until all α learning sample pairs are similarly calculated;

(4) adjusting connection weight between layers

And modifying the connection weight of each layer according to the following formula:

in the formula: w is a_ik(n+1)，w_kj(n+1)，w_ji(n +1) are the weight coefficients of the hidden layers after the back propagation update by the gradient method, w_ik(n)，w_kj(n)，w_ji(n) the current weight coefficients of the hidden layers, η represents the step size of the gradient method,

respectively representing gradient information of each hidden layer after being propagated through the neural network;

(5) returning to the step (2), performing forward calculation according to the new connection weight, and if each learning sample pair meets the precision requirement, that is:

wherein, p is 1,2, … …, α, l is 1,2, … …, m, (α, m respectively represents the number of the activation units of each hidden layer and the number of the hidden layers of the neuron), epsilon is a certain value given according to the precision requirement, when the above formula is satisfied, the learning of the network is completed, otherwise, the calculation is repeated until the requirement is satisfied;

the calculation process of the Deep Neural Network (DNN) is actually an updating process of the weight and the threshold of the network, and the weight and the threshold which enable the overall error to be minimum are found;

the final diagnostic model obtained was:

in the formula: f. of₁，f₂，f₃Respectively representing the activation functions of all the hidden layers, and other symbols are described in the description;

inputting the output result (1 or 0) of the LSTM network and physiological indexes of body temperature, heart rate and respiratory rate into the deep neural network, if the output is 1, judging that the patient has pneumonia, and if the output is 0, judging that the patient does not have pneumonia.

The method comprises the steps of carrying out DNN training by using collected physiological data of 260 pneumonia-affected infants and 260 normal infants to obtain a classification recognition model for pneumonia diagnosis, and carrying out model verification by using physiological data of the rest 40 groups of pneumonia-affected infants and 40 normal infants to obtain the accuracy rate of pneumonia diagnosis of the model, wherein the accuracy rate of the pneumonia diagnosis of the model is 80%.

The above description is only for the purpose of illustrating the preferred embodiments of the present invention and is not to be construed as limiting the invention, and any modifications, equivalents and improvements made within the spirit and principle of the present invention are intended to be included within the scope of the present invention.

Claims (6)

1. A rapid intelligent diagnosis method for infant pneumonia based on a hybrid deep learning model comprises the following steps:

the method comprises the following steps: measuring and collecting various physiological indexes of the infant, including respiratory audio data, body temperature, heart rate and respiratory frequency;

step two: cleaning the obtained infant non-audio physiological data, removing abnormal data, marking the data, and constructing an infant pneumonia diagnosis physiological index data set, wherein the data set comprises a training set and a testing set and is used for training a Deep Neural Network (DNN);

step three: analyzing the respiratory audio data by utilizing Fourier transform, extracting corresponding characteristic information, and constructing a data set about the spiral sound identification, wherein the data set also comprises a training set and a testing set and is used for training a long-time memory network LSTM;

step four: training a long-time memory neural network LSTM by using the frequency domain characteristics of the respiratory audio and the labels thereof, and judging the spiroid of the respiratory audio data;

step five: and establishing and training a deep neural network DNN for identifying pneumonia by using a spiral voice identification result of the breathing audio data and physiological indexes of body temperature, heart rate and breathing frequency, and interpreting new input data so as to judge whether the patient has pneumonia.

2. The method for rapidly and intelligently diagnosing infant pneumonia based on a hybrid deep learning model as claimed in claim 1, wherein the first step is specifically as follows: according to local conditions, various physiological indexes of infant pneumonia patients and normal infants are collected, and the method comprises the following steps: heart rate, body temperature, respiratory rate, respiratory audio data, and then labeling and storing the data by the category of whether or not pneumonia is present.

3. The method for rapidly and intelligently diagnosing infant pneumonia based on a hybrid deep learning model according to claim 1, wherein the second step is specifically as follows: processing data except respiratory audio by using filtering, linear transformation and standardization technologies, and eliminating data groups corresponding to abnormal data and invalid data for deep neural network DNN training;

assuming that there are N detection samples and M characteristic parameters, a matrix from which the original data can be obtained is:

after standardization treatment, each index is obtained:

in the formula (I), the compound is shown in the specification,

and σ_jMean and standard deviation of the jth characteristic parameter:

4. the method for rapidly and intelligently diagnosing infant pneumonia according to claim 1, wherein the method for analyzing respiratory audio data in the third step and extracting corresponding characteristic information specifically comprises:

for the respiratory audio data, Fourier transform is utilized to extract characteristic frequency domain characteristics, the method is used for training a long-time memory network (LSTM), and the MFCC solving comprises the following steps:

a. decomposing a voice signal into a plurality of frames;

b. pre-emphasizing the voice signal, and passing the voice signal through a high-pass filter;

c. performing fourier transform to transform the signal into the frequency domain, namely:

d. passing the frequency spectrum obtained by each frame through a Mel filter to obtain Mel scales;

e. extracting logarithmic energy on each mel scale;

f. the result obtained above is subjected to inverse discrete fourier transform and transformed into the cepstrum domain, that is:

MFCC is the magnitude of this cepstrum; typically 12 coefficients are used, superimposed with the frame energy to obtain 13-dimensional coefficients.

5. The method for rapidly and intelligently diagnosing infant pneumonia based on a hybrid deep learning model according to claim 1, wherein the fourth step is specifically as follows:

training a long-time memory neural network LSTM by using the frequency domain characteristics of the respiratory audio and the labels thereof to obtain a first part of a model, namely a part for recognizing the spiral sound;

the LSTM has three gate structures, namely an input gate, an output gate and a forgetting gate;

(1) forget the door:

let the network "forget" the information that was not useful in the previous step, it will depend on the current input x^(t)State of the last moment c^(t-1)Output h of the previous moment^(t-1)The input part of the forgetting gate mainly comprises a hidden state h at the previous moment^(t-1)And input data x of the current time^(t)It uses an activation function sigma, generally a sigmoid function, to ensure the output of the forgetting gate is [0,1 ] due to the characteristics of the sigmoid function]Output f^(t)Comprises the following steps:

f^(t)＝σ(W_fh^(t-1)+U_fx^(t)+b_f) (6)

wherein W_f,U_f,b_fRespectively, the cyclic weight, the input weight and the offset of the forgetting gate;

(2) an input gate:

after the neural network 'forgets' the information of the previous state, supplementing new memory; according to the current input x^(t)State of the last moment c^(t-1)Output h of the previous moment^(t-1)Jointly determining useful information into a new state c^(t)And stored in memory cells; the output of which comprises two parts, i^(t)And a^(t)：

i^(t)＝σ(W_ih^(t-1)+U_ix^(t)+b_i) (7)

a^(t)＝tanh(W_ah^(t-1)+U_ax^(t)+b_a) (8)

Wherein, W_i,W_aFor different connection weights, U, of the previous state_i，U_aFor different connection weights of the inputs, i_b，a_bRespectively, a respective bias portion;

(3) an output gate:

using status cells c^(t)Determining how much information is output to hidden layer state unit h^(t)When the user passes through the forgetting gate and the input gate, the results of the two gates act on the state unit c^(t)，c^(t)The mathematical expression implemented is:

c^(t)＝i^(t)Θa^(t)+f_tΘc^(t)(9)

wherein i^(t)Θa^(t)For remaining in state unit c after passing through input gate^(t)Information of (1), f_tΘc^(t-1)To remain at c after passing through a forgetting door^(t)The information in (1). At the same time, the layer state h is hidden^(t)Updated mainly by two parts, x^(t)、h^(t-1)Result of the combined action o^(t)And status unit c^(t)To determine o^(t)And h^(t)The formula implemented is:

o^(t)＝σ(W_oh^(t-1)+U_ox^(t)+b_o) (10)

h^(t)＝o^(t)Θtanh(c^(t)) (11)

wherein U is_o，W_o，b_oRespectively inputting x for the current time^(t)Output gate weight, last hidden state unit h^(t-1)Round robin weight and output gate offset;

and (3) processing the respiratory audio signal according to the flow, inputting the respiratory audio signal into an LSTM network, if the output result is 1, judging that the respiratory audio contains spiral sound, and if the output result is 0, judging that the respiratory audio does not contain spiral sound.

6. The method for rapidly and intelligently diagnosing infant pneumonia based on a hybrid deep learning model according to claim 1, wherein the fifth step is specifically as follows:

inputting data including body temperature, heart rate, respiratory rate and whether the respiratory audio data contains spiral sound into a deep neural network DNN for training to obtain a second part of the model, namely a pneumonia diagnosis part;

(1) initialization

The convergence of the network is greatly influenced by the setting of the initial value of the connection weight comprising the threshold; the net input value of each neuron is small due to the expected initial value so as to ensure that the neuron works in an activation region, namely a region with sensitive change, and the initial value of each connection weight is a random number;

(2) forward pass process computation

(a) Input learning sample pair (X)_q,T_q) Wherein Q is 1,2, … Q;

(b) for each input sample, sequentially calculating the output value of each neuron according to the following formula;

in the formula:

output of each hidden layer of DNN, w_ji，w_kj，w_lkThe weight coefficients of each hidden layer are respectively, and because the signals in the neural network are transmitted layer by layer, the output of the previous hidden layer in the neural network is used as the input of the next hidden layer, so that

Is the output of the first hidden layer, is also the input of the second hidden layer,

is the output of the second hidden layer and is also the input of the third hidden layer;

(3) error back propagation

Calculating equivalent errors of neurons of each layer from the output layer to the input layer, then returning to the step (2), and performing forward propagation calculation and error backward propagation on other learning sample pairs until all α learning sample pairs are similarly calculated;

(4) adjusting connection weight between layers

And modifying the connection weight of each layer according to the formula (13), namely:

in the formula: w is a_ik(n+1)，w_kj(n+1)，w_ji(n +1) are the weight coefficients of the hidden layers after the back propagation update by the gradient method, w_ik(n)，w_kj(n)，w_ji(n) the current weight coefficients of the hidden layers, η represents the step size of the gradient method,

respectively representing gradient information of each hidden layer after being propagated through the neural network;

(5) returning to the step (2), performing forward calculation according to the new connection weight, and if each learning sample pair meets the precision requirement, that is:

wherein, p is 1,2, … …, α, l is 1,2, … …, m, (α, m respectively represents the number of the activation units of each hidden layer and the number of the hidden layers of the neuron), epsilon is a certain value given according to the precision requirement, when the above formula is satisfied, the learning of the network is completed, otherwise, the calculation is repeated until the requirement is satisfied;

the calculation process of the deep neural network DNN is actually an updating process of the weight and the threshold of the network, and the weight and the threshold which enable the overall error to be minimum are found;

the final diagnostic model (classification recognition model) is:

in the formula: f. of₁，f₂，f₃Respectively representing the activation functions of all the hidden layers, and other symbols are described in the description;

inputting the output result (1 or 0) of the LSTM network and physiological indexes of body temperature, heart rate and respiratory rate into the deep neural network, if the output is 1, judging that the patient has pneumonia, and if the output is 0, judging that the patient does not have pneumonia.

Images