CN110731778A - A visualization-based breathing sound signal recognition method and system - Google Patents

Abstract

The invention relates to the field of audio-frequency signal identification, and discloses a visualization-based breathing sound signal identification method and system. The short-time Fourier transform is used to perform time-frequency analysis on a cut breath sound periodic signal, and a one-dimensional audio signal is converted into In order to visualize the signal in two dimensions, through the processing and analysis of the image, a data set is formed, and the image classification of the convolutional neural network is performed to realize the distinction between normal and three pathological breath sounds. The pathological breath sound signal has obvious noise, and the noise formed in the process of exhalation and inhalation has special spectral information. The present invention uses the time-frequency analysis method and uses short-time Fourier transform to perform the cut breath sound cycle signal. Time-frequency analysis converts one-dimensional audio signals into two-dimensional visual signals. Through image processing and analysis, a data set is formed, and visual images are classified based on convolutional neural networks to distinguish between normal and three pathological breath sounds.

Description

A visualization-based breathing sound signal recognition method and system

technical field

The present invention relates to the field of audio signal recognition, and more particularly, to a method and system for recognizing breath sound signals based on visualization.

Background technique

The breath sound signal is a physiological signal produced by the human respiratory system and the outside world during the ventilation process. Breath sounds contain a lot of physiological and pathological information, which can well reflect the health of the human respiratory system. Therefore, it has very important research significance in breath sounds and clinical medicine. In recent years, the increase in the incidence of respiratory diseases caused by environmental problems such as frequent haze weather has also greatly increased the demand for the rapidity and accuracy of respiratory disease diagnosis.

Cardiopulmonary auscultation has attracted widespread attention due to its advantages of speed, convenience and non-invasiveness. However, it is difficult for unskilled medical personnel to monitor directly with stethoscopes. The development of automatic breath sound diagnosis technology will undoubtedly bring important help to the diagnosis of respiratory diseases. The development of hardware equipment such as electronic stethoscope and other signal acquisition technologies and software such as automatic identification and disease early warning has further promoted the research and progress of modern breath sound signal analysis and identification technology.

Common breath sound feature extraction algorithms include Auto-Regressive (AR) algorithm, power spectral density (PSD)-based algorithm, cepstrum-based (Mel-frequency cepstrum) MFCC coefficient method, and Discrete wavelet decomposition and wavelet packet decomposition of wavelet transform technology (Wavelet Transform, WT). The features extracted by these methods are not visualized and robust, and provide little reference value for clinicians. The methods currently used generally perform manual analysis and judgment through the time-domain waveform and spectrogram of the signal, and do not realize the combined analysis of automation and visualization.

The invention uses the short-time Fourier transform to perform time-frequency analysis on the cut breath sound cycle signal, converts the one-dimensional audio signal into a two-dimensional visual signal, and processes and analyzes the image to form a data set for volume The image classification of the integrated neural network realizes the distinction between normal and three pathological breath sounds. Pathological breath sound signal murmur is obvious, and the murmur formed during exhalation and inhalation has special spectral information.

SUMMARY OF THE INVENTION

The object of the present invention is to provide a method and system for recognizing breath sound signals based on visualization. The present invention uses a time-frequency analysis method, uses short-time Fourier transform to perform time-frequency analysis on the cut breath sound cycle signal, and converts a The three-dimensional audio signal is converted into a two-dimensional visual signal, and a data set is formed through the processing and analysis of the image. The visual image is classified based on the convolutional neural network, and the normal and three pathological breath sounds are distinguished.

In order to achieve the above object, the technical scheme adopted in the present invention is as follows:

A method for recognizing a breath sound signal based on visualization, comprising: S1, collecting an original breath sound signal, filtering and separating the signal to obtain a preprocessed breath sound signal; S2, dividing the period of the preprocessed breath sound signal to obtain a set period S3, perform Fourier transform on the breath sound signal of the set period to obtain the frequency information of the breath sound signal; S4, according to the division point of the breath sound cycle signal, to The two-dimensional spectrogram of the breath sound signal is processed to obtain a single-cycle two-dimensional spectrogram; S5, establish a frequency map data set according to the two-dimensional spectrogram; S6, establish a convolutional neural network model according to the frequency map data set, S7, pass The convolutional neural network model performs predictive analysis on new types of breath sounds.

Further, the specific process of the filtering and separation processing in the step S1 includes: S100, performing high-pass filtering processing on the original breath sound signal, which can effectively take out environmental murmurs, current murmurs and other murmurs in the original breath sound signal, and obtain heart sounds and breathing. S101, performing wavelet transformation on the heart sound and breath sound mixed signal to obtain the heart sound interference signal in the breath sound and separating it separately; S102, subtracting the heart sound interference signal from the heart sound and breath sound mixed signal A preprocessed breath sound signal (a relatively pure breath sound signal) is obtained.

Further, in the step S2, the preprocessed breath sound signal adopts a moving rectangular window to perform periodic division, and the division point of the breath sound periodic signal is determined, and the respective breath sounds of normal breath sound, stridor sound, crepitus, and pleuropathy sound are determined. The periodic signal is cut separately, and the method of moving the rectangular window is used to obtain the minimum value before the next cycle (one exhalation and one inhalation as one cycle). 2*fs number of sample points, the vertical line in the figure is the cutting point, it can be seen that the cut breath sound signal is divided into obvious breath sound cycles of exhalation and inhalation, and the small peak part corresponding to the small amplitude represents exhalation, and the amplitude The large peak portion corresponding to the large represents inspiration.

Further, the types of the original breath sounds include normal breath sounds, stridor sounds, crepitus and pleuropathy sounds.

Further, the specific steps of the step S3 include: S300, using an analysis window that moves with time to window and truncate the breath sound signal of a set period, and decompose it into a series of approximately stationary short-term signals; S301, The two-dimensional spectrogram of each short-term stationary signal is obtained by Fourier transform; a time-sliding analysis window is used to perform windowing and truncation operations on the non-stationary signal, and then the non-stationary signal is decomposed into a series of approximately stationary short-term signals. Time signal (for a long time non-stationary signal, the signal of a short period of time is cut out by adding a sliding window, and the signal can be considered to be approximately stationary in this short period of time), and finally the Fourier transform is used to analyze each short-term stationary signal. spectrum.

The short-time Fourier transform first multiplies a function and a window function, then performs a one-dimensional Fourier transform, and finally obtains the result of the transformation by sliding the window function, and arranging the obtained results can obtain a two-dimensional representation. . The short-time Fourier formula is as follows:

In the formula, Z(t) is the original signal, g(t) is the window function, and u is an integral variable used for integral calculation.

Further, the step S5 includes: S500, cutting the two-dimensional spectrogram according to the division point of the breath sound signal to form a single-period two-dimensional spectrogram; S501, performing specification processing on the single-period two-dimensional spectrogram Form a spectrogram dataset.

Further, the specific steps of the specification processing in the step S501 include: S5010, unify the size of the two-dimensional time-frequency atlas; S5011, perform RGB component analysis on the unified size of the two-dimensional time-frequency atlas; S5012, perform a picture Compressed to get a frequency map dataset.

Further, since the four time-frequency maps are separately processed and cut, the coordinate sizes are different, so the width and height of the images are inconsistent, so the images need to be processed. From the cut breath sound image database to the data set that can be convolved and trained, the processing flow is as follows: S700, acquire new types of breath sound signals and perform high-pass filtering and wavelet transform processing on them; S701, use the moving matrix window to The noise-removed breath sound signal is divided into periods, and the division point of the breath sound period signal is determined; S702, short-time Fourier transform is performed on the divided breath sound period signal; S703, according to the division point of the divided breath sound signal , perform periodic segmentation on the spectrogram that has undergone short-time Fourier transform to obtain a single-period two-dimensional spectrogram; S704, extract the R component in the RGB in the periodic spectrogram, and compress it into spectrogram data of a set size; S705, The compressed data is put into the pre-trained neural network model for prediction, and each category is predicted as the final breath sound category.

This solution also provides a system for recognizing a breath sound signal based on visualization, including: a signal acquisition and processing unit for collecting the original breath sound signal, filtering and separating the signal to obtain a preprocessed breath sound signal; a period dividing module , using a moving rectangular window to divide the period, and determine the division point of the breath sound periodic signal; the Fourier transform module is used to perform Fourier transform on the preprocessed breath sound signal to obtain the frequency information of the breath sound signal; the cutting module, It is used to process the two-dimensional spectrogram of the breath sound signal according to the division point of the breath sound periodic signal to obtain a single-period two-dimensional spectrogram; the frequency chart data set establishment module establishes a frequency chart data set according to the two-dimensional spectrogram; The convolutional neural network module is used to establish a convolutional neural network model according to the frequency map data set, and predict and analyze new types of breath sounds through the convolutional neural network model.

Further, the signal acquisition processing unit includes: a high-pass filter module for removing environmental noises, current sounds and other noises in the original breath sound signal; a wavelet transform module for dividing out the heart sound components in the filtered breath sound signal , reconstruct the heart sound component in the breath sound signal, and separate the heart sound interference signal separately; the copy module is used to copy the mixed signal of heart sound and breath sound obtained after high-pass filtering; the separation module is used to The heart sound interference signal is separated separately; the subtraction module is used to subtract the heart sound interference signal from the mixed signal of the heart sound and the breath sound to obtain the preprocessed breath sound signal.

Compared with the prior art, the advantages of the present invention are: this scheme provides a method and system for recognizing breath sound signals based on visualization, the present invention uses a time-frequency analysis method, and uses short-time Fourier transform to Perform time-frequency analysis on the periodic signal of breath sounds, convert the one-dimensional audio signal into a two-dimensional visual signal, form a data set through image processing and analysis, classify the visual image based on convolutional neural network, and distinguish between normal and three pathological breaths sound.

Description of drawings

In order to explain the embodiments of the present invention or the technical solutions in the prior art more clearly, the following briefly introduces the accompanying drawings that need to be used in the description of the embodiments or the prior art. Obviously, the accompanying drawings in the following description are only These are some embodiments of the present invention. For those of ordinary skill in the art, other drawings can also be obtained according to these drawings without creative efforts.

Fig. 1 is the method block diagram of the present invention;

Fig. 2 is a method block diagram for filtering and separating processing of original breath sound signal in the present invention;

Fig. 3 is the method block diagram of establishing frequency chart data set according to two-dimensional spectrogram in the present invention;

Fig. 4 is the method block diagram of specification processing in the present invention;

5 is a block diagram of a method for predicting and analyzing new types of breath sounds by a convolutional neural network model in the present invention;

Fig. 6 is a system block diagram in the present invention;

7 is a waveform diagram of a normal breath sound signal in the present invention;

Fig. 8 is the waveform diagram after the division of normal breath sound signal in the present invention;

FIG. 9 is a system block diagram of a signal acquisition processing unit in the present invention.

Detailed ways

The preferred embodiments of the present invention will be described in detail below with reference to the accompanying drawings, so that the advantages and features of the present invention can be more easily understood by those skilled in the art, and the protection scope of the present invention can be more clearly defined.

Referring to FIG. 1, a method for recognizing breath sound signals based on visualization includes: S1, collecting original breath sound signals, filtering and separating the signals to obtain preprocessed breath sound signals; S2, performing preprocessing breath sound signals on the signal Carry out cycle division to obtain the breath sound signal of the set period, and determine the division point of the breath sound signal; S3, perform Fourier transform on the breath sound signal of the set period to obtain the frequency information of the breath sound signal; S4, according to the breath sound At the division point of the periodic signal, the two-dimensional spectrogram of the breath sound signal is processed to obtain a single-period two-dimensional spectrogram; S5, establishing a frequency map data set according to the two-dimensional spectrogram; S6, establishing a convolution according to the frequency map data set Neural network model, S7. Predictive analysis of new types of breath sounds through a convolutional neural network model.

The convolutional neural network adopts two layers of convolution, two layers of pooling, and two layers of full connection. The pooling kernel size is 2*2. Trained with LeNet model.

LeNet counts the input and output as a total of eight layers:

The first layer: the data input layer, the data is normalized first, and the interval range is 0-255 of the grayscale image.

The second layer: convolutional layer c1

The convolutional layer is the core of the convolutional neural network, and the features of the image are obtained through different convolution kernels. The convolution kernel is equivalent to a filter, and different filters extract different features.

The third layer: pooling pooling layer

Basically, there is a pooling layer behind each convolutional layer for the purpose of dimensionality reduction. Generally, the size of the output matrix of the original convolution layer is changed to half of the original, which is convenient for subsequent operations. In addition, the pooling layer increases the robustness of the system and turns the original accurate description into a rough description to prevent overfitting to a certain extent.

Fourth layer: convolutional layer

Similar to before, the features are further extracted, and the original samples are expressed at a deeper level.

Fifth layer: pooling layer

The sixth layer: convolutional layer (full connection)

There are 100 convolution kernels here, which are fully connected here. Convolve the matrix into a number, which is convenient for the subsequent network to determine.

The seventh layer: fully connected layer

Like the hidden layer in MLP, the representation of high-dimensional spatial data is obtained.

Eighth layer: output layer

The RBF network is generally used here. The center of each RBF is the symbol of each category, and the minimum output value is the final classification result predicted by the network. For this experiment, it is the final predicted breath sound category.

In this embodiment, please refer to FIG. 2 , the specific process of the filtering and separation processing in the step S1 includes: S100 , performing high-pass filtering processing on the original breath sound signal, which can effectively extract the environmental noise, current murmur and other murmurs, obtain the mixed signal of heart sound and breath sound and copy it; S101, perform wavelet transformation on the mixed signal of heart sound and breath sound to obtain the heart sound interference signal in the breath sound and separate it separately; S102, pass the heart sound and The preprocessed breath sound signal (a relatively pure breath sound signal) is obtained by subtracting the heart sound interference signal from the breath sound mixed signal.

In this embodiment, please refer to FIG. 7 and FIG. 8 , in the step S2, the preprocessed breath sound signal adopts a moving rectangular window to perform periodic division, and determines the division point of the breath sound periodic signal. The breath sound cycle signals of the sound, crepitus, and pleuropathy are respectively cut, and the minimum value before the next cycle (one exhalation and one inhalation as a cycle) is obtained by moving the rectangular window. The processing parameters are: rectangle The window size is 0.8~2s, that is, the number of sample points of 0.8*fs~2*fs. The vertical line in Figure 7 is the cutting point. It can be seen that the cut breath sound signal is divided into obvious breath sound cycles of breathing and breathing. The small peak part corresponding to the small value represents exhalation, and the large peak part corresponding to the large amplitude represents inhalation.

In this embodiment, the types of the original breath sounds include normal breath sounds, stridor sounds, crepitus sounds and pleuropathy sounds.

In the present embodiment, the specific steps of the step S3 include: S300, using an analysis window that moves with time to window and truncate the breath sound signal of the set period, and decompose it into a series of approximately stationary short-term signals; S301. Obtain a two-dimensional spectrogram of each short-term stationary signal through Fourier transform; use a time-sliding analysis window to perform windowing and truncation operations on the non-stationary signal, and then decompose the non-stationary signal into a series of approximately stationary signals The short-term signal (long-term non-stationary signal, the signal of a short period of time is cut out by adding a sliding window, the signal can be considered to be approximately stationary in this short period of time), and finally each short-term stationary signal is analyzed by Fourier transform. the spectrum of the signal.

The short-time Fourier transform first multiplies a function and a window function, then performs a one-dimensional Fourier transform, and finally obtains the result of the transformation by sliding the window function, and arranging the obtained results can obtain a two-dimensional representation. . The short-time Fourier formula is as follows:

In the formula, Z(t) is the original signal, g(t) is the window function, and u is an integral variable used for integral calculation.

In this embodiment, please refer to FIG. 3 , the step S5 includes: S500 , cutting the two-dimensional spectrogram according to the division point of the breath sound signal to form a single-period two-dimensional spectrogram set; S501 , for the single-period spectrogram The two-dimensional spectrogram set is subjected to specification processing to form a spectrogram dataset.

In this embodiment, please refer to FIG. 4 , the specific steps of the specification processing in step S501 include: S5010, unify the size of the two-dimensional time-frequency atlas; S5011, unify the size of the two-dimensional time-frequency atlas Perform RGB component analysis; S5012, perform image compression to obtain a frequency map data set.

In this embodiment, since the four time-frequency maps are separately processed and cut, the coordinate sizes are different, so the width and height of the pictures are inconsistent, so the pictures need to be processed. From the cut breath sound image database to the data set that can be convolved and trained, the processing flow is as follows: Please refer to Figure 5, S700, obtains new types of breath sound signals and performs high-pass filtering and wavelet transform processing on them S701, utilize moving matrix window to carry out periodic division to the breath sound signal of removing noise, and determine the division point of breath sound periodic signal; S702, carry out short-time Fourier transform to the divided breath sound periodic signal; S703, according to dividing A good dividing point of the breath sound signal, perform periodic division on the short-time Fourier-transformed spectrogram to obtain a single-period two-dimensional spectrogram; S704, extract the R component in the RGB in the periodic frequency graph, and compress it into a set Spectrogram data of the size; S705, put the compressed data into the pre-trained neural network model for prediction, and a predicted category is the final breath sound category.

This solution also provides a system for recognizing breath sound signals based on visualization, as shown in FIG. 6 , including: a signal acquisition and processing unit 802 for collecting original breath sound signals, filtering and separating the signals to obtain pre- Process the breath sound signal; the period division module 804 uses a moving rectangular window to divide the period, and determines the division point of the breath sound period signal; the Fourier transform module 806 is used to perform Fourier transform on the preprocessed breath sound signal to obtain The frequency information of the breath sound signal; the cutting module 808 is used to process the two-dimensional spectrogram of the breath sound signal according to the division point of the breath sound periodic signal to obtain a single-period two-dimensional spectrogram; the frequency map data set establishment module 810, A frequency map data set is established according to the two-dimensional spectrogram; the convolutional neural network module 812 is used to establish a convolutional neural network model according to the frequency map data set, and predict and analyze new types of breath sounds through the convolutional neural network model .

In this embodiment, please refer to FIG. 9 , the signal acquisition processing unit 802 includes: a high-pass filter module 8021, which is used to remove environmental noises, current sounds and other noises in the original breath sound signal; a wavelet transform module 8023, which is used for Divide the heart sound component in the filtered breath sound signal, reconstruct the heart sound component in the breath sound signal, and separate the heart sound interference signal separately; the copy module 8022 is used to perform high-pass filtering on the mixed signal of the heart sound and the breath sound. Duplication; the separation module 8024 is used to separate the heart sound interference signal obtained after wavelet transformation; the subtraction module 8025 is used to subtract the heart sound interference signal from the heart sound and breath sound mixed signal to obtain the preprocessed breath sound signal.

This is the working principle of the method and system for recognizing breath sound signals based on visualization. Meanwhile, the contents not described in detail in this specification belong to the prior art known to those skilled in the art.

Although the embodiments of the present invention are described in conjunction with the accompanying drawings, the patent owner can make various changes or modifications within the scope of the appended claims, as long as the protection scope described in the claims of the present invention is not exceeded, all should be within the protection scope of the present invention.

Claims (10)

1, A visual-based breath sound signal identification method, comprising:

s1, collecting original breath sound signals, and filtering and separating the signals to obtain preprocessed breath sound signals;

s2, carrying out periodic division on the preprocessed breathing sound signal to obtain a breathing sound signal with a set period, and determining a division point of the breathing sound signal;

s3, carrying out Fourier transform on the breath sound signal with the set period to obtain frequency information of the breath sound signal;

s4, processing the two-dimensional spectrogram of the breath sound signal according to the division points of the breath sound periodic signal to obtain a single-period two-dimensional spectrogram;

s5, establishing a frequency spectrum data set according to the two-dimensional frequency spectrum diagram;

s6, establishing a convolutional neural network model according to the frequency map data set;

and S7, carrying out predictive analysis on the new breath sounds through the convolutional neural network model.

2. The visualization-based breath sound signal identification method of claim 1, wherein the specific process of the filtering separation process in step S1 comprises:

s100, carrying out high-pass filtering processing on the original breath sound signal to obtain a heart sound and breath sound mixed signal and copying the heart sound and breath sound mixed signal;

s101, performing wavelet transformation on the heart sound and breath sound mixed signals to obtain heart sound interference signals in the breath sounds and separating the heart sound interference signals separately;

and S102, subtracting the heart sound interference signal from the heart sound and respiration mixed signal to obtain a preprocessed respiration sound signal.

3. The visualization-based breath sound signal identification method of claim 1, wherein the pre-processed breath sound signal in step S2 is periodically divided by a moving rectangular window, and the division point of the breath sound periodic signal is determined.

4. The visualization-based breath sound signal identification method of claim 1, wherein the types of the original breath sound comprise normal breath sound, wheezing sound, twiddle sound, and pleural lesion sound.

5. The visualization-based breath sound signal identification method of claim 1, wherein the specific step of step S3 comprises:

s300, windowing and cutting the breath sound signal with the set period by adopting an analysis window moving along with time, and decomposing the breath sound signal into series of approximately stable short-time signals;

s301, obtaining a two-dimensional spectrogram of each short-time stationary signal through Fourier transform.

6. The visualization-based breath sound signal identification method of claim 1, wherein the step S5 comprises:

s500, cutting the two-dimensional frequency spectrum graph according to the dividing point of the breathing sound signal to form a single-period two-dimensional frequency spectrum graph set;

s501, performing specification processing on the two-dimensional spectrogram set of the single period to form a spectrogram data set.

7. The visualization-based breath sound signal identification method of claim 6, wherein the specification processing in step S501 comprises the following specific steps:

s5010, calculating the size of a two-dimensional time-frequency atlas of a system ;

s5011, performing RGB component analysis on the two-dimensional time-frequency atlas with the size of the system ;

and S5012, carrying out picture compression to obtain a frequency picture data set.

8. The visualization-based breath sound signal identification method of claim 1, wherein the step S7 specifically comprises:

s700, acquiring new various respiratory sound signals and carrying out high-pass filtering and wavelet transformation processing on the signals;

s701, periodically dividing the breath sound signals with the noise removed by using a mobile matrix window, and determining division points of the breath sound periodic signals;

s702, carrying out short-time Fourier transform on the divided breath sound periodic signals;

s703, according to the divided points of the breathing sound signal, carrying out periodic division on the spectrogram subjected to short-time Fourier transform to obtain a single-period two-dimensional spectrogram;

s704, extracting R components in RGB in the periodic frequency chart, and compressing into spectrogram data with set size;

s705, the compressed data is put into a pre-trained neural network model for prediction, and the predicted category is the final breath sound category.

9, A system based on visual breath sound signal identification method, comprising:

the signal acquisition and processing unit is used for collecting original breath sound signals and filtering and separating the signals to obtain preprocessed breath sound signals;

the periodic division module is used for carrying out periodic division by adopting a movable rectangular window and determining division points of the periodic signals of the breath sounds;

the Fourier transform module is used for carrying out Fourier transform on the preprocessed breathing sound signal to obtain frequency information of the breathing sound signal;

the cutting module is used for processing the two-dimensional spectrogram of the breathing sound signal according to the division points of the breathing sound periodic signal to obtain a single-period two-dimensional spectrogram;

the frequency map data set establishing module is used for establishing a frequency map data set according to the two-dimensional frequency spectrogram;

and the convolutional neural network module is used for establishing a convolutional neural network model according to the frequency map data set and carrying out predictive analysis on various new breath sounds through the convolutional neural network model.

10. The system of the visual-based breath sound signal identification method of claim 9, wherein the signal acquisition processing unit comprises:

the high-pass filtering module is used for removing environmental noise, current noise and other noise in the original breathing sound signal;

the wavelet transform module is used for dividing the heart sound components in the filtered respiratory sound signals, reconstructing the heart sound components in the respiratory sound signals and independently separating the heart sound interference signals;

the copying module is used for copying the heart sound and breath sound mixed signal obtained after the high-pass filtering processing;

the separation module is used for separating the heart sound interference signals obtained after the wavelet transformation;

and the subtraction module is used for subtracting the heart sound interference signal from the heart sound and respiratory sound mixed signal to obtain a preprocessed respiratory sound signal.

Images