TWI798111B - Cough identification method and system thereof - Google Patents

Abstract

The present invention relates to a cough identification method and a system thereof. The identification method comprises the following steps: inputting a plurality of training audios and storing them in a storage device; converting the plurality of training audios into a plurality of audio signals, and inputting them into a personal audio feature extraction module performs convolutional neural network operations to establish a personal audio feature model; the personal audio features obtained by the personal audio feature model and the cough audio are input to the cough audio analysis and identification module then performs convolutional neural network operations to establish a cough audio analysis and identification model. When performing disease identification, pre-input audio into the personal audio feature model to obtain personal audio features, then input real-time cough audio and the personal audio features to the cough audio analysis and identification model to identify the corresponding respiratory disease.

Description

Cough sound recognition method and system

The present invention relates to a cough recognition method and system thereof, in particular to a method that uses convolutional neural networks (CNN) operations to analyze individual voices and their coughs, and then correctly identify the corresponding coughs. Identification method and system for respiratory diseases.

Cough is a common symptom of the respiratory tract, which is caused by inflammation, foreign body, physical or chemical stimulation of the trachea, bronchial mucosa or pleura. Cough is a physiological symptom of various cough diseases, and different cough diseases exhibit different cough characteristics.

In medicine, experienced doctors can diagnose coughing diseases based on the characteristics of the patient's coughing sound. Common coughing diseases and their characteristics include:

1. Pure dry cough or pure wet cough-postnasal drip syndrome.

2. A dry cough that ends with a wheezing sound - asthma.

3. Roaring cough with high pitch - acute laryngitis

4. Cough with a clicking sound - COPD.

5. Dry cough weak but short-pneumonia.

6. Dry cough with spasticity - pertussis.

7. Single cough - upper airway inflammation.

Since it takes a certain amount of experience to judge the type of respiratory disease corresponding to a cough by cough audio, generally only experienced doctors can make an accurate judgment, and people with no experience or less experience cannot judge the corresponding respiratory disease based on cough audio. disease.

In view of this, how to establish a technology that does not require manual identification, so that it can directly and correctly identify the corresponding respiratory diseases by directly using the cough sound, will be the goal that the relevant medical industry hopes to achieve. Therefore, the inventors of the present invention conceived and designed a cough recognition method and its system to improve the deficiencies of the conventional technology, thereby enhancing the implementation and utilization in the industry.

In view of the above-mentioned problems in the prior art, the purpose of the present invention is to provide a cough recognition method and system thereof, so as to solve the conventional problem of insufficient accuracy of manual interpretation and difficulty in automation.

According to one object of the present invention, a cough recognition method is proposed, which includes the following steps: Step S1: input a plurality of training audio and its corresponding training cough audio through an input device, and store them in a storage device; Step S2: borrow The storage device is accessed by the processor, and the plurality of training audios are converted into a plurality of audio signals; Step S3: The processor inputs the plurality of audio signals to the personal audio feature extraction module to perform convolutional neural network calculations to Establish a personal audio feature model and obtain a plurality of personal audio features; Step S4: Input the plurality of personal audio features and the corresponding plurality of training cough audio to the cough audio analysis and identification module through the processor to perform convolutional neural network calculations to establish a cough audio analysis and identification model; Step S5: Input the audio of the person to be identified and the audio of the cough to be identified through the input device, and perform the interpretation program through the processor, and interpret the corresponding respiratory disease type according to the personal audio feature model and the cough audio analysis and identification model; Step S6: Pass The output device accesses the storage device and outputs the types of respiratory diseases that have been interpreted and analyzed.

According to another object of the present invention, a cough recognition system is provided, which includes an input device, a storage device, a processor, and an output device. Among them, the input device is used to input a plurality of training audio and its corresponding plurality of training cough audio, the audio of the individual to be identified and its audio of cough to be identified; the storage device is connected to the input device and the output device to store the plurality of training audio The audio and its corresponding multiple training cough audio, the audio of the individual to be identified and the audio of the cough to be identified; the output device is connected to the storage device to output the types of respiratory diseases that have been interpreted and analyzed; the processor is connected to the storage device to execute multiple Instructions are used to perform the following steps: convert a plurality of training audios into a plurality of audio signals, and input the plurality of audio signals to the personal audio feature extraction module to perform convolutional neural network operations to establish a personal audio feature model, and obtain A plurality of personal audio features; the plurality of personal audio features and their corresponding plurality of training cough audio (also converted into a plurality of training cough audio signals) are input to the cough audio analysis and identification module for convolutional neural network Calculation to establish a cough audio analysis and identification model; according to the personal audio feature model and the cough audio analysis and identification model, the audio of the person to be identified and the cough audio are interpreted to analyze the corresponding type of respiratory disease.

Preferably, the plurality of audio signals and the plurality of training cough audio signals may be Mel-Frequency Cepstral Coefficients (MFCC); Mel-Frequency Cepstral Coefficients are a group used to establish Mel The key coefficients of the Mel cepstrum, from the segment of the sound signal, a group of cepstrums (Cepstrum) that can represent the sound signal can be obtained, and the Mel cepstrum coefficients are the cepstrum derived from the cepstrum. Different from the general cepstrum, the characteristic of the Mel cepstrum is that the frequency bands on it are uniformly distributed on the Mel scale, that is, compared with the generally seen, linear cepstrum representation method, this type of frequency band is different from the The human nonlinear auditory system is much closer. For example: In audio compression technology, Mel cepstrum is often used for processing.

Based on the above, using the cough sound recognition method and system of the present invention, it is possible to quickly and conveniently know the type of respiratory disease that the patient is suffering from. Through this method, the type of respiratory disease can be identified, which can assist clinical interpretation and improve the accuracy of subsequent diagnosis results. Correct rate.

1: Personal audio to be identified

2: Cough audio to be identified

3: Personal audio feature model

4: Convert to the cough audio signal to be identified

5: Cough audio analysis and identification model

6: Types of respiratory diseases

7: Training Audio

8: Training cough audio

9: Convert to training cough audio signal

10: Complex convolution layer

11: Complex long and short memory layers

12: Fully connected layer

13: Complex fully connected layer

20: Cough sound recognition system

21: Input device

22: storage device

23: Processor

24: output device

A:Audio input

B: Feature extraction and analysis and identification

C: Cough sound audio analysis and identification module

S1~S6: steps

In order to make the technical features, content and advantages of the present invention and the effects that can be achieved more obvious, the present invention is hereby combined with the accompanying drawings, and described in detail in the form of embodiments as follows: The first figure is an embodiment of the present invention The flow chart of the steps of the cough recognition method; Figure 2 is a schematic block diagram of the cough recognition method of the embodiment of the present invention; Figure 3 is the personal audio feature extraction module training the personal audio feature model of the embodiment of the present invention Figure 4 is a schematic diagram of the cough audio analysis and recognition module training cough audio analysis and recognition model of the embodiment of the present invention; Figure 5 is a schematic diagram of the cough recognition system of the embodiment of the present invention.

In order for the Ligui Examiner to understand the technical features, content and advantages of the present invention and the effects it can achieve, the present invention is hereby combined with the accompanying drawings and described in detail in the form of an embodiment as follows, and the drawings used therein, its The subject matter is only for illustration and auxiliary instructions, and not necessarily the true proportion and precise configuration of the present invention after implementation, so it should not be interpreted based on the proportion and configuration relationship of the attached drawings, and limit the scope of rights of the present invention in actual implementation. Together first describe.

Unless otherwise defined, all terms (including technical and scientific terms) used herein have the meaning as commonly understood by one of ordinary skill in the art to which this invention belongs. It will be further understood that terms such as those defined in commonly used dictionaries should be interpreted to have meanings consistent with their meanings in the context of the relevant art and the present invention, and will not be interpreted as idealized or excessive formal meaning, unless expressly so defined herein.

Please refer to Fig. 1, Fig. 2 and Fig. 4 together. Fig. 1 is a flow chart of the steps of the cough recognition method of the embodiment of the present invention; Fig. 2 is a flowchart of the cough recognition method of the embodiment of the present invention Block diagram; and Figure 4 is a schematic diagram of the cough audio analysis and recognition module training cough audio analysis and recognition model of the embodiment of the present invention. As shown in Figure 1, the cough recognition method includes the following steps (S1~S6):

Step S1: Input a plurality of training audios 7 and corresponding training cough audios 8 through an input device, and store them in a storage device.

Input a plurality of training audio 7 and its corresponding training cough audio 8 through the input device, and input them into the storage device of the system. The input device described here is an audio collection device, such as a microphone, or an audio collection device. Electronic devices such as smartphones, tablets, Notebook computers, cameras, etc., but not limited to, any device that can capture audio can be used as an input device.

Step S2: Convert the plurality of training audio signals 7 into a plurality of audio signals through the processor accessing the storage device.

This step is to convert the training audio 7 into a specific audio signal, preferably the Mel cepstrum coefficient, because the Mel cepstrum is closer to the human nonlinear auditory system, and it will be used as the effect of subsequent establishment of a personal audio feature model more significant.

Step S3: Input the plurality of audio signals to the personal audio feature extraction module through the processor to perform convolutional neural network calculations to establish a personal audio feature model 3 and obtain a plurality of personal audio features.

The personal audio feature extraction module includes a plurality of convolutional layers, a plurality of long short-term memory layers, and a plurality of fully-connected layers, and each layer includes A trigger function.

Through the personal audio feature extraction module, the audio of different people can be input and then the convolutional neural network operation is performed, so that the audio signal is mapped to a high-dimensional continuous feature space (latent space), and the high-dimensional continuous The feature space is the personal audio feature model 3, which is a high-dimensional continuous feature space with multiple high-dimensional vectors (latent vectors), and the multiple high-dimensional vectors are the personal audio features of different people; When training the convolutional neural network model, you can use but not limited to Euclidean distance to minimize the distance between high-dimensional vectors obtained from different audios belonging to the same person, and at the same time maximize the distance between audio features belonging to different people. distance.

Please also refer to FIG. 3 . FIG. 3 is a schematic diagram of a personal audio feature extraction module training a personal audio feature model 3 according to an embodiment of the present invention. It can be seen from the figure that the audio A1 sent by individual A to An is n independent audio and the audio content does not have to be the same (such as different sentences or sounds), and the personal audio feature extraction module needs to map the audio from A1 to An to similar regions in the high-dimensional continuous feature space; Similarly, m audio B1 to Bm provided by another person B also needs to be mapped to a nearby area. On the other hand, during the training process of the personal audio feature extraction module, an additional loss function (loss function) will be used to maximize the difference between different people's audio, so it has a higher accuracy than traditional recognition methods. Discrimination. When training the model, multiple audios from different people will be used, and each person will provide multiple audios covering different content for training the model.

Step S4: Input the multiple personal audio features and the corresponding multiple training cough audios 8 to the cough audio analysis and identification module C through the processor to perform convolutional neural network calculations to establish cough audio analysis and identification Model 5.

The cough audio analysis and recognition module C includes a plurality of convolutional network layers 10, a plurality of long-term short-term memory layers 11 and a plurality of fully connected layers 13 (one of which is independently a single fully connected layer 12), and each layer also includes A trigger function.

Referring to Fig. 4 again, it can be seen from the figure that the cough sound audio analysis and recognition module C includes two kinds of input data, one is a plurality of personal audio characteristics obtained in the personal audio characteristic model 3 established in step S3; and the other The other is the plurality of training cough audio 8 corresponding to the plurality of personal audio features, and the plurality of training cough audio 8 is also converted into a plurality of training cough audio signals 9, and the plurality of training cough audio The signal 9 is preferably Mel cepstral coefficients; that is, the personal audio characteristics of individual A and its cough audio signal, the personal audio characteristics of individual B and its cough audio signal, etc., and so on; wherein, the plurality of training The cough audio signal 9 passes through a plurality of convolutional network layers 10, a plurality of long-term short-term memory layers 11 and a single fully connected layer 12, and then passes through the remaining plurality of fully connected layers 13 together with the plurality of personal audio features. Training, and finally establish a cough audio analysis and recognition model 5.

Step S5: Input the personal audio 1 to be identified and the cough audio 2 to be identified through the input device, and execute the interpretation program through the processor, and interpret the corresponding respiratory disease type 6 according to the personal audio feature model 3 and the cough audio analysis and identification model 5 .

Input the personal audio 1 to be identified into the personal audio feature model 3 to obtain a personal audio feature, and then combine the personal audio feature with the cough audio 2 to be identified (that is, the cough audio of the same person, which can be recorded in real time or pre-recorded) It is converted into a cough audio signal 4 to be identified, and is input to the cough audio analysis and identification model 5 for interpretation, so as to obtain the corresponding respiratory disease type 6 . Using the personal audio features as the personalized correction information for subsequent identification of respiratory diseases can greatly improve the accuracy of identification of cough diseases.

Step S6: Access the storage device through the output device, and output the types of respiratory diseases that have been interpreted and analyzed.

The identification result 6 of the type of respiratory disease analyzed by the cough audio analysis and identification model 5 is read from the storage device through the output device to display the corresponding type of respiratory disease; the output device may include various display interfaces, such as computer screens, monitors Or handheld device display, etc.

Please refer to FIG. 5, which is a schematic diagram of a cough recognition system according to an embodiment of the present invention. As shown in the figure, the cough recognition system 20 may include an input device 21 , a storage device 22 , a processor 23 and an output device 24 . The input device 21 may include various audio collection devices, such as a microphone or an electronic device with audio collection function, such as a smart phone, a tablet computer, a notebook computer, a camera, etc., and transmit a plurality of training audio and its corresponding A plurality of training cough audios, individual audios to be identified and cough audios to be identified are stored in the memory of the storage device 22. The memory can include read-only memory, flash memory, disk or cloud database, etc. .

Then, the cough recognition system 20 accesses the storage device 22 through the processor 23. The processor 23 may include a central processing unit, an image processor, a microprocessor, etc. in a computer or a server, and it may include a multi-core A processing unit or a combination of multiple processing units. The processor 23 executes instructions to access the plurality of training audios 7 in the storage device 22 and the corresponding plurality of training cough audios 8 to perform convolutional neural network operations to obtain the personal audio feature model 3 and cough audio analysis and identification Model 5; thereafter, the processor 23 executes instructions to access the personal audio 1 to be identified and the cough audio 2 to be identified in the storage device 22, and use the personal audio feature model 3 and the cough audio analysis and identification model 5 to identify the individual The audio 1 and the cough audio 2 to be identified are interpreted to obtain a corresponding identification result 6 of the type of respiratory disease; finally, the output device 24 accesses the storage device 22 to output the identification result 6 of the type of respiratory disease obtained through interpretation, and outputs The device 24 may include various display interfaces, such as, but not limited to, a computer screen, a display, or a display of a handheld device.

To sum up, using the cough sound recognition method and system of the present invention, the type of respiratory disease that the patient is suffering from can be quickly and conveniently known. By identifying the type of respiratory disease in this way, it can assist clinical interpretation and improve the accuracy of subsequent diagnosis results. Correct rate.

The above descriptions are illustrative only, not restrictive. Any equivalent modification or change made without departing from the spirit and scope of the present invention shall be included in the scope of the appended patent application.

S1~S6: steps

Claims (10)

A cough recognition method, the following steps: Step S1: input a plurality of training audios and corresponding training cough audios through an input device, and store them in the storage device; Step S2: use a processor to access the storage device, convert the plurality of training audio into a plurality of audio signals; step S3: input the plurality of audio signals to a person audio feature extraction module through the processor to perform convolutional neural network calculations to create a person The audio feature model obtains a plurality of personal audio features; step S4: input the plurality of personal audio features and the plurality of training cough audio to a cough audio analysis and identification module through the processor to perform convolutional neural network to establish a cough audio analysis and identification model; step S5: input a personal audio to be identified and a cough audio to be identified through the input device, and perform an interpretation program by the processor, according to the personal audio feature model and The cough audio analysis and recognition model interprets a type of respiratory disease corresponding to it; and step S6: access the storage device through an output device, and output the type of respiratory disease that has been interpreted and analyzed, wherein the personal audio frequency described in step S3 The feature extraction module includes a plurality of convolutional layers, a plurality of long short-term memory layers and a plurality of fully-connected layers, and each layer includes a The trigger function is to perform a convolutional neural network operation on the plurality of audio signals by the personal audio feature extraction module, so that the plurality of audio signals are mapped to a high-dimensional continuous feature space (latent space), The high-dimensional continuous feature space is the personal audio feature model, and has a plurality of high-dimensional vectors (latent vector), the plurality of The high-dimensional vectors are the plurality of personal audio features. The cough recognition method as described in Claim 1, wherein the plurality of training cough audio systems described in step S4 are converted into a plurality of training cough audio signals, and then input together with the plurality of personal audio features into the The cough audio analysis and identification module performs convolutional neural network operations to establish the cough audio analysis and identification model. The cough recognition method according to claim 1, wherein the plurality of audio signals in step S2 are Mel cepstral coefficients. The cough recognition method according to claim 2, wherein the plurality of training cough audio signals in step S4 are Mel cepstrum coefficients. The cough recognition method according to any one of claim 1 to claim 4, wherein the cough audio to be recognized in step S5 is real-time recording or pre-recording. A cough recognition system, which includes: an input device, which is used to input a plurality of training audio and its corresponding plurality of training cough audio, a personal audio to be identified, and a cough audio to be identified; a storage device connected to The input device is used to store the plurality of training audios and the corresponding plurality of training cough audios, the audio to be identified and the audio to be identified; an output device is connected to the storage device and is The type output of the respiratory disease for interpretation and analysis; and a processor, connected to the storage device, for executing a plurality of instructions to perform the following steps: convert the plurality of training audio into a plurality of audio signals, and convert the plurality of An audio signal is input to a human audio feature extraction module for convolutional neural network operations to establish A human audio feature model obtains a plurality of personal audio features; the plurality of personal audio features and the plurality of training cough audio are input to a cough audio analysis and identification module for convolutional neural network operations to create a cough Sound audio analysis and identification model; and according to the personal audio feature model and the cough audio analysis and identification model, interpret the personal audio to be identified and the cough audio to be identified to analyze the corresponding type of respiratory disease, wherein the personal audio feature The capture module includes a plurality of convolutional layers, a plurality of long short-term memory layers, and a plurality of fully-connected layers, and each layer contains a trigger The function uses the personal audio feature extraction module to perform convolutional neural network operations on the multiple audio signals, so that the multiple audio signals are mapped (mapped) to a high-dimensional continuous feature space (latent space), the The high-dimensional continuous feature space is the personal audio feature model, and has multiple high-dimensional vectors (latent vectors), and the multiple high-dimensional vectors are the multiple personal audio features. The cough recognition system as described in claim 6, wherein the plurality of training cough audio systems are converted into a plurality of training cough audio signals, and then input to the cough audio analysis and identification module together with the plurality of personal audio features The group performs convolutional neural network operations to establish the cough audio analysis and identification model. The cough recognition system as claimed in claim 6, wherein the plurality of audio signals are Mel cepstral coefficients. The cough recognition system according to claim 7, wherein the plurality of training cough audio signals are Mel cepstral coefficients. The cough recognition system according to any one of claim 6 to claim 9, wherein The cough audio to be identified is recorded in real time or pre-recorded.

Images