JP2023174854A - Respiratory sound analysis device, respiratory sound analysis method and computer program - Google Patents

Abstract

To classify and output a plurality of sound types included in respiratory sound.SOLUTION: A respiratory sound analysis device includes: classification means for classifying respiratory sound into a plurality of sound types; input means for receiving an input for selecting a type of the respiratory sound to be output; output means for outputting the respiratory sound of the sound type selected according to the input received by the input means, as a spectral image; and change means capable of changing a color of the spectral image for every sound type selected according to the input received by the input means.SELECTED DRAWING: Figure 1

Description

The present invention relates to the technical field of a breathing sound analysis device, a breathing sound analysis method, and a computer program that analyze breathing sounds including a plurality of sound types.

As this type of device, one is known that distinguishes between normal breathing sounds and abnormal breathing sounds of a living body detected by an electronic stethoscope or the like. For example, Patent Document 1 proposes a technique of outputting the analysis results of abnormal sounds on a three-dimensional display device. Patent Document 2 proposes a technique for detecting rales as abnormal sounds included in breathing sounds. Patent Document 3 proposes a technique in which a sound waveform is divided into a plurality of sections and the type of sound is discriminated.

Japanese Patent Application Publication No. 2002-538921 Japanese Patent Application Publication No. 2009-106574 Japanese Patent Application Publication No. 2013-123494

However, the techniques described in Patent Documents 1 to 3 mentioned above have a technical problem in that they cannot arbitrarily output a plurality of sound types included in breathing sounds.

Examples of the problems to be solved by the present invention include those mentioned above. An object of the present invention is to provide a breathing sound analysis device, a breathing sound analysis method, and a computer program that can separate and output a plurality of sound types included in breathing sounds.

A breathing sound analysis device for solving the above problems includes a classification means for classifying breathing sounds into a plurality of sound types, an input means for accepting input for selecting the type of breath sounds to be output, and the input means. output means for outputting a sound type of breathing sound selected according to the received input as a spectral image; and a color of the spectral image can be changed for each sound type selected according to the input received by the input means. and a changing means.

A breathing sound analysis method for solving the above problems includes a classification step of classifying breathing sounds into a plurality of sound types, an input step of accepting input for selecting the type of breathing sounds to be output, and the input means. an output step of outputting a sound type of breathing sound selected according to the received input as a spectral image; and a color of the spectral image can be changed for each sound type selected according to the input received by the input means. and a change step.

A computer program for solving the above-mentioned problems includes a classification step of separating breath sounds into a plurality of sound types, an input step of accepting input for selecting the type of breath sounds to be output, and a step of accepting input by the input means to select the type of breath sounds to be output. an output step of outputting the sound type of breath sound selected according to the input as a spectral image; and a changing step of changing the color of the spectral image for each sound type selected according to the input received by the input means. and make the computer execute it.

FIG. 1 is a block diagram showing the overall configuration of a breathing sound analysis device according to the present embodiment. 3 is a flowchart showing the operation of the breathing sound analysis device according to the present embodiment. FIG. 3 is a spectrogram diagram showing frequency analysis results of breathing sounds including crepitus sounds. FIG. 3 is a spectrogram diagram showing frequency analysis results of breathing sounds including whistling sounds. It is a graph showing the spectrum of breathing sounds including crepitus sounds at a predetermined timing. FIG. 2 is a conceptual diagram showing a method of approximating the spectrum of breathing sounds including crepitus sounds. It is a graph showing the spectrum of breathing sounds including whistling sounds at a predetermined timing. FIG. 2 is a conceptual diagram showing a method of approximating the spectrum of breathing sounds including whistling sounds. It is a graph showing an example of a frequency analysis method. It is a figure showing an example of a frequency analysis result. It is a conceptual diagram which shows the peak detection result of a spectrum. It is a graph showing normal alveolar breath sound base. It is a graph showing a crepitation base. It is a graph showing a continuous rales base. It is a graph showing a white noise basis. FIG. 7 is a graph showing a frequency-shifted continuous rales base; FIG. It is a figure which shows the relationship between a spectrum, a basis, and a coupling coefficient. FIG. 3 is a diagram showing an example of an observed spectrum and a basis used for approximation. It is a figure which shows each base and coupling coefficient which show a spectrum. FIG. 7 is a conceptual diagram (part 1) showing a continuous rales classification method according to a first modification. FIG. 7 is a conceptual diagram (Part 2) showing a continuous rales classification method according to the first modification. It is a graph which shows the threshold value used for classification of the whistle sound and the snoring sound based on a 2nd modification. It is a graph which shows the initial value of the threshold value used for classification of the whistle sound and the snoring sound based on a 3rd modification. It is a graph (part 1) showing the adjusted value of the threshold value used for classification between the whistle sound and the snoring sound according to the third modification. It is a graph (part 2) showing the adjusted value of the threshold value used for classification between the whistle sound and the snoring sound according to the third modification. It is a spectrogram diagram of breathing sounds including whistling sounds. It is a graph which shows the peak frequency and the number of peaks of a whistle sound. It is a spectrogram diagram of breathing sounds including rhinoid sounds. It is a graph showing the peak frequency and the number of peaks of sonar sounds. FIG. 3 is a plan view showing an example of display on the display unit. It is a spectrogram diagram showing extraction results for each sound type. FIG. 2 is a conceptual diagram (Part 1) showing an example of audio output for each classified sound type. FIG. 7 is a conceptual diagram (part 2) showing an example of audio output for each classified sound type. FIG. 3 is a conceptual diagram showing a volume adjustment method for each classified sound type. FIG. 2 is a conceptual diagram showing a volume adjustment method for each frequency band. FIG. 2 is a conceptual diagram showing an example of image processing performed for each sound type. FIG. 3 is a plan view showing an example of an image generated by superimposing images subjected to image processing for each sound type. FIG. 3 is a conceptual diagram showing a display color adjustment method for each classified tone type.

<1>
The respiratory sound analysis device according to the present embodiment includes a classification means for classifying respiratory sounds into normal sounds and abnormal sounds, and an output means for outputting any breath sound among the classified respiratory sounds.

According to the breathing sound analysis device according to the present embodiment, during its operation, breathing sounds are first classified into normal sounds and abnormal sounds. The respiratory sounds separated by the classification means do not have to be two sounds, normal sounds and abnormal sounds, and each of the normal sounds and abnormal sounds may be further separated into a plurality of sound types. For example, abnormal sounds may be classified into whistling sounds, snoring sounds, crepitus sounds, etc., respectively. Note that there are no particular limitations on the specific classification method, as long as the classification can be performed in a state that enables the output of any breath sounds described below.

When the breath sounds are separated, any breath sound among the separated breath sounds is output. That is, a plurality of separated breath sounds are selectively output. Thereby, for example, it is possible to output only the breathing sounds desired by the user. More specifically, for example, only whistle sounds included in breathing sounds, or only whistle sounds and snoring sounds can be output. Note that the output mode is not particularly limited, and may be output as audio or image, or may be output in other formats.

By separating and outputting breathing sounds as described above, it is possible to easily diagnose, for example, a health condition. Specifically, for example, when multiple breathing sounds are heard mixed together, it is difficult for even experienced doctors to distinguish between each type of sound. It becomes possible to easily distinguish the types of sounds included. In this way, if only specific types of sounds included in breathing sounds can be made easier to hear, it can also be utilized in physician education and research.

As explained above, according to the breathing sound analysis device according to the present embodiment, it is possible to separate the breathing sounds and output any desired breathing sounds, so that it is possible to suitably analyze the breathing sounds that include a plurality of sound types. It is possible.

<2>
In one aspect of the breathing sound analysis device according to the present embodiment, the output means simultaneously outputs a plurality of breath sounds among the classified breath sounds.

According to this aspect, since a plurality of breathing sounds can be output simultaneously (in other words, in a superimposed state) as arbitrary breathing sounds, desired breathing sounds can be output in an appropriate combination.

<3>
In another aspect of the breathing sound analysis device according to the present embodiment, the output means outputs the arbitrary breathing sound as a voice or a spectral image.

According to this aspect, any breath sound among the separated breath sounds is output as audio using, for example, a speaker or headphones. Alternatively, any breath sounds are output as a spectral image using a display such as a liquid crystal monitor. Therefore, any outputted breathing sound can be suitably used.

<4>
In another aspect of the breathing sound analysis device according to the present embodiment, the apparatus further includes a changing means capable of changing the output state of the arbitrary breathing sounds for each sound type.

According to this aspect, since the output state (for example, output volume, image display mode, etc.) of any breathing sound can be changed for each sound type, any outputted breathing sound can be used more suitably. For example, by increasing the volume and outputting only a specific abnormal sound included in the breathing sounds, it is possible to make the specific abnormal sound easier to hear even when a plurality of breathing sounds are mixed. Furthermore, even if the volume is set back to normal once it has been set to an easy-to-listen state, it is considered that the state will become easy-to-listen from then on. Therefore, it can be effectively used for training of inexperienced doctors.

<5>
In an embodiment further including the above-mentioned changing means, the changing means may be capable of changing the output volume of the arbitrary breathing sound for each sound type.

In this case, since the output volume can be changed for each sound type, convenience can be improved by making it easier to listen to only desired breathing sounds.

<6>
In the aspect described above in which the output volume can be changed for each sound type, the changing means may be able to change the output volume of the arbitrary breathing sound for each predetermined frequency band.

In this case, since the output volume for each sound type can be changed in more detail, for example, only the output volume of a predetermined frequency band can be increased to make it easier to hear the desired breathing sound. Note that the predetermined frequency band may be set according to the characteristics of the sound types to be classified.

<7>
Alternatively, in an embodiment further including a changing means, the changing means may be able to perform predetermined image processing for each sound type on the image showing the arbitrary breathing sound.

In this case, it is possible to perform predetermined image processing on an image showing arbitrary breathing sounds (for example, a spectrogram or the like in which only one type of breathing sound is extracted) so that the image can be easily recognized visually. Examples of the predetermined image processing include color change (for example, RGB adjustment), binarization, edge detection, and the like.

<8>
In an embodiment in which the above-described image processing can be performed for each sound type, the changing means may be capable of outputting images obtained by performing the predetermined image processing for each sound type by superimposing them for a plurality of sound types. .

In this case, multiple images of each sound type that have been separately image-processed can be superimposed and displayed, so multiple sound types that have been displayed in an appropriate manner through image processing can be recognized together in one image. Comparisons between a plurality of sound types can be suitably performed.

<9>
In another aspect of the breath sound analysis device according to the present embodiment, the classification means includes an acquisition means for acquiring information regarding a frequency corresponding to a predetermined feature of a spectrum of breath sounds, and a reference for classifying the breath sounds. a shifting means for shifting a plurality of reference spectra according to the information regarding the frequency to obtain a frequency-shifted reference spectrum; and ratio output means for outputting the ratio of the reference spectrum.

According to this aspect, the classification means first obtains information regarding frequencies corresponding to predetermined features of the spectrum of breath sounds. Note that the "predetermined feature" here refers to a feature that occurs at a specific frequency depending on the type of sound included in the spectrum of body sounds, such as a peak that appears in a frequency-analyzed signal. . Furthermore, "information regarding frequency" is not limited to information that directly indicates a frequency, but includes information that allows the frequency to be derived indirectly.

When the information regarding the frequency is acquired, a plurality of reference spectra that serve as standards for classifying breath sounds are shifted according to the information regarding the frequency, and a frequency-shifted reference spectrum is acquired. Note that the "reference spectrum" here refers to a spectrum that is used to classify multiple sound types included in breathing sounds (for example, normal breathing sounds, continuous rales, crepitus sounds, etc.). This is a preset spectrum. The reference spectrum is frequency-shifted according to, for example, a peak position, which is a predetermined feature obtained from the breath sounds, and is used as a frequency-shifted reference spectrum.

When the frequency-shifted reference spectrum is acquired, the ratio of the plurality of reference spectra included in the breathing sound is output based on the breathing sound and the frequency-shifted reference spectrum. Specifically, the ratio of sound types corresponding to a plurality of reference spectra included in the breath sound to be analyzed is calculated, and the result is output. More specifically, for example, a calculation based on a plurality of reference spectra is performed on the spectrum of a breathing sound, and the ratio of the reference spectra is calculated as a coupling coefficient.

As a result of the above, according to the classification means according to the present embodiment, breath sounds including a plurality of sound types can be suitably classified. In this embodiment, in particular, even when a plurality of breath sounds are mixed on the same frequency axis, the proportion of each sound type can be suitably distinguished.

<10>
In the embodiment using the frequency-shifted reference spectrum described above, the predetermined feature may be a local maximum value.

In this case, for example, frequency analysis using fast Fourier transform (FFT) is performed on a signal indicating breathing sounds, and information regarding the frequency corresponding to the local maximum value (i.e., peak) of the analysis result is obtained. Ru. Note that information regarding the frequency is acquired as corresponding to the position of the local maximum value, but even if the frequency does not completely match the position of the local maximum value, it is acquired as information regarding the frequency corresponding to a position near the local maximum value. Good too.

As described above, by using the maximum value as a predetermined feature of the spectrum of breath sounds, information regarding the frequency can be obtained more easily and accurately.

<11>
The breathing sound analysis method according to the present embodiment includes a classification step of classifying breathing sounds into normal sounds and abnormal sounds, and an outputting step of outputting any breath sounds among the classified breath sounds.

According to the breathing sound analysis method according to this embodiment, similar to the above-described breathing sound analysis device according to this embodiment, it is possible to suitably analyze breathing sounds including a plurality of sound types.

Note that the breathing sound analysis method according to the present embodiment can also adopt various aspects similar to the various aspects of the breathing sound analysis device according to the present embodiment described above.

<12>
The computer program according to the present embodiment causes a computer to execute a classification step of classifying breathing sounds into normal sounds and abnormal sounds, and an output step of outputting any breathing sound among the classified breathing sounds.

According to the computer program according to the present embodiment, it is possible to cause the computer to perform the same processing as the breathing sound analysis method according to the present embodiment described above, so that breathing sounds including a plurality of sound types can be suitably analyzed.

Note that the computer program according to the present embodiment can also adopt various aspects similar to the various aspects of the breath sound analysis device according to the present embodiment described above.

<13>
The computer program described above is recorded on the recording medium according to this embodiment.

According to the recording medium according to the present embodiment, by causing a computer to execute the above-described computer program, it is possible to suitably analyze breathing sounds including a plurality of sound types.

The effects and other benefits of the breathing sound analysis device and the breathing sound analysis method, as well as the computer program and recording medium according to this embodiment will be explained in more detail in the examples shown below.

Hereinafter, embodiments of a breathing sound analysis device, a breathing sound analysis method, a computer program, and a recording medium will be described in detail with reference to the drawings.

<Overall configuration>
First, the overall configuration of a breathing sound analysis device according to this embodiment will be described with reference to FIG. 1. FIG. 1 is a block diagram showing the overall configuration of a breathing sound analysis device according to this embodiment.

In FIG. 1, the respiratory sound analysis device according to the present embodiment includes a body sound sensor 110, a signal storage section 120, a signal processing section 125, an audio output section 130, and a base holding section 140 as main components. , a display section 150, an input section 160, and a processing section 200.

The body sound sensor 110 is a sensor configured to be able to detect the breathing sounds of a living body. The body sound sensor 110 includes, for example, an ECM (Electret Condenser Microphone), a piezo-based microphone, a vibration sensor, and the like.

The signal storage unit 120 is configured as a buffer such as a RAM (Random Access Memory), and temporarily stores a signal indicating breathing sounds detected by the body sound sensor 110 (hereinafter referred to as a “breathing sound signal”). to be memorized. The signal storage unit 120 is configured to be able to output the stored signals to the audio output unit 130 and the processing unit 200, respectively.

The signal processing unit 125 processes the sound acquired by the body sound sensor 110 and outputs the processed sound to the audio output unit 130. The signal processing unit 125 functions, for example, as an equalizer or a filter, and processes the acquired sound into a state that is easy for people to listen to.

The audio output unit 130 is configured as, for example, a speaker or a headphone, and outputs the breathing sound detected by the body sound sensor 110 and processed by the signal processing unit 125.

The base holding unit 140 is configured as, for example, a ROM (Read Only Memory), and stores bases corresponding to predetermined types of sounds that may be included in breath sounds. Note that the base according to this example is an example of the "reference spectrum" of the present invention.

The display unit 150 is configured as, for example, a display such as a liquid crystal monitor, and displays image data output from the processing unit 200.

The input unit 160 is a device that receives input from a user, and is configured as, for example, a keyboard, a mouse, a touch panel, various switches, and the like. The input unit 160 is configured to allow an input operation for selecting at least a breathing sound to be output.

The processing unit 200 is configured to include a plurality of arithmetic circuits, memories, and the like. The processing section 200 includes a frequency analysis section 210, a frequency peak detection section 220, a basis set generation section 230, a coupling coefficient calculation section 240, a signal strength calculation section 250, an image generation section 260, and a breathing sound selection section 270.

The operation of each part of the processing section 200 will be described in detail later.

<Operation explanation>
Next, the operation of the breathing sound analysis device according to this embodiment will be explained with reference to FIG. 2. FIG. 2 is a flowchart showing the operation of the breath sound analysis device according to this embodiment. Here, a brief explanation will be given to understand the overall flow of processing executed by the breathing sound analysis device according to this embodiment. Details of each process will be described later.

In FIG. 2, when the breathing sound analysis device according to the present embodiment operates, first, breathing sounds are detected by the body sound sensor 110, and a breathing sound signal is acquired by the processing unit 200 (step S101).

When the breathing sound signal is acquired, frequency analysis (for example, fast Fourier transform) is performed in the frequency analysis unit 210 (step S102). Furthermore, the frequency peak detection unit 220 detects a peak (maximum value) using the frequency analysis result.

Subsequently, a basis set is generated in the basis set generation unit 230 (step S103). Specifically, the basis set generating section 230 generates a basis set using the bases stored in the basis holding section 140. At this time, the basis set generation unit 230 shifts the basis based on the peak position (that is, the corresponding frequency) obtained from the frequency analysis result.

Once the basis set is generated, the coupling coefficient calculation unit 240 calculates a coupling coefficient based on the frequency analysis result and the basis set (step S104).

Once the coupling coefficient is calculated, the signal strength calculation unit 250 calculates the signal strength according to the coupling coefficient (step S105). In other words, the proportion of each sound type included in the breathing sound signal is calculated.

Once the signal strength is calculated, the image generation unit 260 generates image data indicating the signal strength. The generated image data is displayed as an analysis result on the display unit 150 (step S106).

After displaying the analysis results, when the user inputs the sound type to be output (step S107: YES), the breathing sound selection unit 270 selects the breathing sound to be output, and the selected sound type is output to the audio output unit 130. Alternatively, it is output to the display unit 150 (step S108).

Thereafter, a determination is made as to whether or not to continue the analysis process (step S109). If it is determined that the analysis process is to be continued (step S109: YES), the process from step S101 is executed again. If it is determined not to continue the analysis process (step S109: NO), the series of processes ends.

<Specific example of breathing sound signal>
Next, a specific example of a breathing sound signal analyzed by the breathing sound analysis apparatus according to this embodiment will be described with reference to FIGS. 3 and 4. FIG. 3 is a spectrogram diagram showing the frequency analysis results of breathing sounds including crepitus sounds, and FIG. 4 is a spectrogram diagram showing the frequency analysis results of breathing sounds including whistling sounds.

In the example shown in FIG. 3, in addition to the spectrogram pattern corresponding to normal breathing sounds, a spectrogram pattern corresponding to crepitus sound, which is one of abnormal breathing sounds, is observed. The spectrogram pattern corresponding to the crepitus sound has a shape close to a rhombus, as shown in the enlarged part in the figure.

In the example shown in FIG. 4, in addition to the spectrogram pattern corresponding to normal breathing sounds, a spectrogram pattern corresponding to a whistling sound, which is one of abnormal breathing sounds, is observed. The spectrogram pattern corresponding to the whistle sound is shaped like a swan's neck, as shown in the enlarged part of the figure.

In this way, there are multiple types of abnormal breathing sounds, and spectrogram patterns with different shapes are observed depending on the type of sound. However, as can be seen from the figure, normal breathing sounds and abnormal breathing sounds are detected in a mixed state. The breathing sound analysis device according to this embodiment performs an analysis for separating a plurality of sound types mixed together in this way.

<Method of approximating breathing sound signals>
Next, an analysis method using the breath sound analysis apparatus according to this embodiment will be briefly described with reference to FIGS. 5 to 8. FIG. 5 is a graph showing a spectrum of a breathing sound including a crepitus sound at a predetermined timing, and FIG. 6 is a conceptual diagram showing a method of approximating the spectrum of a breathing sound including a crepitus sound. Further, FIG. 7 is a graph showing a spectrum of a breathing sound including a whistling sound at a predetermined timing, and FIG. 8 is a conceptual diagram showing a method of approximating the spectrum of a breathing sound including a whistling sound.

In FIG. 5, when a spectrum is extracted from a breathing sound signal (see FIG. 3) that includes a crepitus sound at a timing when a spectrogram pattern corresponding to a crepitus sound appears strongly, the result shown in the figure is obtained. This spectrum is considered to include normal breathing sounds and crepitus sounds.

In FIG. 6, the spectrum corresponding to normal breathing sounds and the spectrum corresponding to crepitus sounds can be estimated in advance through experiments or the like. Therefore, by using the pre-estimated pattern, it is possible to know in what proportion the above-mentioned spectrum contains components corresponding to normal breathing sounds and components corresponding to crepitation sounds.

In FIG. 7, when a spectrum is extracted from a breathing sound signal (see FIG. 4) that includes a whistling sound at a timing when a spectrogram pattern corresponding to a whistling sound appears strongly, the result shown in the figure is obtained. This spectrum is considered to include normal breathing sounds and whistling sounds.

In FIG. 8, similarly to the normal breathing sounds and crepitus sounds described above, the spectrum corresponding to the whistling sound can also be estimated in advance through experiments or the like. Therefore, by using the pre-estimated pattern, it is possible to know the proportion of components corresponding to normal breathing sounds and components corresponding to whistling sounds included in the above-mentioned spectrum.

Below, each process for realizing such analysis will be explained in more detail.

<Frequency analysis>
Frequency analysis of breathing sound signals and detection of peaks in the analysis results will be described in detail with reference to FIGS. 9 to 11. FIG. 9 is a graph showing an example of a frequency analysis method, and FIG. 10 is a diagram showing an example of a frequency analysis result. Further, FIG. 11 is a conceptual diagram showing the result of spectrum peak detection.

In FIG. 9, frequency analysis is first performed on the acquired breathing sound signal. Frequency determination can be performed using existing techniques such as fast Fourier transform. In this embodiment, the amplitude value for each frequency (ie, amplitude spectrum) is used as the frequency analysis result. Note that the sampling frequency, window size, and window function (for example, Hanning window, etc.) at the time of data acquisition may be determined as appropriate.

As shown in FIG. 10, the frequency analysis result is obtained as being composed of n values. Note that "n" is a value determined by the window size and the like in frequency analysis.

In FIG. 11, peak detection is performed on the spectrum obtained by frequency analysis. In the example shown in the figure, peaks p1 to p4 are detected at positions of 100 Hz, 130 Hz, 180 Hz, and 320 Hz. Note that the peak detection process may be a simple process since it is only necessary to know at which frequency the peak exists. However, it is preferable to set peak detection parameters so that even small peaks are not missed.

In this embodiment, a point having a maximum value is found, and a maximum of N points (N is a predetermined value) are detected in order from the smallest second-order differential value (that is, the largest absolute value) of that point. . The maximum value is determined from the point where the sign of the difference changes from positive to negative. The second-order differential value is approximated by the difference of differences. A maximum of N items whose values are smaller than a predetermined threshold (negative value) are selected in descending order of value, and their positions are stored.

<Generation of basis set>
Next, generation of the basis set will be described in detail with reference to FIGS. 12 to 16. FIG. 12 is a graph showing the basis of normal alveolar breath sounds. Further, FIG. 13 is a graph showing the crepitus base, FIG. 14 is a graph showing the continuous rales base, and FIG. 15 is a graph showing the white noise base. FIG. 16 is a graph showing a frequency-shifted continuous rales basis.

As shown in FIGS. 12 to 15, the base corresponding to each tone type has a unique shape. Note that each base is composed of n numerical values (ie, amplitude values for each frequency), which are the same as the frequency analysis results. Note that each base is normalized so that the area surrounded by the line indicating the amplitude value for each frequency and the frequency axis becomes a predetermined value (for example, 1).

Incidentally, although four bases are shown here: normal alveolar breath sound base, crepitus base, continuous rales base, and white noise base, analysis can be performed even if there is only one base. Furthermore, bases other than those listed here can also be used. Note that if, for example, a basis corresponding to heartbeat sounds or bowel sounds is used instead of the basis corresponding to the breath sounds mentioned here, it becomes possible to analyze the heartbeat sounds or bowel sounds.

In FIG. 16, among the bases described above, the base corresponding to continuous rales is frequency-shifted in accordance with the peak position detected from the result of frequency analysis. Here, an example is shown in which the frequency of the continuous rales basis is shifted in accordance with each of the peaks p1 to p4 shown in FIG. 11. Note that bases other than those corresponding to continuous rales may be frequency shifted.

As a result of the above, the base set is generated as a set of normal alveolar breath sound bases, crepitus sound bases, continuous rales bases corresponding to the number of detected peaks, and white noise bases.

<Calculation of coupling coefficient>
Next, calculation of the coupling coefficient will be explained in detail with reference to FIGS. 17 to 19. FIG. 17 is a diagram showing the relationship between a spectrum, a basis, and a coupling coefficient, and FIG. 18 is a diagram showing an example of an observed spectrum and a basis used for approximation. Further, FIG. 19 is a diagram showing the approximation result by non-negative matrix factorization.

The relationship between the spectrum y to be analyzed, the base h(f), and the coupling coefficient u can be expressed by the following equation (1).

As shown in FIG. 17, the spectrum y and each basis h(f) have n values. On the other hand, the coupling coefficient has m values. Note that "m" is the number of bases included in the base set.

The breathing sound analysis device according to this embodiment uses non-negative matrix factorization to calculate the coupling coefficient of each basis included in the basis set. Specifically, what is necessary is to find u (however, each component value of u is non-negative) that minimizes the optimization criterion function D shown by the following equation (2).

Note that general non-negative matrix factorization is a method of calculating both a basis matrix representing a set of basis spectra and an activation matrix representing a coupling coefficient, but in this example, the basis matrix is fixed and Only the coupling coefficient is calculated.

Incidentally, as a means for calculating the coupling coefficient, an approximation method other than non-negative matrix factorization may be used. However, even in this case, the condition of non-negativeness is desired. Below, we will explain the reason for using the non-negative approximation method using a specific example. As shown in Figure 18, when the coupling coefficient is calculated by approximating the observed spectrum using four bases A to D. think of. Note that the expected coupling coefficient u under the condition that it is non-negative is 1 for base A, 1 for base B, 0 for base C, and 0 for base D. There is nothing to do. That is, when the condition is non-negative, the observed spectrum is approximated as a spectrum obtained by adding base A multiplied by 1 and base B multiplied by 1.

On the other hand, when non-negativeness is not a condition, the expected coupling coefficient u is 0 for base A, 0 for base B, 1 for base C, and 1 for base D. The one that does is -0.5. That is, when non-negativeness is not a condition, the observed spectrum is approximated as a spectrum obtained by adding base C multiplied by 1 and base D multiplied by -0.5.

When comparing the two examples described above, higher approximation accuracy may be obtained when non-negativeness is not a condition than when non-negativeness is a condition. However, since the coupling coefficient u here represents the amount of components for each spectrum, it must be obtained as a non-negative value. In other words, if the coupling coefficient u is obtained as a negative value, it cannot be interpreted as a component amount. On the other hand, if the approximation is performed by imposing a non-negative condition, it is possible to calculate the coupling coefficient u corresponding to the component amount.

In FIG. 19, the biological analysis device according to this embodiment uses a basis set consisting of a normal alveolar breath sound base, a crepitus base, four continuous rales bases, and a white noise base, as described above. In order to calculate the coupling coefficient u, the coupling coefficient u is calculated as having seven values from u ₁ to u ₇ .

Here, the coupling coefficient u ₁ corresponding to the normal alveolar breath sound base can be said to be a value indicating the ratio of normal alveolar breath sounds to the breath sounds. Similarly, the coupling coefficient u ₂ corresponds to the crepitation base, the coupling coefficient u ₃ corresponds to the white noise base, the coupling coefficient u 4 corresponds to the continuous rales base shifted to 100 Hz, and the coupling coefficient u ₄ corresponds to the continuous rales base shifted to 130 Hz. For each of the coupling coefficient u ₅ corresponding to the sound base, the coupling coefficient u ₆ corresponding to the continuous rales base shifted to 180 Hz, and the coupling coefficient u ₇ corresponding to the continuous rales base shifted to 320 Hz, the breathing It can be said that this value indicates the ratio of each sound type to the sound. Therefore, the signal strength of each tone type can be calculated from the coupling coefficient u.

As described above, in this embodiment, a plurality of sound types included in breath sounds are classified using a plurality of bases corresponding to each sound type. However, the above-described classification method is just an example, and other classification methods may be used to classify a plurality of sound types.

<Modified example of separation method>
In the following, several examples will be described of classification methods other than the classification method using a plurality of bases as described above.

<First modification example>
First, the separation method according to the first modification will be described with reference to FIGS. 20 and 21. Here, FIGS. 20 and 21 are conceptual diagrams showing a continuous rales classification method according to the first modification.

In the classification method according to the first modification, breathing sounds are classified into continuous rales and other sounds. Specifically, if the peak frequency detected from the frequency analysis result of the breathing sound signal fluctuates within a predetermined range, it is determined that it is a continuous rales.

As shown in FIG. 20, the continuous rales such as whistle sounds and snoring sounds vary so that the positions of peaks detected continuously on the time axis fall within a predetermined range. In other words, the peak frequency changes temporally with continuity. Therefore, if consecutive peak positions are within a predetermined range, it can be determined that the sound is a continuous rales.

On the other hand, as shown in FIG. 21, sounds other than continuous rales fluctuate so that the positions of peaks detected continuously on the time axis do not fall within a predetermined range. In other words, the peak frequency has no temporal continuity and changes discretely. Therefore, if the successive peak positions are not within a predetermined range, it can be determined that the sound is not a continuous rales.

Note that a plurality of determination results can also be used to determine continuous rales. Specifically, if the position of peaks detected continuously on the time axis fluctuates within a predetermined range for a predetermined number of times or more, the sound is considered to be a continuous rales. It may be determined that

<Second modification example>
Next, a separation method according to a second modification will be described with reference to FIG. 22. Here, FIG. 22 is a graph showing the threshold values used to distinguish between whistling sounds and snoring sounds according to the second modification.

In the classification method according to the second modification, continuous rales are classified into whistling sounds and snoring sounds. Here, whistle sounds and snoring sounds can be distinguished by their pitch (i.e., frequency), so that whistle sounds are called high-pitched continuous rales, and snoring sounds are called low-pitched continuous rales. It is possible. However, the peak frequency of whistling sounds and snoring sounds changes over time. Therefore, if you try to judge whistling sounds and snoring sounds using a single threshold value for the peak frequency (i.e., one threshold value that does not fluctuate), the judgment results may change over time. There is. For example, if the peak frequency changes so as to straddle the determination threshold, what was previously determined accurately will be determined to be an incorrect sound type. Therefore, in the second modification, the determination threshold value is varied depending on the peak frequency.

As shown in FIG. 22, in the classification method according to the second modification, the threshold value is varied such that the proportion determined to be a whistling sound and the proportion determined to be a snoring sound smoothly change according to the peak frequency. For example, when the peak frequency is 200 Hz, it is determined that 7% of whistle sounds are included and 93% of snoring sounds are included. When the peak frequency is 250 Hz, it is determined that 50% of whistle sounds and 50% of snoring sounds are included. When the peak frequency is 280 Hz, it is determined that 78% of whistle sounds are included and 22% of snoring sounds are included. Note that the specific numerical values here are just examples, and different values may be set. Further, the variation characteristics may vary depending on the sex, age, height, weight, etc. of the living body to be measured.

By using the above-mentioned fluctuating threshold value, it is possible to suitably prevent erroneous determinations caused by fluctuations in peak frequency. That is, in the classification method according to the second modification, the threshold value for determining whistling sounds and snoring sounds changes to an appropriate value according to the peak frequency, and therefore, for example, a single threshold value that does not change is used. This allows for more accurate separation compared to the conventional method.

<Third modification example>
Next, a separation method according to a third modification will be described with reference to FIGS. 23 to 25. Here, FIG. 23 is a graph showing the initial value of the threshold value used to distinguish between the whistling sound and the snoring sound according to the third modification. Moreover, FIGS. 24 and 25 are graphs showing the adjusted values of the threshold values used to distinguish between whistle sounds and snoring sounds according to the third modification.

The classification method according to the third modification is also a method of classifying continuous rales into whistling sounds and snoring sounds, similar to the already explained second modification. Furthermore, the second modified example is also similar to the second modified example in that the determination is made using a threshold value for the peak frequency obtained from the frequency analysis result.

As shown in FIG. 23, in the classification method according to the third modification, the determination result is set to change at a threshold of 250 Hz. Specifically, when the peak frequency is 250 Hz or more, it is determined that the continuous rales contain 100% whistle sound components and do not contain rhonal sounds. On the other hand, if the peak frequency is less than 250 Hz, it is determined that the continuous rales contain 100% rhonchitic components and do not contain whistling sounds.

As shown in FIG. 24, in the classification method according to the third modification, if it is determined in the previous determination that the sound contains 100% of the whistling sound component, the threshold value is lowered from 250 Hz to 220 Hz. Therefore, it is easier to determine that the sound contains 100% of the whistle sound component. Specifically, if we consider the case where the peak frequency is 230Hz, it will be determined to be a sonarous sound according to the initial threshold value (see Figure 23), but according to the adjusted threshold value (see Figure 24) It is determined to be a whistling sound.

As shown in FIG. 25, in the classification method according to the third modification, if it is determined in the previous determination that the sound contains 100% of the rhinocerosal component, the threshold value is increased from 250 Hz to 280 Hz. Therefore, it is easier to determine that the sound contains 100% snoring sound components. Specifically, if we consider the case where the peak frequency is 270Hz, it will be determined to be a whistling sound according to the initial threshold value (see Figure 23), but it will be determined to be a similar sound according to the adjusted threshold value (see Figure 25). The sound is determined to be snoring.

By adjusting the threshold as described above, it is possible to suitably prevent erroneous determinations caused by fluctuations in peak frequency. That is, in the classification method according to the third modification, the threshold values for determining whistle sounds and snoring sounds are adjusted to appropriate ones based on past determination results. More accurate judgments can be made compared to when using this method.

Note that the threshold value may be adjusted based not only on the immediately previous determination result but also on a plurality of past determination results. Furthermore, when using multiple past determination results, each determination result may be weighted. For example, weighting may be performed such that the more past the determination result, the smaller the influence. Furthermore, the smooth threshold value of the second modified example may be used as the initial value of the threshold value to be adjusted (see FIG. 22).

<Fourth variation>
Next, a separation method according to a third modification will be described with reference to FIGS. 26 to 26. FIG. 26 is a spectrogram diagram of breathing sounds including whistling sounds, and FIG. 27 is a graph showing the peak frequency and number of peaks of the whistling sounds. Further, FIG. 28 is a spectrogram diagram of breathing sounds including snoring sounds, and FIG. 29 is a graph showing the peak frequency and peak number of snoring sounds.

The classification method according to the fourth modification is also a method of classifying continuous rales into whistling sounds and snoring sounds, similar to the already explained second and third modifications.

In FIG. 26, breathing sounds including whistling sounds are detected as a spectrum waveform having a predetermined peak. In order to detect the peak frequency F and the number of peaks N from this, first, a frequency-amplitude graph corresponding to a single time of the spectrum waveform (ie, the area surrounded by a white frame in the figure) is created.

From the graph shown in FIG. 27, the peak frequency F1 and the number N1 of peaks of the whistle sound can be detected. It is known that the peak frequency distribution of whistle sounds is approximately 180 to 900 Hz. Further, as can be seen from the figure, the number N1 of peaks of the whistle sound is one.

In FIG. 28, breathing sounds including snoring sounds are detected as spectrum waveforms having predetermined peaks different from whistling sounds. In order to detect the peak frequency F and the number of peaks N from this, a frequency-amplitude graph corresponding to a single time of the spectrum waveform is similarly created.

From the graph shown in FIG. 29, the peak frequency F2 and the number of peaks N2 of the sonar sounds can be detected. It is known that the distribution of peak frequencies of rhinocerosic sounds is approximately 100 to 260 Hz. That is, the peak frequency F2 of the snoring sound is distributed in a region lower than the peak frequency F1 of the whistling sound. Further, as can be seen from the figure, the number of peaks N2 of assonorous sounds is, for example, three. That is, the number N2 of peaks of the snoring sound is not one like the number N1 of peaks of the whistling sound, but is plural.

In the classification method according to the fourth modification, determination is performed using the difference in characteristics between the whistle sound and the snoring sound described above. Specifically, the whistle sound and the snoring sound are classified based on each of the peak frequency F and the number N of peaks. In this way, more accurate classification can be achieved than, for example, when distinguishing between whistling sounds and snoring sounds using only the peak frequency F.

<Display of analysis results>
Next, display of analysis results will be described in detail with reference to FIG. 30. FIG. 30 is a plan view showing an example of display on the display unit.

As shown in FIG. 30, the analysis results are displayed as a plurality of images in the display area 155 of the display unit 150. Specifically, the acquired breath sound waveform is displayed in the area 155a. The area 155b displays the spectrum of the acquired breath sounds. In the area 155c, a spectrogram of the acquired breath sounds is displayed. In the area 155d, a graph showing a time-series change in the component amount of each classified sound type (here, five sound types: normal breathing sound, snoring sound, whistle sound, crepitus sound, and blister sound) is displayed. ing. In the area 155e, the ratio of each sorted note type is displayed as a radar chart.

Note that this display format of the analysis results is just an example, and the analysis results may be displayed in other display formats. For example, the ratio of each classified note type may be displayed as a bar graph or a pie chart, or may be displayed numerically.

<Tone type selection and output>
Next, the selection of a note type by the user and the output for each selected note type will be explained with reference to FIGS. 31 to 33. FIG. 31 is a spectrogram diagram showing the extraction results for each sound type. Further, FIGS. 32 and 33 are conceptual diagrams showing an example of audio output for each classified sound type, respectively.

As shown in FIG. 31, the spectrogram displayed in the area 155c of the display unit 150 may be displayed for each sound type selected by the user. That is, instead of the original (obtained original breath sounds) spectrogram shown in FIG. 31(a), a spectrogram of normal breath sounds shown in FIG. , the spectrogram of the whistling sound shown in FIG. 31(d), the spectrogram of the crepitus sound shown in FIG. 31(e), and the spectrogram of the bubbling sound shown in FIG. 31(f) may be displayed. Further, a plurality of spectrograms for each of these sound types may be displayed side by side.

As shown in FIGS. 32 and 33, a graph for each note type displayed in the area 155d of the display section 150 may be selected, and only the selected note type may be output in volume. For example, in the example shown in FIG. 32, only normal breathing sounds are selected, and none of the other sounds such as rhinoceros, whistling, crepitus, and water bubbles are selected. Therefore, the audio output unit 130 outputs only normal breathing sounds as audio. Furthermore, in the example of FIG. 33, not only normal breathing sounds are selected, but all of the other sounds, such as rhinoceros, whistling, crepitus, and blister, are selected. Therefore, the audio output unit 130 outputs a sound that is a combination of the snoring sound, the whistling sound, the crepitus sound, and the water bubble sound.

<Change of output mode>
Next, a method of changing the output mode for each sound type will be specifically explained with reference to FIGS. 34 to 38. Here, FIG. 34 is a conceptual diagram showing a volume adjustment method for each classified sound type, and FIG. 35 is a conceptual diagram showing a volume adjustment method for each frequency band. Further, FIG. 36 is a conceptual diagram showing an example of image processing performed for each sound type, and FIG. 37 is a plan view showing an example of an image generated by superimposing images subjected to image processing for each sound type. . FIG. 38 is a conceptual diagram showing a display color adjustment method for each classified tone type.

As shown in FIG. 34, the output volume of each classified sound type may be adjustable for each sound type. In the operation screen shown in the figure, ON/OFF of each tone type can be switched by a check box, and the volume of each tone type can be adjusted by a slider. In the example shown in the figure, a snoring sound and a whistle sound are respectively output, and the whistle sound is output at a louder volume than the snoring sound.

As shown in FIG. 35, in addition to adjusting the output volume for each sound type, it may be possible to adjust the output volume for each frequency band. In the example shown in the figure, the gain can be adjusted in each frequency band of 125 Hz, 250 Hz, 500 Hz, 1 kHz, and 2 kHz.

As shown in FIG. 36, image processing (eg, binarization, edge detection, etc.) may be performed on the spectrogram extracted for each sound type. In this way, things that are difficult to recognize when only extracted can be displayed in a more visually easy-to-understand state. Note that the image processing may be a combination of multiple processes. Further, different image processing may be performed depending on the type of sound.

As shown in FIG. 37, images for each sound type that have been subjected to image processing (see FIG. 36) may be displayed in a superimposed manner. In this way, the spectrograms for each sound type can be recognized together in one image, so that visual understanding can be suitably performed. Here, an example was shown in which images of normal breathing sounds and whistling sounds were displayed in an overlapping manner, but the sound types to be overlaid can be selected, and only the desired sound types can be appropriately selected and displayed. I can do it.

As shown in FIG. 38, the color of the image may be adjustable for each type of sound. In the example shown in the figure, it is possible to adjust the RGB values for each note type by adjusting the sliders corresponding to R (red), G (green), and B (blue), respectively. In this way, it is possible to display a plurality of tone types in mutually different colors, and it is possible to realize a display that is easier to visually understand.

As explained above, according to the breathing sound analysis device according to the present embodiment, after classifying the breathing sounds, it is possible to select and output them as appropriate. In addition, since the output mode can be changed for each sound type, the classified data for each sound type can be suitably used.

The present invention is not limited to the embodiments described above, and can be modified as appropriate within the scope or idea of the invention that can be read from the claims and the entire specification. An apparatus and a breath sound analysis method, as well as a computer program and a recording medium are also included within the technical scope of the present invention.

110 Body sound sensor 120 Signal storage section 125 Signal processing section 130 Audio output section 140 Base holding section 150 Display section 155 Display area 160 Input section 200 Processing section 210 Frequency analysis section 220 Frequency peak detection section 230 Basis set generation section 240 Coupling coefficient calculation Section 250 Signal strength calculation section 260 Image generation section 270 Breathing sound selection section y Spectrum h(f) Basis u Coupling coefficient

Claims (4)

A classification means for classifying breath sounds into multiple sound types;
an input means for accepting input for selecting the type of breathing sound to be output;
output means for outputting a sound type of breath sound selected according to the input received by the input means as a spectral image;
a changing means capable of changing the color of the spectrum image for each sound type selected according to the input received by the input means;
A breathing sound analysis device comprising: The changing means is capable of changing the color of the spectrum image to a different color for each selected tone type according to the input received by the input means,
The breathing apparatus according to claim 1, wherein the output means superimposes and outputs a plurality of spectral images of the breathing sounds selected according to the input received by the input means, each in a different color. Sound analysis device. A classification step of separating breath sounds into multiple sound types;
an input step of accepting input for selecting the type of breathing sound to be output;
an output step of outputting a sound type of breath sound selected according to the input received by the input means as a spectral image;
a changing step in which the color of the spectrum image can be changed for each sound type selected according to the input received by the input means;
A breathing sound analysis method characterized by comprising: A classification step of separating breath sounds into multiple sound types;
an input step of accepting input for selecting the type of breathing sound to be output;
an output step of outputting a sound type of breath sound selected according to the input received by the input means as a spectral image;
a changing step in which the color of the spectrum image can be changed for each sound type selected according to the input received by the input means;
A computer program that causes a computer to execute.