TW201703029A - Method and device for recognizing stuttered speech and computer program product - Google Patents

Abstract

A method and a device for recognizing a stuttered speech and a computer program product are provided. The method includes: dividing an audio signal into multiple syllables; performing a dynamic time warping algorithm according to several feature values on a first syllable and a second syllable of the syllables that are adjacent to each other in order to determine whether the first syllable is similar to the second syllable; if the first syllable is similar to the second syllable, determining that a stuttered phenomenon exists in the audio signal.

Description

Stutter detection method and device, computer program product

The invention relates to a method, a device and a computer program product capable of automatically detecting stuttering voice.

In the field of language therapy, it takes a lot of manpower to assess whether a person has a stutter, and these assessments rely heavily on the subjective judgment of the listener. It is difficult to find consistent and objective criteria from different judges. Therefore, if a computer can be used to automatically judge the phenomenon of stuttering, there can be objective standards and manpower can be saved. Some conventional methods are to record a person's voice to obtain an audio signal, and divide the voice signal into a plurality of learning samples, and then execute a machine learning algorithm, and the obtained model can be used to determine whether the test voice signal has The phenomenon of stuttering. However, the use of machine learning methods requires the collection of many vocal samples, and if the vocal samples are insufficient, the accuracy of the judgment will be reduced.

Embodiments of the present invention provide a stutter detection method. The method includes: obtaining an audio signal; for the audio signal, calculating a plurality of feature values Obtaining a plurality of characteristic signals; dividing the sound signal into a plurality of syllables according to one of the characteristic signals; and performing a dynamic time axis correction algorithm according to the characteristic signals for determining the first syllable and the first syllable for the adjacent first syllable and the second syllable Whether the two syllables are similar; and if the first syllable is similar to the second syllable, it is judged that the sound signal is stuttering.

In an embodiment, the feature signals are calculated according to the following equations (1) to (6):

VH=α×Volume+(1-α)×HOD...(5)

VE=Volume×(1-entropy)...(6)

Volume is the volume characteristic signal, ZCR is the zero-crossing characteristic signal, Entropy is the entropy characteristic signal, HOD is the differential characteristic signal, VH is the volume differential characteristic signal, and VE is the volume entropy characteristic signal. s _i is the sound signal, n is the length of the sound segment, sgn[ ] represents the sign function, s(f _k ) represents the amplitude of the k-th frequency of the sound signal in the frequency domain, and N represents the length of the sound signal in the frequency domain. α is a constant.

In an embodiment, the step of dividing the audio signal into syllables includes: determining whether the volume differential feature signal is greater than a syllable threshold; and obtaining a portion greater than the syllable threshold from the volume differential feature signal to determine the plurality of syllables.

In an embodiment, the determining the first syllable and the second syllable Whether the similar steps include: performing a dynamic time axis correction algorithm on the first syllable and the second syllable for each characteristic signal to obtain an error value, and determining whether the error value is smaller than an error threshold; and the error value in the characteristic signal If the number is less than the error threshold, if it is greater than the similar threshold number, it is determined that the first syllable is similar to the second syllable. In an embodiment, the similar critical number is four.

Embodiments of the present invention also provide a computer program product and a stutter detection device for performing the stutter detection method described above. Thereby, it is possible to automatically judge whether the person has a stutter without collecting too many learning samples.

The above described features and advantages of the invention will be apparent from the following description.

100‧‧‧ Stuttering detection device

110‧‧‧ processor

120‧‧‧ memory

130‧‧‧Sound capture module

140‧‧‧Transmission module

210‧‧‧Volume differential feature signal

211‧‧‧ syllable threshold

220, 230‧‧ syllables

300‧‧‧Matrix

310‧‧‧ Path

S401~S406‧‧‧Steps

FIG. 1 is a schematic diagram showing a stutter detection device according to an embodiment.

FIG. 2 is a schematic diagram showing dividing an audio signal into a plurality of syllables according to an embodiment.

FIG. 3 is a schematic diagram showing a matrix in a dynamic time axis correction algorithm according to an embodiment.

FIG. 4 is a flow chart showing a stutter detection method according to an embodiment.

The terms "first", "second", "etc." used in this document are not intended to mean the order or the order, and are merely to distinguish between elements or operations described in the same technical terms.

1 is a schematic diagram showing a stutter detection device according to an embodiment. Referring to FIG. 1 , the stutter detecting device 100 includes a processor 110 , a memory 120 , a sound capturing module 130 , and a transmission module 140 . The stutter detection device 100 can be implemented as any form of electronic device, such as a mobile phone, a tablet computer, a personal computer, or an embedded system.

The processor 110 is, for example, a central processing unit, a microprocessor, or a general purpose processor of any executable instructions. The memory 120 can be a random access memory or a flash memory. The memory 120 stores a plurality of instructions, which can be executed by the processor 110 to complete the stutter detection method. The sound capture module 130 is, for example, a microphone, and the transmission module 140 can be a circuit that conforms to any suitable communication protocol, such as a universal serial bus (USB) module or a Bluetooth module. . However, the stutter detecting device 100 may further include other modules, such as a display module, a power module, etc., and the present invention is not limited thereto. The specific content of the stutter detection method will be described below.

First, the user can speak to the sound capture module 130, and the sound capture module 130 transmits the obtained sound signal to the processor 110. In this embodiment, the processor 110 obtains the sound signal through the sound capture module 130. However, in other embodiments, the sound signal can be obtained from other devices through the transmission module 140. The present invention is not limited thereto. .

Next, the processor 110 calculates a plurality of feature values for the sound signal to obtain a plurality of feature signals. In this embodiment, the number of characteristic signals is 6, which are a volume characteristic signal, a zero-crossing rate characteristic signal, an entropy characteristic signal, a differential characteristic signal, a volume differential characteristic signal, and a volume entropy characteristic signal. It is calculated by the following equations (1) to (6).

VH=α×Volume+(1-α)×HOD...(5)

VE=Volume×(1-entropy)...(6)

Specifically, the volume is a volume characteristic signal, and s _i is the amplitude of the sound signal at time point i. n is the length of the sound segment, for example 20 microseconds, but may be other lengths in other embodiments. In other words, the sound signal is cut into a plurality of sound segments, and for each sound segment, a value can be calculated according to the above equation (1), and all the values in the sound signal constitute a volume feature signal.

ZCR is a zero-crossing rate characteristic signal, and sgn[ ] represents a sign function. For example, if the variable x is positive, sgn[x] is 1, and vice versa.

Entropy is the entropy characteristic signal, s(f _k ) represents the amplitude of the k-th frequency of the audio signal in the frequency domain, and N represents the length of the audio signal in the frequency domain. For example, for each sound segment, the sound signal can be first converted from the time domain to the frequency domain, for example, Fourier transform or Fast Fourier Transform (FFT) to obtain the amplitude of each frequency. (N total of N values), and s(f _k ) represents the kth amplitude.

HOD is a differential feature signal, VH is a volume differential characteristic signal, and VE is a volume entropy characteristic signal, where α is a constant between 0 and 1.

After obtaining the above six characteristic signals, the sound signal can be divided into a plurality of syllables according to at least one characteristic signal. Please refer to FIG. 2. FIG. 2 is a schematic diagram showing dividing an audio signal into a plurality of syllables according to an embodiment. Taking the volume differential feature signal 210 as an example, it can be determined whether the amplitude of each sampling point in the volume differential characteristic signal 210 is greater than the syllable threshold 211, and then the portion of the volume differential characteristic signal 210 that is greater than the syllable threshold 211 is determined. The syllable 220 and the syllable 230. In other embodiments, other characteristic signals may be used to divide the sound signal into a plurality of syllables, and the present invention is not limited thereto.

If the user speaks Chinese, the syllable obtained above may represent a word. Next, it can be judged whether two adjacent syllables in the syllables are similar. If two adjacent syllables are similar, it is judged that there is stuttering in the sound signal. For example, assuming that the above syllables have first syllables and second syllables adjacent to each other, it is worth noting that the lengths of the two syllables may not be the same. For each feature signal, the processor 110 performs a dynamic time warping on the first syllable and the second syllable to obtain an error value. Referring to FIG. 3, assuming that there are a sound segments in the first syllable and b sound segments in the second syllable, wherein a and b are positive integers, a multiplication is generated when performing the dynamic time axis correction algorithm. Matrix 300 of b. Taking the volume feature signal as an example, each element in the matrix 300 represents the difference in volume between the corresponding sound segment in the first syllable and the corresponding sound segment in the second syllable. The dynamic time axis correction algorithm is to find the smallest path 310 after the difference is accumulated in the matrix 300, and the accumulated difference is The error value between the first syllable and the second syllable. In this embodiment, there are six characteristic signals, so each characteristic signal has a corresponding matrix and error value, and these error values are compared with an error threshold. Next, it is calculated that the error value of several characteristic signals is smaller than the error threshold, and the error value in the characteristic signal is less than the error threshold. If it is greater than a similar critical number, it is determined that the first syllable is similar to the second syllable. This similar critical number is, for example, 4, but the invention is not limited thereto. In other words, if four of the six characteristic signals are very similar, it means that the first syllable is similar to the second syllable, and the two syllables are stuttered.

In the above embodiment, the number of feature signals is 6, and the similar critical number is 4. However, in other embodiments, more feature values may be used and more feature signals may be generated. In these embodiments, the similar critical numbers described above may also be adjusted accordingly. On the other hand, the above-described stutter detection method can be applied to any language, and the present invention is not limited thereto.

The invention also provides a computer program product, which can complete the above stutter detection method after the computer loads the computer program product and executes it. For example, the computer program product can be loaded into the memory 120 of FIG. 1 and executed by the processor 110. However, the present invention does not limit the programming language in which the computer program product is to be implemented.

On the other hand, the stutter detection method proposed by the present invention can be expressed as the flowchart of FIG. 4, and in step S401, an audio signal is obtained. In step S402, for the audio signal, a plurality of feature values are calculated to obtain a plurality of feature signals. In step S403, the sound signal is divided into a plurality of syllables according to one of the characteristic signals. In step S403, for the first sound adjacent in the syllable The node and the second syllable perform a dynamic time axis correction algorithm according to the feature signal to determine whether the first syllable is similar to the second syllable. If there are two adjacent syllables similar, it is determined in step S405 that there is a stutter phenomenon in the audio signal. If all adjacent syllables are not similar, it is determined in step S406 that there is no stuttering phenomenon in the audio signal. However, the steps in FIG. 4 have been described in detail above, and will not be described again here. It should be noted that the steps in FIG. 4 can be implemented as multiple code codes or circuits, and the present invention is not limited thereto. In addition, the method of FIG. 4 can be used in combination with the above embodiments or can be used alone. In other words, other steps can be added between the steps of FIG. 4.

Although the present invention has been disclosed in the above embodiments, it is not intended to limit the present invention, and any one of ordinary skill in the art can make some changes and refinements without departing from the spirit and scope of the present invention. The scope of the invention is defined by the scope of the appended claims.

S401~S406‧‧‧Steps

Claims (7)

A stutter detection method includes: obtaining an audio signal; calculating, for the audio signal, a plurality of feature values to obtain a plurality of feature signals; and dividing the sound signal into a plurality of syllables according to one of the feature signals; An adjacent first syllable and a second syllable in the syllables perform a dynamic time axis correction algorithm according to the characteristic signals to determine whether the first syllable is similar to the second syllable; and if the first syllable Similar to the second syllable, it is judged that the sound signal has a stuttering phenomenon. The stutter detection method of claim 1, wherein the characteristic signals are calculated according to the following equations (1) to (6): VH=α×Volume+(1-α)×HOD...(5) VE=Volume×(1-entropy) (6) where Volume is the volume characteristic signal, ZCR is the zero-crossing rate characteristic signal, and Entropy is Entropy characteristic signal, HOD is differential characteristic signal, VH is volume differential characteristic signal, VE is volume entropy characteristic signal, s _{i is} the amplitude of the sound signal at time point i, n is the length of a sound segment, sgn[ ] Represents a sign function, s(f _k ) represents the amplitude of the k-th frequency of the audio signal in the frequency domain, N represents the length of the audio signal in the frequency domain, and α is a constant. The stutter detection method of claim 2, wherein the step of dividing the sound signal into the syllables according to one of the characteristic signals comprises: determining whether the volume differential feature signal is greater than a syllable threshold And obtaining a portion greater than the syllable threshold from the volume differential feature signal to determine the syllables. The stutter detection method of claim 1, wherein the step of determining whether the first syllable is similar to the second syllable comprises: for each of the characteristic signals, the first syllable and the second syllable Performing the dynamic time axis correction algorithm to obtain an error value, and determining whether the error value is less than an error threshold; and if the error value of the characteristic signals is less than a certain threshold number of the error threshold, It is determined that the first syllable is similar to the second syllable. The stutter detection method of claim 4, wherein the number of the characteristic signals is 6, and the similar critical number is 4. A computer program product, which is capable of performing the stutter detection method according to any one of the above-mentioned claims, wherein the computer is loaded with the computer program product and executed. A stutter detecting device comprising: a memory for storing a plurality of instructions; and a processor for executing the instructions to perform a plurality of steps: obtaining an audio signal; and calculating a plurality of feature values for the audio signal Obtaining a plurality of characteristic signals; dividing the sound signal into a plurality of syllables according to one of the characteristic signals; performing, for each of the first syllables and the second syllable adjacent to the syllables, performing a signal according to the characteristic signals The dynamic time axis correction algorithm determines whether the first syllable is similar to the second syllable; and if the first syllable is similar to the second syllable, determining that the audio signal has a stuttering phenomenon.