JP2006084619A - Device and program for automatic detection of breathy area of speech data - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide a device that automatically detects a breathy area of speech data. <P>SOLUTION: The breathy area automatic detecting device includes 1st and 2nd filters 81 and 82 which filter the speech data by filters of 1st and 2nd frequency bands corresponding to frequencies of 1st and 3rd formants, 1st and 2nd envelope estimation parts 83 and 84 which find envelope waveforms of signal waves filtered by the 1st and 2nd filters 81 and 82, a correlation arithmetic part 85 which finds the correlation between the two found envelope waveforms, a comparison part 86 which compares the found correlation result with a 1st threshold, and a decision part 89 which decides the breathy area of the speech data. <P>COPYRIGHT: (C)2006,JPO&NCIPI

Description

  The present invention relates to a technique for detecting features other than phoneme / prosodic features in speech data, and more particularly to a breath leak region automatic detection device and a breath leak region automatic detection device that automatically detect a breath leak region of speech data. Regarding detection program.

  The breath leak component included in the speech conveys important language / paralinguistic information depending on the language, that is, emotional information such as anger, joy, and sadness. For example, the distinction between weak breathy utterances and vowel utterances in vowels often occurs in various minor languages (Non-Patent Document 1). In English, the relation between various utterance styles and paralinguistic information has been reported (Non-patent Document 2), the relation between strong whispery utterance and “fear”, breath utterance and “sadness” The relationship with is shown.

  In addition, there is a report that “breathiness” is used in Japanese to express attitude and politeness (Non-patent Documents 3 and 4). Furthermore, it has been observed that breath leaks often occur in the vowel section of laughter.

  When the average opening of the glottis is small, as in the case of vowels, the noise component is weaker than the periodic component. Conversely, if the glottis opens too much, vocal cord vibration does not occur and only noise is generated (whispering). When the opening of the glottis is moderate as in a breath sound, the vocal cords continue to vibrate, but two changes occur in the spectrum of the vocal cord sound source (Non-Patent Document 5). One is that the spectrum is significantly lowered by the harmonic component, and the other is that the noise component is increased by the air flow.

  In Non-Patent Documents 6 and 7, a parameter NAQ (Normalized Amplitude Quotient) related to breath leak is proposed, and the peak-to-peak value of the amplitude of the vocal fold vibration waveform and the negative peak value of the differential waveform of the vocal fold vibration are calculated. The ratio is defined as normalized by the fundamental frequency F0.

  However, calculation of this parameter requires vocal tract inverse filtering. Vocal tract inverse filtering is an error-prone process and is difficult to perform stably. In addition, when NAQ is used, there is a problem in distinguishing between breath leak and nasal sound. This is probably because the fundamental frequency component is strong in any voice after inverse filtering. It is considered that the parameter NAQ does not efficiently use the characteristics of the noise component of breath leakage.

On the other hand, a subjective noise evaluation method that takes into account the noise component of breath leakage has also been proposed (Non-Patent Document 8). In this method, irregularities are evaluated by observing a waveform obtained by applying a bandpass filter around the frequency of the third formant (F3) (hereinafter referred to as “F3 frequency”). In the case of breath leakage, the noise component tends to be stronger than the periodic component in the F3 frequency region, and it is an assertion that a significant noise component in the high frequency band is related to the perception of breathy voice quality. . Non-Patent Document 9 also applies this method and shows a high correlation between subjects.
M.M. Gordon, P. Raid Forged, “Voice Type: A Cross-Language Overview”, Phonetic Journal, Vol. 29, pp. 383-406, 2001 (Gordon, M., Ladefoged, P. “Phonation types: a cross-linguistic overview,” J. of Phonetics 29, 383-406, 2001) G. Class Meyer, W. F. Sendle Meyer, “State of Voice and Emotion”, Voice Quality Measurement, Singular Thomson Learning, Chapter 15, pp. 339-358, 2000 (Klasmeyer, G., Sendlmeier, WF “Voice and Emotional States,” In Voice Quality Measurement, Singular Thomson Learning. Ch. 15, 339-358, 2000) N. Campbell, P.A. Moktari, “Voice quality: 4th prosodic parameter”, Proceedings of the 15th International Speech Science Conference, pp. 2417-2420, 2003 (Campbell, N., Mokhtari, P. “Voice quality; the 4th prosodic parameter,” Proc. 15th International Congress of Phonetic Sciences, 2417-2420, 2003) M.M. Ito, “Politeness and Voice Quality-Another Method for Measuring Breathing Noise”, Speech Prosody 2004 Proceedings, pp. 213-216, 2004 (Ito, M., “Politeness and voice quality-The alternative method to measure aspiration noise,” Proc. Speech Prosody 2004, 213-216, 2004) K. Stevens, “Turbulent Noise in the Glottal During Breathing and Speaking”, Acoustic Phonetics, MIT Press, pp. 445-450, 2000 (Stevens, K. “Turbulence Noise at the Glottis During Breathy and Modal Voicing,” In Acoustic Phonetics, The MIT Press, 445-450, 2000) P. ALC, E.I. Wilkman, “Amplitude Index for Description of Glottal Volume Velocity Waveform Estimated by Inverse Filtering”, Spoken Communication, Vol. 18, No. 2, pp. 131-138, 1996 (Alku, P., Vilkman, E. “Amplitude domain quotient for characterization of the glottal volume velocity waveform estimated by inverse filtering,” Speech Communication, Vol. 18, No. 2, 131-138, 1996 .) P. Moktari, N.I. Campbell, "Automatic measurement of tone / breath leak at the center of reliability in continuous speech", IEICE Journal, Information Processing and System Transactions, E-86-D Vol. 574-582, 2003 (Mokhtari, P., Campbell, N. “Automatic measurement of pressed / breathy phonation at acoustic centres of reliability in continuous speech,” IEICE Trans. On Inform. And Systems, Japan, Vol. E-86 -D, No. 3, 574-582, 2003) D. Cratt, L.C. Krat, “Analysis, synthesis, and perception of voice quality changes between female and male speakers”, American Phonetic Society Journal, Vol. 87, pp. 820-857, 1990 (Klatt, D., Klatt, L. “Analysis, synthesis, and perception of voice quality variations among female and male talkers,” J. Acoustic. Soc. Amer., Vol. 87: 820-857. , 1990) H. Hanson, “Characteristics of Female Speakers: Acoustic Correlation”, Acoustical Society of America Journal, Vol. 101, pp. 466-481, 1997 (Hanson, H. “Glottal characteristics of female speakers: Acoustic Correlates,” J. Acoustic. Soc. Amer., Vol. 101: 466-481, 1997)

  However, the non-patent document 8 or the non-patent document 9 described above shows that the irregularity in the peripheral band of the F3 frequency of the third formant is related to breath leakage. It did not disclose a technique for quantifying the degree and automatically detecting a breath leak region from voice data.

  Accordingly, an object of the present invention is to provide an apparatus and a program for automatically detecting a regular sound region of audio data by quantifying the degree of irregularity in the F3 frequency band.

  Another object of the present invention is to provide an apparatus and a program for automatically and accurately detecting a breath leak region from audio data.

  A breath leak region automatic detection device according to an aspect of the present invention is a waveform for obtaining a correlation between an envelope waveform of a signal component of a first formant frequency region and an envelope waveform of a signal component of a third formant frequency region of audio data. Correlation calculation means, comparison means for comparing the correlation result obtained by the correlation calculation means with a predetermined threshold value, and determining a breath leak region of the voice data based on the comparison result by the comparison means Determination means.

  This breath leak region automatic detection apparatus obtains a correlation between the envelope waveform of the signal component in the first formant frequency region of the audio data and the envelope waveform of the signal component in the third formant frequency region. If there is a breath leak, the noise component in the third formant frequency region increases, and the above two envelope waveforms are not similar. Conversely, when there is no breath leak, the noise component in the third formant frequency region is reduced, and the above two envelope waveforms are similar to each other. The similarity of the envelope waveform, that is, the correlation is quantified, and the value and the threshold value are compared with each other by the comparison unit. Based on the comparison result, it is determined whether or not the determination unit is a breath leak region. In this way, the similarity between waveforms is quantified, and the determination means can automatically determine the breath leak area where there is a breath leak, and as a result, the breath leak area included in the audio data is automatically detected. it can.

  Preferably, the waveform correlation calculation means has first filter means for filtering the voice data with a filter in a first frequency band corresponding to the first formant frequency, and second voice data corresponding to the third formant frequency. The second filter means for filtering with a filter of the frequency band of the first, the first envelope estimating means for obtaining the envelope waveform of the signal wave filtered by the first filter means, and the second filter means for filtering Second envelope estimating means for obtaining an envelope waveform of the signal wave obtained, and arithmetic means for obtaining a correlation between the two envelope waveforms respectively obtained by the first and second envelope estimating means.

  In this breath leak region automatic detection device, voice data is filtered by the first filter means and the second filter means, and an envelope waveform of the filtered signal waveform is obtained by two envelope estimation means. By obtaining the correlation between the two estimated envelope waveforms, it is possible to determine the possibility that the portion of the voice data is a breath leak region.

  More preferably, the breath leak region automatic detection device is configured to calculate a difference between the intensity of the signal wave filtered by the first filter means and the intensity of the signal wave filtered by the second filter means. Further, a calculation means is included, and the determination means includes a means for determining a breath leak region of the audio data based on the comparison result by the comparison means and the calculation result by the intensity difference calculation means.

  The determination means makes a determination based on the result of the intensity difference calculation means in addition to the comparison means. If the intensity of the signal wave filtered by the second filter means is relatively small, even if there is a breath leak region, the listener cannot perceive it. Therefore, by determining the presence of the breath leak region by adding the intensity difference to the determination condition, it is possible to detect the breath leak region included in the audio data closer to a human sense and with high accuracy.

  As an example of the calculation of the intensity difference by the intensity difference calculation means, the power of the strongest component in the signal wave filtered by the first filter means and the strongest component in the signal wave filtered by the second filter means The calculation of the difference between values can be mentioned.

  Further, the determination unit is configured to determine, for each frame of the audio data, a frame determination unit for determining a breath-leak area of the audio data based on a comparison result by the comparison unit, and a predetermined number of consecutive frames of the audio data. In response to the determination result of the breath leakage area being obtained by the frame determination means, a means for determining the predetermined number of frames as the breath leakage area may be included.

  With this configuration, it is possible to more accurately and stably determine the breath leak region of the audio data.

  A breath leak region automatic detection program according to another aspect of the present invention is a computer-executable program that, when executed by a computer, causes the computer to operate as one of the above breath leak region automatic detection devices.

-Basic concept-
In the subjective noise evaluation disclosed in Non-Patent Document 8, the subject evaluates the waveform filtered by the bandpass filter around the F3 frequency while viewing it together with the waveform in the F1 frequency region. Therefore, although not pointed out in Non-Patent Document 8, it is considered that when the subject performs this evaluation, the envelope waveforms of these waveforms are compared and evaluated without particular awareness. In the embodiment of the present invention described below, based on this basic idea, an envelope waveform of a waveform component filtered by a bandpass filter in the frequency band around F3 is estimated, and the waveform component around the frequency around F1 is estimated. The correlation between waveforms is quantified by taking the cross-correlation with the envelope waveform.

-Realization by computer-
One embodiment of the present invention described below is realized by a computer and software operating on the computer. Of course, part or all of the functions described below can be realized by hardware instead of software.

  FIG. 1 shows an external view of a computer system 20 used in the present embodiment, and FIG. 2 shows a block diagram of the computer system 20. The computer system 20 shown here is merely an example, and various other configurations are possible.

  Referring to FIG. 1, the computer system 20 includes a computer 40, a monitor 42, a keyboard 46, and a mouse 48, all connected to the computer 40. The computer 40 further includes a CD-ROM (Compact Disk Read-Only Memory) drive 50 and an FD (Flexible Disk) drive 52.

  Referring to FIG. 2, computer system 20 further includes a printer 44 connected to computer 40, which is not shown in FIG. The computer 40 further includes a bus 66 connected to the CD-ROM drive 50 and the FD drive 52, a central processing unit (CPU) 56 connected to the bus 66, a boot-up program for the computer 40, and the like. ROM (Read-Only Memory) 58 that stores data, a RAM (Random Access Memory) 60 that provides a work area used by the CPU 56 and a storage area for programs executed by the CPU 56, and a hard disk 54 that stores audio data.

  The software for realizing the system of the embodiment described below is recorded and distributed on a recording medium such as a CD-ROM 62, and is read into the computer 40 via a reader such as the CD-ROM drive 50. Stored in the hard disk 54. When the CPU 56 executes this program, the program is read from the hard disk 54 and stored in the RAM 60, and an instruction is read from an address designated by a program counter (not shown) and executed. The CPU 56 reads data to be processed from the hard disk 54 and stores the processing result in the hard disk 54 as well.

  Since the operation itself of the computer system 20 is well known, details thereof will not be repeated here.

  Note that the software distribution form is not limited to the form fixed to the storage medium as described above. For example, data may be distributed in the form of receiving data from other computers connected through a network. Further, there may be a distribution form in which a part of software is stored in the hard disk 54 in advance, and the remaining part of the software is taken into the hard disk 54 via the network and integrated at the time of execution.

  In general, modern programs utilize the general purpose functions provided by a computer operating system (OS) and achieve the desired objectives described above by executing them in an organized manner according to the desired objectives. . Therefore, among the functions of the present embodiment described below, it is a program (group) that does not include general-purpose functions provided by the OS or a third party, and specifies only a combination of execution orders of these general-purpose functions. However, as long as it is a program (group) having a control structure that achieves a desired object as a whole using them, it is obvious that they are included in the technical scope of the present invention.

-Functional configuration-
FIG. 3 is a block diagram functionally showing the program of the present embodiment as a breath leak region automatic detection device. Referring to FIG. 3, this device 80 performs the process described below on audio data 90 stored in hard disk 54, detects a breath leak, estimates the presence of a breath leak area, and results thereof. Is output. In this embodiment, the audio data 90 is sampled and framed with a length of 32 msec per frame and a time interval of 10 msec, and the following processing is performed for each frame.

  The apparatus 80 includes an F1 band filter unit 81 for filtering the audio data 90 with a filter in a frequency band (hereinafter referred to as F1 band) corresponding to the frequency of the first formant, and a frequency corresponding to the frequency of the third formant. And an F3 band filter unit 82 for filtering with a band (hereinafter referred to as F3 band) filter.

  The F1 band filter unit 81 and the F3 band filter unit 82 use fixed bandwidths in the F1 band and the F3 band, respectively, in order to avoid automatic extraction of formants that are prone to errors. In this embodiment, the F3 band is fixed to a band of 1800 to 4000 Hz, which is likely to include the third formant frequency for both men and women. The F1 band which is a low frequency band is fixed to a band of 100 to 1500 Hz where the influence of the noise component of breath leakage is small on the periodic component.

  In the following description, for simplicity, the signal filtered by the F1 band filter unit 81 is an F1 wave, the waveform is an F1 waveform, the signal filtered by the F3 band filter unit 82 is an F3 wave, and the waveform is F3. It is called a waveform.

  The apparatus 80 further includes an F1 envelope estimation unit 83 for obtaining the amplitude envelope waveform of the F1 wave, an F3 envelope estimation unit 84 for obtaining the amplitude envelope waveform of the F3 wave, and the envelope estimation units 83 and 84 for F1 and F3, respectively. By calculating a cross-correlation between the two obtained envelope waveforms, a correlation calculation unit 85 for calculating a synchronization rate between these waveforms, a synchronization rate calculated by the correlation calculation unit 85, and a predetermined threshold value are obtained. In comparison, the first comparison unit 86 that outputs a result of whether or not the two envelope waveforms are correlated, and the power value A1 of the F1 wave filtered by the F1 band filter unit 81 and the F3 band filter unit 82 An intensity difference calculation unit 87 for calculating a difference A1-A3 with the power value A3 of the filtered F3 wave, and a comparison between the power value difference A1-A3 calculated by the intensity difference calculation unit 87 and a predetermined threshold value And A second comparison unit 88 for outputting the result, and a determination unit 89 for determining whether or not the sound of the frame is a breath leak region based on the comparison results by the first comparison unit 86 and the second comparison unit 88. Including.

  In this embodiment, first, the instantaneous amplitudes of the F1 waveform and the F3 waveform are estimated by the Hilbert envelope method, and each Hilbert envelope is smoothed by a 1 ms long Hanning window to obtain an amplitude envelope waveform. The method is adopted.

  The correlation calculation unit 85 calculates the synchronization rate between these waveforms by calculating the cross-correlation between the two envelope waveforms obtained by the F1 and F3 envelope estimation units 83 and 84. When the synchronization rate is low, that is, when the two envelope waveforms are not correlated, there is a high possibility that the noise component in the F3 band is generated independently of the vocal cord pulse. Therefore, it is a logic that the input voice data 90 includes breath leak noise and is likely to be a breath leak region.

  In this embodiment, specifically, the intensity difference calculation unit 87 includes the power value A1 of the strongest component in the F1 wave filtered by the F1 band filter unit 81 and the F3 band filtered by the F3 band filter unit 82. The difference A1-A3 of the power value A3 with the highest intensity component in the wave is calculated. As will be described later, even if the correlation between the two envelope waveforms obtained by the F1 and F3 envelope estimation units 83 and 84 is low, it may not be perceived as a breath leak. The measured amount of difference A1-A3 is used.

  Instead of calculating the difference A1-A3 between the power value A3 of the strongest component in the F1 wave and the strongest component in the F3 wave, the average of the power values of several components is calculated, It is good also as an intensity | strength difference with the difference of an average value.

  Further, the determination unit 89, specifically, when the first comparison unit 86 determines that the synchronization rate of the two envelope waveforms is equal to or higher than a predetermined threshold, that is, when both waveforms are correlated. Determines that the voice data is unlikely to contain breath leak noise, and therefore there is no breath leak. Conversely, if the first comparison unit 86 determines that the synchronization rate of the two envelope waveforms is lower than the predetermined threshold value, that is, if the correlation between the two waveforms is low, it can be estimated that there is a breath leak. . However, even in this case, there may be a breath leak that the listener cannot clearly perceive (ie, the listener is not aware of the breath leak). As will be described later, when labeling voices in a large-scale voice database, the results should be as similar as possible to those of a human listener. Therefore, if there is a breath leak, something that the listener cannot clearly perceive should be excluded.

  Therefore, in order to eliminate the presence of breath leakage that cannot be clearly perceived by the listener, the comparison result by the second comparison unit 88 is referred to. According to experiments, even if the synchronization rate is low, if the power value difference A1-A3 is small, the listener determines that there is a breath leak, and conversely, if the power value difference A1-A3 is large, Listeners often determined no breathing. Therefore, if the synchronization rate is low and the value of the power value difference A1-A3 is smaller than the predetermined threshold value, it is determined that there is a breath leak. Even if the synchronization rate is low, the difference in power value A1-A3 If the value is equal to or greater than a predetermined threshold value, it is determined that there is no breath leak.

  Such a determination process is performed for each frame of the audio data 90. Desirably, when it is determined that there is a breath leak a plurality of times (for example, three times or more), it is determined that there is a breath leak in the audio frame. This is because it is considered from experiments that a certain amount of length (30 to 40 ms) is necessary in order for breath leakage to be perceived in a syllable or phrase uttered by a speaker.

-Operation-
The apparatus 80 shown in FIGS. 1-3 operates as follows.

  First, the audio data 90 is filtered by the F1 band filter unit 81 and the F3 band filter unit 82. Next, the F1 envelope estimation unit 83 and the F3 envelope estimation unit 84 obtain the amplitude envelope waveforms of the F1 wave and the F3 wave, respectively, and then the correlation calculation unit 85 obtains the synchronization rate of both envelope waveforms.

  The obtained synchronization rate is compared with a threshold value in the first comparison unit 86. The comparison result is given to the determination unit 89.

  On the other hand, the intensity difference calculation unit 87 calculates the difference A1-A3 between the power value A1 of the F1 wave filtered by the F1 band filter unit 81 and the power value A3 of the F3 wave filtered by the F3 band filter unit 82, This is given to the second comparison unit 88. The second comparison unit 88 calculates the threshold value difference A1-A3 between the power value A1 of the strongest component in the F1 wave and the power value A3 of the strongest component in the F3 wave, which is calculated by the intensity difference calculating unit 87. The values are compared, and the result is output to the determination unit 89. The determination unit 89 determines the presence of breath leak using the comparison result of the first comparison unit 86 and the comparison result of the second comparison unit 88.

  FIG. 4 shows a subroutine of breath leak presence / absence determination processing performed by the determination unit 89. First, it is determined from the comparison result of the first comparison unit 86 whether or not the synchronization rate is equal to or greater than a threshold value (step 101). If it is equal to or greater than the threshold value (YES at step 101), the voice data is unlikely to contain breath leak noise, so it is determined that there is no breath leak (step 106), and the determination process ends.

  When the synchronization rate is smaller than the threshold value (NO at Step 101), the comparison result of the second comparison unit 88 is waited for.

  The determination unit 89 determines whether or not the power value difference A1-A3 is smaller than the threshold value (step 102). If the power value difference A1-A3 is smaller than the threshold value (determination in step 102 is YES), the breath leak is heard by the listener. Is perceived as having a breath leak (step 103). If the difference A1-A3 is not smaller than the threshold value (NO in step 102), it is determined that there is no breath leak even if there is a breath leak (step 106).

  Next, in step 104, the determination unit 89 determines whether or not the determination that there is a breath leak has been made for three consecutive frames (if the determination in step 104 is YES), the determination unit 89 determines whether the breath data frame is breathing. Produce output to label as leaked. If the determination that there is no breath leakage has not been made for three consecutive frames (NO at Step 104), it is determined that there is no breath leakage (Step 106), and the determination process is terminated without doing anything.

  In this way, by using the above-described breath leak region automatic detection device, it is possible to automatically and continuously label the speech data regarding the breath leak region.

-Related experimental results-
FIG. 5 and FIG. 6 show vowels / a / uttered with various voice qualities from the database of female speakers, and the respective F1 waveforms (third waveform from the top) and F3 waveforms (fourth from the top). Waveform) and the amplitude envelope waveform of those waveforms (the waveforms indicated as F1env and F3env at the bottom, in the drawing, the F1 wave envelope waveform is a line connecting the “Δ” marks, and the F3 wave envelope waveform is “ This is indicated by a line connecting “x” marks.). The waveforms shown in FIGS. 5 and 6 are scaled for proper visual comparison.

  FIG. 5 (a) displays a local voice (Modal voice). In this figure, it is observed that the amplitude envelope waveforms (F1env, F3env) estimated for the F1 waveform and the F3 waveform are synchronized (highly correlated). The synchronization rate in this case (denoted as F1F3syn in the figure) is 0.75.

  FIGS. 6 (a) and 6 (b) show examples of weak breath sounds and strong breath sounds. In the case of a weak breath leak sound, a clear periodicity is observed in the F1 waveform, but in the case of a strong breath leak sound, it is not so clear. However, there is no regularity in the F3 waveform for both weak and strong breath leak sounds. Furthermore, the amplitude envelope waveforms F1env and F3env have different shapes, and the synchronization rate F1F3syn is also low (0.05 and 0.03).

  FIG.5 (b) shows the example of a crouching voice (Creaky voice). A squatting voice often occurs with a low F0 (fundamental frequency), and there may be a case where only one vocal cord pulse is included in the analysis window as shown in FIG. In this case, the amplitude envelope waveforms F1env and F3env have the same shape and the synchronization rate F1F3syn is a high value of 0.87 even when there is no vocal cord pulse periodicity in the analysis window.

  As can be understood from this result, by comparing the synchronization rate of the amplitude envelope waveform with the threshold value, a breath such as a local voice of FIG. 5A having a high synchronization rate and a screaming voice of FIG. 5B is obtained. It is possible to surely eliminate the voice quality that does not leak.

  Next, as speech data, 50 utterances were selected from daily natural conversations of one female speaker in their 30s. These 50 utterances were selected to include a variety of utterance styles and voice qualities (weak breath leaks, strong breath leaks, squatting voices, faint voices, tense sounds, laughter, etc.).

  In each utterance, eight subjects performed a task of adding a mark to a syllable in which breath leakage was perceived. It was determined that there was a breath leak in the syllables given by three or more people.

  50 utterances were manually extracted at the phoneme level, and the acoustic parameters extracted in all frames of the vowel section were used in the evaluation experiment.

  As a result, 42 vowels (847 frames) were determined to have a breath leak, and the remaining 345 vowels (4532 frames) were determined to have no breath leak. FIG. 7 shows data of acoustic parameters extracted in all frames of vowels determined to have (a) / no (b) breath leakage. The horizontal axis represents the synchronization rate F1F3syn, and the vertical axis represents the power value difference A1-A3. The frame around the phoneme boundary (20 ms before and after) and the frame of the voice part of the vowel section determined to have a breath leak were omitted from FIG.

  It can be seen that the value of the synchronization rate is low for vowels with breath leakage, and the difference A1-A3 is capped at about 30 dB. However, the synchronization rate is low even in many frames where it is determined that there is no breath leak. Moreover, the low value of the synchronization rate is concentrated on the high value of the difference A1-A3. When A3 is significantly lower than A1, that is, when the difference A1-A3 is a large value (for example, 30 to 40 dB), there is a possibility that the noise component in the F3 band is not clearly heard. Therefore, when the difference A1-A3 is large, the estimated synchronization rate value is not used for breath leak detection.

  Preliminary evaluation of breath leak detection was performed using a threshold value of 0.4 for the synchronization rate F1F3syn and a threshold value of 25 dB for the difference A1-A3. Many of the vowel frames that were judged to have a breath leak were detected correctly. However, false detections occurred even in some frames of vowels where it was determined that no breath leak was perceived.

  Therefore, as in this embodiment, when the restriction that three frames showing the characteristic of breath leakage must be continued, matching with the syllable determined by the subject reached 90% or more. The reason for adding this restriction is that, as described above, it is considered that a certain length is necessary for the syllable or phrase to perceive a breath leak.

  As described above, according to the present embodiment, it is possible to automatically discriminate between a breath-leak area and a non-breath area in a voice section where there is a breath leak. As a result, it is possible to automatically label voices in a large-scale voice database by determining whether or not they are breath-leak areas according to human senses. It can also be applied when recognizing speech of various utterance styles and voice quality, estimating the emotion of the speaker, or synthesizing voice quality speech that makes the listener feel a specific emotion. . It can also be used for automatic detection of laughing voices with many breath leaks and surprised voices.

  The embodiment disclosed herein is merely an example, and the present invention is not limited to the above-described embodiment. The scope of the present invention is indicated by each claim in the claims after taking into account the description of the detailed description of the invention, and all modifications within the meaning and scope equivalent to the wording described therein are intended. Including.

1 is an external view of a computer system that executes a program for realizing a breath leak region automatic detection device according to an embodiment of the present invention. FIG. FIG. 2 is a block diagram of the computer system of FIG. 1. It is a block diagram of breath leak region automatic detection device 80 of one embodiment of this invention. It is a flowchart which shows the determination process of the breath leak area | region which the determination part 89 performs. It is a figure which displays F1 waveform and F3 waveform about those voice data of various voice qualities, and those amplitude envelope waveforms. Similarly, it is a figure which displays F1 waveform and F3 waveform about those voice data of various voice qualities, and those amplitude envelope waveforms. It is a figure which displays the data of the acoustic parameter extracted in all the frames of the vowel which it was judged that there is a breath leak or is not.

Explanation of symbols

80 breath leak region automatic detection device, 81 F1 band filter unit (first filter unit), 82 F3 band filter unit (second filter unit), 83 F1 envelope estimation unit (first envelope estimation unit), 84 F3 Envelope estimation unit (second envelope estimation means), 85 correlation calculation unit, 86 first comparison unit, 87 intensity difference calculation unit, 88 second comparison unit, 89 determination unit

Claims (6)

A waveform correlation calculating means for obtaining a correlation between an envelope waveform of the signal component of the first formant frequency region of the audio data and an envelope waveform of the signal component of the third formant frequency region;
A comparison means for comparing the correlation result obtained by the correlation calculation means with a predetermined threshold value;
An apparatus for automatically detecting a breath leak region, comprising: a determination unit for determining a breath leak region of the audio data based on a comparison result by the comparison unit. The waveform correlation calculation means includes
First filter means for filtering the audio data with a filter in a first frequency band corresponding to the frequency of the first formant;
Second filter means for filtering the audio data with a filter in a second frequency band corresponding to the frequency of the third formant;
First envelope estimation means for obtaining an envelope waveform of the signal wave filtered by the first filter means;
Second envelope estimating means for obtaining an envelope waveform of the signal wave filtered by the second filter means;
The breath leak region automatic detection device according to claim 1, further comprising computing means for obtaining a correlation between two envelope waveforms respectively obtained by the first and second envelope estimating means. Intensity difference calculating means for calculating the difference between the intensity of the signal wave filtered by the first filter means and the intensity of the signal wave filtered by the second filter means;
3. The breath leak area automatic according to claim 2, wherein the determination means includes means for determining a breath leak area of the audio data based on a comparison result by the comparison means and a calculation result by the intensity difference calculation means. Detection device. The intensity difference calculating means calculates a power value difference between the highest intensity component in the signal wave filtered by the first filter means and the highest intensity component in the signal wave filtered by the second filter means. 4. The breath leak region automatic detection device according to claim 3, comprising means for calculating. The determination means includes
Frame determination means for determining a breath leak area of the audio data based on a comparison result by the comparison means for each frame of the audio data;
Means for determining the predetermined number of frames as a breath leak area in response to the determination result of the breath leak area being obtained by the frame determination means for a predetermined number of frames of the audio data; The breath leak area | region automatic detection apparatus in any one of Claims 1-4 containing this. A computer-executable breath leak region automatic detection program that, when executed by a computer, causes the computer to operate as the breath leak region automatic detection device according to any one of claims 1 to 5.

Images