JP4677548B2 - Paralinguistic information detection apparatus and computer program - Google Patents

Description

  The present invention relates to an apparatus for detecting paralinguistic information independent of utterance contents from human speech, and in particular, detects paralinguistic information from prosody information and voice quality information contained in human speech. The present invention relates to a paralinguistic information detection apparatus for performing the above.

  Due to recent technological advances, various devices that emit human language have been produced. An example of such a device is a car navigation system. A car navigation system is a machine in which a machine makes a one-way utterance to a person, but there are also devices that require dialogue with the person. For example, a robot or the like corresponds to this.

  Devices such as robots are more likely to be in close contact with human life than car navigation systems. Therefore, in order to smoothly communicate with humans using such a device, it is necessary to consider not only human speech content but also emotions.

  When estimating the emotion of a speaker accompanying an utterance, it is reasonable to further consider not only the utterance content but also paralinguistic information such as utterance intention, attitude and emotion, which is information independent of the utterance content. That is, it is more reasonable to estimate human emotions using utterance content and paralinguistic information attached to the utterance content than to learn in advance human emotions corresponding to all expected utterance content. It can be said that it is accurate.

  Conventional research related to the extraction of paralinguistic information has emphasized prosodic features as disclosed in Non-Patent Document 1.

  FIG. 1 shows a functional block diagram of a conventional paralinguistic information extraction apparatus 30 using prosodic features. Referring to FIG. 1, this paralinguistic information extraction device 30 processes an utterance speech signal based on prosody and outputs a speech processing unit 40 based on prosody for outputting a parameter called phrase end tone information, and for learning in advance. By using the probability distribution of the relationship between phrase end tone information and paralinguistic information learned using data, paralinguistic information is extracted from the phrase end tone information obtained from the prosodic speech processing unit 40 and output. A paralinguistic information extraction unit 42 for the purpose. In this conventional technique, a parameter representing a change in pitch called F0move is used as phrase end tone information.

When the user speaks, the voice is converted into a voice signal by a microphone (not shown). This speech audio signal is given to the audio processing unit 40. The phrase end tone information is obtained by the processing in the voice processing unit 40. Paralinguistic information is extracted by the paralinguistic information extracting unit 42 using the end-of-phrase tone information obtained by the processing in the speech processing unit 40 based on prosody.
“Acoustic prosodic features at the end of phrases related to perception in natural speech”. Carlos Toshinori Ishii, Parham Moktari, Nick Campbell, Euro Speech: pp. 405-408, 2003 ("Perceptually-related Acoustic-Prosodic Features of Phrase Finals in Spontaneous Speech", Carlos Toshinori Ishi, Parham Mokhtari, Nick Campbell, Eurospeech 2003: 405-408, 2003)

  In the paralinguistic information detection apparatus 30 using only such prosodic features, paralinguistic information extracted from utterances expressing different emotions may overlap each other.

  This overlap will be described with reference to FIG. Here, pitch change F0move and utterance duration are used as prosodic features.

  The vertical axis of the graph represents the speech duration, and the horizontal axis represents the change in pitch. As shown in the legend 50, each of the symbols plotted in the graph represents the emotion of the speaker. As can be seen from this graph, when only prosodic information is used, different paralinguistic information is represented by the same prosodic information and utterance duration. In other words, it is not clear whether the paralinguistic information having a certain utterance duration and a certain pitch change is “listen” or “surprise / unexpected”. Therefore, the accuracy of paralinguistic information detection decreases.

  In addition, the expressive utterance voice includes a voice whose breathing is difficult to extract, such as a breathing voice that is a voice including a breath leak.

  Therefore, an object of the present invention is to solve these problems and provide highly accurate paralinguistic information that can distinguish paralinguistic information more clearly than when only prosodic features are used when detecting paralinguistic information. is there.

  Another object of the present invention is to provide a highly accurate paralinguistic information detection apparatus by extracting paralinguistic information using not only prosodic information but also voice quality information.

  A paralinguistic information detection apparatus according to a first aspect of the present invention is a paralinguistic information detection apparatus for detecting paralinguistic information independent of utterance content from a human utterance voice signal, the prosody of the utterance voice signal. First speech processing means for processing information relating to speech, second speech processing means for processing information relating to voice quality of the speech signal, paralinguistic information relating to speech speech from information relating to prosody and information relating to voice quality And paralinguistic information extracting means for extracting.

  Preferably, the second speech processing means includes vocal / fly ratio calculating means for calculating a ratio of the vocal / fly section in the speech section of the speech signal.

  More preferably, the second sound processing means further includes a non-periodic / double periodicity ratio calculating means for calculating a ratio occupied by the non-periodic / double periodicity period in the utterance period of the speech signal. .

  More preferably, the second sound processing means includes a non-periodic / double periodicity ratio calculating means for calculating a ratio of the non-periodic / double periodicity period in the utterance section of the speech signal.

  More preferably, the second voice processing means further includes a breathing ratio calculation means for calculating a ratio of the breathing section in the utterance section of the utterance voice signal.

  More preferably, the second voice processing means includes a breathing ratio calculation means for calculating a ratio of the breathing section in the utterance section of the utterance voice signal.

  When the computer program according to the second aspect of the present invention is executed by a computer, it causes the computer to operate as one of the paralinguistic information detection devices described above.

  According to this paralinguistic information detection apparatus, not only information related to prosody but also information related to voice quality can be used for information detection. Therefore, the accuracy of paralinguistic information detection can be increased. Therefore, it is possible to provide a more accurate paralinguistic information detection apparatus.

  Hereinafter, an embodiment of the present invention will be described with reference to the drawings. The present embodiment relates to a paralinguistic information detection device for performing parametric sound processing and voice quality voice processing from speech signals and extracting paralinguistic information.

<Configuration>
FIG. 3 shows a functional block diagram of the paralinguistic information detection device 60 according to the present embodiment. Referring to FIG. 3, this paralinguistic information detection device 60 includes a prosodic speech processing unit 70 for processing a speech signal based on prosody and outputting parameters used for extraction of paralinguistic information, and voice quality. Obtained from the voice processing unit 72 by voice quality for processing the speech signal based on the voice and outputting parameters used for extracting paralinguistic information, the voice processing unit 70 by prosody, and the voice processing unit 72 by voice quality A paralinguistic information extracting unit 74 for extracting and outputting paralinguistic information in accordance with a probability distribution indicating a relationship between the parameter and paralinguistic information, which has been learned in advance using learning data.

  FIG. 4 is a functional block diagram showing details of the speech processing unit 70 based on prosody. Referring to FIG. 4, the prosody speech processing unit 70 includes a prosody feature processing unit 80 for performing processing of converting the speech signal into F0move, which is a parameter representing a pitch movement, that is, a change in pitch. Tone for extracting tone parameters from utterance duration extraction unit 84 for extracting information about time, F0move obtained by prosodic feature processing unit 80 and utterance time information obtained by utterance duration extraction unit 84 A parameter extraction unit 82.

  The tone parameters will be described with reference to FIG. Here, Japanese “None” is taken as an example. The tone parameter is a parameterization of the upper and lower pitches included in a word. For example, in the tone parameter 1a (100), the pitch changes between “N” and “I” in the word “None”. And the change is that the pitch goes down when moving from "NA" to "I".

  The symbol 示 shown in FIG. 5 indicates that the pitch is lowered, the symbol ┌ indicates that the lowered pitch returns to the original pitch, and the arrow symbol rising to the right indicates that the pitch is increased.

  Although there are seven types of tone parameters shown in FIG. 5, in the present embodiment, five types of 1a (100), 2a (102), 2b (104), 2c (106), and 3 (108) are used. To do.

  FIG. 6 is a functional block diagram showing details of the prosodic feature processing unit 80. Referring to FIG. 6, the prosody feature processing unit 80 includes a F0 extraction unit 90 for obtaining a parameter F0 that is information about a pitch from a speech signal, and a pitch movement (direction and direction) within a syllable using the parameter F0. Degree), that is, an F0move extraction unit 92 for extracting F0move, which is a parameter representing a change in pitch in semitone units. The F0 extraction unit 90 extracts only F0, which is information relating to the pitch of the sound, from the utterance voice signal, and converts it so as to be expressed by a musical scale.

  FIG. 7 is a functional block diagram showing details of the voice processing unit 72 based on voice quality. Referring to FIG. 7, the voice processing unit 72 based on voice quality calculates a vocal / fly detection unit 120 for detecting vocal / fly from a speech signal, and a ratio of the vocal / fly period in the entire utterance period. And a vocal / fly ratio calculation unit 122. Here, the vocal fly is a pulse sound of a very low frequency of about 7 Hz to 78 Hz, which is generated when the excitation of the vocal tract is almost attenuated.

  The voice processing unit 72 according to voice quality further includes non-periodic section information which is information of a section other than the vocal / fly section and a section having a non-periodic speech waveform and a section having a double period, in the given speech signal. A non-periodic / double-periodicity detecting unit 124 for detecting double-periodic interval information, and vocal / fly detection from the non-periodic and double-periodic interval information detected by the aperiodic / double-periodicity detecting unit 124 Aperiodic / double periodicity ratio calculation for calculating the ratio of the non-periodic section information and the double periodic section information in all utterance sections except for the vocal / fly section information 132 detected by the unit 120 Part 126. Here, aperiodicity means that the speech waveform is aperiodic. Double periodicity means that a speech waveform has a shape in which a set of waveforms composed of two waveforms having different peak lengths and peak widths is periodically repeated.

  The voice processing unit 72 based on voice quality further includes a breathing property detection unit 128 for detecting breathing interval information from a given utterance voice signal, and a breath for calculating the proportion of the breathing interval in all the speech segments. A sex ratio calculation unit 130. Here, the breathability is the degree of breath leakage included in the voice. Examples of breathable voices include whispering voices.

  FIG. 8 is a functional block diagram showing details of the vocal / fly detection unit 120. Referring to FIG. 8, the vocal / fly detection unit 120 uses a band pass filter 140 for passing only a frequency component of 100 Hz to 1500 Hz in an utterance signal, and an utterance signal 154 that has passed through the band pass filter 140 in a very short period. Each frame is converted into a frame length, and information 150 indicating the frame position of each frame having a power larger than that of the two frames before and after the frame and having a difference larger than a predetermined power threshold is set as a power peak candidate. A value related to intra-frame periodicity (IFP value) is calculated for an ultra-short-term peak detection processing unit 142 for outputting as position information and an utterance signal 154 framed with a short-term frame length. Null IFP value for frames other than frames with a predetermined number of periodicities Of the peak position information 150 provided from the short-term periodicity detection unit 144 and the ultra-short-term peak detection processing unit 142, the frame value is determined by the short-term periodicity information 152 provided from the short-term periodicity detection unit 144. A periodicity checking unit 146 for giving only the information 156 of the null portion to the similarity checking unit 148, a waveform near the power peak candidate specified by the information 156, and a waveform near the power peak before that The peak position information is detected when the value related to the inter-pulse similarity (IPS value) is greater than or equal to a predetermined threshold value, and the IPS value between adjacent pulses is detected based on the peak position information. Vocal / fly interval information is detected from the frame between the higher frames, and the vocal / fly ratio calculation unit 122 and the non-periodic / double periodicity ratio calculation unit 1 26 and a similarity checking unit 148 for giving to H.26.

  FIG. 9 is a functional block diagram showing details of the aperiodic / double periodicity detecting unit 124. Referring to FIG. 9, the aperiodic / double periodicity detecting unit 124 filters the speech signal and detects the peak of the speech waveform, thereby calculating a normalized self-correlation function for calculating a normalized autocorrelation function. From the correlation function calculation unit 160 and the waveform of the normalized autocorrelation function based on the normalized autocorrelation function calculated by the normalized autocorrelation function calculation unit 160, the normalized self represented by the relationship between the peak value and the peak position, etc. A normalized autocorrelation function parameter calculation unit 162 for calculating a correlation function parameter, and an aperiodicity for detecting aperiodic and double periodicity interval information from the calculated normalized autocorrelation function parameter value A double periodic section information detection unit 164.

  The normalized autocorrelation function parameter calculation unit 162 detects the first two peaks (P1 and P2) from the normalized autocorrelation function obtained by the normalized autocorrelation function calculation unit 160. However, only the peak value exceeding 0.2 is regarded as a peak.

  The normalized autocorrelation values of these peaks are called NAC (P1) and NAC (P2), and the normalized autocorrelation positions are called TL (P1) and TL (P2), and are treated as normalized autocorrelation function parameters.

  FIG. 10 is a functional block diagram showing details of the normalized autocorrelation function calculation unit 160. Referring to FIG. 10, normalized autocorrelation function calculation section 160 emphasizes a high-pass filter 170 for passing only frequency components of 60 Hz or higher in the speech signal, and a high-frequency portion of the audio signal output from high-pass filter 170. The high-frequency emphasis unit 172 for performing the processing to perform the linear prediction analysis on the speech signal output from the high-frequency emphasis unit 172, the vocal tract parameter extraction unit 174 extracts the vocal tract parameters, and the inverse filter 176 When a vocal tract parameter extracted by the vocal tract parameter extraction unit 174 is used for the voice signal output from the filter 170 and an inverse filter is performed to obtain a residual signal corresponding to the vocal cord sound source waveform, it is necessary for subsequent processing. In order to facilitate peak detection, a low pass filter 178 for passing only an audio signal of 2 kHz or less and a low pass filter 178 are provided. When the received audio signal is given, the window size is set to 80 ms, and the autocorrelation function calculation unit 180 for calculating the autocorrelation function from the audio signal included in the window is calculated by the autocorrelation function calculation unit 180. A peak detection unit 182 for detecting the maximum peak included in each frame from the waveform of the autocorrelation function, and the maximum peak detected by the peak detection unit 182 and the maximum peak immediately before or immediately after the peak The time difference is extracted, the frame length is readjusted so that the time four times the shifted time becomes one frame, and the autocorrelation function is recalculated to calculate the autocorrelation function included in the readjusted frame. It includes a calculation unit 184 and a normalization unit 186 for performing processing for normalizing the obtained autocorrelation function.

  FIG. 11 is a functional block diagram showing details of the breath detection unit 128. Referring to FIG. 11, the breath detection unit 128 includes an F1 pass filter 202 for passing only frequency components of 100 Hz to 1500 Hz in the speech signal, and an entire waveform that has passed through the F1 pass filter 202. From the amplitude envelope extraction unit 204 for extracting the change in amplitude, the F3 pass filter 200 for passing only the frequency components of 1800 Hz to 4000 Hz in the speech signal, and the entire waveform passing through the F3 pass filter 200, Amplitude envelope extraction unit 210 for extracting amplitude change, and for calculating a cross-correlation between the amplitude change obtained from amplitude envelope extraction unit 204 and the amplitude change obtained from amplitude envelope extraction unit 210 Cross-correlation calculation unit 214. Here, the frequency that has passed through the F1 pass filter 202 is referred to as F1 wave, and the frequency that has passed through the F3 pass filter 200 is referred to as F3 wave. Also, the change in amplitude extracted by the amplitude envelope extraction unit 204 is called F1 amplitude envelope, and the change in amplitude extracted by the amplitude envelope extraction unit 210 is called F3 amplitude envelope.

  The breathability detection unit 128 further includes a first maximum frequency component extraction unit 206 for extracting a maximum frequency component from an F1 wave including a component that has passed through the F1 pass filter 202, and a component that has passed through the F3 pass filter 200. A second maximum frequency component extraction unit 212 for extracting a maximum frequency component from the F3 wave, and a spectral tilt that is a difference between the maximum frequency component included in the F1 wave and the maximum frequency component included in the F3 wave And a spectrum inclination calculation unit 216 for calculating the A1-A3 value.

  The breath detection unit 128 further has a threshold value of the spectrum slope A1-A3 obtained from the spectrum slope calculation unit 216 and the F1F3 correlation value obtained from the cross correlation calculation unit 214 is less than a certain threshold value. An air breath determination unit 218 is provided for determining whether it is an air breath interval based on whether it is less than the value and outputting the air breath interval information.

<Operation>
Referring to FIG. 3, first, when the user utters, the uttered voice is converted into an uttered voice signal by a microphone (not shown). The speech signal converted by the microphone is given to the speech processing unit 70 based on prosody and the speech processing unit 72 based on voice quality. Phrase end tone information is obtained by processing in the speech processing unit 70 based on this prosody. Information on the voice / fly ratio, the ratio of non-periodicity and double-periodicity, and the ratio of breathability in the entire utterance can be obtained by processing in the voice processing unit 72 based on voice quality. Details of processing in the speech processing unit 70 based on prosody and the speech processing unit 72 based on voice quality will be described later.

  With reference to FIG. 4, the details of the operation of the speech processing unit 70 based on prosody will be described. When the utterance voice signal is received, the prosody feature processing unit 80 first performs a process of converting the utterance voice signal into F0move which is a parameter representing a change in pitch, that is, a change in pitch. F0move is obtained from F0, which is information about the pitch.

  Details of the operation in the prosodic feature processing unit 80 will be described with reference to FIG. When the utterance voice signal is received, the F0 extraction unit 90 extracts only the information about the pitch of the sound from the utterance voice signal and converts it into scale information to obtain the parameter F0.

  Using the parameter F0, the F0move extraction unit 92 extracts F0move, which is a parameter that represents a pitch movement (direction and degree) within a syllable, that is, a change in pitch in semitone units. F0move can be obtained from the difference between a plurality of F0s.

  Referring to FIG. 4, utterance duration extraction unit 84 extracts information related to the utterance duration from the utterance voice signal.

  A tone parameter is extracted by the tone parameter extraction unit 82 using the F0move extracted by the prosodic feature processing unit 80 and the information related to the utterance duration extracted by the utterance duration extraction unit 84. The extracted tone parameters are used for processing in the later paralinguistic information extraction unit 74.

  Referring to FIG. 7, voice processing unit 72 based on voice quality operates as follows. First, vocal / fly section information is detected by the vocal / fly detection unit 120 from the speech signal.

  Referring to FIG. 8, the vocal / fly detection unit 120 operates as follows. The band pass filter 140 passes only frequency components of 100 Hz to 1500 Hz in the speech signal. The utterance signal 154 that has passed through the bandpass filter 140 is given to the ultra-short-term peak detection processing unit 142, the short-term periodicity detection unit 144, and the similarity check unit 148. The ultra-short-term peak detection processing unit 142 converts the utterance signal 154 into ultra-short-term frames and calculates ultra-short-term power for each frame. Then, for each frame, a frame whose power difference is larger than the power threshold value compared to the two frames before and after the frame is set as a power peak candidate, and information 150 indicating the frame position is output.

  The short-term periodicity detection unit 144 frames the speech signal 154 and calculates an IFP value for each frame. The calculated IFP value is compared with a threshold value, and if it is less than the threshold value, the IFP value of the frame is set to null. If the non-null frames are not continuous by at least 3 frames, the IFP values of those frames are corrected to null. Then, the corrected IFP value is given to the periodicity inspection unit 146.

  The periodicity inspection unit 146 sets the frame IFP value to null by the short-term periodicity information 152 given from the short-term periodicity detection unit 144 among the peak position information 150 given from the ultrashort-term peak detection processing unit 142. Only the information 156 of the existing part is given to the similarity checking unit 148.

  The similarity checking unit 148 calculates an IPS value between the waveform near each power peak of the power peak candidate existing in the section specified by the information 156 and the waveform near the previous power peak. Then, the IPS value is compared with a threshold value, and peak position information of a power peak equal to or greater than the threshold value is detected. Based on this peak position information, a frame between adjacent pulses having a high IPS value is detected as a vocal / fly interval, and information indicating them (vocal / fly interval information) is output.

  Referring to FIG. 7, the detected vocal / fly interval information is provided to vocal / fly ratio calculation section 122. From the vocal / fly section information, the ratio of the vocal / fly section in all utterance sections is calculated by the vocal / fly ratio calculation unit 122. This calculation is obtained by dividing the vocal / fly interval by the total utterance interval. The calculated vocal / fly interval ratio information is provided to the para-language information extraction unit 74 for later processing.

  The non-periodic / double periodicity detecting unit 124 detects an aperiodic section and a double-periodic section, which are information of a section in which the speech waveform is aperiodic and a section in which the speech waveform is a double period, from the speech signal. Aperiodic section information and double periodic section information indicating them are output.

  Referring to FIG. 9, aperiodic / double periodicity detection unit 124 operates as follows. When an utterance voice signal is given, the normalized autocorrelation function calculation unit 160 calculates an autocorrelation function by analyzing a voice waveform obtained by filtering the voice signal. Then, the autocorrelation function is normalized and a normalized autocorrelation function is calculated. Details of processing in the normalized autocorrelation function calculation unit 160 will be described below.

  Referring to FIG. 10, when an utterance signal is given, only a frequency component of 60 Hz or higher is passed by high-pass filter 170. The audio signal of 60 Hz or higher is given to the high frequency emphasizing unit 172 and the inverse filter 176. The high frequency emphasizing unit 172 performs processing for emphasizing the high frequency part of the given audio signal. Then, the vocal tract parameter extraction unit 174 estimates a filter parameter that characterizes the vocal tract. Thereafter, an inverse filter 176 is performed to obtain a vocal cord sound source signal using the vocal tract parameters given by the vocal tract parameter extraction unit 174 to the output speech signal of the high pass filter 170.

  The residual signal processed by the inverse filter 176 is then provided to the low pass filter 178. The low-pass filter 178 allows only frequency components of 2 kHz or less to pass in order to facilitate peak detection required for subsequent processing. The frequency component that has passed through the low-pass filter 178 is given to the autocorrelation function calculation unit 180 and the autocorrelation function recalculation unit 184. The autocorrelation function calculation unit 180 sets the frame size used in the detection process to 80 ms, and obtains the autocorrelation function from the sound signal waveform in the frame. Then, this autocorrelation function is output.

  In the peak detection unit 182, processing for detecting the maximum peak included in the autocorrelation function obtained by the autocorrelation function calculation unit 180 is performed.

  The autocorrelation function recalculation unit 184 first sets a time that is four times the position of the maximum peak detected by the peak detection unit 182 as a new frame length. Such readjustment of the frame is performed in order to appropriately calculate the autocorrelation function. That is, in the case of a fixed frame length, it is difficult to appropriately calculate the autocorrelation function even if the frame is too large or too small. Then, an autocorrelation function is obtained again from the frame.

  Next, a process for normalizing the autocorrelation function obtained by the normalization unit 186 is performed. Referring to FIG. 9, based on the normalized autocorrelation function calculated by normalized autocorrelation function calculation unit 160, a calculation process by normalized autocorrelation function parameter calculation unit 162 is performed. Then, a peak value and a peak position are detected from the waveform of the normalized autocorrelation function in order to extract the non-periodicity and double periodicity of the sound wave. Thereafter, the ratio of the peak values and the ratio of the peak positions are calculated. The ratio of peak values is obtained by 1000 * NAC (P2) / NAC (P1). Further, the ratio of peak positions is obtained by 2000 * TL (P2) / TL (P1).

  Further, using the calculated normalized autocorrelation function parameter, the non-periodic / double-periodic section information detection unit 164 detects a section in which the speech signal has aperiodicity or double-periodicity. Details of this detection processing are as follows.

  That is, if all of the above-mentioned autocorrelation function parameters are values close to 1000, it can be said that the speech speech waveform in the section represented by the waveform of the autocorrelation function has periodicity. Therefore, it is possible to extract utterance sections that take other values as aperiodic and double periodic sections.

  The non-periodic / double-periodic section information detected by the non-periodic / double-periodic section information detection unit 164 is provided to the non-periodic / double-periodicity ratio calculation unit 126.

  Referring to FIG. 7, the ratio of the non-periodic section and the double periodic section in all utterance sections is calculated by a non-periodic / double periodic ratio calculation unit 126. This calculation is performed by dividing the non-periodic section and the double periodic section by the entire speech section.

  Before this calculation process, the vocal / fly detection unit 120 first performs a process of removing the section information detected as the vocal / fly section from the non-periodic / double periodic section information. This is because vocal fly also has non-periodic characteristics, but here, non-periodic / double periodicity other than vocal fly is targeted.

  Referring to FIG. 11, breathability detection unit 128 operates as follows. When the utterance voice signal is given, the F1 pass filter 202 first passes only the frequency components of 100 Hz to 1500 Hz in the utterance voice signal. The amplitude envelope extraction unit 204 extracts the amplitude envelope from the waveform of the F1 wave that has passed through the F1 pass filter 202.

  Similarly, in the F3 pass filter 200, only the frequency component of 1800 Hz to 4000 Hz is passed in the speech signal. Then, the amplitude envelope extraction unit 210 extracts the amplitude envelope from the waveform of the F3 wave that has passed through the F3 pass filter 200.

  A cross-correlation calculation unit 214 calculates a cross-correlation between the F1 amplitude envelope obtained from the amplitude envelope extraction unit 204 and the F3 amplitude envelope obtained from the amplitude envelope extraction unit 210. By this processing, an F1F3 correlation value indicating the mutual relationship between the F1 amplitude envelope and the F3 amplitude envelope is obtained.

  From the F1 wave that has passed through the F1 pass filter 202, the maximum frequency component extraction unit 206 extracts the maximum frequency component from those included in this frequency band. The same processing is performed by the maximum frequency component extraction unit 212 on the F3 wave that has passed through the F3 pass filter 200. A process for calculating a difference between the maximum frequency component included in the F1 wave and the maximum frequency component included in the F3 wave, that is, a spectrum tilt, is performed by the spectrum tilt calculating unit 216. This spectral inclination is defined as A1-A3.

  The breathability determination unit 218 determines whether or not it is breathable by using the F1F3 correlation value and the spectrum inclination A1-A3 value, and outputs breathability section information. In this process, if the F1F3 correlation value is less than a certain threshold value and the A1-A3 value is less than a certain threshold value, it is determined that the breathing interval is present. These threshold values are obtained by learning in advance. By comparing and comparing this threshold value, the actually obtained F1F3 correlation value, and A1-A3, the presence or absence of breathability can be determined.

  The breathability interval information is given to the breathability ratio calculation unit 130. Referring to FIG. 7, the breathing rate calculation unit 130 calculates the ratio of the breathing interval in all speech segments by dividing the breathing interval by all speech segments. The calculated breathing interval ratio is given to the paralinguistic information extraction unit 74 for later processing.

  Referring to FIG. 3, the end-of-phrase tone information obtained by the processing in the speech processing unit 70 based on the prosody, the information on the ratio of the vocal fly to the entire utterance obtained by the processing in the speech processing unit 72 based on the voice quality, Paralinguistic information is extracted by the paralinguistic information extraction unit 74 using information on the ratio of non-periodicity or double periodicity in the entire utterance and information on the ratio of breathability in the entire utterance.

  In this process, the end-of-phrase tone information, the proportion of vocal / fly intervals in the entire utterance, the proportion of non-periodic and double-periodic intervals, and the relationship between the information about the breathability and the paralinguistic information Data needs to be accumulated. By learning, it is possible to create a model of what kind of paralinguistic information can be detected by inputting what kind of parameters from the accumulated data.

  As this model, a decision tree, a neural network, a support vector machine (SVM), and the like can be considered.

[Realization by computer]
The system of this embodiment is realized by computer hardware, a program executed by the computer hardware, and data stored in the computer hardware. FIG. 12 shows the external appearance of the computer system 330, and FIG. 13 shows the internal configuration of the computer system 330.

  Referring to FIG. 12, this computer system 330 includes a computer 340 having an FD (flexible disk) drive 352 and a CD-ROM (compact disk read only memory) drive 350, a keyboard 346, a mouse 348, and a monitor 342. including.

  Referring to FIG. 13, in addition to FD drive 352 and CD-ROM drive 350, computer 340 includes CPU (central processing unit) 356 and bus 366 connected to CPU 356, FD drive 352 and CD-ROM drive 350. And a read only memory (ROM) 358 for storing a boot-up program and the like, and a random access memory (RAM) 360 connected to the bus 366 for storing a program command, a system program, work data, and the like. Computer system 330 further includes a printer 344.

  Although not shown here, the computer 340 may further include a network adapter board that provides a connection to a local area network (LAN).

  A computer program for causing the computer system 330 to operate as the paralinguistic information extracting device 60 is stored in the CD-ROM 362 or FD 364 inserted in the CD-ROM drive 350 or FD drive 352 and further transferred to the hard disk 354. The Alternatively, the program may be transmitted to the computer 340 through a network (not shown) and stored in the hard disk 354. The program is loaded into the RAM 360 when executed. The program may be loaded directly into the RAM 360 from the CD-ROM 362, from the FD 364, or via a network.

  This program includes a plurality of instructions for causing the computer 340 to operate as the paralinguistic information extracting device 60 of this embodiment. Some of the basic functions required to perform this operation are provided by operating system (OS) or third party programs running on the computer 340 or various toolkit modules installed on the computer 340. Therefore, this program does not necessarily include all functions necessary for realizing the system and method of this embodiment. This program calls only an instruction for executing the operation as the paralinguistic information extracting device 60 described above by calling an appropriate function or “tool” in a controlled manner so as to obtain a desired result. It only has to be included. The operation of computer system 330 is well known and will not be repeated here.

  As described above, when detecting paralinguistic information, not only information related to prosody but also information related to voice quality is used, thereby increasing the accuracy of detecting paralinguistic information.

  Specific numbers used in the embodiments disclosed this time are examples.

  Further, the embodiment disclosed this time is merely an example, and the present invention is not limited to the embodiment described above. The scope of the present invention is indicated by each claim in the claims after taking into account the description of the detailed description of the invention, and all modifications within the meaning and scope equivalent to the wording described therein are intended. Including.

It is a functional block diagram of the paralinguistic information extraction device 30 using a prosodic feature. It is a graph showing the overlap of paralinguistic information at the time of detecting paralinguistic information using a prosodic feature. It is a functional block diagram about the paralinguistic information detection apparatus 60 which concerns on this Embodiment. It is a functional block diagram which shows the detail of the process of the audio | voice processing part 70 by a prosody. It is a figure explaining a tone parameter. 3 is a functional block diagram showing details of a prosodic feature processing unit 80. FIG. It is a functional block diagram which shows the detail of the audio | voice processing part 72 by voice quality. 3 is a functional block diagram showing details of a vocal / fly detection unit 120. FIG. 4 is a functional block diagram illustrating details of an aperiodic / double periodicity detection unit 124. FIG. 3 is a functional block diagram illustrating details of a normalized autocorrelation function calculation unit 160. FIG. 3 is a functional block diagram showing details of a breath detection unit 128. FIG. 1 is an external view of a computer system that implements a paralinguistic information extraction device 30 according to an embodiment of the present invention. It is a block diagram of the computer shown in FIG.

Explanation of symbols

70 speech processing unit 72 based on prosody speech processing unit 74 based on voice quality para language information extraction unit 122 vocal / fly ratio calculation unit 126 non-periodic / double periodicity ratio calculation unit 130 breathing ratio calculation unit

Claims (6)

A paralinguistic information detection device for detecting paralinguistic information independent of utterance content from a human utterance voice signal,
First speech processing means for processing information relating to the prosody of the speech signal;
Second voice processing means for processing information relating to voice quality of the speech signal;
And paralinguistic information extracting means for extracting paralinguistic information on speech from the information about the information and the voice quality for said prosodic seen including,
The para-linguistic information detection apparatus , wherein the second voice processing means includes a vocal / fly ratio calculation means for calculating a ratio of a vocal / fly section in an utterance section of the speech signal . The second sound processing means further includes a non-periodic / double periodicity ratio calculating means for calculating a ratio of the non-periodic / double periodicity period in the utterance period of the speech signal. Item 3. The paralinguistic information detection device according to Item 1 . A paralinguistic information detection device for detecting paralinguistic information independent of utterance content from a human utterance voice signal,
First speech processing means for processing information relating to the prosody of the speech signal;
Second voice processing means for processing information relating to voice quality of the speech signal;
Including paralinguistic information extracting means for extracting paralinguistic information about speech from the information about the prosody and the information about the voice quality,
The second speech processing means includes paralinguistic information including a non-periodic / double periodicity ratio calculating means for calculating a ratio of the non-periodic / double periodicity section in the utterance section of the speech signal. Detection device. It said second voice processing means further comprises a breathiness ratio calculating means for calculating a ratio of the breathiness interval during speech period of the speech signal, to any one of claims 1 to 3 The paralinguistic information detection device described. A paralinguistic information detection device for detecting paralinguistic information independent of utterance content from a human utterance voice signal,
First speech processing means for processing information relating to the prosody of the speech signal;
Second voice processing means for processing information relating to voice quality of the speech signal;
Including paralinguistic information extracting means for extracting paralinguistic information about speech from the information about the prosody and the information about the voice quality,
The paralinguistic information detection apparatus, wherein the second voice processing means includes a breathing rate calculation unit for calculating a ratio of a breathing period in the utterance section of the utterance voice signal. A computer program that, when executed by a computer, causes the computer to operate as the paralinguistic information detection device according to any one of claims 1 to 5 .

Images