JP2018180334A - Emotion recognition device, method and program - Google Patents

Abstract

PROBLEM TO BE SOLVED: To recognize emotion at the time of utterance from a voice signal with high accuracy and high reliability.SOLUTION: An autocorrelation processing unit 20 outputs a normalized autocorrelation signal for a segment decomposed by a segmentation unit 10. A feature quantity extraction unit 40 extracts a feature quantity of an analysis frame from the normalized autocorrelation signal and a sound intensity measurement unit 30, using a period of several 10 ms to several 100 ms including one or more segments as an analysis frame. A pattern identification and emotion recognition unit 50 recognizes the emotion at the time of utterance of a voice signal on the basis of the correlation between the feature quantity extracted by the feature quantity extraction unit 40 acquired in the preliminary learning and the emotion, and outputs an emotion label of the recognition result.SELECTED DRAWING: Figure 1

Description

  The present invention relates to an emotion recognition apparatus, method, and program, and more particularly to an emotion recognition apparatus, method, and program capable of highly accurately and reliably recognizing an emotion at the time of speech from a voice signal.

  In order to recognize an emotion at the time of speech from a speech signal, it is necessary to extract a feature quantity that is an index of the emotion from the speech signal.

  Patent Document 1 describes a method of detecting a voice speed, a voice pitch frequency, a volume of voice information, and a voice spectrum of voice information from voice information and using them as emotion information attached to the voice information.

  In Patent Document 2, as a feature derived from a speech signal, the strength and pitch of the speech signal are extracted, and emotions are recognized from statistical representative values such as the average value, maximum value, and minimum value of those time series data. Methods are described.

  In Patent Document 3, an amplitude envelope of an input speech signal is extracted, the frequency of the periodic fluctuation is determined, and when it is within a predetermined range, it is estimated that the speech is in a state of force. The method is described. Patent Document 3 also describes that anger intensity is detected in short units called phonemes.

  According to Patent Document 4, from the speech signal data, the fundamental frequency, the time variation characteristic of the fundamental frequency, the rms value of the amplitude, the time variation characteristic of the rms value of the amplitude, the time fluctuation characteristic of the power and the power, the speech velocity There is described a method of extracting time-varying characteristics of speech velocity as speech feature quantities and estimating an emotion of speech signal data.

Unexamined-Japanese-Patent No. 9-22296 gazette Japanese Patent Application Laid-Open No. 2003-99084 JP, 2009-3162, A JP 2008-204193 A

  Prosodic information such as speech height, speech intensity (power), speech rate (speech rate) is used not only to convey emotions, but also to convey linguistic information. For example, in Japanese, homonyms are distinguished by high and low accents of the fundamental frequency. Therefore, in recognition of emotion from speech signal, it is necessary to distinguish from prosody change as linguistic information and to recognize emotion from prosody change as emotion expression. In addition, it is desirable to extract, from the audio signal, a feature that reflects the emotion at the time of speech as much as possible.

  The method described in Patent Document 1 extracts prosodic features such as speech height (fundamental frequency), speech intensity, speech speed and the like in units of speech or phrase, and based on the prosodic features, features for the entire speech information, for example , “High voice”, “high voice”, etc. to recognize the sensibility associated with speech information, and it does not distinguish between prosody change as emotional expression and prosody change as linguistic information, so it is erroneous. There is a problem that recognition tends to occur. In addition, in this method, there is also a problem that misrecognition tends to occur due to variations in prosodic features caused by individual differences in speech, such as rapid voice / high voice / low voice.

  The method described in Patent Document 2 determines time series data of speech intensity and fundamental frequency in units of speech or phrase, and recognizes emotion from statistical representative values such as average value, maximum value, and minimum value of time series data. There is a problem similar to the method described in Patent Document 1. In addition, in this method, it is difficult to stably and accurately recognize emotion because it is easily affected by the time length of the utterance, the presence or absence of the pause in the utterance, and the depth thereof.

  The method described in Patent Document 3 is for estimating speech in a strong state and its anger intensity, and here also, since it does not distinguish between prosodic change as emotional expression and prosodic change as linguistic information, Patent Document 1 There are the same issues as the described method. Although patent document 3 also describes detecting anger intensity by a short unit called a phonology, the method is the power at the time of the utterance of every phonology about the speech signal presumed to be the utterance in a state of pressure. The strength of the anger is determined from the strength speech generation index indicating the ease of viewing, and the strength speech generation index is an attribute of the phoneme such as a consonant, a vowel, a position in an accent phrase, a relative position from an accent nucleus, etc. It is calculated using the rules for finding the ease of speaking from information.

  It is conceivable to use a statistical model to distinguish linguistic information and to recognize emotions based on prosodic features in consideration of variations in prosody information due to individual differences, but it is conceivable to use the statistical model. Requires a large amount of learning data, and learning requires the labeling of emotions. Therefore, in practice, in addition to voice data, image data, biometric data, etc. are simultaneously acquired, for example, emotion labels are generated from image data or biometric data, and learning is mechanically advanced using the emotion labels, making a mistake It is necessary to perform an operation such as manually confirming the data.

  In the method described in Patent Document 4, the voice feature quantity is extracted for each analysis frame to estimate the emotion, but the voice feature quantity shown here is the basic frequency, amplitude, power, voice speed of voice, etc. It is.

  As described in Patent Documents 1 to 3 and the like, the feature quantities in the case of recognizing emotions in utterance or phrase units are discussed in many documents, but the feature quantities in the case of recognizing emotions in analysis frame units Is discussed only in a small number of documents such as Patent Document 4.

  The feature quantities disclosed in Patent Document 4 are the same as those of Patent Documents 1 to 3, such as the fundamental frequency, amplitude, power, and voice speed of voice. As described above, a feature that is effective when recognizing an emotion in analysis frame units sufficiently shorter than the time length of the utterance or the phrase has not been sufficiently discussed.

  FIG. 29 and FIG. 30 are diagrams showing an example of statistical representative values (average value, maximum value, minimum value, etc.) for each phrase obtained from time series data of fundamental frequency and speech strength, and the figure is This is shown in order to see the relationship between the feature amount in utterance or phrase units and the feature amount in analysis frame units.

  In FIG. 29, the average value of the fundamental frequency in the phrase is 160 Hz, the maximum value 200 Hz when the time is 0.32 seconds, and the minimum value 80 Hz when the time is 0.77 seconds. Further, a period in which the fundamental frequency is not displayed is a period in which the fundamental frequency can not be extracted because it is unvoiced sound (sound without vocal cord vibration), has a low voice intensity, or has no dominant frequency component.

  In the conventional method of recognizing emotions using statistical representative values, the maximum value, minimum value, average value, etc. are used for the fundamental frequency, and in the example of FIG. 29, the maximum value and the range (maximum value And because the difference between the minimum and the maximum) is large, the average is not low, so the emotion is recognized as "anger" or "surprise."

  On the other hand, when the feature value of the fundamental frequency is extracted in analysis frame units, the fundamental frequency takes a value within a typical range in most time zones, and the inclination has many time zones in which the inclination is larger than usual. .

  In addition, in the sense of hearing, assuming the scene that yells at "do not be upset" or "does not care" and yells, "f", "za", "da", "ru", "na" in each sound Even, "I", "I", "D", "D", "D", "D", "D", "D", "D", "D", "E" and "Y" also contain anger emotions. It is thought that Of course, although the feature that the voice intensity is large is extracted from the rough voice, there is also an utterance that feels easy even at a large volume. Therefore, it is considered that the emotion of anger is expressed in some form other than voice intensity.

  As described above, in an analysis frame sufficiently shorter than the time length of speech or phrase, emotion is transmitted by some characteristic other than a statistical representative value such as fundamental frequency of speech signal, speech intensity, amplitude, and speech rate. It is thought that

  By the way, the speech information in the analysis frame can be completely described by an autocorrelation function or a power spectrum except for phase information which can not often be perceived by humans. Therefore, by describing the autocorrelation function as a model parameter and statistically modeling the correlation between the model parameter and the emotion, it is considered that the emotion at the time of speech of the speech signal can be recognized with high accuracy and high reliability.

  An object of the present invention is to solve the above-mentioned problems, and to provide an emotion recognition apparatus, a method and a program capable of well recognizing an emotion at the time of speech from speech signals with high accuracy and high reliability.

  In the present invention, based on the above consideration, the speech signal is decomposed into segments, and low frequency components of the autocorrelation function for the segments are generated, and from the low frequency components as analysis frames of several 10 ms to several hundreds ms, every analysis frame. Then, feature quantities related to emotions are extracted, and furthermore, emotions about speech signals are recognized from feature quantities in a plurality of analysis frames.

  In order to solve the above-mentioned problems, the present invention is an emotion recognition device for recognizing an emotion at the time of speech from an input speech signal, which comprises: segmentation means for decomposing the speech signal into segments; Voice strength measuring means for measuring the voice strength of the voice signal of the voice signal and outputting the voice strength signal for the segment, and calculating the autocorrelation function for each of the segments and outputting the autocorrelation signal for the segment Means, filter means for extracting pitch information from the autocorrelation signal and outputting a post-filtering autocorrelation signal for the segment, normalizing the magnitude of the post-filtering autocorrelation signal with the voice intensity signal for the segment Normalization means for outputting an autocorrelation signal after standardization, and several tens of ms to several hundreds ms including one or more segments Feature quantity extraction means for extracting feature quantities of the analysis frame from the normalized autocorrelation signal and the speech intensity signal using the period as an analysis frame, and sequentially outputting the correlation, based on the correlation between the feature quantity and emotion It is characterized in that it comprises pattern identification / emotion recognition means for recognizing an emotion at the time of speech of a voice signal and outputting an emotion label of the recognition result.

  The pattern identification and emotion recognition means outputs a pattern identification signal for outputting a correlation value between the feature amount and emotion as a pattern identification signal for an analysis frame, a pattern storage means for holding the pattern identification signal, and the pattern storage means The pattern recognition signal sequence can be read out from the above, and the emotion recognition means can recognize the emotion from the correlation between the feature amount for the analysis frame in the pattern recognition signal sequence and the emotion, and output the emotion label of the recognition result.

  Further, pattern identification and emotion recognition means, pattern identification and emotion recognition means, pattern identification means for outputting the feature amount as a pattern identification signal for each analysis frame, pattern storage means for holding the pattern identification signal Reading out a pattern identification signal sequence from the pattern storage means, correlating the occurrence frequency distribution of the pattern identification signal with the emotion in the pattern identification signal sequence, or the largest number of pattern identification signals or important pattern identification signals in the occurrence frequency distribution; It can be comprised by the emotion recognition means which recognizes the emotion at the time of the speech about the said speech signal from correlation with an emotion, and outputs the emotion label of the recognition result.

  Here, the voice intensity measurement means detects a silence period in which the voice signal is equal to or less than a predetermined threshold and outputs a silence detection signal, and the pattern identification / emotion recognition means outputs the silence detection signal. It is also preferable to recognize the emotion with and output an emotion label of the recognition result.

  It is also preferred that the filter means be a low pass filter with a cutoff frequency in the range of 500 Hz to 800 Hz.

  The method further comprises vowel detection means for detecting vowels for each analysis frame from the normalized autocorrelation signal, and the pattern identification and emotion recognition means is for emotion only for analysis frames for which vowels are detected by the vowel detection means. It is also preferable to execute the recognition process of

  Further, the feature quantity extraction means extracts feature quantities for analysis frames of a plurality of different time lengths, and the pattern identification / emotion recognition means is an emotion based on the correlation between all the obtained feature quantities and emotions. It is also preferable to execute the recognition process of and thereby output the emotion label of the recognition result.

  The image processing apparatus further comprises long-term feature quantity extraction means for extracting feature quantities of a plurality of analysis frames using the feature quantity extracted by the feature quantity extraction means as an input, and the pattern identification and emotion recognition means comprises the feature quantity extraction means It is also preferable to execute emotion recognition processing based on the correlation between all the feature quantities extracted by the feature quantity extraction means and the emotion, and thereby output an emotion label of the recognition result.

  The present invention can be realized not only as an emotion recognition apparatus but also as an emotion recognition method and a program for emotion recognition.

  According to the present invention, the speech signal is decomposed into segments, low-frequency components of the autocorrelation function for each segment are generated, feature quantities related to emotion are extracted from the low-frequency components, and several tens of ms from the feature quantities. Since the feature amount of an analysis frame of several hundred ms is extracted and emotions are recognized from the feature amounts of a plurality of analysis frames, the feature amount that correctly reflects the voice information can be extracted, and based on that Therefore, the emotion at the time of speech of the voice signal can be stably recognized with high accuracy and high reliability.

FIG. 1 is a block diagram generally showing an embodiment of an emotion recognition device of the present invention. It is a figure which shows an example of operation | movement in a segmentation part. It is a block diagram which shows an example of a structure of a voice intensity measurement part. It is a block diagram which shows an example of a structure of a feature-value extraction part. It is a figure which shows the waveform of the autocorrelation signal in the case of no band limitation. It is a figure which shows the waveform of the autocorrelation signal in the case with band limitation. It is a figure which shows an example in case the change of the peak value of a sine wave signal is observed in a short time. It is a figure which shows an example in case the change of the peak value of a sine wave signal is continuously observed over a long time. It is a figure which shows the image of the emotion recognition process in a pattern identification * emotion recognition part. FIG. 1 is a block diagram showing a first embodiment of an emotion recognition device of the present invention. It is a figure which shows one specific example of the pattern identification signal series which a pattern memory | storage part hold | maintains. It is a flowchart which shows an example of operation | movement of an emotion recognition part. It is a block diagram which shows 2nd Embodiment of the emotion recognition apparatus of this invention. It is a flowchart which shows the example of operation | movement of an emotion recognition part in the case of recognizing an emotion using the generation frequency distribution of the pattern identification signal in a pattern identification signal series. It is explanatory drawing in the case of recognizing an emotion, after dividing the pattern identification signal in a pattern identification signal series into a group. It is a figure which shows one specific example of the pattern identification signal series given to an emotion recognition part. It is a block diagram which shows 3rd Embodiment of the emotion recognition apparatus of this invention. It is a block diagram which shows an example of a structure of a vowel detection part. It is a figure which shows the block configuration in the case of using the statistical model for pattern identification in 3rd Embodiment. It is a figure which shows the block configuration in the case of using the statistical model for emotion recognition in 3rd Embodiment. It is a block diagram which shows 4th Embodiment of the emotion recognition apparatus of this invention. It is a figure which shows the block configuration of the feature extraction part in the case where extraction of a feature and recognition of emotion are performed in parallel by several analysis frame length in 4th Embodiment. It is a figure which shows the block configuration in the case of using the statistical model for pattern identification in 4th Embodiment. It is a figure which shows the block configuration in the case of using the statistical model for emotion recognition in 4th Embodiment. It is a block diagram which shows 5th Embodiment of the emotion recognition apparatus of this invention. It is a figure which shows the internal block structure of the long-term feature-value extraction part in the case of performing extraction of a feature-value, and recognition of emotion by several analysis frame length in 5th Embodiment. It is a figure which shows the block configuration in the case of using the statistical model for pattern identification in 5th Embodiment. It is a figure which shows the block configuration in the case of using the statistical model for emotion recognition in 5th Embodiment. It is a figure which shows an example of the statistical representative value for every phrase calculated | required from the time series data of fundamental frequency. It is a figure which shows an example of the statistical representative value for every phrase calculated | required from the time series data of audio | voice strength.

  Hereinafter, the present invention will be described with reference to the drawings.

  Although the case where the present invention is realized as an emotion recognition device will be described below, the present invention can also be realized as an emotion recognition method or a program for emotion recognition.

  FIG. 1 is a block diagram generally showing an embodiment of the emotion recognition apparatus of the present invention.

  The emotion recognition apparatus of the present embodiment includes a segmentation unit 10, an autocorrelation processing unit 20, a voice intensity measurement unit 30, a feature quantity extraction unit 40, and a pattern identification / emotion recognition unit 50. Each of these units may be configured by either hardware or software.

  The speech signal input to recognize the emotion at the time of speech is a digital signal sampled at an arbitrary rate, for example, 16 ksps and quantized at an arbitrary accuracy, for example, 16 bits.

  The segmentation unit 10 segments the input audio signal at predetermined time intervals on the time axis. In this segmentation, it is preferable to cut out the segments from the audio signal, for example, as shown in FIG. 2 while overlapping them in time.

  FIG. 2 shows that segmentation is performed by extracting a 10-ms speech signal in the central portion from a 30-ms speech signal that is overlapped by an anteroposterior in time by 10 ms. The segment length here is 10 ms. The segment length is preferably about 10 ms, but may be about several tens of ms. FIG. 2 also shows the relationship between the segments and the analysis frame. Although FIG. 2 is an example in which the analysis frame is composed of five segments, the analysis frame may include one or more segments. As will be described later, speech feature quantities are extracted in units of analysis frames.

Also, as described later, a plurality of analysis frames with different time lengths can be used simultaneously to enhance the detection accuracy of the audio feature quantity. That is, the accuracy of the voice feature may be enhanced by using the voice feature extracted using the short analysis frame and the voice feature extracted using the long analysis frame in combination.
The analysis frame will be described in detail later.

  As for segment length, when recognizing emotion from speech signal, it is preferable to use only vowels with clear spectral structure for analysis, and if consonant is mixed in analysis, recognition accuracy will deteriorate, so consonant mixing into analysis It is preferable to reduce the influence of the following: some consonants have a short duration, and a long segment length leads to an increase in the ratio of segments including consonants in the analysis frame, and thereby to the feature quantities in the analysis frame. It may be set in consideration of the possibility that the influence of consonants may increase, and the like.

  In the consonant section, the spectrum is whitened and there is no sine wave component. Therefore, including this in the analysis will cause a misjudgment, but adding vowel detection as described later makes consonant mixing into the analysis It is possible to reduce the probability of occurrence and to reduce erroneous determinations resulting from this. Although details will be described later, since the zero-cross number of the autocorrelation signal of the vowel relates to the pitch frequency, for example, when a zero-cross equivalent to a pitch frequency of 800 Hz or more is detected, mixing of consonants or noise is suspected. In such a case, it is preferable to stop the extraction of the feature amount.

  The autocorrelation processing unit 20 includes an autocorrelation calculation unit 21, a filter processing unit 22, and a normalization unit 23. The autocorrelation calculation unit 21 receives, for example, audio signals of five consecutive segments, calculates an autocorrelation function for each segment, and outputs the autocorrelation signal to the filter processing unit 22. The audio signals of five consecutive segments may be input to the autocorrelation calculation unit 21 in parallel.

  In the calculation of the autocorrelation function, the amount of calculation can be reduced by using a method of calculating a power spectrum and performing inverse Fourier transform on it. In the calculation of power spectrum, it is also known to perform time window processing beforehand, and according to this, it depends on the shape of the time window when calculating the power spectrum, but it reflects the frequency component of approximately 50 Hz or more. A correlation signal can be obtained.

  The filter processing unit 22 receives the autocorrelation signal output from the autocorrelation calculation unit 21, performs low-pass filter processing to generate an after-LPF autocorrelation signal, and outputs the autocorrelation signal to the normalization unit 23. Since the formant information included in the speech signal is highly related to the linguistic information, and the emotion is mainly transmitted by the pitch information, low-pass filtering is performed here to extract the pitch information. The cutoff frequency of the filter in the filtering unit may be, for example, 500 Hz to 800 Hz.

  The normalization unit 23 receives the post-LPF autocorrelation signal for the segment output from the filter processing unit 22 and the voice intensity signal for the segment output from the voice intensity measurement unit 30 as input, and the LPF post-correlation signal is voice intensity The signal is normalized to generate an after-normalization autocorrelation signal, which is output to the feature amount extraction unit 40. The post-normalization autocorrelation signal is a sine wave signal due to the band limitation in the filter processing unit 22.

  The voice strength measurement unit 30 obtains the voice signal strength for the segment and outputs it to the feature quantity extraction unit 40. Further, the voice intensity measurement unit 30 detects silence in the input voice signal for a predetermined period or more to generate a silence detection signal, and outputs the silence detection signal to the emotion recognition unit 53 of the pattern identification / emotion recognition unit 50. The silence detection signal is used by the emotion recognition unit 53 to determine the end of the speech.

  FIG. 3 is a block diagram showing an example of a configuration for generating an audio strength signal in the audio strength measurement unit 30. As shown in FIG.

  The voice strength measurement unit 30 includes a peak hold circuit 31 and a filter processing unit 32. The peak hold circuit 31 detects the envelope of the audio signal in the segment and outputs it as a peak hold signal. Low-pass filtering is applied to the peak hold signal, and a signal with reduced noise influence is output as an audio strength signal.

  The feature amount extraction unit 40 receives the normalized autocorrelation signal output from the normalization unit 23 and the voice intensity signal output from the voice intensity measurement unit 30 as input, and extracts feature amounts for the analysis frame. Changes in the feature value serving as an indicator of emotion in the autocorrelation signal after normalization may be observed in a short time of about several tens of ms or continuously observed over a period of about several hundred ms.

  Therefore, by setting the analysis frame to a time length in the range of several tens of ms to several hundreds of ms and extracting the feature amount with that time length, it is possible to appropriately extract the feature amount for recognizing the emotion of the audio signal. Since the autocorrelation signal and the voice strength signal are calculated in segment units, the analysis frame length may be an integral multiple of the segment length. As described above, the rate of change of the voice feature amount due to emotion is various, and in order to detect this accurately, it is preferable to use a plurality of analysis frames having different time lengths in combination.

  Although FIG. 1 is a block diagram of an example using a single analysis frame, a block configuration in the case of using a plurality of analysis frames will also be described later. In FIG. 1, the analysis frame length is applied to this block in the sense of clearly indicating the number of analysis frames to be used. Of course, an appropriate analysis frame length may be selected from among a plurality of analysis frame lengths and may be set in the feature amount extraction unit 40, or may be set variably. The same applies to the following embodiments.

  FIG. 4 is a block diagram showing an example of the configuration of the feature quantity extraction unit 40. As shown in FIG.

  The feature amount extraction unit 40 includes a peak detection unit 41, a logarithmic compression unit 42, a time lag analysis unit 43, a peak value analysis unit 44, and a voice strength analysis unit 45.

  The peak detection unit 41 receives the normalized autocorrelation signal output from the normalization unit 23 as input, detects a peak time lag and a peak value of the normalized autocorrelation signal, and outputs the peak time lag to the time lag analysis unit 43. , Peak value is output to the peak value analysis unit 44. The normalized autocorrelation signal is a sine wave signal due to the band limitation in the filter processing unit 22, and the peak detection unit 41 detects the peak time lag and peak value of the sine wave signal.

  FIG. 5 and FIG. 6 show the difference in the waveform of the autocorrelation signal depending on the presence or absence of band limitation.

  As shown in FIG. 5, the autocorrelation signal not subjected to the band limitation greatly changes every moment according to the content of the utterance. On the other hand, as shown in FIG. 6, the autocorrelation signal which has been band-limited by the filter processing unit 22 becomes a stable sine wave signal regardless of the content of the utterance. In addition, in speech including emotion, peak time lag of the sine wave signal and change in peak value are observed.

  In addition, since the autocorrelation signal is a sine wave signal, a plurality of peaks appear at substantially equal intervals. When the maximum value detection is performed on the autocorrelation signal, the peak position of the time lag having the maximum value fluctuates due to the influence of time variation of the voice strength, noise, and the like. In the following, it is assumed that the time lag and peak value of the first peak in the region where the time lag is positive (larger than 0) are the peak time lag and peak value, respectively.

  Also, these changes may be observed in a short time of about several tens of ms or may be observed continuously over a period of about several hundred ms. An example of the case where a change in peak value is observed in a short time of about several tens of ms is shown in FIG. 7 and the case of being continuously observed over a period of several hundred ms is shown in FIG. Although analysis frame 1 for a short period is effective in capturing a rapid change (rise) in a short period, it is necessary to combine it with analysis frame 2 for a long period in order to grasp the continuity of the trend. I understand.

  That is, as in analysis frame 1 in FIG. 8, when the time length of the analysis frame is small compared to the change rate of the feature, the entire image of the change in the feature due to emotion is within one analysis frame. It can not be caught in In this case, although it is not possible to clearly grasp the tendency of the feature amount to be increased if only the analysis frame 1 is used, it is possible to grasp the tendency to increase the feature amount by using the analysis frame 2 in combination.

  On the other hand, as in the analysis frame 2 in FIG. 7, when the time length of the analysis frame is large compared to the change rate of the feature, it is good when the feature is monotonically increasing or monotonically decreasing within the analysis frame. If the feature amount is convex or concave in the analysis frame, the gradient as the whole average can only be determined, and rapid changes can not be captured correctly. Even in such a case, rapid change can be captured by using the analysis frame 1 having a short time length in combination.

  The time lag analysis unit 43 receives the peak time lag output from the peak detection unit 41 as an input, takes a period of several tens ms to several hundreds ms, for example, a period of 50 ms as an analysis frame, and holds the peak time lag detected in the analysis frame, Furthermore, the average time lag, the maximum time lag, and the time lag change rate are output to the pattern identification unit 51 of the pattern identification / emotion recognition unit 50 as feature amounts for the analysis frame.

  Similarly, the peak value analysis unit 44 receives the peak value output from the peak detection unit 41, holds the peak value detected in the analysis frame, and further, average peak value, maximum peak value, peak value change rate The information is output to the pattern identification unit 51 of the pattern identification / emotion recognition unit 50 as the feature amount for the analysis frame.

  The logarithmic compression unit 42 receives the speech strength signal output from the speech strength measurement unit 30 as an input, and outputs the logarithmically compressed speech strength. The voice strength signal output from the voice strength measurement unit 30 usually has a dynamic range of 100 dB or more. Therefore, the speech strength signal is logarithmically compressed here so that speech signal strength in a wide range of input levels can be handled.

  The speech intensity analysis unit 45 receives the logarithmically compressed speech intensity output from the logarithmic compression unit 42, holds the logarithmically compressed speech intensity for the segment (10 ms) of the analysis frame (50 ms), and further The average value, maximum value, and change rate of the logarithmically compressed speech intensity for the segment are calculated, and the average speech intensity, the maximum speech intensity, and the speech intensity change rate are used as feature quantities for the analysis frame. To the pattern identification unit 51 of FIG.

  As described above, the feature extraction unit 40 extracts the feature from the normalized autocorrelation signal and the speech strength signal (in the example of FIG. 4, the average time lag, the maximum time lag, the time lag change rate, the average peak value, the maximum peak value, The peak value change rate, the average speech intensity, the maximum speech intensity, and nine elements of the speech intensity change rate) are extracted, and they are output to the pattern identification / emotion recognizer 50 as feature quantities for the analysis frame.

  The pattern identification / emotion recognition unit 50 includes a pattern identification unit 51, a pattern storage unit 52, and an emotion recognition unit 53.

  The pattern identification unit 51 receives as input the feature amount of the analysis frame output from the feature amount extraction unit 40, and uses it as a pattern identification signal or a feature amount (pattern of each element) according to the correlation between the feature amount and emotion. Output a pattern identification signal according to the request.

  The pattern storage unit 52 holds a pattern identification signal for the analysis frame output from the pattern identification unit 51. The pattern storage unit 52 sequentially stores pattern identification signals for subsequent analysis frames.

  The emotion recognition unit 53 sequentially reads out the pattern identification signals for the analysis frame from the pattern storage unit 52 as a pattern identification signal sequence, and recognizes the emotion at the time of the speech signal utterance based on the correlation between the pattern identification signal sequence and the emotion. And output the emotion label of the recognition result. For example, if the pattern identification signal corresponds to the correlation between the feature amount for the analysis frame and the emotion, the correlation can be used to recognize the emotion. Further, when the pattern identification signal corresponds to the feature amount (pattern of each element) output from the feature amount extraction unit 40, here, the correlation between the pattern identification signal and the emotion in the pattern identification signal sequence is obtained. The emotion can be recognized based on the correlation.

  As described above, the pattern identification / emotion recognition unit 50 recognizes the emotion at the time of the speech signal utterance based on the correlation between the feature amount of the analysis frame output from the feature amount extraction unit 40 and the emotion, and Output emotion label of recognition result.

  FIG. 9 is a view showing an image of an emotion recognition process in the pattern identification / emotion recognition unit 50. As shown in FIG.

  The feature quantity for the analysis frame is a feature quantity having nine elements as many as the extracted number (9 elements in the example of FIG. 4), and can be represented by a vector over a dimension of the number of elements. In FIG. 9, the feature quantity has two feature quantities (elements) 1 and 2, and the feature quantity for the analysis frame can be represented by a vector of a two-dimensional spatial position spanned by the feature quantities 1 and 2 . Here, in order to simplify the explanation, the number of feature quantities for the analysis frame is two, but as shown in the example of FIG. 4, if there are nine feature quantities for the analysis frame, the feature quantities for the analysis frame Is a nine-dimensional vector in space.

  Thus, when the pattern of the feature amount for the analysis frame is represented by a vector in the space in which the feature amount spans, those vectors can be grouped by emotion and the vector of the group can be associated with the emotion. This means that emotions can be recognized based on the feature amount of the analysis frame of the pattern identification signal sequence.

  From the above, the pattern identification and emotion recognition unit 50 recognizes the emotion at the time of the speech signal utterance based on the correlation between the feature amount of the analysis frame and the emotion. The specific method will be described later along with each embodiment.

  FIG. 10 is a block diagram showing a first embodiment of the emotion recognition device of the present invention. The same or equivalent parts as in FIG. 1 are denoted by the same reference numerals.

  In the first embodiment, a statistical model is used to obtain a correlation value between the feature amount and emotion for the analysis frame, and based on this correlation value, the emotion at the time of speech utterance is recognized, and the recognition result is Output emotion labels.

  First, creation of a statistical model used by the pattern recognition unit 51 will be described. A statistical model can be created in advance using training speech signals to create a statistical model.

  As shown in FIG. 10, a speech signal for learning in which emotions are known in advance is input to the segmentation unit 10, and a feature quantity of an analysis frame of the speech signal is extracted by the feature quantity extraction unit 40 to generate a statistical model. Give to part 60. At the same time, a pattern identification signal for the feature amount is given to the statistical model generation unit 60. The statistical model generation unit 60 generates a statistical model representing the correlation between the feature amount of the analysis frame and the pattern identification signal of the feature amount.

  Although FIG. 10 is a block configuration diagram of an example using a single analysis frame, a block configuration in the case of using a plurality of analysis frames later will also be described.

  The statistical model can be expressed, for example, by a matrix W. This matrix W can be used to generate a pattern identification signal corresponding to the feature amount of the analysis frame of the input audio signal. The matrix W can be obtained by the method of maximum likelihood method or SVM (Support Vector Machine).

  The following equation (1) is an example of a matrix W representing a statistical model. As shown in the following equation (2), by multiplying the matrix W by a column vector v consisting of nine feature quantities for the analysis frame, nine feature quantities and four emotions for the analysis frame are obtained. Vector r as a pattern identification signal representing the correlation of

  The pattern identification signal generated using the matrix W is a vector r such as {40, 8, 11, 27}, and each element of the vector is a feature for the analysis frame and each of four emotions, for example The correlation with "anger", "sadness", "boring" and "surprise" is shown. Here, assuming that the larger the number, the larger the correlation, in the above numerical example, the correlation between the feature amount and the emotion of "anger" is the largest, and the correlation with the emotion of "surprise" is also recognized to some extent, "sadness" And correlation with the emotion of "boring" represents low.

  Although the correlation between the feature amount and the emotion in the analysis frame is used as the pattern identification signal in the above, the emotion label of the emotion corresponding to the element that maximizes the correlation between the feature amount and the emotion may be used as the pattern identification signal. For example, when the vector representation is {40, 8, 11, 27}, the emotion label of "anger" can be used as the pattern identification signal. Thus, the pattern identification signal may be an emotion label representing an emotion at the time of speech.

  In addition, the emotion labels may be expressed as numbers such as "1", "2", "3", "4", for example, emotions such as "anger", "sadness", "boring", "surprise" etc. Well, it may be represented by a vector as {1, 0, 0, 0}, {0, 1, 0, 0}, {0, 0, 1, 0}, {0, 0, 0, 1} .

  The pattern storage unit 52 holds pattern identification signals for analysis frames sequentially output from the pattern identification unit 51.

  FIG. 11 is a diagram showing a specific example of a pattern identification signal sequence held by the pattern storage unit 52. As shown in FIG.

  Here, the numerical value in the first line indicates the correlation value between the feature value of each of the first analysis frame and each of the four emotions, with "anger", "sadness", "boring", "surprise" The correlation value is {20, 10, 15, 17}. The numbers on the second line indicate the correlation values between the feature values of the following analysis frame and each of the four emotions, and the correlation values with "anger", "sadness", "boring", "surprise" {80, 7, 25, 37}. The same applies to the following. In addition, although the correlation with the feature-value and four emotions about nine continuous analysis frames is shown here, a pattern memory | storage part is a pattern identification signal about the correlation about the analysis frame of the audio | voice signal input. Hold as.

  The emotion recognition unit 53 sequentially reads out the pattern identification signals for the analysis frame from the pattern storage unit 52 as a pattern identification signal sequence, and based on the correlation for the analysis frame in the pattern identification signal sequence, the emotion at the time of speech about the speech signal Identify

  As in the first embodiment, when the correlation between the feature amount and the emotion is obtained for each analysis frame using a statistical model having the feature amount as an input signal, and the correlation is stored in the pattern storage unit 52 as a pattern identification signal. Can add the correlation for the analysis frame in the pattern identification signal sequence for each emotion, and can recognize the emotion for which the sum is maximum as the emotion at the time of speech of the audio signal.

  For example, in the example shown in FIG. 11, since the sum of correlations with "anger" is maximized, the emotion at the time of speech about the speech signal can be recognized as "anger". In addition, it is possible to recognize an emotion in which the correlation shows the maximum or an emotion in which the number of analysis frames in which the correlation becomes the predetermined threshold is the maximum as the emotion at the time of speech signal without obtaining the total of the correlations. You may

  FIG. 12 is a flowchart showing an example of the operation of the emotion recognition unit 53.

  The emotion recognition unit 53 determines, based on the silence detection signal output from the voice strength measurement unit 30, whether the speech has ended. Here, if it is determined that the utterance has ended, the correlation value for the analysis frame in the pattern identification signal sequence is added for each emotion, and the emotion for which the total value is maximum is the emotion at the time of the utterance for the voice signal. Recognize and output an emotion label of the recognition result.

  As described above, in the first embodiment, the pattern identification unit 51 obtains the correlation value between the feature amount and emotion for the analysis frame, generates the pattern identification signal of the correlation value, and the emotion recognition unit 53 The emotion at the time of speech of the speech signal is recognized by adding the correlation value for the analysis frame in the pattern identification signal sequence for each emotion, but as described below, the feature value for the analysis frame is As a pattern identification signal, it is also possible to recognize an emotion at the time of speech about a voice signal based on the correlation between the occurrence frequency distribution (histogram) of the pattern identification signal in the pattern identification signal sequence and the emotion.

  FIG. 13 is a block diagram showing a second embodiment of the emotion recognition device of the present invention. The same or equivalent parts as in FIG. 1 are denoted by the same reference numerals.

  In the second embodiment, the feature quantity output from the feature quantity extraction unit 40 is used as a pattern identification signal, and using a statistical model, based on the correlation between the occurrence frequency distribution of pattern identification signals in the pattern identification signal sequence and emotion. It is made to recognize the emotion at the time of the utterance about a voice signal.

  Although FIG. 13 is a block configuration diagram of an example using a single analysis frame, a block configuration in the case of using a plurality of analysis frames will also be described later.

  First, creation of a statistical model used by the emotion recognition unit 53 will be described. A statistical model can be created in advance using training speech signals to create a statistical model.

  As shown in FIG. 13, a pattern identification signal and a pattern identification signal sequence are generated from a speech signal for learning in which emotions are known in advance, and the occurrence frequency distribution of the pattern identification signal in the pattern identification signal sequence is generated by a statistical model generator 70. Give to. At the same time, an emotion label corresponding to the occurrence frequency distribution of the pattern identification signal is given to the statistical model generation unit 70.

  The statistical model generation unit 70 generates a statistical model representing the correlation between the pattern identification signal sequence and the occurrence frequency distribution of the pattern identification signal and the emotion label. The statistical model generates, for example, a histogram of pattern identification signals of a pattern identification signal sequence generated from a speech signal for learning, and machine learning such as deep learning or SVM (Support Vector Machine) using each value of the histogram. Can be generated using

  At the time of recognition of the emotion of the voice signal, the emotion recognition unit 53 obtains the occurrence frequency distribution of the pattern identification signal in the pattern identification signal sequence generated from the input voice signal, and uses the statistical model to The highly correlated emotion is recognized as the emotion at the time of speech of the speech signal, and the emotion label of the recognition result is output.

  FIG. 14 is a flowchart showing an example of the operation of the emotion recognition unit 53 in the case of recognizing an emotion using the occurrence frequency distribution of pattern identification signals in a pattern identification signal sequence.

  The emotion recognition unit 53 determines, based on the silence detection signal output from the voice strength measurement unit 30, whether the speech has ended. Here, if it is determined that the utterance has ended, the occurrence frequency of each pattern identification signal in the pattern identification signal sequence is calculated, and the emotion having a high correlation with the occurrence frequency distribution of the pattern identification signal in the pattern identification signal sequence is voice signal It recognizes as an emotion at the time of speaking about, and outputs an emotion label of this recognition result.

  Further, the emotion recognition unit 53 can also recognize, for example, an emotion highly correlated with the pattern identification signal having the highest frequency of occurrence in the pattern identification signal sequence as the emotion at the time of speech. Furthermore, the pattern identification unit 51 divides the feature quantities of each analysis frame into groups and sets them as pattern identification signals for each group, and the emotion recognition unit 53 uses statistical models to select pattern identification signals in the pattern identification signal series. It is also possible to recognize an emotion having a high correlation with the occurrence frequency distribution and the pattern identification signal having the highest occurrence frequency as the emotion at the time of speech.

  FIG. 15 is an explanatory view of a case where the pattern identification signals in the pattern identification signal sequence are grouped and then the emotion is recognized. Here, in order to simplify the description, the number of feature quantities is two.

  In the example of FIG. 15, the pattern identification signals are grouped into 64 by quantizing the feature quantities 1 and 2 in eight stages. In addition, K-MEANS method etc. can be used for this grouping. Also in this case, it is possible to recognize the emotion at the time of the speech signal based on the occurrence frequency distribution for each group of pattern identification signals in the pattern identification signal sequence or the correlation between the pattern identification signal having the highest occurrence frequency and the emotion.

  FIG. 16 is a diagram showing a specific example of a pattern identification signal sequence given to the emotion recognition unit 53. As shown in FIG.

  Here, the pattern identification signal sequence of the pattern identification signal for ten consecutive analysis frames held by the pattern storage unit is shown, but the pattern storage unit 52 is a pattern for each analysis frame of the input audio signal. The pattern identification signal sequence of the identification signal is held. The pattern identification signals “pattern 1, pattern 2,..., Pattern 64” correspond to each of the 64 groups in FIG.

  Since the pattern identification signal having the highest frequency of appearance in the pattern identification signal sequence of FIG. 16 is pattern 5, it is possible to recognize an emotion having a large correlation with the pattern identification signal of pattern 5 as the emotion at the time of speech of the voice signal.

  When a pattern identification signal (important pattern) important for recognizing emotion is held in advance as a statistical model and the pattern identification signal sequence includes an important pattern, the emotion having a large correlation with the important pattern is a speech signal It can also be recognized as the emotion when Furthermore, when a plurality of important patterns are held in advance as a statistical model and the pattern identification signal sequence includes a plurality of important patterns, emotions having a high correlation with the higher priority pattern according to the predetermined priority are It can also be recognized as an emotion at the time of speech about a voice signal.

  FIG. 17 is a block diagram showing a third embodiment of the emotion recognition apparatus of the present invention, and the same or equivalent parts as in FIG. 1 are assigned the same reference numerals.

  As described above, when a consonant is mixed in the analysis frame, the accuracy of emotion recognition is degraded. However, this degradation is performed only when the vowel utterance is being extracted, or when the vowel utterance is in progress Can be prevented by performing pattern identification from.

  Therefore, in the third embodiment, a vowel detection unit 90 is provided which detects during vowel utterance based on the occurrence frequency of the zero cross of the normalized autocorrelation signal output from the normalization unit 23 of the autocorrelation processing unit 20, A vowel detection signal based on this is output to the pattern identification unit 51.

  The pattern identification unit 51 basically outputs a pattern identification signal for each analysis frame, but the vowel detection unit 90 does not output a vowel detection signal (a vowel detection signal indicates invalidity). Does not output

  Although FIG. 17 is a block configuration diagram of an example using a single analysis frame, a block configuration in the case of using a plurality of analysis frames later will also be described.

  FIG. 20 is a block diagram showing an example of the configuration of the vowel detection unit 90. As shown in FIG.

  The vowel detection unit 90 includes a zero crossing count unit 91 and a threshold determination unit 92.

  The zero cross frequency counting unit 91 receives the post-normalization autocorrelation signal output from the normalization unit 23 of the autocorrelation processing unit 20 as an input, and uses the post-normalization autocorrelation signal for one analysis frame (for example, five segments). A zero crossing is detected and an autocorrelation zero crossing number signal which is the number of the zero crossing is output.

  The threshold determination unit 92 receives the autocorrelation zero crossing number signal output from the zero crossing number counting unit 91, and outputs a vowel detection signal when the autocorrelation zero crossing number is less than or equal to a predetermined threshold value. The predetermined threshold value for the number of autocorrelation zero crossings is set to, for example, about 50 to 100 times when the analysis frame length is 50 ms.

  Although omitted in FIG. 17, the pattern identification unit 51 or the emotion recognition unit 53 uses a statistical model generated at the time of prior learning. FIG. 18 shows a block configuration in the case of using a statistical model for pattern identification, and FIG. 19 shows a block configuration in the case of using a statistical model for emotion recognition. The generation of a statistical model and the operation of emotion recognition using the statistical model are the same as in the first and second embodiments, and thus the description thereof is omitted.

  FIG. 21 is a block diagram showing a fourth embodiment of the emotion recognition device of the present invention, and the same reference numerals as in FIG. 1 denote the same parts in FIG.

  As described above, the normalized autocorrelation signal output from the normalization unit 23 of the autocorrelation processing unit 20 is subjected to the band limitation in the filter processing unit 22 and becomes a sine wave signal. The time lag of the sine wave peak corresponding to each emotion and the change of peak value are observed, and these changes may occur rapidly in a short time or may last for a period of several hundred ms.

  In the fourth embodiment, extraction of feature quantities for a plurality of analysis frame lengths is performed in parallel as different analysis frame lengths so as to correspond to those cases, and a pattern is generated using all of the feature quantities. The identification is made to recognize the emotion at the time of speech. The feature quantity extraction unit 40 extracts feature quantities for a plurality of analysis frame lengths in parallel as different analysis frame lengths. For this purpose, the feature amount extraction unit 40 is provided with a plurality of analysis frame lengths.

  In FIG. 21, analysis frame lengths 1 and 2 are applied to the feature amount extraction unit 40 in order to clearly indicate that feature amount extraction is performed using two analysis frames having different time lengths. Note that in order to perform feature amount extraction and emotion recognition in parallel with a plurality of analysis frame lengths, it is sufficient to use multiple systems for performing these processes. FIG. 22 shows a block configuration of the feature quantity extraction unit in this case.

  The pattern identification unit 51 generates a pattern identification signal using all feature amounts of different analysis frame lengths, and the pattern storage unit 52 holds the pattern identification signal thus obtained.

  The emotion recognition unit 53 reads the pattern identification signal sequence from the pattern storage unit 52, recognizes the emotion at the time of speech from the pattern identification signal sequence, and outputs an emotion label of the recognition result. Although omitted in FIG. 21, the pattern identification unit 51 or the emotion recognition unit 53 uses a statistical model generated at the time of prior learning. FIG. 23 shows a block configuration in the case of using a statistical model for pattern identification, and FIG. 24 shows a block configuration in the case of using a statistical model for emotion recognition. The generation of the statistical model and the operation of emotion recognition using the statistical model here are the same as in the first and second embodiments, and thus the description thereof is omitted. In this case, a statistical model is generated by learning the correlation between a plurality of feature quantities obtained using different analysis frame lengths in advance learning and a learning input signal (pattern identification signal or emotion label).

  FIG. 25 is a block diagram showing a fifth embodiment of the emotion recognition apparatus of the present invention, and the same reference numerals as in FIG. 1 denote the same parts in FIG.

  The fifth embodiment is also adapted to cope with the above case, and is provided with a long-term feature quantity extraction unit 80 which processes the feature quantities extracted in the analysis frame and extracts the feature quantities of a plurality of analysis frames. ing. Then, a pattern identification signal sequence is generated using all of the feature amounts for the feature amount of the analysis frame extracted by the feature amount extraction unit 40 and the plurality of analysis frames extracted by the long-term feature amount extraction unit 80, It is used to recognize the emotion at the time of speech.

  FIG. 26 shows an internal block configuration of the long-term feature quantity extraction unit. Here, each of five feature quantities in a predetermined number, for example, five analysis frames, is held, and the average value, maximum value, and change rate are respectively determined to obtain long-term average time lag, long-term maximum time lag, long-term maximum It outputs the time lag change rate, long-term average peak value, long-term maximum peak value, long-term peak value change rate, long-term average voice strength, long-term maximum voice strength and voice strength change rate.

  The pattern recognition unit 51 generates a pattern identification signal using all of the feature amounts for the feature amount of the analysis frame extracted by the feature amount extraction unit 40 and the plural analysis frames extracted by the long-term feature amount extraction unit 80. The pattern storage unit 52 holds the pattern identification signal thus obtained. The emotion recognition unit 53 reads out a pattern identification signal sequence from the pattern storage unit 52, recognizes an emotion from the pattern identification signal sequence, and outputs an emotion label of the recognition result.

  Although omitted in FIG. 25, the pattern identification unit 51 or the emotion recognition unit 53 uses a statistical model generated at the time of prior learning. FIG. 27 shows a block configuration in the case of using a statistical model for pattern identification, and FIG. 28 shows a block configuration in the case of using a statistical model for emotion recognition. The generation of the statistical model and the operation of emotion recognition using the statistical model here are the same as in the first and second embodiments, and thus the description thereof is omitted. In this case, a statistical model is generated by learning the correlation between a plurality of feature quantities obtained for one analysis frame and a plurality of analysis frames in learning in advance and a learning input signal (pattern identification signal or emotion label).

  As mentioned above, although embodiment of this invention was described, this invention is not limited to the said embodiment.

  For example, although the pattern identification and emotion recognition unit in the above embodiment is configured to recognize emotion at the timing of the end of speech, a voice signal of a predetermined period is designated by user operation or the like, and a voice signal in the predetermined period is selected. You may recognize the emotions of In this case, it is preferable to display an operation during a predetermined period. Also, by recognizing the emotion for the audio signal at predetermined intervals, it is possible to recognize the change in emotion throughout the speech.

  The present invention can be realized not only as an emotion recognition device but also as an emotion recognition method and a program for emotion recognition.

  10: Segmentation unit 20: Autocorrelation processing unit 21: Autocorrelation calculation unit 22: Filter processing unit 23: Normalization unit 30: Voice strength measurement unit 31: peak hold circuit 32: filter processing unit 40: feature amount extraction unit 41: peak detection unit 42: logarithmic compression unit 43: time lag analysis unit 44: peak value analysis unit 45: speech intensity analysis unit 50: pattern identification / emotion recognition unit 51: pattern recognition unit 52: pattern storage unit 53: emotion Recognition unit, 60, 70 ... statistical model generation unit, 80 ... long-term feature value extraction unit, 90 ... vowel detection unit, 91 ... zero cross count counting unit, 92 ... threshold determination unit

Claims (14)

An emotion recognition device for recognizing an emotion at the time of speech from an input voice signal, comprising:
Segmentation means for decomposing the audio signal into segments;
Audio intensity measuring means for measuring the audio intensity of the audio signal for each of the segments and outputting an audio intensity signal for the segments;
Autocorrelation calculation means for calculating an autocorrelation function for each of the segments and outputting an autocorrelation signal for the segment;
Filter means for extracting pitch information from the autocorrelation signal and outputting a filtered autocorrelation signal for the segment;
Normalization means for normalizing the magnitude of the post-filtering autocorrelation signal with the voice intensity signal and outputting a post-normalization autocorrelation signal for the segment;
Feature quantity extraction means for extracting feature quantities of an analysis frame from the normalized autocorrelation signal and the voice intensity signal and sequentially outputting the analysis frame as a period of several tens of ms to several hundreds of ms including one or more segments;
An emotion recognition apparatus comprising: pattern identification / emotion recognition means for recognizing an emotion at the time of an utterance of the voice signal based on the correlation between the feature amount and the emotion, and outputting an emotion label of the recognition result. The pattern identification and emotion recognition means
Pattern identification means for outputting the correlation value between the feature amount and emotion as a pattern identification signal for an analysis frame;
Pattern storage means for holding the pattern identification signal;
Emotion recognition means for reading out a pattern identification signal sequence from the pattern storage means, recognizing an emotion from the correlation between a feature amount and an emotion in an analysis frame in the pattern identification signal sequence, and outputting an emotion label of the recognition result The emotion recognition device according to claim 1, characterized in that The pattern identification and emotion recognition means
Pattern identification means for outputting the feature amount as a pattern identification signal for each analysis frame;
Pattern storage means for holding the pattern identification signal;
The pattern identification signal sequence is read out from the pattern storage means, and the correlation between the occurrence frequency distribution and the emotion of the pattern identification signal in the pattern identification signal sequence, or the largest number of pattern identification signals in the occurrence frequency distribution or the important pattern identification signals and the emotion 2. The emotion recognition apparatus according to claim 1, further comprising emotion recognition means for recognizing an emotion at the time of utterance of the voice signal from the correlation with the speech signal, and outputting an emotion label of the recognition result. The voice intensity measurement means detects a silence period in which the voice signal is equal to or less than a predetermined threshold value, and outputs a silence detection signal.
The pattern recognition / emotion recognition means recognizes an emotion at the timing when the silence detection signal is output, and outputs an emotion label as a recognition result. Emotion recognition device.   The emotion recognition apparatus according to any one of claims 1 to 4, wherein the filter means is a low pass filter having a cutoff frequency in the range of 500 Hz to 800 Hz. And vowel detection means for detecting vowels for each analysis frame from the normalized autocorrelation signal.
6. The pattern recognition / emotion recognition means executes an emotion recognition process only on an analysis frame in which a vowel is detected by the vowel detection means. Emotion recognition device. The feature quantity extraction unit respectively extracts feature quantities for analysis frames of a plurality of different time lengths;
The pattern identification and emotion recognition means executes an emotion recognition process based on the correlation between all the obtained feature quantities and emotions, thereby outputting an emotion label of a recognition result. The emotion recognition device according to any one of 6. The image processing apparatus further comprises long-term feature quantity extraction means for extracting feature quantities of a plurality of analysis frames using the feature quantity extracted by the feature quantity extraction means as an input.
The pattern identification / emotion recognition means executes an emotion recognition process based on the correlation between all the feature amounts extracted by the feature amount extraction means and the long-term feature amount extraction means, and thereby the emotion label of the recognition result The emotion recognition apparatus according to any one of claims 1 to 6, characterized in that: A method for recognizing an emotion at the time of speech from an input speech signal, comprising:
A segmentation step of decomposing the speech signal into segments;
Measuring the speech strength of the speech signal for each of the segments and outputting a speech strength signal for the segments;
Calculating an autocorrelation function for each of the segments and outputting an autocorrelation signal for the segment;
A filtering step of extracting pitch information from the autocorrelation signal and outputting a filtered autocorrelation signal for the segment;
Normalizing the magnitude of the post-filtering autocorrelation signal with the speech intensity signal and outputting a post-normalization autocorrelation signal for the segment;
And a step of feature quantity extraction for extracting and sequentially outputting feature quantities of the analysis frame from the normalized autocorrelation signal and the speech intensity signal, with a period of several tens of ms to several hundreds of ms including one or more segments as an analysis frame ,
A method characterized by comprising a step of pattern identification and emotion recognition which recognizes an emotion at the time of speech about the voice signal based on the correlation between the feature amount and an emotion, and outputs an emotion label of the recognition result. The steps of pattern identification and emotion recognition are
Outputting a correlation value between the feature amount and emotion as a pattern identification signal for an analysis frame;
The step of storing the pattern identification signal in the pattern storage means;
It has a step of emotion recognition for reading out a pattern identification signal sequence from the pattern storage means, recognizing an emotion from a correlation between a feature amount and an emotion in an analysis frame in the pattern identification signal sequence, and outputting an emotion label of the recognition result. 10. A method according to item 9. The steps of pattern identification and emotion recognition are
A pattern identification step of outputting the feature amount as a pattern identification signal for each analysis frame;
The step of storing the pattern identification signal in the pattern storage means;
The pattern identification signal sequence is read out from the pattern storage means, and the correlation between the occurrence frequency distribution and the emotion of the pattern identification signal in the pattern identification signal sequence, or the largest number of pattern identification signals in the occurrence frequency distribution or the important pattern identification signals and the emotion 10. The method according to claim 9, further comprising the step of: recognizing emotions at the time of speech about the speech signal from the correlation with the step c) and outputting an emotion label as a recognition result. A program for recognizing an emotion at the time of speech from an input audio signal, comprising:
Segmentation means for decomposing the audio signal into segments,
Audio intensity measuring means for measuring the audio intensity of the audio signal for each of the segments and outputting an audio intensity signal for the segments;
Autocorrelation calculation means for calculating an autocorrelation function for each of the segments and outputting an autocorrelation signal for the segment;
Filter means for extracting pitch information from the autocorrelation signal and outputting a filtered autocorrelation signal for the segment;
Normalization means for normalizing the magnitude of the post-filtering autocorrelation signal with the voice strength signal and outputting a post-normalization autocorrelation signal for the segment;
Feature quantity extraction means for extracting feature quantities of an analysis frame from the normalized autocorrelation signal and the speech intensity signal and sequentially outputting the analysis frame as a period of several tens of ms to several hundreds of ms including one or more segments;
A program that functions as pattern identification / emotion recognition means for recognizing an emotion at the time of an utterance of the voice signal based on the correlation between the feature amount and an emotion, and outputting an emotion label of the recognition result. The pattern identification and emotion recognition means
Pattern identification means for outputting the correlation value between the feature amount and emotion as a pattern identification signal for an analysis frame;
The pattern identification signal sequence is read out from the pattern storage means holding the pattern identification signal, the emotion is recognized from the correlation between the feature amount and the emotion for the analysis frame in the pattern identification signal sequence, and the emotion label of the recognition result is output. The program according to claim 12, comprising emotion recognition means. The pattern identification and emotion recognition means
Pattern identification means for outputting the feature amount as a pattern identification signal for each analysis frame;
The pattern identification signal sequence is read out from the pattern storage means holding the pattern identification signal, and the correlation between the occurrence frequency distribution of the pattern identification signal and the emotion in the pattern identification signal sequence or the largest number of pattern identification signals in the occurrence frequency distribution or important 13. The program according to claim 12, further comprising emotion recognition means for recognizing an emotion at the time of utterance of the voice signal from correlation of the pattern identification signal and the emotion, and outputting an emotion label of the recognition result.

Images