KR20100094182A - Apparatus and method for recognizing emotion using a voice signal - Google Patents

Abstract

PURPOSE: A device and a method for recognizing feelings using voice signal are provided to extract non-linguistic parameter reflecting a emotional state from speed sound, thereby more accurately determining the feeling state of a speaker. CONSTITUTION: A non-linguistic information detector(210) detects one between pitch track and energy track from inputted speed sound as non-linguistic information. A non-linguistic parameter extractor(220) detects a specific interval from at least one between the pitch orbit and energy orbit. The non-linguistic parameter extractor extracts non-linguistic parameter from the detected specific section. A determination unit(230) determines the feeling state recognized about the speed sound through the extracted non-linguistic parameter.

Description

Apparatus and method for emotion recognition using speech signal {APPARATUS AND METHOD FOR RECOGNIZING EMOTION USING A VOICE SIGNAL}

The present invention relates to an apparatus and method for recognizing emotions, and more particularly, to an apparatus and method for recognizing emotions using voice signals.

The fields that are currently attracting much attention and are attracting attention as future technologies are the fields of emotion recognition and emotion understanding. This is also related to the overall flow of IT research moving from PC-centric to network-centric.

As the interest in the interface technology between the user and the machine is increased, technologies for recognizing human emotion from biometric data including voice and facial expressions are being actively researched. Currently, two methods of interface technology between human and machine are generally used.

The first method is a mechanical signal input method such as a keyboard and a mouse, and most of them include an interface method between humans and machines. This has the advantage that the user's intention is accurately and reliably delivered when the signal is input, but it requires a considerable learning to operate the machine effectively and the user's convenience is relatively inferior. In addition, there is a problem that a large portion of the communication of the user is lost.

The second method is a natural signal input method such as voice and video, which is gradually applied to products due to the development and demand of technology, but the frequency of use is still very low compared to the mechanical signal input method described above. This has the disadvantage that the communication is unstable compared to the mechanical signal input method and mechanically incurs additional costs, but it is more convenient for the user and has the advantage of being able to convey various needs.

As a part of the interface technology between humans and machines, the importance of emotion recognition using voice, which is the most basic means of communication and information transmission, has been highlighted.

Emotion recognition technology using voice signals can be used in various fields. For example, if it is determined that the speaker is in an emotional state that is expected to have a rest, an intelligent interface technology that uses a voice signal connected to the service by suggesting a quiet music or a service to relax may be possible. It can also be used to improve the performance of other interface technologies.

Recognition of emotions using these voice signals is generally achieved through verbal or non-verbal information that can be obtained from the speaker's speech.

In this case, the non-verbal information may be obtained by analyzing pitch and energy of the spoken sound. Examples of such research (Kwak Hyun-suk, Kim Soo-hyun, Kwak Yun-geun, "Emotion Recognition Using Rhythm and Characteristic Vector and Gaussian Mixture Model", Korean Society for Noise and Vibration Engineering Society of Autumn, 2002, Han Ho, p. 375, 2002) Emotional recognition was performed by constructing two types of feature vectors by using their first, second, and variation and normalization using pitch, energy, and tempo, which are the rhythm features of. Schuller (Schuller, B. Rigoll, G. and Lang, M., Hidden Markov Model-Based Speech Emotion Recognition, Proc. ICASSP 2003, IEEE, Hong Kong, China, vol. II, 1-4, 2003) In addition to the dynamic characteristics of chords, we propose a method to perform emotion recognition using static statistics using pitch and energy of the chords.

However, the nonverbal information of speech is greatly influenced by environmental and speaker-specific factors. The pitch can vary depending on the vocal cord structure of each speaker, including intonation and accent depending on the language. Energy can also vary according to phoneme composition, location of speech, individual speaker variations, microphone volume, or surroundings. In addition, the pitch and energy of the spoken sound may be affected in the voice signal input process. In other words, environmental factors and speaker-specific factors contribute to the performance degradation of emotion recognition.

As described above, when extracting nonverbal information from a spoken sound to perform emotion recognition, the emotional state of the speaker can be more objectively determined by using a parameter extraction method that can accurately reflect the emotional state of the speaker and the extracted parameters. Emotional awareness is required.

Accordingly, the present invention provides an apparatus and method for recognizing emotions by extracting a parameter for non-verbal information indicating an emotional state from a voice signal.

The present invention also provides an apparatus and method for recognizing emotions by extracting general parameters reflecting emotional states and additional parameters reflecting emotional states from voice signals.

The present invention also provides an apparatus and method for recognizing emotions by extracting additional parameters reflecting an emotional state using voiced sounds and energy of a voice signal.

An emotion recognition apparatus using a speech signal, the apparatus comprising: a non-verbal information detector for detecting at least one of a pitch track and an energy track from non-verbal information; A non-verbal parameter extraction unit detecting a specific section from at least one of the detected pitch trajectory and the energy trajectory, and extracting a nonverbal parameter from the detected specific section; And a determination unit to determine an emotional state recognized for the speech using the extracted nonverbal parameters.

In addition, the emotion recognition method using the voice signal according to the present invention, the step of detecting at least one of the pitch trajectory and the energy trajectory from the input speech sound as non-verbal information; Detecting a specific section from at least one of the detected pitch trajectory and the energy trajectory, and extracting a non-verbal parameter from the detected specific section; And determining an emotional state recognized for the spoken sound by using the extracted non-verbal parameters.

Therefore, according to the present invention, it is possible to determine the emotional state of the speaker more accurately by extracting a nonverbal parameter reflecting the emotional state from the spoken sound.

In addition, according to the present invention, the pitch track and the energy track of the spoken sound are detected as non-verbal parameters indicating the emotional state of the spoken sound, and various statistical parameters extracted from the pitch track and the energy track are combined to obtain the emotional state of the talker. You can judge accurately.

Hereinafter, exemplary embodiments of the present invention will be described in detail with reference to the accompanying drawings. In addition, specific details appear in the following description, which is provided to help a more general understanding of the present invention, and it is obvious to those skilled in the art that the present invention may be practiced without these specific details. Will do.

First, the present invention is proposed to minimize the influence of external factors such as environmental factors, speaker-specific factors, etc. in a method of extracting non-verbal information from speech sounds to recognize emotions. To this end, in the present invention, when extracting non-verbal information, it is simply to measure the absolute value and the amount of change of pitch and energy of speech, and to remove emotion from speech pitch and energy trajectory. By extracting the parameters and comparing the extracted parameters with modeling values for each emotional state, we present a method of objectively determining the emotional state of the speaker.

1 is a flowchart illustrating a process of recognizing emotions by extracting non-verbal parameters according to the present invention.

When the talker's speech is input through the microphone in step 110, various non-verbal parameters for emotion recognition are extracted from the pitch trajectory and the energy trajectory of the speech in accordance with the present invention in step 120. Here, the pitch trajectory of the spoken sound is extracted from the voiced sound, which is a continuous voice accompanied by tremor of the vocal cords. And detecting the rising section, the falling section, the up-convex section, and the up-convex section of each track from the extracted pitch trajectory and the energy trajectory, and calculate statistical parameters from each detected section to extract the non-verbal parameters. do.

Then, in step 130, the emotion is recognized by comparing the extracted non-verbal parameters with predetermined modeling values for each emotion state to determine the emotion state of the speaker.

As described above, the present invention extracts the pitch track and the energy track using the voiced sound interval and the energy measured from the spoken sound, and calculates statistical parameters from the specific section of the pitch track and the energy track to calculate the non-verbal parameters for emotion recognition. Extract it with

Hereinafter, a method of extracting the non-verbal parameters and determining the emotional state using the extracted non-verbal parameters will be described in detail with reference to FIGS. 2 to 4.

2 is a block diagram of an emotion recognition apparatus according to an embodiment of the present invention.

The emotion recognition apparatus of FIG. 2 includes a non-verbal information detector 210, a non-verbal parameter extractor 220, a determiner 230, and a modeler 240.

Referring to FIG. 2, the non-verbal information detector 210 includes a voiced sound detector 211 and an energy detector 213 for detecting voiced sound and energy to detect non-verbal information from input speech. Here, the voiced sound and energy may be detected in units of intervals, and may further include means for detecting other non-verbal information reflecting the speaker's emotional state in addition to the voiced sound and energy as non-verbal information. In addition, the detection units 211 and 213 may be installed in parallel as shown in FIG. 2 or sequentially performed regardless of the order. For example, it may be possible to configure the apparatus to detect voiced sound after detecting voiced sound from the spoken sound, or after detecting energy. In addition, although the apparatus is configured to use both the voiced sound detector 211 and the energy detector 213 in the embodiment of FIG. 2, the apparatus may be configured to detect only the voiced sound or energy. In the following embodiments, it is assumed that both voiced sound and energy are detected to help understand the invention.

First, an embodiment of the present invention detects a pitch trajectory over time in voiced sound except voiced sound using auto-correlation to detect voiced sound. In addition, the energy trajectory over time is detected using a root mean square to detect energy. The method for detecting the pitch trajectory and the method for detecting the energy trajectory may use various methods currently proposed and used in addition to the method using the autocorrelation function and the root mean square.

The non-verbal parameter extractor 220 calculates various statistical parameters from specific intervals of the pitch trajectory and the energy trajectory respectively detected by the voiced sound detector 211 and the energy detector 213, and calculates various statistical parameters from the calculated statistical parameters. Extract a number of nonverbal parameters. 2 may be modified to include the non-verbal parameter extraction unit 220 for the voiced sound detector 211 and the energy detector 213, respectively.

The non-verbal parameter extraction unit 220 compares the slope with respect to the pitch track detected by the voiced sound detector 211 and the rate of change of the slope with a predetermined threshold, and the pitch track is convex downward, convex upward and upward. After acquiring at least one of a phosphorus interval and a descending interval, statistical parameters are calculated for the interval to extract non-verbal parameters that reflect the emotional state.

Specifically, the non-verbal parameters extracted using the pitch trajectory may include at least one of an average, a variance, a maximum value, a median value, and an interquatile range (IQR) for each of the time intervals in which the pitch trajectory is convex downward and convex upward. Contains one. In this case, the IQR refers to, for example, a section of the upper 75% in the distribution of the pitch trajectories.

In addition, the non-verbal parameters extracted using the pitch trajectory are the ratio of the total time of the voiced sound and the time for the convex downward convex section and the convex upper convex section, the convex down section and the convex upper section, respectively. At least one of the mean, median, IQR of the pitch of.

The non-verbal parameters extracted using the pitch trajectory may include at least one of an average, a variance, a maximum value, a median value, and an IQR of each time for the interval in which the pitch trajectory is rising and / or the descending interval. And / or calculate statistical parameters in the descending interval to include at least one of the mean, median, and IQR of each pitch.

On the other hand, after obtaining a rising section and / or a falling section by comparing the slope of the energy trajectory detected by the energy detector 213 with a predetermined threshold, non-verbal that reflects the emotional state by calculating statistical parameters for the section. Extract the parameters.

In this case, the non-verbal parameters extracted using the energy trajectory are the energy trajectories for each frequency band corresponding to 0 to 250 Hz, 0 to 500 Hz, 500 to 1000 Hz, and 2500 to 3500 Hz, and the slope of the slope and the change rate of the slope. And at least one of mean, median, variance, highest, lowest, and IQR in.

In addition, the non-verbal parameters extracted using the energy trajectory are orbits, orbits, in which the energy of each frequency band corresponding to 0 to 250 kHz, 0 to 500 kHz, 500 to 1000 kHz, and 2500 to 3500 kHz is normalized to the total energy of each frame. And at least one of an average, median, variance, maximum, minimum, and IQR at the slope for and the rate of change of the slope.

The frequency bands of 0 to 250 kHz, 0 to 500 kHz, 500 to 1000 kHz and 2500 to 3500 kHz are exemplified and are not necessarily limited to the frequency band.

In addition, the non-verbal parameters extracted using the energy trajectory include at least one of an average, a maximum value, a median value, and an IQR of each time for the section in which the energy trajectory is rising and / or falling, and the rising and / or falling And at least one of the mean, median, and IQR of each energy in the interval.

When a spoken sound is input as described above, various non-verbal parameters are extracted from the pitch trajectory and the energy trajectory. The various non-verbal parameters may be used in combination to improve the performance of emotion recognition. Here, the extracted non-verbal parameters are determined as the emotional state of the speaker as compared to the modeling values for each emotional state that have been pre-modeled (or learned). Here, the emotional state modeling values are learned from, for example, a training voice signal and extracted into an emotional model.

In FIG. 2, the modeling unit 240 extracts the emotion model from the input training voice signal.

Referring to FIG. 3, the process of extracting the emotion model is described. When a training voice signal is input to the modeling unit 240 in step 310 of FIG. 3, the modeling unit 240 performs the input training in step 320. Extract non-verbal parameters from speech signal. In operation 330, the modeling unit 240 extracts an emotion model as follows using the extracted non-verbal parameters.

Table 1 below illustrates average and / or variance values of respective feature vectors of the training voice signal for each emotional state in a database used for training the voice signal. The emotion state modeling values, that is, the emotion model may use average and / or variance values of the feature vectors. The emotion model of Table 1 below illustrates 10 selected parameters among the parameters of each feature vector, and the parameters are the intervals in which the pitch trajectory rises sequentially from the vertical column of Table 1 below. IQR (P1, P2) of the respective pitch values in the descending section, IQR (P3) of the pitch values in the convex down section, the variance (P4) in the slope of the 500-1000 Hz frequency band energy trajectory and its The frequency band energy corresponding to the variance (P5) at the rate of change of the slope, the maximum value P6 and the average (P7) and variance (P8) at the frequency band energy range of 500 to 1000 Hz, and 0 to 250 Hz, The median value (P9) of the orbit normalized by total energy, and the average (P10) of the composition where the frequency band energy corresponding to 0 to 500 Hz is normalized to the total energy of each frame.

When each feature vector is applied to a Gaussian Mixture Model (GMM), a GMM model is generated as the emotion model through a normalization process. The values in Table 1 are examples using a Korean voice database, and may be stored as different values in other languages such as English. However, even if the language is different, the operation of the present invention detects the pitch trajectory and the energy trajectory from the spoken sound (or training voice signal), and extracts the statistical parameters calculated in a specific section of the detected pitch trajectory and the energy trajectory as non-verbal parameters. Applies in common.

The determination unit 230 vectorizes the non-verbal parameters extracted from the non-verbal parameter extraction unit 220, and then uses the vectorized non-verbal parameters by using an emotion model for determining an emotional state pre-configured in the modeling unit 240. The emotional state of the speaker is determined by comparing the likelihood for each emotional state in the emotional model. That is, the determination unit 230 determines (recognizes) the emotional state corresponding to the modeling value having the closest value by comparing the vectorized nonverbal parameters with modeling values for each emotional state as the speaker's emotional state.

In this case, GMM or HMM (Hidden Markov Model) may be used as an algorithm for determining the emotional state. In addition, any pattern classification algorithm that can be used in emotion recognition can be used.

4 is a flowchart illustrating a method of recognizing an emotional state from an input utterance according to an embodiment of the present invention.

First, when a speech sound is input, in step 410, non-verbal information on the input speech sound is detected. Specifically, the non-verbal information includes at least one of voiced sound and energy extracted from the spoken sound. In step 411, the pitch track is detected from the voiced sound, and in step 413, the energy track is detected from the energy of the spoken sound. In addition, it may be possible to detect the energy of the spoken sound in the period of the voiced sound.

Steps 411 and 413 may be performed in parallel, and may be performed sequentially regardless of the order of progress. It will also be possible to selectively perform at least one of each step.

In operation 420, statistical parameters are calculated from specific sections of the pitch track and the energy track, which are non-verbal information detected in steps 411 and 413, respectively, and extracted as non-verbal parameters for emotion recognition. A specific method of calculating the non-verbal parameters may include obtaining at least one of a section in which the pitch track and the energy track are convex downward, convex upward, rising section and falling section as described in the description of FIG. 2. In addition, statistical parameters are calculated for the interval to extract non-verbal parameters that reflect the emotional state.

In operation 430, the extracted non-verbal parameters are calculated as vectorized values, and in step 440, the vectorized non-verbal parameters are compared with modeling values for each emotional state, and in operation 450, the closest to the vectorized non-verbal parameters are obtained. The modeling value having the value is checked, and in step 460, the emotional state corresponding to the identified modeling value is determined (recognized) as the speaker's emotional state.

5A to 5C show simulation results of applying an emotion recognition method according to an embodiment of the present invention, respectively.

A graph showing the pdf distribution (x-axis normalized) using two Gaussians for each emotional state, with each emotional state being normal (510), joy (520), sadness (530), and angry (540). It is clearly shown. First, Figure 5a is a graph showing the pdf of the IQR in the interval of the pitch trajectory in accordance with an embodiment of the present invention, Figure 5b is a median of energy corresponding to 0 ~ 250 kHz standardized according to an embodiment of the present invention This is a graph showing the pdf for. In addition, Figure 5c is a graph showing a pdf of the average value of the energy corresponding to the standardized 0 ~ 500㎐. 5A to 5C, it can be seen that the joy 520 and the anger 540 generally representing the excited state, and the normal 510 and the sadness 530 representing the calm state have similar distributions. In the simulation, a combination of a plurality of nonverbal parameters extracted according to the present invention was used after vectorization in order to improve emotion recognition performance.

1 is a flowchart illustrating a process of recognizing emotions by extracting non-verbal parameters according to the present invention;

2 is a block diagram of an emotion recognition apparatus according to an embodiment of the present invention;

3 is a flowchart illustrating a process of extracting an emotion model according to an embodiment of the present invention;

4 is a flowchart illustrating a method of recognizing an emotional state from an input utterance according to an embodiment of the present invention;

5a is a graph showing a pdf for an IQR in a section in which a pitch trajectory falls according to an embodiment of the present invention;

Figure 5b is a graph showing a pdf of the median of energy corresponding to 0 ~ 250 kHz standardized according to an embodiment of the present invention,

Figure 5c is a graph showing the pdf of the average value of the energy corresponding to 0 ~ 500 Hz standardized.

Claims (21)

A non-verbal information detection unit for detecting at least one of a pitch track and an energy track from the input speech sound as non-verbal information; A non-verbal parameter extraction unit detecting a specific section from at least one of the detected pitch trajectory and the energy trajectory, and extracting a nonverbal parameter from the detected specific section; And And a determination unit that determines an emotional state recognized for the speech using the extracted non-verbal parameters. The method of claim 1, wherein the detected specific section is, At least one of a downward convex section, an upward convex section, a rising section, and a falling section, which are detected by comparing a slope and a rate of change of the slope with respect to at least one of the detected pitch trajectory and the energy trajectory with a predetermined threshold value. Emotion recognition apparatus using a voice signal, characterized in that the interval. The method of claim 1, wherein the emotion recognition device, And a modeling unit which generates a modeling value for each emotional state as a database using a plurality of non-verbal parameters extracted from the input training signal. The method of claim 1, The non-verbal information detection unit, A voiced sound detector for detecting the pitch trajectory according to time in voiced sound by detecting a voiced sound section from the spoken sound; And And an energy detector for detecting the energy trajectory over time by detecting energy from the spoken sound. The method of claim 3, wherein the determination unit, The emotional state corresponding to the modeling value having the closest value is compared to the emotional state recognized by the speech sound by comparing the extracted non-verbal parameter from the input speech sound with the modeling values for each emotional state provided through the modeling unit. Emotion recognition apparatus using a voice signal, characterized in that for judging. The method of claim 1, wherein the determination unit, The non-verbal parameter extracted from the input speech sound is compared with predetermined modeling values for each emotional state, and the voice signal is determined as an emotional state recognized for the speech sound corresponding to the modeling value having the closest value. Emotion Recognition Device. The method of claim 1, wherein the determination unit, Emotion recognition apparatus using a speech signal, characterized in that using one of the Gaussian Mixture Model (GMM) and Hidden Markov Model (HMM) as an algorithm for determining the emotional state. The method of claim 1, wherein the non-verbal parameters are: IQR (Interquatile range) of each pitch value in the section where the pitch track is rising and falling, IQR of the pitch value in the section where the pitch track is convex downward, The normalization of the variance in the slope and the variance in the rate of change of the slope, the maximum and mean and variance in the 500–1000 Hz frequency band energy trajectory, and the frequency band energy corresponding to 0 to 250 Hz And a vectorized value of at least one of an average of a composition in which a frequency band energy corresponding to a median of the track and a frequency band energy corresponding to 0 to 500 Hz is normalized to the total energy of each frame. The method of claim 1, wherein the emotional state, Emotion recognition apparatus using a voice signal comprising at least one of a sad state, a normal state, a joy state, an angry state. The method of claim 8, wherein, among the non-verbal parameters, IQR of each pitch value in the section in which the pitch track is rising and falling, IQR of the pitch value in a section in which the pitch track is convex downward, in the slope with respect to the energy band of 500 to 1000 kHz frequency band. Variance, the mean and variance in the 500-1000 Hz frequency band energy trajectory, the median of the trajectory where the frequency band energy corresponding to 0 to 250 kHz is normalized to the total energy of each frame, and the frequency corresponding to the 0 to 500 Hz Characterized in that at least one of the magnitudes of the average values of the average of the composition of band energy normalized to the total energy of each frame corresponds to the sadness state, the normal state, the joy state, and the angry state, respectively, in the ascending order. Emotion Recognition Apparatus Using Voice Signal. The method of claim 8, wherein, among the non-verbal parameters, At least one of the variance in the rate of change of the slope with respect to the 500-1000 Hz frequency band energy trajectory and the magnitude of each mean value with respect to the maximum value in the 500-1000 Hz frequency band energy trajectory is each sad in the emotional state in descending order. Emotion recognition apparatus using a voice signal, characterized in that corresponding to the state, normal state, joy state, anger state. Detecting at least one of a pitch trajectory and an energy trajectory as non-verbal information from the input speech; Detecting a specific section from at least one of the detected pitch trajectory and the energy trajectory, and extracting a non-verbal parameter from the detected specific section; And And determining the emotional state recognized for the spoken sound using the extracted non-verbal parameters. The method of claim 12, wherein the detected specific section, At least one of a downward convex section, an upward convex section, a rising section, and a falling section, which are detected by comparing a slope and a rate of change of the slope with respect to at least one of the detected pitch trajectory and the energy trajectory with a predetermined threshold value. Emotion recognition method using a speech signal, characterized in that the interval. The method of claim 12, wherein the emotion recognition method, And generating a database for each emotional state modeling value using a plurality of non-verbal parameters extracted from the input training signal. 13. The method of claim 12, The detecting process, Detecting the pitch trajectory according to time in the voiced sound by detecting a voiced sound section from the spoken sound; And And detecting at least one energy trajectory over time by detecting energy from the spoken sound. The method of claim 12, wherein the determining comprises: The non-verbal parameter extracted from the input speech sound is compared with modeling values for each emotional state, and the emotional state corresponding to the modeling value having the closest value is determined as the emotional state recognized for the speech sound. Emotion recognition method using the speech signal. The method of claim 12, wherein the determining comprises: Emotion recognition method using a voice signal, characterized in that using one of the Gaussian Mixture Model (GMM) and Hidden Markov Model (HMM) as an algorithm for determining the emotional state. The method of claim 12, wherein the non-verbal parameter is IQR (Interquatile range) of each pitch value in the section where the pitch track is rising and falling, IQR of the pitch value in the section where the pitch track is convex downward, The normalization of the variance in the slope and the variance in the rate of change of the slope, the maximum and mean and variance in the 500–1000 Hz frequency band energy trajectory, and the frequency band energy corresponding to 0 to 250 Hz And a vectorized value of at least one of a mean of a composition in which a frequency band energy corresponding to a median value of a track, 0 to 500 Hz, is normalized to the total energy of each frame. The method of claim 12, wherein the emotional state, Emotion recognition method using a voice signal comprising at least one of a sad state, a normal state, a joy state, an angry state. 19. The method of claim 18 wherein, among the non-verbal parameters, IQR of each pitch value in the section in which the pitch track is rising and falling, IQR of the pitch value in a section in which the pitch track is convex downward, in the slope with respect to the energy band of 500 to 1000 kHz frequency band. Variance, the mean and variance in the 500-1000 Hz frequency band energy trajectory, the median of the trajectory where the frequency band energy corresponding to 0 to 250 kHz is normalized to the total energy of each frame, and the frequency corresponding to the 0 to 500 Hz Wherein at least one of the magnitudes of the average values of the average of the band energy normalized to the total energy of each frame corresponds to the sadness state, the normal state, the joy state, and the angry state among the sadness states in ascending order. Emotion recognition method using the speech signal. 19. The method of claim 18, wherein, among the non-verbal parameters, At least one of the variance in the rate of change of the slope with respect to the 500-1000 Hz frequency band energy trajectory and the magnitude of each mean value with respect to the maximum value in the 500-1000 Hz frequency band energy trajectory is each sad in the emotional state in descending order. Emotion recognition method using a voice signal, characterized in that corresponding to the state, normal state, joy state, anger state.

Images