KR101066228B1 - Emotion classification system and method thereof - Google Patents

Abstract

An emotional classification system and an emotional classification method are disclosed. The emotion classification system and the emotion classification method are based on a range of emotion vectors extracted from a voice signal and at least one first feature vector value extracted from a training voice signal. The voice classified into any one emotion group based on a range of emotion vectors of the voice signal and at least one second feature vector value extracted from the training voice signal after classifying into one emotion group The emotion classification operation is performed by classifying the emotion of the signal sender once again.

Emotion classification, feature vector, pitch, energy, mel frequency cepstral coefficients (MFCC)

Description

Emotion classification system and method {EMOTION CLASSIFICATION SYSTEM AND METHOD THEREOF}

The present invention relates to an emotion classification technique, and more particularly, to an emotion classification system and a method for classifying an emotional state of a sender based on a feature vector extracted from a voice signal.

There is a growing interest in emotion classification or emotion classification systems in areas that require flexible responses, such as call centers, marriage information companies, and mobile content industries. Emotional information from human behavior can be obtained through facial expressions, voice, gestures, heart rate, blood pressure, body temperature, brain waves, and the like. In particular, since the emotional classification method using voice is relatively convenient to input and process voice signals, research on emotional classification technology using the same has been actively conducted.

While general training of emotion classification system is performed using voice signals for emotion training, inquiries of many unspecified customers are recorded in completely different environments. Therefore, classifying the emotions of customers from the voice of the unspecified number of customers using the emotion classification system inevitably leads to poor performance of the system and inaccurate accuracy of the emotion classification due to differences in the recording environment.

Accordingly, the technical problem to be achieved by the present invention is to provide an emotional analysis system and method having a high system efficiency and accuracy of emotional classification even for emotional classification for a plurality of unspecified voice signal transmitters, which are trained based on training voice signals. It is.

An emotional classification system for achieving the above technical problem may include a feature vector extraction block, a feature vector storage block, and an emotion classification block. The feature vector extraction block may extract feature vectors of a received speech signal.

The feature vector storage block includes a plurality of emotion groups including at least one emotion group including at least two emotions among a plurality of emotions based on feature vectors for training speech signals extracted by the feature vector extraction block. Store at least one first feature vector value corresponding to each of the sentiment groups and at least one second feature vector value corresponding to each of the at least two sentiments included in the at least one sentiment group have.

The emotion classification block is a result of emotional classification of the voice signal sender based on the feature vectors of the voice signal and the range of the at least one first feature vector value and the range of a second feature vector value stored in the feature vector storage block. May occur.

The emotional classification block may include a first emotional classification block and a second emotional classification block. The first emotion classification block classifies the emotion of the voice signal sender into one of the plurality of emotion groups based on the range of emotion vectors of the voice signal and the at least one first feature vector value. can do.

The second emotion classification block may classify the at least two emotions included in the at least one emotion group based on the emotion vectors of the voice signal and the range of the at least one second feature vector value.

For example, the feature vector storage block may include a range of average values of pitches corresponding to each of a plurality of emotion groups including at least one emotion group including at least two emotions among the plurality of emotions and the at least one emotion group. It is possible to store a range of energy and a range of MFCC corresponding to each of the at least two emotions included in.

Then, the first emotion classification unit selects one of the plurality of emotion groups the emotion of the sender of the voice signal based on the range of the average value of the pitch of the voice signal and the average value of the pitch stored in the feature vector storage block. And the second emotional classification unit is based on the average value of the energy of the speech signal and the average value of the MFCC and the range of the energy stored in the feature vector storage block and the range of the MFCC. At least two emotions included in the emotion group may be classified.

An emotional classification system according to an embodiment of the present invention may include a feature vector extraction block, a feature vector storage block, and an emotion classification block. The feature vector extraction block may extract an average value, energy, and MFCC of a pitch of the received speech signal.

The feature vector storage block classifies and stores an average value of pitches for training voice signals extracted by the feature vector extraction block into a range of pitch average values corresponding to male normal, male angry and female ordinary, and female angry, respectively. The energy and MFCC of the training voice signal extracted by the feature vector extraction block may be classified and stored into a range of energy and a range of MFCC corresponding to a male anger and a female phase, respectively.

The emotion classification block is based on the average value of the pitch of the speech signal, energy, and the range of the average value of the pitch stored in the MFCC and the feature vector storage block, the range of energy, and the range of the MFCC. May occur.

The emotional classification block may include a first emotional classification unit and a second emotional classification unit. The first emotion classification unit is based on the average value of the pitch of the voice signal and the range of the average value of the pitch stored in the feature vector storage block, the sentiment of the voice signal sender being male ordinary, male angry and female ordinary, and female angry. It can be classified into any one of the emotional groups.

The second emotion classification unit selects one of the male angry male and female female emotional groups based on the energy of the speech signal and the range of the energy stored in the MFCC and the feature vector storage block and the range of the MFCC. Can be classified as

Emotion classification method for solving the technical problem comprises the steps of extracting feature vectors of the received speech signal; At least one first feature vector corresponding to each of the plurality of emotion groups including at least one emotion group including at least two emotions among the plurality of emotions based on feature vectors extracted from the training speech signals Storing a range of values and a range of at least one second feature vector value corresponding to each of the at least two sentiments included in the at least one sentiment group; And generating an emotion classification result of the voice signal sender based on the feature vectors of the voice signal and the range of the at least one first feature vector value and the range of the second feature vector value stored in the feature vector storage block. It may include.

The generating of the emotion classification result comprises: an emotion group of any one of the plurality of emotion groups based on a range of emotion vectors of the voice signal and the at least one first feature vector value. Classifying to; And classifying the at least two emotions included in the at least one emotion group based on a range of emotion vectors of the voice signal and the at least one second feature vector value.

For example, storing the range of the first feature vector and the range of the second feature vector may include each of a plurality of sentiment groups including at least one sentiment group including at least two sentiments among the plurality of sentiments. Storing a range of MFCCs and a range of energy corresponding to each of the at least two sentiments included in the at least one sentiment group and a range of average values of corresponding pitches.

The generating of the emotion classification result of the sender of the voice signal may include the emotion of the sender of the voice signal based on a range of the average value of the pitch of the voice signal and the average value of the pitch. Categorizing into an emotion group; And classifying at least two emotions included in any one emotion group based on the energy of the voice signal and the range of the energy stored in the MFCC and the feature vector storage block and the range of the MFCC.

The emotion classification method according to an embodiment of the present invention may be implemented by executing a computer program for executing the emotion classification method stored in a computer-readable recording medium.

As described above, the emotion classification system and method according to an embodiment of the present invention classify the emotion of the voice signal sender into one of a plurality of emotion groups including at least one emotion according to the range of the feature vector of the voice signal, By classifying the primarily classified results based on the range of different feature vectors, it is possible to improve system performance and negative accuracy of emotional classification that may occur due to differences in the recording environment of the training voice signal and the caller's voice signal. There is.

In order to fully understand the present invention, the operational advantages of the present invention, and the objects attained by the practice of the present invention, reference should be made to the accompanying drawings which illustrate preferred embodiments of the present invention and the contents described in the accompanying drawings.

In the present specification, when one component 'transmits' data or a signal to another component, the component may directly transmit the data or signal to the other component, and at least one other component. Through this means that the data or signal can be transmitted to the other component.

Hereinafter, exemplary embodiments of the present invention will be described in detail with reference to the accompanying drawings. Like reference numerals in the drawings denote like elements.

1 is a block diagram of an emotion classification system 100 according to an embodiment of the present invention. Referring to FIG. 1, the emotion classification system 100 includes a feature vector extraction block 110, a feature vector storage block 120, an emotion classification block 130, and a controller 140. The overall operation of the emotion classification system 100 may be controlled by the controller 140.

The feature vector extraction block 110 may extract feature vectors of the received voice signal. The feature vector of the speech signal may include pitch, energy, speech rate, mel frequency cepstral coefficients (MFCC) including a plurality of mathematical coefficients for modeling the speech signal, but the present invention is not limited thereto.

2 is a block diagram of the feature vector extraction block 110 shown in FIG. 1. Referring to FIG. 1, the feature vector extraction block 110 may include a speech signal division unit 111, a sampling window application unit 112, a non-voice interval elimination unit 113, and a feature vector generation block 114. Can be.

The voice signal dividing unit 111 may divide and output the received voice signal in predetermined frame units. This is to improve the accuracy of feature vector extraction by dividing the speech signal into smaller units and extracting the feature vectors from the divided speech signals.

The sampling window applying unit 112 may multiply and output windows having a predetermined overlapping range with respect to the speech signal divided in units of frames. For example, the sampling window applying unit 112 multiplies and outputs a 50% overlapping Hemming window with a neighboring frame by a speech signal divided by a frame so that unwanted high frequency components affect the feature vector extraction. Can be reduced. If a rectangular window is applied, high frequency components included at the edges of the rectangle may impair the accuracy of feature vector extraction.

The non-voice section removing unit 113 may extract a voice section from the voice signal. This is to improve the performance of the system by removing the non-speech interval from the speech signal in the feature vector extraction. The non-voice interval removing unit 113 may be positioned before or after the division of the voice signal or before or after applying the sampling window. The feature vector generation block 114 may extract feature vectors of the speech signal.

3 is a block diagram of the feature vector generation block 114 shown in FIG. Referring to FIG. 3, the feature vector generation block 114 includes a pitch extraction unit 115, an energy extraction unit 116, a mel frequency cepstral coefficients (MFCC) extraction unit 117, and a feature vector generation unit 119. It may include. The pitch extraction unit 115 may extract and output the pitch of the audio signal. Here, the pitch refers to the frequency of the voice signal and is one of the most basic and important feature vectors for emotion classification of the voice signal sender.

The energy extraction unit 116 may extract and output energy of the voice signal. The MFCC extraction unit 117 may extract a plurality of mathematical coefficients for modeling the speech signal. MFCC is a sine wave component of the frequency-specific power spectrum of a speech signal expressed in mel-scale. The feature vector generating unit 119 can generate the pitch, energy, and MFCC of the speech signal and the mean and standard deviation values thereof. Here, the pitch, energy, and MFCC of the speech signal, and average and standard deviation values thereof may be used as feature vectors for classifying the emotions of the speech signal sender.

The feature vector generation block 114 may further include a delta value extraction unit for extracting pitch, energy, and temporal variation of the MFCC from the sound signal, and also varying the pitch, energy, and temporal variation of the sound signal from the MFCC. It can be used as a feature vector for classifying emotions of voice signal senders.

The feature vector generation unit 119 may further generate an average value and a standard deviation value of the pitch, energy, and delta values of the speech signal, respectively, and the average value of the pitch, energy, and delta values of the speech signal, respectively. And the standard deviation value may also be used as a feature vector that classifies the emotion of the voice signal sender.

The feature vector storage block 120 may classify and store feature vectors extracted from training voice signals by emotion. For example, the feature vector storage block 120 classifies a plurality of emotions into a plurality of emotion groups including at least one emotion group including at least two emotions, and includes at least one feature corresponding to each of the plurality of emotion groups. You can store a range of vector values.

Also, the feature vector storage block 120 may store a range of at least one feature vector value corresponding to each of at least two emotions included in an emotion group including at least two emotions among a plurality of emotion groups. .

Here, the plurality of emotions may include emotions such as ordinary, angry, joy, sadness, fear, surprise, crush, and the like, and the plurality of emotions may be classified by gender, age, etc., but the scope of the present invention is It is not limited to this.

For example, the feature vector storage block 120 may include a range of average values of pitches corresponding to each of the plurality of emotion groups including at least one emotion group including at least two emotions among the plurality of emotions, and the at least The range of the average value of energy and the range of the average value of MFCC corresponding to each of the at least two emotions included in one emotion group may be stored.

In a more specific example, the feature vector storage block 120 calculates an average value of pitches for training voice signals extracted by the feature vector extraction block 110 for male normal, male angry and female ordinary, and female angry, respectively. And store the energy and MFCC of the training voice signal extracted by the feature vector extraction block 110 into the range of the energy corresponding to the male anger and the female normal and the range of the MFCC, respectively. Can be sorted and stored.

The emotion classification block 130 classifies the emotion of the voice signal sender based on the feature vectors of the speech signal and the range of the at least one first feature vector value and the range of the second feature vector value stored in the feature vector storage block 120. Can result.

4 is a block diagram of the emotion classification block 130 shown in FIG. 1. Referring to FIG. 4, the emotion classification block 130 may include a first emotion classification unit 131 and a second emotion classification unit 132. In FIG. 4, only two emotional classification units 131 and 132 are illustrated, but the scope of the present invention is not limited thereto.

The first emotion classification unit 131 is configured to select the emotion of the voice signal originator from among the plurality of emotion groups based on the range of emotion vectors of the voice signal and at least one first feature vector value stored in the feature vector storage block 120. It can be classified into any one emotion group. The second sentiment classification unit 132 is classified by the first sentiment classification unit 131 based on a range of sentiment vectors of the speech signal and at least one second feature vector value stored in the feature vector storage block 120. At least two emotions may be classified in an emotion group including a plurality of emotions among the emotion groups.

For example, the first emotion classification unit 131 may determine the emotion of the sender of the speech signal based on the range of the average value of the pitch of the speech signal and the average value of the pitch stored in the feature vector storage block 120. Can be categorized into one emotion group, and the second emotion classification unit 132 is based on the energy of the voice signal and the range of the energy stored in the MFCC and the feature vector storage block 120 and the range of the MFCC. At least two emotions included in the group may be classified.

In a more specific example, the first emotion classification unit 131 may use the emotion of the voice signal sender based on the average value of the pitch of the voice signal and the range of the average value of the pitch stored in the feature vector storage block 120. Emotion group of any one of the angry and female normal, and the female angry, the second emotional classification unit 132 is the energy of the speech signal and the range of the energy stored in the MFCC and feature vector storage block 120 and Based on the range, an emotional group of male anger and female ordinary can be classified as either an emotional male or female ordinary.

That is, the first emotion classification unit 131 may classify the emotions of the voice signal sender into three types of male ordinary, male angry and female ordinary, and female angry, and the second emotional classification unit 132 may include the first emotional. An emotion group including the male anger and the female normal classified by the classification unit 131 may be classified into the emotion of either the male anger or the female normal.

5 is a graph illustrating an average value of pitches of voice signals according to gender and emotions of men and women. Fig. 5 shows an average value of pitches of a speech signal in which 10 men and women each speak 30 sentences for each of the normal emotion and the angry emotion. As described above, the average value of the pitch of the voice signal illustrated in FIG. 5 may be a criterion of the emotion classification of the first emotion classification unit 131.

Referring to FIG. 5, it can be seen that, among the emotions, the average value of the pitch of the emotion of the male flat and the average value of the pitch of the emotion of the female flat are the most overlapping with each other. At this time, the feature vector storage block 120 may classify and store the range of the average value of the pitch of the voice signal into three types of male normal, male angry and female ordinary, and female angry. Then, the first emotion classification unit 131 may classify the emotion of the voice signal sender into an emotion corresponding to the range of the average value of the pitch stored in the feature vector storage block 120 including the average value of the pitch of the voice signal.

As described with reference to FIG. 5, the emotion classification criterion of the first emotion classification unit 131 may be an average value of pitches of voice signals, but the scope of the present invention is not limited thereto. For example, the first emotion classification unit 131 may be configured as one emotion group among a plurality of emotion groups including a plurality of emotion groups based on a range in which energy, voice speed, and the like of the voice signal overlap. You can also classify.

FIG. 6 shows the emotion classification result of the emotion classification block shown in FIG. 1. Referring to FIG. 6, the first emotion classification unit 131 classifies the emotion of the voice signal sender into three types of male normal, male angry and female ordinary, and female angry, and the second emotional classification unit 132 It can be seen that the group of male angry and female ordinary emotions is again classified into male angry and female ordinary emotions.

As described above, the emotion classification system 100 according to an embodiment of the present invention may perform a two-step emotion classification process through the first emotion classification unit 131 and the second emotion classification unit 132. However, the scope of the present invention is not limited thereto, and the emotion classification system 100 may perform three or more emotion classification processes. The number of emotion classifications performed by the emotion classification system 100 may increase as the number of emotion groups including a plurality of emotions increases.

On the other hand, each of the components of the emotion classification system 100 according to an embodiment of the present invention may be implemented by software, hardware, or a combination of software and hardware for carrying out the technical spirit of the present invention. That is, each of the components of the emotion classification system 100 according to the embodiment of the present invention may be implemented by a logical combination of a predetermined code and a hardware resource for performing the predetermined code.

7 is a schematic flowchart of an emotion classification method according to an embodiment of the present invention. Hereinafter, the process will be described with reference to FIGS. 1 to 4 and 7.

The feature vector extraction block 110 extracts feature vectors for each emotion by using a training voice file, and the feature vector storage block 120 stores feature vectors for each emotion (S70). The emotion vectors stored in the feature vector storage block 120 may store a range of at least one feature vector value corresponding to each of the plurality of emotion groups including at least one emotion group including at least two emotions. A range of at least one feature vector value corresponding to each of at least two emotions included in an emotion group including at least two emotions among the emotion groups of may be stored.

When the training of the emotion classification system 100 using the training voice file is completed, the feature vector extraction block 110 extracts feature vectors of the input voice signal (S80). Then, the emotion classification block 130 classifies the emotion of the voice signal sender based on the emotion-specific feature vectors stored in the feature vector storage block 120 and the feature vectors of the voice signal to generate an emotion result (S90). ).

8 is a detailed flowchart of an emotion classification method according to an embodiment of the present invention. More specifically, FIG. 8 is a flowchart of an emotion classification method of performing a first emotional classification on the basis of the average value of the pitch of the speech signal, and performing a second emotional classification based on the average value of the energy of the speech signal and the average value of the MFCC. to be. Hereinafter, the process will be described with reference to FIGS. 1 to 4 and 8.

The feature vector extraction block 110 extracts and stores the average value, the energy, and the MFCC of each pitch by using the training voice file (S70). When the training of the emotion classification system 100 using the training voice file is completed, the feature vector extraction block 110 extracts an average value, energy, and MFCC of the pitch of the input voice signal (S80).

When the extraction of the feature vectors for the input voice signal is completed, the first emotion classification unit 131 may generate the first emotion based on the average value of the pitch for each emotion stored in the feature vector storage block 120 and the average value of the pitch of the voice signal. The classification is performed (S90a). For example, the first sentiment classification unit 131 is based on the average value of the pitch and the average value of the pitch of the voice signal corresponding to the emotional groups stored in the feature vector storage block 120 of the voice signal sender. And female normal, and female anger.

When the first emotional classification is completed, the second emotional classification unit 132 may perform the second emotional classification based on the emotion-specific energy and the MFCC and the energy of the speech signal and the MFCC stored in the feature vector storage block 120. 90b. For example, the second emotion classification unit 132 may include emotions of voice signal callers classified into an emotion group of male angry and female angry including two or more emotions among the emotional groups classified by the first emotional classification unit 131. Based on the range of energy for each emotion stored in the feature vector storage block 120 and the range of the MFCC, the energy of the voice signal and the MFCC can be classified into male male or female ordinary emotion.

The emotion classification method according to an embodiment of the present invention may also be embodied as computer readable code on a computer readable recording medium. The computer-readable recording medium includes all kinds of recording devices in which data that can be read by a computer system is stored.

For example, a computer-readable recording medium may include a ROM, a RAM, a CD-ROM, a magnetic tape, a floppy disk, an optical data storage device, and the like. The program code for performing the emotion classification method according to an embodiment of the present invention may be May be transmitted in the form of a carrier wave (eg, transmission over the Internet).

The computer readable recording medium can also be distributed over network coupled computer systems so that the computer readable code is stored and executed in a distributed fashion. And functional programs, codes and code segments for implementing the emotion classification method according to an embodiment of the present invention can be easily inferred by programmers in the art to which the present invention belongs.

Hereinafter, a performance evaluation result using a plurality of databases of the emotion classification system 100 according to an embodiment of the present invention will be described.

Table 1 shows three databases for evaluating the performance of the emotion classification system 100.

In Table 1, speaker independence means that utterance sentences do not overlap with each other, and sentence independence means that utterance sentences are different for different emotions. In real life, since different people express their emotions through different sentences, the database of speaker independence and sentence independence will be referred to as a suitable database for performance evaluation of real emotion classification system.

The first database DB1 is composed of emotion-dependent recording files of speaker independence-sentence subordination of emotional expression trainers (eg, performers), and the second database DB2 is speaker independence-sentence in drama dialogue. The third database DB3 is composed of emotional recording files of speaker independence-sentence independence of emotional expression trainees. Therefore, the voice files stored in the third database DB3 are the emotion-specific voice files closest to the emotional expression in real life.

Table 2 trains a conventional normal / angry binary emotional classification system using the voice files of the third database DB3, and uses the remaining voice files of the third database DB3 to train the normal / anger binary sensitivity. The performance of the emotion classification system 100 is evaluated based on the classification method.

Referring to Table 2, the binary sentiment classification system classified 493 out of 500 voice files representing normal emotions as normal emotions, and evaluated 480 out of 500 voice files representing angry emotions as angry emotions. Therefore, the accuracy of the binary emotional classification system is 97.3%. This means that the third database DB3 is a stable database.

Table 3a trains a conventional ordinary / angry binary emotional classification system using voice files of a third database DB3, and shows the performance of the binary emotional classification system using voice files of a first database DB1. It is evaluated.

Referring to Table 3a, the binary sentiment classification system classified 500 out of 500 voice files representing ordinary emotions as normal emotions and evaluated 10 out of 500 voice files representing angry emotions as angry emotions. Therefore, the accuracy of the binary emotional classification system is 51.0%. Referring to Tables 2 and 3a, even if the binary emotional classification system is trained based on a stablely constructed third database (DB3), the performance of the binary emotional classification system is degraded for databases recorded in different environments. Able to know.

Table 3b trains the emotion classification system 100 according to an embodiment of the present invention using the voice files of the third database DB3, and uses the voice files of the first database DB1. 100) is evaluated.

Referring to Table 3b, the emotion classification system 100 classified 195 out of 250 voice files representing male normal emotion as male normal, and classified 205 out of 250 voice files representing male angry emotion as male angry. Of the 250 voice files representing female normal sensibilities, 220 were classified as female normals. Therefore, the accuracy of the emotion classification system 100 is 86.3%.

3A and 3B, it can be seen that the emotional classification performance of the same database recorded in different environments of the emotional classification system 100 according to the embodiment of the present invention is superior to the conventional binary emotional classification system. .

Table 4a trains the conventional binary emotion classification system using the voice files of the third database DB3, and evaluates the performance of the binary emotion classification system using the voice files of the second database DB2.

Referring to Table 4a, the binary emotional classification system classified 490 out of 500 voice files representing normal emotions as normal emotions, and evaluated 115 out of 500 voice files representing angry emotions as angry emotions. Therefore, the accuracy of the binary emotional classification system is 60.5%. Referring to Tables 2 and 4a, even if the binary emotional classification system is trained based on a stablely constructed third database (DB3), the performance of the binary emotional classification system is degraded for databases recorded in different environments. Able to know.

Table 4b shows the training of the emotion classification system 100 using the voice files of the third database DB3, and the performance of the emotion classification system 100 using the voice files of the second database DB2. will be.

Referring to Table 4b, the emotion classification system 100 classifies 198 of 250 voice files representing male normal emotion as male normal, and classifies 165 of 250 voice files representing male anger emotional as male angry. Of the 250 voice files representing female normal emotion, 188 were classified as female normal, and 200 of 250 voice files representing female anger emotional were classified as female male. Therefore, the accuracy of the emotion classification system 100 is 75.1%.

4A and 4B, it can be seen that the emotional classification performance of the same database recorded in different environments of the emotional classification system 100 according to the embodiment of the present invention is superior to the conventional binary emotional classification system. .

Currently, the emotional classification system used in the customer center is mostly recorded in different environments for the training voice and the unspecified customer's voice, so the performance of the traditional flat / angry binary emotional classification system is low. However, even if the emotion classification system 100 according to an embodiment of the present invention is applied to a complaint management system of customers in a corporate environment such as a customer center, it is possible to prevent the performance degradation of the system due to differences in recording environments.

Furthermore, the emotion classification system 100 according to the embodiment of the present invention may be usefully used in a field in which the emotions of a plurality of unspecified customers, such as a fire station, a police station, a marriage information company, etc., must be quickly recognized and responded to.

Although the invention has been described with reference to one embodiment shown in the drawings, this is merely exemplary, and it will be understood by those skilled in the art that various modifications and equivalent other embodiments are possible. Therefore, the true technical protection scope of the present invention will be defined by the technical spirit of the appended claims.

1 is a block diagram of an emotion classification system according to an embodiment of the present invention.

FIG. 2 is a block diagram of the feature vector extraction block shown in FIG. 1.

3 is a block diagram of the feature vector generating unit shown in FIG.

4 is a block diagram of the emotion classification block shown in FIG. 1.

5 is a graph illustrating an average value of pitches of voice signals according to gender and emotions of men and women.

FIG. 6 shows the emotion classification result of the emotion classification block shown in FIG. 1.

7 is a schematic flowchart of an emotion classification method according to an embodiment of the present invention.

8 is a detailed flowchart of an emotion classification method according to an embodiment of the present invention.

Claims (19)

A feature vector extraction block for extracting feature vectors of a received speech signal; Corresponding to each of a plurality of emotion groups including at least one emotion group including at least two emotions among a plurality of emotions based on feature vectors for training speech signals extracted by the feature vector extraction block A feature vector storage block for storing a range of at least one first feature vector value and at least one second feature vector value corresponding to each of the at least two emotions included in the at least one emotion group; And Emotion classification that generates an emotion classification result of the voice signal sender based on the range of the feature vectors and the range of the at least one first feature vector value and the range of the second feature vector value stored in the feature vector storage block. Block, The emotion classification block A first emotion classification unit that classifies the emotion of the voice signal sender into any one emotion group among the plurality of emotion groups based on the range of emotion vectors of the voice signal and the at least one first feature vector value; And An emotion classification system including a second emotion classification unit for classifying the at least two emotions included in the at least one emotion group based on a range of emotion vectors of the speech signal and the at least one second feature vector value . delete The method of claim 1, wherein the feature vector extraction block is A voice signal dividing unit for dividing and outputting the voice signal in predetermined frame units; A sampling window application unit which multiplies and outputs windows having a predetermined overlapping range with respect to the speech signal divided in the frame unit; A non-voice section removing block for extracting a voice section from the voice signal; And And a feature vector generation block for extracting feature vectors of the speech signal. The method of claim 3, wherein the feature vector generation block is A pitch extraction unit for extracting and outputting a pitch of the voice signal; An energy extraction unit for extracting and outputting energy of the voice signal; Mel frequency cepstral coefficients (MFCC) extraction unit for extracting a plurality of mathematical coefficients for modeling the speech signal; And And a feature vector generating unit for generating an average value and / or a standard deviation value for at least one of pitch, energy, and MFCC of the speech signal. The method of claim 4, wherein the feature vector generation block is A delta value extracting unit for extracting pitch, energy, and temporal change amount for MFCC of the speech signal, The feature vector generating unit And an average value and a standard deviation value of pitch, energy, and delta values for the MFCC of the speech signal. The method of claim 4, wherein the feature vector storage block is A range of an average value of a pitch corresponding to each of the plurality of emotion groups including the at least one emotion group including the at least two emotion groups among the plurality of emotions and the at least two included in the at least one emotion group An emotional classification system that stores a range of energy and a range of MFCCs corresponding to each of the emotions. The method of claim 6, wherein the first emotional classification unit Classify the sentiment of the sender of the voice signal into any one of the plurality of emotion groups based on a range of the average value of the pitch of the voice signal and the average value of the pitch stored in the feature vector storage block, The second emotional classification unit An emotion classification system for classifying at least two emotions included in the at least one emotion group based on the energy of the speech signal and the range of MFCC and the range of energy stored in the feature vector storage block and the MFCC. A feature vector extraction block for extracting an average value, an energy, and an MFCC of a pitch of the received speech signal; The average value of the pitches for the training voice signals extracted by the feature vector extraction block is classified into a range of pitch average values corresponding to each of the male normal, the male angry and the female ordinary, and the female angry, and the feature vector extraction is performed. Characteristic vector storage block for storing the energy and the MFCC of the training voice signal extracted by the block into a range of energy and a range of MFCC corresponding to the male anger and the female normal, respectively: Emotion classification that generates the emotion classification result of the voice signal sender based on the average value of the pitch of the voice signal, the energy, and the range of the average value of the pitch stored in the MFCC and the feature vector storage block, the energy range, and the range of the MFCC. Block, The emotion classification block Based on the average value of the pitch of the voice signal and the range of the average value of the pitch stored in the feature vector storage block, the sentiment of the voice signal sender is sent to the emotional group of any one of male ordinary, male angry and female ordinary, and female angry. A first emotional classification unit to classify; And A second classifying an emotional group of male angry and female ordinary into one of male angry or female ordinary based on the energy of the speech signal and the range of the energy stored in the MFCC and the feature vector storage block and the range of the MFCC Emotion classification system comprising an emotion classification unit. delete The method of claim 8, wherein the feature vector extraction block is A voice signal dividing unit for dividing and outputting the voice signal in predetermined frame units; A sampling window application unit which multiplies and outputs windows having a predetermined overlapping range with respect to the speech signal divided in the frame unit; A non-voice section removing block for extracting a voice section from the voice signal; And And a feature vector generation block for extracting feature vectors of the speech signal. 11. The method of claim 10, wherein the feature vector generation block is A pitch extraction unit for extracting and outputting a pitch of the voice signal; An energy extraction unit for extracting and outputting energy of the voice signal; Mel frequency cepstral coefficients (MFCC) extraction unit for extracting a plurality of mathematical coefficients for modeling the speech signal; And And a feature vector generating unit for generating an average value and / or a standard deviation value for at least one of pitch, energy, and MFCC of the speech signal. The method of claim 11, wherein the feature vector generation block is A delta value extracting unit for extracting pitch, energy, and temporal change amount for MFCC of the speech signal, The feature vector generating unit And an average value and a standard deviation value of pitch, energy, and delta values for the MFCC of the speech signal. Extracting feature vectors of the received speech signal; At least one first feature vector corresponding to each of the plurality of emotion groups including at least one emotion group including at least two emotions among the plurality of emotions based on feature vectors extracted from the training speech signals Storing a range of values and a range of at least one second feature vector value corresponding to each of the at least two sentiments included in the at least one sentiment group; And Generating an emotional classification result of the voice signal sender based on the feature vectors of the voice signal and the stored range of the at least one first feature vector value and a range of a second feature vector value, Generating the emotion classification result Classifying the emotion of the voice signal sender into any one of the plurality of emotion groups based on the range of emotion vectors of the voice signal and the at least one first feature vector value; And And classifying the at least two emotions included in the at least one emotion group based on a range of emotion vectors of the speech signal and the at least one second feature vector value. delete The method of claim 13, wherein extracting feature vectors of the speech signal comprises: Dividing and outputting the voice signal into predetermined frame units; Multiplying and outputting windows having a predetermined overlapping range with respect to the speech signal divided in the frame unit; Extracting a voice section from the voice signal; And And generating feature vectors of the speech signal. 16. The method of claim 15, wherein generating feature vectors of the speech signal Extracting and outputting a pitch of the voice signal; Extracting and outputting energy of the voice signal; Extracting an MFCC including a plurality of mathematical coefficients for modeling the speech signal; And Generating a mean value and / or a standard deviation value for at least one of pitch, energy, and MFCC of the speech signal. 17. The method of claim 16, wherein storing the first feature vector value range and the second feature vector value range comprises: A range of an average value of a pitch corresponding to each of the plurality of emotion groups including the at least one emotion group including the at least two emotion groups among the plurality of emotions and the at least two included in the at least one emotion group Storing the range of energy and range of MFCC corresponding to each of the emotions. 18. The method of claim 17, wherein generating the emotion classification result of the sender of the voice signal Classifying the emotion of the sender of the speech signal into any one emotion group among the plurality of emotion groups based on a range of the average value of the pitch of the speech signal and the average value of the pitch; And And classifying at least two emotions included in the at least one emotion group based on the energy of the speech signal and the MFCC and the stored range of energy and the range of MFCC. A computer-readable recording medium storing a computer program for executing the emotion classification method according to any one of claims 13 and 15 to 18.

Images