KR101006049B1 - Apparatus and method for recognizing emotion - Google Patents

Abstract

Disclosed are an apparatus and method for analyzing an input voice signal and recognizing an emotion contained in the voice signal according to the result. An emotion recognition apparatus according to an embodiment may include an input unit configured to receive a target voice signal; A converter for converting the input voice signal into an input matrix using a spectrogram; An NMF performer for calculating a feature vector by performing non-negative matrix factorization (NMF) on the input matrix; An emotion model database that stores model vectors for each emotion model; A comparison unit comparing the feature vector with a model vector stored in the emotion model database; And a determination unit that determines an emotional state of the target voice signal as an emotion model corresponding to the model vector most similar to the feature vector as a result of the comparison in the comparison unit. Provided are an emotion recognition apparatus and method for determining an emotion model vector from a spectrogram for each emotion model using non-negative matrix factorization, and using the same to recognize emotions contained in a speaker's voice.

Emotion, Recognition, Speech, Analysis, Nonnegative Matrix Factorization

Description

Apparatus and method for recognizing emotion

The present invention relates to an emotion recognizing apparatus, and more particularly, to an apparatus and a method for analyzing an input voice signal and recognizing an emotion contained in the voice signal according to the result.

Voice is the most natural means of communication and information delivery. As the field of speech information technology (SIT) for the effective processing and digitization of human voice has been made a remarkable development, it has been applied to real life one after another.

Such voice information processing techniques are classified into speech recognition, speech synthesis, speaker identification and verification, speech coding, and the like.

In relation to the voice information processing technology, it is possible to consider an emotion recognition technology that estimates and recognizes the speaker's emotional state by analyzing the voice signal. Emotion recognition technology attempts to numerically recognize emotions embedded in language, voice, etc. that people use in their daily lives.

A representative example of the emotion recognition technology based on the voice signal analysis is a lie detector. A polygraph is a type of polygraph, which means a system that detects a person's state of excitement, tension or emotional conflict by a predefined standard. When a person lie, the blood pressure of the vocal cords decreases due to mental tension, and the distorted sound waves come out of the vocal cords due to unavoidable neural action. However, such a lie detector has a problem in that it is not possible to analyze and determine from the voice signal various emotions contained in the human voice in everyday life.

In addition, there is a need for a system capable of providing various services to a user by performing predetermined signal processing through an emotion recognition technique based on the voice signal analysis.

Accordingly, the present invention determines an emotion model vector from a spectrogram for each emotion model using non-negative matrix factorization, and uses the same to recognize emotions contained in the speaker's voice. A recognition apparatus and method are provided.

In addition, the present invention provides an emotion recognition apparatus and method that can provide a variety of customer service, such as switching the response method or connection of the best counselor on the basis of the results of the emotion recognition based on the analyzed voice. .

According to an aspect of the present invention, there is provided an emotion recognition apparatus for determining an emotional state by analyzing a voice signal.

An emotion recognition apparatus according to an embodiment may include an input unit configured to receive a target voice signal; A converter for converting the input voice signal into an input matrix using a spectrogram; An NMF performer for calculating a feature vector by performing non-negative matrix factorization (NMF) on the input matrix; An emotion model database that stores model vectors for each emotion model; A comparison unit comparing the feature vector with a model vector stored in the emotion model database; And a determination unit that determines an emotional state of the target voice signal as an emotion model corresponding to the model vector most similar to the feature vector as a result of the comparison in the comparison unit.

The comparison unit may perform the comparison by using a square of an Euclidean distance between the feature vector and the model vector.

The converter may set a frequency axis and a time axis as rows and columns of the input matrix with respect to a spectrogram of the target voice signal, and set an amplitude value of the target voice signal according to a change in frequency and time as an element. . Here, each element of the input matrix may be non-negative.

The NMF performing unit may select a basis matrix representing a frequency characteristic of the target speech signal among the results of performing the non-negative matrix factorization of the input matrix as the feature vector.

The input unit receives one or more model voice signals, and the converter generates a basic input matrix by using the spectrogram of the model voice signal, and generates a composite input matrix horizontally pasting the basic input matrix for each emotion model. The NMF execution unit may perform non-negative matrix factorization on the synthesized input matrix to calculate the model vector and store it in the emotion model database. The model voice signal may include the target voice signal in which an emotional state is determined by the determination unit.

According to another aspect of the present invention, there is provided an emotion recognition method for analyzing the voice signal to determine the emotional state and a recording medium having recorded thereon a program for performing the same.

According to an embodiment, a method of recognizing emotions may include: receiving a target voice signal (a); (B) converting the received target voice signal into an input matrix using a spectrogram; (C) calculating a feature vector by performing non-negative matrix factorization (NMF) on the input matrix; (D) comparing the feature vector with a model vector stored in an emotion model database; And (e) determining an emotional state of the target voice signal using an emotion model corresponding to a model vector most similar to the feature vector as a result of the comparison.

In step (d), a comparison may be performed using a square of an Euclidean distance between the feature vector and the model vector.

In the step (b), the frequency axis and the time axis are set as rows and columns of the input matrix with respect to the spectrogram of the target voice signal, and the amplitude value of the target voice signal according to the change of frequency and time is an element. can do. Here, each element of the input matrix may be non-negative.

In step (c), a basis matrix representing a frequency characteristic of the target speech signal may be selected as the feature vector among the results of performing the non-negative matrix factorization of the input matrix.

Receiving at least one model voice signal; Generating a basic input matrix using a spectrogram of the model speech signal; Generating a composite input matrix by horizontally pasting the basic input matrix for each emotion model; Calculating a model vector by performing non-negative matrix factorization on the composite input matrix; And storing in the emotion model database may be performed prior to step (a).

Other aspects, features, and advantages other than those described above will become apparent from the following drawings, claims, and detailed description of the invention.

Emotion recognition apparatus and method according to the present invention by using a non-negative matrix factorization to determine the emotion model vector from the spectrogram for each emotion model, using the emotion recognition apparatus and method for recognizing the emotions contained in the speaker's voice to provide.

In addition, on the basis of the result of recognizing the emotion based on the analyzed voice, it is possible to provide a variety of customer service such as switching the method of answering the phone call or connecting excellent counselors.

As the invention allows for various changes and numerous embodiments, particular embodiments will be illustrated in the drawings and described in detail in the written description. However, this is not intended to limit the present invention to specific embodiments, it should be understood to include all transformations, equivalents, and substitutes included in the spirit and scope of the present invention. In the following description of the present invention, if it is determined that the detailed description of the related known technology may obscure the gist of the present invention, the detailed description thereof will be omitted.

The terms first, second, etc. may be used to describe various components, but the components should not be limited by the terms. The terms are used only for the purpose of distinguishing one component from another.

The terminology used herein is for the purpose of describing particular example embodiments only and is not intended to be limiting of the present invention. Singular expressions include plural expressions unless the context clearly indicates otherwise. In this application, the terms "comprise" or "having" are intended to indicate that there is a feature, number, step, operation, component, part, or combination thereof described in the specification, and one or more other features. It is to be understood that the present invention does not exclude the possibility of the presence or the addition of numbers, steps, operations, components, components, or a combination thereof.

Hereinafter, embodiments of the present invention will be described in detail with reference to the accompanying drawings.

In the present invention, the target voice signal is a voice signal that is an object of emotion determination, and may be, for example, a voice when a customer calls a call center. The model voice signal is a voice signal that is a criterion for emotion determination, and is contained in such a manner that the characteristic of a specific emotion model is sufficient and can be distinguished from other emotion models. The model voice signal may be a voice signal represented by a trained voice actor or the like.

1 is a block diagram of an emotion recognition apparatus according to an embodiment of the present invention.

The emotion recognition apparatus 100 analyzes the input target voice signal to recognize and display or inform what emotions are included in the target voice signal. In the present invention, non-negative matrix factorization is used for emotion recognition, which will be described in detail later.

The emotion recognizing apparatus 100 classifies one of the following emotion models in recognizing the emotions included in the target voice signal. The emotion model refers to an emotional state that can be judged from a voice signal, and may be classified into neutral, happy, angry, sad, and the like. In some embodiments, an emotional model such as bored may be further added.

The emotion recognition apparatus 100 is connected to a wired or wireless communication network (hereinafter referred to as a "communication network") and receives a voice signal with a user terminal connected to the communication network. Alternatively, the emotion recognition apparatus 100 may receive a voice signal through an input interface provided separately.

That is, the voice signal is transmitted from the user terminal to the emotion recognizing apparatus 100 in real time or in the form of a pre-stored file through a communication network, or directly through a voice input device such as a microphone provided in the emotion recognizing apparatus 100. A voice signal input in advance or stored in a file form may be input through a data input unit included in the emotion recognition apparatus 100. In addition, a method of inputting a voice signal to the emotion recognition apparatus 100 may vary within the spirit of the present invention.

The emotion recognition apparatus 100 may include an input unit 110, a converter 120, an NMF performer 130, a comparator 140, a determiner 150, and an emotion model database 160.

The input unit 110 receives a voice signal through a communication network or receives a voice signal through an input interface provided separately in the emotion recognition apparatus 100. The voice signal may be a target voice signal and / or a model voice signal. The voice signal may be input in real time or may be input in advance in a file form.

The voice signal input to the input unit 110 may include a background noise. The emotion recognition apparatus 100 further includes a background noise filtering unit (not shown) to filter the background noise to be more accurate and reliable. High emotional awareness can be achieved.

The converter 120 converts the voice signal input to the input unit 110 into a matrix according to a spectrogram. Spectrograms allow you to visualize and identify sounds or waves, combining a combination of waveform and spectrum characteristics. In the waveform, only the change in amplitude according to the change of the time axis and the change in the amplitude according to the change of the frequency axis in the spectrum can be identified, whereas in the spectrogram, the change in the amplitude according to the change in the time axis and the frequency axis can be identified.

Here, rows are classified according to frequency units of the frequency axis, and columns are classified according to time units of the time axis. Thereafter, the amplitude of an arbitrary frequency and time is an element of a row corresponding to the frequency and a column corresponding to the corresponding time, thereby converting the speech signal into a matrix. For example, when the voice signal is divided into 512 frequencies and separated into 200 time units, the converter 120 may convert the voice signal into a 512 x 200 matrix.

Each element of the matrix is a non-negative amplitude and time amplitude of a speech signal.

The NMF execution unit 130 calculates a basis vector by performing non-negative matrix factorization (NMF) on the matrix converted by the conversion unit 120. Since each element of the matrix converted by the transform unit 120 is non-negative as described above, it is possible to perform non-negative matrix factorization.

The basic vector calculated when the speech signal is a model speech signal is a model vector, and the basic vector calculated when the speech signal is a target speech signal is a feature vector. The model vectors are distinguished from each other according to the emotion model to which the model speech signal belongs.

When the NMF performer 130 performs the non-negative matrix factorization on the model speech signal, the model vector, which is the calculated basic vector, is stored in the emotion model database 160.

The comparison unit 140 is a feature vector, which is a basic vector calculated when the NMF execution unit 130 performs a non-negative matrix factor on the target speech signal, and a model for each emotion model stored in the emotion model database 160. Compare vectors.

The emotion model database 160 stores model vectors having features that are distinguished from each other according to the emotion model, and compares similarities between the feature vector and one or more model vectors. According to an embodiment, Euclidean distance may be used as a method for comparing similarity.

The determination unit 150 uses the result of comparing the at least one model vector stored in the emotion model database 160 with the feature vector of the target voice signal calculated by the NMF execution unit 130 to determine whether the emotion included in the target voice signal is included. Determine what it is. That is, a model vector most similar to the feature vector of the target voice signal is selected, and it is determined that the emotion model to which the model vector belongs is the emotion contained in the target voice signal.

The emotion selected by the determination unit 150 may be displayed in various ways. The similarity of the corresponding target speech signal to each emotion model is indicated by a bar or number, in a color that represents the selected emotion model (for example, blue for neutral, red for anger, green for happiness, etc.) or for each emotion model. It can be displayed as a figure, such as a character representing the facial expression.

Alternatively, various services may be provided based on the selected emotion according to the determination result of the determination unit 150. For example, it is assumed that the emotion recognition device 100 is applied to the call center system.

The call center system connects one of the many external customers with one of the internal agents. At this time, it is possible to analyze the voice of the customer during the consultation or before the consultation to recognize the current emotional state of the customer and to provide a suitable service. For example, if the customer's emotional state is anger, anger, excitement, etc., the automatic response process may be omitted and directly connected to an agent, or may be connected to an experienced counselor with experience in customer response.

The emotion recognition apparatus 100 according to another embodiment may further include an update unit. After determining the emotional state included in the target voice signal that is the object of emotion recognition, the update unit updates the model vector stored in the emotion model database 160 by using the target voice signal as an additional model voice signal belonging to the determined emotional state. .

An emotion recognition method in the emotion recognition apparatus 100 will be described in detail with reference to the drawings below with reference to FIG. 2.

2 is a flowchart of a emotion recognition method according to an embodiment of the present invention, and FIG. 3 is an exemplary diagram of a matrix in which a voice signal is converted according to an embodiment of the present invention.

The input unit 110 receives a target voice signal (step S210). It may be received through a communication network or through an input interface provided separately in the emotion recognition apparatus 100. The target voice signal may be transmitted / input in real time or may be transmitted / input in the form of a pre-stored file.

The conversion unit 120 converts the input speech signal into an input matrix by using the spectrogram of the received target speech signal (step S220).

Human emotions have a different frequency spectrum depending on the vocal cords. For example, a neutral emotional voice has a strong frequency component in the 0 to 4000 Hz band, and an angry or happy voice is evenly distributed throughout the 0 to 8000 Hz band. In addition, the shape of the spectrum may have a peak and a valley accurately classified according to the emotion model. Therefore, when using a spectrogram for the speech signal, it is possible to use characteristics distinguished according to each emotion model.

In the spectrogram, we can see the change in amplitude with the change of time and frequency axis. Referring to FIG. 3, an example of converting a target speech signal into an input matrix is illustrated. The spectrogram is used for the target speech signal to determine the amplitude value according to the change of the time axis and the frequency axis as each element of the input matrix. Each element of the input matrix is a non-negative bar that is an amplitude value over time and frequency of the target speech signal.

The input matrix divides the rows according to the frequency units of the frequency axis and the columns according to the time units of the time axis. For example, if the frequency axis is divided into 512 frequencies f1 to f512 and the time axis is divided into 200 times t1 to t200, the input matrix may be converted into a 512 x 200 matrix. The amplitude value at the frequency f3 and the time t2 becomes the three-row, two-column element V (3, 2) of the input matrix V.

The NMF execution unit 130 performs non-negative matrix factorization on the input matrix to calculate a feature vector (step S230).

Non-negative matrix factorization (NMF) has properties that are useful for decomposing multivariate data. The update rule is applied to decompose the input matrix into two matrices.

In Equation 1, V is an input matrix, W is a basis matrix, and H is an encoding matrix. Here, the input matrix V must be non-negative of each element.

The base matrix W has the characteristic information of the input matrix, and the encoding matrix H has the encoding information. The base matrix W and the encoding matrix H are updated by applying a multiplicative update rule of Equation 2 below.

가 증가하지 않는 특징을 가지고 있다. 그리고 계속적인 업데이트를 수행하며 하기 수학식 3의 비용 함수(cost function)을 이용하여 오차를 계산하게 된다. Under the update rule of Equation 2, the Euclidean distance between V and WH

Has the characteristic of not increasing. The continuous update is performed and an error is calculated using a cost function of Equation 3 below.

Here, A is the input matrix V, B is the product of the base matrix W and the encoding matrix H.

이 최소가 되는 경우에 입력 행렬 V에 대하여 기초 행렬 W와 인코딩 행렬 H를 결정하게 된다. The cost function in Equation 3 uses the square of the Euclidean distance between A and B.

In this case, the base matrix W and the encoding matrix H are determined for the input matrix V.

In addition, a cost function such as Equation 4 below may be used as the cost function.

The base matrix W has the basic information of the input matrix V. Therefore, if the properties of the input matrix V are different, the properties of the base matrix W are also different. In the present invention, when the input matrix V uses the spectrogram, the base matrix W has information related to the frequency of the speech signal, and the encoding matrix H has encoding information over time of the speech signal. Therefore, the base matrix W representing the frequency characteristic of the input matrix V becomes a feature vector.

It is assumed that the input matrix V is an n x m matrix. In this case, the input matrix V may determine that there are m data vectors in n-dimensions. For example, in the case of the input matrix V illustrated in FIG. 3, it may be determined that there are 200 data vectors of 512 dimensions with respect to frequency.

By matrix operation, the base matrix W has a dimension of n x r, and the encoding matrix H has a dimension of r x m. In this case, since the characteristics of the input matrix V are in n instead of m, the base matrix W represents the characteristics of the input matrix V. Here, r is an arbitrary natural number that is not determined and can be set by the user. In the following description, it is assumed that r is 1, but the present invention is not limited thereto.

In this way, a base vector W representing a feature of the input matrix V is calculated and determined as a feature vector.

The comparison unit 140 compares the feature vector calculated as a result of performing the non-negative matrix factorization with the model vector W _E stored in the emotion model database 160 (step S240). Here, the model vector W _E is a result of performing the non-negative matrix factorization on the model voice signal as described above, and is previously stored in the emotion model database 160 for each emotion model. This will be described later in detail with reference to FIG. 4.

The method of comparing the feature vector and the model vector is to use the square of Euclidean distance. Euclidean distance is to select the most similar model vector by finding the difference between the stored model vector and the new feature vector.

If only Euclidean distance is used, only the difference value is obtained.However, if additional squares are used, the larger value is larger and the smaller value is smaller. And the non-characterized part will not have a big difference, which can lead to better recognition rate.

In this embodiment, Euclidean distance is used as a method for finding similarity between the feature vector and the model vector. However, it is to be understood that this is only one embodiment and that various methods of finding similarity between vectors may be applied to the present invention.

The determination unit 150 selects a model vector most similar to the feature vector as a result of the comparison between the model vector and the feature vector (step S250). In addition, it is determined that the emotion model indicated by the selected model vector is an emotion state included in the target speech signal having the feature vector that is the current determination target (step S260).

Thus, the emotion recognition apparatus 100 recognizes what emotions are included in the target voice signal. And, depending on the result, a predetermined task can be performed. In an embodiment, the determined emotion may be displayed as a bar bar, a numerical value, a color, a character, or the like so that the user can check the determined emotion. In another embodiment, when the emotion recognition apparatus 100 is applied to the call center system, flexibility may be added to the counselor connection according to the emotion state of the customer determined from the voice of the customer.

The emotion recognition method using the target voice signal has been described above, and a model vector generation method using the model voice signal will be described below.

4 is a flowchart of a method for generating a model vector according to an embodiment of the present invention, FIG. 5 is a diagram illustrating examples of model speech signals for various emotion models, and FIG. 6 is an input matrix generated using a model speech signal. Is an illustration of an example.

When generating the model vector, the comparator 140 and the determiner 150 among the components of the emotion recognition apparatus 100 may not be activated.

The input unit 110 receives one or more model voice signals (step S410). It may be received through a communication network or through an input interface provided separately in the emotion recognition apparatus 100. The model voice signal may be transmitted / inputted in real time or transmitted / input in the form of a pre-stored file.

The model voice signal is a voice signal serving as a reference for determining an emotional state included in a target voice signal to be input later, and the larger the number, the more accurate the voice signal may be. Therefore, when generating a model vector, it is preferable to receive a plurality of model voice signals.

The input model voice signal is classified for each emotion model (step S420). The model voice signal is judged as to what implied emotional state is completed, and each model voice signal is grouped according to which emotion model belongs.

Here, in the case of the target voice signal in which emotion recognition is completed by the method illustrated in FIG. 2, the target voice signal may function as a model voice signal indicating a specific emotional state.

In the present embodiment, the description has been made on the assumption that a plurality of model voice signals belonging to various emotion models are input. Alternatively, only model voice signals belonging to a specific emotion model may be input. In this case, the classification process of step S420 may be omitted.

After the model voice signal is classified for each emotion model, steps S430 to S450 are separately performed according to each emotion model.

The model speech signal is converted into a basic input matrix using a spectrogram (step S430). The conversion to the basic input matrix is the same as in the above-described step S220 of the emotion recognition method, and thus a detailed description thereof will be omitted.

One basic input matrix is generated for each model speech signal, and the same number of basic input matrices as the number of input model speech signals are generated.

If there is only one input model voice signal, step S440 may be omitted since there is also one basic input matrix. The default input matrix is the composite input matrix.

If there is more than one input model voice signal, since the basic input matrix is more than one, step S440 is performed. A composite input matrix is formed by pasting two or more basic input matrices in a horizontal direction (step S440). Pasting in the horizontal direction means incrementing columns while keeping rows for two or more basic input matrices.

Referring to FIG. 5, a composite input matrix divided by emotion models is illustrated. The first composite input matrix 500N for neutral, the second composite input matrix 500A for anger, the third composite input matrix 500H for happiness, and the fourth composite input matrix 500S for sadness are shown. It is.

Referring to the first synthesis input matrix 500N, the first synthesis input matrix 500N includes n basic input matrices 510-1, 510-2, 510-3, ..., 510-n. . The first composite input matrix 500N is generated by increasing columns while maintaining rows for the n basic input matrices 510-1, 510-2, 510-3, ..., 510-n. This is because, in the present invention, the frequency is a reference for identifying an important characteristic of the speech signal, and the row representing the frequency has the important characteristic of the input matrix.

Assuming that the primary input matrix is a 512 x 200 matrix, the (1, 1) element of the first basic input matrix 510-1 becomes the (1, 1) element of the first composite input matrix 500N (N1 (1, 1) = N (1, 1)), and the (1, 1) element of the second basic input matrix 510-2 becomes the (1, 201) element of the first composite input matrix 500N ( N2 (1, 1) = N (1, 201)).

The NMF execution unit 130 calculates a model vector by performing non-negative matrix factorization on the synthesized input matrix (step S450). Since the method of performing the non-negative matrix factorization is similar to the process of performing the non-negative matrix factorization of the emotion recognition method described above (step S230), a detailed description thereof will be omitted.

Here, the input matrix is a composite input matrix when the nonnegative matrix factorization is performed, and the base vector is a model vector. In non-negative matrix factorization, the larger the number of columns in the composite input matrix, the better the basis vector will represent the multiple frequency features for that emotion model.

By performing steps S430 to S450 for each emotion model, a model vector for each emotion model may be obtained. The acquired model vector is stored in the emotion model database 160 (step S460). Thereafter, when the target voice signal is input, the target voice signal is used for emotion recognition.

7A-7C are examples of model vectors obtained in accordance with one embodiment of the present invention. Here, the horizontal axis corresponds to the row of the model vector and represents 512 frequencies. And the vertical axis represents the amplitude characteristic at each frequency. In particular, the horizontal axis divides the 0 to 8000 Hz band into 512 units.

The emotion models represent neutral (see FIG. 7A), anger (see FIG. 7B), and sadness (see FIG. 7C), and features distinguished according to each emotion model. Such a model vector may have a slightly different value depending on the type, number, etc. of the model voice signal, but the main features are shared.

For example, it is shown in FIG. 7A that the speech signal has a strong value in the 0 to 4000 Hz band when the emotion model is neutral. In addition, when the emotional model is angry, it is shown in FIG. 7B that voice signals are evenly distributed throughout the 0 to 8000 Hz band. In addition, it is shown in FIG. 7C that a peak and a valley appear repeatedly in a specific band when the emotional model is sad.

Meanwhile, the above-described emotion recognition method and / or model vector generation method can be created by a computer program. The codes and code segments that make up the program can be easily deduced by a computer programmer in the field. In addition, the program is stored in a computer readable media, and read and executed by a computer to implement the emotion recognition method and / or model vector generation method. The information storage medium includes a magnetic recording medium, an optical recording medium, and a carrier wave medium.

It is to be understood that one or more of the components in the embodiments of the present invention may be implemented in an integrated manner, or some of the components may be functionally subdivided and implemented, which is within the scope of the present invention.

It will be apparent to those skilled in the art that various modifications and variations can be made in the present invention without departing from the spirit or scope of the invention as defined in the appended claims. It will be understood that the invention may be varied and varied without departing from the scope of the invention.

1 is a block diagram of an emotion recognition apparatus according to an embodiment of the present invention.

2 is a flowchart of a emotion recognition method according to an embodiment of the present invention.

3 is an exemplary diagram of a matrix in which a voice signal is converted according to an embodiment of the present invention.

4 is a flowchart of a model vector generation method according to an embodiment of the present invention.

5 is a diagram illustrating examples of model speech signals for various emotion models.

6 shows an example of an input matrix generated using a model speech signal.

7A-7C are illustrations of model vectors obtained in accordance with one embodiment of the present invention.

<Description of Drawing>

100: emotion recognition device

110: input unit 120: conversion unit

130: NMF execution unit 140: comparison unit

150: judgment unit 160: emotion model database

500N, 500A, 500H, 500S: composite input matrix

510-1, 510-2, 510-3, 510-n: default input matrix

Claims (14)

An input unit configured to receive a target voice signal; A converter for converting the input voice signal into an input matrix using a spectrogram; An NMF performer for calculating a feature vector by performing non-negative matrix factorization (NMF) on the input matrix; An emotion model database that stores model vectors for each emotion model; A comparison unit comparing the feature vector with a model vector stored in the emotion model database; And And a determination unit for determining an emotion state of the target voice signal using an emotion model corresponding to a model vector most similar to the feature vector as a result of the comparison in the comparison unit. The input unit receives one or more model voice signals, The transform unit generates a basic input matrix by using the spectrogram of the model speech signal, and generates a composite input matrix by horizontally pasting the basic input matrix for each emotion model. And the NMF performing unit calculates the model vector by performing non-negative matrix factorization on the synthesized input matrix and stores the model vector in the emotion model database. The method of claim 1, And the comparing unit performs a comparison by using a square of an Euclidean distance between the feature vector and the model vector. The method of claim 1, The conversion unit sets a frequency axis and a time axis as rows and columns of the input matrix with respect to a spectrogram of the target voice signal, and sets an amplitude value of the target voice signal according to a change in frequency and time as an element. Emotion recognition device. The method of claim 3, Wherein each element of the input matrix is non-negative. The method of claim 1, And the NMF performing unit selects a basis matrix representing a frequency characteristic of the target speech signal as the feature vector among the results of performing the non-negative matrix factorization of the input matrix. delete The method of claim 1, And the model voice signal includes the target voice signal in which an emotional state is determined by the determination unit. Receiving a target voice signal (a); (B) converting the received target voice signal into an input matrix using a spectrogram; (C) calculating a feature vector by performing non-negative matrix factorization (NMF) on the input matrix; (D) comparing the feature vector with a model vector stored in an emotion model database; And (E) determining an emotional state of the target speech signal using an emotion model corresponding to a model vector most similar to the feature vector as a result of the comparison, Receiving at least one model voice signal; Generating a basic input matrix using a spectrogram of the model speech signal; Generating a composite input matrix by horizontally pasting the basic input matrix for each emotion model; Calculating a model vector by performing non-negative matrix factorization on the composite input matrix; And And storing in the emotion model database is performed prior to step (a). The method of claim 8, The step (d) is a emotion recognition method characterized in that the comparison using the square of the Euclidean distance (Euclidean distance) between the feature vector and the model vector. The method of claim 8, In the step (b), the frequency axis and the time axis are set as rows and columns of the input matrix with respect to the spectrogram of the target voice signal, and the amplitude value of the target voice signal according to the change of frequency and time is an element. Emotion recognition method, characterized in that. The method of claim 10, Wherein each element of the input matrix is non-negative. The method of claim 8, Wherein the step (c) selects a basis matrix representing a frequency characteristic of the target speech signal as the feature vector among the results of performing the non-negative matrix factorization of the input matrix. delete A recording medium having recorded thereon a program of instructions which can be executed in a computer device for carrying out the emotion recognition method according to any one of claims 8 to 12, wherein the program can be read by the computer device.

Images