KR20200092166A - Server, method and computer program for recognizing emotion - Google Patents

Abstract

The present invention relates to a server, a method, and a computer program for recognizing the emotion of a user based on speech by the user while the user is performing an interaction through speech or a video. The server for recognizing the emotion of a user comprises: a receiving part for receiving input data including speech information and utterance moment information and user identification information from a user terminal; an attribute information extraction part for extracting user attribute information based on the received user identification information; a selection part for selecting at least one emotion recognition model among a plurality of pre-registered emotion recognition models based on the speech information, the utterance moment information, and the user attribute information; and an emotion recognition part that recognizes the emotion of the user by analyzing the speech information by the selected emotion recognition model.

Description

Server, method and computer program for recognizing emotions{SERVER, METHOD AND COMPUTER PROGRAM FOR RECOGNIZING EMOTION}

The present invention relates to a server, method and computer program for recognizing emotions.

Based on the artificial intelligence engine and speech recognition, intelligent personal assistants that collect and provide customized information to users and perform various functions according to voice commands are being provided through various devices.

The intelligent personal assistant may not only perform the user's voice command, but also analyze the user's emotion by recognizing the user's voice, and recommend appropriate content to the user based on the analyzed user's emotion.

With regard to technology for recognizing a user's emotion in an intelligent personal assistant, Korean Patent Registration No. 10-1398218, which is a prior art, discloses an apparatus and method for recognizing emotion speech.

Conventional techniques for recognizing a user's emotion were able to recognize the user's emotion based on the volume, height, and speech rate of the voice from the user's voice data. However, since the voice may be expressed in a calm voice according to the user's personality, when the user's emotion is recognized using only the voice data, there is a disadvantage in that the accuracy is somewhat lower.

An object of the present invention is to provide a server, method, and program for recognizing a user's emotion based on a voice spoken by the user while the user performs an interaction such as voice or video.

In addition to the voice spoken by the user, it is intended to provide a server, method, and program for recognizing the user's emotions based on complex situation information including video of a user's face, conversation content, user information, and past service usage history. .

It is intended to provide a server, method, and program for recognizing a user's emotion that enables a counterpart who is interacting with the user to recognize the user's emotion in advance, thereby responding to the recognized user's emotion in advance.

However, the technical problems to be achieved by the present embodiment are not limited to the technical problems as described above, and other technical problems may exist.

As a means for achieving the above-described technical problem, an embodiment of the present invention, a receiving unit for receiving input information and identification information of the user including voice information and instantaneous speech information from the user terminal, the identification information of the received user An attribute information extraction unit for extracting user attribute information based on the selection unit for selecting at least one emotion recognition model from among a plurality of emotion recognition models registered based on the voice information, the spoken moment information, and the user attribute information And an emotion recognition unit that recognizes the user's emotion by analyzing the voice information by the selected emotion recognition model.

According to another embodiment of the present invention, receiving input data including voice information and instantaneous information from a user terminal and identification information of a user, extracting user attribute information based on the received identification information of the user, Selecting at least one emotion recognition model from among a plurality of pre-registered emotion recognition models based on the voice information, the spoken moment information, and the user attribute information, and analyzing the voice information by the selected emotion recognition model to obtain the It can provide an emotion recognition method comprising the step of recognizing the user's emotion.

According to another embodiment of the present invention, when the computer program is executed by a computing device, input data including voice information and instantaneous speech information from a user terminal and user identification information are received, and the received user identification information Based on the extracted user attribute information, and based on the voice information, the utterance information and the user attribute information, select at least one emotion recognition model among a plurality of registered emotion recognition models, and the selected emotion recognition model By analyzing the voice information, it is possible to provide a computer program stored in a medium including a sequence of instructions to recognize the user's emotion.

The above-described problem solving means are merely exemplary and should not be construed as limiting the present invention. In addition to the exemplary embodiments described above, there may be additional embodiments described in the drawings and detailed description of the invention.

According to any one of the above-described problem solving means of the present invention, a user provides a server, a method and a program for recognizing a user's emotion based on a voice spoken by a user while performing an interaction such as voice or video. can do.

In addition to the voice uttered by the user, a server, method, and program for recognizing the user's emotions based on complex situation information including a video of the user's face, conversation content, user information, and past service usage history can be provided. have.

It is possible to provide a server, a method and a program for recognizing a user's emotion so that a user who is interacting with the user recognizes the user's emotion, so that the user can respond in advance to the recognized user's emotion.

1 is a block diagram of an emotion recognition system according to an embodiment of the present invention.
2 is a block diagram of an emotion recognition server according to an embodiment of the present invention.
3A and 3B are exemplary views illustrating a process of recognizing a user's emotion according to an embodiment of the present invention.
4 is a flowchart of a method for recognizing emotion in an emotion recognition server according to an embodiment of the present invention.

Hereinafter, exemplary embodiments of the present invention will be described in detail with reference to the accompanying drawings so that those skilled in the art to which the present invention pertains may easily practice. However, the present invention can be implemented in many different forms and is not limited to the embodiments described herein. In addition, in order to clearly describe the present invention in the drawings, parts irrelevant to the description are omitted, and like reference numerals are assigned to similar parts throughout the specification.

Throughout the specification, when a part is "connected" to another part, this includes not only "directly connected" but also "electrically connected" with other elements in between. . Also, when a part is said to “include” a certain component, it means that the component may further include other components, not exclude other components, unless specifically stated otherwise. However, it should be understood that the existence or addition possibilities of numbers, steps, actions, components, parts or combinations thereof are not excluded in advance.

In the present specification, the term “unit” includes a unit realized by hardware, a unit realized by software, and a unit realized by using both. Further, one unit may be realized by using two or more hardware, and two or more units may be realized by one hardware.

Some of the operations or functions described in this specification as being performed by a terminal or device may be performed instead on a server connected to the corresponding terminal or device. Similarly, some of the operations or functions described as being performed by the server may be performed in a terminal or device connected to the corresponding server.

Hereinafter, an embodiment of the present invention will be described in detail with reference to the accompanying drawings.

1 is a block diagram of an emotion recognition system according to an embodiment of the present invention. Referring to FIG. 1, the emotion recognition system 1 may include a user terminal 110, an emotion recognition server 120, and another user terminal 130. The user terminal 110, the emotion recognition server 120 and other user terminals 130 exemplarily show components that can be controlled by the emotion recognition system 1.

Each component of the emotion recognition system 1 of FIG. 1 is generally connected through a network. For example, as illustrated in FIG. 1, the emotion recognition server 120 may be connected to the user terminal 110 or another user terminal 130 simultaneously or at a time interval.

The network means a connection structure capable of exchanging information between nodes such as terminals and servers, and a local area network (LAN), a wide area network (WAN), and the Internet (WWW: World) Wide Web), wired and wireless data communication networks, telephone networks, and wired and wireless television communication networks. Examples of wireless data communication networks include 3G, 4G, 5G, 3rd Generation Partnership Project (3GPP), Long Term Evolution (LTE), World Interoperability for Microwave Access (WIMAX), Wi-Fi, Bluetooth communication, infrared communication, ultrasound Communication, Visible Light Communication (VLC), LiFi, and the like are included, but are not limited thereto.

The user terminal 110 may perform a call or interaction such as voice/video with another user terminal 130. For example, the user terminal 110 may receive a voice uttered by the user 100 through a microphone and transmit the received voice to the emotion recognition server 120.

The user terminal 110 may transmit additional information generated during the execution of the video call to the emotion recognition server 120. Here, the additional information may be utterance information related to the moment when the user 100 utters the voice. For example, the user terminal 110 may transmit an image of the user 100 through the camera to the emotion recognition server 120. For another example, the user terminal 110 may transmit the location of the user 100 measured through GPS to the emotion recognition server 120.

The user terminal 110 may transmit the identification information of the user 100 to the emotion recognition server 120. At this time, the identification information of the user 100 may include a line number of the user terminal 110, user account information (ID/PW), and the like.

The emotion recognition server 120 may classify conversation history data collected from a plurality of user terminals according to attribute information for each user, and learn with individual learning models based on the classified attribute information for each user.

The emotion recognition server 120 may receive input data including voice information and instantaneous speech information from the user terminal 110 and identification information of the user 100. The emotion recognition server 120 may convert voice information received from the user terminal 110 into text through STT (Speech-To-Text) conversion.

The emotion recognition server 120 may extract user attribute information based on the received identification information of the user 100. Here, the user attribute information may include the user's 100 gender, age, residence area, past conversation history, past service usage history, preferred content service usage history, service usage time distribution information, and none of the user attribute information. Even if it is not given, the emotion of the user 100 may be recognized.

The emotion recognition server 120 may extract at least one of image information and location information for a moment spoken by the user 100 from the moment information spoken. For example, the emotion recognition server 120 analyzes the face of the user 100 based on the extracted image information, and at least one of a plurality of pre-registered emotion recognition models based on the analyzed face information of the user 100 One emotion recognition model can be selected. For another example, the emotion recognition server 120 identifies a location where the user 100 is located based on the extracted location information, and a plurality of emotion recognition models that are pre-registered based on the location where the identified user 100 is located. At least one emotion recognition model may be selected.

The emotion recognition server 120 may select at least one emotion recognition model from among a plurality of previously registered emotion recognition models based on voice information, instantaneous speech information, and user attribute information. Here, the plurality of pre-registered emotion recognition models may include a gender emotion recognition model, an age-specific emotion recognition model, a regional emotion recognition model, and a specific service user emotion recognition model.

The emotion recognition server 120 may recognize emotions of the user 100 by analyzing voice information according to the selected emotion recognition model. At this time, the emotion recognition server 120 may derive instantaneous emotion data (one-time emotion data) by analyzing voice information by using the emotion recognition model, and perform continuous emotion data (a series of emotion data, multiple times emotion data). It can also be derived.

The emotion recognition server 120 may transmit the emotion of the recognized user 100 to another user terminal 130.

The emotion recognition server 120 may be executed by a computer program stored in a medium including a sequence of instructions for recognizing emotion. When the computer program is executed by the computing device, the user terminal 110 receives input data including voice information and instantaneous speech information and identification information of the user 100, and user attributes based on the received identification information of the user Extracts information, selects at least one emotion recognition model from among a plurality of pre-registered emotion recognition models based on voice information, instantaneous speech information, and user attribute information, and analyzes voice information by the selected emotion recognition model so that the user ( It may include a sequence of commands to recognize the emotion of 100).

The other user terminal 130 may perform a call or interaction such as voice/video with the user terminal 110.

The other user terminal 130 may receive the recognized emotion of the user 100 from the emotion recognition server 120 while performing a voice call or a video call with the user terminal 110.

2 is a block diagram of an emotion recognition server according to an embodiment of the present invention. Referring to FIG. 2, the emotion recognition server 120 includes a learning unit 210, a receiving unit 220, an attribute information extraction unit 230, an instantaneous speech information extraction unit 240, a face analysis unit 250, and a place identification It may include a unit 260, the selection unit 270, the emotion recognition unit 280 and the transmission unit 290.

The learning unit 210 may classify conversation history data collected from a plurality of user terminals according to attribute information for each user, and learn as an individual learning model based on the classified attribute information for each user.

The learning unit 210 may learn with an individual learning model using a neural network learning technique. Here, the learning method is not limited to the neural network learning technique, and a conventional learning method based on machine learning may be used. In this case, when learning is further performed by combining user information in the learning unit 210, accuracy for emotion recognition may be improved.

The learning unit 210 may classify the collected learning data according to user information without using only one learning result, and then train each of them as an individual learning model. The individual learning model may be trained with, for example, a gender emotion recognition model, an age-specific emotion recognition model, a region-specific emotion recognition model, and a specific service user-specific emotion recognition model.

For example, the learning unit 210 may separate male and female learning data into male learning data and female learning data, and then train them as individual learning models, such as a male emotion recognition model or a female emotion recognition model. . At this time, in order to recognize the emotion of the user 100, a female emotion recognition model may be used if it is a female, and a male emotion recognition model may be used if it is a male. This has the advantage that higher accuracy can be obtained than when a single model in which a man and a woman are mixed is used.

For another example, the learning unit 210 may classify the learning data into regions according to user information, and then learn with individual learning models such as regional emotion recognition models. For example, in order to recognize the emotions of the user 100 using the dialect, national emotions are used by using three regional emotion models, such as the Gyeongsang-do regional emotion recognition model, the Seoul regional emotion recognition model, and the Jeolla-do regional emotion recognition model. Higher accuracy can be achieved than using a recognition model. At this time, in order to use the multi-emotional recognition model, additional information on the residential area of the user 100 who wants to recognize the emotion is required.

The reason for learning the regional emotion recognition model as an individual learning model is that there is a high possibility of using the speech, speech, intonation, and vocabulary in which the local color is displayed for each user 100, and thus the user 100 through the regional emotion recognition model using this information This is because, when the emotions of the) are analyzed, the accuracy is higher than that of the analysis using other emotion recognition models.

Specifically, if the user 100 is living in Seoul, the learning unit 210 may tag emotions for each section in which emotions are revealed in the voice information of the user 100 and learn them to learn with the emotion recognition model in Seoul. have. Similarly, an emotion recognition model may be trained for each region. At this time, if it is not possible to obtain information about the area of residence of the user 100, a learning model constructed for the nationals who have learned all the constructed learning data may be used. The learning model built for the national population is somewhat inaccurate than the regional emotion recognition model, but the best results can be obtained in a situation where the user's residence information is not given.

The receiver 220 may receive input data including voice information and instantaneous speech information from the user terminal 110. In the voice information, a voice portion may be extracted from the packet. The voice information and the instantaneous speech information received through the receiving unit 220 may be specified and stored in the database in the order of time input for each user by the storage unit (not shown). At this time, the voice information may be converted into conversation information composed of text through speech-to-text (STT) conversion through a conversion unit (not shown).

The storage unit (not shown) may not store the entire input data in terms of resource management, but may also store only the corresponding input data within a predetermined time. That is, input data within a specific time among all input data may be used for the emotion recognition of the user 100. In this case, instantaneous speech information including voice information, video information, and location information of the user 100 may be data collected and secured at exactly the same time point.

When the input data stored in the storage unit (not shown) needs to continuously check the emotion of the user 100, such as a voice call with a call center agent, the emotion of the user 100 at the time when the new utterance information is acquired This can be analyzed. Alternatively, when it is necessary to grasp the emotion of the talker every time the conversation is finished, such as a video call, it may be analyzed at the time when the conversation information is updated.

The storage unit (not shown) may delete data from which emotion recognition has been stopped from the database for efficient use of resources for information related to input data stored in the database. In addition, in the case of unnecessary data, the storage unit (not shown) may delete past data from a database before a certain time.

When the emotion recognition of the user 100 is completed by the emotion recognition unit 280, the storage unit (not shown) transmits all data on the user information, voice information, utterance information, and recognized emotions at the corresponding time and the user 100. You can store and manage each, and save it as data for learning a new model later. At this time, all data related to the user 100 may be subsequently corrected by the emotion recognition result by the manager, and the corrected data may be used to train the emotion recognition model. Through this, the emotion recognition function may be continuously improved or a new emotion recognition model may be learned.

The receiver 220 may receive identification information of the user 100. The identification information of the user 100 may include a line number capable of specifying the user 100, user account information, and the like.

The attribute information extraction unit 230 may extract user attribute information based on the identification information of the received user 100. Here, the user attribute information includes user profile information including the gender, age, and residence area of the user 100 and a user including past conversation history, past service usage history, preferred content service usage history, service usage time distribution information, and the like. It may be composed of information related to the service used by (100). At this time, the user attribute information is information related to the user 100, which may be pre-stored in the database, or may be extracted based on the identification information of the user 100 from the moment of speech information, but is not limited thereto. . In addition, even when no user attribute information is given, the emotion of the user 100 may be recognized using voice information and video information.

The utterance instantaneous information extraction unit 240 may extract at least one of image information and location information for the moment uttered by the user 100 from the utterance instantaneous information. Here, the video information may include all images of a face image within a few minutes, such as a video call, and some face images detected instantaneously.

The face analysis unit 250 may analyze the face of the user 100 based on the extracted image information. For example, if gender or age is not included in the user attribute information, the gender and age of the user may be derived based on the face information of the user analyzed by the face analyzer 250. In addition, the face analysis unit 250 may analyze the facial expression of the user 100 based on the facial muscles located in the eyes, mouth, and the like of the user 100.

The location identification unit 260 may identify a location where the user 100 is located based on the extracted location information. For example, the location identification unit 260 may identify the region where the user 100 is located based on the extracted location information.

The selector 270 may select at least one emotion recognition model from among a plurality of pre-registered emotion recognition models based on voice information, instantaneous speech information, and user attribute information. Here, the plurality of pre-registered emotion recognition models may include a gender emotion recognition model, an age-specific emotion recognition model, a regional emotion recognition model, and a specific service user emotion recognition model.

The selector 270 may select at least one emotion recognition model from among a plurality of pre-registered emotion recognition models based on the analyzed face of the user 100. For example, the selection unit 270 may select a female emotion recognition model from among pre-registered emotion recognition models based on the face of the user 100.

The selector 270 may select at least one emotion recognition model from among a plurality of pre-registered emotion recognition models based on the identified location of the user 100. For example, when the place where the user 100 is located is'Seoul', the selector 270 may select the emotion recognition model of the Seoul region from among a plurality of registered emotion recognition models.

For example, when the user 100 is'a man in his 20s who lives in Seoul', the selection unit 270 includes a male emotion recognition model, a Seoul regional emotion recognition model, and a 20th emotion recognition model from a plurality of previously registered models. Either one or a combination of these may be selected.

The emotion recognition unit 280 may recognize emotions of the user 100 by analyzing voice information according to the selected emotion recognition model. Here, the type of emotions to be recognized may be recognized as one of dozens of emotions, at least two. The emotion recognition unit 280 may derive one emotion by analyzing voice information, or may analyze a unit of seconds to derive a sequence of emotions close to real time. Emotions are defined by a dictionary system (not shown), and can be divided into 2 to up to dozens of types. For example, it may be differentiated from two emotions: neutral, angry emotion, to six emotions: neutral, angry, sad, surprise, fear, and pleasure.

Specifically, the emotion recognition unit 280 may derive instantaneous emotion data or continuous emotion data by analyzing voice information using the emotion recognition model. For example, when recognizing the emotion of the user 100 using only the voice information of the user 100, the emotion recognition unit 280 analyzes the voice information by the emotion recognition model and analyzes the instant emotion data such as'angry'. Can be derived. On the other hand, in the case of recognizing the emotion of the user 100 by using the video information of the face of the user 100 among the voice information and the instantaneous information of the user 100, the emotion recognition unit 280 recognizes the emotion By analyzing the voice information by the model, it is possible to derive continuous emotion data such as'differential-angry-outrage'.

The transmission unit 290 may transmit the emotion of the recognized user 100 to another user terminal 130. For example, when the emotion information of the recognized user 100 is transmitted to the other user terminal 130, the other user terminal 130 is based on the emotion of the recognized user 100 to respond to the related user 100 Can provide. Here, the result of the emotion recognition may be the emotion data at the moment of final determination, but may be transmitted to other user terminals 130 in various forms such as a probability for each emotion or a confidence pair between the emotion and the emotion.

3A and 3B are exemplary views illustrating a process of recognizing a user's emotion according to an embodiment of the present invention.

3A is an exemplary diagram for explaining a process of recognizing a user's emotion through a voice call according to an embodiment of the present invention. Referring to FIG. 3A, the user 100 may perform a voice call with the call center agent 300. Here, the other user terminal 130 may be a terminal of the call center agent 300.

The emotion recognition server 120 may receive input data including voice information and instantaneous speech information from the user terminal 110. For example, the voice information may include the voice of the user 100, such as "The object is broken." The utterance information may include the current location of the user 100.

The emotion recognition server 120 may receive user identification information from the user terminal 110. For example, the emotion recognition server 120 may receive a user account from the line number of the user terminal 110 or the user terminal 110.

The emotion recognition server 120 may extract user attribute information based on the identification information of the user 100. For example, the emotion recognition server 120 may determine that the current residence of the user 100 is'Jeonolla area' based on the user attribute information.

The emotion recognition server 120 may use a region-specific emotion recognition model among a plurality of registered emotion recognition models. At this time, the emotion recognition server 120 selects the Jeolla-do regional emotion recognition model corresponding to the user's residence from among the region-specific emotion recognition models, and analyzes voice information through the selected Jeolla-do regional emotion recognition model to derive instant emotion data to the user The emotion of (100) can be recognized as'complaint'.

The emotion recognition server 120 may transmit the emotion of the recognized user 100 to the terminal 130 of the call center agent 300.

The call center agent 300 may provide the user 100 with a response suitable for a complaint situation based on the emotion of the recognized user 100.

3B is an exemplary diagram for explaining a process of recognizing a user's emotion through a video call according to an embodiment of the present invention. Referring to FIG. 3B, the user 100 may make a video call with the other party 310. Here, the other user terminal 130 may be the terminal of the other party 310 making a video call.

The emotion recognition server 120 may receive input data including voice information and instantaneous speech information from the user terminal 110. For example, the voice information may include the voice of the user 100, such as "I was so happy today. I want to go there again." The utterance information may include the image of the face of the user 100 and the current location.

The emotion recognition server 120 may receive user identification information from the user terminal 110. For example, the emotion recognition server 120 may receive the line number of the user terminal 110.

The emotion recognition server 120 may extract user attribute information based on the identification information of the user 100. For example, the emotion recognition server 120 may determine that the gender of the user 100 is “female” based on the user attribute information.

The emotion recognition server 120 may use a gender emotion recognition model among a plurality of registered emotion recognition models. At this time, the emotion recognition server 120 selects a female emotion recognition model corresponding to the user's gender from the gender emotion recognition model, analyzes voice information and video information through the selected female learning model, and derives continuous emotion data to generate a user. The emotion of (100) can be recognized as'happy-shinnam'.

The emotion recognition server 120 may transmit the emotion of the recognized user 100 to the terminal 130 of the other party 310.

4 is a flowchart of a method for recognizing emotion in an emotion recognition server according to an embodiment of the present invention. The method for recognizing emotion in the emotion recognition server 120 illustrated in FIG. 4 includes steps that are processed in time series according to the embodiment illustrated in FIGS. 1 to 3B. Therefore, even if it is omitted below, it is also applied to a method of recognizing emotions in the emotion recognition server 120 according to the embodiment illustrated in FIGS. 1 to 3B.

In step S410, the emotion recognition server 120 may receive input data including voice information and instantaneous speech information from the user terminal 110 and identification information of the user 100.

In step S420, the emotion recognition server 120 may extract user attribute information based on the identification information of the received user 100.

In step S430, the emotion recognition server 120 may select at least one emotion recognition model from among a plurality of pre-registered emotion recognition models based on voice information, instantaneous speech information, and user attribute information.

In step S440, the emotion recognition server 120 may recognize the emotion of the user 100 by analyzing voice information according to the selected emotion recognition model.

In the above description, steps S410 to S440 may be further divided into additional steps or combined into fewer steps, according to an embodiment of the present invention. In addition, some steps may be omitted if necessary, and the order between the steps may be switched.

The method for recognizing emotions in the emotion recognition server described with reference to FIGS. 1 to 4 may also be implemented in the form of a computer program stored in a medium executed by a computer or a recording medium including instructions executable by a computer. In addition, the method for recognizing emotion in the emotion recognition server described with reference to FIGS. 1 to 4 may also be implemented in the form of a computer program stored in a medium executed by a computer.

Computer readable media can be any available media that can be accessed by a computer, and includes both volatile and nonvolatile media, removable and non-removable media. In addition, computer readable media may include computer storage media. Computer storage media includes both volatile and nonvolatile, removable and non-removable media implemented in any method or technology for storage of information such as computer readable instructions, data structures, program modules or other data.

The above description of the present invention is for illustration only, and those skilled in the art to which the present invention pertains can understand that it can be easily modified to other specific forms without changing the technical spirit or essential features of the present invention. will be. Therefore, it should be understood that the embodiments described above are illustrative in all respects and not restrictive. For example, each component described as a single type may be implemented in a distributed manner, and similarly, components described as distributed may be implemented in a combined form.

The scope of the present invention is indicated by the following claims rather than the above detailed description, and it should be interpreted that all changes or modified forms derived from the meaning and scope of the claims and equivalent concepts thereof are included in the scope of the present invention. do.

110: user terminal
120: emotion recognition server
130: another user terminal
210: learning department
220: receiver
230: attribute information extraction unit
240: instantaneous information extraction unit
250: facial expression analysis unit
260: location identification
270: selection section
280: emotion recognition unit
290: transmission unit

Claims (20)

In the emotion recognition server,
A reception unit for receiving input data including voice information and instantaneous speech information from the user terminal and identification information of the user;
An attribute information extracting unit extracting user attribute information based on the received identification information of the user;
A selection unit for selecting at least one emotion recognition model from among a plurality of emotion recognition models pre-registered on the basis of the voice information, the spoken moment information, and the user attribute information; And
And an emotion recognition unit that recognizes the user's emotion by analyzing the voice information by the selected emotion recognition model.
According to claim 1,
The user attribute information includes at least one of the user's gender, age, residence area, past conversation history, preferred content service usage history, and service usage time distribution information.
According to claim 1,
Emotion recognition server further comprises a speech instantaneous information extraction unit for extracting at least one of the video information and location information for the user's speech moment from the speech moment information.
The method of claim 3,
Further comprising a face analysis unit for analyzing the face of the user based on the extracted image information,
The selection unit selects at least one emotion recognition model from among a plurality of emotion recognition models registered in advance based on the analyzed face information of the user,
The emotion recognition unit analyzes the voice information by the selected emotion recognition model, and recognizes the user's emotion based on the analyzed voice information, emotion recognition server.
The method of claim 3,
Wherein based on the extracted location information further comprises a place identification unit for determining the location where the user is located,
The selection unit selects at least one emotion recognition model among a plurality of emotion recognition models that are pre-registered based on the location where the identified user is located,
The emotion recognition unit recognizes the user's emotion by analyzing the voice information according to the selected emotion recognition model.
According to claim 1,
The pre-registered plurality of emotion recognition models include at least one of a gender emotion recognition model, an age-specific emotion recognition model, a region-specific emotion recognition model, and a specific service user-specific emotion recognition model.
According to claim 1,
Emotion recognition server further comprises a transmission unit for transmitting the emotion of the recognized user to another user terminal.
According to claim 1,
Emotion recognition server further comprises a learning unit that classifies the conversation history data collected from a plurality of user terminals according to user-specific attribute information, and learns with an individual learning model based on the classified user-specific attribute information.
According to claim 1,
The emotion recognition unit derives instant emotion data by analyzing the voice information by the emotion recognition model.
According to claim 1,
The emotion recognition unit derives continuous emotion data by analyzing the voice information by the emotion recognition model.
In a method for recognizing emotion in the emotion recognition server,
Receiving input data and user identification information including voice information and utterance information from a user terminal;
Extracting user attribute information based on the received identification information of the user;
Selecting at least one emotion recognition model among a plurality of pre-registered emotion recognition models based on the voice information, the spoken moment information, and the user attribute information; And
And recognizing the user's emotion by analyzing the voice information by the selected emotion recognition model.
The method of claim 11,
The user attribute information includes at least one of the user's gender, age, residence area, past conversation history, preferred content service usage history, and service usage time distribution information.
The method of claim 11,
And extracting at least one of image information and location information of the user's utterance from the utterance moment information.
The method of claim 13,
Analyzing the user's face based on the extracted image information;
Selecting at least one emotion recognition model from among a plurality of registered emotion recognition models based on the analyzed face information of the user; And
And analyzing the voice information by the selected emotion recognition model and recognizing the user's emotion based on the analyzed voice information.
The method of claim 13,
Determining a location where the user is located based on the extracted location information;
Selecting at least one emotion recognition model from among a plurality of registered emotion recognition models based on the identified user's location; And
And recognizing the user's emotion by analyzing the voice information by the selected emotion recognition model.
The method of claim 11,
The pre-registered plurality of emotion recognition models include at least one of a gender emotion recognition model, an age-specific emotion recognition model, a region-specific emotion recognition model, and a specific service user-specific emotion recognition model.
The method of claim 11,
And transmitting the emotion of the recognized user to another user terminal.
The method of claim 11,
The method further includes classifying conversation history data collected from a plurality of user terminals according to attribute information for each user, and learning with an individual learning model based on the classified attribute information for each user.
The method of claim 11,
Recognizing the emotion,
Emotion recognition method, by analyzing the voice information by the emotion recognition model to derive either of the instantaneous emotion data or continuous emotion data.
A computer program stored on a computer readable medium comprising a sequence of instructions for recognizing emotion, comprising:
When the computer program is executed by a computing device,
Receiving input data and user identification information including voice information and instantaneous speech information from a user terminal,
Extract user attribute information based on the received identification information of the user,
Selecting at least one emotion recognition model from among a plurality of emotion recognition models pre-registered based on the voice information, the utterance moment information, and the user attribute information,
And a sequence of instructions for analyzing the voice information by the selected emotion recognition model to recognize the user's emotion.

Images