JP6772881B2 - Voice dialogue device - Google Patents

Description

The present invention relates to a voice dialogue device, and more particularly to a voice dialogue device that estimates emotions.

A voice dialogue device that has a conversation with a user is known. Regarding such a technique, for example, Patent Document 1 discloses an interactive processing device that responds according to a user's emotion. In this dialogue processing device, whether the user's emotion is positive, negative, or neutral is estimated based on the content of the user's utterance according to a certain criterion, and a response is performed according to the estimation result.

As described above, there is a technique of indexing a user's emotion based on a feature amount such as linguistic information, phonological information, or image information, and estimating the emotion based on a certain criterion regardless of the user.

Japanese Unexamined Patent Publication No. 2006-178603

However, even if the utterance content is the same, the emotions contained in the utterance may differ depending on the user. That is, the degree of reflection of emotions on the features used for emotion estimation may differ depending on the user. For example, some users are rich in word expressions and facial expressions, while others are not. In the case of the technique described in Patent Document 1, for users whose emotions are difficult to be reflected in the feature amount, the estimation result is often neutral, and for users whose emotions are easily reflected in the feature amount, As an estimation result, there are many positives or negatives. In this way, the variation among users becomes large, and it is not possible to perform appropriate emotion estimation.

On the other hand, if the judgment criteria for emotion estimation are set in advance for each user, the judgment criteria can be changed according to the user, and it is considered that appropriate emotion estimation according to the user becomes possible. However, in that case, the preset setting becomes complicated. For this reason, it has not been easy to appropriately estimate emotions according to the user.

The present invention has been made against the background of the above circumstances, and an object of the present invention is to provide a voice dialogue device capable of easily performing emotion estimation according to a user.

One aspect of the present invention for achieving the above object is based on a user identification unit that identifies a user who is a dialogue partner, a feature amount acquisition unit that acquires a feature amount used for emotion estimation from the user, and the feature amount. When the amount of dialogue between the emotion estimation unit that calculates the index value of the user's emotion and the user specified by the user identification unit is less than the predetermined reference implementation amount, a predetermined response guideline. A response generation unit that generates an initial response according to the above, an index value calculated by the emotion estimation unit based on the feature amount of the user's reaction to the initial response, and a predetermined reference index value. It has an emotion threshold setting unit that sets a threshold unique to the user for estimating emotions according to the comparison result, and the emotion estimation unit is set by the calculated index value and the emotion threshold setting unit. It is a voice dialogue device that estimates the emotion of the user according to the comparison result with the threshold value.
According to this voice dialogue device, when the dialogue execution amount is less than a predetermined standard implementation amount, communication is performed according to a predetermined initial response, and the communication is performed in advance as an index value of the user's emotion at that time. A threshold value unique to this user is set according to the comparison result with the reference index value. Therefore, it is possible to easily set the threshold value according to the degree of reflection of emotions on the feature amount. Therefore, according to this voice dialogue device, emotion estimation according to the user can be easily performed.

According to the present invention, it is possible to provide a voice dialogue device capable of easily performing emotion estimation according to a user.

It is a figure which shows the hardware configuration of the voice dialogue apparatus which concerns on Embodiment 1. FIG. It is a block diagram which shows the structure of the control device of the voice dialogue device which concerns on Embodiment 1. FIG. It is a flowchart which shows an example of the operation of the voice dialogue apparatus which concerns on Embodiment 1. FIG.

Hereinafter, embodiments of the present invention will be described with reference to the drawings. In each drawing, the same elements are designated by the same reference numerals, and duplicate explanations are omitted as necessary.

FIG. 1 is a diagram showing a hardware configuration of the voice dialogue device 1 according to the first embodiment. The voice dialogue device 1 has a dialogue with the user using voice. Specifically, the voice dialogue device 1 interacts with the user by outputting voice to the user in response to an utterance from the user. That is, the voice dialogue device 1 recognizes the spoken voice of the user who is the conversation partner, and outputs the voice to this user. The voice dialogue device 1 can be mounted on, for example, robots such as life support robots and small robots, cloud systems, smartphones, and the like.

The voice dialogue device 1 includes a microphone 2 that collects surrounding sounds, a camera 3 that acquires images of the surroundings, a speaker 4 that emits sounds, and a control device 10. The control device 10 has a function as, for example, a computer. The control device 10 is connected to the microphone 2, the camera 3, and the speaker 4 by wire or wirelessly.

The control device 10 has a CPU (Central Processing Unit) 12, a ROM (Read Only Memory) 14, and a RAM (Random Access Memory) 16 as a main hardware configuration. The CPU 12 has a function as an arithmetic unit that performs control processing, arithmetic processing, and the like. The ROM 14 has a function for storing a control program, an arithmetic program, and the like executed by the CPU 12. The RAM 16 has a function for temporarily storing processing data and the like.

The control device 10 analyzes the user's utterance collected by the microphone 2 and generates a response to the user in response to the user's utterance. Then, the control device 10 outputs a voice (response voice) corresponding to the generated response via the speaker 4.

FIG. 2 is a block diagram showing a configuration of a control device 10 of the voice dialogue device 1 according to the first embodiment. The control device 10 includes a user identification unit 101, a feature amount acquisition unit 102, a user database 103, an initial dialogue determination unit 104, an emotion estimation unit 105, a response generation unit 106, an initial dialogue scenario database 107, and emotions. It has an ease of output determination unit 108, an emotion threshold setting unit 109, and a voice synthesis unit 110. The user identification unit 101, the feature amount acquisition unit 102, the initial dialogue determination unit 104, the emotion estimation unit 105, the response generation unit 106, the emotion easiness determination unit 108, the emotion threshold setting unit 109, and the voice shown in FIG. The synthesis unit 110 can be realized, for example, by the CPU 12 executing the program stored in the ROM 14. Further, the necessary program may be recorded on an arbitrary non-volatile recording medium and installed if necessary. Further, the user database 103 and the initial dialogue scenario database 107 are realized by a storage device such as a ROM 14. It should be noted that each component is not limited to being realized by software as described above, and may be realized by some hardware such as a circuit element.

The user identification unit 101 identifies a user who is a conversation partner. In the present embodiment, the user identification unit 101 identifies the user by using the image information acquired from the camera 3. More specifically, the dialogue partner is specified by performing face recognition processing using the face image information for each user stored in advance in the user database 103 and the image information acquired from the camera 3. In the present embodiment, the user identification unit 101 identifies the user ID associated with the face image information in the user database 103.

The feature amount acquisition unit 102 acquires the feature amount used for emotion estimation from the user who is the dialogue partner. In the present embodiment, the feature amount acquisition unit 102 generates text data as a feature amount by performing voice recognition processing on the user's voice data acquired by the microphone 2. Therefore, the feature amount acquisition unit 102 may be referred to as a voice recognition unit. In the present embodiment, the feature amount acquired by the feature amount acquisition unit 102, that is, the text data is used not only for emotion estimation but also for response generation.

The user database 103 stores face image information and dialogue history for each user. The user database 103 stores, for example, these data and the user ID in association with each other. The dialogue history includes information indicating the amount of dialogue performed, such as the number of dialogues or the dialogue time. That is, the dialogue execution amount is a value indicating for each user how much the dialogue between the voice dialogue device 1 and the user has been performed so far. Further, the user database 103 stores a reference index value described later.

The initial dialogue determination unit 104 determines whether or not the amount of dialogue with the user specified by the user identification unit 101 is less than a predetermined reference implementation amount. That is, the initial dialogue determination unit 104 determines whether or not the dialogue with the user specified by the user identification unit 101 is in the initial stage. Specifically, in the present embodiment, the initial dialogue determination unit 104 refers to the user database 103, and the number of dialogues associated with the user ID specified by the user identification unit 101 is less than a predetermined number of times. It is determined whether or not it is.

The emotion estimation unit 105 calculates an index value of the user's emotion specified by the user identification unit 101 based on the feature amount acquired by the feature amount acquisition unit 102. Specifically, the emotion estimation unit 105 analyzes the text data generated by the feature amount acquisition unit 102, and calculates an index value indicating the user's emotion according to a predetermined calculation rule. Further, the emotion estimation unit 105 estimates the emotion of the dialogue partner who is the user specified by the user identification unit 101 according to the comparison result between the calculated index value and the threshold value. Here, the threshold value is a value set by the emotion threshold value setting unit 109 described later, and is a value corresponding to each user.

The emotion estimation unit 105 may estimate the emotion based on the calculation of the emotion index value and the comparison between the index value and the threshold value, and any known method can be applied as such an emotion estimation method. is there. For example, as one of the emotion estimation methods, the technique described in "Emotion estimation based on the emotion-causing factor corpus acquired from the Web" (Ryoko Tokukura et al., Proceedings of the 14th Annual Meeting of the Natural Language Processing Society, March 2008). May be used.

In the present embodiment, the emotion estimation unit 105 calculates a numerical value in the range of −1.0 to +1.0 as an index value from the analysis result of the text data indicating the utterance content of the user. Here, when the analysis result shows a negative emotion, the index value becomes a negative value, and when the analysis result shows a positive emotion, the index value becomes a positive value.

The emotion estimation unit 105 estimates the emotion of the user who is the dialogue partner by using the threshold value set by the emotion threshold setting unit 109 after the communication by the initial response described later is completed. In the present embodiment, specifically, the emotion estimation unit 105 determines one of positive, negative, and neutral emotions by using the calculated index value and the threshold value. Neutral is an emotion that is neither positive nor negative. For example, the threshold value for estimating positive emotions is +0.5, and the threshold value for estimating negative emotions is -0.5. In this case, the emotion estimation unit 105 determines that the user's emotion is negative when the index value calculated based on the feature amount acquired by the feature amount acquisition unit 102 is −0.5 or less. Further, the emotion estimation unit 105 determines that the user's emotion is positive when the index value calculated based on the feature amount acquired by the feature amount acquisition unit 102 is +0.5 or more. Then, the emotion estimation unit 105 determines that the user's emotion is neutral when the index value calculated based on the feature amount acquired by the feature amount acquisition unit 102 is greater than −0.5 and less than +0.5. To do.

The response generation unit 106 generates a response to the utterance of the user who is the conversation partner. The response is typically textual data. Here, in particular, when the dialogue execution amount with the user specified by the user identification unit 101 is less than the predetermined reference implementation amount, the response generation unit 106 responds according to the predetermined response guideline. Generate an initial response that is. That is, when the initial dialogue determination unit 104 determines that the dialogue is in the initial stage, it generates an initial response according to a predetermined response guideline. Specifically, the response to the user's utterance is determined according to the scenario stored in advance in the initial dialogue scenario database 107.

The initial dialogue scenario database 107 stores predetermined scenario data performed to determine the threshold value for estimating emotions. The scenario data includes, for example, chats or greetings regarding the weather, etc., but is not limited to these. When it is determined that the dialogue is in the initial stage, the response generation unit 106 refers to the initial dialogue scenario database 107 and generates a response by selecting a response statement from this database.

Further, the response generation unit 106 generates a normal response when the dialogue execution amount with the user specified by the user identification unit 101 is equal to or more than a predetermined reference implementation amount. The normal response is, for example, a response having a higher degree of freedom than the initial response, and can include, for example, a response other than the scenario stored in the initial dialogue scenario database 107. In the present embodiment, the response generation unit 106 generates an appropriate response according to the emotion estimated by the emotion estimation unit 105. Therefore, for example, the response generation unit 106 generates a response sentence by referring to the response sentence table including the response sentence associated with the emotion type and selecting an appropriate response sentence from the response sentence table. You may.

As described above, the response generation unit 106 generates an initial response at the initial stage of the dialogue with the user specified by the user identification unit 101, and generates a normal response after the initial stage is completed. Hereinafter, the dialogue when the response generation unit 106 is performing the initial response will be referred to as an initial dialogue. During the initial dialogue, the dialogue history of the user database 103 will be updated. Therefore, the amount of dialogue performed by the user during the initial dialogue is increased. Then, when the dialogue implementation amount becomes equal to or more than a predetermined standard implementation amount, the initial dialogue ends and the dialogue by the normal response is started.

The emotional susceptibility determination unit 108 determines the emotional susceptibility of the user specified by the user identification unit 101 based on the index value during the initial dialogue. That is, the emotional easiness determination unit 108 determines the degree of reflection of emotions on the feature amount used for emotion estimation for each user. Specifically, the emotion easiness determination unit 108 compares the index value calculated by the emotion estimation unit 105 based on the feature amount of the user's reaction to the initial response with a predetermined reference index value. This determines how easily the user feels. Here, the reference index value is, for example, the average of the index values when a plurality of people have an initial dialogue. In this case, the emotional susceptibility determination unit 108 determines the emotional susceptibility of the determination target user by comparing the index value of the determination target user with the average index value of the user. .. For example, it is assumed that the average value of the index values during the initial dialogue of the user to be determined (hereinafter referred to as the average value to be determined) is 0.75. Further, it is assumed that the average value of a plurality of users (hereinafter referred to as a reference average value) of the average value of the index values during the initial dialogue calculated in advance is 0.52. For example, the emotional easiness determination unit 108 subtracts the absolute value of the reference average value from the absolute value of the determination target average value, and determines the emotional easiness in three stages according to the determination value obtained as the subtraction result. To do. For example, the ease of emotional determination unit 108 determines that emotions are likely to occur when the determination value is larger than +0,2, and determines that emotions are less likely to occur when the determination value is smaller than −0,2. If the judgment value is -0.2 or more and +0.2 or less, it is judged that neither is the case. In the case of the above example, since the determination value is 0.75-0.52 = 0.23> 0.2, it is determined that the user to be determined is likely to express emotions.

The emotion threshold setting unit 109 sets a user-specific threshold value for estimating emotions according to the determination result by the emotional ease determination unit 108. When the initial dialogue is completed, the emotion threshold setting unit 109 sets a threshold value to be used when the emotion estimation unit 105 estimates the user's emotion according to the determination result by the emotion easiness determination unit 108. For example, when the emotional ease determination unit 108 determines that emotions are difficult to express, the emotional threshold setting unit 109 is looser than a predetermined basic threshold value so that positive or negative emotions can be easily estimated. Set the threshold. Further, when the emotional ease determination unit 108 determines that emotions are likely to be expressed, the emotional threshold setting unit 109 sets a threshold value tighter than the basic threshold value so that positive or negative emotions are less likely to be estimated. .. Therefore, for example, when the emotional ease determination unit 108 determines that emotions are difficult to produce, the emotional threshold setting unit 109 sets the threshold value for presuming positive emotions to +0.5, which is the basic threshold value. The threshold value for estimating negative emotions is increased by 0.2 from the basic threshold value of -0.5 to be -0.3.

The voice synthesis unit 110 converts the response generated by the response generation unit 106 into voice data. That is, the voice synthesis unit 110 converts the text data of the response sentence generated by the response generation unit 106 into voice data. The generation of speech data from text data can be realized by various known speech synthesis techniques and the like. After that, typically, a D / A converter (not shown) converts the audio data into an analog audio signal, and the speaker 4 outputs the analog audio signal as audio.

Next, the operation of the voice dialogue device 1 will be described. FIG. 3 is a flowchart showing an example of the operation of the voice dialogue device 1. Hereinafter, an operation example of the voice dialogue device 1 will be described with reference to FIG.

In step 100 (S100), the user identification unit 101 identifies a user who is a dialogue partner.
Next, in step 101 (S101), the feature amount acquisition unit 102 performs voice recognition processing on the user's voice data acquired by the microphone 2 to generate text data. For example, in step 101, text data "It's sunny today" is generated from the user's utterance.

Next, in step 102 (S102), the initial dialogue determination unit 104 determines whether or not the dialogue with the user is an initial dialogue. That is, the initial dialogue determination unit 104 determines whether or not the dialogue execution amount with the user specified in step 100 is less than the predetermined reference implementation amount. When the dialogue execution amount with the user specified in step 100 is less than the predetermined reference implementation amount, the initial dialogue determination unit 104 determines the implementation of the initial dialogue, and the process proceeds to step 103. On the other hand, when the dialogue execution amount with the user specified in step 100 is equal to or more than the predetermined reference implementation amount, the initial dialogue determination unit 104 determines the execution of the dialogue by the normal response, and the process is the step. Move to 106.

In step 103 (S103), the emotion estimation unit 105 calculates an index value of the user's emotion specified in step 100.
Next, in step 104 (S104), the response generation unit 106 generates an initial response. That is, the response generation unit 106 selects a response statement from the initial dialogue scenario database 107. For example, in step 104, "Hello, becomes you feel when you are clear?" Is generated as an initial response.

After step 104, in step 105 (S105), the voice synthesis unit 110 converts the text data of the initial response into voice data. As a result, the voice of the initial response is output from the speaker 4. After step 105, the process returns to step 101. That is, the voice recognition process is performed on the utterance from the user who received the initial response of the voice dialogue device 1. The transition from step 102 to step 103 is repeated until the amount of dialogue with the user reaches the reference amount. That is, the initial response is repeated.

When the amount of dialogue with the user reaches the reference amount, the process shifts from step 102 to step 106 as described above.
In step 106 (S106), it is determined whether or not the threshold value for the user specified in step 100 is set. If the threshold is not set, the process proceeds to step 107. When the threshold value is set, the process skips step 107 and step 108 for setting the threshold value and proceeds to step 109. That is, immediately after the end of the initial conversation, the step for setting the threshold value is executed, and thereafter, the threshold value setting process is skipped.

In step 107 (S107), the emotional susceptibility determination unit 108 determines the emotional susceptibility of the user specified in step 100 based on the index value calculated in step 103 during the initial dialogue.
Next, in step 108 (S108), the emotion threshold setting unit 109 sets a threshold value specific to the user specified in step 100 according to the determination result in step 107. After step 108, the process proceeds to step 109.

In step 109 (S109), the emotion estimation unit 105 calculates an index value of the user's emotion based on the text data obtained in step 101, and compares the calculated index value with the threshold value set in step 108. Estimate the user's emotions accordingly. For example, based on the text data "When it is sunny, the laundry dries well and feels good" obtained in step 101, in step 109, a threshold value is used in consideration of the user's emotional susceptibility. It is presumed to be positive as an emotion.
Next, in step 110 (S110), the response generation unit 106 generates a normal response corresponding to the emotion estimated in step 109. For example, if the emotion estimated in step 109 is positive, in step 110 a response "it is like" is generated as a response to the positive utterance.

After step 110, in step 105, the voice synthesis unit 110 converts the text data of the normal response into voice data. As a result, the sound of the normal response is output from the speaker 4. After that, the dialogue based on the normal response will be repeated.

The voice dialogue device 1 according to the embodiment has been described above. As described above, the voice dialogue device 1 determines the degree of reflection of emotions on the feature amount used for emotion estimation for each user, and estimates emotions with a threshold value set according to the determination result. Therefore, even if the degree of reflection of emotions on the feature amount differs for each user, it is possible to accurately capture the user's emotions. In this way, since the response can be generated after accurately capturing the emotion, the voice dialogue device 1 can communicate with the user more smoothly. Here, in particular, the voice dialogue device 1 first makes an initial response, sets a threshold value based on the index value at that time, and then shifts to a normal response. Therefore, it is possible to easily set the threshold value for each user.

The present invention is not limited to the above embodiment, and can be appropriately modified without departing from the spirit. For example, in the above embodiment, the user's utterance content is used as the feature amount, but the present invention is not limited to this, and the index value is calculated and the emotional value is calculated based on other feature amounts such as the user's facial expression and prosody. Estimates may be made. That is, as another example, the feature amount acquisition unit 102 may perform image processing on the image acquired by the camera 3 to acquire the user's facial expression as the feature amount.
Further, in the above-described embodiment, the ease of expressing emotions is determined in three stages, but this is an example and may be determined in two stages or four or more stages. Further, each value related to the index value and the threshold value is an example and is not limited to the above values.

1 Voice dialogue device 101 User identification unit 102 Feature amount acquisition unit 103 User database 104 Initial dialogue judgment unit 105 Emotion estimation unit 106 Response generation unit 107 Initial dialogue scenario database 108 Emotion easiness judgment unit 109 Emotion threshold setting unit 110 Speech synthesis Department

Claims (1)

A user identification unit that identifies the user with whom you interact
A feature amount acquisition unit that acquires a feature amount used for emotion estimation from the user,
An emotion estimation unit that calculates an index value of the user's emotion based on the feature amount,
A response generation unit that generates an initial response according to a predetermined response guideline when the amount of dialogue with a user specified by the user identification unit is less than a predetermined reference implementation amount.
The above-mentioned for estimating an emotion according to a comparison result between the index value calculated by the emotion estimation unit based on the feature amount of the user's reaction to the initial response and a predetermined reference index value. An emotion threshold setting unit that sets a user-specific threshold,
Have,
The emotion estimation unit is a voice dialogue device that estimates the user's emotions according to a comparison result between the calculated index value and the threshold value set by the emotion threshold setting unit.

Images