JP2002239256A - Emotion determination device in automatic response toy and automatic response toy - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide an emotion determination device in an automatic response toy capable of determining emotions of the automatic response toy on the basis of stimulation from the outside. SOLUTION: An emotion determination device in an automatic response toy has a sound input means has an image input means, a first detection means to detect elements to change an emotion on the basis of sounds inputted to the sound input means, a second detection means to detect elements to change an emotion on the basis of images inputted by the image input means, and a determination means to determine the emotion of the automatic response toy on the basis of the results of detection by the first and second detection means.

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to an emotion determination device and an automatic response toy in an automatic response toy.

[0002]

2. Description of the Related Art The present applicant has proposed an automatic response toy in the form of a doll, an emotion determining means for determining the emotion of the automatic response toy based on an external stimulus, and an automatic response toy based on an external stimulus. State determining means for determining the current interaction state; and an automatic response toy which performs an operation according to a combination of the current emotion of the automatic response toy determined by the emotion determination means and the current interaction state determined by the state determination means. We have developed an automatic response toy equipped with a response operation control means to be performed by the user.

[0003]

SUMMARY OF THE INVENTION An object of the present invention is to provide an emotion determining apparatus for an automatic response toy capable of determining the emotion of the automatic response toy based on an external stimulus.

[0004] It is an object of the present invention to provide an automatic response toy having an emotion determining means capable of determining the emotion of the automatic response toy based on an external stimulus.

[0005]

According to the present invention, there is provided an emotion determining apparatus for an automatic response toy according to the present invention, which detects an element for changing an emotion based on voice input to voice input means, image input means, and voice input means. A first detecting means, a second detecting means for detecting an element which changes an emotion based on an image inputted by the image inputting means, and an automatic response toy based on a detection result by the first and second detecting means. Determining means for determining the emotion of the subject.

An automatic response toy according to the present invention has an emotion determining means for determining an emotion of the automatic response toy based on an external stimulus, and a state determination for determining a current interaction state of the automatic response toy based on an external stimulus. Means, and a response operation control means for causing the automatic response toy to perform an operation in accordance with a combination of the current emotion of the automatic response toy determined by the emotion determination means and the current interaction state determined by the state determination means. Wherein the emotion determining means is provided by the voice input means, the image input means, the first detection means for detecting an element that changes the emotion based on the voice input to the voice input means, and the input by the image input means. Second detection means for detecting an element that changes emotion based on an image, and detection by the first and second detection means Based on the result, characterized in that it comprises a determining means for determining an emotion of the automatic response toy.

[0007] The first detecting means includes means for determining the volume of the input voice at predetermined time intervals based on the voice input to the voice input means.
There is provided a means for determining whether or not a face is detected at predetermined time intervals based on the image input by the image input means.

The first detecting means includes means for judging the volume of the input voice at predetermined time intervals based on the voice input to the voice input means, and based on the voice input to the voice input means. Means for determining whether or not it is a nodding timing at predetermined time intervals; and a second detection means detects a face at predetermined time intervals based on the image input by the image input means. It is provided with a means for judging whether or not it has been fired.

The first detecting means includes means for determining the volume of the input voice at predetermined time intervals based on the voice input to the voice input means, and the second detecting means comprises:
Means for determining whether or not a face has been detected at predetermined time intervals based on the image input by the image input means, and movement of the face at predetermined time intervals based on the image input by the image input means. There is provided means for determining the size.

The first detecting means includes means for determining the loudness of the input voice at predetermined time intervals based on the voice input to the voice input means, and based on the voice input to the voice input means. Means for determining whether or not it is a nodding timing at predetermined time intervals; and a second detection means detects a face at predetermined time intervals based on the image input by the image input means. Means for determining whether or not the face has moved, and means for determining the magnitude of the face movement at predetermined time intervals based on the image input by the image input means.

As the determining means, for example, an emotion parameter indicating the degree of happiness, an emotion parameter indicating the degree of sadness, and an emotion parameter indicating the degree of anger based on the elements detected by the first and second detection means. Are calculated, and the emotion corresponding to the largest of these emotion parameters is determined as the emotion of the automatic response toy.

[0012] More specifically, the deciding means sets a point value set in advance for each element that changes each emotion and a gain set in advance for each element that changes each emotion and for each emotion parameter. Each emotion parameter is calculated using the above.

[0013]

Embodiments of the present invention will be described below with reference to the drawings.

[1] Description of the appearance of the automatic response toy

FIGS. 1 to 3 show the appearance of an automatic response toy in the form of a doll.

This automatic response toy has the following drive mechanism (drive circuit).

(1) A mechanism for opening and closing the eyes (blinking the eyes) (2) A mechanism for moving the eyeball in the left-right direction (3) A mechanism for tilting the neck left and right (4) A mechanism for tilting the neck back and forth (5) Forward, Mechanism for reversing, turning and rotating (6) Speaker drive circuit

The automatic response toy has the following sensors.

(1) A microphone for inputting a voice of a player (2) A CCD camera for inputting an image of a player's face and the like (3) A doll is located at an edge of a running surface such as a table. Road surface sensor for detecting

[2] Description of electrical configuration of automatic response toy

FIG. 4 shows the electrical configuration of the automatic response toy.

The automatic response toy has a central processing unit 10. The central processing unit 10 includes a storage device 11.

The automatic answering toy includes, as input devices of the central processing unit 10, a voice input device 1 having a microphone 1a, an image input device 2 having a CCD camera 2a, and a danger / safety determining device 3 having a road surface sensor 3a. I have.

The automatic response toy is used as an output device of the central processing unit 10 as a drive circuit 4 of the above-described various drive mechanisms 5.
And an audio output device 6 having a speaker 6a.

[3] Description of Functions of Central Processing Unit 10

FIG. 5 shows the functions of the central processing unit 10.

The central processing unit 10 includes a voice processing unit 20, an image processing unit 30, an emotion model generation unit 40, and a state management unit 50.
And a response operation control unit 60.

The emotion model generator 40 determines whether the automatic response toy is happy, sad, or angry. The state management unit 50 determines an interaction state of the automatic response toy. The response operation control unit 60 determines and determines an operation according to a combination of the current emotion of the automatic response toy determined by the emotion model generation unit 40 and the current interaction state determined by the state management unit 50. Cause the automatic response toy to perform the action.

[4] Description of the audio processing unit 20

The voice processing section 20 includes a nodding timing detection section 21, an input voice loudness determination section 22, and an input voice presence / absence determination section 23.

FIG. 6 shows a processing procedure of the nodding timing detecting section 21.

First, the analog audio signal input from the microphone 1a is A / D-converted at a cycle of 8 msec (step 1). Every time the number of conversions reaches 5, that is, 40
An average value of the five AD conversion values is calculated for each ms cycle (step 2) (step 3). The obtained average value is set to ON /
By performing threshold processing using the OFF threshold,
A time series ON / OFF pattern for the past 1.6 seconds is generated (step 4).

Then, it is determined whether or not it is a nodding timing in the linear prediction model of the time-series ON / OFF pattern. If the nodding timing is reached, a message indicating the nodding timing is transmitted to the emotion model generating unit 40 ( Step 5). Here, the nodding timing is a timing suitable for returning a response to the word when the other party speaks the word. The iRT technology for estimating the nodding timing using a linear prediction model of a speech time series (ON / OFF pattern) has already been developed by Prof. Tomio Watanabe, Faculty of Information Engineering, Okayama Prefectural University.

That is, the nodding timing detecting section 21
Makes a determination as to whether or not it is a nodding timing every 40 msec, and transmits a message indicating that it is the nodding timing to the emotion model generation unit 40 when it is determined that the nodding timing is reached.

The input voice loudness determination unit 22 determines that the
Each time, the average value of the AD conversion values obtained in step 3 is subjected to threshold processing using three thresholds, and the loudness of the input voice is classified into four types: large, medium, small, and none. I do. Then, when the value is large, medium, or small, a determination signal indicating that fact is output. This discrimination signal is sent to the emotion model generation unit 40
Sent to

The input voice presence / absence determination unit 23 calculates the ratio of the ON pattern from the ON / OFF pattern obtained in step 4 every 1.6 seconds, and determines the ratio of the ON pattern to a predetermined value (for example, 0). .5), a determination signal indicating that there is a voice input is output if the value is larger than a predetermined value. This determination signal is sent to the state management unit 50.
Sent to

[5] Description of Image Processing Unit 30

The image processing section 30 includes a face detecting section 31, a face motion magnitude detecting section 32, and a dark detecting section 33.

The face detecting section 31 first extracts a flesh color region based on the input image data from the image input device 2.
Next, it is determined whether or not a face area exists in the input image based on the size and shape of the extracted skin color area.
Then, a message indicating the determination result is transmitted to emotion model generation unit 40 and state management unit 50. Face detection unit 3
1 is performed at predetermined time intervals.

When the face area is detected by the face detecting section 31, the face motion magnitude detecting section 32 determines the magnitude of the face motion by detecting the magnitude of the movement of the face area. I do. Then, a message indicating the determination result is transmitted to emotion model generation unit 40. The processing of detecting the magnitude of the face motion by the face motion magnitude detection unit 32 is performed at predetermined time intervals.

When detecting that the surroundings have become dark based on the input image data from the image input device 2, the dark detection unit 33 sends a message indicating this to the state management unit 50. The dark detection processing by the dark detection unit 33 is performed at predetermined time intervals.

A message from the danger / safety determination device 3 is also sent to the state management unit 50. Danger / safety judgment device 3
Sends a message indicating safety when the road surface is detected by the road surface sensor 3a to the state management unit 50.
When the road surface is no longer detected by the road surface sensor 3a, a message indicating danger is transmitted to the state management unit 50.

[6] Description of Emotion Model Generation Unit 40

Here, it is assumed that the automatic response toy has three feelings of happiness, sadness, and anger. Then, for each happiness, sadness, and anger, a parameter indicating the degree (hereinafter referred to as an emotion parameter) Level _Hap , Lev
Has el _Sad and Level _Ang .

Each emotion parameter changes according to some emotion-changing elements (stimuli given from the outside), and the largest of the three emotion parameters is the emotion at that time. However, if all three emotion parameters are the same, it is determined that the user is happy.

Elements that change the emotion include a nodding timing detected by the nodding timing detecting section 21, a loudness of the input voice determined by the loudness determining section 22, a face detected by the face detecting section 31, and the like. And the magnitude of the face motion detected by the face motion magnitude detection unit 32. The nodding timing is
It is an index of whether or not the story is bouncing, and it is interpreted that the more times the nod timing is detected, the more bouncing the story.

For example, the voice of the input voice is medium,
When the player's face is detected, since the player is talking happily from the toy's side and the player's face is visible, the emotion parameter Level _Hap indicating the degree of happiness rises, and the automatic response Toys will be happy.

On the other hand, when the voice of the input voice is low and the face of the player is not detected, the degree of sadness is utterly spoken from the toy side and the face of the player is not visible. The emotional parameter Level _Sad increases, and the automatic answering toy becomes sad. When the input voice is loud, since the user is yelling when viewed from the toy side, the emotion parameter Level _Ang indicating the degree of anger increases, and the automatic response toy is in an angry state.

FIG. 7 shows a processing procedure of the emotion model generation unit 40.

The emotion model generation unit 40 generates an emotion parameter indicating the degree of happiness based on the element that changes the emotion.
Level _Hap , the emotion parameter Lev that indicates the degree of sadness
emotion parameter Level representing the el _{Sa d} and the degree of anger
Control _Ang .

First, each emotion parameter Level _Hap , Lev
Initialize el _Sad and Level _Ang (step 11).
Each emotion parameter Level _Hap , Level _Sad , Level _Ang
Takes a value between 0 and 1000. In the initial setting of step 11, each emotion parameter Level _Hap , Level _Sad ,
Level _Ang is set to 500 (neutral value).

Next, it waits for a message from the nodding timing detecting section 21, the input voice loudness judging section 22, the face detecting section 31, or the face motion loudness detecting section 32 (step 12).

When a message is received from any of these units, the received signal is analyzed (step 13), and each emotion parameter is calculated (step 1).
4).

The emotion corresponding to the largest emotion parameter among the calculated emotion parameters is transmitted to the response operation control unit 60 as the current emotion of the automatic response toy (step 15). However, if all the emotion parameters are equal, happiness is transmitted to the response operation control unit 60 as the current emotion of the automatic response toy. Thereafter, the process returns to step S12.
Each emotion parameter is calculated based on the following equation (1).

[0055]

(Equation 1)

Here, Gain _All is the overall gain,
For example, 1. Point _Local is a point predetermined for each element that changes emotion. Gain _Hap , Gain
_Sad and Gain _Ang are gains predetermined for each element that changes emotion.

Point _Local and Gain _Hap , Gain _Sad ,
Gain _Ang is set, for example, as shown in Table 1. Point _Local is empirically determined based on the frequency of appearance of elements that change emotion, the importance of elements in emotion change, and the like. Gain _Hap , Gain _Sad , and Gain _Ang are positive values for the element that increases the corresponding emotion parameter, and negative values for the other elements.

[0058]

[Table 1]

For example, each emotion parameter Level _Hap ,
When Level _Sad and Level _Ang are 500 and a message indicating that a face is detected from the face detection unit 31 is received, Point _Local = 5, Gain _Hap = + 1,
Since Gain _Sad = -1 and Gain _Ang = -1, Point
_Ctl = 5, Level _Hap = 505, Level _Sad = 495,
Level _Ang = 495. As a result, the emotion parameter Level _Hap indicating the degree of happiness increases, and the emotion parameters Level _Sad and Level indicating the degree of sadness and anger.
_Ang falls.

When the state in which no message is received continues, each emotion parameter is set to a neutral value (50) based on the idea that each emotion should calm down.
It is preferable to control so as to approach 0).

In the above-described embodiment, the nodding timing detected by the nodding timing detecting section 21, the loudness of the input voice determined by the loudness determining section 22, and the face detecting section 31
There is a face detected by the method and a magnitude of the face motion detected by the face motion magnitude detection unit 32. Any combination of these may be used as an element that changes the emotion.

[7] Description of State Management Unit 50

The state management section 50 displays a message from the input voice presence / absence determination section 23, a message from the face detection section 31, a message from the dark detection section 33, and a message from the danger / safety determination device 3 as an event. The interaction state of the automatic response toy is determined by transiting the interaction state according to the state transition table of FIG. Then, the current interaction state of the determined automatic response toy is transmitted to the response operation control unit 60.

[0064]

[Table 2]

In this example, there are five types of interaction states that the automatic answering toy can take, namely, "conversation state", "conversation interruption state", "autonomous action state", "sleep state", and "emergency state". .

The event includes an event indicating that there is an input voice detected by the input voice presence / absence determining unit 23, an event indicating that there is no input voice detected by the input voice presence / absence determining unit 23, and a face detection unit 31. An event indicating that the detected speaker is absent (no face has been detected), an event indicating that darkness is detected by the dark detection unit 33, and a danger detected by the danger / safety determination device 3. And an event indicating that the safety is determined by the danger / safety determination device 3.

For example, when the event indicating that there is no input voice occurs while the interaction state is the “conversation state”, the state transits to the “conversation interruption state”. In this state, when an event indicating absence of a speaker occurs, the state transits to the “autonomous action state”.

If the event indicating danger occurs when the interaction state is "autonomous action state", the state transits to "emergency state". In this state, when an event indicating safety is generated, the state changes to the state of the transition source, that is, the “autonomous action state”.

When the event indicating darkness occurs in the case where the interaction state is the “conversation suspended state” or the “autonomous action state”, the state transits to the “sleep state”. In this state, when an event indicating the presence of the input voice occurs, the state transits to the “conversation state”.

The event includes at least an event indicating that there is an input voice detected by the input voice presence / absence determining unit 23, an event indicating that there is no input voice detected by the input voice presence / absence determining unit 23, and the face detection unit 31. It is sufficient if there is an event indicating that the speaker is absent, which is detected by.

[8] Description of Response Operation Control Unit 60

The response operation control unit 60 determines the current emotion of the automatic response toy determined by the emotion model generation unit 40,
The operation according to the combination with the current interaction state determined by the state management unit 50 is determined based on the response operation table in FIG. 8, and the determined operation is performed by the automatic response toy. Although not shown in FIG. 8, actually, the squeal according to the emotions, that is, a happy scream if happy, a sloppy call if sad, and a scream if angry. An angry call is output from the audio output device 6.

In the above embodiment, the response operation control unit 60
Determines the operation of the automatic response toy according to a combination of the current emotion of the automatic response toy determined by the emotion model generation unit 40 and the current interaction state determined by the state management unit 50. Alternatively, the operation of the automatic response toy may be determined in consideration of the characteristics of the automatic response toy.

That is, a personality setting unit for setting the personality of the automatic response toy is provided. The player sets the character of the automatic answering toy to his liking. Types of personality include, for example, "scared", "tempered", "gentle", "
For example, "active", "calm", etc. are used, and a response operation table as shown in Fig. 8 is created for each character, and an automatic response is performed using the response operation table for the set character. Determine the behavior of the toy.

For example, if the personality is set to "temporarily", and if the state transitions to "autonomous action state" in the absence of a speaker, if the feeling of "anger" rises, it will be "violently violent". Cause rough movements.

Further, if the character is set to “gentle”, when the state changes to the “autonomous action state” in the absence of the speaker, when the emotion of “anger” is raised, “screaming” etc. The emotional expression of the anger hidden inside is caused to appear.

[0077]

According to the present invention, the emotion of the automatic response toy can be determined based on the stimulus from the outside.

[Brief description of the drawings]

FIG. 1 is a front view of an automatic response toy.

FIG. 2 is a side view of the automatic response toy.

FIG. 3 is a rear view of the automatic response toy.

FIG. 4 is a block diagram showing an electrical configuration of the automatic response toy.

FIG. 5 is a functional block diagram showing functions of the central processing unit 10.

FIG. 6 is a flowchart showing a processing procedure of a nodding timing detecting section 21;

FIG. 7 is a flowchart showing a processing procedure of an emotion model generation unit 40.

FIG. 8 is a schematic diagram showing a response operation table.

[Explanation of symbols]

 Reference Signs List 1a microphone 1 voice input device 2a CCD camera 2 image input device 3a road surface sensor 3a 3 danger / safety determination device 4 drive circuit 5 various drive mechanisms 6a speaker 6 voice output device 10 central processing unit 11 storage device 20 voice processing unit 30 image processing Unit 40 emotion model generation unit 50 state management unit 60 response operation control unit

Claims (14)

[Claims] An input unit configured to detect an element that changes an emotion based on a voice input to the voice input unit, based on an image input by the image input unit; , A second detection means for detecting an element that changes the emotion, and a detection result by the first and second detection means,
An emotion determination device for an automatic response toy, comprising a determination means for determining an emotion of the automatic response toy. 2. The method according to claim 1, wherein the first detecting unit includes a unit that determines a loudness of the input voice at predetermined time intervals based on the voice input to the voice input unit. , Based on the image input by the image input means,
The emotion determination device for an automatic response toy according to claim 1, further comprising means for determining whether a face is detected at predetermined time intervals. 3. The first detecting means includes means for judging the volume of the input voice at predetermined time intervals based on the voice input to the voice input means, and based on the voice input to the voice input means. Means for determining whether or not it is a nodding timing at predetermined time intervals. The second detecting means detects a face at predetermined time intervals based on the image input by the image input means. The emotion determination device for an automatic response toy according to claim 1, further comprising means for determining whether or not the emotion is detected. 4. The first detecting means includes means for determining a loudness of the input voice at predetermined time intervals based on the voice input to the voice input means, and the second detecting means includes: , Based on the image input by the image input means,
Means for determining whether or not a face has been detected at predetermined time intervals, and means for determining the magnitude of face movement at predetermined time intervals based on an image input by the image input means. 2. The emotion determination device in the automatic response toy according to 1. 5. A first detecting means for determining a loudness of an input voice at predetermined time intervals based on a voice input to the voice input means and a voice based on the voice input to the voice input means. Means for determining whether or not it is a nodding timing at predetermined time intervals. The second detecting means detects a face at predetermined time intervals based on the image input by the image input means. 2. The automatic response toy according to claim 1, further comprising: means for determining whether or not the face is detected, and means for determining the magnitude of the face movement at predetermined time intervals based on an image input by the image input means. Emotion determination device in 6. The determining means determines, based on the elements detected by the first and second detecting means, an emotion parameter representing a degree of happiness, an emotion parameter representing a degree of sadness, and an emotion parameter representing a degree of anger. The emotion in the automatic response toy according to any one of claims 1, 2, 3, 4, and 5, wherein the emotion corresponding to the largest of these emotion parameters is calculated as the emotion of the automatic response toy. Decision device. 7. The determination means uses a point value preset for each element that changes each emotion and a gain preset for each element that changes each emotion and for each emotion parameter. The emotion determining apparatus for an automatic response toy according to claim 6, wherein the emotion parameter is calculated. 8. An emotion determining means for determining an emotion of the automatic response toy based on an external stimulus, a state determining means for determining a current interaction state of the automatic response toy based on an external stimulus, and an emotion determining means Response operation control means for causing the automatic response toy to perform an operation in accordance with a combination of the current emotion of the automatic response toy determined by the current interaction state and the current interaction state determined by the state determination means. Means, voice input means, image input means, first detection means for detecting an element that changes emotion based on voice input to the voice input means, emotion detection based on an image input by the image input means. Based on a detection result by the second detection means for detecting an element that changes
An automatic response toy comprising a determination means for determining an emotion of the automatic response toy. 9. The first detecting means includes means for determining a loudness of the input voice at predetermined time intervals based on the voice input to the voice input means, and the second detecting means includes: , Based on the image input by the image input means,
9. The automatic response toy according to claim 8, further comprising: means for determining whether a face is detected at predetermined time intervals. 10. A first detecting means for determining a loudness of an input voice at predetermined time intervals based on a voice input to the voice input means and a voice based on the voice input to the voice input means. Means for determining whether or not it is a nodding timing at predetermined time intervals. The second detecting means detects a face at predetermined time intervals based on the image input by the image input means. 9. The emotion determination device for an automatic response toy according to claim 8, further comprising means for determining whether or not the emotion is detected. 11. The first detecting means includes means for determining a loudness of the input voice at predetermined time intervals based on the voice input to the voice input means, and the second detecting means includes: , Based on the image input by the image input means,
Means for determining whether or not a face has been detected at predetermined time intervals, and means for determining the magnitude of face movement at predetermined time intervals based on an image input by the image input means. An emotion determination device for an automatic response toy according to claim 8. 12. A first detecting means for determining a loudness of an input voice at predetermined time intervals based on a voice input to the voice input means, and a first detecting means for determining a loudness of the voice input to the voice input means. Means for determining whether or not it is a nodding timing at predetermined time intervals. The second detecting means detects a face at predetermined time intervals based on the image input by the image input means. 9. The automatic response toy according to claim 8, further comprising: means for determining whether or not the face has been detected, and means for determining the magnitude of the face movement at predetermined time intervals based on an image input by the image input means. . 13. A determining means for converting an emotion parameter representing a degree of happiness, an emotion parameter representing a degree of sadness, and an emotion parameter representing a degree of anger based on the elements detected by the first and second detection means. The automatic response toy according to any one of claims 8, 9, 10, 11 and 12, wherein the emotion corresponding to the largest of these emotion parameters is calculated as the emotion of the automatic response toy. 14. The determining means uses a point value set in advance for each element that changes each emotion and a gain set in advance for each element that changes each emotion and for each emotion parameter. The emotion determination device for an automatic response toy according to claim 13, wherein the emotion parameter is calculated respectively.