JP2020056935A - Controller for robot, robot, control method for robot, and program - Google Patents

Abstract

To properly control expressions of a robot according to a flow of utterance even when predetermined object utterance contents change largely.SOLUTION: A controller 30 for robot controls expressions of a robot configured to change its expressions. The controller 30 for robot comprises: a voice detection part 31 and a voice recognition part 32 which acquire a voice that a predetermined object utters; a keyword determination part 34 which analyzes the voice acquired by the voice detection part 31 and voice recognition part 32 so as to acquire a keyword and its attribute information from utterance contents of the predetermined object; a storage part 22 which stores the attribute information acquired by the keyword determination part 34; and an expression decision part 35 which decides an expression of the robot based upon relation between the attribute information acquired during utterance of the predetermined object and the past attribute information stored in the storage part 22.SELECTED DRAWING: Figure 3

Description

  The present invention relates to a robot control device, a robot, a robot control method, and a program.

  2. Description of the Related Art A control device of a robot configured to change a facial expression is known. For example, Patent Literature 1 discloses that, during a conversation, the content of the conversation is determined by analyzing a voice, and the expression of the robot is controlled based on the determined content of the conversation.

JP 2018-62042 A

  However, the conventional robot control device merely controls the expression of the robot based on the content of the dialogue determined at each time.For example, when the content of the dialogue changes significantly, the expression of the robot is appropriately adjusted. There is a problem that cannot be controlled.

  The present invention has been made in view of the above, and it is an object of the present invention to appropriately control the expression of a robot along the flow of an utterance even when the utterance content of a predetermined target such as a user greatly changes. Aim.

The robot control device according to the present invention includes:
A robot control device that controls a facial expression of a robot configured to be able to change its own facial expression,
Voice acquisition means for acquiring a voice uttered by a predetermined object;
An attribute parameter obtaining unit that obtains an attribute parameter representing an attribute of the utterance content of the predetermined target by analyzing the voice obtained by the voice obtaining unit;
Storage means for storing the attribute parameters obtained by the attribute parameter obtaining means,
Expression control means for controlling the expression of the robot, based on a relationship between the attribute parameters acquired during the utterance of the predetermined target and the past attribute parameters stored in the storage means,
Is provided.

  ADVANTAGE OF THE INVENTION According to this invention, even when the utterance content of a predetermined object changes greatly, the expression of a robot can be controlled appropriately along the flow of utterance.

FIG. 2 is a front view of the appearance of the robot according to the embodiment of the present invention. FIG. 2 is a view of the external appearance of the robot according to the embodiment of the present invention as viewed from the back. FIG. 2 is a hardware configuration diagram of the robot according to the embodiment of the present invention. FIG. 2 is a functional configuration diagram of the robot control device according to the embodiment of the present invention. 5 is a flowchart illustrating a keyword determination process according to the embodiment of the present invention. FIG. 3 is a diagram illustrating an example of a configuration of a keyword information database of the robot control device according to the embodiment of the present invention. It is a flowchart which shows the expression determination process which concerns on embodiment of this invention. 6 is a flowchart of a facial expression determination process according to the embodiment of the present invention. 6 is a flowchart of a facial expression determination process according to the embodiment of the present invention. 6 is a flowchart of a facial expression determination process according to the embodiment of the present invention. 6 is a flowchart of a facial expression determination process according to the embodiment of the present invention. FIGS. 8A and 8B are diagrams illustrating examples of facial expressions determined in the facial expression determination processing of FIGS. 6 and 7, wherein FIG. 8A is an expression of “normal”, FIG. 8B is an expression of “happy”, and FIG. (D) is a diagram showing “pupil change”. FIG. 9 is a diagram showing an example of a facial expression determined in the facial expression determination process of FIG. 8, wherein (A) is an expression of “good mood”, (B) is an expression of “impressed”, (C) is an expression of “confused”, (D) is a figure showing the expression of “expectation”. 10A and 10B are diagrams illustrating examples of facial expressions determined in the facial expression determination process of FIG. 9, wherein FIG. 9A illustrates an expression of “depressed”, FIG. 9B illustrates an expression of “relief”, FIG. 10C illustrates an expression of “sadness”, and FIG. D) is a diagram showing the expression of “anxiety”. 11A is a diagram showing an example of a facial expression determined in the facial expression determination process of FIG. 10, (A) a facial expression of “confirmation”, (B) a facial expression of “happy”, (C) a facial expression of “worried”, (FIG. D) is a diagram showing “pupil change”. It is a figure showing an example of change of the expression of a robot when talking to the robot. It is a figure showing the example which displayed the expression on the screen of the personal digital assistant.

  Hereinafter, a robot control device, a robot, a robot control method, and a program according to an embodiment of the present invention will be described in detail with reference to the drawings.

  The robot 1 according to the embodiment of the present invention is a robot that operates autonomously according to the utterance of a predetermined target such as a user. The predetermined target is a target that communicates with and exchanges with the robot 1. Specifically, a user (human) who is the owner of the robot 1, humans around the user (user's family, friends, and the like) and the like are listed.

  FIG. 1A is a diagram of the appearance of the robot 1 viewed from the front, and FIG. 1B is a diagram of the robot 1 viewed from the back. The robot 1 includes a head 2, a body 3, a hand 4, a foot 5, and a tail 6. The head 2 includes an ear 7, an eye 8, and a mouth 9. A camera serving as an image sensor 23 (see FIG. 2) for photographing a predetermined facing object is provided on the forehead 10 of the head 2. Further, each of the pair of left and right ears 7 is provided with a microphone 24 (see FIG. 2) for detecting an utterance of a predetermined target. The mouth 9 is provided with a speaker 27 (see FIG. 2) for the robot 1 to speak.

  A display device 28 (see FIG. 2) such as a liquid crystal display or an organic EL (Electro Luminescence) display is provided in the eye 8 of the robot 1, and images of both eyes are displayed on the screen of the display device 28. The facial expression of the robot 1 is represented by changes in the eyes such as the pupils and eyelids. For example, when sad, the sadness is expressed by lowering the eyelids and turning the eyes downward. The robot 1 recognizes a predetermined target by collating a person photographed by a camera (the image sensor 23) with a database, analyzes a voice detected from the microphone 24, and utters contents (attributes) of the predetermined target. ) Is determined, and the expression is changed according to the result of the determination.

  As shown in FIG. 2, the robot 1 includes a control unit 21, a storage unit 22, the above-described image sensor 23, a microphone 24, a moving unit 25, a movable unit 26, and a speaker 27, as shown in FIG. And a display device 28.

  The storage unit 22 is an example of a storage unit described in the claims. The moving unit 25 is an example of the moving mechanism described in the claims. The display device 28 is an example of a display unit described in the claims.

  The control unit 21 controls the operation of the entire robot. The control unit 21 includes a CPU (Central Processing Unit), and executes a program stored in the storage unit 22. In addition, the control unit 21 generates a control signal for moving the robot 1 according to various data acquired from the imaging device 23 and the microphone 24 based on the operation program and transmits the control signal to the moving unit 25. To control the movement of the robot.

  The storage unit 22 includes a random access memory (RAM), a read only memory (ROM), a flash memory, and the like. The ROM stores various operation programs. The RAM functions as a work area for temporarily storing various operation programs executed by the CPU, data necessary for executing these operation programs, and the like. The storage unit 22 is used as a person database and a keyword information database described later.

  The imaging device 23 is a camera that captures an image to recognize a predetermined target. The microphone 24 is a voice input device for acquiring voice data spoken by a predetermined target.

  The moving unit 25 is a part for moving the robot 1. The moving unit 25 includes left and right wheels provided on the bottom surface of the robot's feet, motors for rotating the left and right wheels, and a drive circuit for driving and controlling the motors. The movable portion 26 is a portion that is displaced by driving of an actuator or the like. The movable unit 26 includes driven members such as the head 2, the hand 4, the foot 5, and the tail 6, a driving member such as an actuator, and a driving circuit that drives and controls the driving member.

  The speaker 27 converts the sound data generated by the control unit 21 into sound and outputs the sound to the outside. The robot 1 can have a simple conversation with the predetermined target by outputting a voice corresponding to the utterance content of the predetermined target from the speaker 27. The display device 28 displays an image of the expression of the robot 1 generated by the control unit 21 on a screen.

  As shown in FIG. 3, the control device 30 of the robot functionally includes a voice detection unit 31, a voice recognition unit 32, a personal authentication unit 33, a keyword determination unit 34, a facial expression determination unit 35, It includes a database 36, a keyword information database 37, an utterance direction identification unit 38, a distance information acquisition unit 39, a movement control unit 40, and a movable control unit 41.

  The voice detection unit 31 and the voice recognition unit 32 are examples of a voice acquisition unit described in the claims. The personal authentication unit 33 is an example of a personal authentication unit described in the claims. The keyword determining unit 34 is an example of an attribute parameter acquiring unit described in the claims. The facial expression determination unit 35 is an example of a facial expression control unit described in the claims. The keyword information database 37 is an example of an attribute parameter database described in the claims.

  The sound detection unit 31 detects a sound included in an utterance by a predetermined target (a human including a user) from the sound collected by the microphone 24. When a silent section continues for a fixed time or more, for example, 1.5 seconds or more, the voice detecting unit 31 separates the utterance based on the silent section and detects voice data for each separated utterance section. The voice recognition unit 32 generates text data by performing voice recognition on the voice data detected by the voice detection unit 31 for each utterance section and analyzing the voice data using a well-known analysis method.

  The personal authentication unit 33 compares the image data of the predetermined target imaged by the imaging element 23 with the data of the person database 36 in the information of the individual in the storage unit 22 in which the information on the individual is registered, and performs the individual determination. The personal authentication unit 33 detects a person's face from a frame image captured by the robot at a predetermined frame rate (for example, 30 fps) using the image sensor 23, and identifies the detected face. In the person database 36, information on the person to be authenticated by the personal authentication unit 33 is registered. Specifically, the person database 36 stores a person ID, a name, and a face image for each person. The detected face image and the face image registered in the person database 36 are compared with each other for feature amounts such as the distance between eyes and the aspect ratio of the face, and face identification is performed to determine who the face is. Determine.

  The keyword determination unit 34 analyzes the text data generated by the voice recognition unit 32, compares the text data with the data in the keyword information database 37, and outputs keywords in the text data and expression determination information associated with the keywords. The keyword information database 37 is classified for each individual, and selects keyword information according to the individual based on the personal information authenticated by the individual authentication unit 33.

  In the keyword information database 37, words for determining the attributes of human utterance contents are registered as keywords in word units or short sentence units. Keywords include various parts of speech, such as nouns, verbs, and adjectives. Each keyword is tagged with its attribute information. The attribute information of the keyword is facial expression determination information used for determining a facial expression, and is an example of an attribute parameter described in the claims.

  The keyword attribute information is classified into positive, negative, and neutral. The state in which the attribute information of the keyword is positive includes a state in which the utterance content of the utterance including the keyword indicates joy / enjoyment, a state in which peace is expressed, a state in which the utterance is interested, and the like. . For example, keywords such as "happy", "fun", and "funny" are classified as positive. The state where the attribute information of the keyword is negative includes a state in which the utterance content of the utterance including the keyword indicates anger / sadness, a state indicating anxiety, a state indicating boredom, and the like. For example, keywords such as "scary", "sad", and "boring" are classified as negative. The state where the attribute information of the keyword is neutral includes a normal state in which the utterance content of the utterance including the keyword is other than positive and negative. Regarding the words used to determine the attributes of the utterance content, what words represent what attributes (positive, negative, neutral) depends on personal experiences, memories, hobbies, tastes, etc. Even so, the attribute of the utterance content including the word differs depending on the person. Therefore, the keyword information database 37 is provided with data having different contents for each predetermined target.

  As an example of the configuration of the keyword information database 37, as shown in FIG. 5, keywords and keyword attribute information are recorded for each predetermined target. Of the keywords, “happy”, “sad”, and “go” have the same attribute information of the keywords of the users A and B. On the other hand, the attribute information of the keyword of the user A is “negative”, but the attribute information of the keyword of the user B is positive, in other words, the utterance represented by the “yaw” in both A and B. The attributes of the contents are different from each other. In this way, keyword information corresponding to an individual is created, and information for determining a facial expression suitable for the individual is provided.

  The facial expression determination unit 35 determines the facial expression of the robot based on the facial expression determination information such as positive or negative from the keyword determination unit 34, and causes the display device 28 to display an image corresponding to the determined facial expression. At this time, the facial expression is determined based on not only the facial expression determination information obtained from the keywords included in the current utterance of the predetermined target, but also two facial expression determination information obtained from two keywords that are continuously acquired. That is, the expression is determined based on the relationship between the expression determination information obtained from the keyword included in the previous utterance and the expression determination information obtained from the keyword included in the current utterance.

  The utterance direction identification unit 38 recognizes a phase difference or a volume difference between sounds acquired by the microphones 24 provided in each of the pair of left and right ears 7 in accordance with the utterance from the predetermined object, thereby making the utterance. Specify the direction. The distance information obtaining unit 39 obtains distance information from the image sensor 23 to a predetermined target. The distance information is obtained by the autofocus function of the image sensor. Note that the distance information may be obtained by providing a distance sensor. By executing the operation program, the movement control unit 40 generates a control signal for moving the robot 1 based on, for example, data acquired from the speech direction identification unit 38 and the distance information acquisition unit 39, and sends the control signal to the movement unit 25. By inputting, the movement of the robot 1 is controlled. The movable control unit 41 controls the operation of the movable unit 26 by executing an operation program.

  Next, an operation of the robot 1 when a predetermined object talks to the robot 1 will be described. When a predetermined target speaks to the robot 1, the robot 1 collects a voice of the predetermined target using a microphone 24 provided in the ear 7. The utterance direction specifying unit 38 specifies the utterance direction according to the utterance from the predetermined target. When the direction is specified, the movement control unit 40 controls the moving unit 25 to move the robot 1 in the specified direction. Further, the distance information obtaining unit 39 obtains predetermined target and distance information from the image sensor 23.

  The robot 1 specifies a relative positional relationship with a predetermined target according to an utterance from the predetermined target. The movement control unit 40 specifies the relative positional relationship between the robot 1 and a predetermined target based on the distance information acquired by the distance information acquisition unit 39 and the information on the speech direction acquired by the speech direction identification unit 38. Then, the movement control unit 40 generates a control signal for moving toward a predetermined target, and inputs this signal to the moving unit 25. Thus, the movement control unit 40 controls the operation of the moving unit 25. The movement control unit 40 stops the operation of the movement unit 25 when determining that the distance between the robot and the predetermined target has reached the predetermined distance.

  As described above, the robot 1 approaches a predetermined target. Thereafter, personal authentication of a predetermined target is performed, and a keyword to be used for expression determination and attribute information thereof are acquired from the utterance content of the predetermined target by keyword determination processing.

  Next, the keyword determination processing will be described with reference to the flowchart shown in FIG. The keyword determination unit 34 acquires text data obtained by voice-recognizing a predetermined target utterance and converting it into text, from the voice recognition unit 32 (step S101). The keyword determination unit 34 acquires the personal information of the predetermined target that has been personally authenticated from the personal authentication unit 33 (step S102). When acquiring the personal information for specifying the predetermined target, the keyword determination unit 34 selects the keyword information for the individual based on the acquired personal information (step S103). Subsequently, the keyword determination unit 34 collates the selected individual keyword information with the text data, acquires a matching keyword and attribute information [Property_0] tagged with the keyword, and stores the keyword in the storage unit 22. (Step S104). As described above, since a predetermined target is authenticated and a personal keyword information database corresponding to the predetermined target can be used, the facial expression can be determined in accordance with the sense of the individual.

  When the keyword attribute information (expression determination information) is acquired, the expression determining unit 35 determines the expression of the robot 1 using the acquired keyword attribute information. The facial expression determination processing for determining the facial expression of the robot 1 will be described with reference to the flowchart shown in FIG.

  When the facial expression determining process is started, first, the facial expression determining unit 35 determines whether the attribute information [Property_0] of the keyword acquired from the keyword determining unit 34 is unknown (Step S201). Even if the text data acquired from the voice recognition unit 32 is compared with the keyword information database 37, there is a case where the corresponding keyword does not exist. In this case, it is determined that the current attribute information [Property_0] is unknown (step S201: Yes).

  If it is determined that the current attribute information [Property_0] is unknown, a “normal” expression is determined as the expression of the robot (step S202). The expression “normal” is an expression in a calm situation that is neither an expression showing a positive emotion such as happy or fun nor an expression showing a negative emotion such as sad or boring. As shown in FIG. 11A, the expression of “normal” is a state in which the eyelids are open and the diameter of the pupil with respect to the pupil is about half. The determined normal expression image data is sent to the display device 28, and the display device 28 displays the normal expression image. That is, a normal expression is given to the utterance of the predetermined target as a natural correspondence according to the utterance content.

  When it is determined that the current attribute information [Property_0] is not unknown (step S201: No), it is determined whether the current attribute information is “positive” (step S203). If the current attribute information is determined to be "positive" (step S203: Yes), it is determined that the predetermined object has been pleased, and a "happy" expression is determined as the expression of the robot (step S204). ). A facial expression that can be widely applied as a facial expression when the attribute information is positive is a “happy” facial expression, which is accepted without any discomfort as a reaction to the utterance of the predetermined target. The “happy” expression is as shown in FIG. This is a state in which the eyelids are open, the pupil is almost the same size as the pupil, and a large number of reflections of external light are represented in the pupil by ellipses, stars, and crescents. The expression that sparkles his eyes is shown.

  If the current attribute information is not “positive” (step S203: No), it is determined whether the attribute information is “negative” (step S205). If it is determined that the current attribute information is “negative” (step S205: Yes), it is determined that the predetermined object has been sad or dissatisfied, and the expression “worried” is determined as the expression of the robot. Is performed (step S206). The expression of “worried” is a state in which the eyelids are slightly lowered and the pupil is upward, as shown in FIG.

  If the current attribute information is neither “positive” nor “negative” (step S205: No), it is determined that the current attribute information is “neutral” (step S207). In this case, since the content of the utterance represented by the keyword is neither positive nor negative, the expression is set to indicate that the utterance is recognized by pupil change / blinking or the like (step S208). As shown in FIG. 11D, the expression of “pupil change” changes so that the size of the pupil gradually increases, and blinks the eyelid up and down a plurality of times.

  When the current attribute information is determined and the facial expression is determined by executing any of steps S202, S204, S206, and S208, it is determined whether or not there is subsequent text data (step S209). If the utterance by the predetermined object continues and there is subsequent text data (step S209: Yes), the process proceeds to the next processing step S301 shown in FIG. If there is no subsequent text data (step S209: No), the process ends.

  Since the processing steps in FIG. 6 are executed at the first time immediately after the start of the facial expression determination processing, there is no keyword acquired before that. Therefore, the expression of the robot 1 is determined based only on the current attribute information. On the other hand, in FIG. 7 and subsequent figures, the expression is determined based on the relationship between the attribute information of the keyword acquired last time and the attribute information of the keyword acquired this time. That is, the flow of the utterance is read based on the transition of the attribute information, and an expression along the utterance flow of a predetermined target is selected.

  First, the attribute information [Property_0] of the keyword acquired in the process of FIG. 6 is shifted to the previous attribute information [Property_1] (step S301). That is, [Property_0] = [Property_1]. As described later, in each of the processes of FIGS. 7 to 10, when there is the next text data, the process returns to step S301. In this case, the attribute information [Property_0] of the keyword acquired in each processing of FIGS. 7 to 10 is shifted to the previous attribute information [Property_1]. The facial expression determining unit 35 determines whether the previous attribute information [Property_1] is unknown (step S302). If the previous attribute information is unknown (step S302: Yes), it is determined whether the current attribute information [Property_0] newly acquired from the keyword determination unit 34 is unknown (step S303). If neither the previous attribute information nor the current attribute information is unknown (step S303: Yes), the expression of the robot is maintained as “normal” (step S304).

  If the current attribute information is not unknown (step S303: No), it is determined whether the current attribute information is “positive” (step S305). If the previous attribute information is unknown and the current attribute information is “positive” (step S305: Yes), it is determined that there is something happy, and it is determined that the expression is “happy” (step S306). .

  If the current attribute information is not “positive” (step S305: No), it is determined whether the current attribute information is “negative” (step S307). If it is determined that the previous attribute information is unknown and the current attribute information is “negative” (step S307: Yes), it is determined that the content of the utterance of the user is sad or dissatisfied, and the robot Is determined to be the expression of “worried” (step S308).

  If the current attribute information is neither “positive” nor “negative” (step S307: No), it is determined that the previous attribute information is unknown and the current attribute information is “neutral” (step S309). In the case of "neutral", in order to bring out the continuation of the utterance, the expression is set to an expression indicating that recognition is made by pupil change, blinking, or the like (step S310).

  When the attribute information is determined and the facial expression is determined, it is determined whether or not there is subsequent text data (step S311). If the utterance by the predetermined target continues and there is text data to follow (step S311: Yes), the process moves to processing step S301. If there is no subsequent text data (step S311: No), the process ends.

  If the previous attribute information is not unknown (Step S302: No), the process proceeds to the processing step in FIG. 8, and it is determined whether the previous attribute information is “positive” (Step S401). If the previous attribute information is “positive” (step S401: Yes), it is determined whether the current attribute information is unknown (step S402).

  If the current attribute information is unknown (step S402: Yes), the utterance content of the predetermined object inherits the previous positive state, so that it is determined that the expression of the robot is “good mood”. (Step S403). As shown in FIG. 12A, the expression of “good mood” is a state in which a crescent-shaped pupil is displayed at the center of the eye. The expression is more calm as time passes than the "happy" expression, but expresses a good mood.

  Conversely, if the current attribute information is not unknown (step S402: No), it is determined whether the current attribute information is “positive” (step S404). If both the previous attribute information and the current attribute information are “positive” (step S404: Yes), the attribute information becomes “positive” continuously, so the joy and enjoyment expressed by the utterance content It is determined that the expression has further increased, and it is determined that the expression of the robot is “expressed” (step S405). As shown in FIG. 12 (B), the expression of “excitement” is a state in which the pupil is almost the same size as the pupil, and the pupil is displayed at a plurality of places in the form of an ellipse and a crescent shape with reflection of external light, An elliptical tear is displayed below the eye. The emotional expression expresses the expression of tears and moisturizing eyes.

  If the current attribute information is not “positive” (step S404: No), it is determined whether the current attribute information is “negative” (step S406). If it is determined that the previous attribute information is “positive” and the current attribute information is “negative” (step S406: Yes), the attribute information has changed from “positive” to “negative”. It is determined that the content of the utterance has changed from joy or pleasure to sadness, and it is determined that the expression of the robot is "embarrassed" (step S407). As shown in FIG. 12C, the expression of “confused” is a state in which the right eyelid is slightly lowered and the diameter of the pupil is about one third of the pupil. The pupil is made smaller to express a confused expression.

  If the current attribute information is neither “positive” nor “negative” (step S406: No), it is determined that the previous attribute information is “positive” and the current attribute information is “neutral” (step S406). S408). In this case, since the attribute information has changed from "positive" to "neutral", the content of the utterance does not change from joy or enjoyment, but other keywords are recognized. The expression of "expectation" is determined as the expression of the robot so as to expect the development (step S409). As shown in FIG. 12D, the expression of “expectation” is a state in which the size of the pupil and the reflection of external light are somewhat less than the expression of “happy”. She shines her eyes and expresses the expression of expectations.

  When the attribute information is determined and the facial expression is determined, it is determined whether or not there is subsequent text data (step S410). If the utterance by the predetermined object continues and there is subsequent text data (step S410: Yes), the process moves to the processing step S301 in FIG. If there is no subsequent text data (step S410: No), the process ends.

  On the other hand, when the previous attribute information is not “positive” (step S401: No), the process proceeds to the processing step of FIG. In this case, it is determined whether the previous attribute information is “negative” (step S501). When it is determined that the previous attribute information is “negative” (Step S501: Yes), it is determined whether the current attribute information is unknown (Step S502).

  If the current attribute information is unknown (step S502: Yes), since the previous attribute information is “negative”, it is determined that the expression of the robot is “declined” (step S503). If the current attribute information is “negative” and the facial expression is determined based solely on it, the expression “worried” is used as described in step S206. In the above case, however, the content of the utterance can be turned around. Instead, it is considered that the negative state is continuing, and the expression of “fall” is given. As shown in FIG. 13A, the expression of “depression” is a state in which the eyelids are close to half of the eyes and the pupils are located below. The eyes are lower than the expression of "anxiety", and the expression of depression and depression is expressed.

  If the current attribute information is not unknown (Step S502: No), it is determined whether the current attribute information is “positive” (Step S504). If the previous attribute information is “negative” and the current attribute information is “positive” (step S504: Yes), the attribute information changes from “negative” to “positive”, and the attribute of the utterance content is changed. Is determined to have changed from sadness, anger, etc. to joy, pleasure, etc., and it is determined that the expression of the robot should be the expression of "relief" (step S505). As shown in FIG. 13B, the expression “relieved” is a state in which the crescent-shaped pupil is arranged obliquely downward. The expression is relieved as the eyes are drooping.

  If the current attribute information is not “positive” (step S504: No), it is determined whether the current attribute information is “negative” (step S506). When both the previous attribute information and the current attribute information are “negative” (step S506: Yes), the state in which the attribute information is “negative” is continuous, so the sadness expressed by the utterance content, Assuming that dissatisfaction and the like have further increased, it is determined that the expression of the robot should be the expression of "sadness" (step S507). As shown in FIG. 13C, the expression of "sadness" is a state in which the eyelids are lowered to about two thirds of the eyes, the pupils are located below the eyes, and the elliptical tears are located below the eyes. is there. The expression of "sadness" is expressed by lowering the eyelids more than the expression of "depression" and tearing.

  If the current attribute information is neither “positive” nor “negative” (step S506: No), it is determined that the previous attribute information is “negative” and the current attribute information is “neutral” (step S508). ). In this case, since the attribute information changes from “negative” to “neutral”, the utterance content of the predetermined target does not change from the state of sadness, but other keywords are recognized. It is determined that the expression of the robot is “anxiety” so as to be worried about the development (step S509). As shown in FIG. 13D, the expression of “anxiety” is a state where the eyelids are lowered by about a quarter and the pupil and the pupil are located below the eyes. The eyelids are lower than the expression of "worry", the depression is moderate, and the situation is uncertain and an uneasy expression is expressed.

  When the attribute information is determined and the facial expression is determined, it is determined whether or not there is subsequent text data (step S510). If the utterance by the predetermined object continues and there is subsequent text data (step S510: Yes), the process moves to the processing step S301 in FIG. If there is no subsequent text data (step S510: No), the process ends.

  If the previous attribute information is unknown and neither “positive” nor “negative” (step S501: No), the process proceeds to the processing steps in FIG. In this case, it is determined that the previous attribute information is "neutral" (step S601).

  Next, it is determined whether the current attribute information is unknown (step S602). If the current attribute information is unknown (step S602: Yes), the previous attribute information is "neutral", and the utterance content represented by the keyword is neither positive nor negative, so that the robot is asked as the expression of the robot. It is determined that the expression should be such as to be confirmed (step S603). As shown in FIG. 14A, the expression of “confirmation” is a state in which the pupil and the pupil are located above the eyes. It expresses the expression as if you ask it with an upper eye. If the current attribute information is not unknown (step S602: No), it is determined whether the current attribute information is “positive” (step S604).

  If the previous attribute information is “neutral” and the current attribute information is “positive” (step S604: Yes), the attribute information changes from “neutral” to “positive”. In this case, it is assumed that the utterance content of the predetermined target is happy, and the expression of the robot is determined to be “happy” (step S605). FIG. 14B shows an example of a “happy” expression. If the current attribute information is not “positive” (step S604: No), it is determined whether the current attribute information is “negative” (step S606).

  If the previous attribute information is “neutral” and the current attribute information is “negative” (step S606: Yes), the attribute information changes from “neutral” to “negative” and the utterance content of the predetermined target It is determined that there is something sad or dissatisfied, and the expression of the robot is determined to be the expression of "worried" (step S607). FIG. 14C shows an example of an expression of “worried”.

  If the current attribute information is neither “positive” nor “negative” (step S606: No), it is determined that both the previous attribute information and the current attribute information are “neutral” (step S608). The attribute information is "neutral" in succession, and the utterance content expressed by the keyword was neither positive nor negative, so the robot's facial expression was recognized by pupil change, blinking, etc. to elicit the next utterance Is determined (step S609). FIG. 14D shows an example of an expression of “pupil change”.

  When the attribute information is determined and the expression is determined, it is determined whether or not there is subsequent text data (step S610). If the utterance by the predetermined object continues and there is text data to follow (step S610: Yes), the process moves to the processing step S301 in FIG. If there is no subsequent text data (step S610: No), the process ends.

  As described above, after the second facial expression determination is performed after the start of the facial expression determination process, the facial expression is determined based on the relationship between the previous attribute information and the current attribute information, and such a facial expression is determined. The decision method is repeatedly executed.

FIG. 15 shows an example of how the expression of the robot changes when a predetermined object speaks based on the above-described expression determination process and the example of the expression. It is assumed that a predetermined target has spoken to the robot 1 as described below.
"I had a stylish lunch with my friends yesterday."
"But the incident is ..."
"I couldn't make a reservation because of a clerk's mistake ... I was really impatient."
When the predetermined object speaks to the robot, "I had a stylish lunch with a friend yesterday," the voice detecting unit 31 detects the voice of the predetermined object, and the voice recognizing unit 32 detects the predetermined predetermined voice. Convert the target audio data to text. The text data is sent to the keyword determination unit 34. The keyword determination unit 34 compares the text data with a keyword information database 37 corresponding to a predetermined target, and acquires a keyword and attribute information tagged with the keyword. Here, a keyword of “fashionable lunch” + “had” and “positive” as its attribute information are acquired. The facial expression determining unit 35 executes a facial expression determining process from the acquired attribute information. In this case, this corresponds to Yes in step S203 of the facial expression determination process in FIG. 6, and a “happy” facial expression is determined in subsequent step S204. The expression of the robot 1 changes to a “happy” expression based on this determination.

  Subsequently, the predetermined object utters "But the incident is ...". The keyword determination unit 34 extracts “happening” as a keyword, and acquires “negative” as its attribute information. Since the attribute information of the previous keyword “fashionable lunch” + “was” was “positive”, the attribute information changes from “positive” to “negative”. In this case, this corresponds to Yes in step S406 of the facial expression determination process in FIG. 8, and the facial expression of “embarrassment” is determined in the subsequent step S407. Based on this determination, the expression of the robot 1 changes from a “happy” expression to a “confused” expression.

  Subsequently, the predetermined object utters, "I could not make a reservation because of a clerk's mistake ... I was really rushed." The keyword determination unit 34 extracts “hurled” as a keyword and acquires “negative” as its attribute information. Since the attribute information of the previous keyword “happing” was “negative”, the attribute information changes from “negative” to “negative”. In this case, this corresponds to Yes in step S506 of the facial expression determination process in FIG. 9, and the facial expression of “sadness” is determined in the subsequent step S507. Based on this determination, the expression of the robot 1 changes from the expression "confused" to the expression "sad".

  The robot 1 can interact with a predetermined target by outputting a voice from the speaker 27 according to the determination result of the facial expression determining unit 35. For example, in the example of FIG. 15, in response to the utterance “I had a stylish lunch with a friend yesterday”, a reply indicating “I'm glad” and “I am happy” is given. In response to the next "but happening ...", he responds with "what happened" and "embarrassed". He responds to "The clerk made a mistake and couldn't make a reservation ... I was really irritated."

  As described above, the control device 30 of the robot 1 acquires the voice uttered by the predetermined target by the voice detection unit 31 and the voice recognition unit 32, and determines “positive”, “negative” from the obtained utterance of the predetermined target. The attribute information of “neutral” is acquired by the keyword determination unit 34. Then, the control device 30 of the robot 1 determines the flow of the utterance based on the “positive” or “negative” attribute transition, which is the relationship between the current attribute information acquired during the utterance of the predetermined target and the previous attribute information. The facial expression is read and the facial expression is determined by the facial expression determining unit 35. As a result, it is possible to realize a natural change in facial expression along with the flow of the story.

  The control device 30 of the robot 1 further includes a personal authentication unit 33 that authenticates a predetermined target, and a person database 36 in which attribute information corresponding to a keyword is registered for each predetermined target. 34 selects a keyword information database 37 corresponding to a predetermined target authenticated by the personal authentication unit 33, matches text data acquired from the voice recognition unit 32 with the keyword information database 37, and acquires attribute information. As a result, it is possible to determine an individual to be faced and use the keyword information database for the individual, and to realize a facial expression selection suited to the individual.

  Further, the control device 30 of the robot 1 analyzes the voice acquired by the voice detecting unit 31 and the voice recognizing unit 32 by the keyword determining unit 34, extracts a keyword included in the voice, and tags the extracted keyword. The attribute information of “positive”, “negative”, and “neutral” is acquired. Thus, it is possible to perform voice recognition in real time, obtain a keyword and attribute information used for determining a facial expression from the voice recognition, and perform a rapid facial expression change.

  In addition, the robot 1 is configured to be able to interact with a predetermined target, and the control device 30 of the robot 1 uses the expression determining unit 35 to compare the current attribute information acquired during the dialog with the predetermined target with the current attribute information. The expression of the robot 1 is controlled based on the relationship with certain attribute information. This makes it possible to realize a natural change in facial expression along with the flow of the dialogue.

  Further, the robot 1 includes a display device 28 configured to display an image corresponding to the expression of the robot 1 itself, and the control device 30 of the robot 1 uses the expression determination unit 35 to acquire the image during the utterance of the predetermined target. Expression data is generated based on the relationship between the current attribute information and the previous attribute information, and is displayed on the display device 28. Thereby, a natural change of the expression along the flow of the story can be displayed as the expression of the robot 1.

  Further, the robot 1 includes a moving unit 25 configured to move the robot 1. The control device 30 of the robot 1 controls the robot 1 to a predetermined position when controlling the facial expression of the robot 1 by the facial expression determining unit 35. It further includes a movement control unit 40 that controls the movement unit 25 so as to approach the target. This facilitates communication with the opponent.

  Note that the present invention is not limited to the above embodiment, and various modifications and applications are possible. The above embodiment may be modified as follows.

  In the above-described embodiment, the expression determination process is performed based on the relationship between two pieces of continuous attribute information including the attribute information of the emotion represented by the previous keyword and the attribute information of the emotion represented by the keyword. The present invention is not limited to this, and the facial expression determination process may be performed based on the relationship between attribute information represented by keywords included in three or more utterances of the subject who are temporally continuous. By increasing the number of combinations, more expressions can be expressed. However, when the number of keywords increases, it takes time to acquire the number of keywords necessary for performing the facial expression determination process, and the determination of the next facial expression is delayed. As a result, it is difficult to instantaneously change the facial expression with respect to the utterance of the predetermined target. With the combination of the attribute information of two keywords, it is possible to instantaneously change the expression with respect to the utterance of the predetermined target without acquiring time for the necessary keyword. Further, as for the number of expressions to be expressed, the required types of expressions are almost covered by the variations of the expressions shown in the present embodiment, and natural expressions corresponding to the utterance contents of a predetermined target can be expressed. Therefore, a combination of the attribute information of the two keywords is desirable for implementation.

  In the present embodiment, the change in the facial expression of the robot is represented by the state of the eyes, but the present invention is not limited to this, and may be represented by other parts of the face such as the mouth and eyebrows in addition to the eyes. In addition, facial expressions include not only faces but also gestures. The robot 1 has a movable tail 6 as shown in FIG. By adding the movement of the tail 6 to the facial expression determined in the facial expression determination processing, the facial expression can be shown by the tail 6 according to the utterance of the predetermined target. For example, the feeling of "happy" is expressed by swinging the tail 6 largely left and right, and the feeling of "security" is expressed by slowly shaking the tail 6 diagonally downward. The movable control unit 41 generates a drive signal for performing the above-described movement in accordance with the determination result of the facial expression determining unit 35, and transmits this signal to the movable unit 26. The movable section 26 moves the tail section 6 by controlling a drive member such as an actuator by a drive circuit according to a drive signal. In addition, the facial expression may be expressed not only by the tail part 6 but also by moving other parts such as the head part 2, the hand part 4, and the foot part 5. However, it is difficult to instantaneously change the expression in response to the utterance of a predetermined object in a part or movement that takes a long time to drive. Therefore, a part or movement that does not take a long time to drive is preferable.

  In the present embodiment, a doll-shaped robot has been described as an example. However, the robot is not limited to this, and includes a robot in which an image of the robot is displayed on the screen 51 of the mobile terminal 50 as shown in FIG. Using the image sensor 23 built in the portable terminal 50, an image of a predetermined target facing is photographed, and the utterance of the predetermined target is detected using the built-in microphone 24. When a predetermined object speaks toward the mobile terminal 50, the expression of the robot displayed on the screen 51 of the mobile terminal 50 changes according to the dialogue. The screen 51 may display not only the face but also the upper body or the whole body. In this case, the expression can be expressed not only by the expression of the face but also by the gesture by moving the hand, the foot, or the like.

  In the present embodiment, the keyword attribute information is classified into positive, negative, and neutral. However, the present invention is not limited to this, and may be classified into, for example, joy, anger, sorrow, ease, and the like.

  In the present embodiment, the keyword information database 37 is divided for each individual, and a keyword and attribute information corresponding to the individual are registered. In the present embodiment, the creation of the keyword information database for each individual is not particularly mentioned. The keyword information database for each individual, for example, determines positive / negative based on the tone of the voice for each individual with respect to the keyword, or captures the facial expression of each individual when speaking the keyword by using the image sensor and outputs the positive from the captured image. -Created by determining negative.

  Further, in the present embodiment, the expression is determined only from the attribute information extracted from the keyword information database 37. The present invention is not limited to this, and in addition to the attribute information extracted from the keyword information database, a positive / negative determination may be separately performed, and the expression may be determined based on the result of both. For example, in the case of a keyword that can be both positive and negative, it is possible to determine whether the keyword is positive or negative based on the tone and expression of the voice. It is also possible to judge by parsing. However, in order to achieve a natural change in facial expression, it is necessary to react quickly to successive keywords. Therefore, the condition is that the determination does not take much time.

  In the present embodiment, a description has been given assuming that various databases necessary for determining an execution operation are stored in the storage unit 22 provided in the robot 1. However, these data may be stored in an external storage device or the like. The robot 1 may be configured to appropriately acquire necessary data from an external storage device that can communicate via a communication network.

  While some embodiments of the present invention have been described, the scope of the present invention is not limited to the above-described embodiments, but includes the scope of the invention described in the claims and equivalents thereof. In the following, the inventions described in the claims first attached to the application form of this application are appended. The appended numbers are as set forth in the claims originally attached to the application for this application.

(Appendix 1)
A robot control device that controls a facial expression of a robot configured to be able to change its own facial expression,
Voice acquisition means for acquiring a voice uttered by a predetermined object;
An attribute parameter obtaining unit that obtains an attribute parameter representing an attribute of the utterance content of the predetermined target by analyzing the voice obtained by the voice obtaining unit;
Storage means for storing the attribute parameters obtained by the attribute parameter obtaining means,
Expression control means for controlling the expression of the robot, based on a relationship between the attribute parameters acquired during the utterance of the predetermined target and the past attribute parameters stored in the storage means,
Control device for a robot, comprising:

(Appendix 2)
Personal authentication means for authenticating a predetermined object;
An attribute parameter database in which attribute parameters corresponding to the utterance content are registered for each predetermined target,
Further comprising
The attribute parameter acquisition unit selects an attribute parameter database corresponding to a predetermined target authenticated by the personal authentication unit, compares the voice acquired by the voice acquisition unit with the attribute parameter database, and To get the
A control device for a robot according to supplementary note 1.

(Appendix 3)
The attribute parameter obtaining unit analyzes the voice obtained by the voice obtaining unit, extracts a keyword included in the voice, and obtains attribute information of the extracted keyword as the attribute parameter.
3. The control device for a robot according to claim 1 or 2.

(Appendix 4)
The robot is configured to be able to interact with the predetermined object,
The expression control unit controls the expression of the robot based on a relationship between the attribute parameter acquired during the dialogue with the predetermined target and the past attribute parameter.
4. The control device for a robot according to any one of supplementary notes 1 to 3.

(Appendix 5)
The robot includes a display unit configured to display an image according to the expression of the robot,
The expression control unit generates expression data based on a relationship between the attribute parameter acquired during the utterance of the predetermined target and the past attribute parameter, and displays the expression data on the display unit.
The control device for a robot according to any one of supplementary notes 1 to 4.

(Appendix 6)
The robot includes a movement mechanism configured to move the robot,
When controlling the facial expression of the robot by the facial expression control means, further comprising a movement control means for controlling the moving mechanism so as to bring the robot closer to the predetermined target,
6. The control device for a robot according to any one of supplementary notes 1 to 5.

(Appendix 7)
A robot comprising the control device for a robot according to any one of supplementary notes 1 to 6.

(Appendix 8)
A voice obtaining step of obtaining a voice uttered by a predetermined target;
An attribute parameter acquiring step of acquiring an attribute parameter representing an attribute of the utterance content of the predetermined target by analyzing the acquired voice;
A storing step of storing the obtained attribute parameters;
An expression control step of controlling the expression of the robot based on the relationship between the attribute parameter acquired during the utterance of the predetermined target and the attribute parameter stored in the past,
Robot control method including:

(Appendix 9)
On the robot computer,
Voice acquisition processing for acquiring a voice uttered by a predetermined target;
An attribute parameter acquisition process of acquiring an attribute parameter representing an attribute of the utterance content of the predetermined target by analyzing the acquired voice;
A storage process for storing the acquired attribute parameters;
An expression control process for controlling the expression of the robot based on the relationship between the attribute parameter acquired during the utterance of the predetermined target and the attribute parameter stored in the past,
A program that executes

DESCRIPTION OF SYMBOLS 1 ... Robot, 2 ... Head, 3 ... Body part, 4 ... Hand part, 5 ... Foot part, 6 ... Tail part, 7 ... Ear part, 8 ... Eye part, 9 ... Mouth part, 10 ... Forehead, 21 ... Control unit, 22 storage unit, 23 imaging device, 24 microphone, 25 moving unit, 26 movable unit, 27 speaker, 28 display device, 30 control device, 31 voice detection unit, 32 voice Recognition unit, 33: individual authentication unit, 34: keyword determination unit, 35: expression determination unit, 36: person database, 37: keyword information database, 38: speech direction identification unit, 39: distance information acquisition unit, 40: movement control Unit, 41: movable control unit, 50: portable terminal, 51: screen

Claims (9)

A robot control device that controls a facial expression of a robot configured to be able to change its own facial expression,
Voice acquisition means for acquiring a voice uttered by a predetermined object;
An attribute parameter obtaining unit that obtains an attribute parameter representing an attribute of the utterance content of the predetermined target by analyzing the voice obtained by the voice obtaining unit;
Storage means for storing the attribute parameters obtained by the attribute parameter obtaining means,
Expression control means for controlling the expression of the robot, based on a relationship between the attribute parameters acquired during the utterance of the predetermined target and the past attribute parameters stored in the storage means,
Control device for a robot, comprising: Personal authentication means for authenticating a predetermined object;
An attribute parameter database in which attribute parameters corresponding to the utterance content are registered for each predetermined target,
Further comprising
The attribute parameter acquisition unit selects an attribute parameter database corresponding to a predetermined target authenticated by the personal authentication unit, compares the voice acquired by the voice acquisition unit with the attribute parameter database, and To get the
A control device for a robot according to claim 1. The attribute parameter obtaining unit analyzes the voice obtained by the voice obtaining unit, extracts a keyword included in the voice, and obtains attribute information of the extracted keyword as the attribute parameter.
The control device for a robot according to claim 1. The robot is configured to be able to interact with the predetermined object,
The expression control unit controls the expression of the robot based on a relationship between the attribute parameter acquired during the dialogue with the predetermined target and the past attribute parameter.
The control device for a robot according to claim 1. The robot includes a display unit configured to display an image according to the expression of the robot,
The expression control unit generates expression data based on a relationship between the attribute parameter acquired during the utterance of the predetermined target and the past attribute parameter, and displays the expression data on the display unit.
A control device for a robot according to any one of claims 1 to 4. The robot includes a movement mechanism configured to move the robot,
When controlling the facial expression of the robot by the facial expression control means, further comprising a movement control means for controlling the moving mechanism so as to bring the robot closer to the predetermined target,
A control device for a robot according to any one of claims 1 to 5.   A robot comprising the control device for a robot according to any one of claims 1 to 6. A voice obtaining step of obtaining a voice uttered by a predetermined target;
An attribute parameter acquiring step of acquiring an attribute parameter representing an attribute of the utterance content of the predetermined target by analyzing the acquired voice;
A storing step of storing the obtained attribute parameters;
An expression control step of controlling the expression of the robot based on the relationship between the attribute parameter acquired during the utterance of the predetermined target and the attribute parameter stored in the past,
Robot control method including: On the robot computer,
Voice acquisition processing for acquiring a voice uttered by a predetermined target;
An attribute parameter acquisition process of acquiring an attribute parameter representing an attribute of the utterance content of the predetermined target by analyzing the acquired voice;
A storage process for storing the acquired attribute parameters;
An expression control process for controlling the expression of the robot based on the relationship between the attribute parameter acquired during the utterance of the predetermined target and the attribute parameter stored in the past,
A program that executes

Images