JP5751610B2 - Conversation robot - Google Patents

Description

  The present invention relates to a conversation robot, and is suitable for application when, for example, a conversation performed by a plurality of people while sharing a topic (hereinafter referred to as a group conversation).

  In recent years, many robots have been commercialized that use electric or magnetic actions to perform movements resembling human or animal movements. For such robots, there is also known a robot equipped with a voice conversation function capable of performing conversations with users similar to conversations that humans perform daily, for example (see Non-Patent Document 1, for example). ).

  In fact, these various conversation robots are equipped with various external sensors such as CCD (Charge Coupled Device) cameras and microphones, recognize external conditions based on the output of these external sensors, and autonomously based on the recognition results. Some are designed to be able to act. For example, as an example, based on an external sensor, the direction of a user who speaks is recognized, and for example, the body part is moved so that the front part of the body part and the front part of the head face the direction of the user, as if the user's speech There is also known a conversation robot that reacts and talks to a user.

Seita Ibuki, Kenji Kimura, Natsuyoshi Takeda: Life support system for the elderly using communication robots, Journal of the Japan Society of Mechanical Engineers, Vol.108, No.1038, pp.392-395 (2005)

  However, in such a conversation robot, when the group conversation is actually performed, the front of the body and the front of the head are moved in the direction of the speaking user, and the line of sight is directed only to the speaking user. When this action is performed, the unnaturalness as if the other user who was not speaking is also a participant in the group conversation, as if it was removed from the group conversation, There was a problem that the user felt.

  The present invention has been made in consideration of the above points, and an object of the present invention is to propose a conversation robot that can perform a natural conversation according to the situation.

In order to solve this problem, a first aspect of the present invention includes an upper body portion that is rotatably provided on the body portion and a head portion that is rotatably provided on the upper body portion. in based on the output results with a plurality of objects conversation robots that autonomously group conversation, position detecting means for detecting the positions of a plurality of objects before on the basis of the output results obtained from the external sensor, the external based on the output results obtained from the sensor, recognizes one of the plurality of objects as the main object of interest, recognizing the role identification means other remaining of the object as従注th object, before Symbol Body part control for controlling the rotation angle of the upper body part so that the front surface of the upper body part is directed toward the center of gravity direction indicated by the center of gravity direction line obtained from the positions of the main attention object and the secondary attention object. Means in front of the main object of interest And a head control means for controlling the rotation angle of the head so as to direct the front of the head, the head control unit and the body unit control means, based on data from the roles identification means, It is possible to control the head so that the front of the head is directed toward the target object by rotating only the head while the front of the body is directed toward the center of gravity indicated by the center of gravity direction line. And, with the front of the head directed toward the main object of interest, only the upper body is rotated so that the front of the upper body is moved toward the center of gravity indicated by the center of gravity direction line. It is the structure which can be controlled to turn .

  According to a second aspect of the present invention, the plurality of objects are users, and the role identification unit is a user who speaks among the plurality of users based on images and / or sounds acquired from the external sensors. Alternatively, a user who is a main listener is recognized as the main target object, and the other remaining users are recognized as the sub target object.

  Further, according to a third aspect of the present invention, the role identifying means recognizes a next target object estimation object noticed by the user recognized as the main target object, and the user recognizes the next target object estimation object after the utterance ends. As a new main target object.

  According to a fourth aspect of the present invention, the center-of-gravity direction angle formed between the center-of-gravity direction line and a predetermined reference line is a main target-of-interest angle formed between the main target-of-interest direction line extending in the direction of the main target-of-interest and the reference line. And a secondary attention target angle formed by the secondary attention object direction line extending in the direction of the secondary attention object and the reference line, and the plurality of values obtained by adding all the primary attention object angle and the secondary attention object angle It is obtained by dividing by the total number of objects.

  According to a fifth aspect of the present invention, when the body part control means directs the front of the upper body part in the direction of the center of gravity obtained from the plurality of objects, the rotation angle of the upper body part is a predetermined number of times. When the moving angle is equal to or greater than the moving angle, the center of gravity direction is obtained by excluding a predetermined object from the plurality of objects so that the rotation angle of the upper body part is within the predetermined rotation angle range. It is characterized by.

  According to a sixth aspect of the present invention, there is provided an instruction unit control means for controlling the movement of the instruction unit.

  According to a seventh aspect of the present invention, when the front surface of the upper body part is directed in the direction of the center of gravity, the upper body part is rotated when the rotation angle of the upper body part is equal to or greater than a predetermined rotation angle. It is characterized by comprising movement control means for moving the body part by moving means so as to be within the range of the rotation angle.

  According to the present invention, not only the main target object but also the subordinate target object can be given an impression as if it is focused, and thus a natural conversation according to the situation can be performed.

It is the schematic which shows the external appearance structure of the conversation robot of this invention. It is the schematic which shows the mode of operation | movement of the conversation robot when the user B newly joins. It is a block diagram which shows the circuit structure of the conversation apparatus mounted in the conversation robot. It is the schematic where it uses for description of how to obtain | require a gravity center direction. It is the schematic which shows the mode of operation | movement of the conversation robot when the user B detaches | leaves from a group conversation.

  Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings.

(1) Outline of the Present Invention In FIG. 1, reference numeral 1 denotes a conversation robot. The conversation robot 1 has an external appearance imitating a person and is connected to a body portion 2 so as to be rotatable. A portion 3 and arm portions 4a and 4b movably connected to the left and right of the body portion 2 are provided. In practice, the body portion 2 has a base 6 having movable wheels 5, and the upper body portion 8 is rotatable with respect to the base 6 in a rotation direction Ca with the z direction as a rotation axis. It has the structure provided in.

  The upper body portion 8 is provided with arm portions 4a and 4b on the upper left and right via shoulder joint portions 9a and 9b, and a head 3 that is movable vertically and horizontally is connected to a neck portion 8a provided on the upper portion. Yes. The arm 4a is provided with an upper arm 11 that is movable in the x, y, and z directions with respect to the upper body 8 by the shoulder joint 9a, and the elbow joint 12 is connected to the upper arm 11 through the elbow joint 12. The front heel part 13 is rotatably provided, and the front heel part 13 is provided with a hand part 15 via the wrist joint part 14 so as to be rotatable. Thereby, the arm portions 4a and 4b drive the shoulder joint portion 9a, the elbow joint portion 12 and the wrist joint portion 14 by respective actuators (not shown), and the upper arm portion 11, the forehead portion 13 and the hand portion 15 are moved. An action imitating the action of a person's arm, such as protruding toward the front side of the upper body part 8 (hereinafter simply referred to as the front side of the body part) 8b, can be performed.

  In addition, the head 3 imitates the eyes 21 and the mouth of a person, with the same side as the body front 8b as the front of the head 3 (hereinafter simply referred to as the head front) 3a. A mouth portion 22 is formed on the head front surface 3a. In addition, the head 3 is provided with a pair of CCD (Charge Coupled Device) cameras 23 that function as “eyes” at the eyes 21 of the head front surface 3a, and functions as an uttering device inside the mouth 22. A speaker (not shown) is provided.

  Further, the conversation robot 1 collects the user's voice with a microphone (not shown in FIG. 1), and utters an optimum response, question, etc. from the speaker according to the content of the user's utterance. Or by moving the head 3, the upper body 8, the arms 4a and 4b, and the wheels 5 so as to execute autonomous actions according to the situation with the user. In addition to such a configuration, the conversation robot 1 according to the present invention rotates the head 3 and the upper body 8 in the rotation direction Ca according to the number of users who have a conversation, and thus the head front 3a and the trunk. The direction in which the front surface 8b is directed is appropriately changed to an optimal direction so that a natural group conversation can be performed even if the number of users having a conversation increases. Hereinafter, the operation of the conversation robot 1 when performing a group conversation with a plurality of users will be described.

  Here, FIG. 2A shows a situation of a one-to-one conversation between the conversation robot 1 and the user A as viewed from above, whereas FIG. FIG. 4 shows a situation where a user B is newly added and a situation of group conversation between a plurality of persons performed between the conversation robot 1, the user A, and the user B is viewed from above. 2A and 2B, the “Δ” mark indicates the front face A1, B2 of the user A or the user B and the head front face 3a of the conversation robot 1, and the solid line arrows indicate the user A and the user A. A direction in which the front faces A1 and B2 of the user B and the head front face 3a of the conversation robot 1 are directed is shown. The dotted arrows indicate the directions in which the body fronts A2 and B2 of the user A and the user B and the body front 8b of the conversation robot 1 are directed.

  Incidentally, in the case of this embodiment, as an example of a method for collecting the voices of the user A and the user B, a dedicated microphone is prepared for each of the user A and the user B, and the predetermined parts of the user A and the user B are respectively provided. The attached microphone is connected to the conversation robot 1 via wiring. Thereby, the conversation robot 1 acquires each voice of the user A and the user B collected from the microphone as a voice signal, and recognizes the voice situation and the contents of the voice of the user A and the user B by analyzing the voice signal. Has been made to get.

  Here, as shown in FIG. 2A, when the conversation robot 1 has a conversation with the user A, the conversation robot 1 rotates the head 3 so that the head front 3a faces the user A. The body part 8 is also rotated so that the body part front face 8b faces the user A, and an operation is performed as if the user A is being watched. In this way, the conversation robot 1 directs the head front 3a and the torso front 8b to the user A so that the user can have an impression as if he / she clearly indicated that he / she would have a conversation with the user A. Can be given to A.

  In addition to such a configuration, the conversation robot 1 is connected to the user A during the conversation with the user A, for example, when the user B approaches, the face B1 or the body front of the user B based on the moving image captured by the CCD camera 23. If the direction of B2 is detected and it is recognized from this detection result that a part of user B's face front B1 and a part of body front B2 (described later) are facing the direction of conversation robot 1, user B It is possible to judge that a group conversation is newly performed by joining a conversation.

  When the conversation robot 1 recognizes that the conversation is a group conversation including the user A and the user B as shown in FIG. 2B, the participant of the group conversation is based on the moving image captured by the CCD camera 23. (User A and user B in FIG. 2B) identify the positions of all the members, calculate the center of gravity direction line CG1 (described later) from the positional relationship, and point the head front 3a toward the user A who is currently talking In this state, only the upper body portion 8 is rotated, and the body portion front surface 8b is directed toward the center of gravity direction indicated by the center of gravity direction line CG1. Thereby, the conversation robot 1 can give the user A and the user B the impression that the conversation with the user A is continued by directing the head front 3a to the user A. In addition to this, the conversation robot 1 looks not only at the user A but also at the user B by directing the body front 8b in the direction of the center of gravity obtained from the positions of the users A and B. A natural impression can be given to the user A and the user B, and a natural conversation between the user A and the user B can be realized.

(2) Circuit Configuration of Conversation Robot Next, the behavior of the conversation robot 1 as shown in FIGS. 2A and 2B will be described below using the circuit configuration shown in FIG. In the case of this embodiment, the conversation robot 1 incorporates a conversation device 30 as shown in FIG. 3. For example, the CCD camera 23 provided on the head front surface 3a images the head front surface 3a direction. The moving image obtained in this way is sent as moving image data to the face direction / body direction detection unit 32 and the face recognition unit 33, respectively. The face recognizing unit 33 identifies an almost elliptical skin color area from the moving image generated from the moving image data based on the skin color likelihood determined in advance by a statistical method, and uses this as the face area. Extract. The face recognition unit 33 normalizes this skin color area, and then calculates features (face feature amounts) such as the distance between both eyes and the width of the nose from the normalized image. Here, in the face recognition unit 33, face feature amounts such as the distance between both eyes of the user A and the user B and the width of the nose are stored in advance as face data. As a result, the face recognition unit 33 compares the face feature amount detected based on the moving image data with the feature amount of the registered face data, so that the user A and the user B exist in the moving image. Can be recognized and sent to the position detection unit 35 as face identification result data.

  Further, in the case of this embodiment, when the face direction / body direction detection unit 32 receives moving image data from the CCD camera 23, it is determined in advance by a statistical method from moving images generated from the moving image data. Based on the skin color likelihood, after identifying an almost elliptical skin color area and extracting it as a face area, normalize this skin color area, and the features such as the distance of both eyes and the width of the nose from this normalized image (Facial feature amount) is calculated. Further, the face direction / body direction detection unit 32 is configured to detect each of the user A and the user B based on the contour data of the upper body of the person predetermined by a statistical method from the moving images generated from the moving image data. After specifying the upper body and extracting it as an upper body region, the upper body region is normalized, and features such as shoulder width (upper body contour feature amount) are calculated from the normalized image. Here, the face orientation / body orientation detection unit 32 stores, for example, face orientation template information (described later) and body orientation template information (described later) in advance. Is used to identify the orientation of the face front A1, B1 from changes in the distance between the eyes, etc., and by comparing the body orientation template information with the upper body contour features, the body front A2, B2 It is made to be able to identify the direction of the.

  For example, the face orientation template information is statistical when the human face is in front of the CCD camera 23, when it is 30 degrees diagonally left and right, when it is 60 degrees diagonally right and left, and when it is 90 degrees diagonally right and left. A model of a facial feature value (hereinafter referred to as a statistical facial orientation feature value) is shown. By comparing these statistical facial orientation feature values with the facial feature values of the users A and B, the front of the face is shown. The orientations of A1 and B1 are identified, and this is obtained as face orientation identification data. The body orientation template information is statistically calculated when, for example, a person's upper body is front to the CCD camera, 30 degrees diagonally left and right, 60 degrees diagonally right and left, and 90 degrees diagonally right and left. A model of the upper body contour feature quantity (hereinafter referred to as a statistical body orientation feature quantity) is shown, and by comparing these statistical body orientation feature quantities with the upper body outline feature quantities of the user A and the user B, The directions of the body front faces A2 and B2 are identified, and this is obtained as body orientation identification data. Then, the face direction / body direction detection unit 32 sends the face direction identification data and the body direction identification data of the user A and the user B detected based on the moving image data to the role identification unit 36, respectively.

  Here, as a method for identifying the orientations of the front faces A1, B1 and the front faces A2, B2 of the user A and the user B, for example, “upper body contour extraction considering the appearance and shape variation tendency of the face and body”・ Technology described in “Tracking Method” (Naoya Hiroshi, Shinya Fujie, Tetsunori Kobayashi (“Image Recognition and Understanding Symposium (MIRU2010)”, July 2010)) may be applied. User A and User B Various other methods may be applied as long as the orientations of the face fronts A1 and B1 and the body fronts A2 and B2 can be identified.

  At this time, when the microphones 37a and 37b collect the voices of the user A and the user B, respectively, the microphones 37a and 37b send them to the voice processing unit 39 as voice signals. The voice processing unit 39 stores in advance the voice features (voice feature quantities) of the user A and the user B as voice identification data, and extracts the feature quantities from the respective voice signals received from the microphones 37a and 37b, By comparing the feature quantity with the voice identification data, it is possible to recognize which microphone 37a, 37b is used by the user A or the user B.

  Then, for example, the voice processing unit 39 generates voice identification result data indicating that one microphone 37a collects the voice of the user A and the other microphone 37b collects the voice of the user B. Is sent to the position detector 35. The speech processing unit 39 performs frame-synchronous continuous speech recognition using a bigram language model, HMM (Hidden Markov Model), and converts the speech signal into a sequence of words. This is sent to the action selection unit 40 as word string data.

  On the other hand, the position detection unit 35 associates the face identification result data received from the face recognition unit 33 with the voice identification result data received from the voice processing unit 39, so that the positions of the user A and the user B in the moving image are And the user A or the user B can be recognized.

  In practice, the position detection unit 35 associates the user A recognized in the moving image with the audio signal obtained by the one microphone 37a, and obtains the user B recognized in the moving image and the other microphone 37b. The voice signals are associated with each other, and it is recognized whether any of the user A or the user B in the moving image is speaking, and this is transmitted to the role identifying unit 36 as position detection result data.

  The role identification unit 36 can determine whether or not the user A and the user B are participating in the conversation from the face direction identification data and the body direction identification data received from the face direction / body direction detection unit 32. Yes. In practice, the role identifying unit 36 determines that, for example, a part of the face front A1 and a part of the body front A2 of the user A are obtained from the face orientation identification data and the body orientation identification data of the user A obtained from the moving image of the user A. Both of them determine whether or not they are facing the conversation robot 1 side. As a result, when a part of the face front A1 and a part of the body front A2 of the user A are facing the conversation robot 1 from the face orientation identification data and the body orientation identification data of the user A (for example, the conversation robot 1 In contrast, when the face of the user A is front, when the front left / right is 30 degrees diagonally, and when the front left / right is 60 degrees diagonally), this is because the user A has a group conversation with the conversation robot 1 or another user B. Then, it is determined that the face front A1 and the body front A2 are directed to the conversation robot 1 or another user B side, and the role identifying unit 36 recognizes that the user A is a participant in the group conversation.

  Similarly, for the user B, the role identifying unit 36 also determines a part of the user B's face front B1 and the body front from the user B's face orientation identification data and body orientation identification data obtained from the user B moving image. It is determined whether a part of B2 is facing the conversation robot 1 side. As a result, if the face direction identification data and the body direction identification data of the user B indicate that a part of the front face B1 and a part of the front face B2 of the user B face the conversation robot 1 side, this means that the user B Determines that the front face B1 and the front face B2 are facing the conversation robot 1 or the other user A side in order to perform a group conversation with the conversation robot 1 or the other user A, and the role identifying unit 36 Recognize that they are participants in a group conversation.

  Next, the role identification unit 36 determines whether one of the user A and the user B determined as a group conversation participant based on the position detection result data received from the position detection unit 35 is a speaker. Can be judged. For example, when the role identification unit 36 obtains an audio signal from the microphone 37a of the user A determined to be a participant in the group conversation based on the position detection result data received from the position detection unit 35, the role identification unit 36 utters the user A. A user (main attention object), the other user B as a listener (subordinate attention object), the roles of these users A and B (in this case, user A as a speaker (main attention object), The user B is a listener (subordinate attention object), and main / subject attention position detection data associated with the position detection result data is generated, and this is generated in the head drive control section 41 and the trunk section drive control section 42, respectively. Send it out.

  At this time, if the role identifying unit 36 determines that the orientation of the face front A1 of the user A is the user B direction based on the face orientation identification data of the user A who is the currently speaking speaker, for example, User A is currently speaking to user B, and after user A's utterance ends, user B estimates that there is a high probability of speaking because he / she replies, and this user B is assumed to be the next utterance estimator (user A's After the utterance is completed, the next utterance estimator (the next attention target estimation object is set as the main attention object) is sent to the head drive control unit 41 in association with the main / subject attention position detection data.

  Furthermore, when the role identification unit 36 receives the utterance content selected by the behavior selection unit 40 described later from the behavior selection unit 40 and recognizes that the conversation robot 1 is currently speaking, the role orientation / body orientation From the face orientation identification data and the body orientation identification data from the detection unit 32, it is determined whether or not there is a user A or a user B pointing the face front A1, B1 and the body front A2, B2 to the conversation robot 1. . As a result, the role identifying unit 36 uses the user A or the user B facing the conversation robot 1 as the face front A1, B1 and the body front A2, B2 as the main listener (main attention object), and detects the main / subject attention position. The data is sent to the head drive control unit 41 and the body drive control unit 42 in association with the data. Incidentally, in the case of this embodiment, when both user A and user B face front A1, B1 and body front A2, B2 are facing conversation robot 1, either user A or user B is randomly selected as the main listener. And is sent to the head drive control unit 41 in association with the main / subordinate attention position detection data.

  Upon receiving the master-slave attention position detection data from the role identification unit 36, the head drive control unit 41 receives the direction of the user A or the user B as the main target object such as the speaker or the main listener (hereinafter, this is the main target object). Then, the angle difference between the head front surface 3a and the main attention target direction is calculated. Actually, as shown in FIG. 4A, the head drive control unit 41 uses the center line (hereinafter referred to as the image frame center line) d of the imaging image frame 51 of the CCD camera 23 as the head front surface 3a. A main attention target direction line d1 extending in the body center direction of the user A, for example, as the main attention object is calculated from the angle center point O with the lower end of the image frame center line as the angle center point O. . The head drive control unit 41 calculates an angle θ1 from the image frame center line d to the main target object direction line d1 with reference to the angle center point O (hereinafter, referred to as a main target object angle) θ1, The main target angle θ1 is sent to the head actuator 43 as a head rotation command.

  In FIG. 4A, since the image frame center line d and the main attention target direction line d1 already coincide with each other, the main attention target angle θ1 is 0 degree. If the image frame center line d and the main attention target direction line d1 are deviated, the head actuator 43 rotates the head 3 by the main attention target angle θ1 based on the head rotation command. The frame center line d and the main attention target direction line d1 can be matched, and the head front surface 3a can be directed to the main attention target direction (FIGS. 2A and 2B).

  Incidentally, when the conversation robot 1 is talking only with the user A, the body drive control unit 42 determines the body part rotation angle necessary to rotate the body 8 to the main target angle θ1. The upper body part rotation angle is calculated and sent to the body part actuator 44 as a body part rotation command. The torso part actuator 44 rotates the torso part 8 by the torso part turning angle based on the torso part turning command, so that the torso part front face 8b is also directed in the main target direction toward which the head front face 3a faces. Can be directed (FIG. 2A).

  On the other hand, when the body part drive control unit 42 has conversation with not only the user A but also the user B and receives the master / slave attention position detection data from the role identification unit 36, as shown in FIG. Then, the main attention target direction line d1 extending in the body center direction of the user A as the main attention object is calculated from the angle center point O, and the main attention target direction line from the image frame center line d is calculated based on the angle center point O. The main target object angle θ1 up to d1 is calculated.

Further, at this time, the body part drive control unit 42 calculates, from the angle center point O, a secondary attention target direction line d2 extending in the body center direction of the user B as a secondary attention object, and displays the image based on the angle center point O. An angle θ2 from the frame center line d to the secondary attention target direction line d2 (hereinafter referred to as the secondary attention target angle) is calculated. Next, the body part drive control unit 42 adds up all of the main target target angle θ1 and the subordinate target target angle θ2, and divides by “2”, which is the total number of recognized users (user A and user B), An angle from the image frame center line d to the center of gravity direction line CG1 (hereinafter referred to as the center of gravity direction angle) θ _CG1 is calculated with respect to the angle center point O as a body part rotation command, and the body part actuator 44 To send.

As a result, the body part actuator 44 can turn the upper body part 8 by the center-of-gravity direction angle θ _CG1 based on the body part rotation command, thereby turning the body part front face 8b toward the center-of-gravity direction line CG1 (see FIG. 2 (A) and (B)). Thus, the conversation robot 1 turns only the upper body part 8 and directs the body part front face 8b toward the center of gravity so that it is as if all of the conversation robot 1, user A, and user B are having a group conversation. Such intention display can be made to the user A and the user B.

  Incidentally, in the above-described embodiment, as the group conversation, the operation of the conversation robot 1 when the user A and the user B are the participants of the group conversation has been described, but the present invention is not limited to this, The conversation robot 1 can perform the same operation even when three or four or more other persons are participating in the group conversation. For example, as shown in FIG. 4B, the operation of the conversation robot 1 when a user C is newly added in addition to the users A and B and three persons are group conversation participants will be described below.

  In this case, for the user C, the role identification unit 36 similarly uses a part of the face C1 and the body of the face C1 of the user C from the face orientation identification data and the body orientation identification data of the user C obtained from the moving image of the user C. It is determined whether a part of the front C2 is facing the conversation robot 1 side. As a result, if a part of the front face C1 and a part of the front face C2 of the user C face the conversation robot 1 side from the face orientation identification data and the body orientation identification data of the user C, this means that the user C Determines that the front face C1 and the front face C2 are facing the conversation robot 1 side in order to have a group conversation with the conversation robot 1 and other users A and B, and the role identification unit 36 determines that the user C is a group. Recognize that you are a participant in a conversation.

  Further, when receiving the main-slave attention position detection data from the role identification unit 36, the body part drive control unit 42, similar to the above, from the image frame center line d to the main target-of-interest direction line d1 based on the angle center point O. Is calculated, and the secondary attention target angle θ2 from the image frame center line d to the secondary attention target direction line d2 is calculated on the basis of the angle center point O. Here, as shown in FIG. 4B, in addition to user A and user B, user C is newly recognized. As a result, the torso drive control unit 42 calculates, from the angle center point O, the subordinate target direction line d3 extending in the trunk center direction of the user C as the subordinate target object, and the image frame based on the angle center point O. A slave target angle θ3 from the center line d to the slave target direction line d3 is calculated.

Then, the body drive control unit 42 adds up the main target object angle θ1, the subordinate target object angle θ2, and the subordinate target object angle θ3, and determines the total number of recognized users (user A, user B, and user C). Dividing by a certain “3”, a center-of-gravity direction angle θ _CG2 from the image frame center line d to the center-of-gravity direction line CG1 is calculated, and this is sent to the body part actuator 44 as a body part rotation command. That is, the body drive control unit 42 can calculate the center-of-gravity direction angle θ _CG from the image frame center line d based on the angle center point O based on the calculation formula of θ 1 + θ 2 + θ 3 +... + Θ n / n. (However, θ1 is the main target object angle, θ2 to θn are the subordinate target object angles, and n is the total number of recognized users).

  Incidentally, as shown in FIG. 3, when the action selection unit 40 receives word string data from the voice processing unit 39, the action selection unit 40 reads a keyword stored in advance in the database 47, and includes the keyword included in the word string data. And a template corresponding to the extracted keyword string is read out from the database 47 from predetermined templates. Thereby, the action selection unit 40 can determine what meaning the keyword string indicates from a predetermined template selected based on the keyword string. Here, in the database 47, an action pattern table in which action contents indicating action contents such as utterance contents uttered by the conversation robot 1 and moving the arms 4a and 4b are stored in advance for each template. Yes.

  Thereby, the action selection unit 40 selects an action pattern corresponding to the keyword string extracted from the character string data from the action pattern table based on the template, and a predetermined utterance associated with the selected action pattern. The contents and the action contents are read from the database 47, and the utterance contents are sent to the speech synthesizer 48, and the action contents are sent to each drive unit such as the arm portions 4a and 4b. The voice synthesizing unit 48 has a function of converting the utterance content given from the action selecting unit 40 into a voice signal, and sends the voice signal thus obtained to the speaker 49. As a result, sound based on the sound signal can be output from the speaker 49. In addition, the driving units such as the arm units 4a and 4b are autonomous in accordance with the situation, such as raising the hand unit 15 according to the utterances of the user A and the user B based on the operation content given from the action selecting unit 40. Operation can be realized.

  Incidentally, when the conversation robot 1 has a group conversation between the user A and the user B as shown in FIG. 2B, for example, as shown in FIG. When the direction is changed so that part of the face front B1 and part of the body front B2 do not face the robot 1, the user B is determined not to be a participant in the group conversation, as shown in FIG. 5 (B). The body part front face 8b is directed to the user A side, and a conversation with the user A is performed.

  In practice, the role identifying unit 36 determines that a part of the user B's face front B1 and a part of the body front B2 are obtained from the user B's face orientation identification data and body orientation identification data obtained from the user B's moving image. If it is determined that it is not suitable for the conversation robot 1 (for example, when the face of the user B is 90 degrees or more in front of the conversation robot 1), the master-slave indicating that the user B is a participant in the group conversation Attention position detection data is generated and sent to the body part drive control unit 42.

  The body part drive control unit 42 calculates a body part rotation angle necessary to rotate the body part 8 to the main target object angle θ1, and uses this body part rotation angle as a body part rotation command. It is sent to the body actuator 44. The torso part actuator 44 rotates the torso part 8 by the torso part turning angle based on the torso part turning command, so that the torso part front face 8b is also directed in the main target direction toward which the head front face 3a faces. Can be directed (FIG. 5B). Thereby, even if the user B leaves the group conversation, the conversation robot 1 can indicate the intention to continue the conversation with the user A by directing the trunk front 8b toward the user A. Can have a natural conversation.

(3) Operation and Effect In the above configuration, the conversation robot 1 recognizes which of the user A and the user B is the speaker based on the voices and moving images of the user A and the user B, and the user A Or, depending on the utterance content of the user B, the utterance content uttered from the speaker 49 and the behavior pattern such as the operation of the arms 4a, 4b can be changed to realize autonomous behavior according to the conversation with the user A or the user B. .

  In the conversation robot 1, of the user A and the user B, the utterer, the main listener, or the next utterance estimator after the utterance is recognized as the main target object, and the head in the direction of the main target object The head 3 is rotated so that the front 3a faces. Thereby, in the conversation robot 1, according to the operation of the user A or the user B as the main target object, it is possible to give an impression as if the user is gazing to talk with the user A or the user B. A natural conversation according to the situation can be performed with the user A or the user B.

In addition to this, the conversation robot 1 calculates the main target object angle θ1 and the subordinate target object angle θ2, sums up all of the main target object angle θ1 and the subordinate target object angle θ2, and then recognizes the user (user) The center-of-gravity direction angle θ _CG1 is calculated by dividing by the total number of A and user B), and the body portion front face 8b is rotated to the center-of-gravity direction angle θ _CG1 . In this way, in the conversation robot 1, by rotating the trunk front 8b to the center of gravity direction angle _θCG1 , the trunk front 8b is directed toward the center of gravity, so that not only the user A but also the user B It is possible to give an impression as if the user is paying attention, and thus a natural conversation according to the situation can be realized with the user A or the user B.

(4) Other Embodiments The present invention is not limited to this embodiment, and various modifications can be made within the scope of the gist of the present invention. For example, the present invention resembles an animal such as a dog. A conversation robot may be applied. In the above-described embodiment, the case where the user A is the main target object and the user B and the user C are the subordinate target objects among the users A and B as the target objects is described. The invention is not limited to this. For example, a bulletin board on which characters can be written with a writing instrument, a display device on which various types of information are displayed, a bulletin-type printed matter, etc., are used as a main attention object or a secondary attention object (object) Also good.

  In this case, when the conversation robot 1 recognizes the board as the main target object, and recognizes the user B as the subordinate target object, the conversation robot 1 uses an instruction unit control means (not shown) that controls the arms 4a and 4b to move. The movement of the arm portions 4a and 4b may be controlled so that the arm portions 4a and 4b as the instruction portions are directed toward the board as the main target object.

At this time, the conversation robot 1 uses the trunk drive control unit 42 to determine the target attention angle θ1 determined from the position of the board as the target attention object and the secondary attention determined from the position of the user B as the secondary attention object. All the target angles θ2 are added together and divided by “2”, which is the total number of recognized objects (board and user B), and from the frame center line d to the center of gravity direction line CG based on the angle center point O The center-of-gravity direction angle θ _CG is calculated and sent to the body part actuator 44 as a body part rotation command.

Thereby, in the conversation robot 1, the body part actuator 44 rotates the body part 8 by the center-of-gravity direction angle θ _CG based on the body part rotation command, and the body part front 8b faces the center-of-gravity direction line CG1. be able to. Thus, by turning only the upper body part 8 of the conversation robot 1 and directing the front part 8b of the body part toward the center of gravity, it is as if the conversation robot 1 indicates the board with the arms 4a and 4b. A part of 8 is directed to the user B side, and an impression can be given as if a conversation is being performed with the user B while paying attention to the board.

  Further, in the above-described embodiment, the direction of the center of gravity is obtained based on the positions of the user A and the user B in the imaging image frame 51, and only the upper body part 8 is rotated without moving the base 6, thereby the center of gravity. However, the present invention is not limited to this. For example, the center of gravity direction is obtained based on the positions of the user A and the user B in the imaging frame 51, and the center of gravity is obtained. When the rotation angle for directing the body portion front surface 8b in the direction is equal to or greater than a predetermined rotation angle, the movement control means controls the rotation of the upper body portion 8 to be within a predetermined rotation angle range. The body 5 is rotated in the rotation angle direction by rotating the wheel 5 of the base 6 and rotating the body portion 2 itself in the rotation angle direction, or by moving the body portion 2 parallel to the rotation angle direction side. 2 may be moved automatically.

  Furthermore, in the above-described embodiment, the case where the direction of the center of gravity is obtained from the positions of all the users A and B in the imaging frame 51 and the upper body part 8 is rotated in the direction of the center of gravity has been described. However, the present invention is not limited to this, and when the body front 8b is directed in the direction of the center of gravity obtained from the positions of all the users A and B in the imaging image frame 51, the upper body portion 8 exceeds a predetermined rotation angle. When the user A or the user B needs to be rotated, the user A or the user B existing at a position greater than or equal to the predetermined rotation angle is excluded so as to be within the predetermined rotation angle. The center-of-gravity direction may be obtained by excluding the portion.

  Further, in the above-described embodiment, as an external sensor, a CCD camera 23 as an imaging unit is provided in the eye 21 of the conversation robot 1, and the user A and the user A in the imaging image frame 51 captured by the CCD camera 23 are used. Although the case where the direction of the center of gravity is obtained from the position of B has been described, the present invention is not limited to this. For example, a camera as an imaging unit is installed on the indoor ceiling as an external sensor, and the user A, the user B, and the conversation robot 1 May be captured from above by a camera, the center of gravity direction may be obtained from the positions of the users A and B in the captured image, and the body front 8b of the conversation robot 1 may be directed toward the center of gravity.

  In addition, as another form of the above-described embodiment, for example, regarding the specification of the sound source direction, for example, “sound source localization using two microphones installed on the robot head” 470 1999 Tetsunori Kobayashi, Daisuke Miyata, Kayo Matsuzaka) ”. In this case, two microphones are installed in the head 3, and the sound source localization is based on the signal received by each microphone. After obtaining the approximate direction of arrival, the moving image obtained by the CCD camera 23 may be subjected to image processing to search for a user in that direction and to be recognized as a speaker.

1 Conversation robot 2 Torso 3 Head
4a, 4b Arm (instruction part)
5 wheels (moving means)
8 upper body
23 CCD camera (external sensor)
37a, 37b Microphone (external sensor)
35 Position detector (position detection means)
36 Role identification part (role identification means)
41 Head drive control unit (head control means)
42 Fuselage part drive control part (torso part control means)

Claims (7)

A body part rotatably provided on the body part and a head part rotatably provided on the body part, and autonomously with a plurality of objects based on an output result from an external sensor In a conversation robot that performs group conversations ,
Position detecting means for detecting positions of a plurality of previous objects based on an output result obtained from the external sensor;
On the basis of the output result acquired from the external sensor, recognizes one of the plurality of objects as the main object of interest, recognizing the role identification means other remaining of the object as従注th object,
The centroid direction indicated in the previous SL main object of interest and the center of gravity direction line obtained from the position of the従注eye object, body to control the rotation angle of the upper body portion so as to direct a front of the upper body portion Part control means;
A head control means for controlling a rotation angle of the head so that the front of the head faces the direction of the main target object ;
Based on the data from the role identifying means, the torso part control means and the head control means are configured so that the front of the upper body part is oriented in the center of gravity direction indicated by the center of gravity direction line. The upper body part can be controlled so that the front of the head is directed toward the target object by rotating only the head, and the front of the head is directed toward the main target object. The conversation robot is characterized in that it can be controlled so that the front of the upper body part is directed in the direction of the center of gravity indicated by the center of gravity direction line . The plurality of objects are users;
The role identification means is
Based on the image and / or sound acquired from the external sensor, among the plurality of users, the user who speaks or is the main listener is recognized as the main target object, and the other remaining users are the subordinate attention. The conversation robot according to claim 1, wherein the conversation robot is recognized as an object. The role identification means is
The next attention target estimation object which the user recognized as the main attention object recognizes is recognized, and the user sets the next attention object estimation object as a new main attention object after the utterance ends. 2. The conversation robot according to 2. The center-of-gravity direction angle formed by the center-of-gravity direction line and a predetermined reference line is
The main target object angle formed by the main target object direction line extending in the direction of the main target object and the reference line, and the sub target object angle formed by the reference line and the sub target object direction line extending in the direction of the sub target object To calculate
4. The method according to claim 1, wherein a value obtained by adding all the main attention target angles and the sub attention object angles is calculated by dividing the sum by the total number of the plurality of objects. 5. Conversation robot. The body part control means includes:
When the front surface of the upper body part is directed in the direction of the center of gravity obtained from the plurality of objects, when the rotation angle of the upper body part is equal to or greater than a predetermined rotation angle, the rotation angle of the upper body part is 5. The center-of-gravity direction is calculated by excluding a predetermined object from the plurality of objects so as to be within a predetermined rotation angle range. Conversation robot. The conversation robot according to any one of claims 1 to 5, further comprising instruction unit control means for controlling movement of the instruction unit. When the front surface of the upper body part is directed in the direction of the center of gravity, the rotation of the upper body part is within the range of the rotation angle when the rotation angle of the upper body part is equal to or greater than a predetermined rotation angle. The conversation robot according to claim 1, further comprising movement control means for moving the body portion by movement means.

Images