JP2015148701A - Robot control device, robot control method and robot control program - Google Patents

Abstract

PROBLEM TO BE SOLVED: To achieve an action as if a robot views a video picture together with a user, and to cause the robot to generate empathy for the user.SOLUTION: A robot control device is configured to: acquire a comment about a program under viewing from a social media server 7; determine an utterance content caused to be uttered by a robot 2 from comments of a personality match utterer matching a personality set in the robot 2; and extract an action content caused to be executed by the robot 2 from an action database 15 on the basis of a dialogue state of the utterance content and an emotional state of the robot 2. Thus, the robot control device allows the robot 2 to execute the action in accordance with the content of the program under viewing. When determining the utterance content, the robot control device narrows dowm the comments by the personality set in the robot to allow the robot 2 to execute an utterance and action with consistency.

Description

  The present invention relates to a technique for controlling a robot.

  It is known that when a person expresses an emotion in a certain expression between humans, the happiness expression of the subject is promoted and the anger / sadness expression is weakened by the synchronized reaction. Therefore, co-viewing the video with the robot and co-responding to the video content together, such as laughter, joy, sadness, and anger, promotes emotions such as laughter and joy rather than watching the video alone. It is possible to suppress emotions such as sadness and anger.

  Further, in communication research between a person and a CG person, it has been pointed out that an affinity motivation is given to a person by causing the CG person to perform a facial expression change that gives empathy. Affinity motivation is defined as a desire to approach, cooperate with, or reward the other person, and people are considered to have affinity motivation for others who have a similar attitude to them. Yes.

  In Non-Patent Document 1, as a technique for making a user feel that a robot has sympathy with a robot during video viewing, the user's evaluation of the viewing program is estimated as a profile from the user's viewing program log and the utterance being viewed, and the user is viewing However, if the robot seems to be bored, a technique is disclosed in which the robot recommends another TV program using a profile, thereby generating user empathy for the robot.

  Further, in Non-Patent Document 2, the robot uses the comments on the social media regarding the viewing program as the utterance sentence, performs the dialogue toward the user, and the robot posts the utterance from the user as a comment on the social media. A technique in which a robot acts as an intermediary for social media is disclosed.

Tatsuhashi Takahashi and two others, “Social media brokering robot for increasing speech opportunities for the elderly”, IEICE Technical Report, IEICE, October, 112, 233, pp.21- 26 Yasushi Takama and 5 others, “Human-Robot Communication Based on Recommendation of Information when Watching TV”, 21st Annual Conference of Japanese Society for Artificial Intelligence, Japan Society for Artificial Intelligence, 2007, 2D5-5

  However, Non-Patent Document 1 only introduces other programs based on user evaluation in units of programs, and cannot express emotions or speak about the content of the currently viewed video. Also, emotional expressions and utterances regarding user operations such as power ON / OFF for video and display and volume adjustment are only for channel changes, so the effect of creating empathy was limited.

  Also, even in Non-Patent Document 2, emotional expression by action according to television contents is not realized. Concerning the utterance contents, it is difficult to make the utterances of robots consistent when using social media, and even in Non-Patent Document 2, there is a negative impression that robots utter inconsistent utterances. Yes. As a cause of negative impressions of inconsistent utterances, for example, a robot that was speaking with a female speaking utterance on a program being watched with a user suddenly uttered a male utterance It seems that the agency of the robot that had been watching TV together until then collapsed. Therefore, it is important to make the utterance consistent. In Non-Patent Document 2, it is stated that the method of realizing consistency is realized by interpreting the semantic attributes of the text on social media as a “sequential method”, and extracting and speaking only consistent opinions. However, it is generally not easy to extract only consistent opinions from social media, and no concrete implementation method is described. Further, Non-Patent Document 2 does not describe emotion expressions and utterances for user operations. Therefore, even in Non-Patent Document 2, the effect of generating empathy was limited.

  The present invention has been made in view of the above, and an object of the present invention is to realize an action as if a robot is watching a video together with a user, so that the robot can sympathize with the user.

  A robot control apparatus according to a first aspect of the present invention is a robot control apparatus that causes a robot to perform an operation of viewing a video together with a user, and a comment acquisition unit that acquires a comment posted on the video; An utterance sentence generating means for generating an utterance sentence to be uttered by the robot from the comment, an emotion determination means for determining an emotion state of the robot from the comment, an interactive state of the utterance sentence to be uttered by the robot, and an emotion of the robot Action storage means for storing the state and the action to be executed by the robot in association with each other, and referring to the action storage means, the dialogue state of the utterance sentence generated by the utterance sentence generation means and the emotion state determined by the emotion determination means Action deciding means for deciding an action to be executed by the robot, and generating the spoken sentence The utterance sentence stage generated causes output to the robot by voice synthesis, and having a control means for executing the operation based on the action the action determining means has determined the robot.

  In the robot control device, the robot control device has personality storage means for storing personality information set for the robot, and the speech sentence generation means generates the speech sentence from the comment posted by a poster who matches the personality. It is characterized by doing.

  The robot control apparatus includes a direction acquisition unit that acquires the direction of the user, and the action includes an action of viewing the direction of the user.

  A robot control method according to a second aspect of the present invention is a robot control method by a computer that causes a robot to perform an operation of viewing a video together with a user, and obtaining a comment posted on the video; Generating an utterance sentence to be uttered by the robot from the comment; determining an emotion state of the robot from the comment; an interaction state of an utterance sentence to be uttered by the robot; an emotion state of the robot; and Referring to the action storage means stored in association with the action to be executed, the robot is caused to execute from the conversation state of the utterance sentence generated in the step of generating the utterance sentence and the emotion state determined in the step of determining the emotion state A step of determining an action, and a step of generating the spoken sentence. The spoken sentence causes output to the robot by voice synthesis resulting in, and having the steps of: executing the operation based on the action determined in the robot in the step of determining the action.

  A robot control program according to a third aspect of the present invention is a robot control program for causing a robot to perform an operation of viewing a video together with a user, the process for obtaining a comment posted on the video, and the comment Processing for generating an utterance sentence to be uttered by the robot, processing for determining the emotional state of the robot from the comment, dialogue state of the utterance sentence to be uttered by the robot, emotion state of the robot, and causing the robot to execute An action to be executed by the robot from the emotion state determined in the conversation state of the utterance sentence generated in the process of generating the utterance sentence and the emotion state determined in the process of determining the emotion state with reference to the action storage means stored in association with the action Speech synthesis of the utterance sentence generated in the process of determining and the process of generating the utterance sentence It causes outputs to the robot, characterized in that to perform the operation based on the action determined in the process of determining the actions and processes to be executed by the robot, to the computer.

  According to the present invention, it is possible to realize an action as if the robot is watching a video together with the user, and the robot can generate empathy for the user.

It is a whole block diagram including the robot control apparatus in this Embodiment. It is a figure which shows the example of robot personality attribute information. It is a figure which shows the example of the action determination table and control sequence table which are stored in an action database. It is a figure which shows the example of the fixed form speech sentence stored in a fixed form speech sentence database. It is a figure which shows the example of the data stored in a positive / negative word database. It is a figure which shows the example of the data stored in a tone conversion database. It is a figure which shows the example of the data stored in a program related speech / action tag database. It is a figure which shows the example of the data which an electronic program guide information database hold | maintains. It is a figure which shows the example of the data which a program-social media tag related database hold | maintains. It is a figure which shows the example of the social media comment information which a social media server hold | maintains. It is a figure which shows the example of the user display direction information which a position acquisition server hold | maintains. It is a flowchart which shows the flow of the process which determines speech content and action content based on a display state change. It is a flowchart which shows the flow of a process which acquires an excitement value and personality matching speaker comment information. It is a flowchart which shows the flow of the process which determines speech content and action content based on an excitement value and personality matching speaker comment information. It is a figure which shows the map of the emotional state based on a climax value and a positive / negative value. It is a flowchart which shows the flow of the process which determines utterance content and action content based on the scenario accompanying a program. It is a flowchart which shows the flow of the process which makes a robot perform speech and an action based on the determined speech content and action content. It is a figure which shows the mode of the robot controlled by the robot control apparatus.

  Hereinafter, embodiments of the present invention will be described with reference to the drawings.

  FIG. 1 is an overall configuration diagram including a robot control apparatus according to the present embodiment.

  The robot control apparatus 1 according to the present embodiment changes the state of the display 4 having a TV program display function operated by the remote controller 3 (for example, power ON / OFF, volume change, etc.), viewing posted on the social media server 7. This is a device for controlling the robot 2 by determining the utterance / action content based on the comment associated with the channel (= program) in the middle and the scenario associated with the channel being viewed.

[Robot controller configuration]
First, the configuration of the robot control apparatus 1 in the present embodiment will be described.

  The robot control apparatus 1 includes a display information processing unit 11, a social media information acquisition unit 12, an utterance / action determination unit 13, a robot personality attribute information database 14, an action database 15, a fixed utterance sentence database 16, a positive / negative word database 17, and a tone conversion. A database 18 and a program-related utterance / action tag database 19 are provided. Each unit included in the robot control device 1 may be configured by a computer including an arithmetic processing device, a storage device, and the like, and the processing of each unit may be executed by a program. This program is stored in a storage device included in the robot control apparatus 1 and can be recorded on a recording medium such as a magnetic disk, an optical disk, or a semiconductor memory, or provided through a network. In FIG. 1, the robot control device 1 and the robot 2 are shown separately, but the robot control device 1 may be incorporated in the robot 2.

  The display information processing unit 11 acquires display state change information and viewing channel change information including the content of the operation of the display 4 by the remote controller 3, and the display state change information is transmitted to the utterance / action determination unit 13 to change the viewing channel. When the information is acquired, information on a channel to be newly viewed is acquired and transmitted to the social media information acquisition unit 12. If the utterance / action tag information related to the channel being viewed exists in the program-related utterance / action tag database 19, the utterance / action tag information related to the channel being viewed is acquired and transmitted to the utterance / action determination unit 13. .

  The social media information acquisition unit 12 receives information on the channel being viewed from the display information processing unit 11, acquires a comment regarding the channel being viewed from the social media server 7, and has a personality that matches the personality set for the robot. An excitement value indicating the excitement level of the comment of the speaker (hereinafter referred to as “personality matching speaker”) and the channel being viewed is obtained and transmitted to the utterance / action determination unit 13.

  The utterance / action determination unit 13 determines whether the robot is based on the display state change information and the utterance / action tag information received from the display information processing unit 11, the personality match speaker comment information received from the social media information acquisition unit 12, and the excitement value. The robot controls the robot by determining the utterance content to be uttered and the action content to be performed by the robot. The robot personality set in the robot 2 is taken into consideration when determining the utterance / action content.

[Data held by the robot controller]
Next, data held by the robot control apparatus 1 will be described.

  The robot personality attribute information database 14 stores robot personality attribute information representing the personality set for the robot. The robot personality attribute information is used when the social media information acquisition unit 12 extracts personality-matching speakers and when the utterance / action determination unit 13 determines the utterance / action content.

  FIG. 2 shows an example of the robot personality attribute information stored in the robot personality attribute information database 14. The robot personality attribute information includes two types of personality attribute information, that is, a video-dependent robot personality shown in FIG. 2A and a fixed robot personality shown in FIG.

  In the video-dependent robot personality, a set of one or more attributes, attribute values, and attribute weights is set for each category. The category is composed of items representing the type of video such as sports, news, and dramas, and some categories have subcategories. For example, when the main category is sports, soccer, baseball, basketball, and the like are subcategories. The attribute represents an element of hobbies or preferences that an individual can have with respect to the program, and includes, for example, a favorite team in a soccer program, a favorite player, a favorite genre in a news program, and the like. The attribute value is an item of a specific hobby or preference for the attribute. For example, the attribute value is the team B attribute for the favorite team attribute, and the attribute value is the entertainment value for the favorite genre attribute. The The attribute weight is given to each attribute, and is a weight indicating how much the attribute value match of the attribute contributes to the personality similarity determination.

  The fixed robot personality is a fixed personality that does not depend on the image, and is composed of a set of attributes, attribute values, and attribute weights. If the attribute is gender or age, and the attribute is gender, male or female is set as the attribute value, and if the attribute is age, the twenties, thirties, and so on are set. The attribute weight is a weight for each attribute similar to the video-dependent robot personality.

  The action database 15 stores an action determination table used when the utterance / action determination unit 13 determines the action content of the robot 2 and a control sequence table describing a robot control sequence for each action.

  FIG. 3A is an example of an action determination table, and FIG. 3B is an example of a control sequence table.

  The action determination table stores data including a dialogue state trigger, an emotion state trigger, an action name, and an execution speed. The conversation state trigger includes either “talk” or “impression”. The conversation state is determined by the utterance content determined by the utterance / action determination unit 13, and is spoken if the utterance content is a content to be spoken, and is an impression if the content is to describe an impression. The emotional state trigger includes values of emotional states set in the robot 2, such as joy and surprise. The emotional state is determined according to the content of the utterance / action determined by the utterance / action determining unit 13 and stored in a storage area included in the utterance / action determining unit 13. The action name is a label for identifying an action to be performed by the robot 2 such as nodding, shaking his / her head, or taking a long time. The execution speed is a parameter related to the speed when executing the motion specified by the action name. The larger the value, the faster the action is executed. The utterance / action determination unit 13 refers to the action determination table and determines the action content to be made to the robot 2 based on the dialogue state and the emotion state.

  The control sequence table stores data in which operation names and actuator control sequences are set. The operation name is a label corresponding to the action determination table, and indicates a state of the robot operation when the robot executes a series of actuator control sequences. The actuator control sequence includes a list of data including motor control locations and values, and sequence movement intervals, in addition to whether or not user orientation is necessary and whether or not display orientation is necessary. Whether or not the user orientation is necessary and whether or not the display orientation is necessary specify whether or not the user orientation and the display orientation are necessary when performing the corresponding action. The motor control location and value include an actuator location to be controlled and an angle value set for the actuator, such as a head tilt angle of 0 degrees and a left arm tilt angle of 0 degrees. For example, in the case of the nod operation shown in FIG. 3B, the motor control location and the value data start from a head tilt angle of 0 degrees, a head tilt angle of −40 degrees, and a head tilt angle of 0 degrees in a list format. The actuator control is performed until the specified angle is reached in order from the left side. The sequence movement interval includes a value indicating a transition interval at the time of transition from the motor control location and value in the same row to the next motor control location and value. The smaller the value, the faster the transition speed.

  The fixed utterance sentence database 16 stores fixed utterance sentences describing the utterance contents of the robot 2 when the state of the display 4 changes. When the utterance / action determination unit 13 receives the display state change information, the utterance / action determination unit 13 refers to the standard utterance sentence database 16 and makes the robot 2 utter from the content of the display state change information, the current emotion state, and the robot personality. And get the emotional state at run time.

  FIG. 4 shows an example of a fixed utterance sentence stored in the fixed utterance sentence database 16. The fixed utterance sentence shown in the figure consists of a set of data including a television state transition, an original emotion state, an emotion state to be executed, a dialogue state, an utterance content, and a robot personality. The television state transition includes a change in the state of the display 4 that has been changed by the operation of the display 4, such as power ON, power OFF, channel change, high volume, and low volume. The original emotional state and the emotional state to be executed contain values indicating the emotional state of the robot such as surprise and joy. The original emotional state may include a plurality of emotional states such as ALL representing all emotional states, fatigue, sleepiness, and sadness. The dialogue state is an utterance attitude when executing the utterance content, and a value of talking or impression is entered. The content of the utterance contains a character string actually spoken by the robot, such as “Let's watch TV together”. The robot personality contains a value indicating the personality that matches the utterance content. In the example of FIG. 4, sex and age are described.

  The positive / negative word database 17 stores data indicating whether the meaning of the word is positive or negative. When the utterance / action determination unit 13 receives personality match speaker comment information, the utterance / action determination unit 13 refers to the positive / negative word database 17 to determine whether a word in the comment is positive or negative, and thus a positive / negative value. Is calculated. The positive / negative value is used to determine the emotional state together with the excitement value. It is also used to determine the level of voice to be uttered by the robot 2.

  FIG. 5 shows an example of data stored in the positive / negative word database 17. The data stored in the positive / negative word database 17 is composed of three elements: word, positive, and negative. The words include great, beautiful, and regretful words. If the word has a positive meaning, 1 is entered in the positive column, 0 is entered in the negative column, and if the word has a negative meaning, 0 is entered in the positive column, and 1 is entered in the negative column.

  The tone conversion database 18 stores conversion data for converting the tone of the utterance content. The utterance / action determination unit 13 determines the utterance contents from the personality matching speaker comment information, and refers to the tone conversion database 18 to convert the tone of the utterance contents.

  FIG. 6 shows an example of data stored in the tone conversion database 18. The data stored in the tone conversion database 18 includes a set of conversion sources and conversion destinations. Both the conversion source and the conversion destination are character strings.

  The program-related utterance / action tag database 19 stores utterance contents to be uttered by the robot 2 in accordance with the program shown on the display 4. The display information processing unit 11 acquires information on the channel being viewed, determines whether or not the data of the corresponding program is stored in the program-related utterance / action tag database 19, and corresponds to the program being viewed. If the data is stored, the data is transmitted to the utterance / action determination unit 13, and the utterance / action determination unit 13 determines the utterance content to be uttered by the robot 2 based on the reproduction time of the program and the robot personality. .

  FIG. 7 shows an example of data stored in the program-related utterance / action tag database 19. The data stored in the program-related utterance / action tag database 19 includes channel information, program name, operation start time, execution action, emotion state to be executed, utterance content, and robot personality. The channel information represents a television program channel. The program name indicates the name of the television program being performed on each channel. The operation start time is a time indicating the timing at which the execution action and the content of the utterance are performed based on the start time of the program. The execution action indicates the name of the operation executed by the robot. The emotional state to be executed indicates the emotional state of the robot when the action is executed. The utterance content is a text indicating an utterance sentence uttered by the robot simultaneously with the execution of the action. The robot personality is a value indicating the personality according to the execution action and the content of the utterance. In the example of FIG. 7, sex and age are described.

[External data used by the robot controller]
Subsequently, data held by an external server or an external database used by the robot control apparatus 1 will be described.

  FIG. 8 is a diagram showing an example of data held in the electronic program guide information database 5. The display information processing unit 11 refers to the electronic program guide information database 5 and acquires the program name and category of the channel being viewed on the display 4.

  The electronic program guide information shown in FIG. 8 includes a set of channel information, category, program name, start time, and end time. The electronic program guide information indicates from what time to what time each program is broadcast on which channel. The category is information indicating the classification according to the program content.

  FIG. 9 is a diagram illustrating an example of data held in the program-social media tag related database 6. The program-social media tag related database 6 is used by the social media information acquisition unit 12 to extract comments related to the program being viewed by the user.

  The program-social media tag related database 6 shown in FIG. 9 holds a program name and a set of a plurality of program related tags. A program-related tag is a common character string that is intentionally included in a comment to indicate that a user using a social media server has commented on a specific television program. In the example of FIG. 9, the social media tag is a single-byte English uppercase character string starting with the symbol #. By extracting a comment including the program-related tag, only a comment related to the program can be extracted.

  FIG. 10 is a diagram illustrating an example of social media comment information held by the social media server 7. The social media information acquisition unit 12 accesses the social media server 7 and acquires social media comment information.

  The social media comment information shown in FIG. 10 includes a set of user ID, comment time, and comment content. The user ID is a unique ID given to each user who posts a comment to the social media server 7. The comment time is the time when the user of the social media transmits the comment to the social media server 7. The comment content is a character string of a comment transmitted to the social media server 7 by the user.

  FIG. 11 is a diagram illustrating an example of user display orientation information held by the position acquisition server 8. The user / display direction information is information indicating the direction from the robot 2 to the user and the display 4, and is used when the utterance / action determination unit 13 causes the robot 2 to execute an action.

  The user display azimuth information shown in FIG. 11 has an azimuth angle and an elevation angle for each of the two objects of the user and the display 4. The azimuth angle indicates in which direction each object from the robot 2 is located when the north is 0 ° in the horizontal direction on the ground. The elevation angle indicates the angle at which each object from the robot 2 is at 0 ° in the direction perpendicular to the ground and 90 ° directly above the ground. It is assumed that the azimuth angle and elevation angle of each object are sequentially updated according to the movement of the user, the robot 2 and the display 4.

[Robot controller operation]
Next, the operation of the robot control device 1 will be described. Below, the process which acquires operation with respect to the display 4, the process which determines speech content and action content, and the process which makes the robot 2 perform speech and action are demonstrated in order.

[Process to acquire operation for display]
First, a process for acquiring a user operation on the display 4 will be described.

  When the user operates the remote controller 3, the remote controller 3 transmits a display state operation signal or a channel operation signal corresponding to the operation content to the display 4. The display state operation is an operation for changing the state of the display 4 such as turning the display 4 on and off, increasing the volume, and decreasing the volume. The channel operation is a signal for changing the video displayed on the display, and is an operation for changing to another channel such as 1ch to 2ch, for example.

  The remote controller 3 transmits a signal to the display 4 and transmits display state change information and viewing channel change information to the robot controller 1. The display state change information includes a character string (for example, power ON, volume level, etc.) indicating the operation content for the display 4. The viewing channel change information includes channel information after the change.

  When receiving the viewing channel change information and the display state change information, the robot control device 1 executes processing for determining the utterance content and action content described later.

  In the present embodiment, the viewing channel change information and the display state change information are transmitted from the remote controller 3 to the robot control apparatus 1. However, the robot control apparatus 1 is transmitted from the remote control 3 to the display 4. The received signal may be converted into viewing channel change information and display state change information by receiving the signal and processing in the robot control apparatus 1. For example, when the display 4 is operated by a wireless LAN using a smartphone application as the remote controller 3, the robot control device 1 is added to the transmission destination of the operation information.

  The display 4 may have a function of transmitting viewing channel change information and display state change information to the robot control device 1. For example, when a change in display state is detected in response to a TV operation by an infrared remote controller, an infrared signal is received from the infrared remote controller near the infrared light receiving port of the display 4, and the operation content indicated by the infrared signal is wirelessly controlled by the robot. Transmit to device 1.

  Further, when the robot 2 mediates the display operation according to the user's voice or remote control operation, a change in the display state is detected from the user voice or remote control operation on the robot. Alternatively, the robot 2 may notify the robot controller 1 of the display state.

[Process to determine utterance content and action content]
Next, processing for determining the utterance content to be uttered by the robot 2 and the action content to be executed will be described.

  When receiving the display state change information and the viewing channel change information, the robot control device 1 determines the utterance content and the action content suitable for the display state change and the channel being viewed. The robot control apparatus 1 according to the present embodiment uses three methods: (A) a method based on a display state change, (B) a method using social media, and (C) a method based on a scenario associated with a program. Determine the action content. Hereinafter, the processes (A) to (C) will be described in order. In addition, you may use any method of (A)-(C), and may combine. It is preferable to provide the method of using the social media of (B) in that it is not an operation according to a predetermined rule.

(A) Method Based on Display State Change First, a method for determining speech content and action content based on display state change will be described.

  FIG. 12 is a flowchart showing a flow of processing for determining the utterance content and the action content based on the display state change.

  Upon receiving the display state change information from the remote controller 3, the display information processing unit 11 transmits the display state change information to the utterance / action determining unit 13 (step S11).

  When the utterance / action determination unit 13 receives the display state change information, the utterance / action determination unit 13 acquires the emotional state from the storage area of the utterance / action determination unit 13 (step S12). Note that the emotional state is not stored when the robot 2 is caused to act for the first time after power-on or initialization.

  The utterance / action determination unit 13 extracts the fixed utterance sentence information corresponding to the robot personality, the emotional state, and the received display state change information from the fixed utterance sentence database 16 (step S13). Specifically, the utterance / action determination unit 13 acquires the robot personality attribute information set for the robot 2 from the robot personality attribute information database 14 and refers to the standard utterance sentence database 16 to obtain the emotion acquired in step S12. The standard utterance text information in which the state and the original emotional state value in the standard utterance text database 16 match and the attribute value in the robot personality attribute information matches the personality attribute value in the standard utterance text database 16 is extracted. Then, in the extracted standard utterance text information, the standard utterance text information whose TV state transition value matches the received display state change information is extracted. The utterance content of the fixed utterance sentence information extracted in step S13 is the utterance content that the robot 2 utters. It should be noted that a plurality of original emotional states set in the standard utterance sentence database 16 match if the acquired emotional states are included. When ALL is set as the original emotional state, it is assumed that they match regardless of the emotional state. If there are a plurality of results extracted in step S13, one of them is selected at random.

  Then, the value of the emotional state executed by the fixed utterance sentence information extracted in step S13 is stored as an emotional state in the storage area of the utterance / action determining unit 13 (step S14).

  Subsequently, the action content to be executed by the robot 2 is determined.

  The utterance / action determination unit 13 extracts an action corresponding to the dialogue state and emotional state of the utterance content from the action database 15 (step S15). Specifically, the utterance / action determination unit 13 refers to the action determination table of the action database 15, the dialog state trigger of the action database 15 matches the dialog state of the fixed utterance sentence information extracted in step S 13, and the action Data whose emotional state trigger in the database 15 matches the emotional state executed by the fixed utterance sentence information extracted in step S13 is extracted. When a plurality of data matches, one of them is selected at random. If there is no matching data, no action is taken.

  Then, the operation name of the data extracted from the action determination table in step S15 is searched from the control sequence table of the action database 15. The actuator control sequence of the retrieved data is the action content that the robot 2 executes. A process for controlling the robot 2 based on the action content will be described later.

  Through the above processing, the utterance content and action content considering the robot personality are determined based on the received display state change information. After the utterance / action content is determined, a process for causing the robot described later to execute the utterance and action is executed.

(B) Method Using Social Media Next, a method for determining utterance content and action content using social media will be described.

  FIG. 13 is a flowchart showing a flow of a process of acquiring the excitement value and the personality matching speaker comment information among the processes of determining the utterance content and the action content using social media. Through the processing described later, the utterance / action determination unit 13 determines the utterance content and the action content based on the excitement value and the personality matching speaker comment information.

  When the display information processing unit 11 receives the viewing channel change information from the remote controller 3 (step S21), the display information processing unit 11 refers to the electronic program guide information database 5 and acquires the data of the currently viewed program from the received viewing channel change information and the current time. (Step S22).

  Then, the display information processing unit 11 refers to the program-social media tag related database 6 and acquires a program related tag related to the program being viewed (step S23). The display information processing unit 11 transmits the category of the program being viewed and the program-related tag to the social media information acquisition unit 12 as program domain information and program-related social media tag information.

  When receiving the program domain information and the program related social media tag information, the social media information acquisition unit 12 acquires social media comment information including the received program related social media tag information from the social media server 7 (step S24).

  The social media information acquisition unit 12 calculates a climax value from the acquired social media comment information (step S25). In this embodiment, the excitement value is calculated based on the increase / decrease in the number of comments corresponding to the time of each scene. Specifically, the total number x of social media comments posted within one minute from the current time, the average value μ of the number of comments per minute from the start of the program to the current time, and 1 from the start of the program to the current time Using the variance value σ of the number of comments counted every minute, a rising value is obtained by the following equation (1).

  When the value calculated by Equation (1) is less than −1.0, the rising value is −1.0, and when the value is higher than 1.0, the rising value is 1.0. When the value calculated by the expression (1) is within the range of -1.0 to 1.0, the value is set as a rising value.

  Then, the social media information acquisition unit 12 refers to the robot personality attribute information database 14 and extracts the robot personality attribute information that matches the received program domain information (step S26). In the coincidence determination between the program domain information and the robot personality attribute information, it is first determined whether there is a match in the main category from the video-dependent robot personality. If there is no matching main category, it is determined whether or not there is a match in all subcategories. If there is a matching main category and there are subcategories in the main category, select one of the subcategories at random and combine the attributes, attribute values, attribute weights of the selected subcategory with a fixed robot personality. Thus, the robot personality attribute information used for specifying the personality matching speaker is used. If there is no subcategory in the matching main category, or if there is no matching main category and there is a matching subcategory, combine the category's attributes, attribute values, attribute weights, and a fixed robot personality to create a personality matching speaker. Let it be the robot personality attribute information used for identification. If there is no category that matches the program domain information in the video-dependent robot personality, only the fixed robot personality is used as the robot personality attribute information used to identify the personality matching speaker.

  And the social media information acquisition part 12 extracts a personality matching speaker using the extracted robot personality attribute information (step S27). Specifically, first, the attribute values of all users of the social media comment information acquired in step S24 are estimated for each attribute of the robot personality attribute information. The technique described in Jun ITO, “What is he / she like ?: Estimating Twitter User Attributes from Contents and Social Neighbors” is used to estimate the attribute value of each attribute of all users. Then, using the attribute values in each attribute of all the estimated users and the attribute values in each attribute of the robot personality attribute information, the degree of coincidence of each user and the robot's personality is calculated, and the degree of coincidence exceeding a predetermined value Are designated as personality matching speakers. In calculating the degree of match, for each user and the same attribute of the robot personality attribute information, the attribute value estimation result of each user and the attribute value of the robot personality attribute information are compared, and the attribute weight of the attribute with the matching attribute value Calculate the sum of The sum of the attribute weights divided by the number of attributes of the robot personality attribute information is used as the degree of coincidence between the personality of the user and the robot.

  Then, the comment of the personality matching speaker is extracted (step S28). The extracted comment of the personality matching speaker is transmitted to the utterance / action determination unit 13 together with the excitement value as personality matching speaker comment information.

  Through the above processing, personality matching speaker comment information matching the excitement value of the program being watched by the user and the personality set in the robot is transmitted to the utterance / action determining unit 13. Subsequently, processing in which the utterance / action determination unit 13 determines the utterance content and the action content will be described.

  FIG. 14 is a flowchart showing a flow of processing for determining speech content and action content based on the excitement value and personality match speaker comment information among the processing for determining speech content and action content using social media.

The utterance / action determination unit 13 refers to the positive / negative word database 17 and calculates a positive / negative value using the received personality matching speaker comment information (step S31). Specifically, the morphological analysis is performed on all comment contents of the personality matching speaker comment information, and each word of the comment contents subjected to the morphological analysis matches the word of the positive / negative word information stored in the positive / negative word database 17. Determine whether there is something to do. When the number of all words subjected to morphological analysis is wordNMB and the i-th number of all words subjected to morphological analysis is w _i , a positive / negative value PN is obtained by the following equation (2).

In equation (2), J (w _i ) is one when w _i matches the word of the positive / negative word information, and positive is 1 and negative is 0, and w _i is the positive / negative word information. This is a function that matches a word and returns -1 if positive is 0 and negative is 1, and returns 0 otherwise. As a result of the calculation of Expression (2), PN = 100 when PN> 100, and PN = −100 when PN <−100. When −100 ≦ PN ≦ 100, the calculated PN is used as a positive / negative value as it is.

  Then, the utterance / action determination unit 13 determines the utterance content from the received personality matching speaker comment information (step S32). Specifically, first, a comment posted within a certain time from the current time is extracted from personality matching speaker comment information, and morphological analysis is performed on the extracted comment. Then, TF-IDF values are calculated for all the words in the comment subjected to morphological analysis. At this time, TF is the number of occurrences of each word in all words of the comment subjected to morphological analysis, and IDF is calculated from a general document corpus. A typical document corpus includes a newspaper corpus. And the total value of TF-IDF value of each word which each comment sentence has is calculated, and the comment having the largest total value is determined as the utterance content.

Then, the utterance / action determination unit 13 determines the emotional state using the excitement value and the positive / negative value (step S33). The emotional state is determined based on the emotional state map based on the positive value and the positive value shown in FIG. 15 applying Russell's emotional ring model (James A. Russell, “A Circumplex Model of Affect”). By applying positive and negative values, the closest emotional state among the emotional states shown in the map is set as the robot's emotional state. In the map of FIG. 15, the maximum value of the climax value is ActMAX = 1, the minimum value is ActMIN-1, the maximum value of the positive / negative value is PNMAX = 100, and the minimum value is PNMIN = -100. The emotional distance EmDist _n with the emotional state EM _n is given by _{assuming that} the rising value of the emotional state EM _{n in} the map of FIG. 15 is Act _n , the positive / negative value is PN _n , the rising value is Act and the positive / negative value is PN. Is represented by the following equation (3).

Using equation (3), the emotional distance EmDist _n with all emotional states EM _n in the map is calculated, and the emotional state EM _n having the smallest emotional distance EmDist _n is set as the emotional state of the robot. The determined emotional state is stored in the storage area of the utterance / action determination unit 13.

  In addition, as a method of determining the emotional state of the robot 2, a method of analogizing emotions from sound in a video (JP 2009-111938 A, JP 2009-251469 A) or a method of analogizing emotions from video (special May be used to extract emotions of a particular character.

  Then, the utterance / action determination unit 13 performs noise removal and tone conversion on the utterance content determined in step S32 (step S34). Noise removal removes unnecessary social media tags and emoticons using symbols when speaking. When removing the social media tag, the character string is removed according to the rule representing the social media tag. In the present embodiment, single-byte English uppercase character strings starting with # are removed. In emoticon removal, emoticons in the utterance content are extracted and removed using an emoticon analysis system described in Michal Ptaszynski, “CAO: A Fully Automatic Emoticon Analysis System Based on Theory of Kinesics”. If the utterance content includes a word registered as a conversion source in the tone conversion database 18, the corresponding character string is replaced with the conversion destination character string.

  Then, the utterance / action determination unit 13 determines an action to be executed by the robot 2 from the utterance content converted in step S34 and the emotion state determined in step S33 (step S35). First, it is determined whether or not the utterance content is a question sentence, and the dialog state is specified. If the utterance content ends with a question mark or “ka” or “kana”, it is determined to be a question sentence. If the utterance content is a question sentence, the conversation state is set as a talk, and if not, the dialog state is set as an impression. Then, similarly to step S15 of the method based on the display state change, the action contents to be executed by the robot 2 are extracted by extracting data in which the dialogue state and the emotion state respectively match the dialogue state trigger and the emotion state trigger in the action database 15. decide.

  Through the above processing, the utterance content and action content considering the robot personality are determined using the comments posted on the social media server 7. After the utterance / action content is determined, a process for causing the robot described later to execute the utterance and action is executed.

  In the above, the comment including the tag related to the program is extracted from the social media server 7 where the user freely posts the comment and the utterance / action content is determined. However, the comment related to the program is read from the electronic bulletin board installed for each channel. May be extracted to determine the utterance / action content.

  In addition, when viewing a video to which a comment is given corresponding to the playback time of the video, rather than a program broadcast in real time, the comment given to the video can also be used.

(C) Method Based on Scenario Attached to Program Next, a method of determining the utterance content and action content based on the scenario associated with the program will be described.

  FIG. 16 is a flowchart showing a flow of processing for determining the utterance content and the action content based on the scenario accompanying the program.

  The display information processing unit 11 refers to the program-related utterance / action tag database 19 and extracts utterance / action tag information corresponding to the program being viewed (step S41). The program being viewed is identified with reference to the electronic program guide information database 5 as in step S22 of the method using social media. The display information processing unit 11 extracts, from the program-related utterance / action tag database 19, utterance / action tag information that matches the currently viewed channel and program name.

  The display information processing unit 11 transmits the extracted utterance / action tag information that has a near operation start time to the utterance / action determination unit 13 (step S42). For example, when the difference between the time obtained by adding the operation start time of the extracted utterance / action tag information to the start time of the program being viewed and the current time is within a predetermined time (about 10 seconds), the corresponding utterance / action tag The information is transmitted to the utterance / action determination unit 13 as utterance / action tag candidate information. The process of step S42 is repeated until the display 4 is turned off or the program being viewed is changed. If the program being viewed has been changed, the process returns to step S41.

  Upon receiving the utterance / action tag candidate information, the utterance / action determination unit 13 selects the utterance / action tag information suitable for the robot personality set in the robot 2 from the utterance / action tag candidate information, The action content is determined (step S43). Specifically, among the utterance / action tag information received as the utterance / action tag candidate information, the utterance / action tag information having the robot personality attribute value that matches the robot personality stored in the robot personality attribute information database 14 is obtained. select. When there are a plurality of matching utterance / action tag information, one of them is selected at random. The utterance content of the selected utterance / action tag information is determined as the utterance content that causes the robot 2 to utter, and the execution action of the utterance / action tag information is determined as an action that causes the robot 2 to execute. If there is no matching utterance / action tag information, there is no utterance content and no action content.

  When the utterance / action determination unit 13 determines the action to be executed, the utterance / action determination unit 13 searches the control sequence table of the action database 15 for data having the action to be executed as an operation name. The actuator control sequence of the retrieved data is the action content that the robot 2 executes. In addition, data having an action to be executed as an operation name is retrieved from the action determination table of the action database 15 to obtain an action execution speed.

  Further, the utterance / action determination unit 13 stores the emotional state executed by the utterance / action tag information selected in step S43 as an emotional state in the storage area of the utterance / action determination unit 13 (step S44).

  Through the above processing, the utterance content and action content considering the robot personality are determined based on the scenario associated with the program stored in the program-related utterance / action tag database 19. After the utterance / action content is determined, a process for causing the robot described later to execute the utterance and action is executed.

[Process to make robot execute speech and action]
Next, processing for causing the robot 2 to execute utterance and action based on the determined utterance content and action content will be described.

  FIG. 17 is a flowchart showing a flow of processing for causing the robot 2 to execute speech and action based on the determined speech content and action content.

  First, the utterance / action determination unit 13 determines whether the action to be executed by the robot 2 requires user orientation information or display orientation information (step S51). The utterance / action determination unit 13 determines whether or not the user orientation information is necessary and the display orientation information is necessary in the actuator control sequence of the action to be executed by the robot 2. If neither user orientation information is required nor display orientation information is required, the process proceeds to step S54.

When user orientation information or display orientation information is required, the user orientation information and display orientation information are acquired from the position acquisition server 8 (step S52), and are substituted into the necessary portions of the actuator control sequence of actions to be executed by the robot 2. (Step S53). For example, in the “see user” action shown in FIG. 3B, the azimuth angle of the user azimuth information is substituted for the head tilt angle y ₁ degree of the actuator control sequence, and the head roll angle y of the actuator control sequence is set. _{At 2} degrees, the elevation angle of the user orientation information is substituted. Further, in the action of “seeing the user with the right hand pointing at the display”, the azimuth angle of the user azimuth information is substituted for the head tilt angle y ₁ degree of the actuator control sequence, and the head roll angle y ₂ degrees of the actuator control sequence Is substituted with the elevation angle of the user orientation information. Further, the azimuth angle of the display azimuth information is substituted for the right arm tilt angle d ₁ degree of the actuator control sequence, and the elevation angle of the display azimuth information is substituted for the right arm roll angle d ₂ degrees of the actuator control sequence.

  As a method for acquiring user orientation information and display orientation information for the robot 2, an indoor position measurement method can be used. For example, Y. Gu, “A Survey of Indoor Positioning Systems for Wireless Personal Networks”. By attaching indoor position information of the user, the robot 2 and the display 4 acquired by this method, and a sensor capable of acquiring the orientation to the robot 2, line-of-sight control and pointing control are performed in the user display direction with respect to the robot 2. be able to. As another method, there is a method of identifying the user display by attaching a camera to the robot 2 and performing image processing. User orientation information and display orientation information viewed from the robot 2 are acquired from the user display position in the camera image.

  After acquiring the actuator control sequence of the action to be executed by the robot 2, the utterance / action determination unit 13 synthesizes speech with the utterance content (step S54). When performing speech synthesis, the pitch of the sound, the speaking speed, and the magnitude of the sound are determined according to the emotional state. The utterance / action determination unit 13 acquires the emotional state from the storage area, and obtains the excitement value and the positive / negative value corresponding to the emotional state from the map of FIG.

The maximum value of the climax value is ActMAX, the minimum value is ActMIN, the maximum value of the positive / negative value is PNMAX, the minimum value is PNMIN, the climax value corresponding to the emotional state obtained from the map of FIG. 15 is Act _m , and the positive / negative value Is PN _m , the maximum value of the sound pitch in speech synthesis is STH_MAX, the minimum value is STH_MIN, the maximum value of speech rate is SS_MAX, the minimum value is SS_MIN, the maximum value of sound volume is SV_MAX, and the minimum value is SV_MIN Then, the sound pitch STH _m , speech rate SS _m , and sound volume SV _m in speech synthesis can be obtained by the following equations (4) to (6), respectively.

  The speech content is synthesized according to the pitch, the speaking speed, and the loudness obtained by the equations (4) to (6). The result of speech synthesis is generated as a speech file.

  Then, the utterance / action determination unit 13 transmits an actuator control signal and a voice file to the robot 2 (step S55). The actuator control signal is generated from the actuator control sequence and includes an actuator part to be controlled, a control angle, and a time until the control angle is reached.

  The robot 2 controls a specified portion of the actuator with a specified angle and time based on the received actuator control signal, reproduces the received audio file, and outputs it from a speaker included in the robot 2.

  FIG. 18 is a diagram illustrating a state of the robot 2 controlled by the robot control device 1. FIG. 18A is a diagram illustrating a state in which the line of sight is controlled to be directed toward the display 4, and FIG. 18B is directed to the direction of the user while pointing the direction of the display 4. It is a figure which shows a mode that it was controlled to.

  By controlling the robot 2 so that its face and eyes are directed toward the display 4, it is possible to naturally notify the user that the robot 2 is paying attention to the display 4. Similarly, although not shown, by controlling the robot 2 so that its face and eyes are directed toward the user, it is possible to naturally inform the robot 2 that the user is paying attention.

  In addition, when the robot 2 pays attention to the user and speaks the video content, the robot 2 controls the user so that his / her face or eyes are directed toward the user and the display 4 is pointed to the display 4. It is possible to naturally inform the user that the content is being spoken.

  In this way, by performing line-of-sight control and pointing control of the robot toward the user and the display 4, the user perceives that the robot 2 recognizes the presence of the user and the display 4, and the robot 2 It is possible to give the user a feeling as if viewing video.

  As described above, according to the present embodiment, the utterance content to be uttered by the robot 2 is determined based on the operation content on the display 4 and the emotional state of the robot 2 with reference to the fixed utterance sentence database 16 and the utterance. By extracting the action content corresponding to the dialogue state of the content and the emotional state of the robot 2 from the action database 15, it is possible to cause the robot 2 to execute an action corresponding to the state change of the display 4.

  According to the present embodiment, a comment regarding the program being viewed is acquired from the social media server 7, and the utterance content to be uttered by the robot 2 is determined from the comment of the personality matching speaker that matches the personality set in the robot 2. At the same time, by extracting from the action database 15 the action content to be executed by the robot 2 based on the conversation state of the utterance content and the emotional state of the robot 2, the robot 2 is caused to execute an action corresponding to the content of the program being viewed. Is possible. As a result, the user can have a feeling of watching the program together with the robot 2, and the robot can give empathy to the user. When the user feels that the user has sympathized with the robot when viewing the video, the user accepts the user when the robot 2 is given a motivation for the robot 2 and the robot 2 performs a product, service, action recommendation, etc. for the user. It can be expected to increase the probability.

  According to the present embodiment, when utterance contents are determined, it is possible to cause the robot 2 to execute consistent utterances / actions by narrowing down comments by the personality set in the robot.

  According to the present embodiment, the utterance / action tag information related to the program being viewed is acquired from the program-related utterance / action tag database 19, and the utterance content to be uttered by the robot 2 and the action content to be executed are determined. Thus, it is possible to cause the robot to execute an action along the program that the user is viewing.

  According to the present embodiment, the robot 2 is controlled in accordance with the user direction and the display direction, and the user is given a sense that the robot 2 is both viewing the display 4 and a user when speaking. It is possible to make the robot 2 recognize the video and the presence of the user, and to give more empathy to the viewing content.

DESCRIPTION OF SYMBOLS 1 ... Robot control apparatus 11 ... Display information processing part 12 ... Social media information acquisition part 13 ... Utterance and action determination part 14 ... Robot personality attribute information database 15 ... Action database 16 ... Regular utterance sentence database 17 ... Positive negative word database 18 ... Tone Conversion database 19 ... Program related utterance / action tag database 2 ... Robot 3 ... Remote control 4 ... Display 5 ... Electronic program guide information database 6 ... Program-social media tag related database 7 ... Social media server 8 ... Location acquisition server

Claims (5)

A robot control device that causes a robot to perform an operation such as viewing a video with a user,
Comment acquisition means for acquiring a comment posted to the video;
An utterance sentence generating means for generating an utterance sentence to be uttered by the robot from the comment;
Emotion determining means for determining the emotional state of the robot from the comment;
Action storage means for storing the dialogue state of the utterance sentence to be uttered by the robot, the emotional state of the robot, and the action to be executed by the robot in association with each other;
Referring to the action accumulating means, action determining means for determining an action to be executed by the robot from the dialogue state of the spoken sentence generated by the utterance sentence generating means and the emotion state determined by the emotion determining means;
Control means for causing the robot to perform an operation based on the action determined by the action determination means, while causing the robot to synthesize and output the speech sentence generated by the speech sentence generation means;
A robot control apparatus comprising: Having personality storage means for storing personality information set in the robot;
The robot control apparatus according to claim 1, wherein the utterance sentence generation unit generates the utterance sentence from the comment posted by a contributor who matches the personality. Direction acquisition means for acquiring the direction of the user;
The robot control apparatus according to claim 1, wherein the action includes an action of looking at a direction of the user. A robot control method by a computer that causes a robot to perform an operation such as viewing a video with a user,
Obtaining a comment posted to the video;
Generating an utterance sentence to be uttered by the robot from the comment;
Determining an emotional state of the robot from the comments;
The dialogue of the utterance sentence generated in the step of generating the utterance sentence with reference to the action storage means stored in association with the dialogue state of the utterance sentence to be uttered by the robot, the emotion state of the robot and the action to be executed by the robot Determining an action to be executed by the robot from the state and the emotional state determined in the step of determining the emotional state;
Synthesizing and outputting the utterance sentence generated in the step of generating the utterance sentence to the robot, and causing the robot to perform an action based on the action determined in the step of determining the action;
A robot control method comprising: A robot control program for causing a robot to perform an operation such as viewing a video with a user,
Processing for obtaining a comment posted to the video;
Processing to generate an utterance sentence to be uttered by the robot from the comment;
A process for determining the emotional state of the robot from the comment;
Dialogue of the utterance sentence generated by the process of generating the utterance sentence with reference to the action storage means stored in association with the conversation state of the utterance sentence to be uttered by the robot, the emotion state of the robot and the action to be executed by the robot A process for determining an action to be executed by the robot from the emotional state determined by the process for determining the state and the emotional state;
A process of causing the robot to perform an action based on the action determined in the process of determining the action, while synthesizing and outputting the speech sentence generated in the process of generating the spoken sentence to the robot;
A robot control program for causing a computer to execute.

Images