WO2018203501A1

WO2018203501A1 - Object control system and object control method

Info

Publication number: WO2018203501A1
Application number: PCT/JP2018/016759
Authority: WO
Inventors: 真一本多; 大場　章男; 博之勢川
Original assignee: 株式会社ソニー・インタラクティブエンタテインメント
Priority date: 2017-05-02
Filing date: 2018-04-25
Publication date: 2018-11-08
Also published as: JP2018187712A; US11759959B2; JP6814089B2; US20200114520A1

Abstract

A feeling deduction unit 100 deduces a user's feeling. An internal state management unit 110 manages the internal state of an object and the internal state of the user on the basis of the user's deduced feeling. An action management unit 120 determines the action of the object on the basis of the internal state of the object. An output processing unit 140 causes the object to carry out the action determined by the action management unit 120.

Description

Object control system and object control method

The present invention relates to a technique for controlling the behavior of an object that is an actual object or a virtual object.

”It is said that in order to build a good relationship with others, not“ face-to-face relationships ”but“ co-view experience ”in which the same thing is seen together is effective. It is known that by looking at the same thing at the same place and empathizing with each other, the distance to others is reduced and the sense of familiarity is increased.

The present inventor has paid attention to the possibility of utilizing the robot as a user's experience of co-viewing. For example, during game play by the user, the robot watching the game play beside the user, and being happy or sad with the user increases the user's affinity for the robot, and motivation to play the game. It is expected to improve. Further, not only for games but also for movies, TV programs, and the like, it is expected that the user can enjoy the content more by watching with the robot than when watching alone.

The present invention has been made in view of these problems, and an object of the present invention is to provide a technique by which a user can obtain a co-view experience with an object such as a robot.

In order to solve the above problems, an aspect of the present invention is an object control system that controls an object, and an emotion estimation unit that estimates a user's emotion and a user that stores an internal state of the user including the user's emotion Internal state storage unit, object internal state storage unit that stores the internal state of the object including the emotion of the object, and internal state management that manages the internal state of the object and the internal state of the user based on the estimated user emotion A behavior management unit that determines the behavior of the object based on the internal state of the object, and an output processing unit that causes the object to perform the behavior determined by the behavior management unit.

Another aspect of the present invention is an object control method. The method includes estimating a user's emotion, managing a user's internal state including the user's emotion, managing an object's internal state including the object's emotion, and estimating the user's emotion. Basically, the method includes updating the internal state of the object and the internal state of the user, determining the behavior of the object based on the internal state of the object, and causing the object to perform the determined behavior.

Note that any combination of the above components, the expression of the present invention converted between a method, an apparatus, a system, a computer program, a recording medium on which the computer program is recorded so as to be readable, a data structure, and the like are also included in the present invention. It is effective as an embodiment of

It is a figure which shows the example of an external appearance of an actual object. It is a figure for demonstrating the outline | summary of an object control system. It is a figure which shows the input-output system of a robot. It is a figure which shows the structure of the object control system which controls an object. It is a figure which shows the example of the external appearance shape of HMD.

The object control system according to the embodiment provides a mechanism for realizing a co-view experience with an actual object or a virtual object. The actual object may be a human-type or pet-type robot, and it is preferable that at least sound can be output, and a motor is provided at the joint part to move an arm, leg, neck, or the like. The user puts a robot beside them and watches the content together, and the robot outputs a reaction that sympathizes with the user based on the estimated emotion of the user, or outputs a reaction that repels the user, Communicate with users.

The virtual object may be a character such as a person or a pet composed of a 3D model, and exists in a virtual space generated by a computer. In the embodiment, in a virtual space constructed when the user wears a head mounted display (HMD), the content is played in front of the user, and when the user turns sideways, the virtual character watches the content together. We propose a mechanism that allows you to see how you are. The virtual character also communicates with the user by outputting a reaction that sympathizes with the user, or by outputting a reaction that repels the user.

Fig. 1 shows an example of the appearance of an actual object. This object is a humanoid robot 20 and includes a speaker that outputs sound, a microphone that inputs sound of the outside world, a motor that moves each joint, and a drive mechanism that includes a link that connects the motors. The robot 20 preferably has an interactive function of having a conversation with the user, and more preferably has an autonomous movement function.

FIG. 2 is a diagram for explaining the outline of the object control system 1. FIG. 2 shows a situation where the user sits on the sofa and plays a game, and the robot 20 sits on the same sofa and watches the user's game play. Ideally, the robot 20 should have a high level of autonomous movement function that allows the user to sit on the sofa. If the robot 20 does not have such a function, the user can bring the robot 20 to the sofa and Sit sideways. The robot 20 may participate in the game as a virtual player, and for example, may virtually operate the opponent team of the baseball game that the user is playing.

The information processing apparatus 10 receives operation information input to the input device 12 by the user and executes an application such as a game. The information processing apparatus 10 may be capable of reproducing content media such as a DVD or streaming content from a content server by connecting to a network. The camera 14 is a stereo camera, captures a user sitting in front of the display device 11, which is a television, at a predetermined cycle, and supplies the captured image to the information processing apparatus 10.

The object control system 1 controls the behavior of the robot 20 that is a real object by estimating the emotion from the reaction of the user. The robot 20 that supports the user's game play is basically controlled in behavior so as to express sympathy to the user happily together with the user and to give the user a common vision experience. Note that the object control system 1 of the embodiment manages the internal state of the robot 20 and the user. For example, even if the user is pleased, if the preference of the robot 20 to the user is low, the object control system 1 is not pleased (not sympathized) together. Control behavior. When the robot 20 virtually participates in the game as the user's opponent, the behavior may be controlled so as to be sad if the user is happy. The robot 20 basically expresses emotions by outputting the utterance content indicating emotions as voice, but may express emotions by moving the body.

FIG. 3 shows the input / output system of the robot 20. The control unit 30 is a main processor that processes and outputs various data such as voice data and sensor data and commands. The microphone 22 collects ambient sounds and converts them into audio signals, and the camera 24 captures the surroundings to obtain a captured image. The storage unit 26 temporarily stores data, commands, and the like that are processed by the control unit 30. The communication unit 32 transmits data output from the control unit 30 to the external information processing apparatus 10 by wireless communication via the antenna. In addition, the communication unit 32 receives the audio data and the drive data from the information processing apparatus 10 via the antenna through wireless communication and outputs them to the control unit 30.

When the control unit 30 receives the sound data, the control unit 30 supplies the sound to the speaker 36 and outputs the sound. When the control data is received, the control unit 30 rotates the motor of the drive mechanism 34. The drive mechanism 34 includes a motor incorporated in a joint part that is a movable part of the robot 20 and a link mechanism that connects the motors, and the arm, leg, neck, and the like of the robot 20 are moved by driving the motor. .

FIG. 4 shows the configuration of an object control system that controls objects. The object control system 1 includes an emotion estimation unit 100, an internal state management unit 110, an action management unit 120, an internal state storage unit 130, and an output processing unit 140. The internal state storage unit 130 includes an object internal state storage unit 132 and a user internal state storage unit 134. The emotion estimation unit 100 includes a first emotion estimation unit 102, a second emotion estimation unit 104, and a third emotion estimation unit 106, and performs a process of estimating emotions in three systems.

The object control system 1 also includes a camera 14 that captures the user, a biosensor 15 that detects the user's biometric information, a motion sensor 16 that detects the user's movement, a microphone 17 that obtains an audio signal around the user, and an event occurrence. It has the event detection part 40 to detect.

In FIG. 4, each element described as a functional block for performing various processes can be configured by a circuit block, a memory, and other LSIs in terms of hardware, and loaded into the memory in terms of software. Realized by programs. Therefore, it is understood by those skilled in the art that these functional blocks can be realized in various forms by hardware only, software only, or a combination thereof, and is not limited to any one.

4, each configuration of the emotion estimation unit 100, the internal state management unit 110, the behavior management unit 120, the internal state storage unit 130, and the output processing unit 140 is provided in the information processing apparatus 10, and the information processing apparatus 10 The behavior of the robot 20 may be controlled. Each configuration of the emotion estimation unit 100, the internal state management unit 110, the behavior management unit 120, the internal state storage unit 130, and the output processing unit 140 is provided in the robot 20, and the robot 20 autonomously performs its own behavior. You may control. In this case, the robot 20 is configured as an autonomous robot and can act alone. The emotion estimation unit 100, the internal state management unit 110, the behavior management unit 120, the internal state storage unit 130, and a part of the output processing unit 140 may be provided in the information processing apparatus 10 and the rest may be provided in the robot 20. . In this case, the information processing apparatus 10 and the robot 20 work together to control the behavior of the robot 20.

As described above, the object control system 1 shown in FIG. 4 may be realized in various modes. When each of the above configurations is provided in the information processing apparatus 10, the audio signal acquired by the microphone 22 of the robot 20 and the captured image captured by the camera 24 are transmitted to the information processing apparatus 10 for use in processing in the information processing apparatus 10. May be. When each of the above components is provided in the robot 20, the audio signal acquired by the microphone 17 connected to the information processing apparatus 10 and the captured image captured by the camera 14 are transmitted to the robot 20 for use in processing in the robot 20. It's okay. In addition, an audio signal or a photographed image may be acquired by a microphone or camera other than those illustrated, and provided to the information processing apparatus 10 or the robot 20. In the following, for convenience of explanation, the camera 14 representatively captures the state of the user, and the microphone 17 collects the user's utterance and converts it into an audio signal.

The emotion estimation unit 100 estimates a user's emotion based on outputs from various sensors and data from an external server. The emotion estimation unit 100 estimates a user's emotion by deriving an evaluation value for each of emotion indexes such as joy, anger, affection, and surprise. In the embodiment, the user's emotion is expressed by a simple model, and the emotion estimation unit 100 converts the user's emotion into a good emotion (positive emotion), a bad emotion (negative emotion), a good or bad emotion (neutral). Estimate from three evaluation values. In practice, each of “positive” and “negative” may be subdivided into a plurality of stages so that a fine evaluation value can be derived.

The first emotion estimation unit 102 is a non-language based on paralinguistic information or non-linguistic information based on outputs of various sensors, that is, outputs of the camera 14, the biological sensor 15, the motion sensor 16 and the microphone 17 which are image sensors. Understand behavior and estimate user's emotions. Compared with the second emotion estimation unit 104, the first emotion estimation unit 102 is characterized by estimating the user's emotion based on the utterance content.

The first emotion estimation unit 102 extracts changes in facial expression, line-of-sight direction, hand gesture, and the like of the user's face from the captured image of the camera 14, and estimates the user's emotion. The biological sensor 15 is attached to the user and detects biological information such as the user's heart rate and sweating state, and the first emotion estimation unit 102 derives the user's psychological state from the biological information and estimates the user's emotion. To do. The motion sensor 16 is attached to the user and detects the user's movement, and the first emotion estimation unit 102 estimates the user's emotion from the user's movement. The role of the motion sensor 16 can be substituted by analyzing the captured image of the camera 14. Further, the first emotion estimation unit 102 estimates the user's emotion from the voice signal of the microphone 17 using the feature amount of the paralinguistic information. Paralinguistic information includes information such as speech speed, volume, voice inflection, intonation, and wording. The first emotion estimation unit 102 collects the user's emotions estimated from the outputs of the various sensors as one evaluation value for each evaluation index, and supplies the evaluation value to the internal state management unit 110.

The second emotion estimation unit 104 estimates the user's emotion based on the user's utterance content, and specifically performs natural language understanding that analyzes the user's utterance content from the output of the microphone 17 to obtain the user's emotion. Is estimated. As a natural language understanding method, a known algorithm may be used. If the user speaks “Yo Sha. I hit a home run” while playing the baseball game, the second emotion estimation unit 104 estimates the emotion of the user's “joy” index as “positive”, while the user If he / she utters “I have hit a home run”, the second emotion estimation unit 104 may estimate the emotion of the user's “joy” index as “negative”. The second emotion estimation unit 104 supplies the estimated evaluation value of the user's emotion to the internal state management unit 110.

The event detection unit 40 detects the occurrence of an event in the object control system 1 and notifies the third emotion estimation unit 106 of the content of the event that has occurred. Regarding the game event, the event detection unit 40 may be notified of the event by executing the game program on the emulator so that the emulator can detect an event such as a home run. Note that the event detection unit 40 may detect a home run event by referring to access to effect data reproduced during a home run, or may detect a home run event from an effect actually displayed on the display device 11. . The third emotion estimation unit 106 estimates the user's emotion from the notified event content, and supplies the evaluation value to the internal state management unit 110.

The event detection unit 40 may be provided with event occurrence timing from an external server that accumulates big data. For example, when the information processing apparatus 10 plays a movie content, the event detection unit 40 obtains a correspondence table between the time information of the attention scene of the movie content and the user emotion estimated in the attention scene from an external server in advance. Acquired and provided to the third emotion estimation unit 106. When the time of the scene of interest arrives, the event detection unit 40 notifies the third emotion estimation unit 106 of time information, and the third emotion estimation unit 106 refers to the correspondence relationship table and is associated with the time information. Emotion is acquired, and the user's emotion in the scene of interest is estimated. As described above, the third emotion estimation unit 106 may estimate the user's emotion based on information from the external server.

In the internal state storage unit 130, the object internal state storage unit 132 stores the internal state of the robot 20 as an object, and the user internal state storage unit 134 stores the internal state of the user. The internal state of the robot 20 is defined by at least emotions and favorable feelings of the robot 20 with respect to the user, and the internal state of the users is defined by the emotions of the user and the favorable feelings of the user with respect to the robot 20. The likability is generated based on the evaluation values of the latest emotions up to the present, and is evaluated based on the long-term relationship between the user and the robot 20. The internal state of the robot 20 is derived from the user's behavior with respect to the robot 20.

The internal state management unit 110 manages the internal state of the robot 20 and the internal state of the user based on the user's emotion estimated by the emotion estimation unit 100. Hereinafter, the internal state of the user will be described first, and then the internal state of the robot 20 will be described.
The user's “emotion” is set by the emotion estimated by the emotion estimation unit 100. The internal state management unit 110 updates the evaluation value in the user internal state storage unit 134 based on the emotion estimated by the emotion estimation unit 100.

When the “positive” evaluation value is supplied from the emotion estimation unit 100 for the emotion index of the user, the internal state management unit 110 updates the evaluation value of the emotion index to “positive”, and the “negative” evaluation value is When supplied, the evaluation value of the emotion index is updated to “negative”. Here, “update” means a process of overwriting the original evaluation value in the internal state storage unit 130, the original evaluation value is “positive”, the evaluation value to be overwritten is “positive”, and the evaluation value Even if there is no change, the evaluation value is updated.

The emotion estimation unit 100 estimates the user's emotions in the three systems of the first emotion estimation unit 102, the second emotion estimation unit 104, and the third emotion estimation unit 106. The evaluation values may contradict each other. Hereinafter, the evaluation value update processing regarding the “joy” index when the user hits a home run in a baseball game will be described.

The third emotion estimation unit 106 holds an evaluation value table for estimating emotions as “positive” when the user hits a home run and “negative” when the user hits a home run. ing. When the third emotion estimation unit 106 receives an event notification that the user has made a home run from the event detection unit 40, the third emotion estimation unit 106 refers to the evaluation value table and estimates the evaluation value of the “joy” index as “positive”.

Ordinarily, when a user hits a home run, the user expresses the emotion of joy by moving his body or screaming “Yo Sha. I hit the home run.” At this time, both the first emotion estimation unit 102 and the second emotion estimation unit 104 estimate the evaluation value of the “joy” index as “positive”. Therefore, the evaluation values of the “joy” index supplied from the emotion estimation unit 100 to the internal state management unit 110 in three systems are all “positive”, and the internal state management unit 110 stores the internal state management unit 110 in the user internal state storage unit 134. Update the evaluation value of the “joy” index to “positive”.

However, if the user is aiming for something other than a home run, the user may not always exhibit joyful behavior. For example, if a batter hits a batter at a batter's bat, and the user is aiming for a third strike to establish a cycle hit (one batter hits at least one single strike, two strikes, three strikes, and one home run in one game), I can't be happy even if I hit it. In addition to winning and losing games, there are many mechanisms for awarding special plays such as cycle hits, and there are cases where users who are aiming for award are more happy with three strikes than home runs. Therefore, when the user who hit the home run utters “Has a home run?”, Both the first emotion estimation unit 102 and the second emotion estimation unit 104 estimate the evaluation value of the “joy” index as “negative”. .

Therefore, the internal state management unit 110 is supplied with a “negative” evaluation value from the first emotion estimation unit 102 and the second emotion estimation unit 104, and a “positive” evaluation value from the third emotion estimation unit 106. . The internal state management unit 110 may employ an evaluation value that matches in more than half of the evaluation values supplied from the three systems. That is, the internal state management unit 110 employs an evaluation value that matches more than half of the evaluation values supplied independently from a plurality of systems according to the majority rule, and therefore, in this case, the evaluation of the “joy” index. The value may be updated to “negative”.

The internal state management unit 110 may set a priority order for the evaluation values supplied from the three systems, and may determine an evaluation value to be adopted according to the priority order. Hereinafter, ranking of each evaluation value will be described.
First, the evaluation value estimated by the third emotion estimation unit 106 is a unique evaluation value defined in the event, and does not reflect the actual user state. Therefore, the evaluation order estimated by the third emotion estimation unit 106 may be set at the lowest priority order.

Next, when the first emotion estimation unit 102 and the second emotion estimation unit 104 are compared, the second emotion estimation unit 104 performs emotion estimation based only on the user's utterance content, and the amount of information for estimation is small. There are few aspects. In addition, even if the utterance content itself is positive, the user may speak masochistically. For example, it was a case where he was aiming for a three-stroke strike, but hit his home run and dropped his shoulders. In this case, the second emotion estimation unit 104 estimates a “positive” evaluation value, but the first emotion estimation unit 102 estimates a “negative” evaluation value from the user's attitude and paralinguistic information. As described above, since the user may utter words contrary to feelings, the internal state management unit 110 determines the reliability of the evaluation value obtained by the first emotion estimation unit 102 and the evaluation value obtained by the second emotion estimation unit 104. You may set higher than reliability. Therefore, the internal state management unit 110 may set priorities in the order of evaluation values by the first emotion estimation unit 102, the second emotion estimation unit 104, and the third emotion estimation unit 106.

The internal state management unit 110 can update the user's internal state if the evaluation value is obtained in at least one of the systems by setting the priority order. When the user's emotion estimated by the first emotion estimation unit 102 does not match the user's emotion estimated by the second emotion estimation unit 104, the internal state management unit 110 performs the first emotion estimation according to the priority order. Based on the user's emotion estimated by the unit 102, the user's internal state is updated. For example, when the user hits a home run, the first emotion estimation unit 102 and the second emotion estimation unit 104 cannot estimate the user's positive or negative emotion (the user does not move at all and does not speak) ), The internal state management unit 110 may adopt the evaluation value estimated by the third emotion estimation unit 106.

The “favorability” of the user with respect to the robot 20 is derived from a plurality of evaluation values of emotions estimated from the behaviors of the user with respect to the robot 20 so far. Assuming that the name of the robot 20 is “Hikoemon”, when the robot 20 speaks to the user, if the user responds “As Hikoemon says,” the emotion estimation unit 100 evaluates the “love” index of the user. If the value is estimated to be “positive” while the user responds “Hikoemon, quiet from noisy”, the emotion estimation unit 100 sets the evaluation value of the user's “love” index to “negative”. Presume that there is. Further, when the robot 20 speaks to the user “please charge soon”, if the user charges immediately, the emotion estimation unit 100 sets the evaluation value of the user's “love” index to “positive”. If, on the other hand, it is difficult to charge, the emotion estimation unit 100 estimates that the evaluation value of the user's “love” index is “negative”. When the user strokes the head of the robot 20, the emotion estimation unit 100 estimates that the evaluation value of the “love” index of the user is “positive”, and when the user kicks the robot 20, the emotion estimation unit 100 The evaluation value of the “love” index is estimated to be “negative”.

When the internal state management unit 110 receives the evaluation value of the “love” index from the emotion estimation unit 100, the internal state management unit 110 reflects the evaluation value of the user's likability. The internal state management unit 110 stocks a plurality of evaluation values of the “love” index up to the present, and estimates the user's preference for the robot 20 based on, for example, the most recent (for example, 21) evaluation values. . Here, among the plurality of evaluation values excluding the neutral evaluation value, the higher evaluation value of the positive evaluation value and the negative evaluation value may be set as the favorable evaluation value. That is, if the positive evaluation value is 11 or more among the latest 21 evaluation values, the favorable evaluation value is set to “positive”, and if the negative evaluation value is 11 or more, the favorable evaluation value is It may be set as “negative”.

The above is an explanation of the internal state of the user, and then the internal state of the robot 20 is described.
The “favorability” for the user of the robot 20 is derived from a plurality of evaluation values of emotions estimated from the speech and actions that the user has made to the robot 20 so far. The internal state management unit 110 receives the evaluation value of the “love” index from the emotion estimation unit 100 and reflects it in the evaluation value of the favorableness for the user of the robot 20. The user's preference for the robot 20 tends to be linked to the user's preference for the robot 20 described above. The internal state management unit 110 stocks a plurality of evaluation values of the “love” index up to the present, and determines the favorableness of the robot 20 to the user based on, for example, the most recent (for example, 11) evaluation values. To do. Here, the reason why the number of evaluation values to be referred to when estimating the user's favorability is smaller is to make it easier to change the favorability of the robot 20 to the user. The preference for the user of the robot 20 is an important parameter when determining the emotion of the robot 20. It should be noted that the user's favorability may be evaluated not only by “positive” and “negative” but by “multiple” for “positive” and “negative”.

In the embodiment, if the likability of the robot 20 to the user is positive, the robot 20 sympathizes with the user and takes action to realize a common vision experience. On the other hand, if the likability to the user is negative, the robot 20 20 does not sympathize with the user, but rather takes an action repelling the user. The behavior management unit 120 determines the behavior including the voice output of the robot 20 based on the internal state of the robot 20 and the internal state of the user.

The behavior of the robot 20 when the user hits a home run in a baseball game will be described. The emotion estimation unit 100 estimates the evaluation value of the “joy” index as “positive”, and the internal state management unit 110 updates the evaluation value of the user ’s “joy” index to “positive”. Further, the internal state management unit 110 determines the evaluation value of the “joy” index of the robot 20 with reference to the evaluation value of the “joy” index of the user and the evaluation value of the favorableness of the robot 20 to the user.

<When user preference is positive>
When the preference of the robot 20 with respect to the user is set to be positive, the internal state management unit 110 manages the emotion of the robot 20 so as to sympathize with the emotion of the user. Therefore, if the evaluation value of the “joy” index of the user is updated to positive, the internal state management unit 110 also updates the evaluation value of the “joy” index of the robot 20 to positive. When the internal state of the robot 20 is updated, the internal state management unit 110 provides the behavior management unit 120 with opportunity information indicating that it is time to determine the behavior of the robot 20. When the behavior management unit 120 receives the trigger information, the behavior management unit 120 performs behavior including voice output of the robot 20 based on the internal state of the robot 20 updated in the object internal state storage unit 132, specifically, the updated evaluation value of the emotion. To decide. For example, the behavior management unit 120 generates the utterance content “Oh, the player is amazing. I hit the home run.” Also, the behavior of the robot 20 is determined twice. The utterance content and operation are notified to the output processing unit 140, and the output processing unit 140 causes the robot 20 to perform the action determined by the action management unit 120.

The output processing unit 140 generates voice data that controls the output timing of the robot 20 such as the speed of speech and the timing of speech and movement, and drive data that drives the robot 20, and outputs them to the robot 20. When the communication unit 32 receives the voice data and the drive data in the robot 20, the control unit 30 supplies the voice data to the speaker 36 and outputs the voice, and drives the motor of the drive mechanism 34 based on the drive data. As a result, the robot 20 performs utterances and actions that sympathize with the user who hit the home run. The user sees that the robot 20 is happy together, so that the familiarity with the robot 20 is enhanced and the motivation for the game is also enhanced.

On the contrary, when the user hits a home run, the evaluation value of the “joy” index of the user is updated negatively, so the internal state management unit 110 uses the evaluation value of the “joy” index of the robot 20. Update to negative. When the internal state of the robot 20 is updated, the internal state management unit 110 provides opportunity information to the behavior management unit 120. Accordingly, the behavior management unit 120 determines the behavior including the voice output of the robot 20 based on the evaluation value of the “joy” index of the robot 20 updated in the object internal state storage unit 132. For example, the behavior management unit 120 generates an utterance content “Ah. You have been hit by a home run.” Also, as the operation of the robot 20, the behavior that is weak is determined. The utterance content and operation are notified to the output processing unit 140, and the output processing unit 140 causes the robot 20 to perform the action determined by the action management unit 120.

<If user preference is negative>
On the other hand, when the preference of the robot 20 to the user is set to negative, the internal state management unit 110 manages the emotion of the robot 20 so as not to sympathize with the emotion of the user. The internal state management unit 110 rather manages the emotion of the robot 20 so as to repel the user, so that the user can realize that it is better to handle the robot 20 more carefully.

Therefore, even if the evaluation value of the “joy” index of the user who made the home run is updated positively, the internal state management unit 110 updates the evaluation value of the “joy” index of the robot 20 to negative. If the preference of the robot 20 to the user is negative, the internal state management unit 110 updates the evaluation value of the emotion of the robot 20 to a value opposite to the evaluation value of the user's emotion. Therefore, if the user's emotion evaluation value is positive, the emotion evaluation value of the robot 20 may be updated to negative, and if the user's emotion evaluation value is negative, the emotion evaluation value of the robot 20 may be updated to positive.

When the internal state of the robot 20 is updated, the internal state management unit 110 provides the behavior management unit 120 with opportunity information indicating that it is time to determine the behavior of the robot 20. Thereby, the behavior management unit 120 determines the behavior including the voice output of the robot 20 based on the internal state of the robot 20 updated in the object internal state storage unit 132, specifically, the negative evaluation value of the “joy” index. To do. For example, the behavior management unit 120 may determine the behavior that does not generate the utterance content (that is, ignores the user) and faces the user in the opposite direction (facing the side) as the operation of the robot 20. This action content is notified to the output processing unit 140, and the output processing unit 140 causes the robot 20 to perform the action determined by the action management unit 120.

Referring to the behavior of the robot 20, the user notices that he was pleased with him before but not this time. The user looks back on his attitude so far and recognizes that he / she cannot be pleased together because he / she is in contact with the robot 20 coldly. When the robot 20 reacts repulsively, the user is given an opportunity to keep touching the robot 20 from now on.

As described above, the object control system 1 manages the relationship between the user and the robot 20 in the same manner as the human relationship in the actual human society. In real-person communication, good relationships are created by interacting with each other with compassion, but if one loses compassion, the other will not be compassionate. In the object control system 1, this compassion is expressed by the emotion of the “love” index, and the human relationship through the compassion is expressed by the index of “favorability”. Therefore, the evaluation value of the user's preference in the object control system 1 and the evaluation value of the preference of the robot 20 tend to work together. When the user's preference for the robot 20 is lowered, the user's thoughtful contact with the robot 20 again improves the evaluation value of the user's preference for the user from negative to positive. You will be able to get a common vision experience.

The robot 20 can become a kind of friend when the user increases the familiarity with the robot 20. If the user has an irregular life, the robot 20 may suggest improvement of life rhythm, such as “Let's go to sleep soon,” and the user may listen as advice from a friend in the future. In order to realize such a thing, it is also possible to expand the future possibilities of the robot 20 by constructing a mechanism in which the user enhances the familiarity with the robot 20 through the joint viewing experience of the robot 20. .

In the above-described embodiment, a mechanism has been proposed in which the user obtains a co-view experience with the robot 20 that is an actual object, but a mechanism for obtaining a co-view experience with a virtual object is similarly constructed. it can.

In this case, the virtual object may be a character such as a person or a pet composed of a 3D model, and exists in a virtual space generated by a computer. In the embodiment, in a virtual space constructed when the user wears a head mounted display (HMD), the content is played in front of the user, and when the user turns sideways, the virtual character watches the content together. We propose a mechanism that allows you to see how you are. Similarly to the robot 20, the virtual character also outputs a reaction that sympathizes with the user, or conversely, outputs a reaction that repels the user, thereby communicating with the user.

FIG. 5 shows an example of the external shape of the HMD 200. The HMD 200 includes an output mechanism unit 202 and a mounting mechanism unit 204. The wearing mechanism unit 204 includes a wearing band 206 that goes around the head when worn by the user and fixes the HMD 200 to the head.

The output mechanism unit 202 includes a housing 208 shaped to cover the left and right eyes when the user wears the HMD 200, and includes a display panel that faces the eyes when worn. The display panel may be a liquid crystal panel or an organic EL panel. The housing 208 is further provided with a pair of left and right optical lenses that are positioned between the display panel and the user's eyes and expand the viewing angle of the user. The HMD 200 may further include a speaker or an earphone at a position corresponding to the user's ear, and may be configured to connect an external headphone. In addition, a camera that captures the user's face is provided inside the housing 208 and is used to detect the facial expression of the user.

The outer surface of the housing 208 is provided with light emitting

markers

210a, 210b, 210c, and 210d that are tracking LEDs. The light emitting marker 210 is photographed by the camera 14, and the information processing apparatus 10 analyzes the image of the marker position. The HMD 200 is equipped with posture sensors (acceleration sensor and gyro sensor). The HMD 200 is connected to the information processing apparatus 10 by a known wireless communication protocol, and transmits sensor data detected by the attitude sensor to the information processing apparatus 10. The information processing apparatus 10 specifies the direction in which the HMD 200 is facing based on the imaging position of the light emitting marker 210 and the sensor data of the orientation sensor.

Here, the information processing apparatus 10 transmits content data such as a movie to the HMD 200. The information processing apparatus 10 sets a reference direction in which the HMD 200 is directed, sets a virtual display in the reference direction, and displays content reproduction data on the virtual display. In addition, the information processing apparatus 10 arranges a virtual object facing the virtual display at a position rotated by 90 degrees around the viewpoint position with respect to the reference direction, for example. As a result, when the user rotates the line-of-sight direction 90 degrees horizontally, it is possible to see a situation in which a virtual object is watching content together next.

In this way, the user wearing the HMD 200 can obtain a co-view experience with a virtual object. The virtual object action determination process is as described for the robot 20. In addition, since the degree of freedom of action of the virtual object is higher than that of the robot 20, various positive or negative emotion expressions can be expressed.

As described above, according to the embodiment, the user can increase the sense of familiarity through the experience of visual observation with an actual object or a virtual object, and the user's life can be enriched.

The present invention has been described based on the embodiments. It is to be understood by those skilled in the art that the embodiments are exemplifications, and that various modifications can be made to combinations of the respective components and processing processes, and such modifications are within the scope of the present invention.

In the embodiment, the behavior management unit 120 generates an utterance content “XX player is amazing. You hit a home run.” As a sympathy expression when the user hits a home run. In a modified example, in order to emphasize that it is a co-viewing experience, the behavior management unit 120 may positively output from the object a sound in which the target object is replaced with a pronoun. Specifically, the expression “○ × player” may be replaced with “kodoko” which is a pronoun, and the content of the utterance “had this guy run. By using the indicating pronouns, the user can further enhance the sense of co-viewing with the robot 20.

DESCRIPTION OF SYMBOLS 1 ... Object control system, 10 ... Information processing apparatus, 20 ... Robot, 40 ... Event detection part, 100 ... Emotion estimation part, 102 ... 1st emotion estimation part, 104. .. second emotion estimation unit 106... Third emotion estimation unit 110... Internal state management unit 120... Action management unit 130 .. internal state storage unit 132. State storage unit 134... User internal state storage unit 140.

The present invention can be used in the technical field of controlling the behavior of an object.

Claims

An object control system for controlling an object,
An emotion estimation unit that estimates the user's emotions;
A user internal state storage unit for storing the user's internal state including the user's emotions;
An object internal state storage unit for storing the internal state of the object including the emotion of the object;
An internal state management unit for managing the internal state of the object and the internal state of the user based on the estimated emotion of the user;
A behavior management unit that determines the behavior of the object based on the internal state of the object;
An output processing unit that causes the object to perform the behavior determined by the behavior management unit;
An object control system comprising:
The internal state of the object includes the user's likability of the object derived from multiple ratings of emotion estimated to date,
The internal state management unit determines the emotion of the object from the likability,
The object control system according to claim 1.
When the internal state management unit updates the internal state, it provides the behavior management unit with opportunity information indicating that it is time to determine the behavior of the object,
When the behavior management unit receives the opportunity information, the behavior management unit determines the behavior of the object.
The object control system according to claim 1, wherein the object control system is an object control system.
The emotion estimation unit
A first emotion estimation unit for estimating a user's emotion based on the sensor output;
A second emotion estimation unit that estimates the user's emotion based on the user's utterance content,
The internal state management unit updates the internal state of the object and / or the internal state of the user based on the output of the first emotion estimation unit and / or the second emotion estimation unit.
The object control system according to claim 1, wherein the object control system is an object control system.
The internal state management unit is estimated by the first emotion estimation unit when the user's emotion estimated by the first emotion estimation unit and the user's emotion estimated by the second emotion estimation unit do not match. Update the user ’s internal state based on the user ’s emotions,
The object control system according to claim 4.
The object is an actual object or a virtual object.
The object control system according to claim 1, wherein the object control system is an object control system.
The behavior management unit outputs from the object a sound in which the object is replaced with a pronoun.
The object control system according to claim 1, wherein:
Estimating the user's emotions;
Managing the user's internal state including the user's emotions;
Managing the internal state of the object including the emotion of the object;
Updating the internal state of the object and the internal state of the user based on the estimated user emotion;
Determining the behavior of the object based on the internal state of the object;
Causing the object to perform the determined action;
An object control method comprising:
On the computer,
The ability to estimate user emotions,
The ability to manage the user ’s internal state, including the user ’s emotions,
The ability to manage the internal state of the object, including the object's emotions,
A function for updating the internal state of the object and the internal state of the user based on the estimated emotion of the user,
The ability to determine the behavior of an object based on the internal state of the object;
A function that causes the object to perform the determined action;
A program to realize