JP2009131928A - Robot control system, robot, program and information recording medium - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide a robot control system, a robot, a program and an information recording medium capable of facilitating indirect communication between users via the robot. <P>SOLUTION: The robot control system comprises: a user information acquiring part that acquires user information obtained from sensor information from at least one of a behavior sensor for measuring behavior of a user, a status sensor for measuring status of the user and an environment sensor for measuring environment of the user; a presented information deciding part that decides, based on the acquired user information, information to be presented to the user by the robot; and a robot control part that controls the robot to present the information to the users. The user information acquiring part acquires second user information that is user information of a second user, and the presented information deciding part decides, based on the acquired second user information, information to be presented to the first user. The robot control part controls the robot to present the information, which has been decided based on the second user information, to the first user. <P>COPYRIGHT: (C)2009,JPO&INPIT

Description

  The present invention relates to a robot control system, a robot, a program, and an information storage medium.

  2. Description of the Related Art Conventionally, a robot control system that recognizes a user (human) voice and performs a conversation with the user based on the voice recognition result is known (for example, Patent Document 1).

  However, in conventional robot control systems, it is assumed that the robot operates by recognizing the conversation of the owner's user by voice, and robot control reflecting the actions of other users is performed. There wasn't.

  In addition, the conventional robot control systems do not perform robot control that reflects the user's behavior history or state history, and the robot may perform operations that are contrary to the user's feelings and circumstances. It was.

  Further, in the conventional robot control system, it is assumed that one robot converses and talks to one user. For this reason, a complicated algorithm is required for voice recognition processing and conversation processing, and there is a problem that it is difficult in practice to realize a smooth conversation with the user.

JP 2003-66986 A

  The present invention has been made in view of the above problems and the like, and according to some aspects of the present invention, a robot control system, a robot, and a robot that can realize indirect communication between users via a robot, A program and an information storage medium are provided.

  The present invention is a robot control system for controlling a robot, and includes at least one of a behavior sensor that measures a user's behavior, a state sensor that measures a user's state, and an environment sensor that measures a user's environment. A user information acquisition unit that acquires user information obtained from the information, a presentation information determination unit that performs a process of determining presentation information that the robot presents to the user based on the acquired user information, and the presentation information A robot control unit that performs control for the user to present to the robot, wherein the user information acquisition unit acquires second user information that is the user information of the second user, and determines the presentation information The unit performs a process of determining information to be presented to the first user based on the acquired second user information, and the robot control unit The presentation information determined based on the second user information, related to the robot control system for a robotic control for presentation to the first user. The present invention also relates to a program that causes a computer to function as each of the above-described units, or a computer-readable information storage medium that stores the program.

According to the present invention, user information obtained from sensor information from at least one of a behavior sensor, a state sensor, and an environment sensor is acquired. And based on the acquired user information, the determination process of the presentation information which a robot presents to a user is performed, and control for making a robot present presentation information is performed. And according to this invention, based on the 2nd user information of the acquired 2nd user, the determination process of the presentation information to a 1st user is performed, and the determined presentation information is sent to a 1st user. The robot to be presented is controlled. As described above, according to the present invention, the presentation information that the robot presents to the first user is determined based on the second user information of the second user different from the first user. Therefore, the first user can know information such as the behavior and situation of the second user from the presentation information of the robot, and can realize indirect communication between the users via the robot.
In the present invention, the user information acquisition unit acquires the first user information that is the user information of the first user and the second user information that is the user information of the second user. The presentation information determination unit may perform a process for determining the presentation information to the first user based on the acquired first user information and the second user information.

  If it does in this way, presentation information based on the 2nd user information to the 1st user which also considered the 1st user information of the 1st user will be attained.

  In the present invention, the presentation information determination unit determines the presentation information presentation timing based on the first user information, determines the content of the presentation information based on the second user information, and performs the robot control. The unit may perform robot control for presenting the determined information to the first user at the determined presentation timing.

  If it does in this way, it will become possible to notify the information of the 2nd user at the timing suitable for the 1st user, and it will become possible to present information more naturally and smoothly.

  In the present invention, the presentation information determination unit changes the weighting of the first user information and the weighting of the second user information in the determination process of the presentation information to the first user with time. You may let them.

  In this way, when presenting information to the first user, it is possible to present information that takes into account not only the second user information but also the first user information, and weighting that determines the degree of the addition. Changes over time, making it possible to present more diverse and natural information.

  The present invention further includes an event determination unit that determines the occurrence of an available event indicating that the robot can be used by the first user, and the presentation information determination unit includes the available event occurrence. Sometimes, the weighting of the first user information in the determination process is increased, the weighting of the second user information is decreased, and then the weighting of the first user information is decreased, and the second user information May be increased.

  In this way, since the weight of the second user information in the presentation information determination process increases as the time elapses from the occurrence of the robot available event, more natural information presentation is possible. Become.

  In the present invention, the presentation information determination unit performs a process of determining presentation information to be next presented to the first user by the robot based on the reaction of the first user to the presentation of presentation information by the robot. You may go.

  If it does in this way, based on the 1st user's reaction to presentation information, the next presentation information will come to change and the situation where presentation of presentation information by a robot will become monotonous can be prevented.

  The present invention further includes a contact state determination unit that determines a contact state on the sensing surface of the robot, wherein the presentation information determination unit is configured to respond to the first user's response to the presentation information presented by the robot as the first user. Determining whether or not the robot has stroked or hit the robot based on the determination result in the contact state determination unit, and performed a process for determining the next presentation information to be presented to the first user. Also good.

  In this way, it is possible to determine the first user's reaction, such as stroking or hitting the robot, with a simple determination process.

  In the present invention, the contact state determination unit is configured to contact the sensing surface based on output data obtained by performing arithmetic processing on an output signal from a microphone provided inside the sensing surface. The state may be determined.

  In this way, it is possible to detect the first user's reaction, such as stroking or striking the robot, just by using the microphone.

  In the present invention, the output data is signal intensity, and the contact state determination unit performs comparison processing between the signal intensity and a predetermined threshold value, so that the first user strokes the robot. It may be determined whether an action has been performed or a hit action has been performed.

  In this way, it is possible to determine whether the first user has performed an operation of stroking or striking the robot by a simple process of comparing the signal intensity and the threshold value.

  In the present invention, the presentation information determination unit may provide the first user such that the first and second robots present different presentation information for the acquired second user information. The presentation information to be presented may be determined.

  In this way, the first user can indirectly know the information of the second user through the presentation information presented by the first and second robots.

  In the present invention, the first robot is set on the master side, the second robot is set on the slave side, and the presentation information determination unit provided in the first robot on the master side is connected to the slave side. The second robot may be instructed to present the presentation information to the first user.

  In this way, presentation of presentation information by the first and second robots can be realized under stable control with few malfunctions without performing complicated presentation information analysis processing.

  Further, the present invention may include a communication unit that communicates instruction information for instructing presentation of presentation information from the first robot on the master side to the second robot on the slave side.

  In this way, it is only necessary to communicate the instruction information, not the presentation information itself, so that the amount of communication data can be reduced and the processing can be simplified.

  In the present invention, the user information acquisition unit acquires the second user information of the second user via a network, and the presentation information determination unit acquires the second user information acquired via the network. The presentation information to be presented to the first user may be determined based on the user information.

  In this way, for example, even when the second user is away, robot control reflecting the information of the second user can be realized.

  In the present invention, the user information acquisition unit may include at least one of the second user behavior history, the second user status history, and the second user environment history as the second user information. Second presentation history information is acquired, and the presentation information determination unit determines presentation information that the robot presents to the first user based on the acquired second user history information. May be performed.

  In this way, it is possible to present the presentation information reflecting the second user's action history, state history, environment history, or the like by the robot.

  In the present invention, the second user history information may be information obtained by updating with sensor information from the wearable sensor of the second user.

  If it does in this way, the 2nd user's action history, state history, or environmental history can be updated with sensor information from a wearable sensor, and presentation information reflecting these updated history can be presented by a robot. It becomes like this.

  The present invention also includes a user identification unit that identifies an approaching user when the user approaches the robot, and the robot control unit is identified by the user identification unit as the first user approaches. In such a case, robot control for presenting presentation information to the first user may be performed.

  In this way, when the first user approaches the robot and is identified as the first user, the presentation information based on the second user information can be presented to the first user.

  The present invention further includes a presentation permission determination information storage unit that stores presentation permission determination information for determining permission / non-permission of information presentation between users, and the presentation information determination unit includes the presentation permission determination information. Based on this, when it is determined that information presentation between the first and second users is permitted, a process for determining presentation information to the first user based on the second user information is performed. Also good.

  In this way, it is possible to permit indirect communication via a robot only between specific users.

  The present invention further includes a scenario data storage unit that stores scenario data composed of a plurality of conversation phrases as the presentation information, and the presentation information determination unit is configured such that the robot performs the first processing based on the scenario data. The conversation phrase to be uttered to the user may be determined, and the robot control unit may perform control for causing the robot to utter the determined conversation phrase.

  In this way, the utterance of the conversation phrase of the robot can be realized by simple control processing using the scenario data.

  In the present invention, the scenario data storage unit stores scenario data in which a plurality of conversation phrases are linked in a branch structure, and the presentation information determination unit is a response of the first user to a conversation phrase uttered by a robot. Based on the above, a conversation phrase to be subsequently spoken by the robot may be determined.

  In this way, the next conversation phrase changes based on the reaction of the first user to the conversation phrase spoken by the robot, and the situation where the conversation by the robot becomes monotonous can be prevented.

  The present invention further includes a scenario data acquisition unit that acquires scenario data created based on the reaction of the second user with respect to the conversation phrase spoken by the robot, and the presentation information determination unit includes the response of the second user Based on the scenario data acquired by the above, a conversation phrase that the robot speaks to the first user may be determined.

  If it does in this way, based on the scenario data reflecting the reaction of the 2nd user with respect to the conversation phrase of a robot, it will become possible to determine the conversation phrase which a robot speaks to the 1st user.

  Further, in the present invention, the presentation information determination unit provides the first user with the same second user information acquired so that the first and second robots speak different conversation phrases. The conversation phrase to be uttered is determined, and the utterance right of the next conversation phrase is given to either the first or second robot based on the reaction of the first user to the conversation phrase uttered by the robot. An utterance right control unit for controlling whether or not to grant may be included.

  In this way, since the granting of the right to speak is switched according to the reaction of the first user, it is possible to prevent a situation where the conversation becomes monotonous.

  Further, in the present invention, the utterance right control unit may determine whether the first user has made a positive reaction to a conversation phrase uttered by one of the first and second robots. A robot to which the right to speak the next conversation phrase is granted may be determined depending on whether the reaction has been made.

  In this way, for example, it is possible to perform control such as preferentially granting an utterance right to a robot that has been positively reacted by the first user.

  The present invention also relates to a robot including any one of the robot control systems described above and a robot operation mechanism that is a control target of the robot control system.

  Hereinafter, this embodiment will be described. In addition, this embodiment demonstrated below does not unduly limit the content of this invention described in the claim. In addition, all the configurations described in the present embodiment are not necessarily essential configuration requirements of the present invention.

1. User Information In the so-called ubiquitous service, as one direction that is currently aimed at, there has been proposed a convenience providing service that provides users with necessary information anytime and anywhere. This is a service that unilaterally provides information to the user from the outside.

  However, in order for a person to live a lively and fulfilling life, a convenience-providing service that provides information to the user unilaterally from the outside is not sufficient. Inspire-type ubiquitous services that give awareness (inspire) and, as a result, encourage self-growth are desired.

  In this embodiment, in order to realize such an inspire ubiquitous service using information presented by the robot to the user, an action for measuring the action, state, and environment of the user (first and second users). User information (first and second user information) is acquired from sensor information from the sensor, state sensor, and environment sensor. Then, based on the acquired user information, presentation information (for example, conversation) that the robot presents to the user is determined, and robot control is performed to provide the determined presentation information to the robot. First, a method for acquiring this user information (information on at least one of the user's behavior, state, and environment) will be described.

  In FIG. 1, a user (user) has a portable electronic device 100 (mobile gateway). In addition, a wearable display 140 (mobile display) is mounted as a mobile control target device in the vicinity of one eye of the head. Furthermore, various sensors are worn on the body as wearable sensors (mobile sensors). Specifically, indoor / outdoor sensor 510, ambient temperature sensor 511, ambient humidity sensor 512, ambient light intensity sensor 513, arm-mounted motion measurement sensor 520, pulse (heart rate) sensor 521, body temperature sensor 522, peripheral skin temperature sensor 523, sweat sensor 524, foot pressure sensor 530, speech / mastication sensor 540, GPS (Global Position System) sensor 550 provided in portable electronic device 100, facial color sensor 560 provided in wearable display 140, pupil size sensor 561 Etc. are attached. These portable electronic device 100, mobile control target devices such as wearable display 140, and wearable sensors constitute a mobile subsystem.

  In FIG. 1, user information (user history information in a narrow sense) updated by sensor information from the sensor of the user's mobile subsystem is acquired, and the robot 1 is controlled based on the acquired user information. .

  The portable electronic device 100 (mobile gateway) is a portable information terminal such as a PDA (Personal Digital Assistant) or a notebook PC. For example, a processor (CPU), a memory, an operation panel, a communication device, a display (sub-display), or the like. Is provided. The portable electronic device 100 has, for example, a function of collecting sensor information from sensors, a function of performing arithmetic processing based on the collected sensor information, and control (display control) of a control target device (such as a wearable display) based on a calculation result. Etc.), a function of retrieving information from an external database, a function of communicating with the outside, and the like. The portable electronic device 100 may be a device that is also used as a mobile phone, a wristwatch, a portable audio, or the like.

  Wearable display 140 is mounted in the vicinity of one of the user's eyes and the size of the display unit is set to be smaller than the size of the pupil, and functions as an information display unit of a so-called see-through viewer. Information presentation to the user may be performed using headphones, a vibrator, or the like. In addition to the wearable display 140, various devices such as a wristwatch, a mobile phone, or a portable audio can be assumed as the mobile control target device.

  The indoor / outdoor sensor 510 is a sensor that detects whether the user is indoors or outdoors. For example, the user irradiates ultrasonic waves and measures the time until the ultrasonic waves are reflected back from the ceiling or the like. However, the indoor / outdoor sensor 510 is not limited to the ultrasonic method, and may be an active light method, passive ultraviolet method, passive infrared method, or passive noise method sensor.

  The ambient temperature sensor 511 measures the ambient temperature using, for example, a thermistor, a radiation thermometer, a thermocouple, or the like. The ambient humidity sensor 512 measures the ambient humidity using, for example, the change in electrical resistance due to humidity. The ambient light intensity sensor 513 measures the ambient light intensity using, for example, a photoelectric element.

  The arm-mounted motion measurement sensor 520 measures the movement of the user's arm with an acceleration sensor or an angular acceleration sensor. By using the motion measurement sensor 520 and the foot pressure sensor 530, it is possible to more accurately measure the user's daily motion and walking state. A pulse (heart rate) sensor 521 is attached to the wrist, finger, or ear, and measures changes in blood flow associated with pulsation, for example, by changes in infrared light transmittance and reflectance. The body temperature sensor 522 and the peripheral skin temperature sensor 523 measure the user's body temperature and peripheral skin temperature using a thermistor, a radiation thermometer, a thermocouple, and the like. The perspiration sensor 524 measures the perspiration of the skin based on, for example, a change in the surface resistance of the skin. The foot pressure sensor 530 detects the pressure distribution of the sole on the shoe, and measures and determines the user's standing state, sitting state, walking state, and the like.

  The utterance / mastication sensor 540 is an earphone-type sensor for measuring the possibility that the user is speaking (conversation) or mastication (meal), and a bone conduction microphone is included in the body. Built-in external sound microphone. The bone conduction microphone detects a body sound which is a vibration generated from the body during speech and mastication and propagating through the body. The external sound microphone detects sound that is vibration that is conducted outside the body in response to an utterance and external sound including environmental noise. Then, by comparing the power of the sound captured by the bone conduction microphone and the external sound microphone in unit time, etc., the utterance possibility and mastication possibility are measured.

  The GPS sensor 550 is a sensor that detects the position of the user. Instead of the GPS sensor 550, a location information service of a mobile phone or location information of a wireless LAN in the vicinity may be used. The face color sensor 560, for example, arranges an optical sensor near the face, and measures the face color by comparing the light intensity after passing through a plurality of optical bandpass filters. The pupil size sensor 561, for example, places a camera near the pupil and analyzes the camera signal to measure the size of the pupil.

  In FIG. 1, user information is acquired by a mobile subsystem including a portable electronic device 100 and a wearable sensor. However, user information is updated by an integrated system including a plurality of subsystems. The robot 1 may be controlled based on the updated user information. Here, the integrated system may include a subsystem such as a mobile subsystem, a home subsystem, an in-vehicle subsystem, an in-company subsystem, or an in-store subsystem.

  In this integrated system, when the user is outdoors (in a mobile environment), sensor information (including sensor secondary information) is acquired (collected) from the wearable sensor (mobile sensor) of the mobile subsystem, User information (user history information) is updated based on the acquired sensor information. The mobile control target device is controlled based on the user information and the like.

  On the other hand, when the user is at home (in a home environment), sensor information from the home sensor of the home subsystem is acquired, and the user information is updated based on the acquired sensor information. That is, the user information updated in the mobile environment is seamlessly updated even when the home information is transferred to the home environment. Control of home control target devices (TV, audio device, air conditioner, etc.) is performed based on user information and the like. The home sensor is installed in, for example, an environmental sensor that measures the temperature, humidity, light intensity, noise, user conversation, meals, etc. in the home, a robot built-in sensor built into the robot, or in each room or door of the home. A human detection sensor or a urine test sensor installed in a toilet.

  Further, when the user is in the vehicle (in the case of an in-vehicle environment), sensor information from the in-vehicle sensor of the in-vehicle subsystem is acquired, and the user information is updated based on the acquired sensor information. That is, the user information updated in the mobile environment or the home environment is seamlessly updated even when the user information is transferred to the in-vehicle environment. In addition, control of in-vehicle controlled devices (navigation device, car AV device, air conditioner, etc.) is performed based on user information and the like. The in-vehicle sensor measures the driving state sensor that measures the vehicle speed, distance traveled, etc., the operation state sensor that measures the user's driving operation and device operation, and the temperature, humidity, light intensity, and user conversation in the vehicle. For example, environmental sensors.

2. Next, the configuration of the robot 1 (robot 2) in FIG. 1 will be described. This robot 1 is a pet type robot imitating a dog, and includes a plurality of part modules (robot operation mechanism) such as a torso module 600, a head module 610, leg modules 620, 622, 624, 626, and a tail module 630. ).

  The head module 610 includes a contact sensor for detecting a user's stroking and tapping operations, a speech sensor (microphone) for detecting a user's speech, an image sensor (camera) for image recognition, A sound output unit (speaker) for uttering voice or cry is provided.

  Joint mechanisms are provided between the body module 600 and the head module 610, between the body module 600 and the tail module 630, joint portions of the leg module 620, and the like. And these joint mechanisms have actuators, such as a motor, and, thereby, the joint movement of the robot 1 and independent running are implement | achieved.

  Further, for example, the body module 600 of the robot 1 is provided with one or a plurality of circuit boards. The circuit board includes a CPU (processor) for performing various processes, a memory such as a ROM and a RAM for storing various data and programs, a control IC for robot control, a sound generation module for generating sound signals, A wireless module for wireless communication with the outside is mounted. Signals from various sensors mounted on the robot are collected on this circuit board and processed by a CPU or the like. The audio signal generated by the audio generation module is output from the circuit board to the sound output unit (speaker). Further, a control signal from the control IC of the circuit board is output to an actuator such as a motor provided in the joint mechanism, thereby controlling the joint motion and the autonomous running of the robot 1.

3. Robot Control System FIG. 2 shows a system configuration example of this embodiment. This system includes a portable electronic device 100-1 possessed by a first user, a portable electronic device 100-2 possessed by a second user, and the robot 1 controlled by the robot control system of the present embodiment. Have. And the robot control system of this embodiment is implement | achieved by the process part 10 which the robot 1 has, for example.

  Note that, for example, the owner of the robot 1 can be assumed as the first user, and the family, friends, relatives, or lovers of the owner can be assumed as the second user. Alternatively, the first and second users may be owners who share the robot 1.

  The portable electronic device 100-1 possessed by the first user includes a processing unit 110-1, a storage unit 120-1, a control unit 130-1, and a communication unit 138-1. The portable electronic device 100-2 possessed by the second user includes a processing unit 110-2, a storage unit 120-2, a control unit 130-2, and a communication unit 138-2.

  Hereinafter, in order to simplify the description, the portable electronic devices 100-1 and 100-2, the processing units 110-1 and 110-2, the storage units 120-1 and 120-2, and the control unit 130 are described. -1, 130-2, communication units 138-1, 138-2, and the like are collectively referred to as portable electronic device 100, processing unit 110, storage unit 120, control unit 130, and communication unit 138, respectively, as appropriate. The first and second users, the first and second user information, and the first and second user history information are also collectively referred to as user, user information, and user history information, respectively, as appropriate.

  The portable electronic device 100 (100-1, 100-2) acquires sensor information from the wearable sensor 150 (150-1, 150-2). Specifically, the wearable sensor 150 is a behavior sensor that measures a user's (first and second users) behavior (walking, conversation, meal, limb movement, emotional expression, sleep, etc.), and the user's state (fatigue). At least one of a state sensor that measures a user's environment (location, brightness, temperature, humidity, etc.) In addition, the portable electronic device 100 acquires sensor information from these sensors.

  The sensor may be the sensor device itself, or may be a sensor device including a control unit, a communication unit, and the like in addition to the sensor device. The sensor information may be sensor primary information obtained directly from the sensor, or sensor secondary information obtained by processing (information processing) the sensor primary information.

  The processing units 110 (110-1 and 110-2) perform various operations necessary for the operation of the portable electronic device 100 based on operation information from an operation unit (not shown), sensor information acquired from the wearable sensor 150, and the like. Perform the process. The function of the processing unit 110 can be realized by hardware such as various processors (CPU, etc.), ASIC (gate array, etc.), a program stored in an information storage medium (optical disc, IC card, HDD, etc.) not shown. .

  The processing unit 110 includes a calculation unit 112 (112-1, 112-2) and a user information update unit 114 (114-1, 114-2). Here, the calculation unit 112 performs various calculation processes for filtering processing (selection processing) and analysis processing of sensor information acquired from the wearable sensor 150. Specifically, the calculation unit 112 performs a multiplication process or an addition process of sensor information. For example, as shown in the following equation (1), digitized measurement values Xj of a plurality of sensor information from a plurality of sensors and respective coefficients are stored in a coefficient storage unit (not shown), and a two-dimensional matrix (matrix) The product-sum operation with the coefficient Aij represented by Then, as shown in the following equation (2), the result of the product-sum operation is calculated as an n-dimensional vector Yi using multidimensional coordinates. Note that i is the i coordinate in the n-dimensional space, and j is a number assigned to each sensor.

  Filtering processing that removes unnecessary sensor information from the acquired sensor information, and user behavior, state, and environment (TPO information) by performing arithmetic processing such as the above formulas (1) and (2) Analysis processing for identifying based on information can be realized. For example, if the coefficient A multiplied by the measured value X of the pulse (heart rate), the amount of sweat, and the body temperature is set to a value larger than the coefficient for the measured value of other sensor information, the above formulas (1) and (2 The numerical value Y calculated in () represents the “excitability” that is the user's state. In addition, by setting the coefficient multiplied by the measured value X of the utterance amount and the coefficient multiplied by the measured value X of the foot pressure to appropriate values, the user's behavior can sit and talk. You can identify whether you are talking, walking, talking quietly, or sleeping.

  The user information update unit 114 (114-1, 114-2) performs update processing of user information (user history information). Specifically, the user information (first and second user information) is updated based on the sensor information acquired from the wearable sensor 150 (150-1, 150-2). The updated user information (user history information) is stored in the user information storage unit 122 (user history information storage unit) of the storage unit 120. In this case, when storing new user information in order to save the memory capacity of the user information storage unit 122 (122-1, 122-2), the old user information is deleted, and the storage area freed by the deletion May store new user information. Or when giving priority (weighting coefficient) to each user information and memorizing new user information, user information with a low priority may be deleted. Also, the user information may be updated (overwritten) by calculating the user information already stored and the new user information.

  The storage units 120 (120-1 and 120-2) serve as work areas for the processing unit 110, the communication unit 138, and the like, and their functions can be realized by a memory such as a RAM or an HDD (hard disk drive). The user information storage unit 122 included in the storage unit 120 is information (history information) such as the behavior, state, or environment of the user (first and second users), and is updated based on the acquired sensor information. Information (first and second user information) is stored.

  The control unit 130 (130-1, 130-2) performs display control of the wearable display 140 (140-1, 140-2). The communication unit 138 (138-1, 138-2) performs transfer processing of information such as user information to and from the communication unit 40 of the robot 1 by wireless or wired communication. As wireless communication, Bluetooth (Bluetooth is a registered trademark), short-range wireless such as infrared rays, wireless LAN, and the like can be considered. As the wired communication, one using USB, IEEE1394, or the like can be considered.

  The robot 1 includes a processing unit 10, a storage unit 20, a robot control unit 30, a robot operation mechanism 32, a robot mounting sensor 34, and a communication unit 40. Note that some of these components may be omitted.

  The processing unit 10 performs various processes necessary for the operation of the robot 1 based on sensor information from the robot-mounted sensor 34, acquired user information, and the like. The function of the processing unit 10 can be realized by hardware such as various processors (CPU, etc.), ASIC (gate array, etc.), a program stored in an information storage medium (optical disc, IC card, HDD, etc.) not shown. . That is, in this information storage medium, a program for causing a computer (an apparatus including an operation unit, a processing unit, a storage unit, and an output unit) to function as each unit of the present embodiment (a program for causing a computer to execute processing of each unit) ) Is stored, and the processing unit 10 performs various processes of the present embodiment based on a program (data) stored in the information storage medium.

  The storage unit 20 serves as a work area for the processing unit 10 and the communication unit 40, and the functions thereof can be realized by a memory such as a RAM, an HDD (hard disk drive), or the like. The storage unit 20 includes a user information storage unit 22 and a presentation information storage unit 26. The user information storage unit 22 includes a user history information storage unit 23.

  The robot control unit 30 controls a robot operation mechanism 32 (actuator, sound output unit, LED, etc.) to be controlled, and its functions include hardware such as an ASIC for robot control and various processors, It can be realized by a program.

  Specifically, the robot control unit 30 performs control for causing the robot 1 to present presentation information to the user. When the presented information is a conversation (scenario data) of the robot 1, control is performed to cause the robot 1 to speak a conversation phrase. For example, digital text data representing a conversation phrase is converted into an analog voice signal by a known TTS (Text-To-Speech) process, and is output via a sound output unit (speaker) of the robot operation mechanism 32. When the presentation information is information representing the emotional state of the robot 1, control is performed such as controlling the actuators of each joint mechanism of the robot operation mechanism 32 or lighting the LED so as to represent the emotion. .

  The robot-mounted sensor 34 is various sensors such as a contact sensor, a speech sensor (microphone), or an image sensor (camera). Based on the sensor information from the robot-mounted sensor 34, the robot 1 can monitor the user's response to the presentation information presented to the user.

  The communication unit 40 communicates information such as user information between the communication unit 138-1 of the portable electronic device 100-1 and the communication unit 138-2 of the portable electronic device 100-2 by wireless or wired communication. Perform the transfer process.

  The processing unit 10 includes a user information acquisition unit 12, a calculation unit 13, and a presentation information determination unit 14. Note that some of these components may be omitted.

  The user information acquisition unit 12 acquires user information obtained from sensor information from at least one of a behavior sensor that measures the user's behavior, a state sensor that measures the user's state, and an environment sensor that measures the user's environment.

  For example, the user information update unit 114-2 of the portable electronic device 100-2 is based on the sensor information from the wearable sensor 150-2, and the second user (child of the first user, wife, lover, etc.) The second user information (second user history information) that is the user information is updated. The updated second user information is stored in the user information storage unit 122-2.

  Then, the second user information (second user history information) stored in the user information storage unit 122-2 is transferred to the user information storage unit 22 of the robot 1 via the communication units 138-2 and 40. . Specifically, the second user returns home and approaches the robot 1, or connects the portable electronic device 100-2 to the cradle, and connects the portable electronic device 100-2 and the robot 1. When the communication path is established, the second user information is transferred from the user information storage unit 122-2 to the user information storage unit 22. The user information acquisition unit 12 acquires the second user information by reading the second user information transferred in this way from the user information storage unit 22. The user information acquisition unit 12 may acquire the second user information directly from the portable electronic device 100-2 without using the user information storage unit 22.

  In addition, the user information update unit 114-1 of the portable electronic device 100-1 is based on the sensor information from the wearable sensor 150-1 and includes first user information (first user information) that is user information of the first user. User history information) is updated. The updated first user information is stored in the user information storage unit 122-1.

  The first user information (first user history information) stored in the user information storage unit 122-1 is transmitted to the user information storage unit 22 (user information storage unit 22) of the robot 1 via the communication units 138-1 and 40-1. Part 72). Specifically, the first user returns home and approaches the robot 1, or connects the portable electronic device 100-1 to the cradle so that the portable electronic device 100-1 is connected to the robot 1. When the communication path is established, the first user information is transferred from the user information storage unit 122-1 to the user information storage unit 22. The user information acquisition unit 12 acquires the first user information by reading the first user information thus transferred from the user information storage unit 22. Note that the user information acquisition unit 12 may acquire the first user information directly from the portable electronic device 100-1 without using the user information storage unit 22.

  The calculation unit 13 performs calculation processing on the acquired user information. Specifically, when necessary, an analysis process and a filtering process for user information are performed. For example, when the user information is primary sensor information or the like, filtering processing for removing unnecessary sensor information from the acquired sensor information by performing the arithmetic processing described in the above formulas (1) and (2) Analyzing processing for identifying the user's behavior, state, and environment (TPO information) based on the sensor information is performed.

  The presentation information determination unit 14 determines the presentation information (conversation, emotional expression, behavioral expression) that the robot 1 presents (provides) to the user based on the acquired user information (user information after the calculation process). I do.

  Specifically, the presentation information determination unit 14 determines the presentation information (conversation phrase, emotion expression, action expression) to the first user based on the acquired second user information of the second user. I do. The robot control unit 30 performs robot control for presenting the presentation information determined based on the second user information to the first user. For example, when the first user approaches the robot 1, the presentation information is determined based on the second user information of the second user who is away from the robot 1, and the determined presentation information is changed to the first user information. Present to the user.

  When the first user information of the first user is acquired by the user information acquisition unit 12, the presentation information determination unit 14 is based on both the acquired first user information and second user information. Then, the process for determining the information presented to the first user may be performed.

  Specifically, the TPO (Time Place Occasion) of the first user is estimated based on the first user information, and the TPO information is acquired. That is, time information, location information of the first user, and situation information are acquired. Then, based on the acquired TPO information of the first user and the second user information of the second user, presentation information determination processing is performed.

  More specifically, the presentation information determination unit 14 determines the presentation information presentation timing (conversation start timing, utterance timing) based on the first user information (TPO information), and based on the second user information. To determine the content of the presentation information (conversation content, scenario data). Then, the robot control unit 30 performs robot control for presenting the presentation information of the decided content to the first user at the decided presentation timing.

  That is, if it is determined based on the first user information (the first user's TPO) that the first user is busy, has no mental margin, and is not at the timing to present the presentation information, the robot 1 Do not present the presentation information by. On the other hand, if it is determined that the first user has sufficient time or spare time and is presenting the presentation information based on the first user information, the second user information The content of the presentation information is determined based on the information, and information for notifying the situation, action, etc. of the second user is presented to the first user using the robot 1.

  In this way, the situation of the second user can be notified at a timing that is appropriate and timely for the first user, and more natural and smooth information presentation is possible.

  When the user information acquisition unit 12 acquires second user history information, which is at least one of the second user's action history, state history, and environment history, as second user information, presentation information determination is performed. The unit 14 performs a process of determining presentation information that the robot 1 presents to the first user based on the acquired second user history information. The second user history information in this case is information obtained by the portable electronic device 100-2 or the like performing update processing based on sensor information from the wearable sensor 150-2 of the second user, for example. The data is transferred from the user information storage unit 122-2 of the electronic device 100-2 to the user history information storage unit 23 of the robot 1. The user behavior history, state history, and environment history include user behavior (walking, speech, meal, etc.), state (fatigue, tension, hunger, mental state, physical state, etc.), environment (location, brightness, temperature). And the like (log information) stored in association with the date and time.

  In addition, the presentation information determination unit 14 performs a process of determining the presentation information that the robot 1 presents to the first user next based on the first user's response to the presentation of presentation information by the robot 1. Specifically, for example, when the robot 1 presents presentation information to the first user and the first user reacts to the presentation information, the reaction is detected by the robot-mounted sensor 34. Then, the presentation information determination unit 14 determines (estimates) the response of the first user based on the sensor information from the robot-mounted sensor 34, and determines the next presentation information to be presented based on the response.

4). Operation Next, the operation of the present embodiment will be described. In general, a conversation between a user and a robot (conversation) is realized in a one-to-one confrontation relationship, such as one user and one robot. However, in such a one-to-one relationship, there is a problem that the conversation of the robot becomes monotonous and the user gets bored immediately.

  In this regard, in this embodiment, the robot that has a conversation with the first user has a conversation based on the second user information of the second user different from the first user. Therefore, the first user can know the information of the second user such as his / her family, friend, or lover through communication with the robot. Therefore, the robot conversation is prevented from becoming monotonous, and a robot that does not get bored by the user can be realized.

  In this case, the information provided to the user by the conversation of the robot is based on the second user information acquired by the sensor information from the behavior sensor, the state sensor, and the environment sensor that the wearable sensor has. For this reason, the first user can indirectly know the behavior of the second user with whom he / she is close, the state of the second user, and the environment surrounding the second user through the conversation of the robot. become. Therefore, for example, in a home where the father's return home is always late and communication with the child cannot be made, the father can know the state of the child indirectly through robot conversation. In addition, it becomes possible to know the state of friends and lovers living in remote places through conversations with robots, and a robot as an unprecedented type of communication means can be provided.

  For example, in FIG. 3A, a first user who is a father who has returned home connects the portable electronic device 100 (100-1) to the cradle 101 and performs charging or the like. In FIG. 3A, it is determined that an event that can be used by the robot 1 has occurred due to the connection of the portable electronic device 100 to the cradle 101, and the robot 1 is activated and can be used. Instead of connecting to the cradle 101, the robot 1 may be activated by determining the approach between the first user and the robot 1. For example, when information transfer between the portable electronic device 100 and the robot 1 is performed wirelessly, the occurrence of an available event of the robot 1 can be determined by detecting the wireless strength.

  When such an available event occurs, the robot 1 is activated and can be used. In this case, the second user information of the second user who is a child is stored in the user information storage unit 22 of the robot 1. Specifically, information such as the behavior, state, and environment of the second user at school or the like is transferred and stored as the second user information. Thereby, it is possible to control the conversation of the robot 1 based on the second user information. As will be described later, the second user information may be collected and acquired through a conversation between the robot 2 and the second user who is a child.

  For example, in FIG. 3A, when the father (first user) returns home from the company and approaches the robot 1, the robot 1 starts a conversation about the child (second user). Specifically, for example, based on scenario data to be described later, the utterance phrase “Bo-chan seems to be busy with club activities recently” is spoken to inform the father of the child's today.

  Further, in FIG. 3B, the robot 1 speaks “I want to go on a trip during the summer vacation”, and for example, the father informs the father of the child's request acquired through the conversation with the child. . In FIG. 3B, a father who is interested in a topic related to this child's request strokes the robot 1. That is, the father requests the robot 1 to stroke the robot 1 and teach more detailed information in order to know more about the child's request. Then, as shown in FIG. 3 (C), the robot 1 utters “I wonder if summer is the sea?” Based on the information collected from the children. This allows the father to know that the place where the child wants to go summer vacation is the sea. That is, in FIG. 3 (B), the conversation phrase to be uttered next by the robot 1 based on the father's (first user) reaction (browsing motion) to the conversation phrase (presentation of presentation information by the robot) uttered by the robot 1. (Presentation information to be presented next) is determined.

  For example, fathers who are slow to go home every day have little time to interact with their children, so it is difficult to know their children's condition and demands. Even if there is time to interact with the child, the child may not communicate his wishes directly to the father.

  Even in such a case, in the present embodiment, indirect communication between the father and the child is realized using the robot 1 as a medium. For example, even when the child does not directly transmit his / her request to the father, according to the present embodiment, the child's request can be smoothly transmitted to the father via the robot 1. Further, even when a child unconsciously speaks his / her request to the robot 1, the request can be transmitted to the father.

  In addition, since there is little time for contact with children, it becomes possible to give fathers who are less interested in children their awareness of their children. Therefore, it is possible to realize an inspire type ubiquitous service in which a user is aware through a conversation with the robot 1 instead of a so-called convenience providing type service.

  As shown in FIG. 3A, when a robot 1 usable event occurs and the robot 1 is activated, the first user information that is father's user information is stored in the user information storage unit 22 of the robot 1. You may transfer and memorize | store. In other words, information such as father's behavior, state, environment, etc. at the company or on the go is transferred and stored. In this way, it is possible to control the conversation or the like of the robot 1 using both the first user information of the father and the second user information of the child.

  For example, it is assumed that it is determined based on the first user information that the father's return time is later than usual. Specifically, the father's daily return home time is measured based on the location information from the GPS sensor of the wearable sensor and the time information from the timer. Then, the average of the father's return time in the past and the current return time are compared, and it is determined whether or not the father's return time is late.

  When the father's return time is much later than usual, the father's TPO can be estimated to be very tired due to work or the like. Therefore, in this case, the robot 1 does not immediately conduct a conversation on the topic of the child, but performs a laborious conversation such as “Today, it was late because of work and it was difficult”. Or the conversation which talks about the result of the game of the baseball team which the father likes.

  Then, after the time has passed and the state of being tired from work is reduced, the robot 1 starts a conversation about the child based on the second user information. That is, the weighting of the first user information (first user history information) and the weighting of the second user information (second user history information) in the presentation information (conversation) determination process change over time. Let Specifically, when an available event of the robot 1 occurs, the weighting of the first user information that is the father's user information is increased, and the weighting of the second user information that is the child's user information is decreased and presented. Information determination processing is performed. Then, after that, the weighting of the first user information is decreased and the weighting of the second user information is increased, and the presentation information determination process is performed. This makes it possible to present information in a timely manner according to the father's TPO.

  FIG. 4 shows a flowchart for explaining the operation of the present embodiment.

  First, the user information acquisition unit 12 acquires second user information that is user information of a second user (child) (step S1). Specifically, the second user information from the portable electronic device 100-2 of the second user is transferred to the user information storage unit 22, and the transferred user information is read. And the content of the presentation information (conversation etc.) which the robot 1 presents to the first user (father) is determined based on the acquired second user information (child user information) (step S2).

  Next, the user information acquisition part 12 acquires the 1st user information which is the user information of a 1st user (father) (step S3). Specifically, the first user information from the portable electronic device 100-1 of the first user is transferred to the user information storage unit 22, and the transferred user information is read. Next, if necessary, the TPO of the first user is estimated based on the first user information (step S4). Here, TPO (Time Place Occasion) information includes time information (year, month, week, day, time, etc.), user location information (location, location, distance, etc.) and user status information (mental / physical status). At least one piece of information). For example, the meaning of latitude / longitude information obtained by a GPS sensor differs depending on the user. If the location of the latitude / longitude is the user's home, the user's location is estimated to be the home.

  Next, based on the first user information (first user's TPO), it is determined whether it is time to present the presentation information to the first user (step S5). For example, if it is determined that the first user is busy or tired based on the first user information, it is determined that it is not the presentation timing, and the process returns to step S3.

  On the other hand, if it is determined that it is time to present to the first user, robot control is performed to present the presentation information to the robot 1 (step S6). Specifically, as shown in FIGS. 3A to 3C, robot control is performed to cause the robot 1 to speak a conversation phrase.

  Next, the first user's reaction to the presentation information presented in step S6 is monitored (step S7). For example, it is determined whether the first user strokes the robot 1, strikes the robot 1, or does nothing. Then, based on the monitored reaction of the first user, presentation information to be presented next by the robot 1 is determined (step S8). That is, the conversation phrase that the robot 1 speaks next is determined.

5). Multiple Robots In the above, a case where there are a plurality of users and a single robot has been described. However, the present embodiment is not limited to this, and a case where there are a plurality of users and there are a plurality of robots is also possible. Applicable. FIG. 5 shows a second system configuration example of the present embodiment when there are a plurality of robots.

  This system includes portable electronic devices 100-1 and 100-2 possessed by first and second users, and robots 1 and 2 (first and second robots) controlled by the robot control system of the present embodiment. ). The robot control system is realized by the processing units 10 and 60 included in the robots 1 and 2, for example. Since the configuration of the robot 2 is the same as that of the robot 1, detailed description thereof is omitted here.

  In FIG. 5, the presentation information determination unit 14 (64) presents different presentation information (different conversation phrases, different emotion expressions, different behavior expressions) for the robots 1 and 2 for the same second user information acquired ( The presentation information (conversation phrase) to be presented to the first user is determined so as to speak. For example, with respect to the acquired second user information, the robot 1 presents first presentation information (first conversation phrase), and the robot 2 presents a second presentation different from the first presentation information. The presentation information is determined so as to present the information (second conversation phrase).

  Next, the operation of FIG. 5 will be described. In general, a conversation between a user and a robot (conversation) is realized in a one-to-one confrontation relationship, such as one user and one robot.

  On the other hand, in FIG. 5, two robots 1 and 2 (a plurality of robots in a broad sense) are prepared. The user does not directly talk to the robots 1 and 2 but takes a form of listening to the conversation between the robots 1 and 2 as long as they look.

  In this way, it is not a convenience-providing service that unilaterally provides information to the user from the outside, but works on the inside of the user through a conversation between the robots 1 and 2, and the user's behavior, state, environment By giving the user an awareness of (inspire), it is possible to realize an inspire ubiquitous service that encourages self-growth.

  FIGS. 6A to 6C are examples of scenes in which second user information of a second user who is a child is acquired. In FIG. 6A, the child who returns home connects the portable electronic device 100 (100-2) to the cradle 101 and performs charging or the like. In FIG. 6A, it is determined that an event that can be used by the robots 1 and 2 has occurred due to the connection of the portable electronic device 100 to the cradle 101, the robots 1 and 2 are activated, and the use is enabled. . Note that the robots 1 and 2 may be activated by determining the approach to the child by detecting the wireless intensity.

  When the robots 1 and 2 are thus activated, the second user information stored in the portable electronic device 100 possessed by the child is transferred to the user information storage units 22 and 72 of the robots 1 and 2. And the conversation of the robots 1 and 2 is controlled based on the second user information of the child updated in the mobile environment. The second user information updated in the mobile environment is further updated in the home environment based on conversations with the robots 1 and 2.

  For example, in FIG. 6A, it is determined based on the second user information that the child's return time is later than usual. As described above, when it is determined that the return time is late, presentation information related to the return time of the child is presented by the robots 1 and 2. Specifically, scenario data related to the topic of the child's return time is selected, and the robots 1 and 2 start a conversation according to the selected scenario data. For example, in FIG. 6A, the robot 1 utters a conversation phrase “It was late for school today,” and the robot 2 utters a conversation phrase “There are many late days recently.” .

  In FIG. 6B, the robot 1 speaks “Is your club activity busy?” And the robot 2 speaks “Why are you walking around?” In this way, in FIG. 6B, the robots 1 and 2 present different presentation information for the same second user information that “the return time is later than usual”. Then, since the child was actually busy with club activities and returned home late, the child strokes the robot 1 saying “Is the club club busy?” As a result, as shown in FIG. 6C, the stroked robot 1 speaks, “That's right, it was a district tournament soon.”

  In this case, the second user information is updated based on the child's reaction (stroking operation) to the contrasting conversation phrases of the robots 1 and 2 as shown in FIG. That is, it is estimated that the reason why the child's return home was delayed was due to the club activities, and that the home return was delayed due to the club activities was recorded as the second user information, and the scenario data presented to the father was Created. That is, scenario data to be presented to the father (first user) is created based on the reaction of the child (second user) to the conversation phrase spoken by the robots 1 and 2.

  FIG. 7A to FIG. 7C are examples of scenes in which the father who is the first user returns home after the child returns home.

  When the father returns home and the connection of the portable electronic device 100 (100-1) to the cradle 101 is detected, the robots 1 and 2 are activated. In this case, the second user information updated by the conversation with the child described with reference to FIGS. 6A to 6C is stored in the user information storage units 22 and 72 of the robots 1 and 2. . The conversations of the robots 1 and 2 are controlled based on the second user information. Specifically, scenario data related to the topic of the child's late home return time is selected, and the robots 1 and 2 start a conversation according to the selected scenario data. For example, in FIG. 7A, the robot 1 speaks “It was late for chan-chan school”, and the robot 2 speaks “It seems that there are many late days now.”

  In this case, for the same second user information “the child's return time is later than usual”, the robots 1 and 2 present different presentation information to the father (first user). Processing for determining the presentation information presented by 1 and 2 is performed. Specifically, in FIG. 7B, regarding the child's late home return time, the robot 1 speaks “It seems that the club activities are busy”, and the robot 2 speaks “But I ’m in a bad mood”.

  For example, if the robot and the user face each other on a one-on-one basis and the robot always conducts a conversation having the same tendency, the user may feel a sense of blockage or a deadlock in the conversation with the robot.

  In contrast, in FIG. 7B, the robots 1 and 2 utter different contrasting conversation phrases. Moreover, instead of having a direct conversation with the user, the robots 1 and 2 have a conversation, and the user watches it. Therefore, it is possible to provide an inspire type ubiquitous service that is not a so-called convenience-providing service but that gives the user awareness through conversations between the robots 1 and 2.

  In FIG. 7B, since the father is more interested in topics related to children's club activities than topics related to today's moods of children, the father is stroking the robot 1. Then, the “browsing motion” that is the user's reaction to the utterance of the conversation phrase by the robot 1 or 2 is detected by the contact sensor 410 or the like of the robot 1.

  Then, based on the user's reaction of such “blowing motion”, a determination process of a conversation phrase (presentation information to be presented next) in which the robots 1 and 2 talk to the father next is performed. Specifically, as shown in FIG. 7 (C), the stroked robot 1 utters “Because it is a district tournament soon”. Thereafter, the conversation between the robots 1 and 2 is continued according to a topical scenario related to the club activities of the child.

  As described above, in FIGS. 6A to 6C, the second user information, which is the child user information, is updated through the conversation between the robots 1 and 2, and scenario data to be presented to the father is created. . Therefore, the second user information is naturally collected and acquired without the child being particularly conscious. Then, based on the acquired second user information, scenario data on the topic of the child is created, and as shown in FIGS. 7A to 7C, the child 1 Topics are provided to the father. Therefore, even if the father and the child are not particularly conscious, the indirect communication between the father and the child can be realized through the robots 1 and 2. This makes it possible to implement an inspire ubiquitous service that gives the user awareness through conversations with the robot.

  FIG. 8 shows a flowchart for explaining the operation of the system of FIG. FIG. 8 differs from the flowchart of FIG. 4 mainly in the process of step S56. That is, if it is determined in step S55 that it is the timing to present to the first user (father), in step S56, robot control for causing the robots 1 and 2 to present different presentation information is performed. Specifically, as described in FIGS. 7A to 7C, the robots 1 and 2 perform different conversations on the same second user information (children's return time). Determine conversation phrases 1 and 2, and let robots 1 and 2 speak. This prevents a situation where the conversation between the user and the robot becomes monotonous.

  FIG. 9 shows a third system configuration example which is a modification of FIG. In FIG. 9, the robot 1 is set on the master side, and the robot 2 is set on the slave side. The robot control system is realized mainly by the processing unit 10 included in the robot 1 on the master side.

  Specifically, the user information acquisition unit 12 provided in the master-side robot 1 acquires user information (second user information), and the master-side presentation information determination unit 14 is based on the acquired user information. The presentation information to be presented to the user by the robots 1 and 2 is determined. For example, when it is determined that the robots 1 and 2 on the master side and the slave side respectively present the first and second presentation information, the robot control unit 30 on the master side determines the determined first Control for causing the robot 1 to present the presentation information is performed. In this way, the master-side robot 1 is controlled. In addition, the master-side presentation information determination unit 14 instructs the slave-side robot 2 to present the presentation information to the user. For example, when it is determined that the master side and the slave side respectively present the first and second presentation information, the slave side robot 2 is instructed to present the second presentation information. . Then, the robot controller 80 on the slave side performs robot control for causing the robot 2 to present the determined second presentation information. In this way, the slave robot 2 is controlled.

  In this case, the communication unit 40 communicates instruction information for instructing presentation of presentation information from the master robot 1 to the slave robot 2 by, for example, wireless communication. When the communication unit 90 on the slave side receives this instruction information, the robot control unit 80 on the slave side performs robot control for causing the robot 2 to present the presentation information instructed by this instruction information.

  Here, the instruction information of the presentation information is, for example, an identification code of the presentation information. When the presented information is a scenario conversation phrase, the instruction information becomes a conversation phrase data code in the scenario.

  For example, when the robots 1 and 2 are engaged in conversation, the robot 2 can recognize the speech phrase spoken by the robot 1, and the robot 2 can speak the speech phrase based on the result of the speech recognition.

  However, according to this method, complicated speech recognition processing and analysis processing are required, leading to high robot costs, complicated processing, and malfunctions.

  In this regard, in FIG. 9, conversations between the robots 1 and 2 are realized under the control of the robot 1 on the master side. That is, from the user's eyes, it seems that the robots 1 and 2 are talking while recognizing each other's words, but in reality, all conversations are performed under the control of the robot 1 on the master side. . Further, since the slave robot 2 determines information to be presented based on the instruction information communicated from the master robot 1, the voice recognition process can be omitted. Therefore, conversations between the robots 1 and 2 can be realized under stable control with few malfunctions without performing complicated speech recognition processing or the like.

6). Acquisition of Second User Information Through Network Although the description has been mainly given of the case where the method of the present embodiment is applied to communication between family members, the present embodiment is not limited to this. For example, the method of this embodiment can be applied to communication between users such as friends, lovers, and relatives who are separated from each other.

  For example, in FIG. 10, the second user information of the second user who is the lover of the first user is acquired. Specifically, in the second user's mobile environment or home environment, the second user information (second user history information) is updated by the method described in FIG. The updated second user information is communicated via a network such as the Internet. That is, the user information acquisition unit 12 of the robot 1 (robot control system) acquires the second user information via the network. And the presentation information determination part 14 performs the determination process of the presentation information shown to a 1st user based on the 2nd user information acquired via the network.

  In this way, the first user can know the behavior, state, or environment (behavior history, state history, environment history) of the second user who is far away from the robot 1 as a medium. . That is, the robot 1 (or the robots 1 and 2) performs the conversation as described in FIGS. 3A to 3C based on the scenario data based on the second user information acquired through the network. Therefore, the first user can indirectly know the state of the second user who is a lover through the conversation of the robot 1. Thereby, the indirect communication between the 1st and 2nd users in a distant place can be implement | achieved, and an unprecedented communication means can be provided. Note that the second user information may be acquired without using a portable electronic device in the system of FIG.

7). System Configuration Example Next, another system configuration example of the present embodiment will be described. FIG. 11 shows a fourth system configuration example of this embodiment. FIG. 11 shows an example in which one robot is provided, but a plurality of robots may be provided as shown in FIG.

  In FIG. 11, a home server 200 which is a local server is provided. The home server 200 performs processing for controlling a control target device of the home subsystem, communication processing with the outside, and the like. The robot 1 (or robots 1 and 2) operates under the control of the home server 200.

  In the system of FIG. 11, the portable electronic devices 100-1 and 100-2 and the home server 200 are communicatively connected by, for example, a wireless LAN or a cradle, and the home server 200 and the robot 1 are communicatively connected by, for example, a wireless LAN. . The robot control system according to this embodiment is mainly realized by the processing unit 210 of the home server 200. Note that the processing of the robot control system may be realized by distributed processing of the home server 200 and the robot 1.

  When a user (first or second user) possessing portable electronic device 100-1 or 100-2 approaches the home, portable electronic devices 100-1, 100-2 and home server 200 are connected by wireless LAN or the like. Communication between them becomes possible. Alternatively, communication can be performed by placing the portable electronic devices 100-1 and 100-2 in a cradle.

  When the communication path is established, user information (first and second user information) is transferred from the portable electronic devices 100-1 and 100-2 to the user information storage unit 222 of the home server 200. Thereby, the user information acquisition unit 212 of the home server 200 acquires user information. Then, the calculation unit 213 performs necessary calculation processing, and the presentation information determination unit 214 determines the presentation information that the robot 1 presents to the user. Then, the determined presentation information or instruction information of the presentation information (for example, speech phrase utterance instruction information) is transmitted from the communication unit 238 of the home server 200 to the communication unit 40 of the robot 1. The robot control unit 30 of the robot 1 performs robot control for presenting the received presentation information or the presentation information indicated by the received instruction information to the user.

  According to the configuration of FIG. 11, for example, when the data size of user information or presentation information (scenario data) is large, it is not necessary to provide a storage unit for user information or presentation information in the robot 1. And downsizing. Further, since transfer and calculation processing of user information and presentation information can be centralized and processed and managed by the home server 200, more intelligent robot control is possible.

  Further, according to the system of FIG. 11, the user information from the portable electronic devices 100-1 and 100-2 is transferred in advance to the user information storage unit 222 of the home server 200 before the event that the robot 1 can be used occurs. Can be kept. For example, before the user returns home and approaches the robot 1 (specifically, information from a GPS sensor that is one of wearable sensors worn by the user indicates that the user has arrived at the nearest station, The user information storage unit of the home server 200 stores the user information updated in the mobile environment, for example, when the information from the door opening / closing sensor, which is one of the home sensors, indicates that the user opened the front door. 222 is transferred and written in advance. When the user approaches the robot 1 and an available event of the robot 1 occurs, the control operation of the robot 1 using the user information previously transferred to the user information storage unit 222 is started. That is, the robot 1 is activated, and the robot 1 is controlled to perform a conversation as shown in FIGS. 3A to 3C, for example. In this way, the conversation based on the user information can be started immediately after the robot 1 is activated, and the control can be made efficient.

  FIG. 12 shows a fifth system configuration example of the present embodiment. In FIG. 12, an external server 300 as a main server is provided. The external server 300 performs communication processing with the portable electronic devices 100-1 and 100-2 and with the home server 200, and performs various management controls. FIG. 12 shows an example in which one robot is provided, but a plurality of robots may be provided as shown in FIG.

  In the system of FIG. 12, the portable electronic devices 100-1 and 100-2 and the external server 300 are communicatively connected by a wireless WAN such as PHS, and the external server 300 and the home server 200 are communicatively connected by a wired WAN such as ADSL. The home server 200 and the robot 1 (robots 1 and 2) are communicatively connected via a wireless LAN or the like. The robot control system of the present embodiment is mainly realized by the processing unit 210 of the home server 200 and the processing unit (not shown) of the external server 300. The processing of the robot control system may be realized by distributed processing of the home server 200, the external server 300, and the robot 1.

  Each unit such as the portable electronic devices 100-1 and 100-2 and the home server 200 appropriately communicates with the external server 300 to perform transfer processing of user information (first and second user information). In addition, it is determined whether or not the user (first and second users) has approached the house using PHS location registration information, a GPS sensor, a microphone, and the like. The user information stored in the user information storage unit that is not to be downloaded is downloaded to the user information storage unit 222 of the home server 200, and the presentation information presentation control by the robot 1 is started. In addition, presentation information such as scenario data described later can also be downloaded from the external server 300 to the presentation information storage unit 226 of the home server 200.

  According to the system of FIG. 12, user information and presentation information can be managed centrally in the external server 300.

8). User History Information Next, a process for updating user history information, which is one of user information, and a specific example of user history information will be described. The user information can include user information obtained in real time from sensor information, user history information that is a history of user information obtained in real time, and the like.

  FIG. 13 is a flowchart illustrating an example of update processing of user history information.

  First, sensor information from the wearable sensor 150 or the like is acquired (step S21). Next, arithmetic processing such as filtering and analysis of the acquired sensor information is performed (step S22). Based on the calculation result, the user's behavior, state, environment, etc. (TPO, feeling) are estimated (step S23). Then, the user history such as the estimated user behavior and state is stored in the user history information storage unit 23 (223) in association with the date and time (year, month, week, day, time), etc., and the user history information is updated. (Step S24).

  FIG. 14 schematically shows a specific example of user history information. The user history information in FIG. 14 has a data structure in which a history of user actions and the like is associated with a time zone, a time, and the like. For example, the user departs from home at 8 o'clock, walks from his home to the station in the time zone from 8 o'clock to 8:20, and arrives at the nearest station A at 8:20. And, from 8:20 to 8:45, I got on the train, got off at the nearest B station of the company at 8:45, arrived at the company at 9:00, and started work. During the time zone from 10:00 to 11:00, meetings are held with members in the company, and lunch is served from 12:00 to 13:00.

  As described above, in FIG. 14, the user history information is constructed by associating the history of the user's behavior or the like estimated by the information from the sensor or the like with the time zone or the time.

  In FIG. 14, measurement values such as a user's utterance amount, a meal amount, a pulse rate, and a sweat amount measured by a sensor or the like are associated with the time zone and the time. For example, in the time zone from 8:00 to 8:20, the user is walking from the home to the station A, but the walking amount at this time is measured by the sensor and is associated with the time zone from 8:00 to 8:20. It is done. In this case, for example, measurement values of sensor information other than the walking amount such as walking speed and sweating amount may be further associated. By doing so, it becomes possible to grasp the user's momentum and the like in this time zone.

  In the time zone from 10 o'clock to 11 o'clock, the user is having a meeting with a colleague, and the amount of speech, etc. at this time is measured by a sensor and correlated with the time zone from 10 o'clock to 11 o'clock. In this case, for example, measurement values of sensor information such as a voice state and a pulse may be further associated. By doing so, it becomes possible to grasp the user's conversation volume, degree of tension, and the like during this time period.

  In the time zone from 20:45 to 21:45 or from 22:00 to 23:00, the user is playing a game or watching TV, but the pulse, sweating amount, etc. at this time are in these time zones. Associated. By doing so, it becomes possible to grasp the degree of excitement of the user in these time zones.

  In the time zone from 23:30, the user is sleeping, but a change in the user's body temperature at this time is associated with this time zone. By doing so, it becomes possible to grasp the health condition of the user during sleep.

  Note that the user history information is not limited to the form as shown in FIG. 14. For example, the user history information can be modified without associating the history of the user's behavior with the date and time.

  For example, in FIG. 15A, the mental state parameter of the user is calculated according to a predetermined calculation formula based on the utterance amount, the voice state, the pulse rate, the sweating amount, and the like, which are measured values of sensor information. For example, if the utterance amount is large, the mental state parameter also becomes high, indicating that the user's mental state is good. In addition, based on a walking amount, walking speed, body temperature, and the like, which are measurement values of sensor information, a user body condition (health state) parameter (exercise amount parameter) is calculated according to a predetermined calculation formula. For example, if the amount of walking is large, the body condition parameter also increases, indicating that the user's body condition is good.

  As shown in FIG. 15B, the mental state and body state parameters (state parameters in a broad sense) of the user can be displayed on a wearable display or a home display by using a bar graph or the like. It is also possible to control the robot in the home environment based on the mental state and body state parameters updated in the mobile environment, and to cause the robot to perform operations that encourage, encourage, or advise the user. become.

  As described above, in the present embodiment, user history information that is at least one of a user action history, a state history, and an environment history is acquired as user information. Then, based on the acquired user history information, a process for determining the presentation information that the robot presents to the user is performed.

9. Robot Conversation Based on Scenario Next, a specific example will be described in detail, taking as an example the case where the information presented to the user is a robot conversation based on the scenario.

9.1 Configuration FIG. 16 shows a more detailed system configuration example of the present embodiment. In FIG. 16, the processing unit 10 further includes an event determination unit 11, a user identification unit 15, a contact state determination unit 16, an utterance right control unit 17, a scenario data acquisition unit 18, and a user information update unit as compared to FIGS. 2 and 5. 19 is included. The storage unit 20 includes a scenario data storage unit 27 and a presentation permission determination information storage unit 28.

  The event determination unit 11 performs determination processing for various events. Specifically, a user who has updated user information in the mobile subsystem or in-vehicle subsystem determines the occurrence of a robot available event indicating that the robot in the home subsystem is newly available. . For example, the event determination unit 11 determines that a robot usable event has occurred when the user approaches (moves) the robot location (house). Alternatively, when information transfer is performed wirelessly, the occurrence of an available event may be determined by detecting the wireless strength. Alternatively, it may be determined that an available event has occurred when a portable electronic device is connected to the cradle. When such a robot usable event occurs, the robots 1 and 2 are activated, and the user information is downloaded to the user information storage unit 22 and the like.

  The scenario data storage unit 27 stores scenario data composed of a plurality of conversation phrases as presentation information. And the presentation information determination part 14 determines the conversation phrase which a robot utters with respect to a user based on this scenario data. Then, the robot control unit 30 performs control for causing the robot to utter the determined conversation phrase.

  Specifically, the scenario data storage unit 27 stores scenario data in which a plurality of conversation phrases are linked in a branch structure. And the presentation information determination part 14 determines the conversation phrase which a robot speaks next based on the reaction of the user (1st user) with respect to the conversation phrase which the robot spoke.

  The user identification unit 15 performs user identification processing. Specifically, when the user approaches the robot, the approaching user is identified. The robot control unit 30 performs robot control for presenting the presentation information to the first user when the user identification unit 15 identifies that the first user has approached.

  This user identification can be realized by, for example, image recognition of the user's face by the robot, voice recognition of the user's voice, or the like. For example, the face image and voice data of the first user are registered in advance. Then, the face image and voice of the user approaching the robot are recognized by an imaging device such as a CCD and a sound sensor such as a microphone, and it is determined whether or not the face image and voice registered are the same. Then, when the recognized face image or sound matches the face image or sound of the first user, presentation of the presentation information to the first user is started. Alternatively, the robot receives the ID information from the portable electronic device possessed by the user, and determines whether or not the approaching user is the first user by determining whether or not it matches the ID information registered in advance. You may identify.

  The contact state determination unit 16 determines the contact state on the sensing surface of the robot, as will be described later. Then, the presentation information determination unit 14 determines whether or not the user has performed an operation of stroking or striking the robot as a response to the utterance of the conversation phrase (presentation of presentation information) by the robot. Judge based on the results. Then, a conversation phrase (presentation information to be presented next) to be uttered next to the user is determined.

  In this case, the contact state determination unit 16 outputs an output obtained by performing arithmetic processing on an output signal (sensor signal) from a microphone (sound sensor) provided inside the sensing surface (inside the robot). The contact state on the sensing surface is determined based on the data. The output data in this case is, for example, signal strength (signal strength data), and the contact state determination unit 16 compares the signal strength with the predetermined threshold value using the output data, so that the user can move the robot. It is possible to determine whether a stroking action or a hitting action has been performed.

  The utterance right control unit 17 utters the next conversation phrase to any of the robots 1 and 2 based on the reaction (for example, stroking, striking, silent) of the user (first user) to the conversation phrase spoken by the robot. Control whether to grant the right (speaking initiative). Specifically, the right to speak the next conversation phrase depends on whether the user has made a positive reaction or a negative reaction to the conversation phrase spoken by one of the robots 1 and 2. Determine the robot to be given. For example, an utterance right is given to a robot to which the user has made a positive reaction, or an utterance right is given to a robot to which the user has not made a negative reaction. This utterance right control process can be realized by using an utterance right flag indicating which of the robots 1 and 2 is to be given the utterance right.

  For example, in FIG. 17A, the father responds positively to the conversation phrase “It seems that the club activities are busy” of the robot 1 by stroking his head. Therefore, in this case, as shown in FIG. 17B, the next utterance right is given to the robot 1 whose head has been stroked (the person who has made a positive reaction). As a result, the robot 1 to which the utterance right is given utters the conversation phrase “Because it is a district tournament soon”. That is, for example, the robots 1 and 2 speak in principle alternately. According to this principle, the next utterance right should be given to the robot 2 in FIG. 17B, but FIG. ), The next utterance right is given to the robot 1 whose head is stroked by the father. In FIG. 17A, when the robot 2 speaks and the father makes a negative reaction that the head of the robot 2 is hit, the right to speak may be granted to the robot 1.

  In FIG. 18A, the father responds positively to the robot 2 in response to the conversation phrase “But somehow bad mood”. Therefore, in this case, as shown in FIG. 18B, the next utterance right is granted to the robot 2 whose head has been stroked as a rule. As a result, the robot 2 to which the right to speak is given speaks the conversation phrase “It was hit three times today! In FIG. 18A, when the robot 1 speaks and the father makes a negative reaction that the head of the robot 1 is struck, the right to speak may be granted to the robot 2.

  For example, if the robots 1 and 2 always talk alternately one after another, the conversation between the robots 1 and 2 becomes monotonous for the user, and the user is quickly bored.

  On the other hand, if the utterance right control method of FIG. 17 (A) to FIG. 18 (B) is used, since the grant of the utterance right is switched depending on the reaction of the user, it is possible to prevent the conversation from becoming monotonous, It is possible to realize conversations between robots that make it difficult for users to get bored.

  The scenario data acquisition unit 18 performs scenario data acquisition processing. Specifically, the scenario data used for the conversation of the robot is acquired by reading the scenario data corresponding to the user information from the scenario data storage unit 27. The scenario data selected based on the user information is downloaded to the scenario data storage unit 27 via the network, and the scenario data used for the robot conversation is selected and read from the downloaded scenario data. May be.

  In this embodiment, for example, as described in FIGS. 6A to 6C, scenario data is created based on the reaction of the second user (child) to the conversation phrase spoken by the robot, The scenario data acquisition unit 18 acquires the created scenario data. And the presentation information determination part 14 determines the conversation phrase which a robot speaks with respect to a 1st user based on the acquired scenario data.

  In this way, the contents of the scenario presented to the first user change according to the reaction of the second user to the conversation phrase of the robot, and various and varied robot conversations can be realized. . For example, in FIG. 6B, a positive reaction that the child strokes the head of the robot 1 is made in response to the utterance of “the club activities are busy” of the robot 1. For this reason, in FIG. 7B and FIG. 7C, a scenario (conversation phrase) related to the topic of a child's club activities is selected and presented to the father.

  The user information update unit 19 performs user information update processing in the home environment. More specifically, the user information is further updated in the home environment by sensing the user's behavior, state, and the like via a conversation with the robot.

  The presentation permission judgment information storage unit 28 stores presentation permission judgment information (presentation permission judgment flag) for judging permission / non-permission of information presentation between users. When the presentation information determination unit 14 determines that information presentation between the first and second users is permitted based on the presentation permission determination information, the first user based on the second user information The process of determining the information presented to

  FIG. 19 shows an example of the presentation permission determination information. For example, in FIG. 19, information presentation is permitted between the user A and the user B, but information presentation is not permitted between the user A and the users C and D. Information presentation is permitted between user B and user E, but information presentation is not permitted between user B and users C and D.

  Therefore, for example, when the user A approaches the robot, the presentation information based on the user information of the user B is permitted to be presented to the user A, but the presentation information based on the user information of the user C is allowed to the user A. Presentation is prohibited.

  For example, it may not be desirable to allow all members of the family to present child information. Therefore, the presentation permission determination information is used to permit the presentation of child information to the father, while the child information is not permitted to be presented to the mother.

  In this way, when the father approaches the robot, the robot determines that presentation of child information is permitted based on the presentation permission determination information, and presents information based on the child user information. To do. On the other hand, when the mother approaches the robot, the robot determines that the presentation of the child information is prohibited based on the presentation permission determination information, and does not present the information based on the child user information. To. In this way, information on other users can be presented only to the necessary users, and it is possible to prevent problems such as privacy infringement.

  Next, the detailed operation of this embodiment will be described with reference to the flowchart of FIG.

  First, as described with reference to FIGS. 6A to 6C, scenario data created based on the reaction of the second user (child) to the conversation phrase spoken by the robot is acquired (step S31).

  Next, it is determined whether or not the user has approached the robot (step S32). That is, it is determined whether or not a robot usable event has occurred by detecting the connection of the portable electronic device to the cradle, detecting the wireless strength, or the like.

  Next, a user approaching the robot is identified (step S33). That is, the user is identified based on image recognition, voice recognition, or the like. Then, presentation permission determination information associated with the identified user is read from the presentation permission determination information storage unit 28 (step S34).

  Next, it is determined whether or not the identified user is the first user permitted to present information based on the presentation permission determination information (step S35). For example, when presentation of information of a second user who is a child is permitted only to the first user who is a father, it is determined whether or not the user who has approached the robot is a father. .

  If it is determined that the user is the first user, as described in FIGS. 7A to 7C, the robot 1 or 2 is uttered based on the scenario data acquired in step S31. The conversation phrase to be performed is determined (step S36). Then, robot control for causing the robots 1 and 2 to speak different conversation phrases is performed (step S37).

  Next, the first user's reaction to the utterances of the robots 1 and 2 is monitored (step S38). Then, the utterance right of the next conversation phrase is determined by the method shown in FIGS. 17A to 18B (step S39). Further, based on the reaction of the first user, a conversation phrase to be spoken next by the robots 1 and 2 is determined (step S40).

9.2 Specific Example of Scenario Next, specific examples of scenario data and its selection method used in the present embodiment will be described.

  As shown in FIG. 21, a scenario number (scenario No.) is assigned to each scenario data in the scenario database (scenario DB). The scenario data specified by the scenario number is composed of a plurality of scenario data codes, and each conversation phrase (text data) is designated by each scenario data code. In FIG. 21, scenario data of scenario number = 0579 is selected by the second user information. The scenario data of scenario number = 0579 is composed of scenario data codes A01 to A06. Here, A01 to A06 are codes of conversation phrases that are sequentially spoken by the robot. By using this scenario data, the robot conversation according to the second user information described in FIGS. 3A to 3C and the like is realized.

  FIG. 22 is an example of a scenario in which the topic of a child is provided to a father.

  For example, the robot first says, “Bo-chan seems to be busy with club activities recently,” and then says, “I want to go on a trip during the summer vacation.” If the father strokes the robot for this utterance, it is assumed that the father is interested in the child's request for a summer vacation trip. Therefore, in this case, the robot utters “It's the sea in the summer” and tells the father of the child's request obtained by the conversation with the child. And continue the conversation on the topic of summer vacation.

  On the other hand, if the father is unresponsive to the utterance “I want to go on a trip during the summer vacation,” it is assumed that the father is not interested in the topic, and the robot Change it and say, for example, "Is studying a mess?" If the father strokes the robot in response to this utterance, it is assumed that the father is interested in studying the child. Therefore, in this case, the robot speaks “It seems that club activities are busy now…” and continues the conversation on the topic of studying the child.

  In this way, in FIG. 22, the conversation phrase to be uttered next by the robot is determined based on the father's reaction to the conversation phrase uttered by the robot. In addition, by detecting reactions such as stroking and hitting the father, it is estimated what kind of child the father is interested in.

  FIG. 23 is an example of a scenario in which child user information is collected through conversations between the robots 1 and 2.

  In addition, the robot 1 speaks to the child, “Today is the school return late,” and the robot 2 speaks, “There are many late days recently.” Subsequently, the robot 1 speaks “Is your club activity busy?”, And the robot 2 speaks “Why are you walking around?”

  If the child strokes the robot 1, it is estimated that the child has returned late due to club activities. In this case, the utterance right is given to the robot 1, and the robot 1 utters “Speaking of that, it is a district tournament soon”. And conversation about the topic of club activities continues.

  On the other hand, when the child hits the robot 2, the robot 2 is given the right to speak, and the robot 2 speaks “Well!

  In this way, in FIG. 23, the second user information of the child is collected and updated through the conversation between the robots 1 and 2. Therefore, even if the child is not particularly conscious, the second user information of the child can be acquired naturally.

  FIG. 24 is an example of a scenario presented to the father based on the second user information collected in FIG.

  First, according to the scenario based on the second user information collected in FIG. 23, the robot 1 speaks “It was late for Botchan school”, and the robot 2 “Looks like there are many late days ~” Say. Subsequently, the robot 1 speaks "It seems that the club activities are busy", and the robot 2 speaks "But somehow bad mood". That is, the robots 1 and 2 utter different conversation phrases for the same second user information.

  And when the father strokes the robot 1, it is estimated that the father is interested in the topic of children's club activities. The robot 1 is given the right to speak, and the robot 1 is "a district tournament soon ~" Speak. And conversation about the topic of children's club activities continues.

  On the other hand, when the father strokes the robot 2, the robot 2 is given the right to speak, and the robot 2 speaks, "Today, I was hit three times!

  In this way, in FIG. 24, the topics of children collected through the conversation between the robots 1 and 2 are presented to the father through the conversation between the robots 1 and 2. Therefore, an indirect communication means through the robots 1 and 2 can be provided.

10. Contact State Determination Next, an example of a specific determination method for operations such as hitting and stroking the robot will be described.

  FIG. 25A shows an example of a stuffed robot 500. The surface of the robot 500 functions as the sensing surface 501. Microphones 502-1, 502-2, and 502-3 are provided inside the sensing surface 501. In addition, a signal processing unit 503 is provided that performs arithmetic processing on output signals from the microphones 502-1, 502-2, and 502-3 and outputs output data.

  As shown in the functional block diagram of FIG. 25B, output signals from the microphones 502-1 502-2... 502-n are input to the signal processing unit 503. The signal processing unit 503 performs noise removal, signal amplification, and the like to process and convert an analog output signal. Then, the signal intensity and the like are calculated and output as digital output data. The contact state determination unit 16 performs processing such as threshold value comparison processing and contact state classification, for example.

  For example, FIG. 26 (A), FIG. 26 (B), and FIG. 26 (C) show three patterns of sound waveforms when the sensing surface 501 is hit, when the sensing surface 501 is stroked, and when speaking to the microphone. It is an example. The horizontal axis of the graph is time, and the vertical axis is signal intensity.

  Paying attention to the signal intensity, it can be seen that the signal intensity is large when tapping in FIG. 26A and when stroking in FIG. In addition, the state becomes temporary when it is struck, but it can be seen that the state continues when it is stroked. Further, as shown in FIG. 26C, for example, the waveform when the speech is “strongly” “a” has a smaller signal intensity when hitting in FIG. 26A than when stroking in FIG. 26B. I understand that.

  By providing a threshold value using these differences, it is possible to detect “striking state”, “boiled state”, and “neither state”. Further, by using a plurality of microphones 502-1, 502-2, and 502-3, a place where the strongest signal is generated can be detected as a “struck part or a stroked part”.

  More specifically, the microphones 502-1, 502-2, and 502-3 embedded in the robot 500 move inside the robot 500 when a user's hand or the like comes into contact with the sensing surface 501 of the robot 500. The propagating sound is detected and converted into an electrical signal.

  The signal processing unit 503 performs noise removal, signal amplification, and A / D conversion on output signals (audio signals) from the microphones 502-1, 502-2, and 502-3, and outputs output data. The signal intensity can be calculated by converting this output data into an absolute value and accumulating it for a certain period of time. Then, the calculated signal strength is compared with a threshold value TH. If the threshold value TH is exceeded, it is determined that “contact” has been detected, and the number of contact state detections is counted. This contact state detection process is repeated for a predetermined time.

  At the time when the predetermined time has elapsed, the contact state determination unit 16 compares the condition set in advance with the above-described contact state detection count, for example, the boiled state according to the following conditions, Detect the hit state. In this case, the number of contact state detections increases because the contact state continues when stroked, but the phenomenon that the number of contact state detections decreases when hit is used. Detect whether it is in the state.

Detection state (number of detections / maximum number of detections) x 100 (%)
Boiled state 25% or more Strike state 10% or more, less than 25% Undetected state Less than 10% By doing this, using at least one microphone, “boiled state”, “striking state”, “none of the states ( Undetected state) ”can be determined. In addition, by dispersing and embedding a plurality of microphones and comparing the number of contact state detections of each microphone, it is possible to determine a portion where contact has occurred.

11. Presentation information determination process based on first and second user information In the present embodiment, the presentation information determination process for the first user is performed in consideration of both the first and second user information, for example. Specifically, the weighting of the first user information and the weighting of the second user information in the process for determining the presentation information to the first user are changed with the passage of time.

  For example, when the first user (father) returns home or comes into contact with the robot, a robot (home subsystem) available event occurs. Specifically, the return of the first user is detected by the GPS of the wearable sensor, the sensor attached to the door of the house, the connection of the portable electronic device to the cradle, etc. When the approach between the first user and the robot is detected by a robot contact sensor or the like, the event determination unit 11 in FIG. 16 determines that a robot usable event has occurred by the first user. That is, it is determined that an available event indicating that the robot is ready for use has occurred.

  In FIG. 27, for example, the first user's outing period (robot unusable period, robot and first user non-proximity period) before the occurrence of the usable event is defined as the first period T1, and the usable event occurs. For example, a later home period (a period in which the robot can be used, a period in which the robot and the first user are in close proximity) is a second period T2.

  In the first period T1, the first user information of the first user (father) and the second user information of the second user (child) are acquired (updated). For example, the first user information (first user history information) is the first user's behavior using the behavior sensor, the state sensor, and the environment sensor of the first user's wearable sensor in the first period T1. It can be acquired by measuring (walking, speaking, eating, etc.), state (fatigue, tension, hunger, mental state, physical state, etc.) or environment (location, brightness, temperature, etc.). Specifically, the user information update unit of portable electronic device 100-1 updates the first user information in the user information storage unit of portable electronic device 100-1 based on the sensor information from these sensors. Thus, the first user information in the first period T1 is acquired.

  Similarly, the second user information of the second user (child) measures the behavior, state, or environment of the second user using the wearable sensor of the second user in the first period T1. You can get it. Specifically, the user information update unit of the portable electronic device 100-2 updates the second user information in the user information storage unit of the portable electronic device 100-2 based on the sensor information from these sensors. Thus, the second user information in the first period T1 is acquired. Note that the second user information can also be acquired through conversation with the robot as described with reference to FIGS. 6 (A) to 6 (C).

  For example, when an available event of the robot 1 occurs, the first and second user information updated in the first period T1 is transferred from the user information storage unit of the portable electronic devices 100-1 and 100-2 to the robot. 1 user information storage unit 22 (user history information storage unit 23). Thereby, the presentation information determination unit 14 performs a presentation information determination process (scenario selection process) that the robot 1 presents to the first user based on the transferred first and second user information. Is possible.

  Even in the second period T2 after the occurrence of the usable event, the action of the first user using the robot-mounted sensor 34 or other sensors (for example, a wearable sensor or a home sensor embedded in the home), The first user information may be updated by measuring the state or environment.

  As illustrated in FIG. 28, the presentation information determination unit 14 determines that the robot 1 is the first based on the first and second user information acquired in the first period T1 (or the second period T2). The presentation information to be presented to the user is determined. Specifically, a scenario determination process for the robot 1 to perform a conversation is performed based on the first and second user information. In this way, for the first user (father) who has returned home, not only the topic about the second user (child), but also the first user's own topic on the go, etc. Can be provided to other users, and awareness of their own actions and the like on the go can be given.

  More specifically, the presentation information determination unit 14 changes the weighting (weighting coefficient) of the first user information and the weighting of the second user information with the passage of time in the presentation information determination process. .

  For example, in FIG. 28, when an available event of the robot 1 occurs (when the user returns home, until a given period elapses from the time of returning home), the weighting of the first user information in the determination process is the second weight. For example, the weight of the first user information is 1.0 and the weight of the second user information is 0.

  In the weight change period TA, the weight of the first user information is reduced, while the weight of the second user information is increased, and the weight is reversed. After the change period TA, the weight of the second user information is larger than the weight of the first user information. For example, the weight of the first user information becomes 0, The user information weight is 1.0.

  Thus, in FIG. 28, when an available event occurs, the weighting of the first user information in the determination process is increased, the weighting of the second user information is decreased, and then the weighting of the first user information is decreased. The weight of the second user information is increased. Specifically, in the second period T2, the weighting of the first user information in the presentation information determination process is decreased with the passage of time, and the weighting of the second user information is increased with the passage of time. Yes.

  In this way, in the first half period of the second period T1, the robot 1 provides topics related to, for example, the behavior of the first user (father) in the first period T1, which is the outing period. Then, after a lapse of time, a topic related to the behavior of the second user (child) is provided.

  In this way, the first user is provided with a topic related to him / herself immediately after returning home, and then the topic of the second user, who is another person, is provided when time has settled. Thus, it is possible to provide a topic more natural for the first user.

  In addition, for example, when the first user returns home when the second user is at home with the robot 1, it is expected that the first user will receive more attention than the second user. The Therefore, immediately after the first user returns home, the topics related to the first user are presented mainly, and when the time has passed and the time has settled, the topics related to the first and second users are provided evenly.

  The method of changing the weight is not limited to the method shown in FIG. For example, contrary to FIG. 28, it is possible to perform a modification in which the weighting of the second user information is increased in the first half period, and then the weighting of the first user information is increased. The weighting may be changed by programming the robot 1 or the like in advance, or by allowing the user to switch freely according to his / her preference.

  Further, when the first user information is acquired (updated) also in the second period T2, the weighting of the first user information acquired in the first period T1 during the presentation information determination process, The weighting of the first user information acquired in the second period T2 may also be changed over time. For example, immediately after the occurrence of an available event of the robot 1, the first user information acquired in the first period T1 is increased in weight, and as time passes, the first user information acquired in the second period T2 is increased. Increase the weight of user information.

  Further, as an example of the weighting of the user information in the presentation information determination process, there is a selection probability of the scenario selected by the user information. Specifically, when the weight of the first user information is increased, the scenario of the first user information is selected rather than the second user information. Specifically, the selection probability of the scenario corresponding to the first user information is increased. On the other hand, when increasing the weight of the second user information, the scenario of the second user information is selected rather than the first user information. Specifically, the selection probability of the scenario corresponding to the second user information is increased.

  For example, in FIG. 28, in the first half of the second period T2, since the weight of the first user information is large, the selection probability of the scenario for the first user information is high. As a result, in the first half period, the robot 1 starts a conversation related to the behavior of the first user on the day. On the other hand, in the second half of the second period T2, since the weight of the second user information is large, the scenario selection probability for the second user information is high. As a result, in the second half period, the robot 1 starts a conversation regarding the behavior of the second user on the day. In this way, the topic of the scenario to be provided can be gradually changed over time, and more natural and diverse robot conversations can be realized.

  Although the present embodiment has been described in detail as described above, it will be easily understood by those skilled in the art that many modifications can be made without departing from the novel matters and effects of the present invention. Accordingly, all such modifications are intended to be included in the scope of the present invention. For example, a term described at least once together with a different term having a broader meaning or the same meaning in the specification or the drawings can be replaced with the different term in any part of the specification or the drawings. Further, the configuration and operation of the robot control system and the robot are not limited to those described in the present embodiment, and various modifications can be made.

Explanatory diagram of user information acquisition method The system configuration example of this embodiment. FIG. 3A to FIG. 3C are explanatory diagrams of the method of this embodiment. The flowchart for demonstrating the operation | movement of this embodiment. The 2nd system configuration example of this embodiment in case there are two or more robots. FIGS. 6A to 6C are explanatory diagrams of a second user information acquisition method. FIG. 7A to FIG. 7C are explanatory diagrams of an information presentation method to the first user. The flowchart for demonstrating operation | movement of a 2nd system configuration. The 3rd system configuration example of this embodiment. Explanatory drawing of the acquisition method of the 2nd user information via a network. The 4th system configuration example of this embodiment. The 5th system configuration example of this embodiment. The flowchart of the update process of user history information. Explanatory drawing of user log | history information. 15A and 15B are explanatory diagrams of user history information. 2 is a detailed system configuration example of the present embodiment. FIG. 17A and FIG. 17B are explanatory diagrams of a speech right control method. FIG. 18A and FIG. 18B are explanatory diagrams of a speech right control method. Explanatory drawing of presentation permission judgment information. The flowchart for demonstrating the detailed operation | movement of this embodiment. Explanatory drawing of scenario data. An example scenario that provides the father with a child's topic. Example scenario used to collect user information for children. The example scenario shown to a father based on the collected 2nd user information. 25A and 25B are explanatory diagrams of the contact determination method. FIG. 26A, FIG. 26B, and FIG. 26C show examples of sound waveforms when the speaker is hit, stroked, or spoken to the microphone. Explanatory drawing of the determination method of the presentation information based on 1st, 2nd user information. Explanatory drawing of the determination process of the presentation information based on 1st, 2nd user information.

Explanation of symbols

1, 2 robot, 10 processing unit, 11 event determination unit, 12 user information acquisition unit,
13 calculation units, 14 presentation information determination unit, 15 user recognition unit, 16 contact state determination unit,
17 utterance right control unit, 18 scenario data acquisition unit, 20 storage unit,
22 user information storage unit, 23 user history information storage unit, 26 presentation information storage unit,
27 scenario data storage unit, 28 presentation permission judgment information storage unit, 30 robot control unit,
32 robot operation mechanism, 34 robot mounted sensor, 40 communication unit,
60 processing units, 62 user information acquisition unit, 63 calculation unit, 64 presentation information determination unit,
70 storage unit, 72 user information storage unit, 73 user history information storage unit,
76 presentation information storage unit, 80 robot control unit, 82 robot operation mechanism,
84 Robot mounted sensor, 90 communication unit,
100-1, 100-2 portable electronic device, 110-1, 110-2 processing unit,
112-1, 112-2 calculation unit, 114-1, 114-2 user information update unit,
120-1, 120-2 storage unit, 122-1, 122-2 user information storage unit,
130-1, 130-2 control unit, 138-1, 138-2 communication unit,
140-1, 140-2 Wearable display,
150-1, 150-2 wearable sensor, 200 home server (local server),
210 processing unit, 212 user information acquisition unit, 213 calculation unit,
214 presentation information determination unit, 220 storage unit,
222 user information storage unit, 223 user history information storage unit,
226 Presentation information storage unit, 300 External server

Claims (25)

A robot control system for controlling a robot,
A user information acquisition unit that acquires user information obtained from sensor information from at least one of a behavior sensor that measures a user's behavior, a state sensor that measures a user's state, and an environment sensor that measures a user's environment;
Based on the acquired user information, a presentation information determination unit that performs a process of determining presentation information that the robot presents to the user;
A robot controller that performs control for causing the robot to present the presentation information to a user,
The user information acquisition unit
Obtaining second user information which is the user information of a second user;
The presentation information determination unit
Based on the acquired second user information, a process for determining the presentation information to the first user is performed,
The robot controller is
A robot control system for performing robot control for presenting presentation information determined based on the second user information to the first user. In claim 1,
The user information acquisition unit
Obtaining the first user information that is the user information of the first user and the second user information that is the user information of the second user;
The presentation information determination unit
A robot control system that performs presentation information determination processing for the first user based on the acquired first user information and second user information. In claim 2,
The presentation information determination unit
Determining the presentation timing of the presentation information based on the first user information, determining the content of the presentation information based on the second user information,
The robot controller is
A robot control system for performing robot control for presenting the presentation information of the decided content to the first user at the decided presentation timing. In claim 2,
The presentation information determination unit
A robot control system characterized in that the weighting of the first user information and the weighting of the second user information in the process of determining the information presented to the first user are changed with time. In claim 4,
Including an event determination unit that determines occurrence of an available event indicating that the robot can be used by the first user;
The presentation information determination unit
When the available event occurs, the weighting of the first user information in the determination process is increased, the weighting of the second user information is decreased, and then the weighting of the first user information is decreased, A robot control system characterized by increasing the weight of the second user information. In any one of Claims 1 thru | or 5,
The presentation information determination unit
A robot control system characterized in that, based on a response of the first user to presentation of presentation information by a robot, the robot performs a process of determining presentation information to be presented next to the first user. In claim 6,
Including a contact state determination unit for determining a contact state on the sensing surface of the robot,
The presentation information determination unit
As a response of the first user to the presentation of presentation information by the robot, it is determined based on the determination result in the contact state determination unit whether the first user has performed an operation of stroking or hitting the robot. A robot control system that performs a process for determining presentation information to be presented next to the first user. In claim 7,
The contact state determination unit
A robot control system for determining a contact state on the sensing surface based on output data obtained by performing arithmetic processing on an output signal from a microphone provided inside the sensing surface . In claim 8,
The output data is signal strength;
The contact state determination unit
A robot control system comprising: comparing the signal intensity with a predetermined threshold value to determine whether the first user has performed an operation of stroking or hitting the robot. In any one of Claims 1 thru | or 9,
The presentation information determination unit
A process for determining the presentation information to be presented to the first user is performed so that the first and second robots present different presentation information for the same second user information acquired. Characteristic robot control system. In claim 10,
The first robot is set on the master side, the second robot is set on the slave side,
The presentation information determination unit provided in the first robot on the master side instructs the second robot on the slave side to present the presentation information to the first user. Robot control system. In claim 11,
A robot control system comprising: a communication unit that communicates instruction information for instructing presentation of presentation information from the first robot on the master side to the second robot on the slave side. In any one of Claims 1 to 12,
The user information acquisition unit
Obtaining the second user information of the second user via a network;
The presentation information determination unit
A robot control system that performs a process of determining presentation information to be presented to the first user based on the second user information acquired via a network. In any one of Claims 1 thru | or 13.
The user information acquisition unit
As the second user information, second user history information that is at least one of the second user behavior history, the second user state history, and the second user environment history is acquired,
The presentation information determination unit
A robot control system that performs presentation information determination processing to be presented to the first user by the robot based on the acquired second user history information. In claim 14,
The robot control system according to claim 2, wherein the second user history information is information obtained by updating with sensor information from the wearable sensor of the second user. In any one of Claims 1 thru | or 15,
A user identification unit for identifying the approached user when the user approaches the robot;
The robot controller is
A robot control system for performing robot control for presenting presentation information to the first user when the user identification unit is identified as approaching the first user. In any one of Claims 1 thru | or 16.
Including a presentation permission judgment information storage unit for storing presentation permission judgment information for judging permission / non-permission of information presentation between users,
The presentation information determination unit
Information presented to the first user based on the second user information when it is determined that information presentation between the first and second users is permitted based on the presentation permission judgment information A robot control system characterized by performing a determination process. In any one of Claims 1 thru | or 17,
A scenario data storage unit that stores scenario data composed of a plurality of conversation phrases as the presentation information;
The presentation information determination unit
Based on the scenario data, determine a conversation phrase that the robot speaks to the first user;
The robot controller is
A robot control system characterized by performing control for causing a robot to utter a determined conversation phrase. In claim 18,
The scenario data storage unit
Stores scenario data in which multiple conversation phrases are linked in a branch structure,
The presentation information determination unit
A robot control system characterized by determining a conversation phrase to be subsequently spoken by the robot based on a reaction of the first user to a conversation phrase spoken by the robot. In claim 18 or 19,
A scenario data acquisition unit that acquires scenario data created based on the reaction of the second user to the conversation phrase spoken by the robot;
The presentation information determination unit
A robot control system, wherein a conversation phrase that the robot speaks to the first user is determined based on scenario data acquired by a reaction of the second user. In any of claims 18 to 20,
The presentation information determination unit
For the same second user information that has been acquired, a conversation phrase to be uttered to the first user is determined so that the first and second robots utter different conversation phrases,
An utterance right control unit that controls which of the first and second robots is granted the utterance right of the next conversation phrase based on the reaction of the first user to the conversation phrase uttered by the robot. A robot control system comprising: In claim 21,
The speaking right control unit
Depending on whether the first user has made a positive reaction or a negative reaction to the conversation phrase spoken by either one of the first and second robots, the next conversation phrase A robot control system characterized by determining a robot to be given the right to speak. The robot control system according to any one of claims 1 to 22,
And a robot operation mechanism which is a control target of the robot control system. A program for robot control,
A user information acquisition unit that acquires user information obtained from sensor information from at least one of a behavior sensor that measures a user's behavior, a state sensor that measures a user's state, and an environment sensor that measures a user's environment;
Based on the acquired user information, a presentation information determination unit that performs a process of determining presentation information that the robot presents to the user;
As a robot controller that performs control for causing the robot to present the presentation information to the user,
Make the computer work,
The user information acquisition unit
Obtaining second user information which is the user information of a second user;
The presentation information determination unit
Based on the acquired second user information, a process for determining the presentation information to the first user is performed,
The robot controller is
A program for performing robot control for presenting presentation information determined based on the second user information to the first user.   An information storage medium readable by a computer, wherein the program according to claim 24 is stored.

Images