JP2023123555A - Autonomous behavior-based robot for recording daily life - Google Patents

Abstract

To convey to a user the daily life experienced by an autonomous behavior-based robot while taking an account of privacy.SOLUTION: A robot 100 comprises: a motion control unit for selecting a motion; a drive mechanism for executing the motion; a recognition unit for recognizing the occurrence of an event; an emotion management unit for changing one or more emotion parameters according to the event; and a history management unit for associating an occurrence history of the event with a change history of the emotion parameters and recording thereof as history information. The robot 100 changes its behavioral characteristics based on the emotion parameters. When the emotion parameter meets a prescribed psychological condition, for example, when a value of the emotion parameter representing curiosity becomes particularly high, the event thereat and the emotion parameter thereat are extracted from the history information for generating a reaction report, and the reaction report is provided to a user.SELECTED DRAWING: Figure 1

Description

The present invention relates to a robot that autonomously selects actions according to its internal state or external environment.

Humans keep pets for comfort. On the other hand, many people give up pets for a variety of reasons, such as not having enough time to take care of pets, lack of living environments that allow pets, allergies, and the pain of bereavement. If there is a robot that can play the role of a pet, it may be possible to give people who cannot keep a pet the comfort that a pet gives (see Patent Documents 1 and 2).

JP-A-2000-323219 WO2017/169826 JP-A-2005-267375 Japanese Patent Application Laid-Open No. 2010-21921 Japanese Patent Application Laid-Open No. 2008-172815

In recent years, robot technology has been rapidly progressing, but it has not yet achieved the presence of a companion like a pet. I don't think robots have free will. By observing the pet's behavior, which can only be assumed to have free will, humans feel the presence of free will in the pet, sympathize with the pet, and are healed by the pet.

Currently, there are many cameras on the market for keeping an eye on pets while they are away. With such a camera, it is possible to know how the pet spends its time when the owner is away. If robots come to have the same presence as pets, it is thought that there will be a need to know the daily life of robots when they are away.

However, simply mounting a camera on a robot and live-relaying the captured image to a smartphone or the like would turn the robot into a mere monitoring device. For example, when user A and the robot are at home, if the robot transmits the photographed image as it is to the smartphone of user B who is out, user A's privacy may not be protected.

The present invention has been completed based on the recognition of the above problems, and its main purpose is to provide a technique for informing a user of the daily life of a robot while taking privacy into consideration.

An autonomous action type robot according to one aspect of the present invention includes a motion control unit that selects a motion of the robot, a drive mechanism that executes the motion selected by the motion control unit, a recognition unit that recognizes occurrence of an event, and an emotion management unit that changes one or more emotion parameters according to the response; and a history management unit that associates event occurrence histories with emotion parameter change histories and records them as history information.
The motion control unit changes behavioral characteristics of the robot based on the emotion parameter.
The history management unit extracts an event when the emotional parameter satisfies a predetermined psychological condition and the emotional parameter when the psychological condition is satisfied from the history information, generates a reaction report, and provides the reaction report to the user.

ADVANTAGE OF THE INVENTION According to this invention, it becomes easy to inform a user about the daily life of a robot, considering privacy.

FIG. 3 is a conceptual diagram for explaining how the robot communicates its daily life to the user; FIG. 2(a) is a front external view of the robot. FIG. 2(b) is a side external view of the robot. It is a cross-sectional view schematically showing the structure of the robot. It is a hardware block diagram of the robot in a basic configuration. 1 is a functional block diagram of a robot system; FIG. FIG. 4 is a schematic diagram showing a reaction report generation process in this embodiment; It is a functional block diagram of the robot in this embodiment. FIG. 4 is a schematic diagram showing a change history of emotion parameters; 4 is a data structure diagram of history information; FIG. FIG. 4 is a data structure diagram of a motion selection table; FIG. 11 is a screen diagram of a first reaction report screen; FIG. 10 is a screen diagram of a second reaction report screen; 4 is a data structure diagram of a user authority table; FIG. 4 is a data structure diagram of an event condition table; FIG. FIG. 10 is a sequence diagram showing a display processing process of a reaction report; FIG. 11 is a sequence diagram showing a detailed display process; FIG. 11 is a screen diagram of a detailed information screen; FIG. 11 is a screen diagram of a third reaction report screen; It is a screen diagram of a rest time setting screen.

FIG. 1 is a conceptual diagram for explaining how the robot 100 communicates its daily life to the user.
The robot 100 in this embodiment has multiple types of “emotions”. Specifically, the robot 100 numerically expresses the intensity of three types of emotions, loneliness, curiosity, and desire for approval, using three types of emotion parameters. As will be described later, the robot 100 may express emotions using emotion parameters other than these three types of emotion parameters.

The robot 100 is equipped with multiple types of sensors, and recognizes the external environment by each sensor.
The robot 100 recognizes some changes in the external environment as "events". The event is
It may be arbitrarily defined as a "event of interest" such as seeing a user or detecting a loud sound. The robot 100 autonomously chooses actions according to events and emotional parameters. The event affects both the behavior selection method and the emotion parameters of the robot 100, and the emotion parameters also affect the behavior selection method. Details of events, emotion parameters, and action selection methods will be described later.

The robot 100 records "everyday events" such as events as "history information" one by one. Based on the history information, the robot 100 generates a "reaction report" for the user to see. A user obtains a reaction report from a user terminal 400 such as a smart phone, laptop PC, tablet computer, or the like. By displaying the reaction report on the user terminal 400, the user can catch a glimpse of the daily life of the robot 100 even when the user is not in contact with the robot 100. FIG.

Even when the user is out, the user can know how the robot 100 at home is spending the day and how the robot 100 spent the day through the reaction report. The reaction report functions as a "diary" for the robot 100, so to speak. The reaction report in this embodiment is "history information for showing", and is not the same as the history information recorded by the robot 100 one by one. Historical information may contain information related to privacy. In this embodiment, privacy is taken into account, and a reaction report that is easy for the user to read is generated again.
Hereinafter, after the basic configuration of the robot 100 will be described with reference to FIGS. 2 to 5, the method of generating the reaction report will be mainly described.

[Basic configuration]
FIG. 2(a) is a front external view of the robot 100. FIG. FIG. 2(b) is a side external view of the robot 100. FIG.
The robot 100 in this embodiment is an autonomous action type robot that determines actions based on the external environment and internal state. The external environment is recognized by various sensors such as cameras and thermosensors. The internal state is quantified as various parameters that express the robot's 100 emotions. The robot 100 has a range of action within the house of the owner's family. below,
A person involved in the robot 100 is called a "user".

A body 104 of the robot 100 has an overall rounded shape and includes an outer skin made of a soft and elastic material such as urethane, rubber, resin, or fiber. robot 1
You can dress 00. Robot 100 has a total weight of 5-15 kg and a height of 0.
.5 to 1.2 meters. Various attributes such as appropriate weight, roundness, softness, and good touch realize the effect that the user can easily hold the robot 100 and want to hold it.

The robot 100 has a pair of front wheels 102 (a left wheel 102a and a right wheel 102b) and one rear wheel.
Including 103. The front wheels 102 are driving wheels and the rear wheels 103 are driven wheels. The front wheel 102 is
Although it does not have a steering mechanism, it is possible to individually control the rotation speed and rotation direction. rear wheel 103
are casters, which are rotatable to move the robot 100 forward, backward, leftward, and rightward. Rear wheel 103 may be an omni wheel.

The front wheels 102 and the rear wheels 103 are connected to the body 10 by a drive mechanism (rotating mechanism, link mechanism).
4 can be fully accommodated. Most of each wheel is hidden in the body 104 even during running,
When each wheel is completely housed in the body 104, the robot 100 becomes immovable. That is, the body 104 descends and seats on the floor surface F as the wheels are retracted. In this seated state, a flat seating surface 108 (ground bottom surface) formed on the bottom of the body 104
abuts on the floor F.

Robot 100 has two hands 106 . The hand 106 does not have the function of grasping an object. The hand 106 can be raised, shaken, or vibrated for simple actions. The two hands 106 are also individually controllable.

The eye 110 is capable of image display using a liquid crystal element or an organic EL element. robot 10
0 is equipped with various sensors such as a microphone array and an ultrasonic sensor that can identify the direction of the sound source. It also has a built-in speaker that can emit simple sounds.

A horn 112 is attached to the head of the robot 100 . Robot 100 as above
is light, the user can lift the robot 100 by grasping the horns 112 . An omnidirectional camera is attached to the horn 112, and the entire upper area of the robot 100 can be imaged at once.

FIG. 3 is a cross-sectional view schematically showing the structure of robot 100. As shown in FIG.
As shown in FIG. 3, the body 104 of the robot 100 includes a base frame 308, a body frame 310, a pair of resin wheel covers 312 and an outer skin 314. As shown in FIG. The base frame 308 is made of metal, configures the axis of the body 104, and supports the internal mechanism. The base frame 308 is configured by vertically connecting an upper plate 332 and a lower plate 334 with a plurality of side plates 336 . multiple side plates 336
Sufficient space is provided between them to allow ventilation. Inside the base frame 308, a battery 118, a control circuit 342 and various actuators are accommodated.

Body frame 310 is made of a resin material, and includes head frame 316 and torso frame 31.
Including 8. The head frame 316 has a hollow hemispherical shape and forms the head skeleton of the robot 100 . The torso frame 318 has a stepped cylindrical shape and forms the torso skeleton of the robot 100 . A torso frame 318 is fixed integrally with the base frame 308 . head frame 31
6 is attached to the upper end of the body frame 318 so as to be relatively displaceable.

The head frame 316 is provided with three axes, a yaw axis 320, a pitch axis 322 and a roll axis 324, and an actuator 326 for rotationally driving each axis. Actuator 32
6 contains multiple servo motors to drive each axis individually. The yaw axis 320 is driven for swing motion, the pitch axis 322 is driven for nod motion, and the roll axis 324 is driven for tilt motion.

A plate 325 that supports the yaw axis 320 is fixed to the top of the head frame 316 . A plurality of ventilation holes 327 are formed in the plate 325 to ensure ventilation between the upper and lower sides.

A metal base plate 328 is provided to support the head frame 316 and its internal mechanisms from below. The base plate 328 is connected to the plate 325 via a cross link mechanism 329 (pantograph mechanism), and is connected via a joint 330 to an upper plate 332 (base frame 308).

Torso frame 318 houses base frame 308 and wheel drive mechanism 370 . Wheel drive mechanism 370 includes pivot 378 and actuator 379 . torso frame 31
The lower half of 8 is narrowed to form a storage space S for the front wheel 102 between the wheel cover 312 and the wheel cover 312 .

Outer skin 314 is made of urethane rubber, and is attached to body frame 310 and wheel cover 31.
2 is covered from the outside. Hand 106 is integrally molded with skin 314 . The upper end of the outer skin 314 is provided with an opening 390 for introducing outside air.

FIG. 4 is a hardware configuration diagram of the robot 100. As shown in FIG.
The robot 100 includes an internal sensor 128, a communication device 126, a storage device 124, a processor 1
22, including drive mechanism 120 and battery 118; Processor 122 and storage device 12
4 is included in the control circuit 342 . Each unit is connected to each other by a power line 130 and a signal line 132 . Battery 118 supplies power to each unit via power line 130 . Each unit transmits and receives control signals through signal line 132 . The battery 118 is a lithium ion secondary battery and is the power source of the robot 100 .

The internal sensor 128 is a collection of various sensors built into the robot 100 . Specifically, they are cameras (omnidirectional cameras), microphone arrays, ranging sensors (infrared sensors), thermosensors, touch sensors, acceleration sensors, odor sensors, and the like. The touch sensor
It is installed between the skin 314 and the body frame 310 and detects the user's touch. An odor sensor is a known sensor that applies the principle that electrical resistance changes due to the adsorption of odor-causing molecules.

The communication device 126 is a communication module that performs wireless communication with various external devices.
The storage device 124 is composed of non-volatile memory and volatile memory, and stores computer programs and various setting information. Processor 122 is a computer program execution means. The drive mechanism 120 includes a plurality of actuators and the wheel drive mechanism 3 described above.
Including 70. It also has a display and speakers.

Drive mechanism 120 primarily controls the wheels (front wheels 102) and the head (head frame 316). The drive mechanism 120 can change the movement direction and movement speed of the robot 100, and can also move the wheels (the front wheels 102 and the rear wheels 103) up and down. When the wheels are raised, the wheels are completely housed in the body 104, and the robot 100 contacts the floor surface F at the seating surface 108,
It becomes a seated state. Drive mechanism 120 also controls hand 106 via wire 134 .

FIG. 5 is a functional block diagram of the robot system 300. As shown in FIG.
Robot system 300 includes robot 100, server 200 and multiple external sensors 1
Including 14. Each component of the robot 100 and the server 200 has a CPU (Central Proc
Processing Unit) and computing units such as various coprocessors, storage devices such as memory and storage, hardware including wired or wireless communication lines connecting them, and software stored in the storage device and supplying processing instructions to the computing unit realized by A computer program may consist of a device driver, an operating system, various application programs located in their higher layers, and a library that provides common functions to these programs. Each block described below represents a functional block rather than a hardware configuration.
A part of the functions of the robot 100 may be realized by the server 200, or the server 200
may be implemented by the robot 100 in part or in whole.

A plurality of external sensors 114 are installed in advance in the house. The position coordinates of the external sensor 114 are registered in the server 200 . The server 200 determines basic actions of the robot 100 based on information obtained from the robot's 100 internal sensor 128 and the plurality of external sensors 114 . The external sensor 114 is for reinforcing the sensory organs of the robot 100 and the server 200 is for reinforcing the brain of the robot 100 . The communication device 126 of the robot 100 periodically communicates with the external sensor 114, and the server 200 identifies the position of the robot 100 using the external sensor 114 (see also Patent Document 2).

(server 200)
Server 200 includes communication unit 204 , data processing unit 202 and data storage unit 206 .
The communication unit 204 is in charge of processing communication with the external sensor 114 and the robot 100 . A data storage unit 206 stores various data. The data processing unit 202 executes various processes based on the data acquired by the communication unit 204 and the data stored in the data storage unit 206. FIG. Data processing unit 202 also functions as an interface for communication unit 204 and data storage unit 206 .

Data store 206 includes motion store 232 and personal data store 218 .
The robot 100 has a plurality of action patterns (motions). Various motions are defined, such as shaking the hand 106, approaching the owner while meandering, and staring at the owner while tilting the head.

The motion storage unit 232 stores a "motion file" that defines the contents of motion control.
to store Each motion is identified by a motion ID. Motion files are also downloaded to the motion store 160 of the robot 100 . Which motion to perform may be determined by the server 200 or may be determined by the robot 100 .

Most of the motions of the robot 100 are configured as compound motions including multiple unit motions. For example, when the robot 100 approaches the owner, it may be expressed as a combination of a unit motion of turning toward the owner, a unit motion of approaching while raising hands, a unit motion of approaching while shaking the body, and a unit motion of sitting while raising both hands. . By combining these four motions, it is possible to achieve the motion of "approaching the owner, raising a hand halfway through, and finally shaking the body before sitting down." In the motion file, the rotation angles and angular velocities of the actuators provided in the robot 100 are defined in association with the time axis. Motion file (actuator control information)
Various motions are expressed by controlling each actuator according to the passage of time.

The transition time when changing from the previous unit motion to the next unit motion is called "interval". The interval may be defined according to the time required to change the unit motion and the contents of the motion. The interval length is adjustable.
Hereinafter, settings related to action control of the robot 100, such as when and which motion to select, output adjustment of each actuator for realizing the motion, etc., are collectively referred to as "behavior characteristics". The behavioral characteristics of the robot 100 are defined by motion selection algorithms, motion selection probabilities, motion files, and the like.

The motion storage unit 232 stores motion files as well as motion selection tables that define motions to be executed when various events occur. In the motion selection table, one or more motions and their selection probabilities are associated with events.

Personal data storage unit 218 stores user information. Specifically, it stores master information indicating familiarity with the user and the user's physical features/behavioral features. Other attribute information such as age and gender may be stored.

The robot 100 has an internal parameter of intimacy for each user. robot 10
When 0 recognizes actions that show goodwill toward him, such as picking him up or calling out to him, his intimacy with that user increases. Users who are not involved in Robot 100,
Intimacy is low for users who act violently, and for users who meet infrequently.

Data processing unit 202 includes location management unit 208 , recognition unit 212 , operation control unit 222 , familiarity management unit 220 and state management unit 244 .
The position management unit 208 identifies position coordinates of the robot 100 . The state management unit 244 manages various internal parameters such as the charging rate, internal temperature, and various physical states such as the processing load of the processor 122 . In addition, the state management unit 244 manages various emotion parameters indicating the emotions of the robot 100 (loneliness, curiosity, desire for approval, etc.). These emotional parameters are constantly fluctuating. The movement target point of the robot 100 changes according to the emotion parameter. For example, when loneliness is increasing, the robot 100 sets the location of the user as the movement target point.

Emotional parameters change over time. In addition, various emotion parameters change depending on the later-described response behavior. For example, when the owner hugs the pet, the emotional parameter indicating loneliness decreases, and when the owner is not seen for a long time, the emotional parameter indicating loneliness gradually increases.

The recognition unit 212 recognizes the external environment. Recognition of the external environment includes various recognitions such as recognition of weather and seasons based on temperature and humidity, and recognition of shadows (safe zones) based on light intensity and temperature.
The recognition unit 156 of the robot 100 acquires various types of environmental information from the internal sensor 128 , performs primary processing on the information, and transfers it to the recognition unit 212 of the server 200 .

Specifically, the recognition unit 156 of the robot 100 extracts an image region corresponding to a moving object, particularly a person or an animal, from the image, and extracts features representing physical and behavioral features of the moving object from the extracted image region. Extract a "feature vector" as a set of quantities. A feature vector component (feature amount) is a numerical value obtained by quantifying various physical/behavioral features. For example, the width of the human eye is 0~1
to form one feature vector component. A technique for extracting feature vectors from a captured image of a person is an application of known face recognition technology. Robot 100 sends the feature vector to server 200 .

The recognition unit 212 of the server 200 compares the feature vector extracted from the image captured by the built-in camera of the robot 100 with the feature vector of the user (cluster) registered in advance in the personal data storage unit 218, thereby obtaining the captured image. It is determined which person the user who receives the information corresponds to (user identification processing). In addition, the recognition unit 212 estimates the user's emotion by performing image recognition of the user's facial expression. The recognition unit 212 also performs user identification processing on moving objects other than people, such as pets such as cats and dogs.

The recognition unit 212 recognizes various behaviors performed by the robot 100 and classifies them into pleasant and unpleasant behaviors. The recognition unit 212 also recognizes the owner's response to the action of the robot 100 and classifies it into positive and negative reactions.
Pleasant/unpleasant behavior is determined by whether the user's response behavior is biologically pleasant or unpleasant. For example, being hugged is a pleasant action for the robot 100, and being kicked is an unpleasant action for the robot 100. A positive/negative reaction is determined depending on whether the user's behavior indicates a user's pleasant feeling or an unpleasant feeling. Being hugged is a positive reaction that indicates a user's pleasant feeling, and being kicked is a negative reaction that indicates a user's unpleasant feeling.

The motion control unit 222 of the server 200 cooperates with the motion control unit 150 of the robot 100 to
Determine the motion of the robot 100. The motion control unit 222 of the server 200 is a robot
Create 100 movement target points and a movement route for them. The motion control unit 222 may create a plurality of travel routes and then select one of the travel routes.

The motion control unit 222 selects the robot 100 from a plurality of motions in the motion storage unit 232.
Select the motion of Each motion is associated with a selection probability for each situation.
For example, when the owner performs a pleasant act, motion A is executed with a probability of 20%, and when the temperature exceeds 30 degrees, motion B is executed with a probability of 5%. .

The familiarity management unit 220 manages the familiarity of each user. As described above, the degree of intimacy is registered as part of the personal data in the personal data storage unit 218. FIG. When a pleasant act is detected, the degree of intimacy management unit 220 increases the degree of intimacy with the owner. The degree of intimacy is lowered when an unpleasant act is detected. In addition, the intimacy level of the owner who has not seen it for a long time gradually decreases.

(Robot 100)
Robot 100 includes communication unit 142 , data processing unit 136 , data storage unit 148 , internal sensors 128 and drive mechanism 120 .
The communication unit 142 corresponds to the communication device 126 (see FIG. 4), the external sensor 114, the server 20
0 and other robots 100 are in charge of communication processing. The data storage unit 148 stores various data. The data storage unit 148 corresponds to the storage device 124 (see FIG. 4). Data processing unit 136 executes various processes based on data acquired by communication unit 142 and data stored in data storage unit 148 . The data processing unit 136 includes the processor 12
2 and a computer program executed by the processor 122 . The data processing unit 136 includes the communication unit 142, the internal sensor 128, the driving mechanism 120 and the data storage unit.
It also functions as an interface for 148.

The data storage unit 148 is a motion storage unit that defines various motions of the robot 100.
Including 160.
The motion storage unit 160 of the robot 100 stores the motion storage unit 23 of the server 200.
Various motion files are downloaded from 2. A motion is identified by a motion ID. Two front wheels, holding front wheels 102 and seated, lifting hands 106
Rotate the robot 100 by rotating 102 in reverse or by rotating only one of the front wheels 102. Rotate the front wheels 102 while the front wheels 102 are retracted to make the robot 100 shake. In order to express various motions such as looking back, the motion timing, motion time, motion direction, etc. of various actuators (driving mechanism 120) are defined in chronological order in the motion file.
Various data may also be downloaded to the data storage unit 148 from the personal data storage unit 218 .

Data processing unit 136 includes recognition unit 156 and operation control unit 150 .
The motion control unit 150 of the robot 100 determines the motion of the robot 100 in cooperation with the motion control unit 222 of the server 200 . Some motions may be determined by the server 200 and other motions may be determined by the robot 100 . Also, although the robot 100 determines the motion, the server 200 may determine the motion when the processing load of the robot 100 is high. A base motion may be determined at the server 200 and an additional motion may be determined at the robot 100 . The robot system 3 determines how the server 200 and the robot 100 share motion determination processing.
It should be designed according to the specifications of 00.

The motion control unit 150 of the robot 100 instructs the drive mechanism 120 to execute the selected motion. Drive mechanism 120 controls each actuator according to the motion file.

The motion control unit 150 can also perform a motion of raising both hands 106 as a gesture of asking for a hug when a user with a high degree of intimacy is nearby, or can perform a motion of hugging.
When you get tired of it, by alternately repeating reverse rotation and stopping while the left and right front wheels 102 are accommodated, you can express a motion that you don't want to hold. The drive mechanism 120 includes the motion control unit 1
By driving the front wheels 102, hands 106, and neck (head frame 316) according to instructions from 50, the robot 100 is made to express various motions.

A recognition unit 156 of the robot 100 interprets external information obtained from the internal sensor 128 . The recognition unit 156 includes visual recognition (visual unit), smell recognition (olfactory unit), and sound recognition (auditory unit).
), tactile recognition (tactile part) is possible.

The recognition unit 156 extracts feature vectors from the captured image of the moving object. As mentioned above,
A feature vector is a set of parameters (feature quantities) representing physical and behavioral features of a moving object. When a moving object is detected, physical characteristics and behavioral characteristics are also extracted from an odor sensor, a built-in sound collecting microphone, a temperature sensor, and the like. These features are also quantified into feature vector components. The recognition unit 156 identifies the user from the feature vector based on the known technique described in Patent Document 2 or the like.

Among a series of recognition processing including detection, analysis, and judgment, the recognition unit 156 of the robot 100 selects and extracts information necessary for recognition, and interpretation processing such as judgment is performed by the recognition unit 212 of the server 200.
Executed by The recognition process may be performed by the recognition unit 212 of the server 200 alone, or may be performed by the recognition unit 156 of the robot 100 alone. good too.

When a strong impact is applied to the robot 100, the recognition unit 156 recognizes this with the touch sensor and the acceleration sensor, and the recognition unit 212 of the server 200 recognizes that a nearby user has acted "violently." User grabs horn 112 and robot 10
Even when you lift 0, you may recognize it as a violent act. When the user facing the robot 100 speaks in a specific sound volume range and a specific frequency band, the recognition unit 212 of the server 200 may recognize that the user has made a “calling action” to himself/herself. Also, when a temperature of about body temperature is detected, it is recognized that the user has made a "contact action", and when an upward acceleration is detected while contact is being recognized, it is recognized that a "hold" has been made. Physical contact when the user lifts the body 104 may be sensed, or the holding may be recognized by reducing the load on the front wheels 102 .
In summary, the robot 100 acquires the user's behavior as physical information through the internal sensor 128, and the recognition unit 212 of the server 200 judges comfort/discomfort. Also, the recognition unit 212 of the server 200 executes user identification processing based on the feature vector.

The recognition unit 212 of the server 200 recognizes various responses of the user to the robot 100 . Pleasant or unpleasant, affirmative or negative are associated with a part of typical responses among various responses. In general, most of the reception behaviors that are pleasant behaviors are positive reactions, and most of the reception behaviors that are unpleasant behaviors are negative reactions. Pleasant/unpleasant actions are related to intimacy, and positive/negative reactions influence the action selection of the robot 100 .

Intimacy management unit 220 of server 200 according to the behavior recognized by recognition unit 156
changes intimacy with the user. In principle, the degree of intimacy with a user who has performed a pleasant act increases, and the degree of intimacy with a user who has performed an unpleasant act decreases.

Based on the basic configuration described above, the implementation of the robot 100 according to the present embodiment will now be described, particularly focusing on the features and purposes of the implementation and the differences from the basic configuration.

[Implementation of reaction report generation function]
FIG. 6 is a schematic diagram showing the reaction report generation process in this embodiment.
The robot 100 records events, emotional parameters, etc. as history information (S10).
. The history information records the feelings (emotional parameters) and occurrences (events) of the robot 100 in chronological order. The user requests acquisition of a reaction report from the user terminal 400 to the server 200 (S12). When server 200 receives the reaction report acquisition request, server 200 generates a reaction report from the history information (S14). Since the history information contains a large amount of information, if the history information is used as the reaction report as it is, the visibility of the reaction report is deteriorated. Also, from the viewpoint of privacy protection, it is not preferable to provide the user with the history information itself as a reaction report. In this embodiment, various types of information included in the history information are selected, and after changing the information expression method, a reaction report is generated as information to be shown (details will be described later). The user terminal 400 acquires the reaction report from the robot system 300 and displays it on the screen (S16).

FIG. 7 is a functional block diagram of the robot 100 in this embodiment.
The robot system 300 in this embodiment has a reaction report generation function. Therefore, the functional block of the robot system 300 shown in FIG. 5 and a plurality of functional blocks are added. In FIG. 7, the added functional blocks will be mainly described.

Note that the robot 100 may recognize the location based on keyframes instead of the external sensor 114 . A key frame is distribution information of feature points (feature amounts) in a captured image. The robot 100 is a graph-based SLAM (Simultaneous Local
mapping) technology, more specifically ORB (Oriented FAST and Rotated BRI
EF) Keyframes may be formed by SLAM technology based on feature values (details are in Japanese Patent Application No. 20
18-014365).

(server 200)
Data processing unit 202 further includes history management unit 252 , icon selection unit 254 , atmosphere determination unit 256 and mode selection unit 258 . In addition, the state management unit 244, the emotion management unit 25
including 0.
The emotion management unit 250 sets an emotion parameter indicating the intensity of loneliness (hereinafter referred to as “lonely parameter”) and an emotion parameter indicating the intensity of curiosity (hereinafter referred to as “curiosity parameter”).
and the emotional parameter that indicates the strength of the desire for approval (hereinafter referred to as the “need for approval parameter”).
Manages 3 types of emotional parameters.

The history management unit 252 manages history information of the robot 100. FIG. In the history information, the occurrence history of events and the change history of emotion parameters are associated with each other. For historical information,
This will be described later with reference to FIG. Also, the history management unit 252 generates a reaction report based on the history information.

Icon selection unit 254 selects an icon according to the event. Details will be described later, but
In the reaction report of this embodiment, an event is image-represented by an icon. The icon selection unit 254 selects an icon associated with the event. Icon selection part 2
54 selects a variety of icons besides events.

The atmosphere determination unit 256 determines the atmosphere of the external environment. A method for determining the atmosphere will be described later. Mode selection unit 258 selects the mode of robot 100 . The mode is setting information regarding the operating state of the robot 100, such as normal mode and power saving mode. The details of the mode selection method will also be described later.

Data storage unit 206 further includes event condition storage unit 260 and history information storage unit 262 . The event condition storage unit 260 defines event conditions. An event condition is, for example, a condition that defines the situation when an event occurs, such as "occurred in a child's room" or "occurred at the entrance". The history information storage unit 262 stores history information.

(Robot 100)
Internal sensor 128 of robot 100 includes imaging unit 158 . The imaging unit 158 corresponds to a camera such as an omnidirectional camera. In this embodiment, the imaging unit 158 constantly captures the peripheral image. The recognition unit 156 of the robot 100 analyzes external information (raw data) obtained from the internal sensor 128 such as the imaging unit 158 and transmits the analysis result (processed data) to the server 200 . The recognition unit 212 of the server 200 determines success or failure of various events based on the analysis results.
The determination of the success or failure of an event may be performed only by the robot 100 side or may be performed only by the server 200 side. The recognition unit 212 may make the determination.
In the following description, it is assumed that the recognition unit 212 of the server 200 determines the success or failure of the event.

The personal data storage unit 218 of the server 200 associates and registers the terminal ID for identifying the user terminal 400 and the user ID. Also, the user ID is associated with the user's physical characteristics and behavioral characteristics.

(User terminal 400)
User terminal 400 displays a reaction report, which will be described later, on the web browser. User terminal 400 may be installed with dedicated application software for displaying reaction reports and inputting instructions to robot 100 (detailed with reference to FIG. 19).

FIG. 8 is a schematic diagram showing change history of emotion parameters.
Emotion management unit 250 changes three types of emotion parameters over time. The horizontal axis in FIG. 8 represents time, and the vertical axis represents the magnitude of the emotion parameter. In this embodiment, the emotion parameter varies in the range of 0-100. In FIG. 8, the loneliness parameter 170 temporarily increases and then gradually decreases. The emotion management unit 250 gradually increases the loneliness parameter 170 as the period in which the robot 100 does not visually recognize the owner becomes longer, and decreases the loneliness parameter 170 when the owner is visually recognized. The event of seeing the owner contributes to the loneliness parameter 170 decrease. Similarly, the emotion manager 250 may also lower the loneliness parameter 170 when the event of hearing the owner's voice occurs.

The approval desire parameter 174 in FIG. 8 gradually increases over time. The emotion manager 250 increases the approval desire parameter 174 over time. Approval needs are one of the basic human needs to be recognized by others. Approval Desire Parameter 174
expresses the strength of the robot 100's desire for approval. The emotion management unit 250 detects pleasant behaviors (attentive behaviors toward the robot 100) such as the user talking to the robot 100 or hugging it.
is performed, the approval desire parameter 174 is lowered (satisfied), and when an unpleasant act is performed, the approval desire parameter 174 is increased (dissatisfied).

The emotion management unit 250 uses the curiosity parameter
Raise 172. Also, when no event occurs, emotion manager 250 gradually lowers curiosity parameter 172 .

The emotion management unit 250 determines whether or not the three types of emotion parameters satisfy various "mental conditions". The psychological condition may be arbitrarily set as a timing indicating a “change in mentality” of the robot 100 . For example, the approval desire parameter 174 is a predetermined threshold (e.g. 80)
When the above is reached, the first psychological condition may be established, and the approval desire parameter 174
The second psychological condition may be established when is dominant over the other two emotional parameters. In FIG. 8, the loneliness parameter 170 is greater than the curiosity parameter 172 and the approval desire parameter 174 during the period t0 to t1. In the period t1 to t2, the approval desire parameter 174 is the largest. Therefore, emotion management unit 250 determines that the second psychological condition is satisfied at time t1. In other words, at time t1, the robot 100
Approval needs (desire to be recognized) prevail over loneliness.

When the first emotional parameter of the three types of emotional parameters changes from being dominant to the second emotional parameter being dominant, in other words, when the superiority or inferiority of a plurality of emotional parameters is reversed, the robot 100 It seems that the mood has changed significantly.

It should be noted that not only the three types of loneliness, curiosity, and desire for approval, but four or more types of emotion parameters may be set. For example, various emotional parameters and events that affect those emotional parameters may be defined, such as joy, anger, anxiety, impatience, and shame. Emotion parameter is 1
can be of any kind. Emotional parameters change over time and with the occurrence of events.

Changes in emotion parameters may be random or periodic. A change in a first affective parameter may affect a second affective parameter. For example, when the loneliness parameter is 50 or higher, the emotion management unit 250 may increase the increase rate of the approval desire parameter.

Emotional parameters do not necessarily correspond to typical human psychological states like "curiosity". For example, "Emotion parameter Q1 is event(E1), event(E2
Emotion parameters having various characteristics may be defined by defining a change condition such as "increase with the occurrence of event (E3) and decrease with the occurrence of event (E3)". A "feeling" specific to the robot 100 may be defined that does not mimic human emotions.

FIG. 9 is a data structure diagram of the history information 270. As shown in FIG.
The history information 270 is stored in the history information storage unit 262 of the server 200. FIG. In the robot system 300, multiple types of events are defined in advance. Events are classified into one of several event types. For example, an event with event type=En (
Hereinafter, “event (En)”) may be visual recognition of the user. The event (Em) may be detection of a voice equal to or greater than a predetermined value, and another event (Ep) may be the robot 100 wearing clothes. Other events can be defined such as being hugged, the user going out, or finding something new in the room.

History management unit 252 records events and emotion parameters in history information 270 periodically, for example, every minute. The history information 270 is associated with an event occurrence history and an emotion parameter change history. Recognition unit 212 recognizes events and emotion management unit 25
0 changes the emotion parameter depending on the event. The motion control unit 150 or the like selects a motion according to the event and emotion parameters.

In the history information 270 of FIG. 9, an event (E1) occurs at 10:03. After the occurrence of event (E1), the loneliness parameter plummets from 72 to 14. event
It can be seen that (E1) is an important event that has eliminated the loneliness of the robot 100 .
In addition to this, the history information 270 includes an atmosphere type indicating the atmosphere of the external environment (details will be described later).
, the motion selected by the robot 100, the position information of the robot 100, the room temperature, and other various information may be registered.

FIG. 10 is a data structure diagram of the motion selection table 280. As shown in FIG.
Motion selection table 280 is stored in motion storage unit 232 . As mentioned above, motions are identified by motion IDs. When recognizing an event, the motion control unit 222 of the server 200 refers to the motion selection table 280 and selects a motion. The robot 100 executes the motion designated by the motion control unit 222. FIG. Motion selection table 280 is also downloaded to robot 100 from server 200 . For this reason, the motion control unit 150 of the robot 100 is set to the server for some events.
You can select the motion corresponding to the event without going through the 200.
In the following description, the motion is selected by the motion control unit 222 (server 200).
0).

In the motion selection table 280, one or more motions and their selection probabilities are associated with events. For example, when event (E1) is detected, motion control unit 222 selects motion (M1) with a selection probability of 5(%) and selects motion (M2) with a selection probability of 15(%). .

In this embodiment, the motion control unit 222 changes the motion selection probability according to the emotion parameter. For example, when the loneliness parameter among the emotion parameters is equal to or greater than a predetermined threshold, motion control section 222 increases the selection probability of motion (M1) for event (E1). When the curiosity parameter is equal to or greater than the predetermined threshold, the operation control unit 222
reduces the selection probability of motion (M1) for event (E1).

As a concrete example, when an event (En) occurs, selectable motion (M1), motion (M2), and motion (M3) selection probabilities are set to 30(%), 20(%), respectively.
Suppose it is 50(%). Below is the motion selected when the event (En) occurs
(M1) is written as "motion (En|M1)".

Motion (En|M1) increased from 30(%) to 40 when loneliness parameter is 80 or higher
(%). The increased 10(%) is adjusted by decreasing the selection probability of motion (En|M2) and motion (En|M3) by 5(%) each. As a result, when the loneliness parameter is 80 or more, the selection probability of motion (En|M2) is reduced from 20(%) to 15(%), and the selection probability of motion (En|M3) is 50(%).
) to 45(%). According to this control method, the motion (En|M1)
is likely to be selected when the robot 100 is "lonely", and "emotion" affects "action selection".

FIG. 11 is a screen diagram of the first reaction report screen 180. As shown in FIG.
The user requests the acquisition of the reaction report from the user terminal 400 to the server 200 . The user can freely switch the display method of the reaction report. A first reaction report screen 180 in FIG. 11 shows a reaction report that associates an event with the robot's feelings (emotional parameters).

When the history management unit 252 receives a reaction report acquisition request from the user terminal 400,
Generate a response report based on historical information 270 . The history management unit 252 extracts emotion parameters when the psychological conditions described above are satisfied.

First, at 9:00, it is assumed that one of a plurality of types of psychological conditions is satisfied. server 2
In 00, an emotion icon 184 is associated in advance with each psychological condition or each emotional parameter state. The icon selection unit 254 selects the emotion icon 184 according to the emotion parameter when the psychological condition is satisfied. In FIG. 11, emotion icon 184a is selected.

The icon selection unit 254 also selects the event icon 182 corresponding to the event when the psychological condition is satisfied. Each of the multiple events also has an event icon 18 in advance.
2 is associated. The ``event when the psychological condition is satisfied'' may be an event that occurred immediately before (most recently) the moment when the psychological condition was met, or an event that occurred within a predetermined time after the moment when the psychological condition was met. good. For example, when the psychological condition is established at time t1, the event that occurred during the seven seconds from time (t1-10 seconds) to time (t1-3 seconds) may be displayed in the reaction report.

In the first reaction report screen 180 of FIG. 11, the robot
An emotion icon 184a representing 100 "feelings" is displayed, and an event icon 182a corresponding to the event (event type) when the psychological condition is established is also displayed. Event icon 182a shown in FIG. 11 corresponds to an event in which robot 100 meets its owner. Emotion icon 184a corresponds to a happy feeling of robot 100, for example, a psychological state in which both the loneliness parameter and the desire for approval parameter are equal to or less than a predetermined value. By referring to the first reaction report screen 180, the user can refer to the "robot 100
I was happy to meet the owner at 9:00." You can see the daily life of the robot 100 and the feelings of the robot 100 at that time.

The user can know the feelings and events of the robot 100 from the emotion icon 184 and the event icon 182 on the first reaction report screen 180 . By symbolically expressing events and the like with icons, the user can enjoy the daily life of the robot 100 like a picture diary.

In this embodiment, the recognition unit 212 recognizes an event based on information of the external environment, and identifies an event type corresponding to the recognized event. Next, the icon selection unit 254
selects the event icon 182 corresponding to the event type. In the reaction report,
The details of the event are not shown, and the event icon 182 shows "What happened?"
is easily communicated, the first reaction report screen 180 prevents excessive disclosure of privacy.

The emotion parameter of the robot 100 may be displayed as a parameter value instead of the emotion icon 184, or the emotion of the robot 100 may be expressed by colors corresponding to the emotion. This is because the emotion parameters of the robot 100 themselves are less likely to affect the user's privacy.

The robot 100 selects actions while recognizing a large number of events. Therefore, showing all events in the reaction report is not preferable not only from the viewpoint of privacy protection but also from the viewpoint of visibility. Therefore, in this embodiment, only the event when the psychological condition is established is registered in the reaction report. According to such a control method, it becomes easier to confirm an event that may have triggered a psychological change of the robot 100 . In other words, it is possible to confirm the situation centering on the "event that moved the heart of the robot 100" among various events occurring around the robot 100. Among the various events recognized by the robot 100, events that are important to (the feelings of) the robot 100 are registered in the reaction report, so that a diary-like reaction report that emphasizes the feelings of the robot 100 can be formed.

At 9:30, an event of the owner going out (event icon 182b) and an event of the owner going home (event icon 182c) occur continuously in a short period of time. When a plurality of events occur in succession in a short period of time, the history management unit 252 causes the reaction report to be displayed with the event icon 182b and the event icon 182c slightly shifted left and right. With such a display method, the visibility of event icon 182 on first reaction report screen 180 can be improved.

The history management unit 252 further sets attention marks 186 next to events belonging to a predefined specific category (specific event type). The attention mark 186 is a highlighting of an event that should be noticed among a plurality of kinds of events. Attention marks 186 help the user to recognize important events. Important events include sighting of an owner, going out or seeing someone off, falling objects, and finding an unregistered person (suspicious person).

It is also conceivable that a plurality of robots 100 are introduced into one household. robot 100
have individuality due to differences in initial settings or differences in environment. For example, there may be a "loneliness vulnerable robot 100" whose loneliness parameter tends to increase, and a "calm robot 100" whose curiosity parameter does not increase easily.
Also, while the robot 100A has a high degree of intimacy with the user (P01), the robot
100B may have a lower affinity than that. In this case, being held by the user (P01) greatly relieves the "loneliness" of the robot 100A, but in the case of the robot 100B, being held by the user (P01) may hardly relieve the "lonelyness". do not have. Also, the robot 100C, which has never seen the user (P01), may rather increase the curiosity parameter when held by the user (P01). in this way,
Each robot 100
changes in the emotional parameters of Therefore, there is a high possibility that the timing at which the psychological condition is established, in other words, the "timing at which the mind is moved" differs for each robot 100. FIG.

Although the history information generated by each of the plurality of robots 100 having the same experience may be substantially the same, the timing at which the psychological condition of the robot 100A is established and the robot 100
Since the timing at which B's psychological condition is established is different, robot 100A and robot 100B
may generate different response reports. For example, robot 100A is an event
(E1) is recorded in the reaction report, but robot 100B may not record event (E1) in the reaction report. By referring to the reaction reports of robots 100A and 100B, the user can recognize that robot 100A was moved by event (E1), but robot 100B is not interested in event (E1). By comparing reaction reports from multiple robots 100, the user can enjoy the individuality and feeling of each of the multiple robots 100. FIG. In addition, by providing an opportunity to feel the individuality of the robot 100 through the reaction report, it is possible to enjoy raising (involvement with) a plurality of robots 100 even more.

FIG. 12 is a screen diagram of the second reaction report screen 190. As shown in FIG.
The reaction report can also display the motion selected by the robot 100 when the psychological condition is satisfied. A second reaction report screen 190 in FIG. 12 shows a reaction report in which events and motions are associated with each other.

A motion icon 192 is associated with the motion ID. Motion icon 192 may be formed as an animation that reproduces the motion of robot 100 . The history management unit 252 identifies the motion selected when the psychological condition is satisfied. Icon selection unit 254 selects motion icon 192 corresponding to the specified motion. The ``motion selected when the psychological condition is satisfied'' may be the first motion selected after the psychological condition is satisfied, or a motion selected during a predetermined period based on the time when the psychological condition is satisfied. There may be. For example, when the psychological condition is satisfied at time t1, time (t
1 + 0.5 seconds) to the time (t1 + 7.5 seconds) for 7 seconds may be registered in the reaction report.

In the second reaction report screen 190 of FIG. 12, the robot
A motion icon 192a representing 100 "actions" is displayed, and an event icon 182a corresponding to the event when the psychological condition is satisfied is also displayed. event icon
182a corresponds to the event that robot 100 encounters an owner. Motion icon 192a represents movement of the robot. By referring to the second reaction report screen 190, the user says, "The robot 100 met the owner at 9:00 and started moving."
The daily life of the robot 100 and the behavior of the robot 100 at that time can be confirmed.

The user operates user terminal 400 to display first reaction report screen 180 and second reaction report screen 180 .
The two reaction report screens 190 can be switched. The robot 100 selects a motion according to the event and emotional parameters at that time. In the robot system 300, the behavior of the robot 100 is determined by the complex influence of events, emotional parameters (feelings), and motions (actions). The behavior of the robot 100 is unpredictable. The user can view first reaction report screen 180 and second reaction report screen 190
By referring to , it is possible to grasp what kind of event occurred around the robot 100, how the robot 100 felt about it, and what action the robot 100 chose as a result of the event. From the first reaction report screen 180 in FIG. 11 and the second reaction report screen 190 in FIG. 12, it can be inferred that the robot 100 was happy to meet the owner and approached the owner.

FIG. 13 is a data structure diagram of the user authority table 290. As shown in FIG.
User authority table 290 is stored in personal data storage unit 218 of server 200 .
In this embodiment, the viewing attribute is set for each user in order to control the amount of information provided by the reaction report according to the user. The viewing attribute in this embodiment indicates how much information the user can extract from the history information, in other words, the viewing authority level. A user is identified by a user ID. A user's physical characteristics and a user ID are associated in advance, and a browsing attribute is set in advance for each user ID. The browsing attribute A1 has the strongest browsing authority, and the browsing attribute A4 has the weakest browsing authority. User (P
01:A1) will be more informative than the reaction report provided to the user (P04:A4).

FIG. 14 is a data structure diagram of the event condition table 292. As shown in FIG.
The event condition table 292 is stored in the event condition storage unit 260 of the server 200. FIG. The event condition table 292 defines viewing conditions for each event condition. Event conditions are conditions defined primarily for determining privacy-related events. Conditions related to the appearance and voice of the user, such as angry voices and cries; conditions related to places such as children's rooms and studies; conditions related to time such as late at night and evening; An event condition is defined as a condition. The event condition regarding location may be classified into events occurring in open spaces such as living rooms, kitchens, and entrances, and events occurring in private spaces such as children's rooms, studies, and bedrooms.

For example, when an event X1 satisfies the event condition (C1), browse attribute A1
Event X1 is registered only in reaction reports for the above users. For this reason,
User (P01:A1) can confirm event X1 in the reaction report, but user (P02
:A2) cannot see event X1 in the reaction report. For example event X1
When (the event of visually recognizing the user) satisfies the event condition (C1) that the user (P06) in his room (study, etc.) is visually recognizable, the user (P01:A1) becomes the user (P0
6) can be confirmed in the reaction report that he was in his room, but the user (P02:A2) cannot know it. Privacy protection is further strengthened by controlling events that can be notified and events that must not be notified for each user according to the browsing attribute (browsing authority level) of the user.

FIG. 15 is a sequence diagram showing the reaction report display processing process.
A reaction report acquisition request is transmitted from the user terminal 400 to the server 200 (S20
). At this time, the user terminal 400 also transmits a terminal ID. History management unit 252 identifies the user ID corresponding to the terminal ID. The history management unit 252 uses the user authority table 29
0 to check the browsing attribute of the user (S22). Based on viewing attributes, the events that can be displayed in the reaction report are narrowed down. Next, the history management unit 252 refers to the history information and checks when the psychological condition is satisfied (S24). The history management unit 252 generates a reaction report from events, emotion parameters, motions, etc. when psychological conditions are established (
S26). Icon selection unit 254 selects an icon as appropriate. A response report is formed as a web page containing icons. The communication unit 204 transmits the reaction report shown in FIGS. 11, 12, etc. to the user terminal 400 (S28). The user terminal 400 displays the reaction report on the screen (S30).

<Display detailed information>
FIG. 16 is a sequence diagram showing the detailed display process.
The history management unit 252 acquires detailed information when an event occurs. The detailed information may be any information indicating the content and situation of the event, such as captured images (still images), short movies, texts, audio information, and temperature distribution information obtained by a thermosensor. At least, the detailed information should provide information other than the icon. For example, the recognition unit 21 of the server 200
2 saves the captured image Y2 at that time when the event X2 of detecting the user (P01) is recognized. The robot 100 transmits captured images to the server 200 at any time. The history management unit 252 associates the event X2 with the captured image Y2 and saves them. This captured image Y2
is "detailed information" about event X2. A short movie of about 5 seconds, for example, may be saved as a detailed image for a predetermined period including the time when the user (P01) is detected.
Alternatively, the recognition unit 212 may generate text information "I found the user (P01) at the entrance". Then, the history management unit 252 may save the text information as detailed information.

When generating a reaction report, the history management unit 252 associates the event with the icon ID of the event icon 182 added to the reaction report and saves them. Icon ID is a number that uniquely identifies the event icon 182 . When the user taps one of the event icons 182 on the user terminal 400, the user terminal 400 transmits the icon ID corresponding to the event icon 182 to the server 200 (S40). History management part 2
52 identifies an event associated with the icon ID and identifies detailed information associated with this event (S42).

Emotion management unit 250 transmits detailed information to user terminal 400 via communication unit 204 (
S44). The user terminal 400 displays detailed information (S46).

FIG. 17 is a screen diagram of the detailed information screen 402. As shown in FIG.
User terminal 400 receives detailed information from server 200 when event icon 182 is tapped. The user terminal 400 displays the detail window above the reaction report at this time.
Display a 404. Detailed information is displayed in the detail window 404 . The detailed information may be still images, moving images, or text information. Also, a still image and text information may be displayed at the same time.

Viewing authority may be set for detailed information as well. For example, for event X3 that satisfies the event condition (C2), the event icon 182 is displayed in the reaction report for users with browsing attribute A2 or higher, but viewing attribute A1 or higher is required to view detailed information. There may be. Alternatively, viewing attribute A2 or higher is required to view event X4 that satisfies the event condition (C4) in the reaction report.
For 4, display of detailed information may be prohibited. Because detailed information is raw data that specifically shows how an event occurred, it may involve privacy. Therefore, whether or not to display detailed information should be managed more strictly than the event icon 182 .

The detailed information may be a moving image without sound, or a partially blurred still image. Further, the history management unit 252 may change the texture of the still image by a known image conversion method and provide detailed information to the user terminal 400 as a painterly image or a blurred image. The history management unit 252 may provide detailed information close to raw data to a user with strong browsing authority, and provide detailed information with a higher degree of processing for users with weaker browsing authority.

<Atmosphere display>
FIG. 18 is a screen diagram of the third reaction report screen 410. As shown in FIG.
In addition to events, emotions, and motions, the history manager 252 can record atmosphere types (ambience information) in reaction reports. The atmosphere determination unit 256 classifies the external environment of the robot 100 into a plurality of atmosphere types. Specifically, they are "pleasant", "lively", "quiet", "dark", and "tense". The icon selection unit 254 selects the atmosphere icon 194 according to the atmosphere type. The history management unit 252 records the atmosphere icon 194 (image) corresponding to the atmosphere type when the psychological condition is established in the reaction report.

The atmosphere determination unit 256 determines the atmosphere type based on external information obtained from the robot 100 . The robot 100 acquires information on images, sounds, smells, and contact by the internal sensor 128 and transmits these to the server 200 . The atmosphere determination unit 256 determines the atmosphere type based on the atmosphere conditions. Specifically, the success or failure of the atmosphere condition is determined based on the sound pressure, whether or not a smile can be detected, laughter, the number of touches, the number of users present around the robot 100, and the like. For example, when a plurality of voices and a plurality of smiles are detected in a predetermined period, the atmosphere condition regarding "lively" is established, and the atmosphere determination unit 256 determines that the atmosphere type is "lively." Further, when an angry voice is detected in a predetermined period, the atmosphere determination unit 256 determines the atmosphere type as "tingly".

In the third reaction report screen 410 of FIG. 18, an atmosphere icon 194 representing the "atmosphere type" when the psychological condition is satisfied is displayed, and an emotion icon 184 corresponding to the emotion of the robot 100 at this time is also displayed. . By referring to third reaction report screen 410, the user can see the atmosphere around robot 100 and robot 100 at that time.
can confirm the feelings of For example, the user may ask whether the robot spent time in a busy environment,
The third reaction report screen 410 lets you know whether you spent a quiet day. A viewing attribute may be set for the atmosphere type as well as for the event.

In this embodiment, the server 200 is described as having the atmosphere determining unit 256 installed therein, but the robot 100 may have the atmosphere determining function by the atmosphere determining unit 256. FIG.
In this case, atmosphere determination unit 256 of robot 100 notifies server 200 of the atmosphere type (ambience information), and icon selection unit 254 selects atmosphere icon 1 corresponding to the atmosphere type.
94 is selected, and the history management unit 252 generates a reaction report to which the atmosphere icon 194 is attached.

<Rest time setting>
FIG. 19 is a screen diagram of a rest time setting screen 420. As shown in FIG.
The user not only confirms the response report on the user terminal 400, but also
Instructions can also be issued from 400 to the robot system 300 . In this embodiment, the user can set the rest time (sleep time) of the robot 100 on the user terminal 400 . On the rest time setting screen 420 of FIG. 19, 23:00 to 5:00 is set as the rest time. The user terminal 400 transmits rest time designation information to the server 200 . Operation control unit 222 of server 200 causes robot 100 to wait at a charging station (not shown) during the rest period.

Rest time can be set for 1 hour or longer in 10 minute increments. The user is a robot 100
Sometimes you don't want to move around in the middle of the night. Moreover, it is considered that the user does not want the robot 100 to move around even when the user is in poor physical condition. By setting the rest time using the user terminal 400, it is possible to prevent the robot 100 from disturbing the user's life.

<Summary>
The robot system 300 has been described above based on the embodiment.
According to this embodiment, even when the user is not near the robot 100, the robot 10
0's daily life can be confirmed by the user terminal 400. FIG. The user can know what kind of experience the robot 100 had, what kind of feeling, and what kind of action it took by a reaction report like a picture diary.

When the psychological condition is satisfied, in other words, when the robot 100 changes its mind, the reaction report is generated by focusing on the event. You can create a reaction report like a diary that emphasizes feelings (sensitivity). Narrowing down the information to be provided as a reaction report not only enhances the visibility of the reaction report, but also helps protect the privacy of users near the robot 100 .

By referring to the reaction report, the user can know how the robot 100 was feeling and how he or she is feeling. For example, the user may want to go home early to see robot 100 when robot 100 is having a lonely day. In addition, even when the father is out and the mother and children are at home with the robot 100, the father can grasp the outline of the activity of the robot 100 from the reaction report, so it is easy to talk about life with the robot 100 even after returning home. It is considered to be. In addition, the user can know the state of the grandfather and grandmother who live far away through the reaction report (details will be described later).

The reaction report does not directly display an event or the like, but expresses the daily life of the robot 100 in a caricature manner using various icons. With beautiful and cute icons, the user can enjoy the reaction report like a picture diary. Icon display not only makes the reaction report easier to understand, but also enhances the appearance of the reaction report and contributes to the protection of privacy.

In addition, viewing attributes can be used to finely control the information that can be provided according to the user.
For example, parents may be able to see many of the events in the home, but grandparents may only be able to see some of the events in the home. The history management unit 252 may reject a reaction report acquisition request from an unregistered user terminal 400. FIG.

By checking the detailed information, the user can check events around the robot 100 from outside without being near the robot 100 . For example, when the robot 100 has a "happy event", by displaying the detailed information of the happy event, even a user on the go can easily share the "happy event" with a user near the robot 100.例文帳に追加Currently, it is common practice to send still images and moving images to close friends via SNS (Social Networking Service). can be communicated to

Also, in the reaction report, the atmosphere around the robot 100 can be represented by the atmosphere icon 194. FIG. A user who is out can know the atmosphere at home through the reaction report. A user on the go may be tempted to go home if the home atmosphere seems pleasant. The father at work can work with peace of mind by checking the reaction report on the smartphone (user terminal 400), even if he does not know the details of the home, if he can know the "fun" atmosphere of the home. Conceivable. Alternatively, the father may ask the family what they had fun with when they got home. Since it is possible to share "fun" information even when you are not at home, it is thought to be a tool that loosely connects fathers and families.

A user can set a rest time for the robot 100 using the user terminal 400 . By setting the rest time, for example, it is possible to prevent the user's sleep from being disturbed by the robot 100 moving around early in the morning. Alternatively, since the user has to wake up early tomorrow, the user may want the robot 100 to move early in the morning. Although it is simple to set the rest time of the robot 100 from the user terminal 400, the user's life pattern and the robot 1
In terms of coordinating 00's life patterns, in other words, it is considered to be a very effective means for realizing symbiosis between the user and the robot 100.

Conventionally, robots that select actions based on emotional parameters are known. In general, it is thought that setting emotion parameters in a robot is to realize a biological action expression as if the robot were moving according to its own emotions. It is conceivable that emotion parameters are often treated as internal parameters invisible to the user. In contrast, in this embodiment, the user can know the emotion of the robot 100 from the reaction report. By knowing how the robot 100 feels in its daily life, the user is expected to consider the feelings of the robot 100 . By considering each other between the robot 100 and the user, the psychological interaction between the robot 100 and the user can be further deepened.

It should be noted that the present invention is not limited to the above-described embodiments and modifications, and can be embodied by modifying constituent elements without departing from the scope of the invention. Various inventions may be formed by appropriately combining a plurality of constituent elements disclosed in the above embodiments and modifications. Also, some components may be deleted from all the components shown in the above embodiments and modifications.

Although the robot system 300 has been described as being composed of one robot 100, one server 200, and a plurality of external sensors 114, part of the functions of the robot 100 are implemented by the server 200.
00, and some or all of the functions of the server 200 may be implemented by the robot 100.
may be assigned to One server 200 may control multiple robots 100 , or multiple servers 200 may cooperate to control one or more robots 100 .

A third device other than the robot 100 and the server 200 may take part of the functions. Figure 7
In general, the aggregate of each function of the robot 100 and each function of the server 200 described in
It is also possible to grasp it as one "robot". How to distribute a plurality of functions necessary for realizing the present invention to one or more pieces of hardware is determined in consideration of the processing capability of each piece of hardware and the specifications required for the robot system 300. It should be decided.

As described above, the “robot in the narrow sense” is the robot 10 that does not include the server 200.
0, but the “robot in a broad sense” is the robot system 300.
It is conceivable that most of the functions of the server 200 will be integrated into the robot 100 in the future.

[Variation]
<reject response report>
The history management unit 252 may reject the reaction report acquisition request from the user terminal 400 existing near the robot 100 . For example, the history management unit 252 may reject the reaction report acquisition request from the user terminal 400 owned by the user (P01) when the imaging unit 158 of the robot 100 is visually recognizing the user (P01). Specifically, the recognition unit 212
recognizes the user (P01) from the captured image, the history management unit 252 recognizes the user (P01)
, and rejects the response report acquisition request from the user terminal 400 having this terminal ID. The reason why such control is executed is to prevent the user (P01) from checking the change of the reaction report on the user terminal 400 while interacting with the robot 100. FIG. If the user (P01) could perform an action such as hitting or hugging the robot 100 and could confirm the change in the emotion of the robot 100 at that time in real time, the robot 100 would be treated as an experimental animal. turn into. In order to prevent the robot 100 from being treated like an experimental animal, the history management unit 252 allows users who are not within the visible range of the robot 100, in other words, users who are far enough away from the robot 100 to directly affect it. A response report may be provided only for

The user (P01) who is the owner of the user terminal 400 (P01) uses the user terminal 400 (
It is also conceivable to obtain the reaction report by another user terminal 400 (P02) instead of P01). In this case, if the user (P02) who is the true owner of the user terminal 400 (P02) is not nearby, the robot 100 will transmit the reaction report to the user terminal 400 (P02), and the user who is nearby will There is a possibility that the reaction report will be seen in (P01).

As a countermeasure against such loopholes, first, the application software built into the user terminal 400 (P02) uses the camera function to obtain the reaction report from the user (
P01) is captured, and the captured image Q1 at that time is sent to the server 20 in response to the response report acquisition request.
0 should be sent. In the case of the above example, this captured image Q1 includes the face image of the user (P01).

When the server 200 receives the reaction report acquisition request, the history management unit 252
00 also acquires the captured image Q2. In the above example, the user (P0
The face image of 1) is included. The recognition unit 212 determines whether the face image of the same person as that included in the captured image Q1 (user terminal 400) is also included in the captured image Q2 (robot 100). When the same person is photographed, it means that a reaction report has been requested by a user near the robot 100, so the history management unit 252 sends the user terminal 400
02) refuses to provide response reports. Provided that the same person is not photographed,
History management unit 252 may provide a reaction report. According to such a control method, the response report acquisition request from the user terminal 400 existing near the robot 100 can be more reliably rejected.

In addition, the server 200 may be equipped with a distance measuring unit (not shown) and measure the distance to the user terminal 400 by a known method such as infrared communication. The history management unit 252 may reject reaction report acquisition requests from user terminals 400 within a predetermined distance. Alternatively, the server 200 stores the position coordinates of the robot 100 and the position coordinates of the user terminal 400 using GPS (Gl
It may be detected by known means such as obal positioning system). The response report acquisition request from the user terminal 400 may be responded to on the condition that the robot 100 is away from the user terminal 400 by a predetermined distance or more.

When the robot 100 receives a reaction report acquisition request from the user terminal 400 of the user (P01) while the robot 100 is visually recognizing the user (P01), the robot 100 receives the user (P01).
01) for a predetermined period based on the time of first viewing, for example, from the time of first viewing
You may provide response reports for only one minute. With such a control method, while the user (P01) is interacting with the robot 100, it is possible to limit the chances of confirming the emotional change of the robot 100 depending on how the robot 100 interacts with the reaction report.

Conversely, the reaction report may be shown only to users at home or near the robot 100 . According to such a control method, it is possible to prevent the reaction report from being viewed by a user at a remote location. Users at home can use robot 100 or server 2
00 may be manipulated to reject remote response report acquisition requests.

When receiving the reaction report acquisition request from the user (P05), the authority confirmation unit (not shown) of the server 200 gives the administrative user (P07), whose viewing permission authority has been set in advance,
You may check whether the response report can be provided. The authority confirmation unit sends information (user ID, name, affiliation, etc.) about the user (P05) to the user terminal 400 of the administrative user (P07).
to send. The history management unit 252 may transmit the reaction report to the user (P05) on the condition that the authority confirmation unit has received an authorization instruction from the administrative user (P07).

Not only whether or not a reaction report can be provided, but also the period covered by the reaction report may be limited. For example, assume that the user (P01) has sent a reaction report acquisition request to the server 200. FIG. At this time, the history management unit 252 may provide the user (P01) with a reaction report whose target period is a time zone (absence period) during which the user (P01) was not near the robot 100. FIG. For example, assume that the user (P01) is away from the robot 100 from time t6 to time t7, and is near the robot 100 after time t8. When the user (P01) transmits a reaction report acquisition request from the user terminal 400 to the server 200 after time t8, the history management unit 252 sends the reaction report to the user (P01) for the absence period from time t7 to time t8. ). On the other hand, no reaction report is provided for the target period after time t8. According to such a control method, even a user who has returned home from outside can look back on the state of the home when he/she is not there, and can get a glimpse of the life of the robot 100 when he/she is not there. , you cannot see the reaction report when you are near the robot 100. When the user is far enough away from the robot 100, the history manager 252 may provide the user with a reaction report including the current time, that is, a real-time reaction report.

The history management unit 252 may exclude the period during which the robot 100 is resting (charging) from the target of the reaction report. For example, when robot 100 is resting from time t9 to time t10, history management unit 252 may not generate a reaction report for the time period from time t9 to time t10. This is because not only is it irrational for the robot 100 to report on its life while it is resting, but it is considered that meaningful information cannot be provided by a reaction report because psychological conditions are not established during rest. Alternatively, even if the robot 100 is resting, the server 200 may be able to provide the user with a reaction report before resting.

The history management unit 252 may not display the emotion of the robot 100 in real time in the reaction report. For example, when the user requests a reaction report, information may be provided in the reaction report limited to emotions from a predetermined time or more ago, for example, 10 minutes or more before the current time.

<record movement history>
The history management unit 252 is not limited to the time and event, but the place where the event occurred or the robot
100 movement histories may be recorded in history information and reaction reports. In addition, the location management part 2
08 may form a map based on the captured image obtained by the imaging unit 158 when the robot 100 moves. The history management unit 252 may provide the map created by the robot 100 as detailed information.

<record mode>
Mode selection unit 258 selects an operation mode of robot 100 . Mode selector 258
, when the amount of activity per unit time of the robot 100 exceeds a predetermined value, the robot
Change the operating mode of 100 from normal mode to power saving mode. The amount of activity may be determined according to the power consumption per unit time or the number of motion selections. In the power saving mode, the types of motions selectable by the robot 100 are narrowed down. Alternatively, the motion of the robot 100 may be slowed down by limiting the movable range of the actuator and lengthening the interval. In the power saving mode, the amount of activity per unit time of the robot 100 is suppressed. 258 changes the operation mode to the normal mode when the power saving mode continues for a predetermined time or longer.

The mode selection unit 258 changes the operation mode to the charge mode (rest mode) when the charging rate of the secondary battery mounted on the robot 100 becomes equal to or less than a predetermined threshold. At this time, the motion control unit 222 places the robot 100 at the charging station. When the charging rate reaches or exceeds the threshold, mode selection unit 258 changes the operation mode from charging mode to normal mode.

The robot system 300 may also have a shootable mode and a shootable mode. A rotary switch (not shown) may be mounted on the horn 112 of the robot 100 . By rotating the switch provided on the horn 112, the user can manually switch between the shooting enabled mode and the shooting disabled mode. In shooting enabled mode, the imaging unit 1
The external world can be imaged by 58 and the captured image can be saved as part of the history information. On the other hand, in the non-imaging mode, the robot 100 does not store the captured image although it performs the imaging itself for user recognition and the like. Alternatively, the robot 100 keeps the captured image for a predetermined time, for example, 3
It may be automatically discarded after seconds. When the user does not want the robot 100 to take pictures, the user can simply turn the switch provided on the horn 112 to switch to the no-picture mode.

When the switch provided on horn 112 is operated, communication unit 142 transmits the setting value to server 200 . The mode selection unit 258 recognizes, based on the setting value received from the robot 100, whether the mode is the photographable mode or the photographable mode. Even in the non-imaging mode, the recognition unit 212 of the server 200 receives a captured image from the robot 100 and executes image recognition processing, but does not permanently store the captured image.

The history manager 252 may record the mode of the robot 100 in history information and reaction reports. The user can check the change history of the operation mode of the robot 100 by referring to the reaction report. It should be noted that, during the period of the non-imaging mode, no captured image remains, so detailed information as an image cannot be provided.

<Record of regular behavior>
The recognition unit 212 may detect regular behavior of the user. Regular actions include, for example, early morning exercise, greetings such as "good morning" and "good night" to the robot 100, brushing teeth, washing face, bathing, sitting in the study, cleaning, eating, taking a nap, etc. It is the user action that takes place. The regular behavior may be registered in advance in the server 200, or the recognition unit 212 may observe the user for a predetermined period, and when the same behavior is repeatedly detected in the same time period, the behavior may be registered as the regular behavior. . Regular behavior is recognized on a per-user basis. Recognition part 2
12 recognizes that a "regular event" has occurred when a regular action is detected.

The history management unit 252 records in history information when a regular event occurs. History management section
252 may record an event icon 182 corresponding to the regular event in the response report. Alternatively, instead of recording the regular event in the reaction report, when the regular event is not detected within the expected time period, that effect may be recorded in the reaction report. According to such a control method, user C at a remote location can confirm whether or not user D near robot 100 is performing a regular action through the reaction report.

Currently, the number of single-person households in Japan is said to exceed 13 million, and it is certain that the number will continue to increase in the future. In particular, the number of elderly (age 65 and over) single-person households has reached 6.2 million households, and patrols (safety checks) by the elderly have become an important issue. Currently, children and grandchildren living in the neighborhood sometimes visit their parents who live alone on a regular basis, and it is necessary to check the situation of their parents who live alone by regularly contacting them by phone, e-mail, SNS, etc. There is also

However, such safety confirmation may be a burden not only for the confirming side but also for the confirmed side. Children are concerned about the lives of their elderly parents who live alone in remote areas, but it is a heavy burden to check on social media every day. Considering that the elderly parents are busy with their children, they often refrain from contacting them.

If a single elderly person (old parent) lives with a robot 100, children will be robots 1
It is thought that it will be easier to confirm the safety of elderly parents through 00. However, from the viewpoint of protecting the privacy of the elderly parents, the robot 100 can interact with the elderly parents' daily lives and other users (children, etc.).
should not be reported to If the privacy of the person being watched over is not taken into consideration, "watching over" is no different from "surveillance."

The history management unit 252 in this embodiment generates a reaction report that considers the privacy of the user near the robot 100, but the child does not want to know the details of the daily life of the elderly parent. Elderly parents also do not want their children to see everything in their daily lives. It may be enough for children to see that their elderly parents are living safely as usual.

As one solution to this problem, it is conceivable to set weak viewing authority for viewers (children) of reaction reports. According to such a control method, the viewer (child)
It is easier to protect the privacy of elderly parents by limiting the information provided to them.

Another solution is to record the success or failure of regular events in reaction reports, which can be viewed by viewers.
provide to (children) For example, an elderly parent's voice saying "good morning" to the robot 100 is a regular event. The viewer can check whether the elderly parents living with the robot 100 performed the regular behavior (morning greetings) in the reaction report every day. In this case, the response report may tell the viewer only the success or failure of the regular event. By simply saying "Good morning" to Robot 100, elderly parents can send a message to children living far away that they are living the same way today. Children can also easily confirm that there is nothing wrong with their elderly parents by knowing whether or not there is a regular event through the user terminal 400 . For the person whose safety is confirmed, the children can be relieved just by living as usual, so it is not a burden.

Viewers (children) only need to confirm the safety of their elderly parents by existing communication means such as telephone only when they cannot confirm regular events in the reaction report.
It is not necessary to notify the success or failure of the regular event by a reaction report, and when the recognition unit 212 detects the regular event, the user who watches that the regular event (behavior as usual) has occurred by a known means of communication, for example, SNS. may be notified to

Moreover, it is not necessary to set the viewing restrictions as described above for regular events. For example, even if the user greets the resting robot 100 with "good night", the robot 100 may provide a response report indicating that a regular event has occurred.

<Manage history information>
In the present embodiment, the history management unit 252 has been described as recording all events in history information. History management unit 252 may limit the storage period of history information, since history information may contain a lot of information about privacy. For example, only history information for one day may be saved, and history management unit 252 may completely delete history information older than one day from history information storage unit 262 .

<Prohibited attribute>
A prohibited attribute may be set for some event conditions. Events with prohibited attributes may not be recorded in history information, may not be recorded in reaction reports,
It may not be provided as detailed information. A plurality of types of prohibited attributes may be provided according to the level of prohibition. For example, an event that satisfies the event condition of the prohibited attribute B1 may not even be recorded in the history information. An event that satisfies the event condition of the prohibition condition B2 is recorded in the history information, but may not be provided as detailed information. Although the event of the prohibition condition B3 is recorded in the history information, it may be erased in a short period of time such as 30 minutes.

Event conditions for which prohibited attributes are set include, for example, crying, yelling (recognition of marital quarrel), laughing face image, sleeping face image, and the like.

<browsing attribute>
In the present embodiment, it has been described that the viewing authority level is represented by the viewing attribute. Browsing attributes are not limited to rights related to privacy. For example, whether or not an event can be registered in a reaction report may be determined based on the external environment, such as an event that can be confirmed in the reaction report only when it occurs in the morning, or an event that can be confirmed in the reaction report only when it occurs on a rainy day.

In the present embodiment, it has been described that the browsing attribute is set for the user and the privacy protection level is controlled by the browsing attribute. In addition to this, the remote user terminal 40
The viewing authority level of the event may be lowered for response report acquisition requests starting from 0. For example, a detailed response report may not be provided to a user far away, or detailed information may not be provided to a user far away. The privacy level may be controlled based on the distance in this way.

Also, the privacy level may be controlled based on time. For example, a detailed reaction report is provided to a user who sends a reaction report acquisition request in the daytime, but the information contained in the reaction report is modest for a user who sends a reaction report acquisition request in the middle of the night. good too.

When the robot 100 is alone, in other words, when the robot 100 is at home with no one around, the history management unit 252 may provide a reaction report of all events. When the robot 100 is in the same room as other users, the information that the reaction report can provide may be suppressed. According to such a control method, while protecting the privacy of the user near the robot 100, when the robot 100 is alone at home, the detailed situation can be conveyed in the reaction report.

When the robot 100 discovers a suspicious person, in other words, a person who has never been seen, the history management unit 252 cancels the privacy protection and sends all the history information to the user terminal 40.
0 may be provided. According to such a control method, the robot 100 can function as a monitoring device only when a suspicious person intrudes into the house. When the robot 100 recognizes a recognized user (owner) and an unregistered person at the same time, it is highly likely that the unregistered person is a visitor. It is also possible to suppress the information to be used.

<emphasis>
In this embodiment, the history management unit 252 displays the attention mark
It was explained as giving 186. In addition, the history management unit 252 may emphasize the event icon 182 of the event of interest by blinking, changing the color, changing the size, or the like. Any display, such as a balloon, that calls attention to an important event may be used.

In the first reaction report screen 180 and the like, multiple types of lines such as a red line (first line) and a blue line (second line) may be displayed. Then, the red line may indicate the time of occurrence of a noteworthy event like the attention mark 186, and the blue line may indicate the time of occurrence of all events. The blue line may display the time points of occurrence of all events registered in the history information, regardless of the event when the psychological condition is satisfied.

<Icon display>
In the present embodiment, the aspect of changing the amount of detailed information according to the browsing attribute has been described. Similarly, the display of icons such as the event icon 182 may be changed according to the viewing attribute. For example, for the event "a customer has come", an event icon 182A using the customer's face photo, an event icon 182B using the customer's portrait, and a face representing the customer without specifying the customer. An event icon 182C using a picture of a visitor, and an event icon 182D such as an exclamation mark that does not even identify a visitor are prepared.

The history management unit 252 uses the event icon 182A for the reaction report for users with the viewing attribute A1 or higher, the event icon 182B for users with the viewing attribute A2, the event icon 182C for the users with the viewing attribute A3, and the viewing Event icon 182D may be used for users with attribute A4. According to such a control method, it becomes easier to protect privacy by controlling the amount of information provided by various icons according to the browsing attribute of the user.

<Command to robot 100>
In this embodiment, the method of setting the rest time for the robot 100 from the user terminal 400 has been described. Various commands may be sent from the user terminal 400 to the robot system 300, not limited to the rest time. For example, even if the user terminal 400 sends a command to the server 200 such as "I want you to stay in the living room from 8:00 to 9:00,""Please do not go to my child's room,""I want you to come to my bedroom at 7:00," etc. good. The motion control unit 222 of the server 200 controls the motion of the robot 100 according to commands. In addition to this, make sure to see the child once in the morning and once in the afternoon, and pay attention to the noise at the entrance when the father comes home between 22:00 and 23:30 (enhance the microphone sensitivity). , etc. can be considered.

However, if the user terminal 400 controls the robot 100 too much, the robot 1
Since the autonomy of 00 is lost, it is desirable that the group of commands that can be issued from the user terminal 400 to the robot 100 is limited to some extent. The operation control unit 222 may randomly select whether or not to follow the command from the user terminal 400 . The operation control unit 222 may follow an instruction from a user with a higher degree of familiarity with a higher probability.

The user may be able to talk to the robot 100 from the user terminal 400 even when he/she is away from home. The user reads aloud the message to the robot 100 from the user terminal 400 .
The server 200 receives the voice message from the user terminal 400 and reproduces it. The robot 100 recognizes the voice message by means of the microphone array (internal sensor 128) and recognizes that an event "a user speaks to me" has occurred. Emotion Management Department 25
0 may cause this audio event to reduce the loneliness parameter. According to such a control method, the user can soothe the loneliness of the robot 100 by speaking to the robot 100 from outside via the user terminal 400 . Even when leaving the robot 100 at home for a long trip, the user can show consideration by speaking to the robot 100 from the user terminal 400 .

<Others>
Instead of the emotion icon 184 corresponding to the emotion parameter, the value of the emotion parameter itself may be displayed in the reaction report. The history management unit 252 may record the value of the emotion parameter or the color corresponding to the value of the emotion parameter in the reaction report. History management part 2
52 may record the change history of the emotional parameter in the reaction report regardless of the success or failure of the psychological condition.

An event may also be represented by an event icon 182, may be displayed as text information, or may be pasted on a reaction report with a photograph of when the event occurred as described in the detailed information.

In this embodiment, the reaction report is transmitted from the server 200 to the user terminal 400,
It has been described that the user terminal 400 displays the reaction report on the screen. A printer may be mounted on or connected to the robot 100 or the server 200, and the reaction report may be printed out from the printer. Alternatively, a monitor may be mounted on or connected to the robot 100 or robot system 300 and the reaction report may be displayed on the monitor.

In the present embodiment, it has been described that events, motions, atmospheres, etc., when psychological conditions are established are displayed in reaction reports. As a modification, regardless of the success or failure of the psychological condition, the history management unit 252 can store events related to predetermined event conditions, important events, regular events, or all events corresponding to the browsing attribute of the user, in other words, the psychological condition All events corresponding to predetermined registration conditions other than the above may be registered in the reaction report. The history management unit 252 registers motions with a low selection probability (rarely performed actions) and events whose detection frequency is less than a predetermined number in a predetermined period (rarely occurring events) in the reaction report regardless of psychological conditions. good.

The history management unit 252 may select one representative event per hour, for example, and register it in the reaction report. An importance level may be set for each event type. The history management unit 252 may select, as a representative event, an event corresponding to the most important event type among one or more events occurring in a unit period. Alternatively, history management section 252 may randomly select a representative event from one or more events occurring in a unit period.

Various icons may be color-coded according to categories. For example, an event related to the user has a blue event icon 182, and an event related to discovery has a red event icon
182, an event related to sound or light may be represented by a yellow event icon 182. It may be color coded according to the event type. Alternatively, the event icon 182 may be color-coded according to whether or not a predetermined event condition is met.

In this embodiment, the feelings of the robot 100, the events, the actions of the robot 100,
It has been explained that the atmosphere and the like are registered in the reaction report. Not limited to this, for example, the reaction report may display only the feelings (emotional parameters) of the robot 100 in the reaction report. Even with such a reaction report, the user can guess how the robot 100 spent the day. When only the emotion of the robot 100 is registered in the reaction report, the user's privacy can be almost completely protected. For the same reason, the reaction report may display only atmosphere types or only events.

In the present embodiment, it has been described that the atmosphere is represented by the atmosphere icon 194 . The atmosphere may be expressed by text information such as "fun" and "lively", or may be expressed by colors. Alternatively, changes in the atmosphere may be represented by numerical values or graphs indicating the goodness of the atmosphere.

The history management unit 252 may record the amount of activity of the robot 100 in history information. Also, the history management unit 252 may record the amount of activity of the robot 100 in the reaction report. According to such a control method, the user can know from the reaction report whether the robot 100 has been actively moving during the day.

The user may notify the robot 100 of events that are scheduled to occur in the near future. For example, the user terminal 400 informs the server 20 of a schedule such as the arrival of a guest or the start of a party.
Notify 0. Emotion management section 250 may change the emotion parameter as the scheduled time approaches. According to such a control method, it is possible to make the robot 100 feel hopefulness and anxiety based on the future prediction. In addition, since the robot 100 changes its behavioral characteristics based on future predictions, the user can understand that the robot 100 is feeling expectations or anxiety.
can be felt from the actions of

In the present embodiment, the description has been made on the premise that the event is an external phenomenon such as seeing the user or being held. Not limited to this, an internal phenomenon such as the temperature inside the body 104 exceeding a predetermined value or the loneliness parameter being 20 or less may be defined as an event.

The robot 100 may have multiple types of emotional parameters, or may change behavioral characteristics based on a single emotional parameter. As mentioned above, the value of the first emotion parameter
The psychological condition may be established when the (first emotion value) is greater than or equal to the value of the second emotion parameter (second emotion value). Alternatively, the psychological condition may be established when the difference between the first emotion value and the second emotion value is greater than or equal to a predetermined threshold.

The psychological condition may be established when the emotion value of a certain emotion parameter is greater than or equal to a first threshold or less than or equal to a second threshold. Alternatively, the psychological condition may be established when the rate of change of the emotion value per unit time is equal to or greater than the third threshold or equal to or less than the fourth threshold. For example, the curiosity parameter may spike when discovering novelty, while the approval-seeking parameter may change only slowly. In this way, the manner of change may be individualized according to the type of emotion parameter. Therefore, various psychological conditions can be defined according to the characteristics of emotion parameters.

When detailed information is provided as a short movie, the start and end of the short movie may be determined according to the period during which psychological conditions are established. For example, assume that the first psychological condition is satisfied at time t5 and the first psychological condition is not satisfied at time t6. At this time, the history management unit 252
Alternatively, a short movie may be generated by clipping a moving image from time t5 to time t6 out of the moving images obtained from the camera of the robot 100. FIG.

Abnormalities occurring in the robot 100 may be recorded in the reaction report. For example, abnormal information such as processor 122 being overheated, emergency stop, actuator not working, battery 118 (rechargeable battery) being overheated, etc. can be sent to the user terminal by a reaction report. 400 may be notified.

Multiple robots 100 may share history information or reaction reports. A user
A description will be given assuming that (P01), the robot 100A, and the robot 100B are in the same household. User (P01) holds robot 100A at time t12, robot 1
Assume that the psychological condition is established at 00A. The "holding" at this time is defined as an event (Ex). The communication unit 142 of the robot 100A requests the robot 100B to acquire a photographed image (still image or moving image) for a predetermined period including time t12. The communication unit 142 of the robot 100B downloads the captured image Q3 in the corresponding period from the captured images stored in the server 200 to the robot 100B from the server 200, and then downloads the captured image Q3 to the robot 100A.
Send the captured image Q3 to The robot 100A registers the captured image Q3 in the server 200 as detailed information of the event (Ex). According to such a control method, the robot 100A
acquires "a figure in which oneself is held" as an image taken from the viewpoint of another robot 100B, and can provide this as detailed information. The robot 100B may transmit the celestial image for the corresponding period to the robot 100A. In this case, the robot 100A may generate a two-dimensional captured image Q3 from a celestial image (three-dimensional image) by a known image processing method.

Robot 100A may request server 200 instead of robot 100B to acquire an image captured by robot 100B at time t12. When one server 200 collectively manages both the robot 100A and the robot 100B, the history management unit 252 of the server 200 may copy and save the image Q3 captured by the robot 100B as the detailed information of the robot 100A.

The robot 100A may specify the start time and end time of the moving image for the robot 100B, and acquire the shot image Q3 as a short movie. For example robot 1
The communication unit 142 of 00A designates 5 seconds before the time t12 as the start time and 5 seconds after the time t12 as the end time.
You can get it from 100B.

When the robot 100A executes some motion, the robot 100A sends a shooting request to the robot 10.
May be sent to 0B. For example, the communication unit 142 of the robot 100A uses motion (M
When executing x), a photographing request is transmitted to the robot 100B before execution. robot 1
00B (the motion control unit 150) moves to the side of the robot 100A when receiving the photographing request. Robot 100A (recognition unit 156) executes motion (Mx) after recognizing that robot 100B is nearby. According to such a control method, the robot 10
It is possible to photograph the situation when 0A executes the motion (Mx) from the viewpoint of the robot 100B. The captured image at this time may be provided as a detailed image of the robot 100A. It may be difficult to understand what kind of motion robot 100A is doing in an image captured by robot 100A itself of the motion executed by robot 100A. If the robot 100B is present, it is possible to provide the user with a photographed image at a good angle. In particular, when the motion (Mx) is a rare motion that is rarely executed, the image captured by the robot B becomes a valuable captured image. By having multiple robots 100 photograph each other,
Various captured images (detailed images) that cannot be obtained with only one robot 100 can be obtained.

100 robot, 102 front wheel, 102a left wheel, 102b right wheel, 103 rear wheel,
104 body, 106 hands, 108 seating surface, 110 eyes, 112 horns, 114 external sensors, 118 batteries, 120 driving mechanisms, 122 processors, 124 storage devices, 126 communication devices, 128 internal sensors, 130 power lines, 132 signal lines, 134
wire, 136 data processing unit, 142 communication unit, 148 data storage unit, 150 motion control unit, 156 recognition unit, 158 imaging unit, 160 motion storage unit, 170 loneliness parameter, 172 curiosity parameter, 174 approval desire parameter, 180 1st
Reaction report screen, 182 event icons, 184 emotion icons, 186 attention marks, 190 2nd reaction report screen, 192 motion icons, 194 atmosphere icons, 200 server, 202 data processing unit, 204 communication unit, 206 data storage unit, 208 position management unit, 212 recognition unit, 218 personal data storage unit, 220 intimacy management unit, 222 motion control unit, 232 motion storage unit, 244 state management unit, 250
Emotion management part, 252 history management part, 254 icon selection part, 256 atmosphere determination part,
258 mode selection section, 260 event condition storage section, 262 history information storage section, 270
history information, 280 motion selection table, 290 user authority table, 292
Event Condition Table, 300 Robot System, 308 Base Frame, 310
body frame, 312 wheel cover, 314 skin, 316 head frame, 318
Torso frame, 320 yaw axis, 322 pitch axis, 324 roll axis, 325 plate, 326 actuator, 327 air vent, 328 base plate, 329 cross link mechanism, 330 joint, 332 upper plate, 334 lower plate, 336 side plate, 342 Control circuit, 370 wheel drive mechanism, 378 rotation axis, 379 actuator, 390 opening, 400 user terminal, 402 detailed information screen, 404 detailed window, 410 third reaction report screen, 420 rest time setting screen

Claims (14)

a motion control unit that selects the motion of the robot;
a drive mechanism that performs the motion selected by the motion controller;
a recognition unit that recognizes the occurrence of an event;
an emotion management unit that changes one or more emotion parameters according to an event;
a history management unit that associates an event occurrence history with an emotion parameter change history and records it as history information,
The motion control unit changes behavioral characteristics of the robot based on the emotion parameter,
The history management unit extracts from the history information an event when the emotional parameter satisfies a predetermined psychological condition and the emotional parameter when the psychological condition is satisfied, generates a reaction report, and provides the reaction report to the user. An autonomous action type robot characterized by: 2. The autonomous action robot according to claim 1, wherein the emotion management unit determines that the psychological condition is met when the first emotion parameter and the second emotion parameter are reversed in superiority. 2. The history management unit further records the motion selected by the motion control unit after the psychological condition is established in the reaction report, and then provides the reaction report to the user. 2. The autonomous action type robot according to 2. an imaging unit that images the surroundings of the robot;
A communication unit that executes communication processing with the communication terminal,
The recognition unit identifies a person from the captured image,
When receiving the reaction report acquisition request from the communication terminal of the first user, the history management unit instructs the communication unit on the condition that the first user is not specified in the captured image. 2. The autonomous action robot according to claim 1, wherein the reaction report is transmitted to the communication terminal of the first user. further comprising an icon selection unit that selects an icon according to an event,
The history management unit records an event when the psychological condition is established as an icon corresponding to the event in the reaction report, thereby representing the event occurrence history in the reaction report as an icon. The autonomous action type robot according to claim 1. 3. The history management unit adds an emphasized expression in the reaction report to an icon of an event belonging to a specific category among a plurality of types of categories.
5. The autonomous action type robot according to 5. The history management unit records detailed information about events, and when one of the one or more icons recorded in the reaction report is selected by the user, records detailed information about the event corresponding to the selected icon. 7. The autonomous action type robot according to claim 5, which is provided to a user. The history management unit specifies a viewing attribute that is preliminarily associated with a user, and selects only events that match the specified viewing attribute from among events that occur when the psychological condition is established, to create a reaction report to be provided to the user. 2. The autonomous action robot according to claim 1, wherein the robot records. wherein the history management unit records only events matching a browsing authority level preset for the user among the events when the psychological condition is satisfied in the reaction report to be provided to the user. Item 1. The autonomous action type robot according to Item 1. an atmosphere determination unit that determines success or failure of each of a plurality of predetermined atmosphere conditions based on external information;
4. The history management unit further records atmosphere information associated with the established atmosphere condition in the reaction report when any one of the atmosphere conditions is established.
1. The autonomous action type robot according to 1. further comprising a mode selection unit that selects the mode of the robot,
2. The autonomous action robot according to claim 1, wherein the history management unit further records a change history of the mode of the robot in the reaction report. The recognition unit recognizes regular behavior of the user as a regular event,
2. The autonomous action robot according to claim 1, wherein said history management unit further records success or failure of said regular event in said reaction report. A communication unit that executes communication processing with the communication terminal,
2. The autonomous action type robot according to claim 1, wherein, when instructed to set a rest period from the communication terminal, the motion control unit causes the robot to be stationary on an external charger during the rest period. robot. A function to select the motion of the robot,
the ability to perform the selected motion;
the ability to recognize the occurrence of an event;
A function to change one or more emotion parameters according to an event;
A function to record the event occurrence history and the emotional parameter change history in association with each other as history information;
A function to change the behavioral characteristics of the robot based on emotional parameters;
a function of extracting from the history information an event when an emotional parameter satisfies a predetermined psychological condition and the emotional parameter when the psychological condition is satisfied, generating a reaction report, and providing the reaction report to the user; An action control program for an autonomous action type robot characterized by exerting.