JP2020032014A - Autonomously acting robot recording daily life - Google Patents

Abstract

To report a daily life which an autonomously acting robot experiences to a user while protecting privacy.SOLUTION: A robot 100 comprises: an operation control unit for selecting a motion; a drive mechanism for executing the motion; a recognition unit for recognizing an occurrence of an event; an emotion management unit for changing one or more emotion parameters according to the event; and a history management unit for recording an occurrence history of events and a change history of the emotion parameters in association as history information. The robot 100 changes a behavior feature on the basis of the emotion parameters. When a prescribed psychological condition for an emotion parameter holds, for example, the event when the value of an emotion parameter indicating curiosity especially increased and the emotion parameter at the time are extracted from the history information to generate a reaction report, which is then provided to the user.SELECTED DRAWING: Figure 1

Description

  The present invention relates to a robot that autonomously selects an action according to an internal state or an external environment.

  Humans keep pets for healing. On the other hand, many people give up pets for various reasons, such as not having enough time to care for pets, lack of living environment for pets, allergies, and difficult bereavement. If there is a robot that plays the role of a pet, healing may be provided to a person who cannot keep a pet as if the pet provided it (see Patent Documents 1 and 2).

JP 2000-323219 A International Publication No. WO 2017/169826 JP 2005-267375 A JP 2010-21921 A JP 2008-172815 A

  In recent years, robot technology has been advancing rapidly, but has not yet achieved a presence as a companion like a pet. This is because the robot does not seem to have free will. By observing the behavior of a pet that only seems to have a free will, a human feels the free will of the pet, sympathizes with the pet, and is healed by the pet.

  At present, a number of cameras have been released for watching over pets that are away. According to such a camera, it is possible to know how the pet spends when there is no owner. If the robot comes to show the same presence as a pet, there is a need to know the daily life of the robot while away.

  However, simply mounting a camera on the robot and broadcasting the captured image live to a smartphone or the like simply turns the robot into a mere monitoring device. For example, when the user A and the robot are at home, if the robot transmits the captured image as it is to the smartphone of the user B who is out, the privacy of the user A may not be protected.

  The present invention has been completed based on the recognition of the above-mentioned problems, and a main object of the present invention is to provide a technique for notifying a user of the daily life of a robot while giving consideration to privacy.

An autonomous behavior robot according to an aspect of the present invention includes an operation control unit that selects a motion of the robot, a driving mechanism that executes the motion selected by the operation control unit, a recognition unit that recognizes occurrence of an event, An emotion management unit that changes one or more emotion parameters in response thereto, and a history management unit that records event occurrence history and change history of the emotion parameter as history information in association with each other.
The motion control unit changes the behavior characteristic of the robot based on the emotion parameter.
The history management unit extracts an event when the emotion parameter satisfies a predetermined psychological condition and an emotion parameter when the psychological condition is satisfied from the history information, generates a reaction report, and provides the user with the reaction report.

  ADVANTAGE OF THE INVENTION According to this invention, it becomes easy to notify a user of the daily life of a robot, considering privacy.

FIG. 3 is a conceptual diagram for explaining a method in which a robot communicates its daily life to a user. FIG. 2A is a front external view of the robot. FIG. 2B is a side external view of the robot. It is sectional drawing which represents the structure of a robot schematically. FIG. 3 is a hardware configuration diagram of a robot in a basic configuration. It is a functional block diagram of a robot system. It is a schematic diagram showing a reaction report generation process in the present embodiment. It is a functional block diagram of a robot in this embodiment. It is a schematic diagram which shows the change history of an emotion parameter. It is a data structure figure of history information. It is a data structure figure of a motion selection table. It is a screen diagram of a 1st reaction report screen. It is a screen diagram of a 2nd reaction report screen. It is a data structure figure of a user authority table. It is a data structure figure of an event condition table. It is a sequence diagram showing a display process of a reaction report. FIG. 9 is a sequence diagram illustrating a process of displaying details. It is a screen figure of a detailed information screen. It is a screen figure of the 3rd reaction report screen. It is a screen diagram of a rest time setting screen.

FIG. 1 is a conceptual diagram for explaining how the robot 100 communicates its daily life to a user.
The robot 100 in the present embodiment has a plurality of types of “emotions”. Specifically, the robot 100 numerically expresses the strengths of three kinds of emotions, such as loneliness, curiosity, and desire for approval, using three kinds of emotion parameters. As will be described later, the emotional expression of the robot 100 may be expressed by emotional parameters other than these three types of emotional parameters.

  The robot 100 includes a plurality of types of sensors, and recognizes an external environment by each sensor. The robot 100 recognizes a part of the change in the external environment as an “event”. The event may be arbitrarily defined as a “notable event” such as visually recognizing the user or detecting a loud sound. The robot 100 autonomously selects an action according to an event and an emotion parameter. The event affects both the behavior selection method and the emotion parameter of the robot 100, and the emotion parameter also affects the behavior selection method. Details of the event, the emotion parameter, and the action selection method will be described later.

  The robot 100 records “daily events” such as events as “history information” one by one. The robot 100 generates a “reaction report” to be shown to the user based on the history information. The user obtains a reaction report using the user terminal 400 such as a smartphone, a laptop PC, and a tablet computer. By displaying the reaction report on the user terminal 400, the user can get a glimpse of the daily life of the robot 100 even when not meeting with the robot 100.

Even when the user is out, the user can know from the reaction report how the at-home robot 100 is spending and what day he spent. The reaction report functions as a so-called “diary” of the robot 100. The reaction report in the present embodiment is “history information for showing”, and is not the same as the history information recorded by the robot 100 one by one. The history information may include privacy-related information. In the present embodiment, a reaction report that is easy to read by the user and is generated while considering the privacy is newly generated.
Hereinafter, the basic configuration of the robot 100 will be described with reference to FIGS. 2 to 5, and then the method of generating a reaction report will be mainly described.

[Basic configuration]
FIG. 2A is a front external view of the robot 100. FIG. 2B is a side external view of the robot 100.
The robot 100 in the present embodiment is an autonomous behavior type robot that determines a behavior based on an external environment and an internal state. The external environment is recognized by various sensors such as a camera and a thermosensor. The internal state is quantified as various parameters expressing the emotion of the robot 100. The robot 100 sets the home of the owner's home as an action range. Hereinafter, a person related to the robot 100 is referred to as a “user”.

  The body 104 of the robot 100 has a rounded shape as a whole, and includes an outer skin made of a soft and elastic material such as urethane, rubber, resin, or fiber. The robot 100 may be dressed. The total weight of the robot 100 is about 5 to 15 kilograms and the height is about 0.5 to 1.2 meters. By various attributes such as moderate weight, roundness, softness, and good touch, the effect that the user can easily hold the robot 100 and want to hold the robot 100 is realized.

  The robot 100 includes a pair of front wheels 102 (left wheel 102a, right wheel 102b) and one rear wheel 103. The front wheel 102 is a driving wheel, and the rear wheel 103 is a driven wheel. Although the front wheel 102 does not have a steering mechanism, the rotation speed and the rotation direction can be individually controlled. The rear wheel 103 is a caster, and is rotatable to move the robot 100 forward, backward, left, and right. The rear wheel 103 may be an omni wheel.

  The front wheel 102 and the rear wheel 103 can be completely housed in the body 104 by a driving mechanism (rotating mechanism, link mechanism). During running, most of the wheels are hidden by the body 104. However, when the wheels are completely stored in the body 104, the robot 100 cannot move. That is, the body 104 descends as the wheels are stored, and sits on the floor F. In this seated state, the flat seating surface 108 (ground contact bottom surface) formed on the bottom of the body 104 abuts on the floor surface F.

  The robot 100 has two hands 106. The hand 106 does not have a function of holding an object. The hand 106 can perform simple operations such as raising, shaking, and vibrating. The two hands 106 can also be individually controlled.

  The eye 110 can display an image using a liquid crystal element or an organic EL element. The robot 100 is equipped with various sensors such as a microphone array and an ultrasonic sensor capable of specifying a sound source direction. It also has a built-in speaker and can emit simple sounds.

  A horn 112 is attached to the head of the robot 100. As described above, since the robot 100 is lightweight, the user can lift the robot 100 by grasping the horn 112. A celestial sphere camera is attached to the horn 112, and can image the entire upper region of the robot 100 at a time.

FIG. 3 is a cross-sectional view schematically illustrating the structure of the robot 100.
As shown in FIG. 3, the body 104 of the robot 100 includes a base frame 308, a main body frame 310, a pair of resin wheel covers 312, and an outer skin 314. The base frame 308 is made of metal, constitutes the axis of the body 104, and supports an internal mechanism. The base frame 308 is configured by vertically connecting an upper plate 332 and a lower plate 334 with a plurality of side plates 336. A sufficient space is provided between the plurality of side plates 336 so as to allow ventilation. The battery 118, the control circuit 342, and various actuators are housed inside the base frame 308.

  The main body frame 310 is made of a resin material, and includes a head frame 316 and a body frame 318. The head frame 316 has a hollow hemisphere shape and forms a head skeleton of the robot 100. The torso frame 318 has a stepped cylindrical shape and forms a torso skeleton of the robot 100. The body frame 318 is fixed integrally with the base frame 308. The head frame 316 is attached to the upper end of the body frame 318 so as to be relatively displaceable.

  The head frame 316 is provided with three axes of a yaw axis 320, a pitch axis 322, and a roll axis 324, and an actuator 326 for driving each axis to rotate. The actuator 326 includes a plurality of servomotors for individually driving each axis. The yaw axis 320 is driven for swinging, the pitch axis 322 is driven for nodding, and the roll axis 324 is driven for tilting.

  A plate 325 that supports the yaw axis 320 is fixed to an upper part of the head frame 316. A plurality of ventilation holes 327 are formed in the plate 325 to ensure ventilation between the upper and lower sides.

  A metal base plate 328 is provided to support head frame 316 and its internal mechanism from below. The base plate 328 is connected to the plate 325 via a cross link mechanism 329 (pantograph mechanism), and is connected to the upper plate 332 (base frame 308) via a joint 330.

  The body frame 318 houses the base frame 308 and the wheel drive mechanism 370. The wheel driving mechanism 370 includes a rotation shaft 378 and an actuator 379. The lower half of the body frame 318 has a small width in order to form a storage space S for the front wheel 102 between itself and the wheel cover 312.

  The outer cover 314 is made of urethane rubber and covers the main body frame 310 and the wheel cover 312 from outside. Hand 106 is integrally formed with outer skin 314. An opening 390 for introducing outside air is provided at the upper end of the outer cover 314.

FIG. 4 is a hardware configuration diagram of the robot 100.
The robot 100 includes an internal sensor 128, a communication device 126, a storage device 124, a processor 122, a driving mechanism 120, and a battery 118. The processor 122 and the storage device 124 are included in the control circuit 342. Each unit is connected to each other by a power supply line 130 and a signal line 132. The battery 118 supplies power to each unit via a power supply line 130. Each unit transmits and receives a control signal via a signal line 132. The battery 118 is a lithium ion secondary battery, and is a power source of the robot 100.

  The internal sensor 128 is an aggregate of various sensors built in the robot 100. Specifically, examples include a camera (omnidirectional camera), a microphone array, a distance measurement sensor (infrared sensor), a thermo sensor, a touch sensor, an acceleration sensor, and an odor sensor. The touch sensor is installed between the outer skin 314 and the main body frame 310, and detects a user's touch. The odor sensor is a known sensor to which the principle of changing the electric resistance by the adsorption of the molecule that causes the odor is applied.

  The communication device 126 is a communication module that performs wireless communication for various external devices. The storage device 124 includes a nonvolatile memory and a volatile memory, and stores a computer program and various setting information. The processor 122 is a means for executing a computer program. The drive mechanism 120 includes a plurality of actuators and the wheel drive mechanism 370 described above. In addition, displays and speakers are also installed.

  The drive mechanism 120 mainly controls the wheels (the front wheels 102) and the head (the head frame 316). The drive mechanism 120 can change the moving direction and the moving speed of the robot 100, and can also raise and lower wheels (the front wheel 102 and the rear wheel 103). When the wheels rise, the wheels are completely housed in the body 104, and the robot 100 comes into contact with the floor surface F at the seating surface 108 to be in a seated state. The driving mechanism 120 controls the hand 106 via the wire 134.

FIG. 5 is a functional block diagram of the robot system 300.
The robot system 300 includes the robot 100, the server 200, and a plurality of external sensors 114. Each component of the robot 100 and the server 200 includes: an arithmetic unit such as a CPU (Central Processing Unit) and various coprocessors; a storage device such as a memory and a storage; hardware including a wired or wireless communication line connecting them; This is realized by software that is stored in the device and supplies processing instructions to the arithmetic unit. The computer program may be configured by a device driver, an operating system, various application programs located in an upper layer thereof, and a library that provides a common function to these programs. Each block described below is not a configuration in a hardware unit but a block in a functional unit.
Some of the functions of the robot 100 may be realized by the server 200, and some or all of the functions of the server 200 may be realized by the robot 100.

  A plurality of external sensors 114 are installed in the house in advance. The position coordinates of the external sensor 114 are registered in the server 200. The server 200 determines the basic behavior of the robot 100 based on information obtained from the internal sensor 128 and the plurality of external sensors 114 of the robot 100. The external sensor 114 is for reinforcing the sensory organs of the robot 100, and the server 200 is for reinforcing the brain of the robot 100. The communication device 126 of the robot 100 periodically communicates with the external sensor 114, and the server 200 specifies the position of the robot 100 using the external sensor 114 (see also Patent Document 2).

(Server 200)
The server 200 includes a communication unit 204, a data processing unit 202, and a data storage unit 206.
The communication unit 204 handles communication processing with the external sensor 114 and the robot 100. The data storage unit 206 stores various data. The data processing unit 202 performs various processes based on the data acquired by the communication unit 204 and the data stored in the data storage unit 206. The data processing unit 202 also functions as an interface between the communication unit 204 and the data storage unit 206.

The data storage unit 206 includes a motion storage unit 232 and a personal data storage unit 218.
The robot 100 has a plurality of motion patterns (motions). Various motions are defined, such as shaking the hand 106, meandering toward the owner, and staring at the owner with his neck stuck.

  The motion storage unit 232 stores a “motion file” that defines the control content of the motion. Each motion is identified by a motion ID. The motion file is also downloaded to the motion storage unit 160 of the robot 100. Which motion is executed may be determined by the server 200 or may be determined by the robot 100.

  Many of the motions of the robot 100 are configured as a composite motion including a plurality of unit motions. For example, when the robot 100 approaches the owner, it may be expressed as a combination of a unit motion facing the owner, a unit motion approaching with the hand raised, a unit motion approaching while shaking the body, and a unit motion sitting with the hands raised. . By such a combination of the four motions, a motion of "approaching the owner, raising his hand on the way, and finally sitting with his body shaken" is realized. In the motion file, a rotation angle, an angular velocity, and the like of an actuator provided in the robot 100 are defined in association with a time axis. Various motions are expressed by controlling each actuator over time according to the motion file (actuator control information).

The transition time when changing from the previous unit motion to the next unit motion is called an “interval”. The interval may be defined according to the time required for changing the unit motion or the content of the motion. The length of the interval is adjustable.
Hereinafter, settings relating to the behavior control of the robot 100, such as when and which motion is selected, output adjustment of each actuator for realizing the motion, etc., are collectively referred to as "behavior characteristics". The behavior characteristics of the robot 100 are defined by a motion selection algorithm, a motion selection probability, a motion file, and the like.

  The motion storage unit 232 stores, in addition to the motion file, a motion selection table that defines a motion to be executed when various events occur. In the motion selection table, one or more motions and their selection probabilities are associated with events.

  The personal data storage 218 stores user information. Specifically, it stores master information indicating intimacy with the user and physical and behavioral characteristics of the user. Other attribute information such as age and gender may be stored.

  The robot 100 has an internal parameter called intimacy for each user. When the robot 100 recognizes an action that favors itself, such as holding up or calling out, the intimacy with the user increases. The intimacy with a user who is not involved in the robot 100, a user who works violently, or a user who meets with a low frequency is low.

The data processing unit 202 includes a position management unit 208, a recognition unit 212, an operation control unit 222, a closeness management unit 220, and a state management unit 244.
The position management unit 208 specifies the position coordinates of the robot 100. The state management unit 244 manages various internal parameters such as various physical states such as a charging rate, an internal temperature, and a processing load of the processor 122. Further, the state management unit 244 manages various emotion parameters indicating the emotions (loneliness, curiosity, approval desire, and the like) of the robot 100. These emotion parameters are always fluctuating. The movement target point of the robot 100 changes according to the emotion parameter. For example, when loneliness is increasing, the robot 100 sets a place where the user is located as a movement target point.

  The emotion parameter changes over time. Various emotion parameters also change depending on a response action described later. For example, the emotion parameter indicating loneliness decreases when the owner “holds”, and the emotion parameter indicating loneliness gradually increases when the owner is not visually recognized for a long time.

  The recognition unit 212 recognizes an external environment. The recognition of the external environment includes various recognitions such as recognition of weather and season based on temperature and humidity, and recognition of a shadow (safety zone) based on light amount and temperature. The recognizing unit 156 of the robot 100 acquires various types of environmental information by the internal sensor 128, performs primary processing on the acquired information, and transfers the information to the recognizing unit 212 of the server 200.

  Specifically, the recognition unit 156 of the robot 100 extracts a moving object, in particular, an image region corresponding to a person or an animal from the image, and indicates a physical or behavioral characteristic of the moving object from the extracted image region. “Feature vector” is extracted as a set of quantities. The feature vector component (feature amount) is a numerical value quantifying various physical and behavioral features. For example, the width of the human eye is quantified in the range of 0 to 1 to form one feature vector component. The method of extracting a feature vector from a captured image of a person is an application of a known face recognition technology. The robot 100 transmits the feature vector to the server 200.

  The recognition unit 212 of the server 200 compares the feature vector extracted from the image captured by the built-in camera of the robot 100 with the feature vector of the user (cluster) registered in the personal data storage unit 218 in advance, so that the image is captured. It is determined which person the user belongs to (user identification processing). In addition, the recognition unit 212 estimates the emotion of the user by image-recognizing the expression of the user. The recognizing unit 212 also performs a user identification process on a moving object other than a person, for example, a cat or dog that is a pet.

The recognizing unit 212 recognizes various response actions performed on the robot 100 and classifies the actions into pleasant and unpleasant actions. The recognition unit 212 also classifies the response into an affirmative / negative response by recognizing the response of the owner to the behavior of the robot 100.
Pleasant or unpleasant behavior is determined based on whether the user's response behavior is comfortable or unpleasant as a living thing. For example, being held is a pleasure for the robot 100, and being kicked is a discomfort to the robot 100. The affirmative / negative response is determined based on whether the user's response action indicates a pleasant feeling or a discomfort feeling of the user. Holding is a positive reaction indicating the user's pleasant feeling, and being kicked is a negative reaction indicating the user's unpleasant feeling.

  The operation control unit 222 of the server 200 determines the motion of the robot 100 in cooperation with the operation control unit 150 of the robot 100. The operation control unit 222 of the server 200 creates a movement target point of the robot 100 and a movement route therefor. The operation control unit 222 may create a plurality of travel routes, and then select any one of the travel routes.

  The operation control unit 222 selects a motion of the robot 100 from a plurality of motions in the motion storage unit 232. Each motion is associated with a selection probability for each situation. For example, a selection method is defined in which the motion A is executed with a probability of 20% when a pleasant action is performed by the owner, and the motion B is executed with a probability of 5% when the temperature exceeds 30 degrees. .

  The closeness management unit 220 manages the closeness for each user. As described above, the intimacy is registered in the personal data storage unit 218 as a part of personal data. When a pleasant activity is detected, the intimacy management unit 220 increases the intimacy with the owner. The intimacy drops when an unpleasant activity is detected. In addition, the intimacy of owners who have not viewed for a long time gradually decreases.

(Robot 100)
The robot 100 includes a communication unit 142, a data processing unit 136, a data storage unit 148, an internal sensor 128, and a driving mechanism 120.
The communication unit 142 corresponds to the communication device 126 (see FIG. 4), and is responsible for communication processing with the external sensor 114, the server 200, and the other robot 100. The data storage unit 148 stores various data. The data storage unit 148 corresponds to the storage device 124 (see FIG. 4). The data processing unit 136 performs various processes based on the data acquired by the communication unit 142 and the data stored in the data storage unit 148. The data processing unit 136 corresponds to the processor 122 and a computer program executed by the processor 122. The data processing unit 136 also functions as an interface of the communication unit 142, the internal sensor 128, the driving mechanism 120, and the data storage unit 148.

The data storage unit 148 includes a motion storage unit 160 that defines various motions of the robot 100.
Various motion files are downloaded from the motion storage unit 232 of the server 200 to the motion storage unit 160 of the robot 100. A motion is identified by a motion ID. A state in which the front wheel 102 is housed, in which the robot 100 rotates by seating and sitting on the front wheel 102, lifting the hand 106, or by rotating the two front wheels 102 in reverse or by rotating only one of the front wheels 102. In order to express various motions such as shaking by rotating the front wheel 102 and stopping and turning back when leaving the user, the operation timing, operation time, operation direction, etc. of various actuators (drive mechanism 120) are expressed. Time series is defined in the motion file.
Various data may be downloaded from the personal data storage unit 218 to the data storage unit 148.

The data processing unit 136 includes a recognition unit 156 and an operation control unit 150.
The operation control unit 150 of the robot 100 determines the motion of the robot 100 in cooperation with the operation control unit 222 of the server 200. Some motions may be determined by the server 200, and other motions may be determined by the robot 100. Further, the robot 100 determines the motion, but when the processing load of the robot 100 is high, the server 200 may determine the motion. The base motion may be determined in the server 200, and the additional motion may be determined in the robot 100. How the motion determination process is shared between the server 200 and the robot 100 may be designed according to the specifications of the robot system 300.

  The operation control unit 150 of the robot 100 instructs the drive mechanism 120 to execute the selected motion. The drive mechanism 120 controls each actuator according to the motion file.

  The motion control unit 150 can also execute a motion of raising both hands 106 as a gesture to hug "Hug" when a user with high intimacy is nearby, and when the user is tired of "Hug", the left and right front wheels 102 By alternately repeating the reverse rotation and the stop with the stowed, it is possible to express a motion that hugs the hug. The drive mechanism 120 causes the robot 100 to express various motions by driving the front wheel 102, the hand 106, and the neck (the head frame 316) according to an instruction from the operation control unit 150.

  The recognition unit 156 of the robot 100 interprets external information obtained from the internal sensor 128. The recognition unit 156 can perform visual recognition (visual unit), odor recognition (olfactory unit), sound recognition (auditory unit), and tactile recognition (tactile unit).

  The recognition unit 156 extracts a feature vector from a captured image of a moving object. As described above, a feature vector is a set of parameters (feature amounts) indicating physical and behavioral features of a moving object. When a moving object is detected, physical characteristics and behavioral characteristics are also extracted from an odor sensor, a built-in sound collecting microphone, a temperature sensor, and the like. These features are also quantified and become feature vector components. The recognizing unit 156 specifies the user from the feature vector based on a known technique described in Patent Literature 2 and the like.

  Among a series of recognition processes including detection, analysis, and determination, the recognition unit 156 of the robot 100 selects and extracts information necessary for recognition, and interpretation processing such as determination is performed by the recognition unit 212 of the server 200. . The recognition process may be performed only by the recognition unit 212 of the server 200, may be performed only by the recognition unit 156 of the robot 100, or may be performed by performing the above-described recognition process while sharing roles. Is also good.

When a strong impact is given to the robot 100, the recognition unit 156 recognizes this by a touch sensor and an acceleration sensor, and the recognition unit 212 of the server 200 recognizes that a “rough act” has been performed by a nearby user. When the user grasps the horn 112 and lifts the robot 100, it may be recognized as a violent act. When the user who is directly facing the robot 100 utters in the specific volume region and the specific frequency band, the recognition unit 212 of the server 200 may recognize that the “calling action” to the user has been performed. Further, when a temperature of about the body temperature is detected, it is recognized that a “touch action” has been performed by the user, and when an upward acceleration is detected in a state where the touch is recognized, it is recognized that a “hold” has been performed. The physical contact when the user lifts the body 104 may be sensed, or the holding may be recognized when the load applied to the front wheel 102 decreases.
In summary, the robot 100 acquires the action of the user as physical information by the internal sensor 128, and the recognition unit 212 of the server 200 determines whether it is pleasant or unpleasant. In addition, the recognition unit 212 of the server 200 executes a user identification process based on the feature vector.

  The recognition unit 212 of the server 200 recognizes various responses of the user to the robot 100. Pleasant or unpleasant, affirmative or negative are associated with some typical responses among various responses. In general, most of the responding acts that are pleasant are positive reactions, and most of the disturbing acts are negative reactions. Pleasant or unpleasant behavior is related to intimacy, and affirmative or negative reactions affect the robot 100's action selection.

  In accordance with the response action recognized by the recognition unit 156, the familiarity management unit 220 of the server 200 changes the familiarity with the user. In principle, the intimacy with the user who performed the pleasure increases, and the intimacy with the user who performs the pleasure decreases.

  Assuming the above basic configuration, the mounting of the robot 100 in the present embodiment will be described below, particularly focusing on the features and purpose of the mounting and the differences between the basic configuration and the present embodiment.

[Implementation of reaction report generation function]
FIG. 6 is a schematic diagram illustrating a reaction report generation process according to the present embodiment.
The robot 100 records an event, an emotion parameter, and the like as history information (S10). The history information records the feelings (emotional parameters) and events (events) of the robot 100 in chronological order. The user requests the server 200 to obtain a response report from the user terminal 400 (S12). When receiving the response report acquisition request, the server 200 generates a response report from the history information (S14). Since the history information includes a large amount of information, when the history information is used as it is as a reaction report, the visibility of the reaction report deteriorates. Also, from the viewpoint of privacy protection, it is not preferable to provide the history information itself to the user as a response report. In the present embodiment, a reaction report as information to be shown is generated after selecting various types of information included in the history information and changing the expression method of the information (described later in detail). The user terminal 400 acquires the reaction report from the robot system 300 and displays it on the screen (S16).

FIG. 7 is a functional block diagram of the robot 100 according to the present embodiment.
The robot system 300 according to the present embodiment has a reaction report generation function. Therefore, a functional block of the robot system 300 shown in FIG. 5 and a plurality of functional blocks are added. In FIG. 7, the description will be focused on the added functional blocks.

  Note that the robot 100 may recognize the location based on a key frame instead of the external sensor 114. A key frame is distribution information of feature points (feature amounts) in a captured image. Even if the robot 100 forms a key frame by a graph-based Simultaneous Localization and Mapping (SLAM) technique using an image feature quantity, more specifically, a SLAM technique based on an ORB (Oriented FAST and Rotated BRIEF) feature quantity. Good (for details, refer to Japanese Patent Application No. 2018-014365).

(Server 200)
The data processing unit 202 further includes a history management unit 252, an icon selection unit 254, an atmosphere determination unit 256, and a mode selection unit 258. The state management unit 244 includes an emotion management unit 250.
The emotion management unit 250 includes an emotion parameter indicating the intensity of loneliness (hereinafter, referred to as “loneliness parameter”), an emotion parameter indicating the intensity of curiosity (hereinafter, referred to as “curiosity parameter”), and a desire for approval. Of three types of emotion parameters (hereinafter, referred to as “approval desire parameters”) indicating the strength of the user.

  The history management unit 252 manages history information of the robot 100. In the history information, an event occurrence history and an emotion parameter change history can be associated with each other. The history information will be described later with reference to FIG. Further, the history management unit 252 generates a reaction report based on the history information.

  The icon selection unit 254 selects an icon according to the event. Although details will be described later, in the reaction report of the present embodiment, the event is represented by an icon as an image. The icon selection unit 254 selects an icon associated with the event. The icon selection unit 254 selects various icons other than the event.

  The atmosphere determination unit 256 determines the atmosphere of the external environment. The method of determining the atmosphere will be described later. The mode selection unit 258 selects a mode of the robot 100. The mode is setting information on the operation state of the robot 100 such as a normal mode and a power saving mode. Details of the mode selection method will be described later.

  The data storage unit 206 further includes an event condition storage unit 260 and a history information storage unit 262. The event condition storage 260 defines an event condition. The event condition is a condition that defines a situation at the time of the event occurrence, such as “occurred in a child room” or “occurred in a front door”. The history information storage unit 262 stores history information.

(Robot 100)
The internal sensor 128 of the robot 100 includes an imaging unit 158. The imaging unit 158 corresponds to a camera such as a spherical camera. In the present embodiment, the imaging unit 158 constantly captures a peripheral image. The recognition unit 156 of the robot 100 analyzes external information (raw data) obtained from the internal sensor 128 such as the imaging unit 158, and transmits an analysis result (processed data) to the server 200. The recognition unit 212 of the server 200 determines the success or failure of various events based on the analysis result. The determination of the success or failure of the event may be performed only on the robot 100 side or may be performed only on the server 200 side. A simple event is determined by the recognition unit 156 of the robot 100, and a complicated event is determined by the server 200. The recognition unit 212 may make the determination.
In the following, a description will be given assuming that the recognition unit 212 of the server 200 determines the success or failure of the event.

  The personal data storage unit 218 of the server 200 registers the terminal ID for identifying the user terminal 400 and the user ID in association with each other. The user ID is associated with the physical characteristics and the behavioral characteristics of the user.

(User terminal 400)
The user terminal 400 displays a reaction report described later on a web browser. In the user terminal 400, dedicated application software for displaying a reaction report and inputting an instruction to the robot 100 (detailed with reference to FIG. 19) may be installed.

FIG. 8 is a schematic diagram showing a change history of the emotion parameter.
The emotion management unit 250 changes three kinds of emotion parameters with time. The horizontal axis in FIG. 8 represents time, and the vertical axis represents the magnitude of the emotion parameter. In the present embodiment, the emotion parameter changes in the range of 0 to 100. In FIG. 8, the loneliness parameter 170 temporarily increases and then gradually decreases. The emotion management unit 250 gradually increases the loneliness parameter 170 as the period during which the robot 100 does not visually recognize the owner increases, and decreases the loneliness parameter 170 when the owner is visually recognized. The event of visually recognizing the owner contributes to a decrease in the loneliness parameter 170. Similarly, when an event of listening to the owner's voice occurs, the emotion management unit 250 may lower the loneliness parameter 170.

  The approval desire parameter 174 of FIG. 8 gradually increases with time. The emotion management unit 250 increases the approval desire parameter 174 over time. The desire for approval is one of the basic desires of human beings to “be recognized by others”. The approval desire parameter 174 expresses the strength of the robot 100's approval desire. The emotion manager 250 lowers the approval desire parameter 174 (satisfaction) when the user performs a pleasant action (attentive action on the robot 100), such as speaking to or holding the robot 100, and the discomfort is performed. Then, the approval desire parameter 174 is raised (dissatisfied).

  The emotion management unit 250 increases the curiosity parameter 172 when the robot 100 finds a new one. When no event occurs, the emotion management unit 250 gradually decreases the curiosity parameter 172.

  The emotion management unit 250 determines whether or not the three types of emotion parameters satisfy various “psychological conditions”. The psychological condition may be arbitrarily set as a timing indicating “change in psychology” of the robot 100. For example, when the approval desire parameter 174 is equal to or greater than a predetermined threshold (for example, 80), the first psychological condition may be satisfied, or the approval desire parameter 174 becomes dominant over the other two emotion parameters. At this time, the second psychological condition may be satisfied. In FIG. 8, in the period t0 to t1, the loneliness parameter 170 is larger than the curiosity parameter 172 and the approval desire parameter 174. In the period t1 to t2, the approval desire parameter 174 is the largest. Therefore, emotion management unit 250 determines that the second psychological condition is satisfied at time t1. In other words, at time t1, the robot 100 is more dominated by the desire for approval (the desire to be recognized) than loneliness.

  When the first emotion parameter changes from the state in which the first emotion parameter is dominant to the state in which the second emotion parameter is dominant, in other words, when the order of the plurality of emotion parameters is reversed, the robot 100 It is thought that feeling changed greatly.

  In addition, not only three types of loneliness, curiosity, and approval desire, but four or more types of emotion parameters may be set. For example, various emotion parameters, such as joy, anger, anxiety, frustration, and shame, and events that affect those emotion parameters may be defined. The emotion parameter may be one type. The emotion parameter changes over time and the occurrence of an event.

  The change of the emotion parameter may have randomness or periodicity. Changes in the first emotion parameter may affect the second emotion parameter. For example, when the loneliness parameter is 50 or more, the emotion management unit 250 may increase the rate of increase of the approval desire parameter.

  The emotion parameter does not necessarily need to correspond to a typical mental state of a human, such as “curiosity”. For example, by defining a change condition such as “the emotion parameter Q1 rises when the event (E1) or the event (E2) occurs, and falls when the event (E3) occurs”, various personalities are defined. May be defined. An “emotion” unique to the robot 100 that does not imitate human emotions may be defined.

FIG. 9 is a data structure diagram of the history information 270.
The history information 270 is stored in the history information storage unit 262 of the server 200. In the robot system 300, a plurality of types of events are defined in advance. Events are classified into one of a plurality of event types. For example, an event of event type = En (hereinafter, referred to as “event (En)”) may be visually recognizing the user. The event (Em) may be detection of a voice of a predetermined value or more, or another event (Ep) may be that the robot 100 wears clothes. A variety of other events can be defined, such as being hugged, a user going out, or finding something new in the room.

  The history management unit 252 records events and emotion parameters in the history information 270 periodically, for example, every minute. History information 270 is associated with an event occurrence history and an emotion parameter change history. The recognition unit 212 recognizes the event, and the emotion management unit 250 changes the emotion parameter according to the event. The operation control unit 150 and the like select a motion according to the event and the emotion parameter.

  In the history information 270 of FIG. 9, an event (E1) occurs at 10:03. After the occurrence of the event (E1), the loneliness parameter sharply drops from 72 to 14. It can be seen that the event (E1) is an important event that has eliminated the loneliness of the robot 100. In addition to the above, various information such as an atmosphere type indicating the atmosphere of the external environment (details will be described later), a motion selected by the robot 100, position information of the robot 100, and room temperature may be registered in the history information 270.

FIG. 10 is a data structure diagram of the motion selection table 280.
The motion selection table 280 is stored in the motion storage 232. As described above, a motion is identified by a motion ID. When recognizing the event, the operation control unit 222 of the server 200 refers to the motion selection table 280 to select a motion. The robot 100 executes the motion specified by the operation control unit 222. The motion selection table 280 is also downloaded from the server 200 to the robot 100. Therefore, the operation control unit 150 of the robot 100 can select a motion corresponding to an event without passing through the server 200 for some events.
In the following, description will be made on the assumption that a motion is selected by the operation control unit 222 (server 200). However, the operation control unit 222 (server 200) can be read as the operation control unit 150 (robot 100).

  In the motion selection table 280, one or more motions and their selection probabilities are associated with the event. For example, when the event (E1) is detected, the operation control unit 222 selects the motion (M1) with a selection probability of 5 (%) and selects the motion (M2) with a selection probability of 15 (%). .

  In the present embodiment, the motion control unit 222 changes the motion selection probability according to the emotion parameter. For example, when the loneliness parameter among the emotion parameters is equal to or greater than a predetermined threshold, the operation control unit 222 increases the selection probability of the motion (M1) for the event (E1). When the curiosity parameter is equal to or more than the predetermined threshold, the operation control unit 222 reduces the selection probability of the motion (M1) for the event (E1).

  As a specific example, the selection probabilities of the motion (M1), the motion (M2), and the motion (M3) that can be selected when the event (En) occurs are 30 (%), 20 (%), and 50 (%), respectively. Suppose there is. Hereinafter, the motion (M1) selected when the event (En) occurs is referred to as “motion (En | M1)”.

  When the loneliness parameter is 80 or more, the motion (En | M1) increases from 30 (%) to 40 (%). The increase of 10% is adjusted by reducing the selection probability of the motion (En | M2) and the motion (En | M3) by 5 (%) each. As a result, when the loneliness parameter is 80 or more, the selection probability of the motion (En | M2) decreases from 20 (%) to 15 (%), and the selection probability of the motion (En | M3) decreases from 50 (%). It falls to 45 (%). According to such a control method, the motion (En | M1) is more likely to be selected when the robot 100 is "lonely", and "emotional" affects "action selection".

FIG. 11 is a screen diagram of the first reaction report screen 180.
The user requests the server 200 to obtain a response report from the user terminal 400. The user can freely switch the display method of the reaction report. The first reaction report screen 180 in FIG. 11 shows a reaction report in which the event is associated with the feeling (emotional parameter) of the robot.

  When a response report acquisition request is received from the user terminal 400, the history management unit 252 generates a response report based on the history information 270. The history management unit 252 extracts an emotion parameter when the above-described psychological condition is satisfied.

  First, it is assumed that one of a plurality of kinds of psychological conditions is satisfied at 9:00. In the server 200, the emotion icon 184 is associated in advance for each psychological condition or each state of the emotion parameter. The icon selection unit 254 selects the emotion icon 184 according to the emotion parameter when the psychological condition is satisfied. In FIG. 11, the emotion icon 184a is selected.

  The icon selection unit 254 also selects an event icon 182 corresponding to the event when the psychological condition is satisfied. An event icon 182 is associated with each of the plurality of events in advance. The “event when the psychological condition is satisfied” may be an event that has occurred immediately before (closest to) the time when the psychological condition is satisfied, or may be an event that has occurred within a predetermined time from the time when the psychological condition is satisfied. Good. For example, when the psychological condition is satisfied at time t1, an event that has occurred during a period of 7 seconds from time (t1-10 seconds) to time (t1-3 seconds) may be displayed on the reaction report.

  In the first reaction report screen 180 of FIG. 11, an emotion icon 184a representing the "feeling" of the robot 100 when the psychological condition is established is displayed, and an event icon corresponding to the event (event type) when the psychological condition is established. 182a is also displayed. An event icon 182a illustrated in FIG. 11 corresponds to an event in which the robot 100 has met the owner. The emotion icon 184a corresponds to a happy state of the robot 100, for example, a psychological state in which both the loneliness parameter and the approval desire parameter are equal to or less than a predetermined value. By referring to the first reaction report screen 180, the user can confirm the daily life of the robot 100 and the feeling of the robot 100 that "the robot 100 was happy to meet the owner at 9:00".

  The first reaction report screen 180 allows the user to know the emotion and the event of the robot 100 by the emotion icon 184 and the event icon 182. By expressing events and the like symbolically by icons, the user can enjoy the daily life of the robot 100 like a picture diary.

  In the present embodiment, the recognition unit 212 recognizes an event based on information on an external environment, and specifies an event type corresponding to the recognized event. Next, the icon selection unit 254 selects an event icon 182 corresponding to the event type. In the reaction report, details of the event are not shown, and by simply transmitting "what happened" in the event icon 182, the privacy is not excessively disclosed by the first reaction report screen 180.

  As the emotion parameter of the robot 100, the parameter value may be displayed as it is, instead of the emotion icon 184, or the emotion of the robot 100 may be expressed by a color corresponding to the emotion. This is because the emotion parameter itself of the robot 100 is considered to be unlikely to affect the privacy of the user.

  The robot 100 selects an action while recognizing a large number of events. For this reason, showing all events in a reaction report is not preferable not only from the viewpoint of privacy protection but also from the viewpoint of visibility. Therefore, in the present embodiment, only the event when the psychological condition is satisfied is registered in the reaction report. According to such a control method, it is easy to confirm an event that may have caused a change in the mental state of the robot 100. In other words, among various events occurring around the robot 100, the situation can be checked mainly on the "event that moved the heart of the robot 100". Of the various events recognized by the robot 100, events important to (the feelings of) the robot 100 are registered in the reaction report, so that a reaction report such as a diary emphasizing the feelings of the robot 100 can be formed.

  At 9:30, an event in which the owner goes out (event icon 182b) and an event in which the owner returns home (event icon 182c) occur consecutively in a short period of time. When a plurality of events occur consecutively in a short period of time, the history management unit 252 displays the reaction report with the event icon 182b and the event icon 182c slightly shifted left and right. With such a display method, the visibility of the event icon 182 on the first reaction report screen 180 can be improved.

  The history management unit 252 sets an attention mark 186 next to an event belonging to a specific category (specific event type) defined in advance. The attention mark 186 is a highlight display for an event of interest among a plurality of types of events. The attention mark 186 makes it easier for the user to recognize important events. Important events may include visual recognition of the owner, going out or seeing off, falling objects, finding unregistered persons (suspicious persons), and the like.

  Note that a plurality of robots 100 may be introduced into one home. Each of the robots 100 has individuality due to a difference in initial settings or a difference in environment. For example, there may be “the robot 100 that is lonely vulnerable” whose loneliness parameter is likely to be high, and “calm robot 100” whose setting of the curiosity parameter is unlikely to increase. Further, the robot 100A may have a high degree of familiarity with the user (P01), while the robot 100B may have a lower degree of familiarity with the user (P01). In this case, being held by the user (P01) heals the "loneliness" of the robot 100A greatly, but in the case of the robot 100B, "loneliness" may hardly be healed by the user (P01). Absent. Also, the robot 100C that has never seen the user (P01) may rather increase the curiosity parameter when held by the user (P01). As described above, the change of the emotion parameter of each robot 100 is diversified with respect to the common event of “held by the user (P01)”. Therefore, there is a high possibility that the timing at which the psychological condition is satisfied for each robot 100, in other words, the “timing to move the heart” is different.

  Although the history information generated by each of the plurality of robots 100 having the same experience may be substantially the same, the timing when the psychological condition of the robot 100A is satisfied and the timing when the psychological condition of the robot 100B are satisfied are different. 100A and robot 100B may generate different reaction reports. For example, the robot 100A may record the event (E1) in the reaction report, but the robot 100B may not record the event (E1) in the reaction report. By referring to the reaction reports of the robots 100A and 100B, the user can recognize that the robot 100A was moved by the event (E1), but the robot 100B was not interested in the event (E1). The user can enjoy the individuality and feeling of each of the plurality of robots 100 by comparing the reaction reports from the plurality of robots 100. In addition, by providing an opportunity to feel the individuality of the robot 100 by the reaction report, it is possible to further enjoy the cultivation (relation) of the plurality of robots 100.

FIG. 12 is a screen diagram of the second reaction report screen 190.
The reaction report may display the motion selected by the robot 100 when the psychological condition is satisfied. A second reaction report screen 190 in FIG. 12 shows a reaction report in which an event is associated with a motion.

  A motion icon 192 is associated with the motion ID. The motion icon 192 may be formed as an animation for reproducing the motion of the robot 100. The history management unit 252 specifies the motion selected when the psychological condition is satisfied. The icon selection unit 254 selects a motion icon 192 corresponding to the specified motion. The “motion selected when the psychological condition is satisfied” may be the motion selected first after the psychological condition is satisfied, or the motion selected in a predetermined period based on the time when the psychological condition is satisfied. There may be. For example, when the psychological condition is satisfied at time t1, a motion selected from time (t1 + 0.5 seconds) to time (t1 + 7.5 seconds) for 7 seconds may be registered in the reaction report.

  In the second reaction report screen 190 of FIG. 12, a motion icon 192a representing the "action" of the robot 100 when the psychological condition is satisfied is displayed, and an event icon 182a corresponding to the event when the psychological condition is satisfied is also displayed. Have been. The event icon 182a corresponds to an event in which the robot 100 has met the owner. The motion icon 192a represents the movement of the robot. By referring to the second reaction report screen 190, the user can confirm the daily life of the robot 100 and the behavior of the robot 100 at that time, "The robot 100 has started to move at 9:00." .

  The user can switch between the first reaction report screen 180 and the second reaction report screen 190 by operating the user terminal 400. The robot 100 selects a motion according to the event and the emotion parameter at that time. In the robot system 300, the behavior of the robot 100 is determined by three factors, that is, the event, the emotion parameter (feeling), and the motion (behavior) that affect each other in a complicated manner. The behavior of the robot 100 has unpredictability. By referring to the first reaction report screen 180 and the second reaction report screen 190, the user can see what event occurs around the robot 100, what kind of feeling the robot 100 feels, and how the robot 100 You can grasp what action 100 has selected. According to the first reaction report screen 180 of FIG. 11 and the second reaction report screen 190 of FIG. 12, it can be inferred that the robot 100 is happy to have met the owner and approached the owner.

FIG. 13 is a data structure diagram of the user authority table 290.
The user authority table 290 is stored in the personal data storage unit 218 of the server 200. In the present embodiment, a browsing attribute is set for each user in order to control the amount of information provided by the reaction report according to the user. The viewing attribute in the present embodiment indicates how much information the user can extract from the history information, in other words, the viewing authority level. A user is identified by a user ID. A user's physical characteristics and a user ID are associated in advance, and a browsing attribute is set in advance for each user ID. The browsing attribute A1 has the highest browsing authority, and the browsing attribute A4 has the weakest browsing authority. The reaction report provided to the user (P01: A1) is richer in information than the reaction report provided to the user (P04: A4).

FIG. 14 is a data structure diagram of the event condition table 292.
The event condition table 292 is stored in the event condition storage 260 of the server 200. The event condition table 292 defines viewing conditions for each event condition. The event condition is a condition mainly defined for determining an event related to privacy. Conditions related to the user's appearance and voice, such as anger and cry, conditions related to places such as the children's room and study, conditions related to time, such as midnight and evening, and facial expressions of the user, such as angry, crying, and laughing faces. Event conditions are defined as conditions. The event condition relating to a place may be classified into an event that has occurred in an open space such as a living room, a kitchen, and an entrance, and an event that has occurred in a private space such as a children's room, a study, and a bedroom.

  For example, when a certain event X1 satisfies the event condition (C1), the event X1 is registered only in a reaction report for a user with the browsing attribute A1 or higher. Therefore, the user (P01: A1) can check the event X1 in the reaction report, but the user (P02: A2) cannot see the event X1 in the reaction report. For example, when the event condition (C1) that the event X1 (event for visually recognizing the user) visually recognizes the user (P06) in his room (such as a study) is satisfied, the user (P01: A1) The user can confirm that he / she was in his / her room by the reaction report, but the user (P02: A2) cannot know it. Privacy protection is further enhanced by controlling events that may or may not be notified for each user according to the user's browsing attributes (viewing authority levels).

FIG. 15 is a sequence diagram illustrating a process of displaying a reaction report.
The user terminal 400 transmits a response report acquisition request to the server 200 (S20). At this time, the terminal ID is also transmitted from the user terminal 400. The history management unit 252 specifies a user ID corresponding to the terminal ID. The history management unit 252 checks the browsing attribute of the user with reference to the user authority table 290 (S22). Events that can be displayed in the reaction report are narrowed down based on the viewing attributes. Next, the history management unit 252 refers to the history information and checks the establishment point of the psychological condition (S24). The history management unit 252 generates a reaction report from events, emotion parameters, motion, and the like when the psychological condition is satisfied (S26). The icon selection unit 254 selects an icon as appropriate. The response report is formed as a web page containing the icons. The communication unit 204 transmits the response report shown in FIGS. 11 and 12 to the user terminal 400 (S28). The user terminal 400 displays the reaction report on the screen (S30).

<Display of detailed information>
FIG. 16 is a sequence diagram illustrating a process of displaying details.
The history management unit 252 acquires detailed information when an event occurs. The detailed information may be any information that indicates the content and status of the event, such as a captured image (still image), a short movie, text, audio information, and temperature distribution information from a thermosensor. At least, the detailed information should just provide information other than an icon. For example, when the recognition unit 212 of the server 200 recognizes the event X2 of detecting the user (P01), the recognition unit 212 stores the captured image Y2 at that time. The robot 100 transmits the captured image to the server 200 as needed. The history management unit 252 stores the event X2 and the captured image Y2 in association with each other. This captured image Y2 becomes “detailed information” about the event X2. A short movie of a predetermined period including the time point when the user (P01) is detected, for example, about 5 seconds, may be stored as the detailed image. Alternatively, the recognizing unit 212 may generate text information that “the user (P01) has been found at the entrance”. Then, the history management unit 252 may store the text information as detailed information.

  When generating a reaction report, the history management unit 252 associates and stores the event and the icon ID of the event icon 182 added to the reaction report. The icon ID is a number that uniquely identifies the event icon 182. When the user taps any event icon 182 on the user terminal 400, the user terminal 400 transmits an icon ID corresponding to the event icon 182 to the server 200 (S40). The history management unit 252 specifies an event associated with the icon ID, and specifies detailed information associated with the event (S42).

  The emotion management unit 250 transmits the detailed information to the user terminal 400 via the communication unit 204 (S44). The user terminal 400 displays the detailed information (S46).

FIG. 17 is a screen diagram of the detailed information screen 402.
When the event icon 182 is tapped, the user terminal 400 receives the detailed information from the server 200. At this time, the user terminal 400 displays the detail window 404 on the reaction report. Detailed information is displayed in a details window 404. The detailed information may be a still image or a moving image, or may be text information. Still images and text information may be displayed simultaneously.

  The browsing authority may be set for the detailed information. For example, the event icon 182 is displayed in the reaction report for the user with the browsing attribute A2 or more for the event X3 that satisfies the event condition (C2), but the browsing attribute A1 or more is required to see the detailed information. There may be. Alternatively, in order to visually recognize the event X4 that satisfies the event condition (C4) in the reaction report, the browsing attribute A2 or more is required, but the display of the detailed information may be prohibited for such event X4. Since the detailed information is raw data that specifically indicates the event occurrence situation, it may be related to privacy. Therefore, it is considered that whether or not the detailed information can be displayed should be managed more strictly than the event icon 182.

  The detailed information may be a moving image that does not include audio, or may be a still image partially blurred. The history management unit 252 may change the texture of the still image by a known image conversion method, and may provide the user terminal 400 with detailed information as a painting-like image or a defocused image. The history management unit 252 may provide detailed information close to raw data to a user with a strong viewing right, and may provide detailed information with a higher processing degree for a user with a weaker viewing right.

<Display of atmosphere>
FIG. 18 is a screen diagram of the third reaction report screen 410.
In addition to the event, the emotion, and the motion, the history management unit 252 can record the atmosphere type (atmosphere information) in the reaction report. The atmosphere determination unit 256 classifies the external environment of the robot 100 into a plurality of types of atmosphere. Specifically, it is "fun,""buzzing,""quiet,""dark,""piripi," and the like. The icon selection unit 254 selects the atmosphere icon 194 according to the atmosphere type. The history management unit 252 records an atmosphere icon 194 (image) corresponding to the atmosphere type when the psychological condition is satisfied in the response report.

  The atmosphere determination unit 256 determines an atmosphere type based on external information obtained from the robot 100. The robot 100 acquires information about an image, a sound, a smell, and a contact by using the internal sensor 128 and transmits the information to the server 200. The atmosphere determination unit 256 determines an atmosphere type based on the atmosphere condition. Specifically, it determines whether the atmospheric condition is satisfied based on the sound pressure, whether or not a smile can be detected, a laugh, the number of touches, the number of users existing around the robot 100, and the like. For example, when a plurality of voices and a plurality of smiles are detected during a predetermined period, an atmosphere condition relating to “buzz” is satisfied, and the atmosphere determination unit 256 determines that the mood type is “buzz”. Further, when an anger is detected during a predetermined period, the atmosphere determining unit 256 determines that the atmosphere type is “piri”.

  On the third reaction report screen 410 in FIG. 18, an atmosphere icon 194 indicating the "atmosphere type" when the psychological condition is satisfied is displayed, and an emotion icon 184 corresponding to the emotion of the robot 100 at this time is also displayed. . By referring to the third reaction report screen 410, the user can check the atmosphere around the robot 100 and the feeling of the robot 100 at that time. For example, the user can know from the third reaction report screen 410 whether the robot spent a busy environment or a quiet day. A browsing attribute may be set for the atmosphere type as well as the event.

  In the present embodiment, the server 200 is described as having the atmosphere determination unit 256, but the robot 100 may have an atmosphere determination function by the atmosphere determination unit 256. In this case, the atmosphere determination unit 256 of the robot 100 notifies the server 200 of the atmosphere type (atmosphere information), the icon selection unit 254 selects the atmosphere icon 194 corresponding to the atmosphere type, and the history management unit 252 displays the atmosphere icon. What is necessary is just to generate the reaction report given 194.

<Setting rest time>
FIG. 19 is a screen diagram of the rest time setting screen 420.
The user can issue an instruction to the robot system 300 from the user terminal 400 as well as check the reaction report on the user terminal 400. In the present embodiment, the user can set a rest time (sleep time) of the robot 100 on the user terminal 400. In the rest time setting screen 420 of FIG. 19, 23:00 to 5:00 is set as the rest time. The user terminal 400 transmits the designation information of the rest time to the server 200. The operation control unit 222 of the server 200 causes the robot 100 to wait at a charging station (not shown) during the rest time.

  The rest time can be set in units of 10 minutes or longer than one hour. The user may not want the robot 100 to move around at midnight. Further, it is considered that the user does not want the robot 100 to move around even when his / her physical condition is poor. By setting the rest time by the user terminal 400, it is possible to prevent the robot 100 from disrupting the user's life.

<Summary>
The robot system 300 has been described based on the embodiment.
According to the present embodiment, the user can check the daily life of the robot 100 with the user terminal 400 even when not near the robot 100. The user can know what kind of experience, what kind of feeling, and what kind of action the robot 100 has experienced by a reaction report such as a picture diary.

  When the psychological condition is satisfied, in other words, since the reaction report is generated by focusing on events and the like when the robot 100 changes in the feeling, not only can the information amount of the reaction report be effectively suppressed, but also the robot 100 You can create a reaction report like a diary that emphasizes feelings (sensitivity). Restricting the information to be provided as the reaction report not only enhances the visibility of the reaction report, but also contributes to protecting the privacy of the user near the robot 100.

  By referring to the reaction report, the user can know what kind of feeling the robot 100 spent and how he feels. For example, when the robot 100 is having a lonely day, the user may want to come home early and meet the robot 100. In addition, even when the father goes out and the mother and the children are spending time with the robot 100 at home, the father can grasp the outline of the activity of the robot 100 by the reaction report, so that it is easy to talk about the life with the robot 100 even after returning home. It is considered to be. In addition, the user can know the state of grandfather and grandmother living far away through a reaction report (details will be described later).

  The reaction report does not directly display an event or the like, but rather expresses the daily life of the robot 100 in a funny manner using various icons. With beautiful and cute icons, users can enjoy the reaction report like a picture diary. The icon display not only makes the response report easier to understand, but also enhances the appearance of the response report and protects privacy.

  Further, information that can be provided according to the user can be finely controlled by the browsing attribute. For example, parents may be able to see many home events, but grandparents may be able to see only some of the home events. The history management unit 252 may reject a response report acquisition request from an unregistered user terminal 400.

  By confirming the detailed information, the user can confirm events around the robot 100 from outside while not being near the robot 100. For example, when the robot 100 has a “happy event”, by displaying detailed information of the happy event, the user on the go can easily share the “happy event” with the user near the robot 100. At present, it is common to send a still image, a moving image, or the like to a close person by using an SNS (Social Networking Service), but according to the present embodiment, a person who wants to transmit detailed information to be transmitted through the robot 100 It is possible to tell.

  In addition, the atmosphere around the robot 100 can be represented by an atmosphere icon 194 in the reaction report. The user who is out can know the atmosphere of the home via the reaction report. A user on the go may want to go home if the atmosphere of the home is enjoyable. If the father at work can check the reaction report on the smartphone (user terminal 400), and know the atmosphere of the "happy" home without knowing the details of the home, he can work safely. Conceivable. Alternatively, the father may ask the family what had been fun when they returned home. It can be a tool for loosely connecting fathers and homes because they can share information of "fun" even when they are not at home.

  The user can set the rest time of the robot 100 using the user terminal 400. By setting the rest time, it is possible to prevent the sleep of the user from being disturbed by the robot 100 moving around early in the morning, for example. Alternatively, there is a case where it is necessary to wake up early tomorrow, so that the robot 100 may want to wake up early in the morning. Although it is simple to set the rest time of the robot 100 from the user terminal 400, it is extremely difficult to cooperate the life pattern of the user and the life pattern of the robot 100, in other words, to realize symbiosis between the user and the robot 100. It is considered to be an effective means.

  Conventionally, a robot that selects an action based on an emotion parameter is known. Generally, it is considered that the emotion parameter is set to the robot in order to realize a biological action expression as if the robot is moving according to its own emotion. It is considered that the emotion parameter is often treated as an internal parameter that is invisible to the user. On the other hand, in the present embodiment, the user can know the emotion of the robot 100 by the reaction report. It is considered that the user will consider the feeling of the robot 100 by knowing how the robot 100 spends his daily life. When the robot 100 and the user consider each other, psychological exchange between the robot 100 and the user can be further deepened.

  Note that the present invention is not limited to the above embodiments and modified examples, and can be embodied by modifying the constituent elements without departing from the gist. Various inventions may be formed by appropriately combining a plurality of components disclosed in the above embodiments and modifications. In addition, some components may be deleted from all the components shown in the above-described embodiment and the modified examples.

  Although one robot 100, one server 200, and a plurality of external sensors 114 have been described as constituting the robot system 300, some of the functions of the robot 100 may be realized by the server 200, or the functions of the server 200. May be allocated to the robot 100. One server 200 may control a plurality of robots 100, or a plurality of servers 200 may control one or more robots 100 in cooperation.

  A third device other than the robot 100 and the server 200 may perform some of the functions. An aggregate of each function of the robot 100 and each function of the server 200 described in FIG. 7 can be globally grasped as one “robot”. How to allocate a plurality of functions necessary to implement the present invention to one or a plurality of hardware is determined in consideration of the processing capacity of each hardware, specifications required for the robot system 300, and the like. It only has to be determined.

  As described above, the “robot in a narrow sense” refers to the robot 100 that does not include the server 200, whereas the “robot in a broad sense” refers to the robot system 300. Many of the functions of the server 200 may be integrated into the robot 100 in the future.

[Modification]
<Rejection of reaction report>
The history management unit 252 may reject a response report acquisition request from the user terminal 400 existing near the robot 100. For example, when the imaging unit 158 of the robot 100 is visually recognizing the user (P01), the history management unit 252 may reject a response report acquisition request from the user terminal 400 owned by the user (P01). Specifically, when the recognition unit 212 recognizes the user (P01) from the captured image, the history management unit 252 specifies the terminal ID associated with the user (P01), and the history management unit 252 specifies the terminal ID from the user terminal 400 having the terminal ID. Reject the response report acquisition request. The reason for performing such control is to prevent the user (P01) from confirming a change in the response report at the user terminal 400 while interacting with the robot 100. If the user (P01) can confirm the change in the emotion of the robot 100 in real time while performing an action such as hitting or holding the robot 100, the robot 100 will be treated as an experimental animal. turn into. In order to prevent such handling of the robot 100 as an experimental animal, the history management unit 252 determines that a user who is not in the viewable range of the robot 100, in other words, a user who is far enough to have no direct influence on the robot 100. Only, a response report may be provided.

  It is also conceivable that the user (P01) who is the owner of the user terminal 400 (P01) acquires the response report by using another user terminal 400 (P02) instead of the user terminal 400 (P01). In this case, if the user (P02) who is the true owner of the user terminal 400 (P02) is not nearby, the robot 100 transmits a response report to the user terminal 400 (P02), and the user who is near There is a possibility that the reaction report may be viewed at (P01).

  As a countermeasure against such a loophole, first, the application software built in the user terminal 400 (P02) takes an image of the user (P01) by a camera function at the time of obtaining a reaction report, and takes an image at that time at the time of a request for obtaining a reaction report. What is necessary is just to transmit the image Q1 to the server 200. In the case of the above example, the captured image Q1 includes a face image of the user (P01).

  When the server 200 receives the response report acquisition request, the history management unit 252 also acquires the captured image Q2 from the robot 100. In the case of the above example, the captured image Q2 also includes the face image of the user (P01). The recognition unit 212 determines whether a face image of the same person as the face image included in the captured image Q1 (the user terminal 400) is also included in the captured image Q2 (the robot 100). When the same person is shown, a response report is requested from a user near the robot 100, and the history management unit 252 refuses to provide the response report to the user terminal 400 (P02). The history management unit 252 may provide a response report on condition that the same person is not shown. According to such a control method, a reaction report acquisition request from the user terminal 400 existing near the robot 100 can be more reliably rejected.

  In addition, the server 200 may include a distance measuring unit (not shown) and measure the distance from the user terminal 400 by a known method such as infrared communication. The history management unit 252 may reject a response report acquisition request from the user terminal 400 within a predetermined distance. Alternatively, the server 200 may detect the position coordinates of the robot 100 and the position coordinates of the user terminal 400 by known means such as GPS (Global Positioning System). On condition that the robot 100 is separated from the user terminal 400 by a predetermined distance or more, a response report acquisition request from the user terminal 400 may be handled.

  When the robot 100 is visually recognizing the user (P01) and receives a response report acquisition request from the user terminal 400 of the user (P01), the time when the robot 100 first visually recognizes the user (P01) is used as a reference. A response report may be provided for a predetermined period, for example, only for one minute from the time of first viewing. With such a control method, while the user (P01) is involved with the robot 100, the opportunity to confirm the emotional change of the robot 100 due to the manner of involvement in the reaction report may be limited.

  Conversely, only the user at home and the user near the robot 100 may show the response report. According to such a control method, it is possible to prevent a user in a remote place from seeing the reaction report. A user at home may operate the robot 100 or the server 200 to set so as to reject a remote reaction report acquisition request.

  Upon receiving the response report acquisition request from the user (P05), the authority confirmation unit (not shown) of the server 200 confirms whether or not the response report can be provided to the management user (P07) whose viewing permission authority has been set in advance. May be. The authority confirmation unit transmits information (user ID, name, affiliation, etc.) regarding the user (P05) to the user terminal 400 of the management user (P07). The history management unit 252 may transmit a response report to the user (P05) on condition that the authority confirmation unit receives a permission instruction from the management user (P07).

  Not only the availability of the response report but also the period for which the response report is targeted may be limited. For example, assume that the user (P01) has transmitted a response report acquisition request to the server 200. At this time, the history management unit 252 may provide the user (P01) with a reaction report for a time period (absence period) in which the user (P01) was not near the robot 100. For example, it is assumed that the user (P01) is away from the robot 100 from time t6 to time t7, and is near the robot 100 after time t8. When the user (P01) transmits a response report acquisition request from the user terminal 400 to the server 200 after time t8, the history management unit 252 generates a response report with the absence period from time t7 to time t8 as the target period. ). On the other hand, no response report is provided for the period after time t8. According to such a control method, even a user who has returned home from home can look back on the state of the home when he is not present, and can glimpse the life of the robot 100 when he is not present. However, a reaction report when the user is near the robot 100 cannot be viewed. When the user is sufficiently away from the robot 100, the history management unit 252 may provide the user with a response report including the current time, that is, a response report with real-time properties.

  The history management unit 252 may exclude a period in which the robot 100 is resting (charging) from the target of the reaction report. For example, when the robot 100 is resting from time t9 to time t10, the history management unit 252 may not generate a reaction report for the time period from time t9 to time t10. This is because it is not only irrational for the robot 100 to report on the resting life, but also because the psychological condition is not satisfied during the rest, meaningful information cannot be provided by the reaction report. Alternatively, even if the robot 100 is resting, a response report before the rest may be provided to the user from the server 200.

  The history management unit 252 may not display the emotion of the robot 100 in real time in the reaction report. For example, when a user requests a response report, information may be provided by the response report only for emotions that are more than a predetermined time before the current time, for example, 10 minutes or more.

<Record of movement history>
The history management unit 252 may record not only the time and the event but also the location where the event occurs or the movement history of the robot 100 in the history information and the reaction report. Further, the position management unit 208 may form a map based on the captured image obtained by the imaging unit 158 when the robot 100 moves. The history management unit 252 may provide the map created by the robot 100 as detailed information.

<Recording the mode>
The mode selection unit 258 selects an operation mode of the robot 100. The mode selection unit 258 changes the operation mode of the robot 100 from the normal mode to the power saving mode when the activity amount of the robot 100 per unit time becomes equal to or more than a predetermined value. The amount of activity may be determined according to power consumption per unit time or the number of times motion is selected. In the power saving mode, the types of motion selectable by the robot 100 are narrowed down. Alternatively, the movement of the robot 100 may be slowed down by limiting the movable range of the actuator and increasing the interval. In the power saving mode, the activity amount of the robot 100 per unit time is suppressed. H.258 changes the operation mode to the normal mode when the power saving mode continues for a predetermined time or more.

  The mode selection unit 258 changes the operation mode to the charging mode (rest mode) when the charging rate of the secondary battery mounted on the robot 100 becomes equal to or less than a predetermined threshold. At this time, the operation control unit 222 causes the robot 100 to be stationary at the charging station. When the charging rate becomes equal to or more than the threshold, the mode selection unit 258 changes the operation mode from the charging mode to the normal mode.

  The robot system 300 may further include a photographable mode and a photographable mode. A rotary switch (not shown) may be mounted on the horn 112 of the robot 100. By rotating a switch provided on the horn 112, the user can manually switch between the photographable mode and the photographable mode. In the photographable mode, the outside world is photographed by the photographing unit 158, and the photographed image can be stored as a part of the history information. On the other hand, in the shooting impossible mode, the robot 100 performs shooting itself for user recognition or the like, but does not save the shot image. Alternatively, the robot 100 may automatically discard the captured image after a predetermined time, for example, three seconds. When the user does not want the robot 100 to take a picture, the user can easily change the mode to the photographing impossible mode by rotating a switch provided on the horn 112.

  When a switch provided on the horn 112 is operated, the communication unit 142 transmits the set value to the server 200. The mode selection unit 258 recognizes, based on the setting value received from the robot 100, which mode is the photographing possible mode or the photographing impossible mode. Also in the photographing disabled mode, the recognition unit 212 of the server 200 receives the photographed image from the robot 100 and executes the image recognition process, but does not permanently store the photographed image.

  The history management unit 252 may record the mode of the robot 100 in the history information and the reaction report. The user can check the change history of the operation mode of the robot 100 by referring to the reaction report. It should be noted that during the period of the shooting impossible mode, no shot image remains, so that detailed information as an image cannot be provided.

<Record of regular activities>
The recognition unit 212 may detect a regular action of the user. The regular behavior includes, for example, daily exercises such as early morning exercises, “good morning” and “good night” calls to the robot 100 (greetings), tooth brushing, face washing, bathing, sitting in the study, cleaning, eating, taking a nap, etc. This is the user's action to be performed. The regular action may be registered in the server 200 in advance, or the recognition unit 212 may observe the user for a predetermined period, and when the same action is repeatedly detected in the same time zone, the action may be registered as the regular action. . Regular actions are recognized for each user. When detecting the regular action, the recognizing unit 212 recognizes that a “regular event” has occurred.

  When a regular event occurs, the history management unit 252 records the event in the history information. The history management unit 252 may record an event icon 182 corresponding to a regular event in the reaction report. Alternatively, instead of recording the regular event in the reaction report, when the regular event is not detected in the assumed time zone, the fact may be recorded in the reaction report. According to such a control method, the user C in a remote place can confirm, through the reaction report, whether the user D near the robot 100 is performing a regular action.

  At present, the number of single households in Japan is said to be more than 13 million, and it is certain that it will continue to increase in the future. In particular, the number of single households of the elderly (65 years old and over) has reached 6.2 million households, and watching around the elderly (checking safety) is an important issue. At present, children and grandchildren living in the neighborhood may visit parents who live alone on a regular basis, or check the status of parents living alone by regular contact by phone, e-mail, SNS, etc. There is also.

  However, such a safety check may be burdensome not only for the checker but also for the checker. Children are concerned about the lives of elderly parents living alone in remote areas, but it is burdensome to check daily with SNS. Older parents often refrain from contacting them because their children are busy.

  If a single elderly person (old parent) comes to live with the robot 100, it is considered that children will be able to easily confirm the safety of the old parent via the robot 100. However, from the viewpoint of protecting the privacy of the old parent, it is not desirable for the robot 100 to report to another user (such as a child) every time the daily life of the old parent. If you don't consider the privacy of the watched person, "watching" is no different from "monitoring."

  Although the history management unit 252 in the present embodiment generates a reaction report in consideration of the privacy of the user near the robot 100, the child does not want to know the details of the daily life of the old parent. Older parents may not want their children to see all of their daily lives. It may be enough for the child to be able to confirm that the old parent lives safely as usual.

  As one of the solutions to such a problem, it is conceivable to set a weak browsing authority to a viewer (child) of the reaction report. According to such a control method, the information provided to the viewer (child) is restricted, so that the privacy of the old parent can be easily protected.

  Another solution is to record the success or failure of the regular event in a response report and provide it to viewers (children). For example, a call to the robot 100 from an old parent saying “Good morning” is a regular event. The viewer can check the reaction report every day whether the old parent living with the robot 100 has performed a regular action (morning greeting). In this case, the response report may inform the viewer of only the success or failure of the regular event. The old parent can send a message saying “I still live as usual today” to children living far away simply by saying “Good morning” to the robot 100. The children can easily confirm that there is no abnormality in the old parents by knowing the presence or absence of the regular event through the user terminal 400. For the person whose safety is confirmed, it is not a burden because they can reassure the children simply by living as usual.

The viewer (child) only needs to confirm the safety of the old parent by existing communication means such as a telephone only when the regular event cannot be confirmed in the reaction report.
There is no need to notify the success or failure of the regular event by a reaction report. When the recognizing unit 212 detects the regular event, the user who watches over the known communication means, for example, the SNS by using the regular event (normal behavior). May be notified.

  In addition, the above-described viewing restriction may not be provided for the regular event. For example, even when the user greets “good night” to the resting robot 100, the robot 100 may provide a response report that a regular event has occurred.

<Management of history information>
In the present embodiment, the history management unit 252 has been described as recording all events in the history information. Since the history information may include abundant information about privacy, the history management unit 252 may limit the storage period of the history information. For example, the history information for one day may be stored, and the history management unit 252 may completely delete the old history information after one day from the history information storage unit 262.

<Prohibited attribute>
A prohibition attribute may be set for some event conditions. An event for which the prohibition attribute is set may not be recorded in the history information, may not be recorded in the response report, and may not be provided as detailed information. A plurality of types of prohibition attributes may be provided according to the prohibition level. For example, an event that satisfies the event condition of the prohibition attribute B1 may not be recorded even in the history information. An event that satisfies the event condition of the prohibition condition B2 is recorded in the history information, but may not be provided as detailed information. Although the event of the prohibition condition B3 is recorded in the history information, it may be deleted in a short time such as 30 minutes.

  Examples of the event condition for which the prohibition attribute is set include a crying, screaming (recognition of marital quarrel), a laughing face image, a sleeping face image, and the like.

<Browsing attributes>
In the present embodiment, the explanation has been given assuming that the viewing authority level is expressed by the viewing attribute. The browsing attribute is not limited to the authority regarding privacy. For example, whether or not an event can be registered in a reaction report may be determined under conditions of an external environment, such as an event that can be confirmed in a reaction report only when it occurs in the morning and an event that can be confirmed in a reaction report only when it occurs on a rainy day.

  In the present embodiment, the description has been given assuming that the browsing attribute is set for the user and the privacy protection level is controlled by the browsing attribute. In addition, in response to a response report acquisition request from a user terminal 400 located at a remote location, the event viewing authority level may be lowered. For example, a detailed response report may not be provided to a user who is far away, or detailed information may not be provided to a user who is farther than a certain distance. In this way, the privacy level may be controlled based on the distance.

  Further, the privacy level may be controlled based on time. For example, provide a detailed response report to users who send a response report acquisition request during the day, but keep the information contained in the response report conservative for users who send a response report acquisition request at night. Is also good.

  When the robot 100 is alone or, in other words, is answering while no one is around, the history management unit 252 may provide all events in a response report. When the robot 100 is in the same room as another user, the information that can provide a reaction report may be suppressed. According to such a control method, while the privacy of the user near the robot 100 is protected, a detailed situation can be reported by a reaction report when the robot 100 is answering alone.

  When the robot 100 finds a suspicious person, in other words, a person who has never seen, the history management unit 252 may release privacy protection and provide all the history information to the user terminal 400. According to such a control method, the robot 100 can function as a monitoring device only when a suspicious person enters a house. When the robot 100 recognizes a recognized user (owner) and an unregistered person at the same time, the unregistered person is likely to be a visitor. At this time, the history management unit 252 provides a response report for privacy protection. Information to be performed may be suppressed.

<Highlighted expression>
In the present embodiment, it has been described that the history management unit 252 attaches the attention mark 186 to an event of interest. In addition to this, the history management unit 252 may perform an emphasized expression by blinking the event icon 182 of the event of interest, changing the color, changing the size, or the like. Any display that prompts attention to important events, such as a balloon, may be used.

  On the first reaction report screen 180 or the like, a plurality of types of lines such as a red line (first line) and a blue line (second line) may be displayed. The red line may display the point of occurrence of a notable event like the attention mark 186, and the blue line may display the point of occurrence of all events. The blue line may display the occurrence points of all the events registered in the history information regardless of the event when the psychological condition is satisfied.

<Icon display>
In the present embodiment, the mode in which the information amount of the detailed information is changed according to the browsing attribute has been described. Similarly, the display of icons such as the event icon 182 may be changed according to the browsing attribute. For example, an event icon 182A using a photograph of the customer's face, an event icon 182B using a portrait of the customer, and a face representing the customer without specifying the customer in response to the event "the customer came" An event icon 182C that does not even identify a visitor, such as an event icon 182C using a picture of the person, is prepared.

  The history management unit 252 uses the event icon 182A for the reaction report for the user with the browsing attribute A1 or higher, the event icon 182B for the user with the browsing attribute A2, the event icon 182C for the user with the browsing attribute A3, and the browsing. The event icon 182D may be used for the user having the attribute A4. According to such a control method, privacy can be more easily protected by controlling the amount of information provided by various icons according to the browsing attribute of the user.

<Instructions to the robot 100>
In the present embodiment, a method of setting a rest time from the user terminal 400 to the robot 100 has been described. Not only the rest time, but also various commands may be transmitted from the user terminal 400 to the robot system 300. For example, even if the user terminal 400 sends a command from the user terminal 400 to the server 200, such as requesting that the user be in the living room from 8:00 to 9:00, do not go to the child's room, or come to the bedroom at 7:00. Good. The operation control unit 222 of the server 200 controls the operation of the robot 100 according to the command. In addition, watch the child once in the morning and afternoon, and pay attention to the sound of the entrance as the father returns home from 22:00 to 23:30 (enhance the microphone's sound collection sensitivity). , And so on.

  However, if the user terminal 400 controls the robot 100 too much, the autonomy of the robot 100 will be lost. Therefore, it is desirable that the command group that can be issued from the user terminal 400 to the robot 100 is limited to some extent. The operation control unit 222 may randomly select whether or not to follow a command from the user terminal 400. The operation control unit 222 may follow a command from a user with a higher degree of intimacy with a higher probability.

  The user may be able to call out to the robot 100 from the user terminal 400 even when away from home. The user reads out a message for the robot 100 from the user terminal 400. The server 200 receives the voice message from the user terminal 400 and plays it. The robot 100 recognizes the voice message by the microphone array (internal sensor 128), and recognizes that an event “a user can speak” has occurred. The emotion management unit 250 may reduce the loneliness parameter by the audio event. According to such a control method, the user can heal the loneliness of the robot 100 by speaking to the robot 100 from an outside place via the user terminal 400. Even when the user leaves the robot 100 at home and travels for a long period of time, the user can indicate consideration by calling the robot 100 from the user terminal 400.

<Others>
Instead of the emotion icon 184 corresponding to the emotion parameter, the response report may display the value of the emotion parameter itself. The history management unit 252 may record the value of the emotion parameter or the color corresponding to the value of the emotion parameter in the response report. The history management unit 252 may record the change history of the emotion parameter in the response report regardless of whether the psychological condition is satisfied.

  The event may also be represented by the event icon 182, displayed as text information, or a photograph at the time of the event occurrence may be pasted on the reaction report as described in the detailed information.

  In the present embodiment, it has been described that the response report is transmitted from the server 200 to the user terminal 400 and the response report is displayed on the user terminal 400 on the screen. A printer may be mounted on or connected to the robot 100 or the server 200, and a response report may be printed out from the printer. Alternatively, a monitor may be mounted on or connected to the robot 100 or the robot system 300, and a reaction report may be displayed on the monitor on the monitor.

  In the present embodiment, it has been described that the event, motion, atmosphere, and the like when the psychological condition is satisfied are displayed in the response report. As a modified example, without being based on the success or failure of the psychological condition, the history management unit 252 determines that all events related to predetermined event conditions, important events, regular events, or events corresponding to the user's browsing attribute, All events corresponding to predetermined registration conditions other than the above may be registered in the reaction report. The history management unit 252 may register a motion with a low selection probability (an action that is rarely performed) or an event whose detection frequency is less than or equal to a predetermined number in a predetermined period (an event that rarely occurs) in a reaction report regardless of psychological conditions. Good.

  The history management unit 252 may select one representative event for a predetermined time, for example, one hour, and register the selected event in the reaction report. The importance may be set for each event type. The history management unit 252 may select, as a representative event, an event corresponding to the most important event type among one or more events that have occurred in the unit period. Alternatively, the history management unit 252 may randomly select a representative event from one or more events that have occurred in the unit period.

  Various icons may be color-coded according to the category. For example, an event relating to the user may be represented by a blue-based event icon 182, an event relating to discovery may be represented by a red-based event icon 182, and an event relating to sound or light may be represented by a yellow-based event icon 182. You may color-code according to the event type. Alternatively, the event icons 182 may be color-coded according to whether a predetermined event condition is met.

  In the present embodiment, the description has been made assuming that the feeling, event, behavior, atmosphere, and the like of the robot 100 are registered in the reaction report. However, the present invention is not limited to this. For example, the reaction report may display only the feeling (emotion parameter) of the robot 100 in the reaction report. Even with such a reaction report, the user can infer how the robot 100 spent a day. When only the emotion of the robot 100 is registered in the reaction report, the privacy of the user can be almost completely protected. For the same reason, in the reaction report, only the atmosphere type may be displayed, or only the event may be displayed.

  In the present embodiment, the description has been made assuming that the atmosphere is represented by the atmosphere icon 194. The atmosphere may be represented by text information such as “fun” or “buzzing”, or may be represented by color. Alternatively, the change of the atmosphere may be expressed by a numerical value or a graph indicating the goodness of the atmosphere.

  The history management unit 252 may record the activity amount of the robot 100 in the history information. Further, the history management unit 252 may record the activity amount of the robot 100 in a reaction report. According to such a control method, the user can know from the reaction report whether the day is a day when the robot 100 has actively moved.

  The user may notify the robot 100 of an event scheduled to occur in the near future. For example, the user terminal 400 notifies the server 200 of a schedule such as the presence of a visitor or the start of a party. The emotion management unit 250 may change the emotion parameter as the scheduled time approaches. According to such a control method, it is possible to make the robot 100 have a sense of expectation or anxiety based on the future prediction. In addition, since the robot 100 changes the behavior characteristics based on the future prediction, the user can sense from the behavior of the robot 100 that the robot feels expectations or anxiety.

  In the present embodiment, the description has been given on the assumption that the event is an external event such as visually recognizing or holding the user. However, the present invention is not limited thereto, and an internal event such as the temperature inside the body 104 being equal to or higher than a predetermined value or the loneliness parameter being equal to or lower than 20 may be defined as an event.

  The robot 100 may include a plurality of types of emotion parameters, or may change behavior characteristics based on a single emotion parameter. As described above, when the value of the first emotion parameter (first emotion value) is equal to or greater than the value of the second emotion parameter (second emotion value), the psychological condition may be satisfied. Alternatively, the psychological condition may be established when the difference between the first emotion value and the second emotion value is equal to or greater than a predetermined threshold.

  The psychological condition may be satisfied when the emotion value of a certain emotion parameter is equal to or greater than the first threshold or equal to or less than the second threshold. Also, when the rate of change of the emotion value per unit time is equal to or greater than the third threshold or equal to or less than the fourth threshold, the psychological condition may be satisfied. For example, the curiosity parameter may spike when something new is discovered, while the approval desire parameter may only change slowly. As described above, the manner of change may be given an individuality according to the type of the emotion parameter. Therefore, various psychological conditions can be defined according to the characteristics of the emotion parameters.

  When providing detailed information as a short movie, the start and end of the short movie may be determined in accordance with the period of establishment of the psychological condition. For example, assume that the first psychological condition is satisfied at time t5, and the first psychological condition is not satisfied at time t6. At this time, the history management unit 252 may generate a short movie by cutting out a moving image from time t5 to time t6 from the moving image obtained from the camera of the robot 100.

  The reaction report may record an abnormality that has occurred in the robot 100. For example, abnormal information such as that the processor 122 has become hot, that an emergency stop has occurred, that the actuator has not moved, that the battery 118 (secondary battery) has become hot, and the like are reported to the user terminal by a reaction report. 400 may be notified.

  A plurality of robots 100 may share history information or a reaction report. The description will be made on the assumption that the user (P01) and the robots 100A and 100B exist in the same home. It is assumed that the user (P01) holds the robot 100A at the time t12, and the robot 100A establishes a psychological condition. “Hug” at this time is defined as an event (Ex). The communication unit 142 of the robot 100A requests the robot 100B to acquire a captured image (still image or moving image) for a predetermined period including the time t12. The communication unit 142 of the robot 100B downloads the captured image Q3 in the corresponding period from the captured image stored in the server 200 to the robot 100B from the server 200, and transmits the captured image Q3 to the robot 100A. The robot 100A registers the captured image Q3 in the server 200 as detailed information of the event (Ex). According to such a control method, the robot 100A can acquire “the figure in which it is held” as a captured image from the viewpoint of another robot 100B, and can provide this as detailed information. The robot 100B may transmit the celestial sphere image in the corresponding period to the robot 100A. In this case, the robot 100A may generate a two-dimensional captured image Q3 from a celestial sphere image (three-dimensional image) by a known image processing method.

  The robot 100A may request the server 200, not the robot 100B, to acquire a captured image from the robot 100B at time t12. When one server 200 manages both the robot 100A and the robot 100B collectively, the history management unit 252 of the server 200 may copy and store the image Q3 captured by the robot 100B as detailed information of the robot 100A.

  The robot 100A may specify the start time and the end time of the moving image with respect to the robot 100B to acquire the captured image Q3 as a short movie. For example, the communication unit 142 of the robot 100A specifies the start time 5 seconds before time t12 and the end time 5 seconds after time t12 as an end time, and thereby the moving image of the robot 100B for 10 seconds before and after the event (Ex) is established. May be obtained from.

  When executing any motion, the robot 100A may transmit a shooting request to the robot 100B. For example, when executing the motion (Mx), the communication unit 142 of the robot 100A transmits a shooting request to the robot 100B before execution. The robot 100B (the operation control unit 150) moves near the robot 100A when receiving the photographing request. The robot 100A (recognition unit 156) executes the motion (Mx) after recognizing that the robot 100B is nearby. According to such a control method, it is possible to capture a state when the robot 100A executes the motion (Mx) from the viewpoint of the robot 100B. The captured image at this time may be provided as a detailed image of the robot 100A. In the captured image of the motion executed by the robot 100A by the robot 100A itself, there is a possibility that it is difficult to understand how the robot 100A is moving. With the robot 100B, a photographed image can be provided to the user at a good angle. In particular, when the motion (Mx) is a rare motion that is rarely executed, an image captured by the robot B is a valuable captured image. By photographing a plurality of robots 100 with each other, various photographed images (detailed images) that cannot be obtained with only one robot 100 can be acquired.

  Reference Signs List 100 robot, 102 front wheel, 102a left wheel, 102b right wheel, 103 rear wheel, 104 body, 106 hand, 108 seating surface, 110 eyes, 112 horn, 114 external sensor, 118 battery, 120 drive mechanism, 122 processor, 124 storage device , 126 communication device, 128 internal sensor, 130 power line, 132 signal line, 134 wire, 136 data processing unit, 142 communication unit, 148 data storage unit, 150 operation control unit, 156 recognition unit, 158 imaging unit, 160 motion storage Part, 170 loneliness parameter, 172 curiosity parameter, 174 approval desire parameter, 180 first reaction report screen, 182 event icon, 184 emotion icon, 186 attention mark, 190 second reaction report screen, 192 motion Icon, 194 atmosphere icon, 200 server, 202 data processing unit, 204 communication unit, 206 data storage unit, 208 position management unit, 212 recognition unit, 218 personal data storage unit, 220 intimacy management unit, 222 operation control unit, 232 Motion storage section, 244 state management section, 250 emotion management section, 252 history management section, 254 icon selection section, 256 atmosphere determination section, 258 mode selection section, 260 event condition storage section, 262 history information storage section, 270 history information, 280 motion selection table, 290 user authority table, 292 event condition table, 300 robot system, 308 base frame, 310 body frame, 312 wheel cover, 314 skin, 316 head frame, 318 torso frame, 3 0 Yaw axis, 322 pitch axis, 324 roll axis, 325 plate, 326 actuator, 327 vent hole, 328 base plate, 329 cross link mechanism, 330 joint, 332 upper plate, 334 lower plate, 336 side plate, 342 control circuit, 370 Wheel drive mechanism, 378 rotation axis, 379 actuator, 390 opening, 400 user terminal, 402 detailed information screen, 404 detailed window, 410 third reaction report screen, 420 rest time setting screen

Claims (14)

An operation control unit for selecting a motion of the robot,
A drive mechanism for executing the motion selected by the operation control unit,
A recognition unit that recognizes the occurrence of an event;
An emotion management unit that changes one or more emotion parameters according to an event;
A history management unit that records the event occurrence history and the change history of the emotion parameter as history information in association with each other,
The operation control unit changes the behavior characteristics of the robot based on the emotion parameter,
The history management unit generates a reaction report by extracting an event when the emotion parameter satisfies a predetermined psychological condition and an emotion parameter when the psychological condition is satisfied from the history information, and provides the reaction report to a user. An autonomous behavior robot characterized by doing.   The autonomous behavior robot according to claim 1, wherein the emotion management unit determines that the psychological condition is satisfied when the order of the first emotion parameter and the second emotion parameter is reversed.   The said history management part further provides the said reaction report to a user, after also recording the motion selected by the said operation control part after the said mental condition is materialized in the said reaction report. 3. The autonomous behavior robot according to 2. An imaging unit for imaging the periphery of the robot,
A communication unit that executes a communication process with the communication terminal,
The recognition unit specifies a person from the captured image,
The history management unit, when receiving the response report acquisition request from the communication terminal of the first user, on the condition that the first user is not specified in the captured image, and instructs the communication unit The autonomous robot according to claim 1, wherein the reaction report is transmitted to a communication terminal of the first user. Further comprising an icon selection unit for selecting an icon according to the event,
The history management unit records an event when the psychological condition is satisfied in the reaction report as an icon corresponding to the event, thereby expressing an event occurrence history in the reaction report with an icon. The autonomous robot according to claim 1.   The autonomous robot according to claim 5, wherein the history management unit adds an emphasis expression in the reaction report to an icon of an event belonging to a specific category among a plurality of types of categories.   The history management unit records detailed information of an event, and when any of one or more icons recorded in the reaction report is selected by a user, the detailed information of the event corresponding to the selected icon is stored in the history management unit. The autonomous robot according to claim 5, wherein the robot is provided to a user.   The history management unit specifies a browsing attribute associated with the user in advance, and among events when the psychological condition is satisfied, only an event that matches the specified browsing attribute is provided to the reaction report to be provided to the user. The autonomous robot according to claim 1, wherein recording is performed.   The history management unit may record, in a response report to be provided to the user, only an event that satisfies a browsing authority level set in advance for the user among events when the psychological condition is satisfied. Item 2. The autonomous behavior robot according to Item 1. An atmosphere determining unit that determines whether each of a plurality of predetermined atmospheric conditions is satisfied based on external information,
The autonomous action type according to claim 1, wherein the history management unit further records, when any of the atmosphere conditions are satisfied, atmosphere information associated with the satisfied atmosphere condition in the reaction report. robot. A mode selection unit for selecting a mode of the robot,
The autonomous robot according to claim 1, wherein the history management unit further records a change history of the mode of the robot in the reaction report. The recognition unit recognizes a user's regular action as a regular event,
The autonomous robot according to claim 1, wherein the history management unit further records the success or failure of the regular event in the reaction report. A communication unit that executes a communication process with the communication terminal,
The autonomous action type according to claim 1, wherein the operation control unit, when instructed to set a rest time zone from the communication terminal, causes the robot to be stationary on an external charger in the rest time zone. robot. A function to select the motion of the robot,
A function for executing the selected motion;
A function that recognizes the occurrence of an event,
A function of changing one or more emotion parameters according to an event,
A function of associating the event occurrence history with the change history of the emotion parameter as history information,
A function for changing the behavior characteristics of the robot based on the emotion parameter,
A function of extracting an event when an emotion parameter establishes a predetermined psychological condition and an emotion parameter when the psychological condition is established from the history information to generate a reaction report, and providing the user with the reaction report; An action control program for an autonomous robot that is characterized by making it work.

Images