WO2020129993A1

WO2020129993A1 - Autonomous robot

Info

Publication number: WO2020129993A1
Application number: PCT/JP2019/049463
Authority: WO
Inventors: 要林; 秀哉南地; 司堀ノ内; 直紀沼口; 克則藁谷
Original assignee: ＧｒｏｏｖｅＸ株式会社
Priority date: 2018-12-17
Filing date: 2019-12-17
Publication date: 2020-06-25
Also published as: JP2024055866A; JPWO2020129993A1

Abstract

In order to provide a user with a sense of security while living with a robot, this autonomous robot comprises: an action control unit that selects robot motion; a drive mechanism that executes the motion selected by the action control unit; a recognition unit that determines whether or not a target object meets prescribed watch-over conditions; a mode setting unit that sets a watch-over mode for the target object when the watch-over conditions have been met; and a communications unit that sends a captured image of the target object to a prescribed communications terminal, in watch-over mode.

Description

[Replenishment based on Rule 26 02.03.2020] Robot

The present invention relates to a robot that autonomously selects an action according to an internal state or an external environment.

Humans keep pets for healing. On the other hand, many people give up their pets for various reasons, such as not being able to secure sufficient time to care for them, having no living environment for keeping pets, having allergies, and having a difficult bereavement. If there is a robot that plays the role of a pet, healing may be given to those who cannot keep the pet (PATENT DOCUMENTS 1 and 2).

Japanese Patent Laid-Open No. 2000-323219 International Publication No. 2017/169826

In recent years, robot technology has been rapidly advancing, but it has not achieved the presence as a companion like a pet. It's because I don't think a robot has free will. By observing the behavior of a pet that seems to be free will, humans feel the free will of the pet, empathize with the pet, and are healed by the pet.

The present invention is an invention completed based on the above recognition of the problems, and a first object thereof is to provide a technology for providing various senses of security to a user in living with a robot. A second purpose is to provide a technique for expressing the affection of the robot user for the user. A third object is to provide a technique for allowing a plurality of robots to act in a coordinated manner.

An autonomous action robot according to an aspect of the present invention includes a motion control unit that selects a motion of a robot, a drive mechanism that executes the motion selected by the motion control unit, and whether or not a target person satisfies a predetermined watching condition. And a communication unit that transmits a captured image of the target person to a predetermined communication terminal in the watching mode, when the watching condition is satisfied, a mode setting unit that sets the watching mode of the target person. ..

According to the present invention, it becomes easier to increase the presence of the robot.

The above-mentioned object and other objects, features and advantages will be further clarified by the preferred embodiments described below and the following drawings accompanying it.

It is a front external view of a robot. It is a side view of a robot. It is a sectional view showing roughly the structure of a robot. It is a hardware block diagram of the robot in a basic configuration. It is a functional block diagram of a robot system in a basic configuration. It is a schematic diagram for explaining an action scene when a plurality of robots watches a baby. It is a schematic diagram for explaining an action scene when a plurality of robots watches a baby. It is a schematic diagram for explaining an action scene when a plurality of robots watches a baby. It is a schematic diagram for explaining an action scene when a plurality of robots watches a baby. It is a schematic diagram for explaining an action scene when a plurality of robots watches a baby. It is a schematic diagram for explaining an action scene when a plurality of robots watches a baby. It is a schematic diagram for explaining an action scene when a plurality of robots watches a baby. It is a schematic diagram for explaining an action scene when a plurality of robots watches a baby. It is a schematic diagram for explaining an action scene when a plurality of robots watches a baby. It is a schematic diagram for explaining an action scene when a plurality of robots watches a baby. It is a schematic diagram for explaining an action scene when a plurality of robots watches a baby. It is a schematic diagram for explaining an action scene when a plurality of robots watches a baby. It is a schematic diagram for explaining an action scene when a plurality of robots watches a baby. It is a schematic diagram for explaining an action scene when the robot is on the answering machine. It is a schematic diagram for explaining an action scene when the robot is on the answering machine. It is a schematic diagram for explaining an action scene when the robot is on the answering machine. It is a schematic diagram for explaining an action scene when the robot is on the answering machine. It is a schematic diagram for explaining an action scene when the robot is on the answering machine. It is a schematic diagram for explaining an action scene when the robot is on the answering machine. It is a schematic diagram for explaining an action scene when the robot is on the answering machine. It is a schematic diagram for explaining an action scene when the robot is on the answering machine. It is a schematic diagram for explaining an action scene when the robot is on the answering machine. It is a schematic diagram for explaining an action scene when the robot is on the answering machine. It is a schematic diagram for explaining an action scene when the robot is on the answering machine. It is a schematic diagram for explaining an action scene when the robot is on the answering machine. It is a schematic diagram for explaining an action scene when the robot is on the answering machine. It is a schematic diagram for explaining an action scene when the robot is on the answering machine. It is a schematic diagram for explaining an action scene when a user who is out is remotely operating a robot. It is a schematic diagram for explaining an action scene when a user who is out is remotely operating a robot. It is a schematic diagram for explaining an action scene when a user who is out is remotely operating a robot. It is a schematic diagram for explaining an action scene when a user who is out is remotely operating a robot. It is a schematic diagram for explaining an action scene when a user who is out is remotely operating a robot. It is a schematic diagram for explaining an action scene when a user who is out is remotely operating a robot. It is a schematic diagram for explaining an action scene when a user who is out is remotely operating a robot. It is a schematic diagram for explaining an action scene when a user who is out is remotely operating a robot. It is a schematic diagram for explaining an action scene when a user who is out is remotely operating a robot. It is a schematic diagram for explaining an action scene when a user who is out is remotely operating a robot. It is a schematic diagram for explaining an action scene when a user who is out is remotely operating a robot. It is a schematic diagram for explaining an action scene when a user who is out is remotely operating a robot. It is a schematic diagram for explaining an action scene when a plurality of robots watches an elderly person. It is a schematic diagram for explaining an action scene when a plurality of robots watches an elderly person. It is a schematic diagram for explaining an action scene when a plurality of robots watches an elderly person. It is a schematic diagram for explaining an action scene when a plurality of robots watches an elderly person. It is a schematic diagram for explaining an action scene when a plurality of robots watches an elderly person. It is a schematic diagram for explaining an action scene when a plurality of robots watches an elderly person. It is a schematic diagram for explaining an action scene when a plurality of robots watches an elderly person. It is a schematic diagram for explaining an action scene when a plurality of robots watches an elderly person. It is a schematic diagram for explaining an action scene when a plurality of robots watches an elderly person. It is a schematic diagram for explaining an action scene when a plurality of robots watches an elderly person. It is a schematic diagram for explaining an action scene when a plurality of robots watches an elderly person. It is a schematic diagram for explaining an action scene when a plurality of robots watches an elderly person. It is a schematic diagram for explaining an action scene when a plurality of robots watches an elderly person. It is a schematic diagram for explaining an action scene when a plurality of robots watches an elderly person. It is a schematic diagram for explaining an action scene when a plurality of robots watches an elderly person.

The robot 100 according to the present embodiment shares a daily life with the user, sometimes considers the user, sometimes struggles to be useful to the user, and actively seeks the affection of the user, so that the robot 100 has a presence as a member of the family. Demonstrate.
Hereinafter, the basic configuration of the robot 100 will be described with reference to FIGS. 1 to 4, and then various action scenes of the robot 100 will be described.

[Basic configuration]
FIG. 1 is a diagram showing an appearance of the robot 100. 1A is a front view and FIG. 1B is a side view.
The robot 100 is an autonomous action type robot that determines an action based on an external environment and an internal state. The external environment is recognized by various sensors such as a camera and a thermo sensor 115. The internal state is quantified as various parameters expressing the emotion of the robot 100. The robot 100 sets the indoor area of the owner's home as an action range. Hereinafter, a person involved in the robot 100 is called a “user”. Of the users, the owner or administrator of the robot 100 is called the “owner”.

The body 104 of the robot 100 has a rounded shape as a whole, and includes an outer skin 314 formed of a soft and elastic material such as urethane, rubber, resin, or fiber. The robot 100 may be dressed. The total weight of the robot 100 is about 5 to 15 kilograms, and the height is about 0.5 to 1.2 meters. Due to various attributes such as appropriate weight, roundness, softness, and good feel, the effect that the user can easily hold the robot 100 and that he/she wants to hold the robot 100 is realized.

The robot 100 includes a pair of front wheels 102 (left wheel 102a, right wheel 102b) and one rear wheel 103. The front wheels 102 are driving wheels and the rear wheels 103 are driven wheels. The front wheels 102 do not have a steering mechanism, but the rotation speed and rotation direction of the left and right wheels can be individually controlled. The rear wheel 103 is a caster and is rotatable to move the robot 100 back and forth and left and right. The rear wheel 103 may be an omni wheel. By making the rotation speed of the right wheel 102b larger than that of the left wheel 102a, the robot 100 can turn left or rotate counterclockwise. By making the rotation speed of the left wheel 102a larger than that of the right wheel 102b, the robot 100 can turn right or rotate clockwise.

The front wheel 102 and the rear wheel 103 can be completely housed in the body 104 by the drive mechanism (rotating mechanism, link mechanism). A pair of left and right covers 312 is provided on the lower half of the body 104. The cover 312 is made of a flexible and elastic resin material (rubber, silicone rubber, or the like), constitutes a soft body, and can accommodate the front wheel 102. The cover 312 is formed with a slit 313 (opening) that opens from the side surface to the front surface, and the front wheel 102 can be advanced through the slit 313 and exposed to the outside.

Most of the wheels are hidden by the body 104 even when the vehicle is running, but when the wheels are completely housed in the body 104, the robot 100 cannot move. That is, the body 104 descends and sits on the floor F as the wheels are retracted. In this seated state, a flat seating surface 108 (ground contact bottom surface) formed on the bottom of the body 104 contacts the floor surface F.

The robot 100 has two arms 106. Although there is a hand at the tip of the arm 106, it does not have a function of grasping an object. The arm 106 can perform simple operations such as raising, bending, waving, and vibrating by driving an actuator described later. The two arms 106 can be individually controlled.

A face area 116 is exposed in front of the head of the robot 100. The face area 116 is provided with two eyes 110. The eye 110 is a device capable of displaying an image with a liquid crystal element or an organic EL element, and expressing a line of sight or a facial expression by moving a pupil or an eyelid displayed as an image. A nose 109 is provided in the center of the face area 116. The nose 109 is provided with an analog stick, and can detect the pushing direction in addition to all the directions of up, down, left and right. Further, the robot 100 is provided with a plurality of touch sensors, and a user's touch can be detected on almost the entire area of the robot 100, such as the head, torso, buttocks, and arms. The robot 100 is equipped with various sensors such as a microphone array and an ultrasonic sensor capable of specifying the sound source direction. It also has a built-in speaker and can emit simple voice.

A horn 112 is attached to the head of the robot 100. A omnidirectional camera 113 is attached to the horn 112 so that the entire upper portion of the robot 100 can be imaged at one time. The horn 112 also has a built-in thermo sensor 115 (thermo camera). Further, the horn 112 is provided with a plurality of modules (not shown) for performing communication using infrared rays, and these modules are annularly installed toward the surroundings. Therefore, the robot 100 can perform infrared communication while recognizing the direction. Further, the horn 112 is provided with a switch for emergency stop, and the user can perform an emergency stop of the robot 100 by pulling out the horn 112.

FIG. 2 is a sectional view schematically showing the structure of the robot 100.
The body 104 includes a main body frame 310, a pair of arms 106, a pair of covers 312, and an outer cover 314. The body frame 310 includes a head frame 316 and a body frame 318. The head frame 316 has a hollow hemispherical shape and forms the head skeleton of the robot 100. The body frame 318 has a rectangular tube shape and forms a body skeleton of the robot 100. The lower end of the body frame 318 is fixed to the lower plate 334. The head frame 316 is connected to the body frame 318 via the connection mechanism 330.

The body frame 318 constitutes the axis of the body 104. The body frame 318 is configured by fixing a pair of left and right side plates 336 to the lower plate 334, and supports the pair of arms 106 and the internal mechanism. The battery 118, the control circuit 342, various actuators, and the like are housed inside the body frame 318. The bottom surface of the lower plate 334 forms the seating surface 108.

The body frame 318 has an upper plate 332 on its upper part. A cylindrical support portion 319 having a bottom is fixed to the upper plate 332. The upper plate 332, the lower plate 334, the pair of side plates 336, and the support portion 319 form a body frame 318. The outer diameter of the support portion 319 is smaller than the distance between the left and right side plates 336. The pair of arms 106 is integrally assembled with the annular member 340 to form an arm unit 350. The annular member 340 has an annular shape, and the pair of arms 106 are attached so as to radially separate the center line thereof. The annular member 340 is coaxially inserted into the support portion 319 and placed on the upper end surfaces of the pair of side plates 336. The arm unit 350 is supported by the body frame 318 from below.

The head frame 316 has a yaw axis 321, a pitch axis 322, and a roll axis 323. The head frame 316 swings around the yaw axis 321 to achieve a swinging motion, and the swinging around the pitch shaft 322 achieves a nod motion, a look-up motion and a look-down motion around the roll shaft 323. The operation of tilting the neck to the left and right is realized by the rotation (rolling). The position and angle of each axis in the three-dimensional space may change according to the driving mode of the connection mechanism 330. The connection mechanism 330 includes a link mechanism and is driven by a plurality of motors installed on the body frame 318.

The body frame 318 houses the wheel drive mechanism 370. The wheel drive mechanism 370 includes a front wheel drive mechanism and a rear wheel drive mechanism that move the front wheel 102 and the rear wheel 103 into and out of the body 104, respectively. The front wheels 102 and the rear wheels 103 function as a “moving mechanism” that moves the robot 100. The front wheel 102 has a direct drive motor in the center thereof. Therefore, the left wheel 102a and the right wheel 102b can be driven individually. The front wheel 102 is rotatably supported by the wheel cover 105, and the wheel cover 105 is rotatably supported by the body frame 318.

The pair of covers 312 is provided so as to cover the body frame 318 from the left and right, and has a smooth curved surface shape so that the outline of the body 104 is rounded. A closed space is formed between the body frame 318 and the cover 312, and the closed space is a storage space S for the front wheels 102. The rear wheel 103 is housed in a housing space provided in the lower rear part of the body frame 318.

The outer skin 314 covers the body frame 310 and the pair of arms 106 from the outside. The outer cover 314 has a thickness that allows a person to feel elasticity, and is formed of a stretchable material such as urethane sponge. As a result, when the user holds the robot 100, he or she can feel appropriate softness and take a natural skinship like a human being makes a pet. The outer cover 314 is attached to the main body frame 310 in such a manner that the cover 312 is exposed. An opening 390 is provided at the upper end of the outer cover 314. The opening 390 is inserted through the horn 112.

A touch sensor is arranged between the body frame 310 and the outer cover 314. A touch sensor is embedded in the cover 312. Each of these touch sensors is a capacitance sensor and detects a touch in almost the entire area of the robot 100. The touch sensor may be embedded in the outer cover 314 or may be provided inside the main body frame 310.

The arm 106 has a first joint 352 and a second joint 354, and an arm 356 between both joints and a hand 358 at the tip of the second joint 354. The first joint 352 corresponds to the shoulder joint, and the second joint 354 corresponds to the wrist joint. A motor is provided in each joint to drive the arm 356 and the hand 358, respectively. The drive mechanism for driving the arm 106 includes these motors and their drive circuit 344.

FIG. 3 is a hardware configuration diagram of the robot 100.
The robot 100 includes an internal sensor 128, a communication device 126, a storage device 124, a processor 122, a drive mechanism 120, and a battery 118. The drive mechanism 120 includes the connection mechanism 330 and the wheel drive mechanism 370 described above. The processor 122 and the storage device 124 are included in the control circuit 342. Each unit is connected to each other by a power supply line 130 and a signal line 132. The battery 118 supplies power to each unit via the power supply line 130. Each unit sends and receives a control signal via a signal line 132. The battery 118 is a lithium-ion secondary battery and is a power source of the robot 100.

The internal sensor 128 is an assembly of various sensors built in the robot 100. Specifically, it is a camera, a microphone array, a distance measuring sensor (infrared sensor), a thermo sensor 115, a touch sensor, an acceleration sensor, an atmospheric pressure sensor, an odor sensor, or the like. The touch sensor corresponds to most of the area of the body 104 and detects a user's touch based on a change in capacitance. The odor sensor is a known sensor that applies the principle that electric resistance changes due to adsorption of molecules that are the origin of odor.

The communication device 126 is a communication module that performs wireless communication with various external devices. The storage device 124 includes a non-volatile memory and a volatile memory, and stores a computer program and various setting information. The processor 122 is a means for executing a computer program. The drive mechanism 120 includes a plurality of actuators. In addition to this, a display and speakers are also installed.

The drive mechanism 120 mainly controls the wheels and the head. The drive mechanism 120 can change the moving direction and moving speed of the robot 100, and can also move the wheels up and down. When the wheels are lifted, the wheels are completely stored in the body 104, and the robot 100 comes into contact with the floor surface F at the seating surface 108 and becomes seated. The drive mechanism 120 also controls the arm 106.

FIG. 4 is a functional block diagram of the robot system 300.
The robot system 300 includes a robot 100, a server 200, and a plurality of external sensors 114. Each constituent element of the robot 100 and the server 200 includes a computing unit such as a CPU (Central Processing Unit) and various coprocessors, a storage device such as a memory and a storage, hardware including a wired or wireless communication line connecting them, and a storage unit. It is realized by software stored in the device and supplying a processing instruction to the arithmetic unit. The computer program may be configured by a device driver, an operating system, various application programs located in their upper layers, and a library that provides common functions to these programs. Each block described below is not a hardware-based configuration but a function-based block.
Some of the functions of the robot 100 may be realized by the server 200, and some or all of the functions of the server 200 may be realized by the robot 100.

A plurality of external sensors 114 are installed in advance in the house. The server 200 manages the external sensor 114 and provides the detection value acquired by the external sensor 114 to the robot 100 as needed. The robot 100 determines a basic action based on the information obtained from the internal sensor 128 and the plurality of external sensors 114. The external sensor 114 is for reinforcing the sensory organs of the robot 100, and the server 200 is for reinforcing the processing capacity of the robot 100. The communication device 126 of the robot 100 may periodically communicate with the server 200, and the server 200 may be responsible for the process of identifying the position of the robot 100 by the external sensor 114 (see also Patent Document 2).

(Server 200)
The server 200 includes a communication unit 204, a data processing unit 202 and a data storage unit 206.
The communication unit 204 is in charge of communication processing with the external sensor 114 and the robot 100. The data storage unit 206 stores various data. The data processing unit 202 executes various processes based on the data acquired by the communication unit 204 and the data stored in the data storage unit 206. The data processing unit 202 also functions as an interface for the communication unit 204 and the data storage unit 206.

The data storage unit 206 includes a motion storage unit 232 and a personal data storage unit 218.
The robot 100 has a plurality of motion patterns (motions). Various motions such as swaying the arm 106, approaching the owner while meandering, and staring at the owner with his/her neck bent are defined.

The motion storage unit 232 stores a “motion file” that defines the motion control content. Each motion is identified by a motion ID. The motion file is also downloaded to the motion storage unit 160 of the robot 100. Which motion is executed may be determined by the server 200 or the robot 100.

Most of the motions of the robot 100 are configured as compound motions including a plurality of unit motions. For example, when the robot 100 approaches the owner, it may be expressed as a combination of a unit motion of turning toward the owner, a unit motion of approaching while raising a hand, a unit motion of approaching while shaking the body, and a unit motion of sitting while raising both hands. .. Such a combination of four motions realizes a motion of “approaching the owner, raising his hand on the way, and finally shaking his body to sit down”. In the motion file, the rotation angle, angular velocity, etc. of the actuator provided in the robot 100 are defined in association with the time axis. According to the motion file (actuator control information), various motions are expressed by controlling each actuator over time.

The transition time when changing from the previous unit motion to the next unit motion is called "interval". The interval may be defined according to the time required to change the unit motion and the content of the motion. The length of the interval is adjustable.
Hereinafter, the settings relating to the behavior control of the robot 100, such as when and which motion to select, output adjustment of each actuator in realizing the motion, are collectively referred to as “action characteristics”. The behavior characteristic of the robot 100 is defined by a motion selection algorithm, a motion selection probability, a motion file, and the like.

The motion storage unit 232 stores, in addition to motion files, a motion selection table that defines motions to be executed when various events occur. In the motion selection table, one or more motions and their selection probabilities are associated with an event.

The personal data storage unit 218 stores user information. Specifically, master information indicating intimacy with the user and physical/behavioral characteristics of the user is stored. Other attribute information such as age and gender may be stored.

The robot 100 has an internal parameter called intimacy for each user. When the robot 100 recognizes an action that is favorable to itself, such as hugging itself or calling out, the intimacy with the user increases. The degree of intimacy with a user who is not related to the robot 100, a user who works violently, or a user who rarely encounters is low.

The data processing unit 202 includes a position management unit 208, a recognition unit 212, an operation control unit 222, an intimacy management unit 220, and a state management unit 244.
The position management unit 208 specifies the position coordinates of the robot 100. The state management unit 244 manages various internal parameters such as the charging rate, the internal temperature, and various physical states such as the processing load of the processor 122. In addition, the state management unit 244 manages various emotion parameters indicating the emotion (loneliness, curiosity, desire for approval, etc.) of the robot 100. These emotional parameters are always fluctuating. The movement target point of the robot 100 changes according to the emotion parameter. For example, when loneliness is increasing, the robot 100 sets the place where the user is as the movement target point.

-Emotional parameters change over time. In addition, various emotion parameters also change due to a response action described later. For example, the emotional parameter indicating loneliness decreases when the owner "hugs", and the emotional parameter indicating loneliness gradually increases when the owner is not visually recognized for a long time.

The recognition unit 212 recognizes the external environment. The recognition of the external environment includes various recognitions such as recognition of weather and season based on temperature and humidity, and recognition of shade (safety zone) based on light intensity and temperature. The recognition unit 156 of the robot 100 acquires various environmental information by the internal sensor 128, performs primary processing on the environmental information, and transfers the environmental information to the recognition unit 212 of the server 200.

Specifically, the recognition unit 156 of the robot 100 extracts a moving object, in particular, an image region corresponding to a person or an animal from the image, and a feature indicating a physical feature or a behavioral feature of the moving object from the extracted image region. A "feature vector" is extracted as a set of quantities. The feature vector component (feature amount) is a numerical value that quantifies various physical and behavioral features. For example, the width of the human eye is digitized in the range of 0 to 1 to form one feature vector component. The method of extracting the feature vector from the captured image of a person is an application of a known face recognition technique. The robot 100 transmits the feature vector to the server 200.

The recognition unit 212 of the server 200 is imaged by comparing the feature vector extracted from the image captured by the built-in camera of the robot 100 with the feature vector of the user (cluster) registered in the personal data storage unit 218 in advance. It is determined which person the user corresponds to (user identification process). Further, the recognition unit 212 estimates the emotion of the user by recognizing the facial expression of the user as an image. The recognition unit 212 also performs user identification processing on moving objects other than people, such as cats and dogs that are pets.

The recognition unit 212 recognizes various kinds of response actions performed on the robot 100 and classifies them into pleasant and unpleasant actions. The recognition unit 212 also recognizes the owner's response to the action of the robot 100, and classifies the action into an affirmative/negative response.
The pleasant/unpleasant behavior is determined depending on whether the user's behavior is pleasant or uncomfortable as a living thing. For example, hugging is a pleasant act for the robot 100, and being kicked is an unpleasant act for the robot 100. The affirmative/negative reaction is determined by whether the user's response action indicates the user's pleasant or unpleasant feeling. Hugging is an affirmative reaction indicating the user's pleasant feeling, and being kicked is a negative reaction indicating the user's unpleasant feeling.

The operation control unit 222 of the server 200 cooperates with the operation control unit 150 of the robot 100 to determine the motion of the robot 100. The operation control unit 222 of the server 200 creates a movement target point of the robot 100 and a movement route therefor. The operation control unit 222 may create a plurality of movement routes and select any one of the movement routes.

The motion control unit 222 selects a motion of the robot 100 from a plurality of motions in the motion storage unit 232. A selection probability is associated with each motion for each situation. For example, a selection method is defined in which when the owner makes a pleasant action, the motion A is executed with a probability of 20%, and when the temperature is 30 degrees or more, the motion B is executed with a probability of 5%. ..

The intimacy degree management unit 220 manages the intimacy degree for each user. As described above, the degree of intimacy is registered in the personal data storage unit 218 as a part of personal data. When the pleasant behavior is detected, the intimacy degree management unit 220 increases the intimacy degree with respect to the owner. The intimacy decreases when an offensive behavior is detected. Moreover, the intimacy of owners who have not been visually recognized for a long period of time gradually decreases.

(Robot 100)
The robot 100 includes a communication unit 142, a data processing unit 136, a data storage unit 148, an internal sensor 128, and a drive mechanism 120.
The communication unit 142 corresponds to the communication device 126 (see FIG. 3) and is in charge of communication processing with the external sensor 114, the server 200, and the other robot 100. The data storage unit 148 stores various data. The data storage unit 148 corresponds to the storage device 124 (see FIG. 3). The data processing unit 136 executes various processes based on the data acquired by the communication unit 142 and the data stored in the data storage unit 148. The data processing unit 136 corresponds to the processor 122 and a computer program executed by the processor 122. The data processing unit 136 also functions as an interface for the communication unit 142, the internal sensor 128, the drive mechanism 120, and the data storage unit 148.

The data storage unit 148 includes a motion storage unit 160 that defines various motions of the robot 100.
Various motion files are downloaded from the motion storage unit 232 of the server 200 to the motion storage unit 160 of the robot 100. The motion is identified by the motion ID. A state in which the front wheel 102 is stowed, the arm 106 is lifted, the two front wheels 102 are reversely rotated, or only one of the front wheels 102 is rotated to rotate the robot 100, and the front wheel 102 is stored. In order to express various motions such as trembling by rotating the front wheel 102, stopping and turning when leaving the user, the operation timing, operation time, operation direction, etc. of various actuators (drive mechanism 120) are It is defined in time series in the motion file.
Various data may be downloaded from the personal data storage unit 218 to the data storage unit 148.

The data processing unit 136 includes a recognition unit 156 and an operation control unit 150.
The operation control unit 150 of the robot 100 cooperates with the operation control unit 222 of the server 200 to determine the motion of the robot 100. The server 200 may determine some motions, and the robot 100 may determine other motions. Although the robot 100 determines the motion, the server 200 may determine the motion when the processing load of the robot 100 is high. The base motion may be determined in the server 200 and the additional motion may be determined in the robot 100. How the server 200 and the robot 100 share the motion determination process may be designed according to the specifications of the robot system 300.

The operation control unit 150 of the robot 100 instructs the drive mechanism 120 to execute the selected motion. The drive mechanism 120 controls each actuator according to the motion file.

The motion control unit 150 can also perform a motion of lifting both arms 106 as a gesture of "hugging" when a user with high intimacy is nearby, and when the user gets tired of "hugging", the left and right front wheels 102 are moved. By alternately repeating the reverse rotation and the stop while the robot is housed, it is possible to express a motion to hate the hug. The drive mechanism 120 drives the front wheel 102, the arm 106, and the neck (head frame 316) in accordance with an instruction from the operation control unit 150 to cause the robot 100 to express various motions.

The recognition unit 156 of the robot 100 interprets external information obtained from the internal sensor 128. The recognition unit 156 can perform visual recognition (visual part), odor recognition (olfactory part), sound recognition (auditory part), and tactile recognition (tactile part).

The recognition unit 156 extracts the feature vector from the captured image of the moving object. As described above, the feature vector is a set of parameters (feature amounts) indicating the physical features and behavioral features of the moving object. When a moving object is detected, physical characteristics and behavioral characteristics are extracted from an odor sensor, a built-in sound collecting microphone, a temperature sensor, and the like. These features are also quantified and become feature vector components. The recognition unit 156 identifies the user from the feature vector based on a known technique described in Patent Document 2 or the like.

Of a series of recognition processing including detection, analysis, and determination, the recognition unit 156 of the robot 100 selects or extracts information necessary for recognition, and interpretation processing such as determination is executed by the recognition unit 212 of the server 200. .. The recognition processing may be performed only by the recognition unit 212 of the server 200, or may be performed only by the recognition unit 156 of the robot 100. As described above, both sides perform the above-described recognition processing while sharing roles. Good.

When a strong impact is applied to the robot 100, the recognition unit 156 recognizes this by the touch sensor and the acceleration sensor, and the recognition unit 212 of the server 200 recognizes that “a violent act” is performed by a user in the vicinity. When the user holds the horn 112 and lifts the robot 100, it may be recognized as a violent act. When the user facing the robot 100 utters a voice in a specific sound volume region and a specific frequency band, the recognition unit 212 of the server 200 may recognize that a “calling action” has been performed on itself. Further, when a temperature around the body temperature is detected, it is recognized that the "contact action" has been performed by the user, and when an upward acceleration is detected in the state of contact recognition, it is recognized that the "hugging" has been performed. Physical contact may be sensed when the user lifts the body 104, or the hug may be recognized when the load applied to the front wheel 102 is reduced.
In summary, the robot 100 acquires the action of the user as physical information by the internal sensor 128, and the recognition unit 212 of the server 200 determines the comfort/discomfort. The recognition unit 212 of the server 200 also executes a user identification process based on the feature vector.

The recognition unit 212 of the server 200 recognizes various responses of the user to the robot 100. Some typical response actions among various response actions are associated with pleasantness or discomfort, affirmation or denial. In general, most of the pleasant behaviors are affirmative reactions, and most of the offensive behaviors are negative responses. Pleasure/discomfort is related to intimacy, and positive/negative reactions influence behavior selection of the robot 100.

The intimacy degree management unit 220 of the server 200 changes the intimacy degree with respect to the user according to the response action recognized by the recognition unit 156. In principle, the degree of intimacy with respect to the user who has performed a pleasant act increases, and the degree of intimacy with respect to the user who has performed an unpleasant act decreases.

Each function of the server 200 is realized by loading a program for realizing the function into the memory and instantiating (instantiating) the program. Various processing by the robot 100 is supplemented by the processing capacity of the server 200. The server 200 can be used as a resource of the robot 100. How to use the resources of the server 200 is dynamically determined according to a request from the robot 100. For example, in the robot 100, when it is necessary to continuously generate complex motions according to detection values from a large number of touch sensors, the processing of the processor 122 in the robot 100 is preferentially assigned to motion selection/generation, The recognition unit 212 of the server 200 may perform the processing for image recognition of the surrounding situation. In this way, various processes of the robot system 300 can be distributed between the robot 100 and the server 200.

A single server 200 can also control multiple robots 100. In this case, each function of the server 200 is materialized independently for each robot 100. For example, the server 200 may prepare the recognition unit 212 for the robot 100B separately from the recognition unit 212 (instance object) for the robot 100A.

Given the above basic configuration, the implementation of the robot 100 in the present embodiment will be described next, focusing on the features and purposes of this implementation, and the differences from the basic configuration.

[SLAM]
The robot 100 of the present embodiment acquires a large number of captured images (still images) by periodically capturing an image of the surroundings with the omnidirectional camera 113. The robot 100 forms a memory (hereinafter referred to as “image memory”) based on the captured image.

Image memory is a collection of multiple keyframes. The key frame is distribution information of feature points (feature amount) in the captured image. The robot 100 of the present embodiment uses a graph-based SLAM (Simultaneous Localization and Mapping) technology using image features, more specifically, a SLAM technique based on ORB (Oriented FAST and Rotated BRIEF) features to form keyframes. It is formed (see Patent Document 3).

The robot 100 periodically forms a key frame while moving to form an aggregate of key frames, in other words, an image memory as an image feature distribution. The robot 100 estimates the current point by comparing the key frame acquired at the current point with a large number of key frames already held. That is, the robot 100 performs “spatial recognition” by comparing the captured image that is actually visually recognized with the captured image (memory) that is visually recognized once, and matching the present situation with the past memory. The image memory formed as a set of feature points is a so-called map. The robot 100 updates the map while moving while estimating the current position.

The basic configuration of the robot 100 is premised on recognizing the position by the external sensor 114 instead of the key frame. The robot 100 of the present embodiment will be described as recognizing a place based only on a key frame.

The robot 100 in this embodiment includes a “mode setting unit” for setting various modes.

<Baby watching>
5 to 7 are schematic diagrams for explaining action scenes when a plurality of robots 100 watch a baby.
First, in the children's room, a baby (hereinafter referred to as the “subject”) who is the target of watching in the bouncer is sleeping (Fig. 5A).
There are two robots 100 in this house. The two robots 100 recognize the presence and position of a baby (infant) through image recognition. The two robots 100 may share the baby's position through mutual communication. The two robots 100 determine the gazing point of each robot from the position of the baby. Through the mutual communication, the two robots 100 correct the gazing points so that the gazing points of the two robots are the same or within a predetermined range. The two robots 100 control the movements and parts so that the heads are directed to the respective determined gaze points, and thus an operation in which the two robots 100 look into the baby is realized (FIG. 5B). ).
One of the two robots 100, 100A, keeps watching over the baby by directing the head of the robot 100A toward the baby's presence direction according to the position of the baby. The robot 100A keeps a constant distance from the baby so as not to get near the baby. The other robot 100B divides the role through communication with the robot 100A, and after a while, performs an operation in which watching is stopped and a play is started (FIG. 5C).
The mother is cooking in the kitchen, leaving the robots 100 to watch over the baby (Fig. 5D).

Mom has a smartphone (communication terminal) in the kitchen (Fig. 6A). The robot 100A (during watching) captures an image of the baby with the omnidirectional camera 113 and continues to send the captured image to the smartphone as a live image. On a smartphone, the area in which the baby is reflected is displayed in the entire sky image. At this time, the image corrected so that the distortion of the whole sky image is displayed on a plane may be displayed on the smartphone.

Suddenly, the baby begins to cry (Fig. 6B).
100B who was playing approaches the baby based on the position of the baby when hearing the baby crying (collecting the voice) through the microphone (FIG. 6C). The robot 100B executes an interference motion. The robot 100B executes the interference motion when at least one of the condition that a voice such as a cry is collected and the condition that the baby is in a specific state such as crying by image analysis are satisfied. To do. The interfering motion is a motion for awakening such as outputting a predetermined sound that catches the baby's attention, outputting a sound like a lullaby, shaking the arm 106, shaking the head or body, and shaking the bouncer. is there. Distract the baby by the interference motion. In addition, by performing the interference motion, it is possible to give a third party the impression that the robot 100B is struggling to manage the crying baby. Instead of or in addition to the robot B, the robot A may perform the interference motion.
A live image relayed from the robot 100A shows the baby crying on his mother's smartphone (Fig. 6D).
The videos before and after the baby starts crying (videos from a predetermined time before the baby begins to cry to a predetermined time after the baby starts crying) may be made permanent by being stored in the HDD or the like.

The mother was surprised to grab the smartphone (Fig. 7A).
The mother hurriedly went to the nursery and rushed to the baby (Fig. 7B).
The mother holds the baby (Fig. 7C).
The robots 100 are beside the mother and the baby and stare at them. The two robots recognize the respective positions of the mother and baby by image analysis, and set the gazing point from the respective positions. The two robots 100 share the gazing point via communication and correct the mutual gazing points so that they are within the same range or within a predetermined range as necessary. The two robots 100 perform an action of directing their heads to the respective gazing points. As a result, when the two robots 100 line up and look at the mother and the baby, it is possible to give the mother the impression that the robots 100 are anxious about the baby.
The baby hugged by the mother falls asleep again (Fig. 7D).
When the robots 100 confirm that the baby fell asleep, they leave the bouncer (FIG. 7E).

The robot 100 may search for the baby in order to start the above-mentioned watching action when manually set by the mother to the “watching mode” via the input switch or the smartphone. When the robot 100 detects that the baby is in the bouncer, the mode setting unit may automatically set the watching mode. The robot 100 may automatically shift to the watching mode when an infant is detected by image recognition and there is no guardian near the infant. The case where there is no guardian near the baby may be, for example, a case where an image of a person of a predetermined age or more is not detected, or a case where an image of a person associated with the baby is not detected. When a plurality of robots 100 are in the same room, all the robots 100 may be set to the watching mode, or only some of the robots 100 may be set to the watching mode.

The robot 100 may shift to the watching mode as a watching condition that "baby" is detected in the captured image and no other user or a specific user such as a mother or father is detected.

The robot 100 being watched may limit the action range within a range in which the baby being watched can be visually recognized. Alternatively, the robot 100 may not leave the room with the baby while watching. At least, it is desirable that the robot 100 keep an eye on the baby (for example, always catch the baby in the angle of view of the camera) while watching. The robot 100 pays attention to the state of the target person by using sensors that measure the external environment, such as the omnidirectional camera 113, a microphone, and a temperature sensor. When the baby moves around, the robot 100 recognizes the position of the baby using a sensor for measuring the external environment such as the omnidirectional camera 113, a microphone, and a temperature sensor so that the robot 100 can turn itself toward the baby. It may be controlled to. Further, the robot 100 may reduce the number of places where the baby comes into contact with the baby, for example, by accommodating the wheels when the baby comes within a predetermined distance.

The robot 100 executes a motion for positively engaging with the baby when a predetermined warning condition is satisfied. Alternatively, the “warning information” may be transmitted to another communication terminal such as a mother's smartphone. The alert condition can be set arbitrarily, for example, when a baby (target person to be watched) crying, when approaching stairs, when trying to get out, when messing with a small object, when falling, etc. , There may be situations where the baby is at risk. The motion for actively engaging may be the above-described interference motion.

When the robot 100A shifts to the watching mode, the robot 100A may notify the robot 100B of the shift to the watching mode. When receiving the notification, the robot 100B may also shift to the watching mode, or may approach the positions of the robot 100A and the baby. By gathering a plurality of robots 100 near a baby, it is possible to express an action as if the robots 100 care about the baby.

At this time, the robot 100 suppresses the operation amount of the drive mechanism 120 (in particular, a mechanism relating to movement and posture adjustment) more than in the normal mode so that the baby does not wake up due to the operation sound of the robot 100, It operates quietly so as not to make operation noise. If the baby enters the watch mode while he is awake, Robot B may perform certain motions on the baby's side that may be of interest to the baby, such as running around a bouncer or dancing. Good. For example, it can be determined that the baby is awake if the condition that the baby's voice is detected by voice analysis, or the condition that the baby has detected an image with the eyes open by image analysis is satisfied. No such motion is performed when the baby is sleeping. The robot B executes an appropriate motion according to the state of the watching target person. A baby is also considered to be at ease if many robots 100 gather around him.

The robot 100 may shift to the watching mode when it receives a specific voice command such as “Watch your baby” from a specific user such as a mother via a microphone or the like. At the start of watching, the robot 100 may perform a confirmation action such as circling around the baby, facing the baby, or pointing the hand 358 to the baby based on the position of the baby detected by image analysis or the like. The mother (instructor) can determine whether or not the robot 100 is mistaken for the person being watched by the confirmation action. It is considered that the mother utters a positive reply such as "I left it to you" if correct, and a negative reply such as "No" if wrong. Therefore, the robot 100 may determine whether or not the target of watching is correct based on the reply of the mother collected by the microphone after the confirmation action.

While watching, the robot 100 keeps the line of sight of the robot 100 at the baby, such as looking into the baby. The baby is relieved to believe that he is being watched by the robot 100. By seeing the robot 100 healthily watching over the baby, the mother can provoke affection and trust for the robot 100. The watching action also has a meaning as an appeal to the user (witness) that "the robot 100 is working hard".

As described above, the robot 100B may freely play while watching the robot 100A. However, the action range of the robot 100B is limited to the range in which the baby or the robot 100A can be visually recognized. The robot 100A may send a live image of the baby up (close-up image) to the smartphone, and the robot 100B may send the live image of the robot 100A watching the baby (wide-angle image) to the smartphone. It is desirable that the robot 100B further limit the range of action so as not to interfere with the shooting of the baby by the robot 100A. For example, the robot 100B may identify the positions of the baby and the robot 100A, and act so as not to be covered on the straight line connecting the baby and the robot 100A. Further, when the robot 100A cannot recognize the baby's figure by the image recognition of the image being captured by the robot 100B, the robot 100A may notify the robot 100B of a command to request movement. The robot 100B moves according to the command.

When it is detected by image analysis or voice analysis that the baby is in a predetermined state, for example, when the baby begins to cry, the robot 100A notifies the baby that the state has changed in addition to the live image. The robot 100B may perform an interference motion. The robot 100A notifies the smartphone of a message that briefly indicates a state associated with a warning condition such as "the baby is crying" or "the baby has started to crawl". The robot 100B may perform a dance as the interference motion, or may play music such as a lullaby with an internal audio player. In any case, the robot 100B soothes the baby by performing a distracting action on the baby. When the mother returns (see FIG. 7B), the robot 100B stops the interference motion. Specifically, the robot 100B stops the interference motion when "Mother" can be confirmed in the captured image.
The robot 100B stops the interfering motion when it recognizes, through a microphone, that the mother has pronounced a keyword (hereinafter referred to as “completion command”) that indicates completion of the watching action such as “thank you” or “it is okay”. May be At this time, it may be determined whether or not the completion command is uttered by the person who has instructed the robot 100A to perform the watching action, and if the instruction is from the person, the watching action may be completed.

The robot 100 recognizes the baby from the captured image. The “baby” detection method may be realized by applying a known face recognition technology. Further, the robot 100 may adjust the moving direction so that the baby can always be visually recognized when the baby is moving regardless of whether the baby is watching or not.

The robot 100A and the robot 100B may take turns watching. For example, when the robot 100A performs watching for 10 minutes, the robot 100B may shift to the watching mode and the robot 100A may take a free action. It is considered that the baby is not bored when the plurality of robots 100 alternately watch. In addition, it becomes easy for a third party to have a feeling as if the robot 100A and the robot 100B are watching together.

　If the robot 100 watches over the baby, it will be easier for mothers who are busy raising children to concentrate on housework. If you do household chores such as washing away from your baby, you may not notice immediately when your baby is crying. A baby may start crying in earnest without realizing that he is crying. This situation puts a great burden on the mother. By the robot 100 watching over the baby, it is possible to quickly detect the sign of the baby crying, such as the baby's "chuckling".

Not only mother but robot 100 participates in childcare. In addition, it is expected that the baby raised while being watched by the robot 100 will have a sense of familiarity with the robot 100 in the future.

<Answer>
8 to 11 are schematic diagrams for explaining an action scene when the robot 100 makes an answering machine.
Assume a home in which two robots 100A and 100B and one female owner live. The female owner goes to work (Fig. 8A). At this time, the female owner calls out to the nearby robot 100A, "Keep me away."
The robot 100A recognizes that the female owner is going out by listening to this word (collecting voice through the microphone), and shifts to the "answering mode" (FIG. 8B).
The female owner goes out from the front door (Fig. 8C). The robot 100A shifts to the entrance and executes a predetermined motion to drop off the female owner.

On the other hand, the robot 100B in the room starts playing after chasing the robot 100A (Fig. 8D). When the female owner goes out, both the robot 100A and the robot 100B may give up, or only the robot 100 whose intimacy with the female owner is a predetermined value or more may give up. ..

After a while, the front door opens during the answering machine (Fig. 9A).
When the robot 100 recognizes that there is a front door by voice analysis or image analysis, the two robots 100 move to the front door in anticipation of the return of the female owner (FIG. 9B).
However, it is assumed that the person (the person identified by the image analysis or the like) who appears is not the female owner but a stranger (suspicious person) with a large bag (FIG. 9C).
At this time, the robot 100 approaches the suspicious person sufficiently to photograph the suspicious person (FIG. 9D). The robot 100 shifts to the alert mode. After shifting to the alert mode, the robots 100 may move to the vicinity of the suspicious individual and continue shooting the suspicious individual.

A female owner works in the office. The robot 100 transmits the photographed image of the suspicious individual to the smartphone of the female owner (FIG. 10A).
A female owner who is working learns that the robot 100 has found a suspicious person with a smartphone (FIG. 10B). In this case, it is assumed that the woman owner's mother thought to be a suspicious person. The robots 100 do not know the mother of the female owner. The woman notifies the robots 100 from the smartphone that the person is not a suspicious person. The female owner may inform the robot 100 that he or she is not a suspicious person, by saying, "I'm a mom, so don't worry." The robot 100 releases the alert mode.
The mother cooks home cooking (Figure 10C).

The female owner (daughter) returns home and chats with her mother while eating home cooking (Fig. 11A).
The robots 100 are playing next to each other (Figs. 11B and 11C).

When a suspicious person (a person who has not been visually recognized in the past or a person whose intimacy degree is equal to or lower than a threshold value) is found in the answering machine through image analysis or voice recognition, the robot 100 photographs the suspicious person and The captured image of the suspicious individual is transmitted to the (specific user) smartphone. According to such a control method, the female owner can leave the security of the absence home to the robot 100 with peace of mind. In addition to this, the robot 100 is out of the office through image analysis and voice recognition, such as when an earthquake occurs, something is broken, a gas leak occurs, a customer (call from the intercom) such as a courier, etc., through image analysis and voice recognition. It detects various events and notifies the owner's smartphone of the content that informs the event. Further, the robot 100 may record an absence event as a life log, and the owner may confirm the absence event by checking the life log via the smartphone after returning home.

The mode setting unit of the robot 100 may set the answering mode based on the operation input from the user. The robot 100 may automatically change the setting to the answering machine mode when detecting a specific event that the user goes out of the front door. Alternatively, the robot 100 may automatically change the setting to the answering machine mode when the user cannot be visually recognized in the room for a certain time or longer.

In the alert mode, the robot 100 may set the action range at a position where a suspicious person can be photographed. This is so that the behavior of a suspicious person is not overlooked. Moreover, in order to prevent violence from a suspicious person, you may act away from the suspicious person. After the suspicious person is notified, when the user receives a notification that the user is not a suspicious person, the robot 100 cancels the warning mode and returns to the answering machine mode. After that, the robot 100 may normally engage with the unidentified person (former/suspicious person). Further, the figure of an unidentified person may be stored and parameters such as intimacy may be managed. You may follow an unidentified person. In the case of the examples shown in FIGS. 8 to 11, although the mother is initially warned by the robot 100, after the female owner (daughter) gives the robot 100 an approval notice, the robot 100 starts to cling to the mother. .. The mother can feel welcomed after being turned over to the robot 100.

In the answering machine, a user (owner) at a remote location may send an indoor confirmation instruction to the robot 100 via a smartphone. When the robot 100 receives the indoor confirmation instruction, the robot 100 patrolles the room according to the map generated based on the SLAM. At this time, the robot 100 may transmit the captured image to the smartphone of the user, or may notify the presence/absence of an abnormal event. The user can confirm the state of the home at any time by transmitting the indoor confirmation instruction.

<Remote operation>
12 to 14 are schematic diagrams for explaining an action scene when a user who is out and about operates the robot 100 remotely.
In the absence house, two robots 100A and 100B are in the absence. There are cats in this house (Fig. 12A).
On the other hand, the rest of the family leave the robots 100 and the cat and go out to the city (Fig. 12B). The boy has a face that does not float (Fig. 12C).
The boy takes out the smartphone (mobile terminal). The boy begins to manipulate the smartphone (Fig. 12D).

Two robots 100 are playing in the absence house (Fig. 13A).
When the robot 100A starts moving, the robot 100B follows the robot 100A (FIG. 13B). The robot 100A and the robot 100B are chasing and playing.
The mother cares about the appearance of the boy (son) (Fig. 13C).
When the boy goes out, he is concerned that the cat was not very well (Fig. 13D).

The boy sends a command from the smartphone to the robot 100 to "check the appearance of the cat". The robot 100A and the robot 100B move to capture an image of a point or an object indicated by the command, and capture an image as appropriate. Images captured by the robot 100A and the robot 100B are sent to the smartphone. Cats are playing well in the cat tree (Fig. 14A).
The family is reassured by the lively appearance of the cat (Fig. 14B).
The robot 100 shoots a cat playing on the cat tower. The robot 100 photographs the cat up close, and the cat gazes at the robot 100 (FIGS. 14C and 14D).

In this way, the user can send various instructions from the smartphone to the robot 100. In particular, the user can instruct the robot 100 to check the room. When the command “check the appearance of a cat” is transmitted as in the above embodiment, the robot 100 detects an object corresponding to “cat” from the captured image and transmits the captured image centering on the cat to the smartphone. To do. Such an instruction may be a voice command or may be input from a graphical user interface included in the smartphone. The user may operate the robot 100 like a radio control car (hereinafter, such an operation method is referred to as “remote operation”).

The image captured by the robot 100 is displayed on the smartphone, and the user may enlarge and display a particularly desired portion of the captured image relayed live on the smartphone. The robot 100 may transmit the whole sky image itself to the smartphone, and the user may confirm the “what he/she saw” of the robot 100 from the whole sky image.

The robot 100 can recognize not only species such as humans and cats but also individual levels such as “who” and “which”. Cats, black cats and white cats, big cats and small cats are treated as different cats. The robot 100 also learns the name of the cat based on the call to the cat by the user. For example, a model that extracts a cat image from a cat name recognized by voice analysis and a captured image when the cat name is recognized and outputs the cat name with the cat image as an input is machine-learned. May be generated. By using such a model, even when there are a plurality of cats, the robot 100 can select the designated cat as an object to be photographed if the user designates and orders a cat name. The user may register the name of the cat and the picture of the cat in advance via a smartphone or the like.

When the user remotely operates the robot 100A, the robot 100B may move with respect to the robot 100A. The captured image from the robot 100A and the captured image from the robot 100B are transmitted to the user's smartphone. When the robot 100A images a cat, the robot 100B near the robot 100A also images the cat by the omnidirectional camera 113. The user can acquire the captured image of the cat not only from the robot 100A but also from the robot 100B simply by remotely controlling the robot 100A. The user can remotely control the robot 100B indirectly by remotely controlling only the robot 100A. This is because the robot 100B has a "following function".

The user can set the robot 100 to the remote control mode from the smartphone. The user can also terminate the remote control mode of the robot 100 from the smartphone. The robot 100 in the remote control mode may change the display of the eyes 110. For example, the robot 100 may change the eyes 110 to red eyes, or may display the icon on the eyes 110 to visually represent “being controlled (remotely operated)”. When the remote control mode ends, the robot 100 returns the eyes 110 to the normal black-eye display, and returns to the location point at the start of the remote control mode. When the remote control mode ends, the robot 100 may sit down, or may shake its head violently to express "a state in which it has regained its ego from control".

The remote mode robot 100 does not change emotional parameters or intimacy.

When switching to the remote control mode, the robot 100 authenticates the person who requested the remote control. Only if the authentication is successful, the robot 100 switches to the remote control mode. For the authentication, a general authentication method using an account name and a password, or an electronic certificate may be registered in advance in a device used for remote operation, and only access from a device having the electronic certificate may be permitted. Further, the person operating the mobile terminal may be authenticated by confirming that the person operating the mobile terminal is the owner of the robot 100, using a camera or a microphone provided in the mobile terminal such as a smartphone. Further, when the remote mode is requested, the users in the vicinity may be allowed to approve the switching to the remote mode. As described above, by sufficiently confirming that the person requesting the remote mode is the owner of the robot 100, it is possible to prevent unintended remote operation by a third party.

<Watching over the elderly>
15 to 18 are schematic diagrams for explaining action scenes when a plurality of robots 100 watch over an elderly person.
An elderly father lives alone. The only daughter lives apart from her father. There are two robots 100 in the father's house (Fig. 15A).
In the living room at home, her daughter is looking at her smartphone (Fig. 15B). The robots 100 record the life with their father as a life log. Lifelog is a diary that shows what is happening around his father in a way that respects his privacy.
The daughter checks the life log on the smartphone (Fig. 15C). The life log contains simple information such as when the father got up and had breakfast.
On the other hand, the father holds the robot 100 and loves him (FIG. 15D). Upon recognizing the father's permission through image analysis, voice analysis, communication, or the like, the robot 100 may send the captured image of the figure playing with the father to the daughter's smartphone. For example, when the robot 100A is held by the father, the robot 100B may serve as a cameraman to capture an image of the robot 100A and the father and transmit the captured image to the daughter's smartphone.

Daughter is relieved to see her father enjoying life with robot 100 in the living room (Fig. 16A).

Next, assume that the daughter is working in the office. Daughter suddenly takes out the smartphone and checks the father's life log (Fig. 16B).
Suppose the lifelog had few records about his father. The daughter suddenly becomes worried about her father (Fig. 16C).
The daughter calls the father from the office corridor (Fig. 16D).

Immediately from the phone, you can hear the humorous voice of your father (Fig. 17A).
My father is in the inn. It seems that he just received a call when he took a bath (Fig. 17B).
The father tells his daughter that he was in a hot spring with a friend (Fig. 17C).
The daughter did not know that her father would go to the hot spring, so she was relieved to know the circumstances (Fig. 17D).

　The conversation between father and daughter continues (Figs. 18A and 18B). At the father's home, two robots 100 are out of office (Fig. 18C).

The robot 100 records various events that occur in life with a father (elderly person to be watched over) as a life log. This life log records the father's daily routines, such as when he got up and whether he did the gymnastics he always does today. The daughter can confirm whether the father is living as usual through the life log provided by the robot 100.

When the event indicating the relationship between the father and the robot 100 does not occur (is not detected) for a predetermined time or longer, the robot 100 may send the incident notification to the daughter's smartphone. For example, the incident notification may be transmitted when the robot 100 has not been touched by the father for a while, or when the father is lying down at noon. Whether or not the father is lying can be determined by image analysis or temperature sensor analysis. The robot 100 may act so as to increase the viewing opportunities of the father by actively moving around in the room.

A life log that abstracts information allows the daughter to check the father's daily life while protecting the father's privacy. The robot 100 does not always need to visually recognize the elderly person when the elderly person is to be watched over. The elderly person lives an independent life, and the robot 100 basically only needs to take an autonomous action. It is considered preferable for the elderly person and the robot 100 to maintain an appropriate sense of distance. The robot 100 may always record a life log when the user desires, not limited to watching. The robot 100 may notify the daughter of the abnormality only when an abnormality occurs in the life of the elderly person.

<Expression of jealousy>
One cannot be indifferent to the affection towards oneself. When more than one person turns to the same person, jealousy can easily develop there. Therefore, when the robot 100A and the robot 100B live with the user, one robot 100 may take an action to make the user feel jealous of the other robot 100.

For example, when the robot 100B is held by the user, the state management unit 244 raises the approval desire value (the desire to be acknowledged), which is a kind of emotion parameter of the robot 100A. The operation control unit 150 of the robot 100</b>A holds the user in a hug as the approval desire value increases. The robot 100A may stare at the user, approach the user, or wander around the user to ask for a hug. When the user walks, the robot 100A may move following the user. The increase in the approval desire value is externally expressed as a behavioral characteristic of the robot 100 as if it was envious.

When the user continues to hold the robot 100B, the robot 100A may positively express "strong jealousy" by clinging to the user. Alternatively, the robot 100A may passively express jealousy by moving to a position away from the user and directing the line of sight from a distance. The expression mode of such jealousy is determined according to the individuality (initialized individuality or nurtured individuality) of each robot 100. Jealousy may be expressed by "sneaking", and an action expression may be expressed such that the user approaches or leaves for a certain period after the event that causes jealousy occurs. The robot 100 may express the behavior of “being tired” by temporarily rejecting the hug.

The robot 100 may behave more like jealousy when the degree of intimacy is higher. For example, it is assumed that the robot 100A has a high degree of intimacy with the user P1 and has a relatively low degree of intimacy with the user P2. At this time, when the user P1 holds the robot 100B, the approval desire value may be higher than when the user P2 holds the robot 100B. According to such a control method, it becomes possible to express an action as if the user has a monopoly desire to monopolize the love of a user he particularly likes.

The robot 100 may notify other robots 100 of its own status (emotion parameter, intimacy, event, etc.) (hereinafter, such notification is referred to as “status notification”). The robots 100 may be able to recognize each other's states based on the state notification. For example, the robot 100B notifies the robot 100B of a state such that the robot 100A is held by the user, stroked by the user, or changed by the user, so that the robot 100B can change the state of the robot 100A. Can be grasped. While the approval request value (desire to be acknowledged) decreases due to an event such as the robot 100A being held, when the approval request value of the robot 100B is higher than a threshold value, the robot 100B is unique in expressing jealousy. Show behavioral characteristics.

In this embodiment, the state management unit 244 of the server 200 collectively manages emotional parameters of each robot 100. In this case, the state management unit 244 may internally notify the robot 100B of the emotion parameter value of the robot 100A, or may change the emotion parameter of the robot 100B based on the emotion parameter of the robot 100A. Good. In this case, the emotion parameter of the robot 100B may be changed on the condition that the robot 100A is in a position visible from the robot 100B. This is because the robot 100B near the robot 100A expresses a state in which the emotional change of the robot 100A is visually sensed and the emotional parameter of the robot 100B is changed.

Not limited to emotion parameters, the robot 100A may notify the robot 100B by short-range wireless communication such as infrared rays. In this case, the robot 100B can receive the state notification of the robot 100A only when the robot 100B is near the robot 100A and there is no obstacle blocking the line of sight. Therefore, "the state can be sensed only when the robot 100B is close enough to be visually recognized." You can express the situation. In addition, the robot 100B may detect an event such as the robot 100A being hugged or stroked by the captured image. The robot 100B may change the emotion parameter when the pleasing action with respect to the robot 100A is image-recognized.

The robot 100 may not only be jealous of another robot 100, but also jealous of a pet or a child. For example, the approval demand value of the robot 100 may be increased even when the user holds a cat. The user may also need to care for the pet in a place where the robot 100 is not looking, or to love the pet and the robot 100 evenly in order not to make the robot 100 jealous. By positively creating an opportunity for the user to think about the feeling of the robot 100, the user's attachment to the robot 100 can be deepened.

The robot 100 of the present embodiment can express the feelings of the robot 100 by its action without having a conversation. The robot 100A may be able to receive the feeling (emotion parameter) of the robot 100B. The server 200 may reflect the change in the emotion parameter of the robot 100B on the action of the robot 100A. For example, the robot 100A may move closer to the robot 100B when the approval desire value of the robot 100B suddenly drops (when it is considered that something has happened to the robot 100B). When the robot 100A is notified that the approval desire of the robot 100B is satisfied, the robot 100A may increase its own approval desire value (the feeling that the user wants to be acknowledged). According to such a control method, even when the user secretly pets the robot 100B, it is possible to realize an action expression as if the robot 100A felt something. In other words, it is possible to realize a mysterious action expression as if the robots 100 communicate with each other through telepathy.

<Multiple robots staring at the same thing>
The robot 100A and the robot 100B may continuously look at the same object. For example, when the robot 100A looks at a relaxing user, the robot 100B may also look at the same user. The robot 100B is gazing at the user in which the robot 100A is relaxing (communication with the robot 100A or image analysis) (that the head of the robot 100A is in the user's presence direction). It may be detected. The robot B may stare at the user after moving near the robot A. Since the user feels a plurality of eyes, the robots 100 can feel that they have a strong interest in themselves. When the user has not bitten the robots 100 for a long time, the robot 100A and the robot 100B may silently seek the “relationship” by staring at the users at the same time.

It is assumed that the robot 100A has a high degree of intimacy with the user P1 that is equal to or higher than a predetermined value, and the robot 100B has a high degree of familiarity with the user P1 that is equal to or higher than the predetermined value. The more intimate the user of the robot 100, the more chances it is to gaze. Therefore, in the above situation, the robot 100A and the robot 100B have an opportunity to gaze at the user P1 at the same time without hesitation. The robot 100A may notify the robot 100B that it is gazing at the user P1. When the robot 100B receives the state notification "looking at the user P1 (also the robot 100A)" from the robot 100A while gazing at the user P1, the robot 100B turns its gaze toward the surprising motion or the direction of the robot 100A. The "accidental coincidence" may be produced by executing a specific motion such as. Further, when the two are gazing at the same user, the motion may be performed so that the robot 100A and the robot 100B approach each other and gaze at the user P1 side by side. When both robots 100 move to the position where the user's face looks as large as possible and stare at the user side by side, strong pressure can be applied to the user.

In addition, when the insect enters the house (when the insect is detected in the house by image analysis or voice analysis), the robot 100A and the robot 100B share the target insect, and the robot 100A and the robot 100B share the target insect. At the same time, an abnormal interest in insects may be expressed by looking at the insects. Furthermore, the robot 100A and the robot 100B can realize an action expression as if the robots 100 are showing each other by gazing at each other.

When the value of the emotion parameter indicating the curiosity of the robot 100A exceeds a threshold value, the robot 100A may notify the robot 100B of a state of "increasing curiosity". At this time, the robot 100B may move closer to the robot 100A, move the hand to touch the robot 100A, and perform a motion as if the robot 100A wants to know the source of curiosity. The robot 100A may notify the robot 100B of the target object of interest in the whole sky image and the direction thereof. When the robot 100B receives this notification, the robot 100B may look at the same object by directing its head or line of sight to the same object as the object of the robot 100A.

<Appeal>
The robot 100 executes a strong appeal action to the user when a predetermined appeal condition is satisfied, for example, when the approval desire value exceeds a threshold value. Here, the appeal action is an action such as a touch, a voice call, a hug, or the like, which actively seeks the user's involvement with the robot 100. For example, suppose the user is doing exercises such as yoga indoors. When the appealing condition of the robot 100 is satisfied while the user is absorbed in yoga, the robot 100 performs appealing actions such as continuing to gaze at the user and wandering around the user, and requests the interruption of yoga. May be.

<Following action>
As described above, when the robot 100A moves, the robot 100B may follow and move from behind the robot 100A while keeping a constant distance from the robot 100A. The robot 100A may similarly follow and move with respect to the user or the pet. For example, when the dog is following the user, the robot 100A may follow the dog or the user. The robot 100B may follow the robot 100A when the robot 100A is following a dog or the like. The distance between the robot and the tracking target (for example, a dog tracking the user) may be equal to the distance between the tracking target and the tracking target of the tracking target (for example, the user), or may be shorter than the distance by a predetermined length. Alternatively, the distance may be longer than the distance by a predetermined length.

When the robot 100 detects that the moving object Q1 and the moving object Q2 are moving in the same direction for a predetermined time or longer, it determines that "following" has occurred. According to such a control method, it becomes possible to express an action that makes the robot 100 feel the instinct that the user wants to keep up with the tracking when the tracking occurs. It is considered that the appearance in which the plurality of robots 100 follow each other is effective in appealing the cuteness of the robot 100 to the user.

The robot 100 may perform a follow-up action on condition that the emotion parameter of the robot 100, for example, the value of the emotion parameter indicating curiosity is equal to or less than a threshold value. According to such a control method, it becomes possible to express the behavior that the follow-up action to another robot is executed when the curiosity is weakened and the user is bored, and the follow-up action is not executed when the curiosity is increased. When the follow-up action is executed, the follow-up action may be ended on the condition that the curiosity is increased to a threshold value or more due to various events. The source of the action of the autonomous action type robot is a predetermined parameter indicating the internal state. In the present embodiment, the parameter indicating curiosity greatly contributes to the source of behavior, but the curiosity parameter may approach 0 if the external environment has little change. In such a case, it is possible to positively change the own parameter by piggybacking on the action of another robot instead of waiting for the change of the own parameter.

As described above, the plurality of robots 100 may take the same action such as the following action, or change the action characteristics while being influenced by the actions of each other. In order to enhance the cooperation and interlocking of the plurality of robots 100, it is desirable that the states of the robots 100 can be understood in the server 200 or between the robots 100. The robot 100B that has grasped the state of the robot 100A may act in synchronization with the state of the robot 100A, or may act independently without synchronizing. An example of an action in which the robot 100B is synchronized with the robot 100A is that the robot B looks at the same thing as the robot A is gazing at, and the robot B looks at the same object as the object of the robot 100A. Is to execute the motion of.

When a plurality of robots 100 take a coordinated action, it is considered that the user feels the robots 100 cute. The user may want to take a picture of how the robots 100 cooperate. When the omnidirectional camera 113 image-recognizes that the user is holding the camera, the robot 100 may maintain at least one of its own behavior and state until the user finishes photographing. In this case, instead of maintaining the action or state, the robot 100 may select a specific motion. For example, the robot 100 may turn its body in the direction in which the user is present, or may temporarily stop the cooperative action to cooperate with the user in photographing. In this way, the robot 100 may temporarily stop the operation when detecting the shooting action. Further, the robot 100 may take a pose or temporarily stop the action when the user's shooting action is detected, not limited to the coordinated action. According to such a control method, the user can easily upload a captured image of the cute appearance of the robot 100 to an SNS (Social Networking Service) or the like. Further, it is considered that various best shot images of the robot 100 related to various things (pets, children, toys, furniture, etc.) in the home can be easily taken.

<Structure of outer skin>
The outer cover 314 of the robot 100 is configured by accommodating a stretchable base material in a cloth bag. The bag may be made of a flexible material that is warm and comfortable to the user. The base material is preferably a flame-retardant material, and more preferably a material that releases a self-extinguishing gas when the temperature rises. For example, the substrate is composed of flame retardant sponge. Since the outer cover 314 is formed so as to wrap a flame-retardant base material in a cloth bag, self-extinguishing gas is released from the base material even when the cloth bag is ignited, so that the spread of the cloth bag is prevented. You can The threshold temperature when the substrate generates the self-extinguishing gas is preferably lower than the ignition temperature of the cloth. In this case, when the cloth becomes hot, self-extinguishing gas is generated before the cloth is ignited, so that the cloth can be prevented from igniting. By forming the outer cover 314 in a dual structure of a flame-retardant base material and a flexible bag, the warmth of the touch of the robot 100 and the safety against high temperature can both be achieved.

It should be noted that the present invention is not limited to the above-described embodiments and modified examples, and constituent elements can be modified and embodied without departing from the scope of the invention. Various inventions may be formed by appropriately combining a plurality of constituent elements disclosed in the above-described embodiments and modifications. In addition, some constituent elements may be deleted from all the constituent elements shown in the above-described embodiments and modifications.

Although the robot system 300 has been described as including one or more robots 100 and one server 200, some of the functions of the robot 100 may be realized by the server 200, or some or all of the functions of the server 200. May be assigned to the robot 100. One server 200 may control a plurality of robots 100, or a plurality of servers 200 may cooperate to control one or more robots 100.

A third device other than the robot 100 and the server 200 may take part of the function. The aggregate of the functions of the robot 100 and the functions of the server 200 described with reference to FIG. 4 can be grasped as one “robot” in a broad sense. How to distribute the plurality of functions required to implement the present invention to one or more pieces of hardware is determined in consideration of the processing capability of each piece of hardware and the specifications required for the robot system 300. It may be decided.

As described above, the “robot in the narrow sense” means the robot 100 that does not include the server 200, but the “robot in the broad sense” means the robot system 300. Many of the functions of the server 200 may possibly be integrated into the robot 100 in the future.

This application claims priority on the basis of Japanese Patent Application No. XX××-×××××× filed on the date of xx××year×month×, and the entire disclosure thereof is here. take in.

Claims

A motion control unit that selects the motion of the robot,
A drive mechanism that executes a motion selected by the operation control unit;
A recognition unit that determines whether the target person satisfies a predetermined watching condition,
A robot comprising a mode setting unit that sets the watching mode of the target person when the watching condition is satisfied.
Equipped with a head,
The robot according to claim 1, wherein the drive mechanism executes a motion of continuously directing the head in a direction in which the target person is present during the watching mode.
In the watching mode, the robot shares the position of the target person with another robot, and the same gazing point as the other robot determined based on the target person or a gazing point within a predetermined distance from the gazing point of the other robot. The robot according to claim 2, wherein the head is turned toward the viewpoint.
The robot according to claim 2 or 3, wherein in the watching mode, at least one of the other robot and the self is configured to direct the head toward the target person. ‥
The robot according to any one of claims 1 to 4, wherein a distance to the object is controlled within a predetermined distance during the watching mode.
The robot according to any one of claims 1 to 5, wherein during the watching mode, the operation amount of the mobile mechanism is made lower than that in at least one mode when the watching condition is not satisfied.
The robot according to any one of claims 1 to 6, characterized in that, when it is determined that the target object satisfies a predetermined condition during the watching mode, a motion for the target object is executed.
The robot according to any one of claims 1 to 7, wherein a motion is performed on the target object when it is determined that the target object satisfies a predetermined condition during the watching mode.
The mode setting unit is configured to set the watching mode when a condition that a person a predetermined age or older is not detected in the surroundings is satisfied in addition to the watching condition. The robot according to any one of 1 to 9.
The mode setting unit is configured to set the watching mode when a condition that a person associated with the target person is not detected in a week is satisfied in addition to the watching condition. The robot according to any one of claims 1 to 9.
The robot according to any one of claims 1 to 10, further comprising a communication unit that transmits a captured image of the target person to a predetermined communication terminal in the watching mode.