JPWO2020129993A1 - robot - Google Patents

Abstract

Providing a sense of security to users in living with robots. The autonomous action type robot has a motion control unit that selects the motion of the robot, a drive mechanism that executes the motion selected by the motion control unit, and a recognition unit that determines whether or not the target object satisfies a predetermined watching condition. It is provided with a mode setting unit for setting the watching mode of the target object when the watching condition is satisfied, and a communication unit for transmitting the captured image of the target object to a predetermined communication terminal in the watching mode.

Description

The present invention relates to a robot that autonomously selects an action according to an internal state or an external environment.

Humans keep pets in search of healing. On the other hand, many people give up on pets for various reasons, such as not having enough time to take care of their pets, lack of living environment to keep pets, allergies, and hard bereavement. If there is a robot that plays the role of a pet, it may be possible to provide healing that the pet gives to those who cannot keep the pet (see Patent Documents 1 and 2).

Japanese Unexamined Patent Publication No. 2000-323219 International Publication No. 2017/169826

In recent years, robot technology has been rapidly advancing, but it has not yet realized its presence as a companion like a pet. I don't think robots have free will. By observing behaviors that make the pet seem to have free will, humans feel the existence of free will in the pet, sympathize with the pet, and are healed by the pet.

The present invention is an invention completed based on the above-mentioned problem recognition, and the first object thereof is to provide a technique for providing a user with various senses of security in living with a robot. The second object is to provide a technique for expressing the love of the robot for the user. A third object is to provide a technique for a plurality of robots to act in cooperation with each other.

In the autonomous action type robot in a certain aspect of the present invention, the motion control unit that selects the motion of the robot, the drive mechanism that executes the motion selected by the motion control unit, and whether or not the subject satisfies a predetermined watching condition. It is provided with a recognition unit for determining, a mode setting unit for setting the monitoring mode of the target person when the monitoring condition is satisfied, and a communication unit for transmitting the captured image of the target person to a predetermined communication terminal in the monitoring mode. ..

According to the present invention, it becomes easier to enhance the presence of the robot.

The above-mentioned objectives and other objectives, features and advantages are further clarified by the preferred embodiments described below and the accompanying drawings below.

It is a front view of the robot. It is a side view of the robot. It is sectional drawing which outlines the structure of a robot. It is a hardware configuration diagram of the robot in the basic configuration. It is a functional block diagram of a robot system in a basic configuration. It is a schematic diagram for explaining the action scene when a plurality of robots watch over a baby. It is a schematic diagram for explaining the action scene when a plurality of robots watch over a baby. It is a schematic diagram for explaining the action scene when a plurality of robots watch over a baby. It is a schematic diagram for explaining the action scene when a plurality of robots watch over a baby. It is a schematic diagram for explaining the action scene when a plurality of robots watch over a baby. It is a schematic diagram for explaining the action scene when a plurality of robots watch over a baby. It is a schematic diagram for explaining the action scene when a plurality of robots watch over a baby. It is a schematic diagram for explaining the action scene when a plurality of robots watch over a baby. It is a schematic diagram for explaining the action scene when a plurality of robots watch over a baby. It is a schematic diagram for explaining the action scene when a plurality of robots watch over a baby. It is a schematic diagram for explaining the action scene when a plurality of robots watch over a baby. It is a schematic diagram for explaining the action scene when a plurality of robots watch over a baby. It is a schematic diagram for explaining the action scene when a plurality of robots watch over a baby. It is a schematic diagram for explaining the action scene when a robot has an answering machine. It is a schematic diagram for explaining the action scene when a robot has an answering machine. It is a schematic diagram for explaining the action scene when a robot has an answering machine. It is a schematic diagram for explaining the action scene when a robot has an answering machine. It is a schematic diagram for explaining the action scene when a robot has an answering machine. It is a schematic diagram for explaining the action scene when a robot has an answering machine. It is a schematic diagram for explaining the action scene when a robot has an answering machine. It is a schematic diagram for explaining the action scene when a robot has an answering machine. It is a schematic diagram for explaining the action scene when a robot has an answering machine. It is a schematic diagram for explaining the action scene when a robot has an answering machine. It is a schematic diagram for explaining the action scene when a robot has an answering machine. It is a schematic diagram for explaining the action scene when a robot has an answering machine. It is a schematic diagram for explaining the action scene when a robot has an answering machine. It is a schematic diagram for explaining the action scene when a robot has an answering machine. It is a schematic diagram for demonstrating an action scene when a user who is going out remotely controls a robot. It is a schematic diagram for demonstrating an action scene when a user who is going out remotely controls a robot. It is a schematic diagram for demonstrating an action scene when a user who is going out remotely controls a robot. It is a schematic diagram for demonstrating an action scene when a user who is going out remotely controls a robot. It is a schematic diagram for demonstrating an action scene when a user who is going out remotely controls a robot. It is a schematic diagram for demonstrating an action scene when a user who is going out remotely controls a robot. It is a schematic diagram for demonstrating an action scene when a user who is going out remotely controls a robot. It is a schematic diagram for demonstrating an action scene when a user who is going out remotely controls a robot. It is a schematic diagram for demonstrating an action scene when a user who is going out remotely controls a robot. It is a schematic diagram for demonstrating an action scene when a user who is going out remotely controls a robot. It is a schematic diagram for demonstrating an action scene when a user who is going out remotely controls a robot. It is a schematic diagram for demonstrating an action scene when a user who is going out remotely controls a robot. It is a schematic diagram for explaining the action scene when a plurality of robots watch over an elderly person. It is a schematic diagram for explaining the action scene when a plurality of robots watch over an elderly person. It is a schematic diagram for explaining the action scene when a plurality of robots watch over an elderly person. It is a schematic diagram for explaining the action scene when a plurality of robots watch over an elderly person. It is a schematic diagram for explaining the action scene when a plurality of robots watch over an elderly person. It is a schematic diagram for explaining the action scene when a plurality of robots watch over an elderly person. It is a schematic diagram for explaining the action scene when a plurality of robots watch over an elderly person. It is a schematic diagram for explaining the action scene when a plurality of robots watch over an elderly person. It is a schematic diagram for explaining the action scene when a plurality of robots watch over an elderly person. It is a schematic diagram for explaining the action scene when a plurality of robots watch over an elderly person. It is a schematic diagram for explaining the action scene when a plurality of robots watch over an elderly person. It is a schematic diagram for explaining the action scene when a plurality of robots watch over an elderly person. It is a schematic diagram for explaining the action scene when a plurality of robots watch over an elderly person. It is a schematic diagram for explaining the action scene when a plurality of robots watch over an elderly person. It is a schematic diagram for explaining the action scene when a plurality of robots watch over an elderly person.

The robot 100 in the present embodiment has a presence as a member of the family by sharing daily life with the user, sometimes considering the user, sometimes struggling to be useful to the user, and actively seeking the affection of the user. Demonstrate.
Hereinafter, the basic configuration of the robot 100 will be described with reference to FIGS. 1 to 4, and then various action scenes of the robot 100 will be described.

[Basic configuration]
FIG. 1 is a diagram showing the appearance of the robot 100. 1A is a front view and FIG. 1B is a side view.
The robot 100 is an autonomous action type robot that determines an action based on an external environment and an internal state. The external environment is recognized by various sensors such as a camera and a thermo sensor 115. The internal state is quantified as various parameters expressing the emotions of the robot 100. The robot 100 has an action range inside the owner's home. Hereinafter, the human being involved in the robot 100 is referred to as a "user". Among the users, the owner or administrator of the robot 100 is called an "owner".

The body 104 of the robot 100 has an overall rounded shape and includes an outer skin 314 formed of a soft and elastic material such as urethane, rubber, resin, or fiber. The robot 100 may be dressed. The total weight of the robot 100 is about 5 to 15 kilograms, and the height is about 0.5 to 1.2 meters. Due to various attributes such as appropriate weight, roundness, softness, and good touch, the effect that the user can easily hold the robot 100 and want to hold it is realized.

The robot 100 includes a pair of front wheels 102 (left wheel 102a, right wheel 102b) and one rear wheel 103. The front wheels 102 are the driving wheels, and the rear wheels 103 are the driven wheels. Although the front wheel 102 does not have a steering mechanism, the rotation speed and rotation direction of the left and right wheels can be individually controlled. The rear wheel 103 is a caster and is rotatable in order to move the robot 100 back and forth and left and right. The rear wheel 103 may be an omni wheel. By increasing the number of rotations of the right wheel 102b more than that of the left wheel 102a, the robot 100 can turn left or turn counterclockwise. By increasing the number of rotations of the left wheel 102a as compared with that of the right wheel 102b, the robot 100 can turn right or rotate clockwise.

The front wheels 102 and the rear wheels 103 can be completely housed in the body 104 by a drive mechanism (rotation mechanism, link mechanism). A pair of left and right covers 312 are provided on the lower half of the body 104. The cover 312 is made of a flexible and elastic resin material (rubber, silicone rubber, etc.), constitutes a soft body, and can accommodate the front wheels 102. A slit 313 (opening) that opens from the side surface to the front surface is formed in the cover 312, and the front wheel 102 can be advanced through the slit 313 and exposed to the outside.

Most of each wheel is hidden in the body 104 even during traveling, but when each wheel is completely housed in the body 104, the robot 100 becomes immovable. That is, the body 104 descends with the retracting operation of the wheels and sits on the floor surface F. In this seated state, the flat seating surface 108 (ground contact bottom surface) formed on the bottom of the body 104 abuts on the floor surface F.

Robot 100 has two arms 106. There is a hand at the tip of the arm 106, but it does not have the function of gripping an object. The arm 106 can perform simple movements such as raising, bending, waving, and vibrating by driving an actuator described later. The two arms 106 can be individually controlled.

The face area 116 is exposed on the front surface of the head of the robot 100. The face area 116 is provided with two eyes 110. The eye 110 is a device capable of displaying an image by a liquid crystal element or an organic EL element, and expressing a line of sight or a facial expression by moving the eyes or eyelids displayed as an image. A nose 109 is provided in the center of the face area 116. An analog stick is provided on the nose 109, and it can detect the pushing direction in addition to the up, down, left, and right directions. Further, the robot 100 is provided with a plurality of touch sensors, and can detect the user's touch on almost the entire area of the robot 100 such as the head, the torso, the buttocks, and the arms. The robot 100 is equipped with various sensors such as a microphone array and an ultrasonic sensor that can specify the direction of a sound source. It also has a built-in speaker and can emit simple voice.

A horn 112 is attached to the head of the robot 100. An all-sky camera 113 is attached to the tsuno 112, and the entire upper part of the robot 100 can be imaged at once. The tsuno 112 also has a built-in thermo sensor 115 (thermo camera). Further, the tsuno 112 is provided with a plurality of modules (not shown) for communication using infrared rays, and these modules are installed in a ring shape toward the periphery. Therefore, the robot 100 can perform infrared communication while recognizing the direction. Further, the tsuno 112 is provided with a switch for emergency stop, and the user can urgently stop the robot 100 by pulling out the tsuno 112.

FIG. 2 is a cross-sectional view schematically showing the structure of the robot 100.
The body 104 includes a body frame 310, a pair of arms 106, a pair of covers 312 and an outer skin 314. The body frame 310 includes a head frame 316 and a body frame 318. The head frame 316 forms a hollow hemisphere and forms the head skeleton of the robot 100. The body frame 318 has a square tube shape and forms the body skeleton of the robot 100. The lower end of the body frame 318 is fixed to the lower plate 334. The head frame 316 is connected to the body frame 318 via a connecting mechanism 330.

The body frame 318 constitutes the axis of the body 104. The body frame 318 is configured by fixing a pair of left and right side plates 336 to a lower plate 334, and supports a pair of arms 106 and an internal mechanism. A battery 118, a control circuit 342, various actuators, and the like are housed inside the body frame 318. The bottom surface of the lower plate 334 forms the seating surface 108.

The body frame 318 has an upper plate 332 on its upper part. A bottomed cylindrical support portion 319 is fixed to the upper plate 332. The upper plate 332, the lower plate 334, the pair of side plates 336 and the support portion 319 form the body frame 318. The outer diameter of the support portion 319 is smaller than the distance between the left and right side plates 336. The pair of arms 106 are integrally assembled with the annular member 340 to form the arm unit 350. The annular member 340 forms an annular shape, and a pair of arms 106 are attached so as to be radially separated on the center line thereof. The annular member 340 is coaxially inserted through the support portion 319 and placed on the upper end surfaces of the pair of side plates 336. The arm unit 350 is supported from below by the body frame 318.

The head frame 316 has a yaw axis 321 and a pitch axis 322 and a roll axis 323. Swinging motion is realized by rotating (yawing) around the yaw axis 321 of the head frame 316, and nodding motion, looking up motion and looking down motion are realized by rotating around the pitch axis 322 (pitching), and around the roll axis 323. The motion of tilting the neck to the left and right is realized by the rotation (rolling) of. The position and angle of each axis in the three-dimensional space may change depending on the driving mode of the connection mechanism 330. The connection mechanism 330 comprises a link mechanism and is driven by a plurality of motors installed on the body frame 318.

The fuselage frame 318 accommodates the wheel drive mechanism 370. The wheel drive mechanism 370 includes a front wheel drive mechanism and a rear wheel drive mechanism that move the front wheels 102 and the rear wheels 103 in and out of the body 104, respectively. The front wheels 102 and the rear wheels 103 function as "movement mechanisms" for moving the robot 100. The front wheel 102 has a direct drive motor in the center thereof. Therefore, the left wheel 102a and the right wheel 102b can be driven individually. The front wheel 102 is rotatably supported by the wheel cover 105, and the wheel cover 105 is rotatably supported by the body frame 318.

The pair of covers 312 are provided so as to cover the body frame 318 from the left and right, and have a smooth curved surface shape so that the outline of the body 104 is rounded. A closed space is formed between the body frame 318 and the cover 312, and the closed space serves as a storage space S for the front wheels 102. The rear wheel 103 is housed in a storage space provided behind the lower part of the fuselage frame 318.

The outer skin 314 covers the main body frame 310 and the pair of arms 106 from the outside. The outer skin 314 has a thickness sufficient for humans to feel elasticity, and is made of an elastic material such as urethane sponge. As a result, when the user hugs the robot 100, he / she feels an appropriate softness and can take a natural skinship like a person makes a pet. The outer skin 314 is attached to the main body frame 310 in a manner that exposes the cover 312. An opening 390 is provided at the upper end of the outer skin 314. This opening 390 inserts the horn 112.

A touch sensor is arranged between the main body frame 310 and the outer skin 314. A touch sensor is embedded in the cover 312. All of these touch sensors are capacitance sensors and detect touches in almost the entire area of the robot 100. The touch sensor may be embedded in the outer skin 314 or may be arranged inside the main body frame 310.

The arm 106 has a first joint 352 and a second joint 354, an arm 356 between the two joints, and a hand 358 at the tip of the second joint 354. The first joint 352 corresponds to the shoulder joint and the second joint 354 corresponds to the wrist joint. A motor is provided at each joint to drive the arm 356 and the hand 358, respectively. The drive mechanism for driving the arm 106 includes these motors and their drive circuit 344.

FIG. 3 is a hardware configuration diagram of the robot 100.
The robot 100 includes an internal sensor 128, a communication device 126, a storage device 124, a processor 122, a drive mechanism 120, and a battery 118. The drive mechanism 120 includes the connection mechanism 330 and the wheel drive mechanism 370 described above. The processor 122 and the storage device 124 are included in the control circuit 342. Each unit is connected to each other by a power line 130 and a signal line 132. The battery 118 supplies power to each unit via the power line 130. Each unit sends and receives control signals by the signal line 132. The battery 118 is a lithium ion secondary battery and is a power source for the robot 100.

The internal sensor 128 is an assembly of various sensors built in the robot 100. Specifically, it is a camera, a microphone array, a distance measuring sensor (infrared sensor), a thermo sensor 115, a touch sensor, an acceleration sensor, a pressure pressure sensor, an odor sensor, and the like. The touch sensor corresponds to most of the area of the body 104 and detects the user's touch based on the change in capacitance. The odor sensor is a known sensor that applies the principle that the electrical resistance changes due to the adsorption of molecules that cause odors.

The communication device 126 is a communication module that performs wireless communication for various external devices. The storage device 124 is composed of a non-volatile memory and a volatile memory, and stores a computer program and various setting information. The processor 122 is a means for executing a computer program. The drive mechanism 120 includes a plurality of actuators. In addition, a display and speakers will be installed.

The drive mechanism 120 mainly controls the wheels and the head. The drive mechanism 120 can change the moving direction and moving speed of the robot 100, and can also raise and lower the wheels. When the wheels are raised, the wheels are completely housed in the body 104, and the robot 100 comes into contact with the floor surface F at the seating surface 108 to be in a seated state. Further, the drive mechanism 120 controls the arm 106.

FIG. 4 is a functional block diagram of the robot system 300.
The robot system 300 includes a robot 100, a server 200, and a plurality of external sensors 114. Each component of the robot 100 and the server 200 includes hardware including a CPU (Central Processing Unit), a computing unit such as various coprocessors, a storage device such as a memory and a storage, and a wired or wireless communication line connecting them, and storage. It is realized by software that is stored in the device and supplies processing instructions to the arithmetic unit. The computer program may be composed of a device driver, an operating system, various application programs located on the upper layer thereof, and a library that provides common functions to these programs. Each block described below shows a block for each function, not a configuration for each hardware.
A part of the function of the robot 100 may be realized by the server 200, and a part or all of the function of the server 200 may be realized by the robot 100.

A plurality of external sensors 114 are installed in advance in the house. The server 200 manages the external sensor 114 and provides the robot 100 with the detection value acquired by the external sensor 114 as needed. The robot 100 determines a basic action based on the information obtained from the internal sensor 128 and the plurality of external sensors 114. The external sensor 114 is for reinforcing the sensory organ of the robot 100, and the server 200 is for reinforcing the processing capacity of the robot 100. The communication device 126 of the robot 100 may periodically communicate with the server 200, and the server 200 may be responsible for the process of identifying the position of the robot 100 by the external sensor 114 (see also Patent Document 2).

(Server 200)
The server 200 includes a communication unit 204, a data processing unit 202, and a data storage unit 206.
The communication unit 204 is in charge of communication processing with the external sensor 114 and the robot 100. The data storage unit 206 stores various data. The data processing unit 202 executes various processes based on the data acquired by the communication unit 204 and the data stored in the data storage unit 206. The data processing unit 202 also functions as an interface between the communication unit 204 and the data storage unit 206.

The data storage unit 206 includes a motion storage unit 232 and a personal data storage unit 218.
The robot 100 has a plurality of motion patterns (motions). Various motions are defined, such as shaking the arm 106, approaching the owner while meandering, and staring at the owner with his neck bent.

The motion storage unit 232 stores a "motion file" that defines the control content of the motion. Each motion is identified by a motion ID. The motion file is also downloaded to the motion storage unit 160 of the robot 100. Which motion is executed may be determined by the server 200 or by the robot 100.

Most of the motions of the robot 100 are configured as compound motions including a plurality of unit motions. For example, when the robot 100 approaches the owner, it may be expressed as a combination of a unit motion of turning toward the owner, a unit motion of approaching while raising a hand, a unit motion of approaching while shaking the body, and a unit motion of sitting while raising both hands. .. By combining these four motions, the motion of "approaching the owner, raising his hand on the way, and finally shaking his body and sitting down" is realized. In the motion file, the rotation angle, the angular velocity, and the like of the actuator provided in the robot 100 are defined in relation to the time axis. Various motions are expressed by controlling each actuator with the passage of time according to the motion file (actuator control information).

The transition time when changing from the previous unit motion to the next unit motion is called "interval". The interval may be defined according to the time required to change the unit motion and the content of the motion. The length of the interval is adjustable.
Hereinafter, the settings related to the behavior control of the robot 100, such as when and which motion to select, and the output adjustment of each actuator for realizing the motion, are collectively referred to as "behavior characteristics". The behavior characteristics of the robot 100 are defined by a motion selection algorithm, a motion selection probability, a motion file, and the like.

In addition to the motion file, the motion storage unit 232 stores a motion selection table that defines a motion to be executed when various events occur. In the motion selection table, one or more motions and their selection probabilities are associated with the event.

The personal data storage unit 218 stores user information. Specifically, it stores master information showing the intimacy with the user and the physical and behavioral characteristics of the user. Other attribute information such as age and gender may be stored.

The robot 100 has an internal parameter of intimacy for each user. When the robot 100 recognizes an action that shows a favor to itself, such as picking it up or calling out to it, the intimacy with the user becomes high. The intimacy with users who are not involved in the robot 100, users who act violently, and users who meet infrequently is low.

The data processing unit 202 includes a position management unit 208, a recognition unit 212, an operation control unit 222, an intimacy management unit 220, and a state management unit 244.
The position management unit 208 specifies the position coordinates of the robot 100. The state management unit 244 manages various internal parameters such as the charging rate, the internal temperature, and various physical states such as the processing load of the processor 122. In addition, the state management unit 244 manages various emotion parameters indicating the emotions (loneliness, curiosity, approval desire, etc.) of the robot 100. These emotional parameters are constantly fluctuating. The movement target point of the robot 100 changes according to the emotional parameter. For example, when loneliness is increasing, the robot 100 sets the place where the user is as a movement target point.

Emotional parameters change over time. In addition, various emotional parameters change depending on the response action described later. For example, the emotional parameter indicating loneliness decreases when the owner "hugs", and the emotional parameter indicating loneliness gradually increases when the owner is not visually recognized for a long period of time.

The recognition unit 212 recognizes the external environment. Recognition of the external environment includes various recognitions such as recognition of weather and seasons based on temperature and humidity, and recognition of shadows (safety zones) based on the amount of light and temperature. The recognition unit 156 of the robot 100 acquires various environmental information by the internal sensor 128, performs primary processing on the information, and then transfers the information to the recognition unit 212 of the server 200.

Specifically, the recognition unit 156 of the robot 100 extracts an image area corresponding to a moving object, particularly a person or an animal, from the image, and a feature showing the physical characteristics and behavioral characteristics of the moving object from the extracted image area. Extract the "feature vector" as a set of quantities. The feature vector component (feature amount) is a numerical value that quantifies various physical and behavioral features. For example, the width of the human eye is quantified in the range 0 to 1 to form one feature vector component. The method of extracting a feature vector from a captured image of a person is an application of a known face recognition technique. The robot 100 transmits the feature vector to the server 200.

The recognition unit 212 of the server 200 is imaged by comparing the feature vector extracted from the image captured by the built-in camera of the robot 100 with the feature vector of the user (cluster) registered in advance in the personal data storage unit 218. It is determined which person the user corresponds to (user identification process). Further, the recognition unit 212 estimates the user's emotion by recognizing the user's facial expression as an image. The recognition unit 212 also performs user identification processing on moving objects other than humans, for example, cats and dogs that are pets.

The recognition unit 212 recognizes various response actions performed by the robot 100 and classifies them into pleasant / unpleasant actions. The recognition unit 212 also classifies the robot 100 into affirmative / negative reactions by recognizing the owner's response to the action of the robot 100.
Pleasure / discomfort is determined by whether the user's response is comfortable or unpleasant as a living thing. For example, being hugged is a pleasant act for the robot 100, and being kicked is an unpleasant act for the robot 100. The affirmative / negative reaction is determined by whether the user's response action indicates the user's pleasant feelings or unpleasant feelings. Being hugged is a positive reaction that indicates the user's pleasant feelings, and being kicked is a negative reaction that indicates the user's unpleasant feelings.

The motion control unit 222 of the server 200 cooperates with the motion control unit 150 of the robot 100 to determine the motion of the robot 100. The operation control unit 222 of the server 200 creates a movement target point of the robot 100 and a movement route for the movement target point. The operation control unit 222 may create a plurality of movement routes and then select one of the movement routes.

The motion control unit 222 selects the motion of the robot 100 from the plurality of motions of the motion storage unit 232. A selection probability is associated with each motion for each situation. For example, a selection method is defined such that motion A is executed with a probability of 20% when the owner makes a pleasant act, and motion B is executed with a probability of 5% when the temperature rises above 30 degrees Celsius. ..

The intimacy management unit 220 manages the intimacy of each user. As described above, the intimacy is registered as part of the personal data in the personal data storage unit 218. When the pleasant behavior is detected, the intimacy management unit 220 increases the intimacy with the owner. Intimacy goes down when discomfort is detected. In addition, the intimacy of owners who have not been visually recognized for a long period of time gradually decreases.

(Robot 100)
The robot 100 includes a communication unit 142, a data processing unit 136, a data storage unit 148, an internal sensor 128, and a drive mechanism 120.
The communication unit 142 corresponds to the communication device 126 (see FIG. 3), and is in charge of communication processing with the external sensor 114, the server 200, and the other robot 100. The data storage unit 148 stores various data. The data storage unit 148 corresponds to the storage device 124 (see FIG. 3). The data processing unit 136 executes various processes based on the data acquired by the communication unit 142 and the data stored in the data storage unit 148. The data processing unit 136 corresponds to the processor 122 and the computer program executed by the processor 122. The data processing unit 136 also functions as an interface for the communication unit 142, the internal sensor 128, the drive mechanism 120, and the data storage unit 148.

The data storage unit 148 includes a motion storage unit 160 that defines various motions of the robot 100.
Various motion files are downloaded from the motion storage unit 232 of the server 200 to the motion storage unit 160 of the robot 100. The motion is identified by the motion ID. A state in which the front wheel 102 is housed and seated, the arm 106 is lifted, the two front wheels 102 are rotated in the reverse direction, or the robot 100 is rotated by rotating only one of the front wheels 102. In order to express various motions such as trembling by rotating the front wheel 102, stopping once when leaving the user, and looking back, the operation timing, operation time, operation direction, etc. of various actuators (drive mechanism 120) are set. It is defined in time series in the motion file.
Various data may be downloaded to the data storage unit 148 from the personal data storage unit 218 as well.

The data processing unit 136 includes a recognition unit 156 and an operation control unit 150.
The motion control unit 150 of the robot 100 determines the motion of the robot 100 in cooperation with the motion control unit 222 of the server 200. Some motions may be determined by the server 200, and other motions may be determined by the robot 100. Further, although the robot 100 determines the motion, the server 200 may determine the motion when the processing load of the robot 100 is high. The server 200 may determine the base motion and the robot 100 may determine additional motions. How the motion determination process is shared between the server 200 and the robot 100 may be designed according to the specifications of the robot system 300.

The motion control unit 150 of the robot 100 instructs the drive mechanism 120 to execute the selected motion. The drive mechanism 120 controls each actuator according to the motion file.

The motion control unit 150 can also perform a motion of lifting both arms 106 as a gesture to plead for "hug" when a user with high intimacy is nearby, and when tired of "hug", the left and right front wheels 102 can be used. It is also possible to express a motion that dislikes a hug by alternately repeating reverse rotation and stop while it is housed. The drive mechanism 120 causes the robot 100 to express various motions by driving the front wheels 102, the arms 106, and the neck (head frame 316) according to the instructions of the motion control unit 150.

The recognition unit 156 of the robot 100 interprets the external information obtained from the internal sensor 128. The recognition unit 156 is capable of visual recognition (visual unit), odor recognition (smell unit), sound recognition (auditory unit), and tactile recognition (tactile unit).

The recognition unit 156 extracts a feature vector from a captured image of a moving object. As described above, the feature vector is a set of parameters (features) indicating the physical and behavioral characteristics of a moving object. When a moving object is detected, physical characteristics and behavioral characteristics are extracted from an odor sensor, a built-in sound collecting microphone, a temperature sensor, and the like. These features are also quantified and become feature vector components. The recognition unit 156 identifies the user from the feature vector based on the known technique described in Patent Document 2 and the like.

Of a series of recognition processes including detection, analysis, and determination, the recognition unit 156 of the robot 100 selects and extracts information necessary for recognition, and interpretation processing such as determination is executed by the recognition unit 212 of the server 200. .. The recognition process may be performed only by the recognition unit 212 of the server 200, or may be performed only by the recognition unit 156 of the robot 100, or the recognition process may be executed while both parties share roles as described above. May be good.

When a strong impact is applied to the robot 100, the recognition unit 156 recognizes this by the touch sensor and the acceleration sensor, and the recognition unit 212 of the server 200 recognizes that the "rough act" has been performed by a nearby user. When the user grabs the horn 112 and lifts the robot 100, it may be recognized as a violent act. When a user facing the robot 100 speaks in a specific volume region and a specific frequency band, the recognition unit 212 of the server 200 may recognize that a "calling action" has been made to itself. Further, when a temperature of about body temperature is detected, it is recognized that the user has made a "contact act", and when an upward acceleration is detected in the state of contact recognition, it is recognized that a "hug" has been made. The user may sense the physical contact when lifting the body 104, or may recognize the hug by reducing the load applied to the front wheel 102.
In summary, the robot 100 acquires the user's action as physical information by the internal sensor 128, and the recognition unit 212 of the server 200 determines whether it is comfortable or unpleasant. Further, the recognition unit 212 of the server 200 executes the user identification process based on the feature vector.

The recognition unit 212 of the server 200 recognizes various responses of the user to the robot 100. Some typical responses of various responses are associated with pleasant or unpleasant, affirmative or negative. In general, most of the pleasant actions are positive reactions, and most of the unpleasant actions are negative reactions. Pleasure / discomfort is related to intimacy, and affirmative / negative reactions affect the action selection of the robot 100.

The intimacy management unit 220 of the server 200 changes the intimacy with respect to the user according to the response action recognized by the recognition unit 156. In principle, the intimacy with the user who performed the pleasant act increases, and the intimacy with the user who performed the unpleasant act decreases.

Each function of the server 200 is realized by loading a program for realizing the function into the memory and instantiating it. The processing capacity of the server 200 supplements various processes by the robot 100. The server 200 can be used as a resource for the robot 100. How to use the resources of the server 200 is dynamically determined according to the request from the robot 100. For example, in the robot 100, when it is necessary to continuously generate a complicated motion according to the detection values from a large number of touch sensors, the processing of the processor 122 in the robot 100 is preferentially assigned to the selection / generation of the motion. The processing for recognizing the surrounding situation as an image may be performed by the recognition unit 212 of the server 200. In this way, various processes of the robot system 300 can be distributed between the robot 100 and the server 200.

A single server 200 can control a plurality of robots 100. In this case, each function of the server 200 is independently materialized for each robot 100. For example, the server 200 may prepare a recognition unit 212 for the robot 100B separately from the recognition unit 212 (instance object) for the robot 100A.

On the premise of the above basic configuration, next, the implementation of the robot 100 in the present embodiment will be described focusing on the features and purposes of the present implementation and the differences from the basic configuration.

[SLAM]
The robot 100 of the present embodiment acquires a large number of captured images (still images) by periodically imaging the surroundings with the all-sky camera 113. The robot 100 forms a memory based on the captured image (hereinafter referred to as "image memory").

Image memory is a collection of multiple keyframes. The key frame is the distribution information of the feature points (feature amount) in the captured image. The robot 100 of the present embodiment uses a graph-based SLAM (Simultaneous Localization and Mapping) technology using image features, and more specifically, a key frame using an ORB (Oriented FAST and Rotated BRIEF) feature-based SLAM technology. Form (see Patent Document 3).

The robot 100 periodically forms keyframes while moving to form an image memory as an aggregate of keyframes, in other words, an image feature distribution. The robot 100 estimates the current position by comparing the key frame acquired at the current position with a large number of key frames already possessed. That is, the robot 100 performs "spatial recognition" by comparing the captured image that is actually visually recognized with the captured image (memory) that was once visually recognized, and matching its current situation with the past memory. The image memory formed as a collection of feature points is a so-called map. The robot 100 updates the map while moving while estimating the current position.

The robot 100 having the basic configuration is premised on recognizing the position not by the key frame but by the external sensor 114. The robot 100 of the present embodiment will be described as recognizing the location based only on the key frame.

The robot 100 in the present embodiment includes a "mode setting unit" for setting various modes.

<Watching over the baby>
5 to 7 are schematic views for explaining an action scene when a plurality of robots 100 watch over a baby.
First, in the children's room, a baby (hereinafter referred to as the "target person") who is the target of watching is sleeping in the bouncer (Fig. 5A).
There are two robots 100 in this house. The two robots 100 each recognize the presence and position of a baby (infant) through image recognition. The two robots 100 may share the position of the baby via mutual communication. The two robots 100 determine the gazing point of each robot from the position of the baby. The two robots 100 correct the gazing points so that the gazing points are the same or within a predetermined range through mutual communication. By controlling the movement and parts of the two robots 100 so that the heads are directed to the determined gazing points, the movements of the two robots 100 looking into the baby are realized (FIG. 5B). ).
One of the two robots 100, the robot 100A, keeps watching the baby by turning the head of the robot 100A toward the baby's existence direction according to the position of the baby. Robot 100A keeps a constant distance from the baby so that it does not move away from the baby. The other robot 100B divides the roles through communication with the robot 100A, and after a while, stops watching and executes an operation as if it is starting to play (FIG. 5C).
The mother leaves the robots 100 to watch over the baby and cooks in the kitchen (Fig. 5D).

The mother has a smartphone (communication terminal) in the kitchen (Fig. 6A). The robot 100A (while watching) captures the baby with the all-sky camera 113 and continues to send the captured image as a live image to the smartphone. The smartphone displays the area where the baby is shown in the all-sky image. At this time, an image corrected so as to display the distortion of the all-sky image on a flat surface may be displayed on the smartphone.

Here, the baby suddenly begins to cry (Fig. 6B).
When the playing 100B hears the baby's crying through the microphone (collecting the voice), it approaches the baby based on the position of the baby (Fig. 6C). Robot 100B executes an interference motion. The robot 100B executes an interference motion when at least one of the conditions that the sound such as crying is collected and the condition that the baby is determined to be in a specific state such as crying by image analysis is satisfied. do. Interference motion is a motion for comforting, such as outputting a predetermined voice that attracts the baby's attention, outputting a voice like a lullaby, shaking the arm 106, shaking the head or body, shaking the bouncer, etc. be. Distract the baby with interference motion. In addition, by executing the interference motion, it is possible to give the third party the impression that the robot 100B is struggling to manage the crying baby. Robot A may perform interfering motion on behalf of or in addition to Robot B.
The live video relayed from the robot 100A shows the baby crying on the mother's smartphone (Fig. 6D).
The video before and after the baby starts crying (the video from a predetermined time before the baby starts crying to after a predetermined time) may be stored in an HDD or the like to be made permanent.

The mother is surprised and involuntarily picks up her smartphone (Fig. 7A).
The mother hurriedly went to the children's room and rushed to the baby (Fig. 7B).
The mother holds the baby (Fig. 7C).
The robots 100 are near the mother and the baby and stare at the mothers. The two robots recognize the positions of the mother and the baby by image analysis, and set the gazing point from each position. The two robots 100 share the gazing point via communication and, if necessary, correct each other's gazing points so that they are the same or within a predetermined range. The two robots 100 perform an action of turning their heads to their respective gazing points. As a result, by looking at the mother and the baby side by side with the two robots 100, it is possible to give the mother the impression that the robots 100 are worried about the baby.
The baby held by the mother falls asleep again (Fig. 7D).
When the robots 100 confirm that the baby has fallen asleep, they move away from the bouncer (Fig. 7E).

The robot 100 may search for a baby in order to start the above-mentioned watching action when the mother manually sets the "watching mode" to the "watching mode" via an input switch, a smartphone, or the like. When the robot 100 detects that the baby is on the bouncer, the mode setting unit may automatically set the watching mode. The robot 100 may detect an infant by image recognition and automatically shift to the watching mode when there is no guardian near the infant. The case where there is no guardian near the infant may be, for example, a case where an image of a person older than a predetermined age is not detected, or a case where an image of a person associated with the infant is not detected. Further, when a plurality of robots 100 are in the same room, all the robots 100 may be set to the watching mode, or only some of the robots 100 may be set to the watching mode.

The robot 100 may shift to the watching mode on condition that the "baby" is detected in the captured image and the other user or a specific user such as a mother or father is not detected.

The watching robot 100 may limit the range of action within a range in which the baby to be watched can be visually recognized. Alternatively, the robot 100 may not leave the room where the baby is present while watching over. At the very least, it is desirable that the robot 100 keep an eye on the baby while watching (for example, always catch the baby at the angle of view of the camera). The robot 100 monitors the state of the subject by using a sensor that measures the external environment, such as an all-sky camera 113, a microphone, and a temperature sensor. When the baby moves around, the robot 100 recognizes the position of the baby by using a sensor that measures the external environment such as an all-sky camera 113, a microphone, and a temperature sensor, and turns its own direction toward the baby. It may be controlled to. Further, the robot 100 may reduce the number of places in contact with the baby by accommodating the wheels when the baby enters within his / her predetermined distance.

When the predetermined caution condition is satisfied, the robot 100 executes a motion for actively engaging with the baby. Alternatively, "warning information" may be sent to another communication terminal such as a mother's smartphone. The alert conditions can be set arbitrarily, but for example, when the baby (the person to be watched over) cries, approaches the stairs, tries to go out, messes with small things, falls, etc. , There may be situations where the baby is in danger. The motion for being actively involved may be the above-mentioned interference motion.

When the robot 100A shifts to the watching mode, the robot 100A may notify the robot 100B that the watching mode has been shifted. Upon receiving the notification, the robot 100B may also shift to the watching mode, or may approach the position of the robot 100A or the baby. By gathering a plurality of robots 100 near the baby, it is possible to express the behavior as if the robots 100 care about the baby.

At this time, the robot 100 suppresses the amount of movement of the drive mechanism 120 (particularly, the mechanism related to movement and posture adjustment) more than in the normal mode so as not to wake up the baby by the operation sound of the robot 100, and the robot 100 suppresses the movement amount from the normal mode. It operates quietly so as not to make an operation noise. If the baby goes into watching mode while awake, Robot B may perform certain motions on the baby's side that are of interest to the baby, such as running around the bouncer or dancing. good. For example, it can be determined that the baby is awake when the condition that the voice of the baby is detected by the voice analysis or the condition that the image of the baby with the eyes open is detected by the image analysis is satisfied. Do not perform such motions when the baby is sleeping. The robot B executes an appropriate motion according to the state of the watching target person. It is thought that babies will be relieved if a large number of robots 100 gather around them.

The robot 100 may shift to the watching mode when a specific voice command such as "Look at the baby" is received from a specific user such as a mother via a microphone or the like. At the start of watching, the robot 100 may perform confirmation actions such as turning around the baby, facing the baby, and pointing the hand 358 at the baby based on the position of the baby detected by image analysis or the like. The mother (instructor) can determine whether or not the robot 100 makes a mistake in the target person to be watched by the confirmation action. It is thought that the mother will give a positive reply such as "I left it to you" if it is correct, and a negative reply such as "No" if it is wrong. Therefore, the robot 100 may determine whether or not the object to be watched is correct based on the mother's reply collected by the microphone after the confirmation action.

While watching, the robot 100 keeps the baby's line of sight, such as looking into the baby. The baby is relieved to believe that the robot 100 is watching over him. The mother sees the robot 100 steadily watching over the baby, and is aroused by the affection and trust in the robot 100. The watching action also has a meaning as an appeal to the user (witness) that "the robot 100 is working hard".

As described above, the robot 100B may freely play while watching over the robot 100A. However, the action range of the robot 100B is limited to the range in which the baby or the robot 100A can be visually recognized. The robot 100A may send a close-up live image (close-up image) of the baby to the smartphone, and the robot 100B may send a live image (wide-angle image) of the robot 100A watching over the baby to the smartphone. It is further desirable that the robot 100B limits the range of action so as not to interfere with the shooting of the baby by the robot 100A. For example, the robot 100B may specify the position of the baby and the robot 100A and act so as not to cover the straight line connecting the baby and the robot 100A. Further, when the robot 100A cannot recognize the baby's figure by the robot 100B by recognizing the image being photographed, the robot 100A may notify the robot 100B of a command requesting movement. The robot 100B moves in response to the command.

When it is detected by image analysis or voice analysis that the baby is in a predetermined state, for example, when the baby starts crying, the robot 100A notifies that the baby's state has changed in addition to the live image. Robot 100B may perform an interference motion. The robot 100A notifies the smartphone of a message simply indicating a state associated with a warning condition such as "the baby is crying" or "the baby has begun to squirm". The robot 100B may dance as an interference motion, or may play music such as a lullaby by a built-in audio player. In any case, the robot 100B comforts the baby by performing a distracting action. When the mother returns (see FIG. 7B), the robot 100B stops the interference motion. Specifically, the robot 100B stops the interference motion when the "mother" can be confirmed in the captured image.
The robot 100B stops the interference motion when it recognizes through a microphone or the like that the mother has pronounced a keyword (hereinafter referred to as "completion command") indicating the completion of the watching action such as "thank you" or "already okay". May be. At this time, it may be determined whether or not the completion command has been uttered by the person who instructed the robot 100A to perform the watching action, and if the instruction is given by the person, the watching action may be completed.

The robot 100 recognizes the baby from the captured image. The "baby" detection method may be realized by applying a known face recognition technique. Further, the robot 100 may adjust the moving direction so that the baby can always be visually recognized when the baby is moving, regardless of whether the baby is being watched over or not.

Robot 100A and robot 100B may take turns performing watching. For example, when the robot 100A watches for 10 minutes, the robot 100B shifts to the watching mode, and the robot 100A may move freely. It is considered that the baby is not bored if the plurality of robots 100 take turns watching over. In addition, it becomes easy for a third party to feel as if the robot 100A and the robot 100B are cooperating and watching over.

If the robot 100 watches over the baby, even mothers who are busy raising children can easily concentrate on housework with peace of mind. If you are doing household chores such as washing away from your baby, you may not notice it immediately even if your baby is crying. Your baby may start crying in earnest before you realize you are crying. Such a situation puts a heavy burden on the mother. When the robot 100 watches over the baby, it can quickly detect signs of crying, such as the baby's "guzzling".

Not only mothers but also robot 100 participate in childcare. In addition, it is expected that the baby who grew up while being watched by the robot 100 will have a sense of familiarity with the robot 100 in the future.

<Answering machine>
8 to 11 are schematic views for explaining an action scene when the robot 100 takes an answering machine.
Imagine a home where two robots 100A and 100B and one female owner live. The female owner goes to work (Fig. 8A). At this time, the female owner calls out to the nearby robot 100A, "Please stay at home."
By listening to this word (collecting voice through a microphone), the robot 100A recognizes that the female owner is going out and shifts to the "answering machine mode" (FIG. 8B).
The female owner goes out from the front door (Fig. 8C). The robot 100A moves to the front door and sends off the female owner by executing a predetermined motion.

On the other hand, the robot 100B in the room chases the robot 100A and starts playing (FIG. 8D). When the female owner goes out, both the robot 100A and the robot 100B may send off, or only the robot 100 whose intimacy with the female owner is equal to or higher than a predetermined value may send off. ..

After a while, the front door opens during the answering machine (Fig. 9A).
When it is recognized by voice analysis or image analysis that the front door has been opened, the two robots 100 "expect" the female owner to return home and move to the front door (FIG. 9B).
However, it is assumed that the person who appeared (the person identified by image analysis or the like) is not a female owner but a strange person (suspicious person) with a large bag (Fig. 9C).
At this time, the robot 100 approaches the suspicious person sufficiently and photographs the suspicious person (FIG. 9D). The robot 100 shifts to the alert mode. After shifting to the alert mode, the robots 100 may move near the suspicious person and continue shooting the suspicious person.

The female owner works in the office. The robot 100 transmits a photographed image of a suspicious person to a smartphone of a female owner (FIG. 10A).
A working female owner learns that the robot 100 has found a suspicious person on her smartphone (Fig. 10B). In this case, it is the mother of the female owner who thought she was a suspicious person. The robots 100 do not know the mother of the female owner. The female notifies the robots 100 from the smartphone that they are not suspicious. The female owner may tell the robot 100 that she is not a suspicious person by voice saying, "I'm a mother, so don't worry." The robot 100 cancels the alert mode.
The mother cooks homemade dishes (Fig. 10C).

The female owner (daughter) returns home and chats with her mother while eating home cooking (Fig. 11A).
Robots 100 are playing near them (Fig. 11B, Fig. 11C).

When the robot 100 finds a suspicious person (a person who has never seen it in the past or a person whose intimacy is below the threshold) during the answering machine through image analysis or voice recognition, the robot 100 photographs the suspicious person and is a female owner. Send the captured image of the suspicious person to the smartphone of (specific user). According to such a control method, the female owner can safely entrust the security of the absence home to the robot 100. In addition to this, the robot 100 occurs while the robot 100 is away through image analysis and voice recognition, such as an earthquake that breaks things, a gas leak occurs, and a visitor such as a courier (call from an interphone). It detects various events and notifies the owner's smartphone of the content that informs the event. Further, the robot 100 may record an event during absence as a life log, and the owner may confirm the event during absence by checking the life log via a smartphone after returning home.

The mode setting unit of the robot 100 may also set the answering machine mode based on the operation input from the user. When the robot 100 detects a specific event that the user goes out from the front door, the robot 100 may automatically change the setting to the answering machine mode. Alternatively, the robot 100 may automatically change the setting to the answering machine mode when the user cannot be visually recognized in the room for a certain period of time or longer.

In the alert mode, the robot 100 may set the action range at a position where a suspicious person can be photographed. This is to not overlook the behavior of suspicious persons. In addition, in order to prevent violence from suspicious persons, they may act away from suspicious persons. After making a suspicious person report, when the user notifies that he / she is not a suspicious person, the robot 100 cancels the alert mode and returns to the answering machine mode. After that, the robot 100 may have a normal relationship with an unidentified person (former / suspicious person). In addition, the appearance of an unidentified person may be memorized and parameters such as intimacy may be managed. You may follow an unidentified person. In the case shown in FIGS. 8 to 11, the mother is initially wary of the robot 100, but after the female owner (daughter) gives the robot 100 an approval notice, the robot 100 begins to cling to the mother. .. The mother can feel welcomed and accepted by the robot 100.

In the answering machine mode, a user (owner) at a remote location may send an indoor confirmation instruction to the robot 100 via a smartphone. When the robot 100 receives the indoor confirmation instruction, it patrols the room according to the map generated based on SLAM. At this time, the robot 100 may transmit the captured image to the user's smartphone, or may notify the presence or absence of an abnormal event. The user can check the state of his / her home at any time by sending the room confirmation instruction.

<Remote control>
12 to 14 are schematic views for explaining an action scene when a user who is out of the office remotely controls the robot 100.
Two robots 100A and 100B are answering machines at the answering machine. There is also a cat in this house (Fig. 12A).
On the other hand, the rest of the family goes out to the city, leaving the robots 100 and the cats (Fig. 12B). The boy has a non-floating face (Fig. 12C).
The boy takes out his smartphone (mobile terminal). The boy begins to operate his smartphone (Fig. 12D).

Two robots 100 are playing in the absence home (Fig. 13A).
When the robot 100A starts to move, the robot 100B follows the robot 100A (FIG. 13B). Robot 100A and Robot 100B are chasing and playing.
The mother is worried about the appearance of the boy (son) (Fig. 13C).
When the boy goes out, he cares that the cat wasn't very well (Fig. 13D).

The boy sends a command from the smartphone to the robot 100, "Check the state of the cat." The robot 100A and the robot 100B move to take a picture of the point or the target indicated by the command, and take an appropriate picture. The captured images of the robot 100A and the robot 100B are sent to the smartphone. Cats are playing well in the cat tree (Fig. 14A).
The family feels relieved to see the cat's energetic appearance (Fig. 14B).
Robot 100 takes a picture of a cat playing on a cat tower. The robot 100 takes a close-up shot of the cat, and the cat looks at the robot 100 (FIGS. 14C and 14D).

In this way, the user can send various instructions from the smartphone to the robot 100. In particular, the user can instruct the robot 100 to confirm the room. When the command "check the state of the cat" is transmitted as in the above embodiment, the robot 100 detects an object corresponding to the "cat" from the captured image and transmits the captured image centered on the cat to the smartphone. do. Such a command may be a voice command or may be input from a graphical user interface provided in the smartphone. The user may be able to operate the robot 100 like a radio control car (hereinafter, such an operation method is referred to as "remote control").

The image captured by the robot 100 is displayed on the smartphone, and the user may magnify and display the portion of the captured image live relayed to the smartphone that he / she particularly wants to see. The robot 100 may transmit the all-sky image itself to the smartphone, and the user may confirm the "what he sees" of the robot 100 by the all-sky image.

The robot 100 can recognize not only species such as humans and cats, but also individual levels such as "who" and "which". As for cats, black cats and white cats, and large cats and small cats are treated as different cats. The robot 100 also learns the name of the cat based on the user's call to the cat. For example, machine learning is used to extract a cat image from the cat name recognized by voice analysis and the captured image when the cat name is recognized, and output the cat name by inputting the cat image. May be generated. By using such a model, even when there are a plurality of cats, the robot 100 can select the designated cat as a shooting target if the user specifies and commands the cat's name. The user may register the cat's name and the cat's photo in advance via a smartphone or the like.

When the user remotely controls the robot 100A, the robot 100B may move with respect to the robot 100A. An image captured by the robot 100A and an image captured by the robot 100B are transmitted to the user's smartphone. When the robot 100A takes an image of a cat, the robot 100B near the robot 100A also takes an image of the cat with the all-sky camera 113. The user can acquire a captured image of a cat not only from the robot 100A but also from the robot 100B simply by remotely controlling the robot 100A. By remotely controlling only the robot 100A, the user can indirectly remotely control the robot 100B as well. This is because the robot 100B has a "following function".

The user can set the robot 100 to the remote control mode from the smartphone. The user can also terminate the remote control mode of the robot 100 from the smartphone. The robot 100 in the remote control mode may change the display of the eyes 110. For example, the robot 100 may change the eyes 110 to red eyes, or may visually express that the eyes 110 are "dominated (remotely controlled)" by displaying an icon on the eyes 110. When the remote control mode ends, the robot 100 returns the eyes 110 to the normal black eye display and returns to the location point at the start of the remote control mode. When the remote control mode ends, the robot 100 may sit down or shake its head violently to express "a state of getting out of control and regaining its ego".

The robot 100 in remote mode does not change emotional parameters or intimacy.

When switching to the remote control mode, the robot 100 authenticates the person who requested the remote control. Only when the authentication is successful, the robot 100 switches to the remote control mode. For authentication, a general authentication method using an account name and a password may be used, or an electronic certificate may be registered in advance in a device used for remote control, and only access from a device having the electronic certificate may be permitted. Further, authentication may be performed by confirming that the person operating the mobile terminal is the owner of the robot 100 by using a camera, a microphone, or the like provided in the mobile terminal such as a smartphone. Further, when the remote mode is requested, the surrounding users may be allowed to approve the switch to the remote mode. In this way, by sufficiently confirming that the person who requested the remote mode is the owner of the robot 100, it is possible to prevent unintended remote control by a third party.

<Watching over the elderly>
15 to 18 are schematic views for explaining an action scene when a plurality of robots 100 watch over an elderly person.
An elderly father lives alone. The only daughter lives away from her father. There are two robots 100 in his father's house (Fig. 15A).
In the living room at home, my daughter is looking at her smartphone (Fig. 15B). The robots 100 record their life with their father as a life log. A life log is a diary that shows what is happening around the father in consideration of his father's privacy.
The daughter checks the life log on her smartphone (Fig. 15C). The lifelog contains simple information such as what time the father woke up and had breakfast.
On the other hand, his father holds the robot 100 and loves it (Fig. 15D). If the permission of the father is recognized through image analysis, voice analysis, communication, or the like, the robot 100 may transmit a captured image of the figure playing with the father to the daughter's smartphone. For example, when the robot 100A is held by the father, the robot 100B may act as a cameraman to take an image of the robot 100A and the father, and transmit the captured image to the daughter's smartphone.

The daughter is relieved to see her father enjoying life with Robot 100 in the living room (Fig. 16A).

Next, assume that the daughter is working in the office. The daughter suddenly takes out her smartphone and checks her father's life log (Fig. 16B).
Suppose there are few records about his father in this lifelog. The daughter suddenly becomes worried about her father (Fig. 16C).
The daughter calls her father from the office corridor (Fig. 16D).

Immediately I heard my father's mood from the phone (Fig. 17A).
My father is at the inn. It seems that a phone call was just received when I was out of the bath (Fig. 17B).
The father tells his daughter that he had come to the hot springs with his friends (Fig. 17C).
The daughter was relieved to know the circumstances because she did not know that her father would go to the hot springs (Fig. 17D).

The conversation between father and daughter continues (Fig. 18A, Fig. 18B). At his father's home, two robots 100 are answering machines (Fig. 18C).

The robot 100 records various events that occur in life with the father (elderly person to be watched over) as a life log. This life log records the father's daily activities, such as when he woke up and whether he did the gymnastics he always does today. The daughter can confirm whether the father is living the usual life through the life log provided by the robot 100.

When the event indicating the relationship between the father and the robot 100 does not occur (not detected) for a predetermined time or longer, the robot 100 may send an incident notification to the daughter's smartphone. For example, when the robot 100 has not been touched by the father for a while, or when the father is lying down even at noon, the incident notification may be transmitted. Whether or not the father is lying down can be determined by image analysis or temperature sensor analysis. The robot 100 may act so as to increase the visibility opportunity of the father by actively moving around the room.

The life log, which abstracts information, allows the daughter to check the father's daily life while protecting the father's privacy. When the elderly person is to be watched over, the robot 100 does not have to always visually recognize the elderly person. Elderly people are living independent lives, and the robot 100 should basically behave autonomously. It is considered preferable that the elderly and the robot 100 maintain an appropriate sense of distance. The robot 100 may record a life log whenever the user so desires, not only when watching over. The robot 100 needs to notify the daughter of the change only when the life of the elderly person changes.

<Expression of jealousy>
One cannot remain indifferent to the affection towards oneself. When multiple people turn their love to the same person, jealousy tends to grow there. Therefore, when the robot 100A and the robot 100B live with the user, one robot 100 may take an action that makes the user feel jealousy toward the other robot 100.

For example, when the robot 100B is held by the user, the state management unit 244 raises the approval desire value (the desire to be recognized), which is a kind of emotional parameter of the robot 100A. The motion control unit 150 of the robot 100A asks the user to hug when the approval desire value increases. The robot 100A may stare at the user, approach the user, or wander around the user to ask for a hug. When the user walks, the robot 100A may move following the user. The increase in the approval desire value is externally expressed as the behavioral characteristics of the robot 100 as if it was aroused by jealousy.

When the user continues to hold the robot 100B, the robot 100A may positively express "strong jealousy" by clinging to the user. Alternatively, the robot 100A may passively express jealousy by moving to a position away from the user and directing the line of sight from a distance. The expression mode of such jealousy is determined according to the individuality (initially set individuality or nurtured individuality) of each robot 100. Jealousy may be expressed by "jealousy", or behavioral expressions such that the user approaches or leaves may be expressed for a certain period after the event of jealousy occurs. The robot 100 may express the behavior of "sneaking" by temporarily refusing to hold it.

The robot 100 may take an action that makes the user feel jealousy as the intimacy is higher. For example, it is assumed that the robot 100A has a high intimacy with respect to the user P1 and a relatively low intimacy with respect to the user P2. At this time, when the user P1 holds the robot 100B, the approval desire value may be higher than when the user P2 holds the robot 100B. According to such a control method, it is possible to express an action as if one has a monopoly desire to monopolize the affection of a user who particularly likes it.

The robot 100 may notify another robot 100 of its own state (emotion parameter, intimacy, event, etc.) (hereinafter, such notification is referred to as "state notification"). Based on the status notification, the robots 100 may be able to grasp each other's status. For example, the robot 100B is in the state of the robot 100A by notifying the robot 100B of the state such as "held by the user", "stroked by the user", "changed by the user", and the like. Can be grasped. While the approval request value (desire to be recognized) decreases due to an event such as the robot 100A being held, when the approval request value of the robot 100B is in a state higher than the threshold value, the robot 100B is peculiar to express jealousy. Shows behavioral characteristics.

In the present embodiment, the state management unit 244 of the server 200 collectively manages the emotion parameters of each robot 100. In this case, the state management unit 244 may internally notify the robot 100B of the value of the emotion parameter of the robot 100A, or may change the emotion parameter of the robot 100B based on the emotion parameter of the robot 100A. good. In this case, the emotional parameters of the robot 100B may be changed on condition that the robot 100A is in a position visible from the robot 100B. This is because the robot 100B near the robot 100A visually senses the change in the emotion of the robot 100A and expresses how the robot 100A changes its own emotional parameters.

Not limited to the emotion parameter, the robot 100A may notify the robot 100B by short-range wireless communication such as infrared rays. In this case, the robot 100B can receive the status notification of the robot 100A only when it is near the robot 100A and there is no obstacle blocking the line of sight. You can express the situation. The robot 100B may detect an event such as the robot 100A being held or stroked from the captured image. The robot 100B may change the emotional parameter when the image recognizes a pleasant action against the robot 100A.

The robot 100 may not only be jealous of another robot 100, but may also be jealous of a pet or a child. For example, even when the user holds a cat, the approval desire value of the robot 100 may be increased. The user may also need to care for the pet in a place not seen by the robot 100 so as not to make the robot 100 jealous, or to love the pet and the robot 100 evenly. By positively creating an opportunity for the user to think about the feelings of the robot 100, the user's attachment to the robot 100 can be deepened.

The robot 100 of the present embodiment can express the feelings of the robot 100 by its actions without having a conversation. The robot 100A may be able to receive the feelings (emotion parameters) of the robot 100B. The server 200 may reflect the change in the emotional parameters of the robot 100B in the behavior of the robot 100A. For example, when the approval desire value of the robot 100B drops sharply (when it is considered that something good has happened to the robot 100B), the robot 100A may move near the robot 100B. When the robot 100A is notified that the approval desire of the robot 100B is satisfied, the robot 100A may increase its own approval desire value (the desire to be recognized by itself). According to such a control method, even when the user secretly pets the robot 100B, it is possible to realize an action expression as if the robot 100A felt something. So to speak, it is possible to realize a mysterious behavioral expression as if telepathy is communicated between the robots 100.

<Multiple robots stare at the same thing>
Robot 100A and robot 100B may continuously stare at the same object. For example, when the robot 100A looks at a relaxing user, the robot 100B may also look at the same user. The robot 100B is looking at the user at which the robot 100A is relaxing through communication with the robot 100A or through image analysis (the direction of the head of the robot 100A is the direction in which the user is present). It may be detected. Robot B may look at the user after moving near robot A. Since the user feels a plurality of lines of sight, he / she can feel that the robots 100 have a strong interest in himself / herself. When the user does not bite the robots 100 for a long time, the robot 100A and the robot 100B may silently ask for "relationship" by staring at the user at the same time.

It is assumed that the robot 100A has a high intimacy with respect to the user P1 by a predetermined value or more, and the robot 100B also has a high intimacy with respect to the user P1 by a predetermined value or more. The robot 100 has more opportunities to stare at the user with higher intimacy. Therefore, in the above situation, the robot 100A and the robot 100B have an opportunity to look at the user P1 at the same time. The robot 100A may notify the robot 100B of the state of staring at the user P1. When the robot 100B receives a status notification from the robot 100A that "(the robot 100A is also) staring at the user P1" while it is staring at the user P1, it makes a surprising motion or turns its gaze toward the robot 100A. You may produce "accidental match" by executing a specific motion such as. Further, when each other is staring at the same user, the robot 100A and the robot 100B may execute a motion so as to approach each other and stare at the user P1 side by side. By moving both robots 100 to a position where the user's face looks as large as possible and staring at the user side by side, strong pressure can be applied to the user.

In addition, when an insect enters the house (when an insect is detected in the house by image analysis or voice analysis), the robot 100A and the robot 100B share the target insect, and the robot 100A and the robot 100B share the target insect. At the same time, by staring at the insects, an abnormal interest in the insects may be expressed in behavior. Further, by staring at each other, the robot 100A and the robot 100B can realize an action expression as if they are showing something between the robots 100.

When the value of the emotion parameter indicating the curiosity of the robot 100A exceeds the threshold value, the robot 100A may notify the robot 100B of the state of "increasing curiosity". At this time, the robot 100B may perform a motion as if he / she wants to know the source of the curiosity of the robot 100A, such as approaching the robot 100A and moving his / her hand to touch the robot 100A. The robot 100A may notify the robot 100B of the object of interest in the all-sky image and its direction. When the robot 100B receives this notification, the robot 100B may look at the same object by directing its head or line of sight to the same object as the object of the robot 100A.

<Appeal>
The robot 100 executes a strong appeal action to the user when a predetermined appeal condition is satisfied, for example, when the approval desire value exceeds the threshold value. The appealing behavior referred to here is an behavior that actively seeks the user's involvement in the robot 100, such as touching, calling, and hugging. For example, suppose a user is exercising indoors, such as yoga. When the user is absorbed in yoga and the appeal condition of the robot 100 is satisfied, the robot 100 continues to stare at the user, performs an appealing action such as wandering around the user, and requests the interruption of yoga. You may.

<Following behavior>
As described above, when the robot 100A moves, the robot 100B may follow and move from behind the robot 100A while keeping a constant distance from the robot 100A. The robot 100A may follow the user or the pet in the same manner. For example, when the dog is following the user, the robot 100A may follow the dog or the user. The robot 100B may follow the robot 100A when the robot 100A is following the dog or the like. The distance between the robot and the tracking target (for example, a dog that follows the user) may be equal to the distance between the tracking target and the tracking target (for example, the user), or may be shorter than the distance by a predetermined length. Alternatively, it may be longer than the distance by a predetermined length.

When the robot 100 detects that the moving object Q1 and the moving object Q2 are moving in the same direction for a predetermined time or longer, it determines that "following" has occurred. According to such a control method, it is possible to express an action that makes the robot 100 feel the instinct of wanting to follow when the follow-up occurs. It is considered that the appearance of the plurality of robots 100 following the behavior is effective in appealing the loveliness of the robot 100 to the user.

The robot 100 may execute the follow-up action on condition that the value of the emotion parameter of the robot 100, for example, the emotion parameter indicating curiosity is equal to or less than the threshold value. According to such a control method, it is possible to express an action in which a follow-up action to another robot is executed when the curiosity is weakened and bored, and a follow-up action is not executed when the curiosity is high. When the follow-up action is executed, the follow-up action may be terminated on condition that the curiosity is increased above the threshold value due to various events. The source of behavior of an autonomous robot is a predetermined parameter indicating an internal state. In this embodiment, the curiosity parameter greatly contributes to the source of behavior, but if there is little change in the external environment, the curiosity parameter may approach zero. In such a case, instead of waiting for the change of its own parameter, it is possible to positively change its own parameter by piggybacking on the behavior of another robot.

As described above, the plurality of robots 100 may take the same action such as a follow-up action, or change their behavioral characteristics while being influenced by each other's actions. In order to enhance the coordination and interlocking of a plurality of robots 100, it is desirable that the robots 100 can grasp each other's states in the server 200 or between the robots 100. The robot 100B that grasps the state of the robot 100A may act in synchronization with the state of the robot 100A, or may perform its own action without synchronizing. An example of the behavior in which the robot 100B synchronizes with the robot 100A is that the robot B looks at the same object as the robot 100A, such as the robot 100B looking at the same thing as the robot A looks at. Is to perform the motion of.

When a plurality of robots 100 take cooperative actions, it is considered that the user feels the robots 100 cute. The user may want to take a picture of how the robots 100 collaborate. When the all-sky camera 113 recognizes that the user is holding the camera, the robot 100 may maintain at least one of its actions and states until the user finishes shooting. Further, in this case, instead of maintaining the action or the state, the robot 100 may select a specific motion. For example, the robot 100 may turn its body in the direction in which the user is, or may cooperate with the user in taking a picture by temporarily stopping the cooperative action. In this way, the robot 100 may temporarily stop its operation when it detects a shooting action. Further, the robot 100 may pose or temporarily stop the action when the user's shooting action is detected, not limited to the cooperative action. According to such a control method, the user can easily upload a photographed image of the cute figure of the robot 100 to an SNS (Social Networking Service) or the like. In addition, it will be easier to take various best shot images of the robot 100 related to various things (pets, children, toys, furniture, etc.) in the home.

<Structure of outer skin>
The outer skin 314 of the robot 100 is configured by accommodating a stretchable base material in a cloth bag. The bag may be made of a flexible material that is warm and comfortable to the user. The base material is preferably a flame-retardant material, and more preferably a material that releases a self-extinguishing gas when the temperature rises. For example, the substrate is composed of a flame-retardant sponge. Since the outer skin 314 is formed so as to wrap the flame-retardant base material in a cloth bag, self-extinguishing gas is released from the base material even if the cloth bag ignites, so that the fire spread of the cloth bag should be prevented. Can be done. The threshold temperature at which the substrate generates the self-extinguishing gas is preferably lower than the ignition temperature of the cloth. In this case, when the temperature of the cloth becomes high, self-extinguishing gas is generated before the cloth is ignited, so that the cloth can be prevented from igniting. By making the outer skin 314 a double structure of a flame-retardant base material and a flexible bag, it is possible to achieve both the warmth of the touch of the robot 100 and the safety against high temperatures.

The present invention is not limited to the above-described embodiment or modification, and the components can be modified and embodied within a range that does not deviate from the gist. Various inventions may be formed by appropriately combining a plurality of components disclosed in the above-described embodiments and modifications. In addition, some components may be deleted from all the components shown in the above embodiments and modifications.

Although the robot system 300 has been described as being composed of one or more robots 100 and one server 200, some of the functions of the robot 100 may be realized by the server 200, and some or all of the functions of the server 200 may be realized. May be assigned to the robot 100. One server 200 may control a plurality of robots 100, or a plurality of servers 200 may cooperate to control one or more robots 100.

A third device other than the robot 100 and the server 200 may play a part of the function. The aggregate of each function of the robot 100 and each function of the server 200 described with reference to FIG. 4 can be grasped as one "robot" in the big picture. How to allocate the plurality of functions necessary for realizing the present invention to one or more hardware is determined in consideration of the processing capacity of each hardware, the specifications required for the robot system 300, and the like. It should be decided.

As described above, the "robot in a narrow sense" is a robot 100 that does not include the server 200, while the "robot in a broad sense" is a robot system 300. It is possible that many of the functions of the server 200 will be integrated into the robot 100 in the future.

This application claims priority based on the Japanese application Japanese Patent Application No. XXXXX-XXXXX, which was filed on XXXXX year XX month XX day, and all of its disclosures are here. take in.

Claims (11)

A motion control unit that selects the motion of the robot, and
A drive mechanism that executes the motion selected by the motion control unit, and
A recognition unit that determines whether or not the target person meets the predetermined watching conditions, and
A robot including a mode setting unit that sets the target person's watching mode when the watching condition is satisfied. Equipped with a head
The robot according to claim 1, wherein the drive mechanism executes a motion of continuously pointing the head in the direction in which the target person exists during the watching mode. In the watching mode, the position of the target person is shared with the other robot, and the same gazing point as the other robot determined based on the target person or the gazing point within a predetermined distance from the gazing point of the other robot. The robot according to claim 2, wherein the head is turned toward a viewpoint. The robot according to claim 2 or 3, wherein in the watching mode, at least one of the other robot and the self is configured to turn the head toward the target person. The robot according to any one of claims 1 to 4, which controls the distance to the object within a predetermined distance during the watching mode. The robot according to any one of claims 1 to 5, wherein the operating amount of the mobile mechanism is reduced during the watching mode as compared with at least one mode when the watching condition is not satisfied. The robot according to any one of claims 1 to 6, wherein when it is determined that the object satisfies a predetermined condition during the watching mode, a motion for the object is executed. The robot according to any one of claims 1 to 7, wherein when it is determined that the object satisfies a predetermined condition during the watching mode, a motion for the object is executed. The mode setting unit is characterized in that, in addition to the watching condition, the mode setting unit is configured to set the watching mode when the condition that a person of a predetermined age or older is not detected in the surroundings is satisfied. The robot according to any one of 1 to 9. The mode setting unit is characterized in that, in addition to the watching condition, the watching mode is set when the condition that the person associated with the target person is not detected in a week is satisfied. The robot according to any one of claims 1 to 9. The robot according to any one of claims 1 to 10, further comprising a communication unit that transmits a captured image of the target person to a predetermined communication terminal in the watching mode.

Images