JP2023104003A - Autonomous behavior robot that behaves based on experience - Google Patents

Abstract

To provide a technique that allows a robot to select its behaviors while making memory associated with places or objects.SOLUTION: An autonomous behavior robot includes a memory unit, a processor, and a driving mechanism. The processor of the robot detects an object, recognizes impression memory of the object, and determines a motion to the object on the basis of the impression memory. The driving mechanism executes a motion determined by the processor.SELECTED DRAWING: Figure 17

Description

The present invention relates to a robot that autonomously selects actions according to its internal state or external environment.

Humans keep pets for comfort. On the other hand, many people give up pets for a variety of reasons, such as not having enough time to take care of pets, lack of living environments that allow pets, allergies, and the pain of bereavement. If there is a robot that can play the role of a pet, it may be possible to give people who cannot keep a pet the comfort that a pet gives (see Patent Documents 1 and 2).

JP-A-2000-323219 WO2017/169826

In recent years, robot technology has been rapidly progressing, but it has not yet achieved the presence of a companion like a pet. I don't think robots have free will. By observing the pet's behavior, which can only be assumed to have free will, humans feel the presence of free will in the pet, sympathize with the pet, and are healed by the pet.

One of the characteristics of living things is "memory". Memories also include associative memories that are linked to places and things. For example, by experiencing the warmth of a window,
The impression of "warm" is associated with "by the window". Alternatively, a positive image may be associated with a red coat when a person has a pleasant experience near it. Organisms form various memories based on the external environment and make action selections while being stimulated by the external environment.

The present invention has been completed based on the recognition of the above problem, and its main purpose is to provide a technique for making a robot select an action while creating a memory associated with a place or an object.

An autonomous action type robot according to one aspect of the present invention includes a storage unit, a processor, and a driving mechanism that executes motions selected by the processor.
The processor recognizes an impression memory for an object and determines motion for the object based on the impression memory.
An autonomous action robot according to another aspect of the present invention includes a motion control unit that selects a motion of the robot, a drive mechanism that executes the motion selected by the motion control unit, a captured image acquisition unit that acquires a captured image, An image feature acquisition unit that acquires image feature information by extracting feature points from the captured image, an environment information acquisition unit that acquires the environmental information of the point where the captured image was acquired, and the image feature information corresponds to the environment information. a map manager that generates an environment memory by adding
Prepare.
The motion control unit refers to the environment storage and sets a point associated with environment information that satisfies a predetermined destination condition as a movement target point.

An autonomous action robot according to another aspect of the present invention includes a motion control unit that selects a motion of the robot, a drive mechanism that executes the motion selected by the motion control unit, a captured image acquisition unit that acquires a captured image, An image feature acquisition unit that acquires image feature information by extracting a feature amount of a captured image; The primary environment information associated with the image feature information whose degree of similarity between the primary environment information acquisition unit to be associated and registered and the newly acquired first image feature information is equal to or greater than a predetermined threshold is selected as the first and a secondary environment information acquiring unit for specifying as secondary environment information corresponding to the image feature information.
The motion control unit changes the behavioral characteristics of the robot based on both the primary environmental information and the secondary environmental information.

An autonomous action robot according to another aspect of the present invention includes a motion control unit that selects a motion of the robot, a drive mechanism that executes the motion selected by the motion control unit, a captured image acquisition unit that acquires a captured image, An image feature acquisition unit that extracts the feature amount of the captured image, a primary environment information acquisition unit that acquires the primary environment information at the point where the captured image was acquired, and a predetermined a secondary environment information acquisition unit that acquires environmental information output from the inference model as secondary environment information by inputting feature values of newly acquired captured images for the inference model; Prepare.
The motion control unit changes the behavioral characteristics of the robot based on both the primary environmental information and the secondary environmental information.

According to another aspect of the present invention, an autonomous action robot includes a motion control unit that selects a motion of the robot, a drive mechanism that executes the motion selected by the motion control unit, an object detection unit that detects an object, and an object that: An environment information acquisition unit that acquires environment information at the time of detection, and an object management unit that generates environment storage by associating the object with the environment information.
The motion control unit changes behavioral characteristics based on environmental memory.

According to another aspect of the present invention, an autonomous action robot includes a motion control unit that selects a motion of the robot, a drive mechanism that executes the motion selected by the motion control unit, an object detection unit that detects an object, and an object that: a state manager that changes the emotion parameter when detected.
The motion control unit changes the behavioral characteristics of the robot according to the emotion parameter.

An autonomous action robot according to another aspect of the present invention includes a motion control unit that selects a motion of the robot, a driving mechanism that executes the motion selected by the motion control unit, an object detection unit that detects a plurality of objects, an environment information acquisition unit that acquires environment information at the time when a plurality of objects are detected;
a state manager for setting impression storage for the plurality of objects based on the environment information.
The state management unit changes the emotion parameter based on the impression memory associated with the plurality of objects when the plurality of objects is re-detected,
The motion control unit changes the behavioral characteristics of the robot according to the emotion parameter.

ADVANTAGE OF THE INVENTION According to this invention, it becomes easy to implement|achieve action selection accompanying memory generation and memory in a robot.

1 is a conceptual diagram for explaining how a robot forms memories based on location; FIG. FIG. 2(a) is a front external view of the robot. FIG. 2(b) is a side external view of the robot. It is a cross-sectional view schematically showing the structure of the robot. It is a hardware block diagram of the robot in a basic configuration. 1 is a functional block diagram of a robot system; FIG. It is a functional block diagram of the robot in this embodiment. FIG. 4 is a schematic diagram showing a method of creating key frames; 4 is a data structure diagram of environment map information; FIG. 4 is a data structure diagram of object information; FIG. FIG. 4 is a schematic diagram showing a chain of key frames; 4 is a flow chart showing a process of updating environment map information; FIG. 4 is a schematic diagram for explaining how the robot selects a destination when an event occurs; 4 is a flow chart showing an event processing process; FIG. 12 is a flow chart showing the details of the object determination process shown in S28 of FIG. 11; FIG. FIG. 11 is a data structure diagram of object environment information in a modified example; FIG. 11 is a data structure diagram of room impression information in a modified example; FIG. 10 is a flow chart showing a process when an object is detected in a modified example; FIG. It is a data structure diagram of action experience information in another modification.

FIG. 1 is a conceptual diagram for explaining how the robot 100 forms memories based on location.
The robot 100 of this embodiment is equipped with a small camera. robot 100
is a large number of captured images by periodically capturing images of the surroundings with this camera.
(still image). Also, the robot 100 acquires environmental information using various sensors such as a microphone and a temperature sensor. Environmental information is a set of values detected by various sensors, such as sound, odor, received strength of radio waves, temperature, presence or absence of a user, and the like. The robot 100 forms a memory based on captured images (hereinafter referred to as "image memory") and a memory based on environmental information (hereinafter referred to as "environmental memory"). By associating these two types of memories, a memory for the location of the robot 100 (hereinafter referred to as "location memory") is formed.

An image storage layer 400 in FIG. 1 is a storage layer conceptually representing image storage, and an environmental storage layer 402 is a storage layer conceptually representing environmental storage. An image store is a collection of keyframes 404 . A key frame 404 is distribution information of feature points (feature amounts) in the captured image. The robot 100 of this embodiment is a graph-based SLAM (Simultaneous Localization and Analysis) using image features.
Mapping) technology, more specifically ORB (Oriented FAST and Rotated BRI
EF) A key frame 404 is formed by the SLAM technique based on the feature amount (details will be described later).

By periodically forming key frames 404 while moving, the robot 100 forms an image storage layer 400 as a collection of key frames 404, in other words, an image feature distribution. The robot 100 acquires a keyframe 404 at the current location and a number of keyframes 404 already possessed.
By comparing (image storage layer 400), it is possible to grasp at which point in the image storage layer 400 the object itself exists. In other words, the robot 100 compares the captured image that is actually viewed with the captured image that was previously viewed (memory), and performs "space recognition" by matching its own current situation with its past memory.

When acquiring the captured image, the robot 100 also acquires environmental information. Ambient memory is a collection of environmental information at each point. The robot 100 selects a movement direction based on the environment information (environment storage layer 402). For example, when the robot 100 is at a hot spot P1, it chooses to move to a cool spot with an average temperature of 20 degrees or less. Assume that the robot 100 first searches the environment storage layer 402 for environment information that satisfies the above conditions, and finds that the environment information of the point P2 satisfies the above conditions. Next, robot 100 identifies keyframe 404 corresponding to point P2. The robot 100 specifies the movement route to the point P2 while referring to the image storage layer 400 (distribution of the keyframes 404).
With the control method described above, the robot 100 moves toward the point P2 that satisfies the condition that the average temperature is 20 degrees or less.

Hereinafter, after explaining the basic configuration of the robot 100 with reference to FIGS. 2 to 5, the method of forming and using memory in this embodiment will be mainly explained.
It should be noted that the robot 100 in the basic configuration is premised on recognizing the position not by the keyframe 404 but by an external sensor. The keyframe 404 and an external sensor may be used together.

[Basic configuration]
FIG. 2(a) is a front external view of the robot 100. FIG. FIG. 2(b) is a side external view of the robot 100. FIG.
The robot 100 in this embodiment is an autonomous action type robot that determines actions based on the external environment and internal state. The external environment is recognized by various sensors such as cameras and thermosensors. The internal state is quantified as various parameters that express the robot's 100 emotions. robot 100
The range of action is the house of the owner's family. A person involved in the robot 100 is hereinafter referred to as a "user".

A body 104 of the robot 100 has an overall rounded shape and includes an outer skin made of a soft and elastic material such as urethane, rubber, resin, or fiber. The robot 100 may be dressed. The robot 100 has a total weight of about 5 to 15 kilograms and a height of about 0.5 to 1.2 meters.
Various attributes such as appropriate weight, roundness, softness, and good touch realize the effect that the user can easily hold the robot 100 and want to hold it.

The robot 100 includes a pair of front wheels 102 (left wheel 102a, right wheel 102b) and one rear wheel 103. As shown in FIG. The front wheels 102 are driving wheels and the rear wheels 103 are driven wheels. The front wheels 102 do not have a steering mechanism, but their rotational speed and rotational direction can be individually controlled. The rear wheels 103 are casters and the robot 1
It is rotatable to move 00 forward, backward, left and right. Rear wheel 103 may be an omni wheel.

The front wheels 102 and rear wheels 103 can be completely housed in the body 104 by the drive mechanism (rotating mechanism, link mechanism). Most of the wheels are hidden by the body 104 even during running, but when the wheels are completely housed in the body 104, the robot 100 becomes immobile. That is, the body 104 descends and seats on the floor surface F as the wheels are retracted. In this seated state, the flat seating surface 108 (grounding bottom surface) formed at the bottom of the body 104 is on the floor surface F.
abut.

Robot 100 has two hands 106 . The hand 106 does not have the function of grasping an object. The hand 106 can be raised, shaken, or vibrated for simple actions. The two hands 106 are also individually controllable.

The eye 110 is capable of image display using a liquid crystal element or an organic EL element.
The robot 100 is equipped with various sensors such as a microphone array and an ultrasonic sensor that can identify the direction of a sound source. It also has a built-in speaker that can emit simple sounds.

A horn 112 is attached to the head of the robot 100 . Since the robot 100 is lightweight as described above, the user can lift the robot 100 by grabbing the horns 112 . An omnidirectional camera is attached to the horn 112, and the entire upper area of the robot 100 can be imaged at once.

FIG. 3 is a cross-sectional view schematically showing the structure of robot 100. As shown in FIG.
As shown in FIG. 3, the body 104 of the robot 100 is attached to the base frame 3
08, includes a body frame 310, a pair of resin wheel covers 312 and an outer skin 314. The base frame 308 is made of metal, configures the axis of the body 104, and supports the internal mechanism. The base frame 308 includes an upper plate 332 and a lower plate 334 connected to a plurality of side plates 33
6 are connected vertically. Sufficient spacing is provided between the side plates 336 to allow for ventilation. Inside the base frame 308, a battery 118, a control circuit 342 and various actuators are accommodated.

Body frame 310 is made of a resin material and includes head frame 316 and body frame 318 . The head frame 316 has a hollow hemispherical shape and forms the head skeleton of the robot 100 . The torso frame 318 has a stepped cylindrical shape and forms the torso skeleton of the robot 100 . A torso frame 318 is fixed integrally with the base frame 308 . The head frame 316 is the torso frame 3
It is attached to the upper end of 18 so as to be relatively displaceable.

The head frame 316 has a yaw axis 320, a pitch axis 322 and a roll axis.
Three axes 324 and an actuator 326 for rotating each axis are provided. Actuator 326 includes multiple servo motors to drive each axis individually. The yaw axis 320 is driven for swing motion, the pitch axis 322 is driven for nod motion, and the roll axis 324 is driven for tilt motion.

A plate 325 that supports the yaw axis 320 is provided on the top of the head frame 316.
is fixed. A plurality of ventilation holes 327 are formed in the plate 325 to ensure ventilation between the upper and lower sides.

A metal base plate 328 is provided to support the head frame 316 and its internal mechanisms from below. The base plate 328 is connected to the plate 325 via a cross link mechanism 329 (pantograph mechanism), and is connected via a joint 330 to an upper plate 332 (base frame 308).

Torso frame 318 houses base frame 308 and wheel drive mechanism 370 . The wheel drive mechanism 370 includes a pivot 378 and an actuator 379
including. A lower half portion of the body frame 318 is narrowed to form a storage space S for the front wheel 102 between the wheel cover 312 and the wheel cover 312 .

Outer skin 314 is made of urethane rubber and covers body frame 310 and wheel cover 312 from the outside. Hand 106 is integrally molded with skin 314 . The upper end of the outer skin 314 is provided with an opening 390 for introducing outside air.

FIG. 4 is a hardware configuration diagram of the robot 100. As shown in FIG.
The robot 100 includes an internal sensor 128, a communication device 126, a storage device 124,
Includes processor 122 , drive mechanism 120 and battery 118 . Processor 122 and storage device 124 are included in control circuitry 342 . Each unit is connected to each other by a power line 130 and a signal line 132 . battery 11
8 supplies power to each unit via the power line 130 . Each unit transmits and receives control signals via signal line 132 . The battery 118 is a lithium ion secondary battery and is the power source of the robot 100 .

The internal sensor 128 is a collection of various sensors built into the robot 100 . Specifically, they are cameras (omnidirectional cameras), microphone arrays, ranging sensors (infrared sensors), thermosensors, touch sensors, acceleration sensors, odor sensors, and the like. A touch sensor is installed between the skin 314 and the body frame 310 to detect the user's touch. An odor sensor is a known sensor that applies the principle that electrical resistance changes due to the adsorption of odor-causing molecules.

The communication device 126 is a communication module that performs wireless communication with various external devices. The storage device 124 is composed of non-volatile memory and volatile memory, and stores computer programs and various setting information. processor
122 is a computer program execution means. The drive mechanism 120 is
It includes a plurality of actuators and the wheel drive mechanism 370 described above. It also has a display and speakers.

The drive mechanism 120 mainly includes wheels (front wheels 102) and a head (head frame).
316). The drive mechanism 120 can change the movement direction and movement speed of the robot 100, and can also move the wheels (the front wheels 102 and the rear wheels 103) up and down. When the wheels are raised, they are fully retracted into the body 104 and
The robot 100 comes into contact with the floor surface F on the seating surface 108 and is in a seated state. Drive mechanism 120 also controls hand 106 via wire 134 .

FIG. 5 is a functional block diagram of the robot system 300. As shown in FIG.
Robotic system 300 includes robot 100 , server 200 and a plurality of external sensors 114 . Each component of the robot 100 and the server 200 includes computing units such as a CPU (Central Processing Unit) and various coprocessors, storage devices such as memory and storage, hardware including wired or wireless communication lines connecting them, and storage. It is implemented by software that is stored in the device and that supplies processing instructions to the calculator. the computer program
It may be composed of a device driver, an operating system, various application programs located in their upper layers, and a library that provides common functions to these programs. Each block described below represents a functional block rather than a hardware configuration.
A part of the functions of the robot 100 may be realized by the server 200,
A part or all of the functions of the server 200 may be realized by the robot 100. FIG.

A plurality of external sensors 114 are installed in advance in the house. server 20
In 0, the position coordinates of the external sensor 114 are registered. Based on the information obtained from the internal sensor 128 of the robot 100 and the plurality of external sensors 114,
Server 200 determines the basic behavior of robot 100 . The external sensor 114 is for reinforcing the sensory organs of the robot 100 and the server 200 is for reinforcing the brain of the robot 100 . Robot 100 Communicator 1
26 periodically communicates with the external sensor 114 and the server 200 communicates with the external sensor 11
4 identifies the position of the robot 100 (see also Patent Document 2).

(server 200)
The server 200 includes a communication unit 204, a data processing unit 202 and a data storage unit.
Including 206.
The communication unit 204 is in charge of processing communication with the external sensor 114 and the robot 100 . A data storage unit 206 stores various data. Data processing unit 20
2 executes various processes based on the data acquired by the communication unit 204 and the data stored in the data storage unit 206 . Data processing unit 202 also functions as an interface for communication unit 204 and data storage unit 206 .

The data storage unit 206 includes the motion storage unit 232 and the personal data storage unit 21.
Including 8.
The robot 100 has a plurality of action patterns (motions). hand 10
Various motions are defined, such as shaking the 6, approaching the owner while meandering, and staring at the owner while tilting the neck.

The motion storage unit 232 stores a “motion file” that defines the content of motion control. Each motion is identified by a motion ID. Motion files are also downloaded to the motion store 160 of the robot 100 . Which motion to perform may be determined by the server 200 or may be determined by the robot 100 .

Most of the motions of the robot 100 are configured as compound motions including multiple unit motions. For example, when the robot 100 approaches the owner, it may be expressed as a combination of a unit motion of turning toward the owner, a unit motion of approaching while raising hands, a unit motion of approaching while shaking the body, and a unit motion of sitting while raising both hands. . like this
By combining four motions, the motion of "approaching the owner, raising a hand in the middle, finally shaking the body and sitting down" is realized. In the motion file, the rotation angles and angular velocities of the actuators provided in the robot 100 are defined in association with the time axis. Various motions are expressed by controlling each actuator over time according to a motion file (actuator control information).

The transition time when changing from the previous unit motion to the next unit motion is called "interval". The interval may be defined according to the time required to change the unit motion and the contents of the motion. The interval length is adjustable.
Hereinafter, settings related to action control of the robot 100, such as when and which motion to select, output adjustment of each actuator for realizing the motion, etc., are collectively referred to as "behavior characteristics". The behavioral characteristics of the robot 100 are defined by motion selection algorithms, motion selection probabilities, motion files, and the like.

The motion storage unit 232 stores motion files as well as motion selection tables that define motions to be executed when various events occur. In the motion selection table, one or more motions and their selection probabilities are associated with events.

Personal data storage unit 218 stores user information. Specifically, it stores master information indicating familiarity with the user and the user's physical features/behavioral features. Other attribute information such as age and gender may be stored.

The robot 100 has an internal parameter of intimacy for each user.
When the robot 100 recognizes an action showing goodwill toward itself, such as picking it up or calling out to the user, the degree of intimacy with the user increases.
Intimacy is low for users who do not interact with the robot 100, users who behave violently, and users who do not meet frequently.

The data processing unit 202 includes a position management unit 208, a recognition unit 212, an operation control unit 2
22, includes a degree of familiarity manager 220 and a state manager 244;
The position management unit 208 identifies position coordinates of the robot 100 . State management part
244 manages various internal parameters such as charging rate, internal temperature, and various physical states such as processing load of processor 122 . In addition, the state management unit 244 manages various emotion parameters indicating the emotions of the robot 100 (loneliness, curiosity, desire for approval, etc.). These emotional parameters are constantly fluctuating. The movement target point of the robot 100 changes according to the emotion parameter. for example,
When the loneliness is increasing, the robot 100 sets the location of the user as the movement target point.

Emotional parameters change over time. In addition, various emotion parameters change depending on the later-described response behavior. For example, from the owner "hug"
When the owner is not seen for a long time, the emotional parameter indicating loneliness decreases, and the emotional parameter indicating loneliness increases little by little.

The recognition unit 212 recognizes the external environment. Recognition of the external environment includes various recognitions such as recognition of weather and seasons based on temperature and humidity, and recognition of shadows (safe zones) based on light intensity and temperature. The recognition unit 156 of the robot 100 uses the internal sensor 1
28 acquires various environmental information, performs primary processing on them, and then sends them to the server 200
is transferred to the recognition unit 212 of the

Specifically, the recognition unit 156 of the robot 100 extracts an image region corresponding to a moving object, particularly a person or an animal, from the image, and extracts features representing physical and behavioral features of the moving object from the extracted image region. Extract a "feature vector" as a set of quantities. A feature vector component (feature amount) is a numerical value obtained by quantifying various physical/behavioral features. For example, the width of the human eye is quantified in the range of 0 to 1,
form one feature vector component. A technique for extracting feature vectors from a captured image of a person is an application of known face recognition technology. Robot 100 sends the feature vector to server 200 .

The recognition unit 212 of the server 200 compares the feature vector extracted from the image captured by the built-in camera of the robot 100 with the feature vector of the user (cluster) registered in advance in the personal data storage unit 218,
It is determined which person the imaged user corresponds to (user identification processing).
In addition, the recognition unit 212 estimates the user's emotion by performing image recognition of the user's facial expression. The recognition unit 212 also performs user identification processing on moving objects other than people, such as pets such as cats and dogs.

The recognition unit 212 recognizes various behaviors performed by the robot 100 and classifies them into pleasant and unpleasant behaviors. The recognition unit 212 also recognizes the owner's response to the action of the robot 100 and classifies it into positive and negative reactions.
Pleasant/unpleasant behavior is determined by whether the user's response behavior is biologically pleasant or unpleasant. For example, being hugged is a pleasant action for the robot 100, and being kicked is an unpleasant action for the robot 100. A positive/negative reaction is determined depending on whether the user's behavior indicates a user's pleasant feeling or an unpleasant feeling. Being hugged is a positive reaction that indicates a user's pleasant feeling, and being kicked is a negative reaction that indicates a user's unpleasant feeling.

The motion control unit 222 of the server 200 controls the motion control unit 150 of the robot 100
determines the motion of the robot 100. The motion control unit 222 of the server 200 creates a movement target point for the robot 100 and a movement route therefor. The motion control unit 222 may create a plurality of travel routes and then select one of the travel routes.

The motion control unit 222 selects a motion for the robot 100 from multiple motions in the motion storage unit 232 . Each motion is associated with a selection probability for each situation. For example, when the owner performs a pleasant act, motion A is executed with a probability of 20%, and when the temperature exceeds 30 degrees, motion B is executed with a probability of 5%. .

The familiarity management unit 220 manages the familiarity of each user. As described above, the degree of intimacy is registered as part of the personal data in the personal data storage unit 218. FIG. When a pleasant act is detected, the degree of intimacy management unit 220 increases the degree of intimacy with the owner. The degree of intimacy is lowered when an unpleasant act is detected.
In addition, the intimacy level of the owner who has not seen it for a long time gradually decreases.

(Robot 100)
The robot 100 includes a communication unit 142, a data processing unit 136, a data storage unit 1
48, including internal sensors 128 and drive mechanism 120;
The communication unit 142 corresponds to the communication device 126 (see FIG. 4), and the external sensor 114
, in charge of communication processing with the server 200 and other robots 100 . The data storage unit 148 stores various data. The data storage unit 148 stores the storage device 1
24 (see Figure 4). Data processing unit 136 executes various processes based on data acquired by communication unit 142 and data stored in data storage unit 148 . The data processing unit 136 corresponds to the processor 122 and computer programs executed by the processor 122 . Data processing unit 136 also functions as an interface for communication unit 142 , internal sensor 128 , drive mechanism 120 and data storage unit 148 .

Data store 148 includes motion store 160 that defines various motions of robot 100 .
Various motion files are downloaded from the motion storage unit 232 of the server 200 to the motion storage unit 160 of the robot 100 . A motion is identified by a motion ID. Sitting with the front wheels 102 retracted, lifting the hand 106, rotating the robot 100 by rotating the two front wheels 102 in reverse, or rotating only one of the front wheels 102, with the front wheels 102 retracted. In order to express various motions such as trembling by rotating the front wheel 102 with the , stopping and looking back when moving away from the user, the operation timing, operation time, operation direction, etc. of various actuators (driving mechanism 120) are adjusted. It is defined chronologically in the motion file.
Various data may also be downloaded to the data storage unit 148 from the personal data storage unit 218 .

Data processing unit 136 includes recognition unit 156 and operation control unit 150 .
The motion control unit 150 of the robot 100 is controlled by the motion control unit 222 of the server 200.
determines the motion of the robot 100 in cooperation with Some motions may be determined by the server 200 and other motions may be determined by the robot 100 . Also, although the robot 100 determines the motion, the robot
The server 200 may determine the motion when the processing load of 100 is high. A base motion may be determined at the server 200 and an additional motion may be determined at the robot 100 . How the server 200 and the robot 100 share the motion determination process may be designed according to the specifications of the robot system 300 .

The motion control unit 150 of the robot 100 transmits the selected motion to the drive mechanism 12
Instruct 0 to execute. According to the motion file, the drive mechanism 120
Control each actuator.

The motion control unit 150 can also perform a motion of raising both hands 106 as a gesture of pleading for a hug when a user with a high degree of intimacy is nearby, or can move the left and right front wheels 102 when the user is tired of hugging. By alternately repeating reverse rotation and stopping while stored, it is possible to express a motion that does not want to be hugged. Drive mechanism 120 drives front wheels 102, hands 106, and neck (head frame 316) according to instructions from motion control unit 150, thereby causing robot 100 to express various motions.

A recognition unit 156 of the robot 100 interprets external information obtained from the internal sensor 128 . The recognition unit 156 is capable of visual recognition (visual unit), smell recognition (olfactory unit), sound recognition (auditory unit), and tactile recognition (tactile unit).

The recognition unit 156 extracts feature vectors from the captured image of the moving object. As described above, a feature vector is a set of parameters (feature amounts) representing physical features and behavioral features of a moving object. When a moving object is detected, physical characteristics and behavioral characteristics are also extracted from an odor sensor, a built-in sound collecting microphone, a temperature sensor, and the like. These features are also quantified into feature vector components. Recognition part 1
56 identifies the user from the feature vector based on the known technology described in Patent Document 2 and the like.

Among the series of recognition processes including detection, analysis, and judgment, the recognition unit of the robot 100
156 selects and extracts information necessary for recognition, and interpretation processing such as determination is executed by the recognition unit 212 of the server 200 . The recognition process may be performed by the recognition unit 212 of the server 200 alone, or may be performed by the recognition unit 156 of the robot 100 alone. good too.

When a strong impact is applied to the robot 100, the recognition unit 156 recognizes this with the touch sensor and the acceleration sensor, and the recognition unit 2 of the server 200
12 recognizes that a "violent act" has been committed by a nearby user. When the user grabs the horn 112 and lifts the robot 100, it may be recognized as a violent act. When the user facing robot 100 speaks in a specific sound volume range and a specific frequency band, recognition unit 212 of server 200
may recognize that the "calling action" to itself has been performed. Also, when a temperature of about body temperature is detected, it is recognized that the user has made a "contact action", and when an upward acceleration is detected while contact is being recognized, it is recognized that a "hold" has been made. Physical contact when the user lifts the body 104 may be sensed, or the holding may be recognized by reducing the load on the front wheels 102 .
In summary, the robot 100 acquires the user's behavior as physical information through the internal sensor 128, and the recognition unit 212 of the server 200 judges comfort/discomfort. Also, the recognition unit 212 of the server 200 executes user identification processing based on the feature vector.

The recognition unit 212 of the server 200 recognizes various responses of the user to the robot 100 . Pleasant or unpleasant, affirmative or negative are associated with a part of typical responses among various responses. In general, most of the reception behaviors that are pleasant behaviors are positive reactions, and most of the reception behaviors that are unpleasant behaviors are negative reactions. Pleasant and unpleasant actions are related to intimacy, and positive and negative reactions are related to robots10
Affects 0 action choices.

Depending on the behavior recognized by the recognition unit 156, the familiarity management unit 220 of the server 200 changes the familiarity with the user. In principle, the degree of intimacy with a user who has performed a pleasant act increases, and the degree of intimacy with a user who has performed an unpleasant act decreases.

Based on the basic configuration described above, the robot 100 in this embodiment will be described next.
will be described, especially focusing on the features and purposes of this implementation and differences from the basic configuration.

[Implementation of place and object memory]
FIG. 6 is a functional block diagram of the robot 100 in this embodiment.
The robot 100 of this embodiment recognizes the location based on the keyframe 404 instead of the external sensor 114 . Robot 100 is keyframe 40
Although the location may be recognized based on both 4 and the external sensor 114, this embodiment will be described as recognizing the location based only on the key frame 404. FIG.

(server 200)
Location manager 208 of data processor 202 includes map manager 168 and object manager 174 . The map management unit 168 manages information that associates the image storage layer 400 and the environment storage layer 402 (hereinafter referred to as "environment map" or "environment map information"). The object management unit 174 manages object information that associates events with objects. An event is a phenomenon recognized as established when environmental information satisfies a predetermined event condition (described later).

Data storage unit 206 further includes map storage unit 170 and object information storage unit 172 . The map storage unit 170 stores environment map information. The object information storage unit 172 stores object information. Although the details will be described later, the state management unit 244 changes the emotion parameter based on environmental information and events.

(Robot 100)
The data processing unit 136, in addition to the recognition unit 156 and the operation control unit 150,
A captured image acquisition unit 162 is included. A captured image acquisition unit 162 acquires a captured image from a camera mounted on the robot 100 . The captured image acquisition unit 162 acquires captured images periodically, for example, twice per second.

Recognition unit 156 includes image feature acquisition unit 164 and environment information acquisition unit 166 .
The image feature acquisition unit 164 generates a key frame 404 by extracting an image feature amount from the captured image. For the method of extracting the image feature quantity, see Fig. 7 below.
will be described in relation to The environment information acquisition unit 166 acquires environment information when acquiring a captured image. Key frame 404 (image feature information) and environment information are transmitted to server 200 by communication unit 142 . Map management unit 168 of server 200 creates an environment map by associating key frames 404 with environment information.

FIG. 7 is a schematic diagram showing a method of creating the keyframe 404. As shown in FIG.
As described above, the captured image acquisition unit 162 periodically acquires captured images. A chair 406 and a shelf 408 are shown in the captured image 412 of FIG. The image feature acquisition unit 164 extracts edge points of these objects as feature points 410 . A keyframe 404 is a set of feature points 410 . The keyframe 404 is characterized by the number and distribution of feature points 410 included in the keyframe 404, in other words, the feature amount of the captured image. Image storage based on the key frame 404 (image feature information) instead of the captured image 412 itself has the advantage of being able to reduce the amount of data.

FIG. 8 is a data structure diagram of the environment map information 420. As shown in FIG.
The environment map information 420 is stored in the map storage unit 170 of the server 200. FIG. The keyframe 404 is identified by a keyframe ID (hereinafter referred to as "FID"). The image feature acquisition unit 164 assigns an FID each time a key frame 404 is formed from the captured image. The environment information acquisition unit 166 acquires the environment information at the same timing as the acquisition of the key frame 404 (captured image) (hereinafter, environment information sensed at the same time as the acquisition of the key frame 404 is referred to as the “primary environment”). information”). The map management unit 168 associates the key frame 404 with the environment information and registers them in the environment map information 420 .
Environment map information 420 connects image storage and environment storage.

The environmental information in this embodiment includes reception intensity, user detection rate, average sound volume, and average temperature. The reception intensity indicates the reception intensity of radio waves transmitted from the server 200, and varies depending on the distance between the robot 100 and the server 200. FIG. The environment information acquisition unit 166 measures the reception strength and ranks it into A, B, and C in descending order. According to FIG. 8, the key frame 404 with FID=F01 (hereinafter referred to as "key frame 404 (F01)") has a reception strength of "A", so it corresponds to the key frame 404 (F01). The point (hereafter referred to as "point (F01
)” is a point with good communication. On the other hand, key frame 404 (F04) has a communication strength of "C", so the location (F04
) is a point with poor communication. A point in this embodiment is defined as an extended area with a radius of about 0.1 to 1 meter.

The user detection rate indicates the frequency with which users are detected at each location, in other words, the likelihood of encountering users at each location. A user detection rate is defined for each user. The user X1 detection rate indicates the detection rate of user X1. For example, when the number of times the robot 100 exists at the point (F01) is 10 times and the user X1 is detected twice in the 10 stay opportunities, the map management unit 168 sets the detection rate of the user X1 to 20(%). calculate. According to FIG. 8, the point (F04) is likely to meet users X1 and X2, and the point (F05) can meet user X2 with a high probability of nearly 50(%). On the other hand, I have never met users X1 and X2 at the point (F07).

The environmental information acquisition unit 166 detects the sound volume at each point with a microphone, and ranks the average sound volume into A, B, and C in descending order. By average volume,
Quiet and noisy areas are identified. Similarly, the environment information acquisition unit 166 detects the temperature at each point with a temperature sensor and calculates the average temperature. Average temperatures identify hot and cool places. These environmental information may be further classified by other parameters such as time of day and season.

Environmental information can also include various other sensor information. For example, the environmental information may include the smell detected by the odor sensor, or may include whether or not there is a "ceiling" such as under a table. Not only the user detection rate (visibility), but also the "probability of being touched by someone"
Experience of interaction with the user such as "probability of being held" and "probability of being hit" may be defined as environment information. The environmental information may be at least information other than the image feature information, and may be information specified as tactile, auditory, or olfactory information such as temperature and wind direction.

The server 200 refers to the environment map information 420 and sets the movement target point of the robot 100 to a point that satisfies a predetermined destination condition. For example, the value of an emotion parameter indicating loneliness (hereafter referred to as "emotion value (lonely)")
) is equal to or higher than the threshold, operation control unit 222 sets a point at which the detection rate of any user is equal to or higher than a predetermined threshold, for example, point (F05), as the movement target point. Destination conditions can be set arbitrarily. For example, when the temperature at the current location is 25 degrees or higher, the destination condition may be satisfied at a location having environmental information with an average temperature of 20 degrees or lower.

The state management unit 244 stores the first
The emotion parameter is changed based on the next environment information (hereinafter referred to as "primary reflection (by environment information)"). The state management unit 244 also collects environmental information acquired in the past at the current location of the robot 100 (hereinafter, the environment information sensed before the acquisition of the key frame 404 is referred to as "secondary environment information"). (hereinafter referred to as "secondary reflection (by environmental information)"). In other words, the state management unit 244 changes the emotion parameter based on not only the current primary environment information but also the stored past secondary environment information. Changes in emotion parameters based on environmental information will be described in detail with reference to FIG.

FIG. 9 is a data structure diagram of the object information 430. As shown in FIG.
The object information 430 is stored in the object information storage unit 17 of the server 200.
2 is stored. Various events occur around the robot 100 . Recognition unit 212 of server 200 determines that an event has occurred when the primary environment information satisfies a predetermined event condition. The event condition can be arbitrarily set, such as seeing the user, being held by the user, hearing a sound of a predetermined volume or more, or being splashed with water. A plurality of event conditions are set in the recognition unit 212 in advance. Events are identified by event IDs.

When the recognition unit 212 detects the occurrence of an event, the communication unit 204 detects the occurrence of an event.
Send an event detection signal to 100. The image feature acquisition unit 164 detects an object from the latest captured image when an event detection signal is received.
Objects can be arbitrarily set, such as coffee cups, chairs, shelves, and dolls. In this embodiment, the image feature acquiring unit 164 has a plurality of object patterns that define features of shape and color in advance for a plurality of types of objects, and performs image recognition of the object based on the object patterns. When any object is specified, the image feature acquiring unit 164 assigns an object ID and transmits it to the server 200 . The object management unit 174 associates event IDs with object IDs. Object recognition can be realized by applying known image recognition technology.

The object management unit 174 manages object information 430. FIG. In the object information 430, a degree of association is defined as an index value indicating the strength of association between an event and an object. The degree of relevance between an event (E01) and an object (B01) is based on the likelihood that the object (B01) will be recognized when the event (E01) occurs. It is an index that expresses the strength of associativity and causality between

When the object (B01) is detected when the event (E01) occurs, the object management unit 174 stores the event (E01) and the object (B01)
) increases the relevance of On the other hand, if the object (B01) is not detected when the event (E01) occurs, the object management unit 174 lowers the degree of association between the event (E01) and the object (B01). In this embodiment, the degree of association is defined as the detection rate of an object when an event occurs. For example, when the object (B01) is detected 3 out of 10 times in the last 10 occurrences of the event (E01), the object management unit 174 sets the degree of association between the event (E01) and the object (B01) to 30.
(%).

Object (B01) with high probability when event (E01) occurs
is detected, the event (E0
It is possible to assume "causality" that 1) is likely to occur. It can also be said that the degree of association is a parameter indicating the degree of certainty for causality. For example, assume that a sofa (object) is detected with high probability at the point where the person is held. In this case, it is possible to predict that if you go near the sofa, there is a high possibility that you will be hugged.

The state management unit 244 changes the emotion parameter based on the event (
Hereinafter referred to as “primary reflection (by event)”). Since an event is also a kind of primary environmental information, the primary reflection by an event usually occurs simultaneously with the primary reflection by the primary environmental information. The state management unit 244 also changes the emotion parameter based on an event associated with the object detected at the current point of the robot 100 (hereinafter referred to as "secondary reflection (by event)"). The state management unit 244 changes emotion parameters not only by events that actually occur, but also by events that are associated with objects.
Event-based affective parameter changes are discussed in more detail in connection with FIGS.

FIG. 10 is a schematic diagram showing a chain of key frames 404. As shown in FIG.
The image feature acquisition unit 164 acquires key frames 404 periodically. Figure 1
At 0, the environment information acquisition unit 166 acquires the environment information (e01) when the key frame 404 (F01) is acquired. The map management unit 168 adds the key frame 404 (F01) and the environment information (
e01) is associated and registered. After a certain period of time, robot 100 acquires key frame 404 (F02) and environmental information (e02). The map management unit 168 associates and registers the key frame 404 (F02) and the environment information (e02), and also records information that the point (F01) and the point (F02) are adjacent to each other.

After obtaining key frame 404 (F02), robot 100 further obtains key frame 404 (F03). The map management unit 168 stores the point (F0
2) is registered in the map storage unit 170 as information that the point (F03) is adjacent. With the above processing, the robot system 300 learns the positional relationship of the keyframe 404 that the point (F03) can be reached from the point (F01) via the point (F02), in other words, the structure of the space. .

After keyframe 404 (F03), keyframe 404 (F04),
Assume that key frame 404 (F05) and key frame 404 (F06) are obtained. The map manager 168 detects keyframe 404 (F02) similar to keyframe 404 (F06). Specifically, keyframe 4
04 (F06) and the key frame 404 (F02) are calculated for the number and distribution of the number of feature points 410, and the similarity is determined when the similarity is equal to or greater than a predetermined threshold. The map management unit 168 determines that the keyframe 404 (F02) and the keyframe 404 (F06) are keyframes 404 at the same point. The map management unit 168 updates the keyframe 404 (F02) with the keyframe 404 (F06) by a method described later. Also, the map management unit 168 recognizes that the key frame 404 (F05) and the key frame 404 (F02) are adjacent. In this way, the positional relationships of multiple keyframes 404 are registered in the map storage unit 170 as a graph database.

FIG. 11 is a flow chart showing the process of updating the environment map information 420. As shown in FIG.
The update process shown in FIG. 11 is periodically executed in the robot system 300. As shown in FIG. First, the captured image acquisition unit 162 acquires a captured image (S10).
The image feature acquisition unit 164 extracts an image feature amount from the captured image and obtains a key frame.
Generate 404 (S12). The environment information acquisition unit 166 acquires (primary) environment information (S14). The state management unit 244 changes the emotion parameter according to the environmental information (S16). This is the "primary reflection by environmental information" described above. It is possible to arbitrarily set how the emotion parameter is changed according to the environmental information. For example, if the user is not detected in the captured image and the user undetected state continues for a predetermined time or longer, the emotional value (loneliness) may be increased. At this time, the recognition unit 212 sets a point having environmental information that alleviates loneliness as the movement target point. The motion control unit 150 may reduce the action selection frequency of the robot 100 when the emotional value (loneliness) increases, or may select a specific motion with a high probability. Environmental information influences emotional parameters, and emotional parameters influence behavioral characteristics.

The emotion value (loneliness) may be lowered when the user hugs the robot.
The emotion value (curiosity) may be increased when the volume within a predetermined range is detected. As described above, the intimacy management unit 220 may change the intimacy as well as the emotion value.

The recognition unit 212 determines whether there is a key frame 404 whose similarity to the key frame 404 extracted in S12 is equal to or greater than a predetermined value, or in other words, a point that the current location has already visited (hereinafter referred to as a "revisited point"). (S18). If it is a revisited point (Y of S18), the map management unit 168 updates the key frame 404 (image feature information) of the revisited point.
(S20).

In the captured image, there are things that exist stably, such as bookshelves and sofas (hereafter referred to as "stable features"), and things that only exist temporarily, such as coffee cups and documents. (hereinafter referred to as "unstable features"). Map management part 1
68 is the first keyframe captured at a revisited point (F05)
404 (F05-1) and the second captured keyframe 404 (F05-
2) is compared. Keyframe 404 (F05-1) and Keyframe 404
(F05-2) is almost the same as a whole, but there is a difference in the part of the unstable feature. For example, keyframe 404 (F05-1) contains image features of notes (unstable features), keyframe 404 (F05
-2) may not include image features of the note. This is the location
This is because the notebook was removed between P5's first and second visits. At this time, the map management unit 168 selects the key frame 404 of the point (F05).
Exclude feature points 410 indicating unstable features from (F05). With such a control method, a key frame 404 containing only stable features, in other words, a key frame 404 containing only image feature amounts that are likely to be used as clues for spatial recognition, is obtained.
can be formed.
As a simpler method, the map management unit 168 may overwrite the previous keyframe 404 with the latest acquired keyframe 404 as the revisited point keyframe 404 .

The state management unit 244 changes the emotion parameter based on the (secondary) environment information previously associated with the revisited point (S22). This is the above-mentioned "secondary reflection by environment information". For example, environment information (e05-1) is associated with location (F05), and robot 100 is located at location (F05).
It is assumed that environment information (e05-2) is obtained when revisiting . The state management unit 244 changes the emotion parameter based on not only the primary environmental information (e05-2) (S16) but also the previous secondary environmental information (e05-1).
According to such a control method, the impression obtained in the past at the revisited point can be reflected in the current emotion. The state management unit 244 updates the environment map information 420 with the environment information (e05-2) newly acquired by the revisit (
S24).

As an example, site (F07) is a "noisy site" with a large average volume.
(See Figure 8). Assume that when the robot 100 revisits the point (F07), no loud noise is heard. In this case, the state management unit 244 increases the emotional value (curiosity), for example, based on the past information "noisy spot". At the same time, the map management unit 168 decreases the average volume, which is a type of environment information for the point (F07).

On the other hand, if it is not a revisited point (N of S18), the recognition unit 212 determines that it is a newly discovered point, and the state management unit 244 registers the newly acquired key frame 404 and environment information in the environment map information 420 (S26 ). After that, object determination processing is executed (S28). Details of the object determination process will be described later with reference to FIG.

For example, assume that at time t1 there is a sofa at point (F04), then the sofa is moved, and at time t2 there is no sofa at point (F04). In this case, the key frame 404 (
F04-1) and key frame 404 (F04-
2) can be so different that they cannot be considered identical. In this case, the robot 100 moves to the point (F04) based on the image memory that the sofa exists, but cannot find the image-stored point (F04) because the sofa does not exist. Although the point (F04) is revisited, the point (F04) cannot be found. At this time, the recognition unit 212 recognizes that the newly discovered point has been reached (N of S18), and a key frame without a sofa is generated. 404 is newly registered in the environment map information 420.

The map management unit 168 stores key frames 40 after a predetermined time or longer has passed since acquisition.
By deleting 4, you may erase the memory of places you haven't revisited in a while. According to such a control method, forgetting of the robot 100 can be expressed, and it is also effective in suppressing the amount of data to be stored in the map storage unit 170. FIG. Also, the old and useless information that "there is a sofa" at the point (F04) can be efficiently removed.

FIG. 12 is a schematic diagram for explaining how the robot 100 selects a destination when an event occurs.
In FIG. 12, when the robot 100 is at the point P5, the neighboring point P
Event E3 occurs at 6. Event E3 is "user X6 is at point P
to exist in 6”. The recognition unit 212 of the server 200 recognizes the user
User X6 is determined to be an unknown user when no user having physical characteristics similar to X6 is remembered. At this time, the operation control unit 2 of the server 200
22 sets a point that satisfies the evacuation destination condition as a movement target point. The save destination condition can be set arbitrarily. Specifically, it may be a point where the appearance probability of a user with a degree of intimacy equal to or greater than a predetermined value is equal to or greater than a predetermined value, or may be a point having a shield such as a ceiling or a wall. The operation control unit 222 associates an event with a save destination condition in advance, and detects from the environment map information 420 a point that satisfies the save destination condition when an event occurs. In FIG. 12, a point P7 and a point P8 that satisfy the evacuation destination conditions are specified.

Operation control unit 222 selects either point P7 or point P8 as the movement target point. In FIG. 12, the motion control unit 222 selects the point P7 closer to the point P5 as the movement target point. For example, point P7 is assumed to be a point with a high detection rate for user X2 with a high degree of intimacy. As mentioned above,
The robot 100 moves to the point P7 while tracing the image memory. robot
100 moves to point P7 and stays at point P7 for a while if it can detect user X2 as expected. If the user X2 cannot be detected even after moving to the point P7, the operation control unit 222 changes the setting from the point P7 to the point P8 as the movement target point. According to such a control method, "the user is surprised by the event E3,
It is possible to cause the robot 100 to express a perplexed behavior of moving to a point P7 in search of X2, and moving to another safe point P3 because the user X2 is not there.

When a specific event occurs, the motion control unit 222 sets a point that satisfies the evacuation destination conditions as the movement target point as described above. State management unit 24
4 changes emotional parameters according to events. For example, when a high-frequency voice is detected for a predetermined time or longer, event E4 is established and the state management unit 2
44 may increase the emotional value (curiosity) in response to event E4. The environment information is a numerical representation of the external environment using detection values of various sensors. An event is a phenomenon that satisfies a predetermined event condition among environmental information, in other words, a special example of environmental information.

FIG. 13 is a flow chart showing the event processing process.
Each time the (primary) environment information is detected in S14 of FIG.
212 determines whether the event condition is met. The event processing shown in FIG. 13 is executed when an event, in other words, the (primary) environment information corresponds to one of the preset event conditions.

When an event is detected, the captured image acquisition section 162 acquires a peripheral image (S30). Peripheral images in this embodiment refer to images in two or more directions.
2 or more captured images. The captured image acquisition unit 162 normally acquires one captured image with a camera targeting one direction, and when an event occurs, acquires a surrounding image with an omnidirectional camera capable of capturing a wide-angle image of the surroundings of the robot 100. may be obtained.

The image feature acquisition unit 164 extracts image features from the surrounding images, in other words, from each of the plurality of captured images (S32). The state management unit 244 changes the emotion parameter according to the event (S34). This is the "primary reflection by event" described above. How the emotion parameter is to be changed is set in advance for each event.

Next, the image feature acquisition unit 164 detects an object from the image features of the peripheral image (S36). If the object exists (Y of S36), the map management unit 168 changes the degree of association between the event and the object (S38).
The degree of association between the event (E01) and the object (B01) is determined by the frequency with which the object (B01) is detected when the event (E01) is detected. When the event (E01) is detected, if the object (B01) is detected with high frequency, the event (E0
1) becomes easier to associate. If the object does not exist (N of S36)
, S38 is skipped.

When the event is a retreat event, in other words, if it is a specific event to escape from the current location (Y of S40), motion control unit 222 sets a location that satisfies the evacuation destination condition as the movement target location (S42). . If it is not a save event, the process of S42 is skipped.

FIG. 14 is a flow chart showing the details of the object determination process shown in S28 of FIG.
The image feature acquisition unit 164 detects an object from the captured image acquired in S10 (S50). The state management unit 244 refers to the image storage and determines whether or not the same object has been detected in the past (S52).
When an object that is the same as an object that has been detected in the past, or more precisely, an object with a degree of similarity greater than or equal to a predetermined value is detected (Y in S52), the state management unit 244
changes the emotion parameter according to the object (S54). this is,
This is the above-mentioned “secondary reflection by event”. When no object is detected or when a new object is detected (N of S52), S54 is skipped.

The state manager 244 manages events associated with objects, and
Emotion parameters are changed according to the degree of association between objects and events. For example, an object (B01) has an event (E03) and an event (
E04) and is associated with an object (B01) and an event (
E03) has a relevance of 20(%), and the object (B01) and the event (E0
Assume that the relevance of 4) is 3%. At this time, the state management unit 244 changes the emotion value corresponding to the event (E03) with the highest degree of relevance. For example, if the event (E03) is an event set to "increase the emotion value (approval request) by 30%", the state management unit 244 considers the degree of relevance of 20%, ) by 6% (=30% x 20%).
According to this control method, even if the event (E03) is not actually occurring, the memory of the event (E03) is stimulated by the object (B01) associated with the event (E03). Emotional parameters are affected, and changes in emotional parameters can be expressed as changes in behavioral characteristics.

The robot 100 and the robot system 300 including the robot 100 have been described above based on the embodiment.
Organisms are thought to recognize space by accumulating environmental information obtained from sensory organs, especially eyes, as memory. According to this embodiment, spatial recognition can be performed in a manner similar to living things. Since the external sensor 114 exemplified as the basic configuration is not an essential component, there is no need to install the external sensor 114 in advance in the planned action range of the robot 100 .

Robot 100 acts based on keyframe 404 . For this reason,
When the internal state of the space to which the robot 100 belongs changes, for example, when the position of an object such as a desk or chair changes, the robot 100 may be temporarily unable to recognize the space. On the other hand, the image memory is updated by newly obtaining the key frame 404 in the new environment. For this reason, it is possible to realize a behavioral characteristic similar to that of a living thing, in which the animal is temporarily puzzled by changes in the environment but eventually adapts.

When the robot 100 enters a space for the first time, it accumulates key frames 404 while moving around and forms a place memory (sense of territory). Keyframe 404 is stored in map storage 170 of server 200 . Map storage unit 170 is preferably formed in a non-volatile recording medium such as a hard disk. Each time the robot 100 is powered on, there is no need to walk around the space and recreate the place memory.

The robot 100 changes its emotional parameters according to environmental information, and changes its behavioral characteristics according to the emotional parameters. In addition, the robot 100 changes the emotion parameter based on not only the (primary) environment information acquired at the "current location/current time point" but also the (secondary) environment information at the "current location/past time point." . According to such a control method, behavioral characteristics change according to past memories, so changes in emotions based on place memories can be expressed in behavior.
Further, by accumulating environmental information in association with feature values obtained from captured images, the robot 100 can act using the existing place memory even in a place different from the place where the place memory is formed.

The robot 100 changes its emotional parameters according to events. In addition, the robot 100 changes its emotion parameter not only by the event itself, but also by objects associated with the event. In particular, when an event that has a strong effect on the emotion parameter is set, it is possible to express behavior that not only the event itself but also the object associated with the event influences the emotion.

It should be noted that the present invention is not limited to the above-described embodiments and modifications, and can be embodied by modifying constituent elements without departing from the scope of the invention. Various inventions may be formed by appropriately combining a plurality of constituent elements disclosed in the above embodiments and modifications. Also, some components may be deleted from all the components shown in the above embodiments and modifications.

One robot 100, one server 200, and a plurality of external sensors 114 have been described to constitute the robot system 300, but the robot 10
0 may be implemented by the server 200 , or some or all of the functions of the server 200 may be assigned to the robot 100 . One server 200 may control multiple robots 100 , or multiple servers 200 may cooperate to control one or more robots 100 .

A third device other than the robot 100 and the server 200 may take part of the functions. The aggregate of each function of the robot 100 and each function of the server 200 described with reference to FIGS. 5 and 6 can also be grasped as one "robot" from a broad perspective. How to distribute a plurality of functions necessary for realizing the present invention to one or more pieces of hardware is determined in consideration of the processing capability of each piece of hardware and the specifications required for the robot system 300. It should be decided.

As described above, the “robot in a narrow sense” refers to the robot 100 that does not include the server 200 , while the “robot in a broad sense” refers to the robot system 300 . It is conceivable that most of the functions of the server 200 will be integrated into the robot 100 in the future.

Although the state management unit 244 has been described as being installed in the server 200 , the function of the state management unit 244 may be installed in the robot 100 . Similarly, the functions of the map management unit 168 and the object management unit 174 may also be implemented in the robot 100 instead of the server 200. FIG.

The environment information acquisition unit 166 of the robot 100 acquires primary environment information.
It functions as a "primary environment information acquisition unit" and is the state management unit of the robot system 300.
244 may include a "secondary environment information acquisition unit" that acquires secondary environment information. Of course, the function of the secondary environmental information acquisition unit may be installed in the robot 100. FIG. At the current position P1 of the robot 100, the primary environmental information acquisition unit
(Assuming that the image feature information R1 is also acquired at the same point), the primary environment information E1 may be acquired and the image feature information R1 and the primary environment information E1 may be associated and registered. Point information that associates the image feature information R1 with the primary environment information E1 is stored in a point information storage unit (not shown) of the robot 100 or the server 200. FIG. Then, the secondary environment information acquisition unit compares the image feature information R1 acquired at the current point P1 with the acquired image feature information group, and selects the image feature information R2 whose image similarity is equal to or greater than a predetermined threshold. To detect. The secondary environment information acquisition unit identifies the environment information E2 associated with the image feature information R2 as the secondary environment information E2 at the current point P1. The motion control unit 222 may change the behavioral characteristics of the robot 100 positioned at the point P1 based on both the primary environment information E1 and the secondary environment information E2 as described above.

When the image feature information R1 and the image feature information R1 are similar, the image feature information R1
and the acquisition point P2 of the image feature information R1 are likely to be the same. On the other hand, image feature information (landscape) of points P1 and P2, which are originally separate points
may have happened to be similar. Therefore, the robot 100, having recognized the scenery at the current point P1, remembers the secondary environment information E2 recognized at the point P2 (another point visited in the past) having scenery similar to that of the point P1.
A process in which the emotion parameter is influenced by the secondary environmental information E2 can be realized.
For example, it is possible to realize a complex behavioral characteristic in which memory is stimulated by an image of "point P2 where father's chair C1 is located" at "point P1 where chair C2 is similar to father's chair C1". Robot 100 is the point
When the user has a positive image of P2, the motion control unit 222
1, by making the robot 100 stay for a predetermined time or more,
You may indirectly express 0's attachment to chair C1.

When the image feature information similar to the image feature information R1 of the point P1 cannot be detected, the secondary environment information acquisition section instructs the captured image acquisition section 162 to change the shooting direction. For example, when the robot 100 has acquired the key frame 404 by photographing the front, the photographing direction may be shifted rightward by 30 degrees and the image characteristic information R3 may be acquired again. Then, the secondary image information acquisition unit may search for acquired image feature information similar to the image feature information R3. The robot 100 may change the orientation of the built-in camera or the orientation of the body, or may change the extraction direction of the key frame 404 in the omnidirectional image captured by the omnidirectional camera. By appropriately changing the shooting direction in this way, it becomes easier to detect image feature information similar to the current location from already acquired image feature information.

An estimation model may be prepared that takes the feature amount of the captured image as an input and the environment information as an output. This inferring model may be a neural network model with feature values f1, f2, . . . , ft as inputs and environmental information e1, e2, . For example, the feature quantity f1 may be the number of feature points 410 included in the keyframe 404 and the feature quantity f2 may be the number of substantially vertical straight lines formed by the feature points 410 in the keyframe 404 . Similarly, the environment information e1 may be the reception intensity, and the environment information e2 may be the detection rate of the user X1. The secondary environment information acquisition unit may input the feature quantity acquired at the current point P1 of the robot 100 to the estimation model and acquire the output as the secondary environment information. The secondary environment information output by the inference model is virtual environment information inferred from the feature amount. Even with such a processing method, the robot 100 that has recognized the scenery at the current point P1 imagines the secondary environmental information based on the image feature amount of the point P1, and the emotion parameter is affected by the secondary environmental information. process can be realized.

The primary environmental information acquisition unit extracts the feature amount from the captured image at the current point P1, and when the primary environmental information is acquired, uses these as input/output values, in other words, as teacher data, and updates the estimation model. good too.

When capturing the key frame 404, a TOF (Time-of-Flight) camera or a depth camera such as a stereo camera may be used. An image captured by a depth camera contains depth information. The image feature acquisition unit 164 may extract feature amounts for each depth range. The primary environment information acquisition unit may associate and register the image feature information for each depth range and the primary environment information E1. The depth range may be defined as an absolute range of distances from the camera,
It may be determined as a relative distance range based on a certain feature point. For example, when five depth ranges are determined, the common primary environment information E1 is registered in association with each of the image feature information of the five depth ranges. Further, the image feature acquisition unit 164 may extract a combination of feature points resulting from the same object as image feature information and register it in association with the primary environment information E1. Then, the secondary environment information acquisition unit may acquire the secondary environment information E2 using the feature amount for each depth range. By extracting the feature quantity according to the depth range in this way, it becomes easier to identify the visual features that are factors that create a certain environment. In other words, the probability of removing feature points that become noise increases, and the learning effect increases in the form using the inference model.

For example, assume a situation where a person is near the television T1. A primary environment information acquisition unit acquires a feature amount for each depth range and registers a plurality of point information. This point information includes image feature information mainly composed of a group of feature points of the television T1 and environment information indicating that a person is present in correspondence with each other. On another occasion, a key frame 404 including TV T2 was shot, and when feature amounts were obtained for each depth range, image feature information mainly composed of feature points of "TV" was extracted. Suppose there is no person near TV T2. When the second environmental information acquisition unit acquires point information similar to the image feature information, the TV T
Based on the image feature information of 1, the secondary environmental information of "there is a person" is obtained. As a result, if image feature information containing many feature points attributed to the television is obtained, even if the place where the experience of "there is a person is near the TV" is actually different from the place where the experience is, "human You can remember the experience of "there is someone", and you can choose learning behavior such as staying long even if there is no one.

In the present embodiment, it has been described that one captured image is acquired periodically, and multiple captured images are acquired when an event occurs. Not limited to this
Acquire n images during normal operation, and m images when an event occurs.
(m>n) captured images may be acquired. The event is robot 100
Therefore, it is desirable to acquire more detailed image information around the event when the event occurs. By acquiring a large number of captured images, an object can be detected with high accuracy when an event occurs. When an event occurs, the robot 100 may acquire a plurality of captured images by directing the mounted camera in a plurality of directions. In this case, it is possible to express the action of checking the surroundings in surprise at the event.

When an event occurs, a captured image may be obtained with higher image quality than normal. The external environment can be recognized in more detail when an event occurs while suppressing the image processing load during normal times. The image feature acquisition unit 164 may recognize a small object from the enlarged image by enlarging the high-quality captured image.

When an event occurs, a feature amount may be extracted for each depth to detect an object. Specifically, the depth camera may separate the depth direction into a plurality of steps. A predetermined range from the robot 100, for example, a range within 30 cm is defined as a first range, a range within 30 cm to 1 meter as a second range, and a range of 1 meter or more as a third range. Then, the degree of similarity between the image feature information of the first range and the existing image feature information of the first range may be determined. In this case, similarity judgment is performed on image feature information that is similar at a short distance, even if it is not similar at a distance. Can perform image processing.

When an event occurs, the detection sensitivity of various sensors such as a microphone may be increased. Alternatively, some sensors, such as the odor sensor, may not be used during normal times, and may be used only when an event occurs, thereby enriching environmental information when an event occurs. Humans are,
In normal times, we act unconsciously, but when we perceive a danger or an event that attracts our interest, so to speak, in an emergency, we work hard to accurately perceive the surrounding environment. By increasing the detection power of the sensor when an event occurs, in other words, in an emergency, it becomes easier to recognize the external environment when the event occurs.

The captured image acquisition unit 162 may constantly capture moving images and periodically extract captured images (still images) from the moving images. The image feature acquisition unit 164 may extract keyframes 404 from the moving image. When an event occurs, the recognition unit 212 may detect an object from a moving image of a predetermined period before and after recognizing the occurrence of the event.

The object determination process shown in FIG. 14 may be executed at a different timing than the update process shown in FIG. If the object determination process is executed with high frequency, the emotion parameter will change each time an object is detected. As one method of suppressing excessive changes in emotion parameters, the object determination process may be executed less frequently than the update process.

In the present embodiment, the causality between an event and an object is defined as the degree of association, and the emotion parameter is changed based on the object. In addition to visible objects, other environmental information may be associated with events, such as specific sounds or smells. For example, the smell of perfume and female user X4
By linking the event of going out, when the smell of perfume is detected, the female user X4 is associated with going out, and the state management unit 244 obtains an emotional value (loneliness
) may be increased.

The object management unit 174 may decrease the degree of association representing the strength of associativity between an event and an object over time. According to such a control method, the robot 100 can express "forgetting" that the associativity between an event and an object fades with time. Object management part 1
74 may reduce the time decay rate of relevance when relevance exceeds a predetermined threshold. According to such a control method, when a strong association (degree of association) is established between an event and an object, such association is difficult to dissolve. In other words, robot 1 can express emotions similar to “strong prejudice”.
It can be expressed in 00.

Some events may be set as special events. For example, a special event may be defined as the occurrence of a sound greater than or equal to a predetermined value, such as a thunderclap or an explosion. The object management unit 174 sets a predetermined degree of relevance, for example, 90(%), to the object visually recognized at the same time as the special event, and
A strong impression such as "trauma (trauma)" may be set on the robot 100. In addition to this, when user X1 hits hard enough to break down,
The object management unit 174 stores the special event of being hit and the user X1 (
object) may be set to 90% relevance. According to such a control method, it is possible to express "feeling of difficulty" by using parameters other than familiarity.

Any event condition can be set. It is not limited to an external event, and may be an emotional value (loneliness) exceeding a predetermined value. Alternatively, a mechanical phenomenon such as a charging rate falling below a predetermined value or a failure may be recognized as an event. When the emotion value (loneliness) reaches a predetermined value or more, the motion control unit 222 selects a candidate point as a destination of the robot 100 where a user whose degree of intimacy is a predetermined value or more has a high probability of appearing. You can have it wander the room in search of

The image feature acquisition unit 164 obtains a total of 10 keyframes 4 at point P6.
Suppose you get 04. At this time, an image feature common to a predetermined number of the ten key frames 404, for example, seven or more key frames 404 may be registered as the key frame 404 (image feature information) of the point P6. Such a control method can also effectively remove unstable features such as documents and clothes that are difficult to mark when recognizing the external environment, that is, noise information in position recognition.

Environmental information may be categorized. For example, environmental information about temperature may be classified into two or more categories such as "warm" and "cold". In addition, environmental information may be classified into a plurality of categories such as "safe", "dangerous", "fun" and "lonely" for each location. According to such a control method, keyframe 4
Since the environmental information linked to 04 can be condensed, the storage capacity can be suppressed and the processing load accompanying detection of the environmental information can be reduced.

The territory of the robot 100 may be represented by the reception strength of the radio signal from the server 200. FIG. The robot 100 cannot receive support from the server 200 if it goes out of the communicable range with the server 200 . The motion control unit 222 may refer to the object information 430 and set a set of points where the reception strength is B or higher as the actionable range. According to such a control method, it is possible to prevent the robot 100 from leaving the communicable range.

The robot 100 may set its territory based on the reception strength from other devices, not limited to the server 200 . For example, when the external sensor 114 is installed, the operation control unit 222 may set the range in which wireless communication with any one of the external sensors 114 can be performed satisfactorily as the actionable range. robot 100
communicates with the server 200 periodically, and when the reception strength becomes equal to or less than a predetermined value, it may restrict its own behavior so that it does not move away from the server 200 any further. On the other hand, when a specific event such as welcoming the user occurs, the operation control unit
222 may set a movement target point even if the reception strength is low.

The robot 100 is periodically charged by a charging device (not shown). The server 200 may suppress communication with the robot 100 during charging. The communication unit 142 of the robot 100 transmits a charging start signal to the server 200 when the charging device starts charging, and transmits a charging completion signal when charging is completed. When the server 200 receives the charging start signal, the robot 10
Communication with the robot 100 may be suspended from 0 until the charging completion signal is received. Alternatively, while the robot 100 is being charged, the frequency of regular communication between the server 200 and the robot 100 may be reduced. According to such a control method, by suppressing communication during charging when the robot 100 is not active,
Power consumption associated with communication between the robot 100 and the server 200 can be suppressed.

The present invention can be applied to other than the robot 100 as well. For example, in a navigation system, the concept of storing locations by keyframes 404 can be applied. A map is created by memorizing scenery by taking pictures of the outside world at regular intervals. Associate environmental information with this. Based on the environment map information, the navigation system may propose a walking course that the user has never been to, or a walking course that includes points where the user has had a pleasant experience. You may suggest a walking course that avoids places where you have had dangerous experiences.

Environment maps or environment map information need not be represented as so-called "maps". The environmental map information may be any information that associates geographical information with environmental information, and does not need to be represented visually. The geographic information here may be the keyframe 404 or various objects that serve as geographical landmarks.

In this embodiment, the key frame 404 is associated with environmental information, the emotion parameter is changed (secondary reflection) based on the environmental information (memory related to the key frame 404) associated with the key frame 404, and the has been described as changing the behavioral characteristics of the robot 100.

Alternatively, the environmental memory may be formed by associating objects with environmental information. When the robot 100 detects an object, the robot 100 may change its behavioral characteristics based on environmental information associated with the object (memory related to the object).

The robot 100 may change the emotional parameter based on the type of object, and change the behavioral characteristics based on the change in the emotional parameter.

In addition, the impression of the robot 100 with respect to a room (space) containing a plurality of objects is stored, and based on the impression memory (described later) of the room
100 emotional parameters may be changed and behavioral characteristics may be changed based on the emotional parameters.

In the following, changes in emotion parameters based on memories of a room, changes in emotion parameters based on environmental information associated with objects, and accompanying changes in behavioral characteristics of robot 100 will be described as modified examples.

The image feature acquisition unit 164 of the robot 100 may function as an "object detection unit". As described above, the image feature acquiring unit 164 may have a plurality of object patterns that define the shape and color features of a plurality of types of objects in advance, and image recognition of the object may be performed based on the object patterns. For example, the image feature acquiring unit 164 may determine that the object (B01) is "TV" based on the feature point distribution of the object (B01). Further, the image feature acquisition unit 164 may determine that this object (B04) is also "television" with respect to the feature point distribution of another object (B04). The image feature acquiring unit 164 or the recognizing unit 212 can recognize that the object (B01) and the object (B04) belong to the same category of "television" even if they are different objects.
The image feature acquisition unit 164 may determine the category of the object using a neural network model that receives feature point distribution as input.

Hereinafter, an object as a unique object such as the object (B01) will be referred to as a "unique object", and a type of object such as a television will be referred to as an "object category". A unique object is identified by an object ID and an object category is identified by a category ID. In addition, they are simply referred to as "objects" when they are not distinguished from each other.

The robot 100 may detect unique objects and identify object categories by methods other than image recognition. For example, the robot 100 may include an "object detection unit" that detects objects by communicating with unique objects. If the home appliance can send the category ID, the object detection unit may determine the object category of the home appliance (unique object) by detecting the category ID sent from the home appliance.
In the following modified example, it is assumed that the object detection unit of the robot 100 detects objects and identifies object categories.

FIG. 15 is a data structure diagram of the object environment information 440 in the modified example.
Object environment information 440 is stored in the object information storage unit 172 . Object environment information 440 includes object ID, category ID,
Associated with environment information and room ID. As noted above, the Object ID identifies a unique object and the Category ID indicates the object category. For example, when the unique object (B01) and the unique object (B04) are both "television", the category IDs of these two unique objects will be the same.

The object detection unit identifies an object ID and a category ID when detecting a unique object. The environment information acquisition unit 166 acquires environment information when the unique object is detected. The object management unit 174 stores the object environment information 4 based on the environment information when the unique object is detected.
Update 40. Updating object environment information 440 is basically the same as the method described in connection with FIG.

The recognition unit 212 recognizes a room as a closed space by detecting walls and doors. Each room is identified by a room ID. Unique objects detected by the object detection unit are managed in association with room IDs. Room ID=R01 indicates "living room" and room ID=R02 indicates "study". According to FIG. 15, the unique object (B01: television) is installed in the room (R01: living room), and the unique object (B04: television) is installed in the room (R02: study). For example, room (R01: living room)
One of the users is detected near the TV (B01) installed in the room (R02: study) with a high probability of 70%, and the TV (B0
Near 4), it is remembered that the user is rarely detected.
In the following description, it is assumed that the object category (C01) is television, the object category (C02) is printer, the object category (C03) is table, and the object category (C04) is sofa.

FIG. 16 is a data structure diagram of room impression information 450 in a modified example.
In this modification, data storage section 206 of server 200 further stores room impression information 450 . The room impression information 450 is the room (room ID
), category ID and impression memory. The "impression memory" is information representing the impression of the robot 100 with respect to the room, and is indicated by a numerical value (hereinafter referred to as "impression memory value"). "Impression" is an index that increases or decreases according to the events experienced in the room, the external state or internal state of the robot, and can be managed as numerical values such as the emotional parameters of the robot 100, the amount of movement of each drive unit, and the volume of the speaker. Determined based on parameters.

For example, in the living room, the robot 100 has many opportunities to interact with its owner and has a large movable space, so it can move freely. The bedroom has less space than the living room, and there are fewer opportunities to interact with each other than in the living room. Therefore, if an index is provided from the viewpoint of the amount of movement of the movable portion of the robot, the index of the amount of movement in the living room is larger than the index of the amount of movement in the bedroom.

If we set an index in terms of the output from the speakers, there are many opportunities to interact with the owner in the living room, and there are many opportunities to speak, so the speaker index is
Greater than the loudspeaker exponent in the bedroom.

If an index is set from the viewpoint of pleasant behavior that the robot 100 receives from the owner, there are more opportunities for the robot 100 to be hugged in the living room, and there are fewer opportunities for hugging in the bedroom than in the living room. be larger than By defining these indices and summing up the actual values of the parameters that form the basis of the defined indices for each room over a certain period of time under predetermined conditions, trends for each room can be understood. These indices are managed as impression memory.

For example, when the robot 100 experiences that it actively acts in the living room, has many opportunities to vocalize, and is often subjected to pleasurable behavior, it obtains the actual values for the living room as the "movement amount index", "speaker index", and "speaker index". Pleasant behavior index” increases. Based on these results, an impression is formed that "in a room such as a living room, you can move actively, speak out, and often receive pleasurable behavior."

These indices also change depending on the time of day. For example, during the daytime on weekdays, it is rare to receive pleasant acts, but during the night on weekdays, it may be possible to receive many pleasant acts. In this case, various indices are associated with weekday afternoons (for example, 11:00 to 13:00). As a result, the impression memory value also changes between daytime on weekdays and nighttime on weekdays.
In FIG. 16, the index of pleasant behavior among these impression memories is illustrated.

When the event occurring in the room (R01: living room) is a pleasant act, the state management unit 244 increases the impression storage value of the room (R01: living room). On the other hand, when the event is an unpleasant act, the state management unit 244 decreases the impression memory value of the room (R01: living room). With such a control method, the impression of the robot 100 on the room (R01: living room) is expressed numerically. According to Figure 16, the room (R0
1: The living room) has an impression memory value of "+10", so the robot 100 moves to the room (
R01: living room) has a positive impression. Such a favorable impression of the living room is cultivated based on the experience that, for example, when the robot 100 was in the living room, it was often given pleasant acts such as being held by the owner.

When the owner scolds the robot 100 in the room (R02: study) (offensive behavior), the impression memory value of the robot 100 for the room (R02: study) decreases. For example, when the robot 100 is in the room (R02: study), the state management unit 244 stores an impression of the room (R02: study) when no owner has been detected for a certain period of time, for example, 10 minutes or longer. value may be decreased. In this way, the impression memory value may be changed based not only on the event that occurred in the room, but also on the fact that no event occurred in the room and the frequency of occurrence of the event.

The robot 100 in this modified example does not recognize that the room (R01) is the "living room". The recognition unit 212 of the robot 100 identifies the room type by a plurality of object categories included in the room, and assigns a room ID to each room.

According to FIG. 16, in the room (R01: living room), the television (C01),
Table (C03) and sofa (C04) are detected. robot 100
is a room (R01:
living room). A high impression memory value for the room (R01: living room) means that the robot 100 has a positive impression for the room that satisfies the condition of including the television (C01), the table (C03), and the sofa (C04). means to have The robot 100 uses environmental information obtained in the room,
In particular, the impression memory value of the room is changed according to the event experienced in the room. The robot 100 identifies a room for impression memory values by a plurality of object categories included in the room.

Assuming the room impression information 450 shown in FIG. 16, the robot 100 has a favorable impression of a room containing a television (C01), a table (C03), and a sofa (C04). When the robot 100 enters the room for the first time (hereinbelow, the room in which the robot 100 is present is referred to as the "occupied room"), the state management unit 244 identifies a plurality of types of object categories included in the existing room. Identify the room type based on For example, a room with a known occupancy (R0
When similar to 1), the state management unit 244 changes the emotion parameter of the robot 100 based on the impression memory value of the room (R01). The occupied room may be the same as the room (R01), or it may be another room that happens to have something in common in that it includes a television (C01), a table (C03) and a sofa (C04). According to such a control method, the robot 100 having a positive impression of a room (R01: living room) with a television, a table, etc.
Even when visiting an unknown home for the first time, it is possible to "be happy" when finding a room similar to the room (R01: living room). Therefore, the emotion of the robot 100 when entering an unknown room can be changed based on the impression of the known room. When in a room similar to a known room, state manager 244 may lower the emotion parameter value indicative of loneliness. Also, when in a room that does not resemble any of the known rooms, the state manager 244 may increase the emotional parameter indicating loneliness and may also increase the emotional parameter indicating curiosity.

For example, the state management unit 244 of the robot 100 may reduce the emotion parameter indicating "lonely" when entering a room similar to or identical to a known room with a high impression memory value. The higher the impression memory value, the greater the decrease in the emotion parameter indicating "lonely".

Also, for example, an index of activity is defined as impression memory. When the index of the amount of movement is smaller than the index of the amount of movement in the living room, the robot 100 can reduce the amount of movement in the room corresponding to the study more than in the room corresponding to the living room.

In summary, the robot 100 intricately changes its emotional parameters, movement amount, etc. while being affected by various external environments in each room. Such action results are recorded as various indices. In addition, based on these behavioral records, impression memories such as "able to move actively" and "easy to receive pleasant behavior" are formed. Then, when the room in which the robot 100 is present resembles a known room, the robot 100 changes its behavioral characteristics based on the impression memory of the known room. The "experience-based behavioral characteristics" are realized by the influence of the actual behavioral characteristics based on the experience-based behavioral performance.

As described above, rooms are identified by a combination of multiple object categories. State manager 244 need not identify rooms based on all unique objects detected in the occupied room.

The object management unit 174 may set an impression storage value for each unique object based on environment information. Specifically, the object management unit 174
is the event condition (described above) when the environment information when the unique object is detected
is satisfied, change the impression storage value of the eigenobject in response to the event. For example, when pleasant behavior is detected near the television (B01), the object management unit 174 may increase the impression storage value for the television (B01). In other words, when a predetermined event occurs, the object manager
174 identifies a unique object within a predetermined range of the robot 100 and changes the impression memory value for that unique object. Whether or not an object is within a predetermined range may be determined by measuring the distance of each object, or may be estimated based on the size of the object included in the captured image.

The object management unit 174 may also identify a unique object that exists in the front direction of the robot 100 and is within a predetermined range, and change the impression storage value for that unique object. Also, the object management unit 174 may set the impression storage value for the object category (C01) instead of the unique object (B01). For example, since there are people around the television, events such as being hugged frequently occur. By setting an impression memory value for the category "television", it is possible to select an action of moving closer to the television based on memory even if it is the first place.

The object management unit 174 may identify the room in which the room is present based on some of the unique objects detected in the room in which the room is present. For example, the object management unit 174 may identify those with particularly high and particularly low impression storage values, such as the top three unique objects and the bottom three unique objects. Then, the object category of each of these six unique objects may be specified, and the occupied room may be identified as a combination of the specified object categories.

The state management unit 244 compares the object category group detected in the room in which the person is present and the object category group associated with each of the plurality of rooms registered in the room impression information 450 . The state management unit 244 may determine the degree of similarity according to the number of matching object categories. For example, 2 of the 3 object categories contained in Room (R01: Living Room)
When one is also detected in the room in which the room is present, the state management unit 244 determines whether the room in which the room is present and the room (R
01: living room) may be determined to be 66(%) (=2÷3). This means that the existing room is likely to be the same as the known room (R01: living room). The state management unit 244 compares the object category detected in the room in which the room is present and the room impression information 450, and determines the room with the highest similarity and a similarity of 70(%) or more as the room in which the room is present. You may determine that it is a room of a type.

FIG. 17 is a flow chart showing the process of detecting an object in the modified example.
The processing shown in FIG. 17 is periodically executed in the robot system 300. As shown in FIG. The environment information acquisition unit 166 first acquires the environment information of the room in which the room is present (S6
0). The state management unit 244 changes the emotion parameter based on the environmental information (S62). This is the same as the “primary reflection by environment information” described with reference to FIG. The object detector detects unique objects (
S64). At this time, the object detector identifies the object category of the unique object.

By comparing the plurality of detected object categories and the room impression information 450, the state management unit 244 identifies the type of room in which the person is present (S66).
. As described above, the state management unit 244 selects a room in which the similarity between the object category group detected from the room in which the room is present and the object category group for each room registered in the room impression information 450 is equal to or greater than a predetermined value. It is specified as "a room similar to a known room (hereinafter simply referred to as a "known room")".

When the room in which the user is present is the "known room", the state management unit 244 changes the emotion parameter based on the impression memory value pre-associated with the known room (S68). For example, for the room (R01: living room), the impression memory value "+1
0”, the state management unit 244 may decrease the emotion parameter value indicating loneliness and the emotion parameter value indicating curiosity when the existing room resembles a living room. In this way, the feeling (emotional parameter) of the robot 100 is changed based on impression memory values pre-associated with a known room, more strictly speaking, with a group of object categories included in the known room. can be done. This is the same as the "secondary reflection by environment information" described in connection with FIG.

On the other hand, when it is an unknown room (N of S66), secondary reflection is not performed.
After that, the state management unit 244 updates the room impression information 450 with the environment information acquired in S62 (S70).

The robot 100 changes its emotional parameters based on the impression memory values of a known room similar to the room in which it is present, and changes its behavioral characteristics based on changes in the emotional parameters. In addition to this, the robot 100 may change its behavioral characteristics based on the impression memory value without using the emotion parameter. For example, in a room with a low impression memory value, the motion control unit 150 of the robot 100
100 movement speed, movement amount (travelable distance in unit time), momentum (
The movable amount or movable range of various actuators may be suppressed. In addition, in a room with a low impression memory value, the motion control unit 150 may suppress the upper limit of the sound volume generated by the robot 100 . According to such a control method, the robot 100 is quiet in a room with a low impression memory value, and behaves actively in a room with a high impression memory value. The feeling of the robot 100 can be expressed in behavior based on the impression memory for the room. An impression memory value may be initialized in advance for each room.

When the robot 100 enters a room for the first time, it refers to the room impression information 450 to determine what kind of room it is. Then, based on the past experience, the action mode such as movement speed is determined. After that, the object environment information 44
By referring to 0, based on past experience, search for objects in the room and set them as target points.

Also, when you are in a house for the first time, or when you are lonely, you search for a "room that seems to be fun (a room with a favorable impression)." Specifically, operation control unit 222 sets a room with a high impression memory value as a movement target point.

The motion controller 222 searches for rooms containing object categories with high impression storage values. For example, assume that the robot 100 has a favorable impression of a room containing a television, a table, and a sofa. In this case, robot 100
Assuming that a room containing a TV, table, and sofa is a "pleasant room," search for a room that meets these conditions.

The motion control unit 222 of the robot 100 refers to the room impression information 450,
Identify the room ID of a room with a high impression memory value. Next, the motion control unit 222 identifies multiple object categories contained in the identified room. The motion control unit 222 selects a room containing a predetermined number or more of object categories from among the plurality of types of object categories included in the “pleasant room” as the movement target point.

The robot 100 can select a room as a movement target point based on experience. For example, owner A takes robot 100A to user B's house. User B's house is unknown to robot 100A. First, the robot 100A receives the room impression information 450 formed in the owner A's house.
to identify object categories contained in rooms with high impression memory values. Then, the robot 100 searches for rooms containing the specified object category group in user B's house as well. For example, when the robot 100 finds a room RB similar to the room (R01: living room) in user B's house, the robot 100
Enter B and change the emotional parameters and behavioral characteristics based on the impression memory value of the room (R01: living room). According to such a control method, it is possible to cause the robot 100 to behave in a manner similar to that at home, even at a home visited for the first time, based on the experience at home.

The motion control unit 222 of the robot 100 further selects a unique object to approach in the existing room. The motion control unit 222 selects a unique object to approach based on the environment information associated with the unique object, using the same method as in the present embodiment. For example, the robot 100 may approach a unique object with a high user detection rate when the emotion parameter value indicating loneliness is high. The state management unit 244 may change the emotion parameter based on the environment information of the unique object. For example, when the robot 100 is near a sofa where the event of being held is likely to occur, the robot 100 may lower the emotional parameter indicating loneliness without actually being held. In addition to changing the emotional parameters and behavioral characteristics of the robot 100 based on the environmental information associated with the keyframe 404, the emotional parameters of the robot 100 can also be changed based on the environmental information associated with objects (unique objects or object categories). and behavioral characteristics may be changed. An impression memory value may be set in advance for the object category. When 100 detects a unique object, 100 may change the behavioral characteristic based on the impression memory value associated with that object category.

A room with a favorable impression includes a television, a table, and a sofa. After entering the room, the motion control unit 222 causes the robot 100 to approach the television. According to such a control method, the robot 100 finds a "pleasant room" based on impression memory, and after entering the room approaches an object (television) that gives a good impression. As a result, I like the living room, or
The robot 100's preference for space and objects, such as his liking for television, can be expressed in behavior.

Similarly, the robot 100 may avoid rooms with low impression memory values. For example, when the robot 100 detects an unpleasant behavior such as being chased away by the owner when entering the bathroom, the robot 100 detects an object in the bathroom,
For example, the impression memory value for a combination of towels, a washstand, a washing machine, etc. is lowered. As a result, they are not good at bathrooms, or they are not good at washing machines.
It can express behavioral characteristics.

Impression memory is not limited to pleasant/unpleasant. For example, the impression memory value may be set based on other evaluation axes such as lively/quiet and hot/cold. room (R
01: Living room) is associated with the impression memory of "bustling", and when the emotional parameter indicating "lonely" increases, the robot 100
may set the room (R01: living room) as the movement target point. room (R0
1: living room), people tend to gather together, so this control method can express the behavioral characteristics of approaching the room (R01: living room) and looking for people when feeling lonely. A unique object need not be a fixed object, and may be a portable object such as a towel, or an autonomously movable object such as a pet or other robot 100 .

FIG. 18 is a data structure diagram of action experience information 460 in another modification.
The object information storage unit 172 may store action experience information 460. FIG. Action experience information 460 associates category IDs with motion IDs and impression memories. For example, when the robot 100 selects a motion with motion ID=M01 within a predetermined range from an object (B01: television), it is assumed that the owner praises it. Since being praised is a pleasant act, the object management unit 174 updates the action experience information 460 by increasing the impression storage value for the combination of object category (C01: television) and motion (M01). On the other hand, suppose that the owner scolded him when he selected motion (M03) near the object (B01: TV). Since being scolded is an offensive act, the object manager 174 decreases the impression storage value for the combination of object category (C01: television) and motion (M03).

The action control unit 222 refers to the action experience information 460, increases the selection probability of the motion (M01) near the television, and decreases the selection probability of the motion (M03). According to such a control method, the behavior characteristics of the robot 100 can be changed by the user's actions such as praising or scolding. For example, by scolding Robot 100 sitting near the cupboard,
Robot 100 can be discouraged from sitting near the cupboard. As a result, the user can "discipline" the robot 100. FIG.

The state of the object may also be considered to change the impression memory. For example, if you approach the vacuum cleaner when it's off, it won't do anything, but if you approach it when it's on, you'll be chased away by the owner. Being shunned by an owner is an offensive behavior, so the robot 100 learns a code of conduct: "Do not approach the vacuum cleaner when it is on." Whether the cleaner is on or off may be determined based on the presence or absence of the sound generated by the cleaner. In addition to the state of the object, the impression memory may be changed in consideration of the time zone. For example, when the robot 100 enters the kitchen while dinner is being prepared, the owner may drive away the robot 100 (offensive behavior). As a result, the robot 100 can be made to learn a code of conduct that "do not approach the kitchen where the refrigerator and cupboards are located in the evening (do not disturb the owner in the kitchen)".

An impression memory may be associated with a combination of room (room ID) and motion. In this case, for example, the robot 100 can learn a code of conduct that loud noises are allowed in the living room but not loud in the study. Similarly, you may learn that you can move around fast in the living room, but you must slow down in the study.

When the robot 100 recognizes an unpleasant act, the robot 100 lowers the degree of intimacy with the owner who performed the unpleasant act. However, when recognizing an unpleasant act from an owner with a degree of intimacy equal to or greater than a predetermined value, the degree of intimacy management unit 220 may not reduce the degree of intimacy, or may reduce the amount of decrease in the degree of intimacy compared to normal times. According to such a control method, the owner who has built a strong relationship of trust with the robot 100 can scold the robot 100 with peace of mind.

As described above, an object may be associated with environment information, or a space (room ID) may be associated with environment information. The state manager 244 may change the impression memory based on environmental information, particularly events. Impression storage for objects or spaces may be initialized. For example, if a high impression memory value is set in advance for the living room, the robot 100 can express the behavioral characteristic that it innately prefers the living room. In addition, the owner transmits the stored impression value for the living room to the server 200 via a mobile terminal such as a smartphone, and the object management unit 174 stores the specified stored impression value as room impression information.
You can set it to 450.

Environment information may be associated with a combination of object categories. A room may be recognized based on the layout of the room and the layout of objects. For example, a room may be identified as "a room with a TV by the window and a sofa placed a short distance from the window".

Environment map information 420, object information
Information shown in 430, room impression information 450, and behavioral experience information 460 may be shared. According to such a control method, when the robot 100A prefers the living room, the robot 100B also prefers the living room, and the action characteristics of the plurality of robots 100 can be interlocked.

Robot 100A may share with robot 100B only places with high impression memory values as fun rooms. For example, among a plurality of rooms, the room impression information 450 may be shared for rooms with higher than average impression memory values or rooms with impression memory values within the top three. Alternatively, room impression information 450 may be shared for rooms with impression memory values lower than the average, or rooms with impression memory values within the bottom three ranks.

The robot 100A and the robot 100B do not need to share the room impression information 450 for all rooms, and may share the room impression information 450 only for some rooms. According to such a control method, for example, since robot 100A does not transmit information about some rooms to robot 100B, it is possible to create a secret room that only robot 100A likes.

Owner A, the room where Robot 100A was held by Owner A for the first time
Robot 100A may not share information with robot 100B in rooms with special conditions (memories), such as a room where a robot holds the robot for a long time.

Impression memory may be visualized. The server 200 may include an impression map generation unit that generates an image (impression map) showing the arrangement of objects and the impression memory for the objects. The communication unit 204 of the server 200 may transmit the impression map to the owner's mobile terminal. For example, an object with a high impression memory value is displayed in blue, and an object with a low impression memory value is displayed in red. can know.

The impression map generator may display changes in impression storage values for the object or room in the impression map. For example, an impression map animation may show how an object that was disliked at first is gradually becoming liked.

Claims (3)

a storage unit;
a processor;
a drive mechanism for performing motions selected by the processor;
The processor
Recognizing impression memory for objects,
A robot that determines a motion for the object based on the impression memory. The robot according to claim 1,
The processor
When a first object is detected, storing a first impression memory for the first object in the storage unit;
when a second object is detected, a second impression that is an impression memory of the second object based on the similarity between the second object and the first object and the first impression memory; recognize memories,
A robot configured to determine a motion for said second object based on said second impression memory.
The robot according to claim 1,
The processor
Display a map showing the space and display objects existing in the space on the map,
A robot that determines the format associated with the display of each object based on the impression memory for the object.