JP2024045247A - robot - Google Patents

Abstract

[Problem] To provide a robot that humans can easily become attached to. [Solution] The robot is equipped with a motion control unit that selects the motion of the robot, a drive mechanism that executes the selected motion, an operation detection unit that detects the deformation of the robot's main body, and an eye generation unit that generates an eye image and displays the eye image in the robot's face area. [Selected Figure] Figure 5

Description

The present invention relates to a robot that humans can easily become attached to.

Humans keep pets to find comfort. However, many people give up on having pets for a variety of reasons, including not having enough time to look after them, not having a living environment that allows them to keep a pet, allergies, and the pain of losing a pet. If there were a robot that could play the role of a pet, it might be able to provide the same comfort that pets provide to people who cannot keep pets (see Patent Documents 1 and 2).

JP 2000-323219 A International Publication No. 2017/169826

In recent years, robot technology has been advancing rapidly, but we have yet to realize a robot that can act as a companion like a pet. This is because we do not believe that a robot has free will. By observing the behavior of our pets, which we can only assume to be of free will, humans can sense the presence of free will in their pets, empathize with them, and feel comforted by them.

The inventors thought that if a robot's response changed depending on how it was touched, people's attachment to the robot might be further increased. As a result of intensive research, the inventors came up with the idea of solving the above problem by installing sensors in places where people unconsciously want to touch, detecting how people touch the robot, and reflecting this in the robot's behavior.

The present invention was completed based on the recognition of the above problems, and its main objective is to provide a robot that humans can easily become attached to.

A robot in one aspect of the present invention includes an operation control unit that selects a motion for the robot, a drive mechanism that executes the selected motion, an operation detection unit that detects deformation of the robot's body, and an eye generation unit that generates an eye image and displays the eye image in the robot's facial area.
When a deformation of the robot body is detected, the eye generation unit changes the eye image in accordance with the deformation state.

A robot in another aspect of the present invention includes an operation control unit that selects a motion of the robot, a drive mechanism that executes the selected motion, and an operation detection unit that detects the movement of a device provided on the robot body.
When the motion of the device is detected, the motion control unit selects a motion corresponding to the motion of the device from among motions that are to be driven within a predetermined range from the device.

In another aspect of the present invention, a robot includes an action control unit that selects a motion of the robot, a drive mechanism that executes the selected motion, and first and second sensors installed on the body surface of the robot that detect contact by a user.
The motion control unit, when a user's contact is detected by both or one of the first and second sensors, selects a motion according to the manner of contact.
The detection accuracy of the first sensor is set to be higher than that of the second sensor, and the first sensor is installed in the face area of the robot.

According to the present invention, it becomes easier for humans to develop attachment to robots.

Fig. 1(a) is a front external view of the robot, and Fig. 1(b) is a side external view of the robot. FIG. 2 is a cross-sectional view schematically showing the structure of the robot. FIG. 2 is a hardware configuration diagram of a robot. FIG. 2 is a functional block diagram of the robot system. FIG. 5(a) is a front external view of the robot in this embodiment. FIG. 5(b) is a side external view of the robot in this embodiment. It is a hardware configuration diagram of a robot in this embodiment. FIG. 2 is a functional block diagram of the robot system according to the first embodiment. It is an external view of an eye image. It is an enlarged view of an eye image. FIG. 2 is a schematic diagram showing a method for generating an eye image. FIG. 11(a) is an external view of the robot's eye when the analog stick is moved upward from the initial position. FIG. 11(b) is an external view of the robot's eye when the analog stick is moved to the right from the initial position. FIG. 11(c) is an external view of the robot's eye when the analog stick is moved downward from the initial position. FIG. 11(d) is an external view of the robot's eye when the analog stick is moved to the left from the initial position. FIG. 11 is a functional block diagram of a robot system according to a second embodiment. 13 is a flowchart showing eye movements and robot motion. FIG. 14(a) is a data structure diagram of the eye movement table. FIG. 14(b) is a data structure diagram of the modified pattern selection table. FIG. 13 is a data structure diagram of a motion selection table. FIG. 2 is a front view of the robot in a stationary state. FIG. 3 is a front view of the robot when the motion is activated.

Embodiments of the present invention will be described in detail below with reference to the drawings. In the following description, for convenience, the positional relationship of each structure may be expressed based on the illustrated state. In addition, in the following embodiments and their modified examples, substantially identical components are given the same reference numerals, and their description will be omitted as appropriate.

When the user touches the robot's nose, the robot in this embodiment responds by moving its eyes and body. Before explaining the implementation of the robot in this embodiment, the basic configuration of the robot will be explained below.

[Basic configuration]
FIG. 1(a) is a front external view of the robot 100. FIG. 1(b) is a side external view of the robot 100.
The robot 100 in this embodiment is an autonomous robot that determines its actions based on the external environment and internal state. The external environment is recognized by various sensors such as cameras and thermosensors. The internal state is quantified as various parameters expressing the emotions of the robot 100. The robot 100 operates within the owner's home. Hereinafter, a person involved in the robot 100 will be referred to as a "user."

The body 104 of the robot 100 has an overall rounded shape and includes an outer skin made of a soft and elastic material such as urethane, rubber, resin, or fiber. The robot 100 may be dressed. The total weight of the robot 100 is about 5 to 15 kilograms, and the height is about 0.5 to 1.2 meters. The various attributes, such as appropriate weight, roundness, softness, and good feel, make it easy for the user to hold the robot 100 and make him or her want to hold the robot 100.

The robot 100 includes a pair of front wheels 102 (left wheel 102a, right wheel 102b) and one rear wheel 103. The front wheel 102 is a driving wheel, and the rear wheel 103 is a driven wheel. Although the front wheels 102 do not have a steering mechanism, their rotational speeds and rotational directions can be individually controlled. The rear wheels 103 are casters, and are rotatable to move the robot 100 back and forth and left and right. The rear wheel 103 may be an omni wheel.

The front wheels 102 and rear wheels 103 can be completely stored in the body 104 by a drive mechanism (rotating mechanism, link mechanism). Even when moving, most of the wheels are hidden by the body 104, but when the wheels are completely stored in the body 104, the robot 100 becomes unable to move. That is, as the wheels are stored, the body 104 descends and sits on the floor surface F. In this seated state, a flat seating surface 108 (ground contact surface) formed on the bottom of the body 104 abuts against the floor surface F.

The robot 100 has two hands 106. The hands 106 do not have the function of grasping objects. The hands 106 are capable of simple movements such as lifting, shaking, and vibrating. The two hands 106 can also be controlled individually.

The eyes 110 are capable of displaying images using liquid crystal elements or organic EL elements. The robot 100 is equipped with various sensors, such as a microphone array and ultrasonic sensors, that can identify the direction of a sound source. It also has a built-in speaker and can emit simple sounds.

Horns 112 are attached to the head of the robot 100. As described above, since the robot 100 is lightweight, the user can also lift the robot 100 by grasping the horns 112. A spherical camera is attached to the horn 112 and can image the entire upper part of the robot 100 at once.

FIG. 2 is a cross-sectional view that shows a schematic structure of the robot 100. As shown in FIG.
As shown in Fig. 2, the body 104 of the robot 100 includes a base frame 308, a main body frame 310, a pair of resin wheel covers 312, and an outer skin 314. The base frame 308 is made of metal, and constitutes the axis of the body 104 and supports the internal mechanism. The base frame 308 is formed by vertically connecting an upper plate 332 and a lower plate 334 with a plurality of side plates 336. Sufficient spaces are provided between the plurality of side plates 336 to allow ventilation. The battery 118, a control circuit 342, and various actuators are housed inside the base frame 308.

The main body frame 310 is made of a resin material and includes a head frame 316 and a body frame 318. The head frame 316 has a hollow hemispherical shape and forms the head skeleton of the robot 100. The trunk frame 318 has a stepped cylindrical shape and forms the trunk skeleton of the robot 100. The torso frame 318 is fixed integrally with the base frame 308. The head frame 316 is attached to the upper end of the body frame 318 so as to be relatively displaceable.

The head frame 316 is provided with three axes, a yaw axis 320, a pitch axis 322, and a roll axis 324, and an actuator 326 for driving the rotation of each axis. The actuator 326 includes a plurality of servo motors for driving each axis individually. The yaw axis 320 is driven for a head shaking motion, the pitch axis 322 is driven for a nodding motion, and the roll axis 324 is driven for a head tilt motion.

A plate 325 that supports the yaw axis 320 is fixed to the upper part of the head frame 316. A plurality of ventilation holes 327 are formed in the plate 325 to ensure ventilation between the upper and lower sides.

A metal base plate 328 is provided to support the head frame 316 and its internal mechanism from below. The base plate 328 is connected to the plate 325 via a cross link mechanism 329 (pantograph mechanism), and is connected to the upper plate 332 (base frame 308) via a joint 330.

The body frame 318 houses the base frame 308 and the wheel drive mechanism 370. Wheel drive mechanism 370 includes a rotation shaft 378 and an actuator 379. The lower half of the body frame 318 is made narrow in order to form a storage space S for the front wheel 102 between it and the wheel cover 312.

The outer skin 314 is made of urethane rubber and covers the main body frame 310 and the wheel cover 312 from the outside. Hand 106 is integrally molded with outer skin 314. An opening 390 for introducing outside air is provided at the upper end of the outer skin 314.

FIG. 3 is a diagram showing the hardware configuration of the robot 100.
The robot 100 includes an internal sensor 128, a communication device 126, a memory device 124, a processor 122, a drive mechanism 120, and a battery 118. The processor 122 and the memory device 124 are included in a control circuit 342. The units are connected to each other by a power line 130 and a signal line 132. The battery 118 supplies power to each unit via the power line 130. Each unit transmits and receives control signals via the signal line 132. The battery 118 is a lithium ion secondary battery, and is the power source of the robot 100.

The internal sensor 128 is a collection of various sensors built into the robot 100. Specifically, it includes a camera (spherical camera), a microphone array, a distance sensor (infrared sensor), a thermosensor, a touch sensor, an acceleration sensor, and an odor sensor. The touch sensor is installed between the outer skin 314 and the main body frame 310, and detects the user's touch. The odor sensor is a known sensor that applies the principle that electrical resistance changes due to the adsorption of odor-causing molecules.

The communication device 126 is a communication module that performs wireless communication with various external devices. The storage device 124 is composed of a nonvolatile memory and a volatile memory, and stores computer programs and various setting information. The processor 122 is a computer program execution means. Drive mechanism 120 includes a plurality of actuators and the wheel drive mechanism 370 described above. In addition, a display and speaker are also installed.

The driving mechanism 120 mainly controls the wheels (front wheels 102) and the head (head frame 316). In addition to changing the direction and speed of movement of the robot 100, the driving mechanism 120 can also raise and lower the wheels (front wheels 102 and rear wheels 103). When the wheels are raised, they are completely stored in the body 104, and the robot 100 comes into contact with the floor surface F at the seating surface 108, and enters a seated state. The driving mechanism 120 also controls the hands 106 via wires 134.

FIG. 4 is a functional block diagram of the robot system 300.
The robot system 300 includes a robot 100, a server 200, and a plurality of external sensors 114. Each component of the robot 100 and the server 200 is realized by hardware including computing units such as a CPU (Central Processing Unit) and various coprocessors, storage devices such as memory and storage, and wired or wireless communication lines connecting them, and software stored in the storage devices and supplying processing instructions to the computing units. The computer program may be configured by a device driver, an operating system, various application programs located at higher layers thereof, and a library that provides common functions to these programs. Each block described below shows a functional block, not a hardware-based configuration.
A part of the functions of the robot 100 may be realized by the server 200, and a part or all of the functions of the server 200 may be realized by the robot 100.

Multiple external sensors 114 are installed in the house beforehand. The position coordinates of the external sensors 114 are registered in the server 200. The server 200 determines the basic behavior of the robot 100 based on information obtained from the internal sensor 128 of the robot 100 and the multiple external sensors 114. The external sensors 114 are intended to reinforce the sensory organs of the robot 100, and the server 200 is intended to reinforce the brains of the robot 100. The communicator 126 of the robot 100 periodically communicates with the external sensors 114, and the server 200 identifies the position of the robot 100 using the external sensors 114 (see also Patent Document 2).

(Server 200)
The server 200 includes a communication unit 204 , a data processing unit 202 , and a data storage unit 206 .
The communication unit 204 is responsible for communication processing with the external sensor 114 and the robot 100. The data storage unit 206 stores various data. The data processing unit 202 executes various processes based on the data acquired by the communication unit 204 and the data stored in the data storage unit 206. The data processing unit 202 also functions as an interface between the communication unit 204 and the data storage unit 206.

The data storage section 206 includes a motion storage section 232 and a personal data storage section 218.
The robot 100 has multiple deformation patterns (motions). Various motions are defined, such as shaking the hand 106, approaching the owner while meandering, and staring at the owner with the head tilted.

The motion storage unit 232 stores a "motion file" that defines motion control details. Each motion is identified by a motion ID. The motion file is also downloaded to the motion storage section 160 of the robot 100. Which motion to execute may be determined by the server 200 or by the robot 100.

Most of the motions of the robot 100 are configured as compound motions including multiple unit motions. For example, when the robot 100 approaches the owner, it may be expressed as a combination of a unit motion of turning toward the owner, a unit motion of approaching while raising its hands, a unit motion of approaching while shaking its body, and a unit motion of sitting while raising both hands. . By combining these four motions, the robot approaches the owner, raises its hands midway, and finally shakes its body before sitting down. In the motion file, the rotation angle, angular velocity, etc. of the actuator provided in the robot 100 are defined in relation to the time axis. Various motions are expressed by controlling each actuator over time according to a motion file (actuator control information).

The transition time when changing from the previous unit motion to the next unit motion is called an "interval." The interval may be defined according to the time required to change a unit motion or the content of the motion. The length of the interval is adjustable.
Hereinafter, settings related to behavior control of the robot 100, such as when to select which motion and output adjustment of each actuator to realize the motion, will be collectively referred to as "behavior characteristics." The behavioral characteristics of the robot 100 are defined by a motion selection algorithm, a motion selection probability, a motion file, and the like.

The motion storage unit 232 stores motion files as well as a motion selection table that defines the motions to be executed when various events occur. In the motion selection table, one or more motions and their selection probabilities are associated with each event.

The personal data storage unit 218 stores user information. Specifically, master information indicating the degree of intimacy with the user and the physical and behavioral characteristics of the user is stored. Other attribute information such as age and gender may also be stored.

The robot 100 has an internal parameter called familiarity for each user. When the robot 100 recognizes actions that show favor toward the user, such as picking the user up or calling out to the user, the robot's intimacy with the user increases. The level of familiarity with users who are not involved with the robot 100, users who are abusive, and users with whom the robot 100 is rarely encountered is low.

The data processing unit 202 includes a position management unit 208 , a recognition unit 212 , a motion control unit 222 , an intimacy management unit 220 , and a state management unit 244 .
The position management unit 208 specifies the position coordinates of the robot 100. The state management unit 244 manages various internal parameters such as various physical states such as the charging rate, internal temperature, and the processing load of the processor 122. The state management unit 244 also manages various emotion parameters indicating the emotions of the robot 100 (loneliness, curiosity, desire for recognition, etc.). These emotion parameters are constantly fluctuating. The movement destination of the robot 100 changes depending on the emotion parameters. For example, when the robot 100 feels increasingly lonely, it sets the location of the user as the movement destination.

Emotional parameters change over time. Furthermore, various emotional parameters change depending on the reception behavior described below. For example, when the owner gives a hug, the emotional parameter indicating loneliness decreases, and when the owner is not seen for a long time, the emotional parameter indicating loneliness gradually increases.

The recognition unit 212 recognizes the external environment. Recognition of the external environment includes a variety of recognitions, such as recognition of weather and season based on temperature and humidity, and recognition of shade (safety zones) based on the amount of light and temperature. The recognition unit 156 of the robot 100 acquires various types of environmental information using the internal sensor 128, performs initial processing on this information, and then transfers it to the recognition unit 212 of the server 200.

Specifically, the recognition unit 156 of the robot 100 extracts an image region corresponding to a moving object, particularly a person or an animal, from the image, and extracts features indicating the physical characteristics and behavioral characteristics of the moving object from the extracted image region. Extract a "feature vector" as a set of quantities. Feature vector components (feature amounts) are numerical values that quantify various physical and behavioral features. For example, the width of the human eye is quantified in the range of 0 to 1 and forms one feature vector component. The method for extracting feature vectors from captured images of people is an application of known face recognition technology. Robot 100 sends the feature vector to server 200.

The recognition unit 212 of the server 200 compares the feature vector extracted from the image captured by the built-in camera of the robot 100 with the feature vectors of users (clusters) preregistered in the personal data storage unit 218 to determine which person the captured user corresponds to (user identification process). The recognition unit 212 also estimates the user's emotions by performing image recognition of the user's facial expressions. The recognition unit 212 also performs user identification process on moving objects other than people, such as pet cats and dogs.

The recognition unit 212 recognizes various behaviors given to the robot 100 and classifies them into pleasant or unpleasant behaviors. The recognition unit 212 also recognizes the owner's behavior in response to the behavior of the robot 100 and classifies it into positive or negative reactions.
A pleasant or unpleasant behavior is determined based on whether the user's behavior is pleasant or unpleasant for the living organism. For example, being held is a pleasant behavior for the robot 100, and being kicked is an unpleasant behavior for the robot 100. A positive or negative reaction is determined based on whether the user's behavior indicates a pleasant emotion or an unpleasant emotion. Being held is a positive reaction that indicates a pleasant emotion for the user, and being kicked is a negative reaction that indicates an unpleasant emotion for the user.

The motion control unit 222 of the server 200 determines the motion of the robot 100 in cooperation with the motion control unit 150 of the robot 100. The operation control unit 222 of the server 200 creates a movement target point for the robot 100 and a movement route therefor. The operation control unit 222 may create a plurality of movement routes, and then select one of the movement routes.

The motion control unit 222 selects a motion for the robot 100 from a plurality of motions in the motion storage unit 232. A selection probability is associated with each motion for each situation. For example, a selection method is defined such that when the owner performs a pleasant action, motion A is executed with a probability of 20%, and when the temperature is 30 degrees or higher, motion B is executed with a probability of 5%.

The intimacy management unit 220 manages intimacy for each user. As described above, the degree of intimacy is registered as part of the personal data in the personal data storage unit 218. When a pleasurable act is detected, the intimacy management unit 220 increases the intimacy with the owner. When an unpleasant act is detected, intimacy decreases. In addition, the intimacy level of owners who have not been visually recognized for a long time gradually decreases.

(Robot 100)
The robot 100 includes a communication section 142, a data processing section 136, a data storage section 148, an internal sensor 128, and a drive mechanism 120.
The communication unit 142 corresponds to the communication device 126 (see FIG. 4) and is in charge of communication processing with the external sensor 114, the server 200, and other robots 100. The data storage unit 148 stores various data. The data storage unit 148 corresponds to the storage device 124 (see FIG. 4). The data processing unit 136 executes various processes based on the data acquired by the communication unit 142 and the data stored in the data storage unit 148. The data processing unit 136 corresponds to the processor 122 and a computer program executed by the processor 122. The data processing unit 136 also functions as an interface for the communication unit 142, internal sensor 128, drive mechanism 120, and data storage unit 148.

The data storage unit 148 includes a motion storage unit 160 that defines various motions of the robot 100 .
Various motion files are downloaded from the motion storage unit 232 of the server 200 to the motion storage unit 160 of the robot 100. The motions are identified by motion IDs. In order to express various motions such as sitting down with the front wheels 102 folded, lifting the hands 106, rotating the two front wheels 102 in reverse or by rotating only one of the front wheels 102 to cause the robot 100 to rotate, trembling by rotating the front wheels 102 with the front wheels 102 folded, stopping once and looking back when moving away from the user, etc., the operation timing, operation time, operation direction, etc. of various actuators (drive mechanisms 120) are defined in a time series in the motion files.
Various data may also be downloaded to the data storage unit 148 from the personal data storage unit 218 .

The data processing section 136 includes a recognition section 156 and an operation control section 150.
The motion control unit 150 of the robot 100 determines the motion of the robot 100 in cooperation with the motion control unit 222 of the server 200. Some motions may be determined by the server 200, and other motions may be determined by the robot 100. Furthermore, the robot 100 determines the motion, but when the processing load on the robot 100 is high, the server 200 may determine the motion. The server 200 may determine a base motion, and the robot 100 may determine additional motions. How the motion determination processing is divided between the server 200 and the robot 100 may be designed according to the specifications of the robot system 300.

The motion control unit 150 of the robot 100 instructs the driving mechanism 120 to execute the selected motion. The driving mechanism 120 controls each actuator according to the motion file.

The motion control unit 150 can execute a motion of lifting both hands 106 as a gesture of asking to be "held" when a user with high intimacy is nearby, and can also express a motion of refusing to be held by alternately rotating in the opposite directions and stopping while keeping the left and right front wheels 102 retracted when the user gets tired of being "held." The drive mechanism 120 drives the front wheels 102, hands 106, and neck (head frame 316) according to instructions from the motion control unit 150, causing the robot 100 to express various motions.

The recognition unit 156 of the robot 100 interprets external information obtained from the internal sensor 128. The recognition unit 156 is capable of visual recognition (visual unit), smell recognition (olfactory unit), sound recognition (auditory unit), and tactile recognition (tactile unit).

The recognition unit 156 extracts a feature vector from the captured image of the moving object. As described above, the feature vector is a collection of parameters (feature amounts) indicating the physical characteristics and behavioral characteristics of a moving object. When a moving object is detected, physical and behavioral characteristics are also extracted from the odor sensor, built-in sound collection microphone, temperature sensor, etc. These features are also quantified and become feature vector components. The recognition unit 156 identifies the user from the feature vector based on the known technology described in Patent Document 2 and the like.

Of the series of recognition processes including detection, analysis, and judgment, the recognition unit 156 of the robot 100 selects and extracts the information necessary for recognition, and interpretation processes such as judgment are performed by the recognition unit 212 of the server 200. The recognition process may be performed by only the recognition unit 212 of the server 200, or by only the recognition unit 156 of the robot 100, or the above recognition process may be performed by both of them with their roles shared as described above.

When a strong impact is given to the robot 100, the recognition unit 156 recognizes this by the touch sensor and the acceleration sensor, and the recognition unit 212 of the server 200 recognizes that a "violent act" has been committed by a nearby user. When a user grabs the horn 112 and lifts the robot 100, this may also be recognized as a violent act. When a user facing the robot 100 speaks in a specific volume range and a specific frequency band, the recognition unit 212 of the server 200 may recognize that a "calling act" has been performed on the robot 100. In addition, when a temperature of about body temperature is detected, it recognizes that a "contact act" has been performed by the user, and when an upward acceleration is detected in a state where contact has been recognized, it recognizes that a "hug" has been performed. Physical contact when the user lifts the body 104 may be sensed, or a hug may be recognized by a decrease in the load on the front wheels 102.
In summary, the robot 100 acquires the user's action as physical information by the internal sensor 128, and the recognition unit 212 of the server 200 judges whether the action is comfortable or uncomfortable. The recognition unit 212 of the server 200 also executes a user identification process based on the feature vector.

The recognition unit 212 of the server 200 recognizes various responses of the user to the robot 100. Pleasure or displeasure, affirmation or negation are associated with some typical reception acts among various reception acts. In general, most of the pleasurable acts are positive reactions, and most of the unpleasant ones are negative reactions. Pleasant and unpleasant actions are related to intimacy, and positive and negative reactions influence the robot 100's behavior selection.

In response to the interaction behavior recognized by the recognition unit 156, the intimacy management unit 220 of the server 200 changes the intimacy with the user. In principle, the intimacy with a user who performs a pleasant behavior increases, and the intimacy with a user who performs an unpleasant behavior decreases.

Based on the above basic configuration, next, the implementation of the robot 100 in this embodiment will be described, focusing in particular on the features and purpose of this implementation and the differences from the basic configuration. The description will be divided into a first embodiment and a second embodiment. When the first embodiment and the second embodiment are described together or when no particular distinction is made, they will be referred to as "this embodiment."

[First embodiment]
Fig. 5A is a front external view of the robot 100 according to the first embodiment. Fig. 5B is a side external view of the robot 100 according to the first embodiment.
The robot 100 in the first embodiment is provided with a face region 107. In addition to eyes 110, a nose 109 is provided within the face region 107. The nose 109 is provided near the center of the face region 107 and at a lower position than the eyes 110. The nose 109 is provided between the left and right eyes, and the distance from the right eye to the nose 109 is equal to the distance from the left eye to the nose 109. The nose 109 is smaller than the eyes 110 and the face region 107. The nose 109 is provided with a physical device (hereinafter, sometimes referred to as a "convex portion" or a "protrusion") that is deformed by the user.

In this embodiment, an analog stick is installed on the nose 109. The user can tilt the analog stick (nose 109) in all directions, up, down, left and right, and can also push it. In this embodiment, tilting and pushing a physical device such as an analog stick corresponds to "deformation of the robot body." The analog stick includes a sensor (hereinafter referred to as a "nose sensor") for detecting touch. As for the method of detecting the analog stick itself and the deformation of the analog stick, the technique described in, for example, Japanese Patent Laid-Open No. 9-134251 is known.

FIG. 6 is a hardware configuration diagram of the robot 100 in the first embodiment.
The robot 100 in the first embodiment includes a monitor 170 and an analog stick 180 in addition to the basic configuration shown in FIG. The monitor 170 is installed on the eye 110 of the robot 100 and displays an eye image (details will be described later).

FIG. 7 is a functional block diagram of the robot system 300 in the first embodiment.
As mentioned above, robot 100 includes monitor 170 and analog stick 180. The data processing unit 136 of the robot 100 in the first embodiment further includes an operation detection unit 182, an operation history management unit 184, and an eye generation unit 172. The operation detection unit 182 sequentially detects the movement of the analog stick 180, that is, the inclination direction, the inclination angle (angle from the initial position), and the amount of depression. The inclination direction in the first embodiment may be referred to as a "deformation direction", and the inclination angle and the pushing amount may be collectively referred to as a "deformation amount".

The operation history management unit 184 stores the detection results of the operation detection unit 182 in chronological order as an operation history. The operation history also includes the duration (deformation duration) of the deformed state of the analog stick 180 (the state in which the analog stick 180 is moved from the initial position). The operation history management unit 184 also calculates the deformation speed as the amount of deformation per unit time. The degree of "deformation of the robot main body" in this embodiment, such as the deformation direction, deformation amount, and deformation speed, is collectively referred to as the "deformation mode". The eye generation unit 172 generates an eye image to be displayed on the eye 110 (monitor 170).

The eye generation unit 172 changes the eye image displayed on the monitor 170 by outputting a signal to the monitor 170 according to the deformation state of the analog stick 180. The operation control unit 150 also selects a motion according to the deformation state of the analog stick 180. Hereinafter, the change of the eye image by the eye generation unit 172 may be referred to as "the robot 100 moves its eyes" or "the eyes of the robot 100 change". Furthermore, the selection of a motion by the operation control unit 150 and the execution of the selected motion by the driving mechanism 120 may be referred to as "the robot 100 moves". Furthermore, when the "robot 100 moves", if the motion is to move a specific part X, it may be referred to as "the robot 100 moves X". For example, when the operation control unit 150 selects a motion to move the hand of the robot 100 and the driving mechanism 120 executes the motion, it may be expressed as "the robot 100 moves its hand". When the user manipulates the nose 109 (analog stick 180), the robot 100 moves its eyes, and when the user continues to manipulate the nose 109, the robot 100 starts to move not only its eyes but also its body, such as its neck and hands. In this embodiment, "motion" means that the robot 100 physically moves its body, and changes in the eye image are not included in "motion". Changes in the image and motion on the monitor 170 when the analog stick 180 is moved will be described in detail below.

FIG. 8 is a diagram showing the appearance of the eye image 171.
The eye generation unit 172 generates an eye image 171 including a pupil image 175 and a peripheral image 176. The eye generation unit 172 displays a moving image of the eye image 171. Specifically, the pupil image 175 is moved to represent a change in the line of sight of the robot 100.

The pupil image 175 includes a pupil region 177 and a cornea region 178. The pupil image 175 also displays a catch light 179 to represent the reflection of external light. The catch light 179 in the eye image 171 is not shining due to the reflection of external light, but is an image region represented as a high-luminance region by the eye generation unit 172.

The eye generation unit 172 moves the pupil image 175 up, down, left, and right. When the recognition unit 156 of the robot 100 recognizes a user, the eye generation unit 172 points the pupil image 175 in the direction of the user. The eye generation unit 172 expresses a change in the virtual line of sight of the robot 100 (hereinafter sometimes referred to as the "virtual line of sight") by changing the pupil image 175 of the eye image 171. The control of the eye image 171 will be described in detail later with reference to FIG. 10.

The eye generation unit 172 may change the shape of the pupil image 175. For example, when the pupil image 175 is in the center of the monitor 170, it is made to have a perfect circle shape, and when it is on the periphery, it is changed to an ellipse shape. By changing the shape of the pupil image 175 depending on the position within the monitor 170, it is possible to make the flat monitor 170 appear as if it has a curved shape like an actual eyeball.

The eye generation unit 172 changes the position of the catch light 179 in response to the direction of the external light source. Figure 8 shows the display position of the catch light 179 when the external light source is located to the upper left as seen by the robot 100. By linking the position of the catch light 179 to the external light source, a more realistic eye image 171 can be expressed. The eye generation unit 172 may determine the direction of the external light source from the captured image by image recognition, or may determine it from detection data of a light sensor (not shown).

FIG. 9 is an enlarged view of the eye image 171.
In the eye image 171, an eyelid image 190 showing the eyelid is superimposed on the pupil image 175 and the peripheral image 176. The eyelid image 190 includes eyelashes 192. The peripheral image 176 is a portion that corresponds to the human conjunctiva.

The eye generation unit 172 changes the eyelid image 190, pupil region 177, cornea region 178, and catch light 179 of the eye image 171. The greater the amount of light, the more the eye generation unit 172 intermittently or continuously enlarges the diameter of the pupil region 177. The eye generation unit 172 may intermittently or continuously enlarge or reduce not only the pupil region 177 but also the entire pupil image 175. When the amount of light is particularly large (for example, when it is greater than a predetermined threshold), the eye generation unit 172 may express a "dazzling appearance" by lowering the eyelid image 190.

FIG. 10 is a schematic diagram showing a method of generating the eye image 171.
The eyeball model 250 is three-dimensional computer graphics that imitates the eyeball of the robot 100. The eye generation unit 172 first forms a three-dimensional sphere using polygons, and forms an eyeball model 250 by applying a texture (hereinafter referred to as "eyeball texture") to this. The eyeball texture is an image including the pupil image 175. The eyeball texture is stored in an eye image storage section (not shown) included in the data storage section 148.

A first surface 252 and a second surface 254 are set in front of the eyeball model 250. The first surface 252 and the second surface 254 are virtual planes corresponding to the display surface of the monitor 170 of the eye 110. The eye generation unit 172 generates a two-dimensional eyeball projection image 256 from the three-dimensional eyeball model 250 by projecting the eyeball model 250 onto the first surface 252 .

The eye generation unit 172 displays the eyelid image 190 on the second surface 254. By superimposing the eyeball projection image 256 on the first surface 252 and the eyelid image 190 on the second surface 254, an eye image 171 shown in FIG. 8 and the like is generated. The eye generation unit 172 generates two eyeball models 250, one for the right eye and one for the left eye, and generates an eye image 171 for each. Hereinafter, the first surface 252 and the second surface 254 will be collectively referred to as "ocular surface 258."

The eye generation unit 172 changes the eyeball projection image 256 by rotating the eyeball model 250. Since this method generates a three-dimensional eyeball model 250 and projects it onto the first surface 252 while rotating it, it is possible to express the movement of the robot 100's gaze more smoothly than if the eye image 171 were drawn directly on the first surface 252. Even though the eye image 171 is two-dimensional, since it is generated and controlled based on the three-dimensional eyeball model 250, this method makes it easier to express the complex movements that are unique to the eyes of living creatures.

The eye generation unit 172 displays the eyelid image 190 on a second surface 254 that is different from the first surface 252, thereby superimposing the eyelid image 190 on the eyeball projection image 256. When people have their hands slapped in front of them, they reflexively close their eyes. In order to implement such a conditioned reflex of the eye in the robot 100, it is necessary to change the eyelid image 190 at high speed. In this embodiment, the image processing for the first surface 252 and the image processing for the second surface 254 are independent. When expressing closing the eyes, the eye generation unit 172 only needs to perform image control on the second surface 254. Even when blinking, the eye generation unit 172 may perform image processing on the second surface 254. Since the eyeball model 250 (eyeball projection image 256) and the eyelid image 190 can be controlled separately, the eyelid image 190 can be controlled at high speed. With such a configuration, the eye generation unit 172 can dynamically generate the eye image 171.

The robot 100 moves its eyes in conjunction with the movement of the analog stick 180. External views of the eyes of the robot 100 are shown in FIGS. 11(a), 11(b), 11(c), and 11(d).

FIG. 11A is an external view of the eyes of the robot 100 when the analog stick 180 is moved upward from the initial position 188.
FIG. 11(b) is an external view of the eyes of the robot 100 when the analog stick 180 is moved to the right from the initial position 188.
FIG. 11C is an external view of the eyes of the robot 100 when the analog stick 180 is moved downward from the initial position 188.
FIG. 11(d) is an external view of the eyes of the robot 100 when the analog stick 180 is moved to the left from the initial position 188.

When the user moves the analog stick 180, the operation detection unit 182 detects the amount and direction of deformation (the tilt angle and the tilt direction) of the analog stick 180. The eye generation unit 172 calculates the rotation direction and amount of rotation of the eyeball model 250 based on the detection result, and rotates the eyeball model 250. Specifically, the same direction as the tilt direction of the analog stick 180 is set as the rotation direction, and the eyeball model 250 is rotated by a rotation amount proportional to the tilt angle. The eye generation unit 172 projects the eyeball model 250 onto the first surface 252. Through such control, the eye image 171 of the robot 100 moves so as to direct its line of sight in the direction in which the analog stick 180 is tilted. The user can change the line of sight of the robot 100 by moving the analog stick 180.

As shown in FIG. 11(a), when the analog stick 180 is tilted upward, the robot 100 also turns its line of sight upward. As shown in FIG. 11(b), when the analog stick 180 is tilted to the right, the line of sight of the robot 100 is directed to the right. When the analog stick 180 is tilted downward, the line of sight of the robot 100 faces downward (FIG. 11(c)), and when it is tilted to the left, the line of sight of the robot 100 moves to the left (FIG. 11(d)). The same applies to directions other than up, down, left, and right.

By using the eyeball model 250, the eye image 171 can be made to quickly follow the movement of the analog stick 180. When the user manipulates the nose 109 (analog stick 180), the robot 100 moves its eyes instantly, allowing the user to experience a rapid response from the robot 100.

When the nose 109 is touched, the robot 100 may move not only its eyes but also its body (at least one of its head, hands, and wheels). When the analog stick 180 is moved quickly, if the robot 100 also responds by immediately moving its head or body, the robot 100 can express biological behavior characteristics called "reflexive behavior."

For example, when the user tilts the analog stick 180 to the right by 30 degrees or more within 0.1 seconds, the robot 100 may turn its head to the right. According to such a control method, by quickly and greatly tilting the nose 109 (analog stick 180), the robot 100 can express motion by moving not only its eyes but also its head.

The robot 100 not only moves its eyes in response to the movement of the nose 109, but also moves its body depending on the movement of the nose 109, thereby showing various reactions to the teasing of the nose 109. Other movements of the nose 109 that trigger the robot 100 to move its body may be the user flicking the analog stick 180 upwards or moving the analog stick 180 left and right at high speed. When the speed at which the analog stick 180 changes shape exceeds a predetermined reference value, a "sudden movement" may be defined, and when the analog stick 180 makes a sudden movement, the motion control unit 150 may select one of a number of motions. These motions may be any that express reflex actions performed by a living being, such as rotating the head in the opposite direction to the movement of the nose 109, or jumping backwards by driving the wheels.

In summary, when the nose 109 (analog stick 180) is moved, the eye generation unit 172 changes the eye image 171 by rotating the eyeball model 250 in accordance with the movement of the analog stick 180. Therefore, the line of sight of the robot 100 can be moved in a tracking manner by the movement of the nose 109. In other words, a user interface is realized in which the nose 109 acts as an eye operation device. Furthermore, when the movement of the nose 109 meets a predetermined condition, the operation control unit 150 selects a motion in accordance with the movement of the nose 109. Therefore, depending on how the nose 109 is moved, the robot 100 can respond to the user's interaction not only by the eye image 171 but also by the movement of the entire body.

The motion control unit 150 may execute a motion that follows the movement of the nose 109. In this manner, the eye generation unit 172 may change the eye image 171 in accordance with the movement of the analog stick 180. The motion control unit 150 may also change the motion in accordance with the movement of the analog stick 180. This gives the user the impression that the operation direction of the analog stick 180 and the change in the eye image 171 or the motion are linked.

As described in relation to Figures 5(a) and (b), the nose 109 is smaller than the eyes 110 and the face region 107. It is necessary to reliably detect the user's contact with any part of the nose 109. For this reason, it is necessary to provide sensors over the entire area or most of the nose 109. When providing sensors of the same size for the robot 100, fewer sensors are required for the nose 109, which is smaller than the face region 107. This allows the cost of manufacturing the robot 100 to be reduced.

Furthermore, although the nose 109 is small, it easily catches the eye of the user. The main reasons for this are that it is provided near the center of the face area 107 and that it has a convex shape with respect to the surface of the robot 100. By providing the analog stick 180 (operation device) on the nose 109, which is easily noticed by the user, the user can easily recognize the operable portion of the robot 100 (the portion that detects the user's touch).

The eye image 171 may be changed based on internal parameters of the robot 100, for example, emotional parameters. For example, when the value of the emotional parameter indicating the degree of irritation of the robot 100 is greater than or equal to a predetermined value, the eye generation unit 172 may rotate the eyeball model 250 in a direction opposite to the direction in which the analog stick 180 is tilted.

A conversion function may be prepared that defines the deformation state (deformation direction, deformation amount, deformation speed) of the analog stick 180 as a variable and defines the rotation direction and rotation amount of the eyeball model 250 as output. The eye generation unit 172 may determine the direction and amount of rotation of the eyeball model 250 based on the manner of deformation of the analog stick 180 and the conversion function.

The eye generation unit 172 may select one of a plurality of types of conversion functions based on the emotion parameters. The eye generation unit 172 may randomly select one of a plurality of types of conversion functions. With this control method, it is possible to provide diversity in the eye movements while maintaining the link between the deformation of the analog stick 180 and the eye movements. The robot 100 responds with a variety of eye movements to the same manner of manipulation, so the user can enjoy the various facial expressions of the robot 100. It is also possible to estimate the emotion of the robot 100 according to the eye movements.

The robot 100 changes its motion according to the operation history of the analog stick 180. As described above, the operation detection unit 182 sequentially detects the movement of the analog stick 180, that is, the direction and amount of deformation. The operation history management unit 184 stores the detection results in chronological order as an operation history. Specifically, the operation history management unit 184 stores the detection result and the detection time in association with each other. The motion control unit 150 selects a motion according to the operation history, that is, how the analog stick 180 moves.

The state management unit 244 may change the emotional parameters depending on how the analog stick 180 is played with (operated). For example, if the user gently tampers with the analog stick 180, the eye generation unit 172 begins to move the eyes in conjunction with the tampering. If the user continues to tamper with the user in the same way, the state management unit 244 increases the emotional parameter indicating a sense of security. When the emotional parameter indicating a sense of security exceeds a predetermined value, the eye generation unit 172 expresses the sleepiness of the robot 100 by lowering the eyelid image 190 (closing the eyes). At this time, the motion control unit 150 may express the state of "drowsing" by selecting a motion of moving the head up and down. The designer may arbitrarily define an easy way to play with the analog stick 180, or in other words, a way to play with the analog stick 180 that makes the robot 100 feel at ease. For example, the analog stick 180 may be reciprocated left and right at approximately the same pace, or the analog stick 180 may be kept stationary and the nose sensor may be struck a predetermined number of times or more with enough force to prevent the analog stick 180 from being pushed. It may be.

If the user continues to mess around with the analog stick 180, the robot 100 will begin to move its eyes in conjunction with the way the user messes with the analog stick 180. If the robot 100 continues to mess around with the robot 100, the emotion parameter indicating irritation increases and the robot 100 stops moving its eyes. Furthermore, the robot 100 moves its head left and right and attempts to release the user's fingertips from the analog stick 180, expressing "no, no". The "messy tinkering method" may also be arbitrarily defined by the designer.

In one embodiment, the state of being handled gently is a state of monotonous movement. If expressed in terms of the output value from the analog stick 180, monotonous movement is a movement in which the output value has periodicity and the amount of deformation is small and continues for a longer period than a predetermined period. In one embodiment, the state of being handled roughly is a state of being non-monotonous movement. If expressed in terms of the output value from the analog stick 180, it is a movement in which the output value has no periodicity and the amount of deformation is large and continues for a longer period than a predetermined period. In this way, the robot 100 not only continues to move its eyes in conjunction with the movement of the analog stick 180, but also changes the way in which the analog stick 180 and its eyes are linked when a predetermined condition is met. The robot 100 expresses a "sleepy" state by closing its eyelids and moving its body, rather than simply stopping its eye movements.

For example, when the analog stick 180 is moved alternately left and right within a range of 30% or less of the maximum tilt angle (hereinafter, this movement is referred to as "alternate left and right tilt"), the robot 100 may stop eye movement, close its eyelids, and move its head up and down to express a "sleepy" state. The operation history management unit 184 refers to the operation history and determines whether the movement of the analog stick 180 corresponds to "alternate left and right tilt". When it corresponds to alternating left and right tilt, the eye generation unit 172 stops the movement of the eyeball model 250 and displays the eyelid image 190. The operation control unit 150 also selects a motion that moves the head of the robot 100 up and down. As a result, the robot 100 closes its eyes and moves its head up and down to express a "sleepy" state.

Living things become sleepy when they feel safe. If the user plays with the robot 100 and the robot 100 makes a behavioral expression that it wants to sleep, the user can feel that the robot 100 is safe by playing with the robot 100. The state management unit 244 increases the emotional parameter indicating a sense of security when the left-right alternating tilt is detected. When the emotional parameter indicating a sense of security exceeds a predetermined value, the robot 100 may close its eyes and move its head up and down.

When the user holds the robot 100 sideways, the robot 100 occupies most of the user's field of vision. At this time, it becomes easier for the user to notice minute changes in the robot 100. The user can easily gaze at the nose 109 and eyes 110 of the robot 100 held sideways. When this state is detected, for example, when a state in which the user is holding the robot 100 sideways is detected from an image captured by the camera of the robot 100, when the user touches the nose 109, the movement of the analog stick 180 is linked. If the eyes change, the user can become immersed in the robot 100 without being distracted by anything other than the robot 100.

Living organisms respond insensitively to short-term stimuli, but can become active when exposed to long-term stimuli. The robot 100 can also express similar behavior by gradually increasing its movements as the user continues to touch it. As the user continues to play with the nose 109, the robot 100, which was initially moving only its eyes, begins to move its head as well. The motion control unit 150 may drive a portion farther from the nose 109 as the deformation duration becomes longer. For example, when the nose 109 is tampered with, the robot 100 moves the eye image 171, and when the nose 109 continues to be tampered with, the robot 100 slowly moves its head. If the user continues to play with the nose 109, the robot 100 may move its hand, and if the user continues to play with the nose 109, the robot 100 may move away from the user. The deformation duration may be defined as the duration of the tilt of the analog stick 180, or may be defined as the duration of the touch of the nose sensor.

As another example, when the user continues alternating left and right tilts for 10 seconds, the robot 100 moves its head up and down to express a state of "getting sleepy and beginning to doze off."

Next, assume that the user continues to alternately lean left and right for 20 seconds or more. In this case, the robot 100 may further move the hands 106 closer to the body 104 and store the front wheels 102 and rear wheels 103 in the body 104. If the user continues to alternately lean left and right in this way, the robot 100 first moves its eyes, then moves its head, becoming "drowsy," and then moves both its arms and legs closer to its body, becoming "asleep."

When the user tilts left and right alternately, the robot 100 may change its motion depending on whether the analog stick 180 is moved quickly or slowly. The operation history management unit 184 records the amount and direction of deformation of the analog stick 180 as an operation history. The operation history management unit 184 calculates the deformation speed (amount of change per unit time) from the operation history.

When the deformation duration is 10 seconds or more and the deformation speed during the 10 seconds is less than or equal to a predetermined value, the motion control unit 150 selects a motion that moves the head up and down. On the other hand, if the deformation speed exceeds the predetermined value, the motion control unit 150 selects a motion that moves the head left and right. Through such control, the robot 100 shakes its head sideways to express "no, no".

The robot 100 of the first embodiment changes its motion depending on how the user plays with the nose 109. The manner in which the analog stick 180 is deformed does not simply mean the direction in which the analog stick 180 is deformed. For example, it includes detailed movements such as whether to move the analog stick 180 small or large, that is, whether it is monotonous or not, and whether to move it quickly or slowly, that is, whether to touch it roughly or in a stroking manner. If the robot 100 changes its motion in response to minute differences in how the user manipulates the robot, the user can interact with the robot 100 without getting bored.

The time it takes for the robot 100 to reach the "sleepy" state may vary depending on the internal parameters of the robot 100, such as the robot's intimacy with the user and the state of its emotional parameters at any given time.

Suppose that the user fiddles with the nose 109 when the robot 100 has its eyes closed and has not moved or performed a motion for a predetermined period of time (when the robot 100 is sleeping). When the robot 100 detects movement of the analog stick 180 in this state, the robot 100 may change the eye image 171 from a closed eye image 171 to an open eye image 171 and perform a movement or motion. In this way, when the robot 100 detects a user's contact with a predetermined eye image 171 and is not performing a motion, the robot 100 may change the eye image 171 and perform a motion.

The motion of the robot 100 may be changed according to the intimacy. For example, when a user with an intimacy level equal to or higher than a predetermined value tweaks the nose 109, the robot 100 moves its head closer to the user. On the other hand, when a user with an intimacy level lower than the predetermined value tweaks the nose 109 in the same manner, the robot 100 turns its head away from the user. In this manner, the motion control unit 150 may select one of a plurality of motions according to the deformation state of the analog stick 180 and the intimacy level of the user who operates the analog stick 180. Specifically, when a predetermined operation input is made by a user with an intimacy level equal to or higher than T1, the motion control unit 150 may randomly select one of the motions M1 to M3, and when the same operation input is made by a user with an intimacy level lower than T1, the motion control unit 150 may randomly select one of the motions M4 to M6.

In order to develop attachment to something, humans need to spend as much time as possible with it. The behavior of the robot 100 of the first embodiment also changes depending on the degree of intimacy with the user. Since the motion of the robot 100 changes depending on the degree of familiarity even if the user tinkers in the same way, the user can continue interacting with the robot 100 without getting bored. By continuing to interact with the robot 100, the user can develop an attachment to the robot 100. Further, by continuing to interact with the user in a gentle manner, the intimacy of the robot 100 with the user increases. As the degree of intimacy changes, the reaction of the robot 100 also changes. The operation on the nose 109, the degree of familiarity (feelings of the robot 100 toward the user), and the emotional parameters (the mood of the robot 100) interact in a complex manner, allowing the robot 100 to realize various behavioral expressions depending on the situation.

When the user touches the nose 109, the robot 100 first moves its eyes, then its head, and then begins to move its body 104. Furthermore, in the robot 100, the "eyes move when you move the analog stick 180 of the nose 109" function (hereinafter, this function may be referred to as "eye linkage function") is enabled when the user moves the robot 100. , the condition may be that the user is holding the user sideways with his or her face facing toward the user.

Even if the user moves the analog stick 180 when the robot 100 is not held sideways, the eyes of the robot 100 may simply blink. The fact that the user is holding the robot 100 sideways is detected by a touch sensor (not shown) that detects contact with the surface of the robot 100 and an acceleration sensor (not shown) that detects the tilt of the robot 100. The recognition unit 156 detects the user's contact with the robot 100 via a touch sensor and an acceleration sensor. A posture determining section (not shown) identifies in what posture the user is in contact with the robot 100 based on the detection result detected by the recognition section 156. If it is determined that the user is holding the robot 100 sideways, the posture determining unit instructs the operation history management unit 184 to enable the pupil linkage function. The operation history management unit 184, which has received the instruction from the posture determination unit, enables the eye-linked function and activates the eye movement according to the movement of the analog stick 180 as described above. In this way, the behavioral characteristics of the robot 100 with respect to the operation of the nose 109 may be changed depending on the posture of the robot 100 such as hugging.

The posture determination unit may enable the pupil link function under the condition that the user and the robot 100 are in a predetermined positional relationship, the user's posture is a predetermined posture, etc. are detected based on a touch sensor, an acceleration sensor, or an image capture. For example, the pupil link function is enabled when the user holds the robot 100 sideways with the face of the robot 100 facing the user. Even if the robot 100 reacts when the user is not focusing on the robot 100, the user is likely to miss the reaction. By linking the analog stick 180 and the eye movement when the robot 100 is in this posture, the user can focus the user's attention on the robot 100 and notice small changes in the robot 100. Note that instead of detecting that the user holds the robot 100 sideways, the pupil link function may be enabled on the condition that the user is in a posture that allows the user to focus the user's attention on the robot 100 and notice small changes in the robot 100, such as sitting on the user's lap facing each other. Furthermore, the condition for enabling the pupil linkage function may be to detect whether the user and the robot 100 are facing each other using face recognition technology of the robot 100, not limited to a touch sensor or an acceleration sensor. The state in which the robot 100 has its wheels completely stored in the body 104 may be defined as the "hug mode", and the pupil linkage function may be enabled when the robot 100 is in the "hug mode" and it is determined by face recognition technology or the like that the user and the robot 100 are facing each other.

[Second embodiment]
The robot 100 of the second embodiment changes the movement of the eyes (eye movement) according to the trajectory (route pattern) of the nose 109 when the nose 109 is moved. Note that the robot 100 of the second embodiment has the same appearance as the robot 100 of the first embodiment.

FIG. 12 is a functional block diagram of a robot system 300 according to the second embodiment.
The data storage unit 148 of the robot 100 in the second embodiment further includes an eye movement storage unit 174. The eye movement storage unit 174 stores the movement (deformation pattern) of the analog stick 180 and the eye movement (eye movement) of the robot in association with each other. The motion storage unit 160 of the robot 100 in the second embodiment also stores motions according to the duration (deformation duration) of the deformed state of the analog stick 180 (the state in which the analog stick 180 is moved from the initial position).

The operation history management unit 184 of the robot 100 in the second embodiment stores the detection results of the operation detection unit 182 in chronological order as an operation history. The operation history management unit 184 compares the stored operation history with the execution conditions of the deformation pattern (described in detail later) stored in the eye movement storage unit 174, and when the operation history matches any of the execution conditions, The eye generation unit 172 is instructed to perform the eye movement associated with the deformation pattern. The eye generation unit 172 receives instructions from the operation history management unit 184 and executes eye movements. “Eye movement” means a change in the eye image 171 due to rotation of the eyeball model 250, as in the first embodiment.

The robot 100 in the second embodiment also changes the eye image 171 according to the operation history of the analog stick 180 provided in the nose 109. Further, a motion is selected depending on the deformation duration time, and the drive mechanism 120 of the robot 100 is controlled.

FIG. 13 is a flowchart showing the eye movements and motions of the robot 100 based on the operation history.
As described above, the operation history management unit 184 stores the detection results of the operation detection unit 182 in chronological order as an operation history (S10). The operation history management unit 184 judges whether the stored operation history matches any of the deformation patterns stored in the eye movement storage unit 174 (S12). If a matching deformation pattern exists (Y in S12), the operation history management unit 184 sequentially judges how long the deformation state continues based on the operation history (S14). If a matching deformation pattern does not exist (N in S12), the subsequent processing is skipped. If the deformation duration is less than 5 seconds (N in S14), the operation history management unit 184 instructs the eye generation unit 172 to execute an eye movement corresponding to the deformation pattern (S16). On the other hand, if the deformation duration is 5 seconds or more (Y in S14), the operation history management unit 184 instructs the eye generation unit 172 to execute an eye closing movement (S18), and issues an instruction to the operation control unit 150 to inquire about a motion from the motion storage unit 160 and have the drive mechanism 120 execute the motion (S20).

Specifically, the operation history management unit 184 of the robot 100 selects an eye motion that is previously associated with a deformation pattern, and instructs the eye generation unit 172 to execute it. For example, it is assumed that an eye movement of spinning the eyes is associated with a deformation pattern defined by a predetermined execution condition of rotating the analog stick 180 once clockwise. In this case, when the user rotates the analog stick 180 once clockwise, the robot 100 rotates its eyes. The motion control unit 150 also selects one of the one or more motions that are previously associated with the deformation duration. When the condition that the deformation duration is 5 seconds or more is satisfied, it is assumed that the motion of moving the head of the robot 100 from side to side is associated with this deformation duration. In this case, when the user continues to move the analog stick 180 for, for example, 6 seconds, the robot 100 shakes its head from side to side.

FIG. 14A is a data structure diagram of the eye movement table 420. FIG. 14B is a data structure diagram of the deformation pattern selection table 440 of the analog stick 180.
The eye movement table 420 defines the correspondence between the deformation pattern of the analog stick 180 and the eye movement. The modified pattern selection table 440 shows what parameters each modified pattern is defined with. The eye movement table 420 and the deformation pattern selection table 440 are stored in the eye movement storage section 174.

The operation history is a record of the time corresponding to the deformation direction and amount of deformation of the analog stick 180 in chronological order. By referring to the operation history, the path (trajectory) followed by the analog stick 180 can be identified. For example, the movement of the analog stick 180 that follows the path R1, which is the trajectory when turning once clockwise, and the movement following the path R1 within one second is defined as deformation pattern A1. In the second embodiment, the "deformation mode" of this embodiment includes the path. Similarly, the movement of the analog stick 180 that moves the analog stick 180 from the initial position to the left and right, follows the path R2, which is the trajectory when returning to the initial position, and the movement following the path R2 within two seconds is defined as deformation pattern A2. The movement of the analog stick 180 that follows the path R3, which is the trajectory when pressing from the initial position, and the movement following the path R3 within 0.5 seconds is defined as deformation pattern A3.

Each deformation pattern is associated with an eye movement. For example, in the case of deformation pattern A1, the eye generation unit 172 performs an action of spinning the eyes (eye action M1). In the case of modified pattern A2, the eye generation unit 172 performs a blinking action (eye action M2). In the case of deformation pattern A3, the eye generation unit 172 executes an action of bringing both eyes together (eye action M3).

When the user moves the analog stick 180, the operation detection unit 182 detects the movement of the analog stick 180 via the internal sensor 128. The movement is transmitted from the operation detection unit 182 to the operation history management unit 184, which compares the movement with the execution conditions of the deformation patterns stored in the eye movement storage unit 174 if the deformation duration is less than 5 seconds (the case where the deformation duration is 5 seconds or more will be described later). If the operation lasts less than 5 seconds and the movement of the analog stick 180 matches the execution conditions of any of the deformation patterns, the operation history management unit 184 identifies the eye movement associated with that deformation pattern. The operation history management unit 184 instructs the eye generation unit 172 to execute the identified eye movement. The eye generation unit 172 executes the identified eye movement.

Specifically, for example, assume that the user moves the analog stick 180 to follow path R1 and completes the movement in 0.5 seconds. Since this movement matches the execution condition for deformation pattern A1, the operation history management unit 184 identifies eye movement M1 that corresponds to deformation pattern A1, and the eye generation unit 172 performs eye movement M1, i.e., a movement of rotating the eyes.

In the second embodiment, the behavior of the robot 100 changes not only in response to eye movements but also in response to the duration of deformation of the analog stick 180.

FIG. 15 is a diagram showing the data structure of the motion selection table 460.
The motion selection table 460 defines the correspondence between the transformation duration and the motion. The motion selection table 460 is stored in the motion storage unit 160.

As described above, when the deformation duration is 5 seconds or more, the operation history management unit 184 instructs the execution of the action of closing the eyes (eye closing action). The eye generation unit 172 executes the eye closing action based on the instruction from the operation history management unit 184.
The motion control unit 150 identifies motion C1, which is a motion corresponding to a deformation duration of 5 seconds or more, from the motion storage unit 160. Motion C1 is a motion of moving the head of the robot 100 from side to side. The robot 100 closes its eyes and shakes its head from side to side, demonstrating the behavior of "turning its face away."

If the deformation duration is 15 seconds or more, the motion control unit 150 selects motion C2. Motion C2 is a movement in which the robot 100 raises and lowers its hand. The motion control unit 150 activates motion C1 and motion C2. Then, the robot 100 moves through motions C1 and C2, and expresses the "no, no" motion of closing its eyes and flapping its hands while shaking its head.

When the duration of the transformation is short, the motion control unit 150 selects a motion that drives the parts of the robot 100 from the nose 109 to the head, which is within a predetermined range. When the duration of the transformation is long, motions that drive parts of the robot 100 outside the head are also selected.

The robot 100 in the second embodiment moves its eyes in accordance with the movement of an analog stick 180 provided on its nose 109. As analog stick 180 continues to be moved, robot 100 begins to move its head. If the analog stick 180 continues to be moved even after the head starts moving, the robot 100 starts moving its body. In this way, as the user continues to play with the nose 109, the robot 100 begins to move its eyes and then its head and body, causing the robot 100 to become tired of being touched, start to feel ticklish, etc. like a living thing. You can feel that the robot 100 is behaving in a similar manner, and you can experience the feeling that you are interacting with a living creature.

In the second embodiment, the analog stick 180 is installed on the nose 109. When communicating, humans often look at the other person's face. The same is true when interacting with a robot, and it is thought that the user will touch the robot 100 while looking at the facial area 107 of the robot 100. At this time, if there is something that the user unconsciously wants to touch on a part that catches the user's eye, such as the face or nose, the user is likely to touch it unconsciously. In the robot 100 of the second embodiment, this is the nose 109. When the analog stick 180 of the nose 109, which is the part that the user unconsciously wants to touch, is moved, the robot 100 behaves in accordance with the movement, so the user gets the impression that the robot 100 is alive and becomes more attached to it.

Fig. 16 is a front view of the robot 100 in a stationary state, in which the robot 100 is standing still and the user is not touching the robot 100.
Fig. 17 is a front view of the robot 100 when motion C1 is activated. Fig. 17 is a diagram showing a state when the robot 100 activates motion C1 as a result of the user continuously touching the analog stick 180 of the robot 100.
In a stationary state where the user is not moving the analog stick 180, the eyes 110 of the robot 100 are open and the robot 100 stares straight ahead. If the user continues to move the analog stick 180 and the deformation duration exceeds 5 seconds, the robot 100 closes the eyes 110 and turns its head away.

The robot 100 has been described above based on the embodiment.
In the robot 100 of this embodiment, an analog stick 180 is provided on the nose 109 .
Since the robot 100 responds to the user's touch of the nose 109, which is easily touched unconsciously, the user feels as if the robot 100 is communicating with the user, and is likely to become attached to the robot 100.

The robot 100 of this embodiment changes its eyes according to the movement of the analog stick 180. Furthermore, if the analog stick 180 is tampered with for a long time, the robot 100 begins to move its head and body. When the user slightly fiddles with the analog stick 180, the robot 100 moves its eyes to appear interested in the user. If the user continues to play with the robot 100, the robot 100 moves its head and pretends to avoid the user's hand, expressing the feeling of "stop". If the robot 100 is too picky, it will start flapping its hands to express "No, no!". The robot 100 of this embodiment changes its expression and behavior not only by the movement of the analog stick 180 but also by the duration of the movement. The user feels that the robot 100 has emotions such as "bored," "ticklish," "annoying," and "do more," and can sense that the robot 100 behaves in a biological manner.

In the robot 100 of this embodiment, when the user plays with an analog stick 180 provided on the nose 109, the eyes change. Furthermore, if the analog stick 180 is played with for a long time, the robot 100 starts to move its head, and if the analog stick 180 is played with for a long time, it starts to move its body. When the user tampers with the nose 109, the behavior of the robot 100 is first shown in the area located in the face area 107 near the nose 109, so the user does not miss the reaction of the robot 100. If you continue to play with it, not only the face area 107 but also the inside of the head of the robot 100 within a predetermined range will begin to move, and if you continue to play with it further, the parts outside the head of the robot 100 (outside the predetermined range) will also start to move. The user can enjoy watching the facial expressions and behavior of the robot 100 that change depending on how the nose 109 is played with and the amount of time the nose 109 is played with.

The robot 100 of this embodiment changes its eyes in response to the movement of the analog stick 180 provided on the nose 109. As a modified example, an image of the user may be captured by an imaging device, the user's movement may be detected from the captured image by a motion sensor, and the eyes may then be changed in response to the user's movement. However, compared to detecting the user's movement through image processing such as a motion sensor, detecting the movement of the analog stick 180 can shorten the time from the user's contact with the robot 100 to the behavior of the robot 100. Furthermore, the user's contact with the robot 100 can be reliably detected.

In this embodiment, an analog stick 180 (operation device) is provided directly on the body of the robot 100. In a modified example, an operation device such as a remote controller may be provided separately from the body of the robot 100. However, when the operation device is provided directly on the body of the robot 100, the user directly touches the robot 100. Therefore, it is easier for the user to intuitively operate and control the robot 100 compared to when the operation device is provided separately from the body of the robot 100.

It should be noted that the present invention is not limited to the above-described embodiments and modified examples, and can be embodied by modifying the constituent elements without departing from the scope of the invention. Various inventions may be formed by appropriately combining a plurality of components disclosed in the above embodiments and modified examples. Furthermore, some components may be deleted from all the components shown in the above embodiments and modified examples.
Other embodiments will be described below.

In the embodiment, the operation detection unit 182 detected the movement (deformation) of the analog stick 180, and determined the behavior of the robot 100 according to the detection result. What is detected by the operation detection unit 182 is not limited to this, and may be any deformation of the robot 100 body or movement of a device provided on the robot 100 body. The detected deformation or movement may be a physical movement such as a movement of the robot 100 shaking its neck, a movement of grasping a hand, a movement of shaking its body, or a deformation of cheeks caused by pinching by the user. Further, the means for detecting these motions may be any means for detecting physical motions such as a joystick, a trackball, or the like. It is also possible to arrange the analog stick 180 and the like inside the flexible outer skin so that these detection means are not exposed on the surface of the robot 100.

In the embodiment, the nose 109 is the location where the operation detection unit 182 detects contact. The location where contact is detected is not limited to the nose 109. For example, it may be any part where contact can be detected, such as a rounded part such as an animal's paw pad, a protruding part such as a tail, or a part where deformation is expected such as a cheek. Preferably, it is a location that the user would like to tinker with unconsciously.

In the embodiment, it is assumed that when the nose 109 is touched, the eyes move. The moving part of the robot 100 is not limited to the eyes, but may be any movable part such as the neck, tail, hands, etc. Further, the location where the touch is detected by the operation detection unit 182 and the location of the action may be close or far apart, for example, the nose may move when the tail is touched.

In the embodiment, the head of the robot 100 is set as a predetermined range, and if the nose 109 is touched for 5 seconds or more and less than 15 seconds, only the head of the robot 100 is touched, and if it is 15 seconds or more, the head of the robot 100 is touched. In addition, the hand 106, which is a part outside the predetermined range, was also moved. The "predetermined range" that defines the location to be driven according to the deformation of the robot 100 body or the movement of a device provided on the robot 100 body is not limited to the head of the robot 100. For example, it may be the face area 107 of the robot 100, or it may be a part of the robot 100 other than the legs. It is sufficient to set an appropriate range among the parts that can be moved by the drive mechanism of the robot 100.

In addition, the transformation duration conditions for activating each motion are not limited to 5 seconds or more or 15 seconds or more, but can be set to various times, such as 1 second or less or 1 minute or more.

In the embodiment, the motion is changed according to the time period during which the nose 109 is continuously being played with (deformation duration time). Not only the deformation duration time, but also the deformation mode of the robot 100 body such as the path and deformation speed, the movement of a device provided in the robot 100, etc. may be used as conditions for selecting a motion.

The eyes or movements of the robot 100 may be changed depending on the cumulative value of the deformation amount, the cumulative value of the deformation duration, or the pressure applied to the robot 100. For example, when touching the nose 109 with a weak force, the transition to eye movement, head movement, and whole body movement is gradual, so it may be sufficient to only change the eyes for a long time. When fiddling with a strong force, it may be possible to transition to a movement of the entire body in a short period of time. As a result, the behavior of the robot 100 can be more varied depending on how it is manipulated.
Unlike a touch such as simply stroking or hitting, the robot 100 of the present invention responds to a touch that involves "deformation." In other words, contact is detected not only by electrical contact detection but also by physical "deformation". The ``methods'' of tampering, such as the strength of tampering, the amount of deformation, and the contact area, vary depending on the user. If the robot 100 responds in a richly expressive manner according to the "touching methods" that differ from user to user, the user will become attached to the robot 100.

In the embodiment, the behavior of the robot 100 is determined according to the movement of the analog stick 180. The trigger for determining the behavior of the robot 100 is not limited to this, and for example, a sensor may be provided at a location where it is expected that the user will unconsciously want to touch it. In addition, multiple sensors may be provided on the robot 100 body. When multiple sensors are provided, one of them may be a sensor that detects the movement of the analog stick 180. In addition, all of the sensors may have the same detection accuracy, or may have different detection accuracy. When multiple sensors with different detection accuracy are provided, a sensor with higher detection accuracy may be provided at a location where it is expected that the user will want to touch it. In this case, the detection accuracy is the detection sensitivity to the amount of change when the user changes the contact point on the robot 100 body. Hereinafter, the detection of the user's contact by any of the sensors provided on the robot 100 body may be referred to as "the user touches the robot 100" or "the user contacts the robot 100".

An example of a sensor provided other than the nose 109 is a sensor that detects the user's contact with two positions on the surface of the robot 100 that sandwich the eye 110. When the user's contact is detected by this sensor, the eye generation unit 172 lowers the eyelid image 190 (closes the eye). This makes it possible to express the behavior of the robot 100 closing its eyes because the eye 110 is covered.

In the case where multiple sensors are provided on the robot 100 body, when any of the sensors detects the user's contact, the eye generation unit 172 may generate an eye image 171 in which the virtual line of sight of the robot 100 is directed toward the contact point. Hereinafter, the eye generation unit 172 may display the eye image 171 with the virtual line of sight directed toward a specific point on the monitor 170 for a specific period of time, which may be referred to as "the robot 100 gazing at a specific point." For example, when the user touches the hand of the robot 100, the robot 100 gazes at the hand. Alternatively, when the user touches the stomach of the robot 100, the robot 100 gazes at the stomach. In addition, when the user's contact exceeds a specific period of time, the robot 100 may move a part including the contact point. For example, when the user touches the hand of the robot 100 for more than 5 seconds, the robot 100 makes a motion of trying to remove the hand of the robot 100 from the user's hand. In addition, when the user touches the stomach of the robot 100 for more than 10 seconds, the robot 100 makes a motion of shaking its stomach from side to side. In this way, when any of the multiple sensors provided at multiple locations on the robot 100 detects a user's touch, the eye generation unit 172 generates an eye image 171 gazing at the contact point. Furthermore, when the detection time of the user's touch at any of the sensors exceeds a predetermined time, the operation control unit 150 executes a motion to move the part including the contact point. This allows the user to feel that the robot 100 is exhibiting a biological behavior of gazing intently at the touched part and then twisting its body.

If highly accurate sensors are attached throughout the entire body of the robot 100, power consumption will increase. By making only some of the sensors highly accurate, power consumption can be reduced. Further, it is assumed that only the sensors at the locations where the user is expected to want to tamper are used with high accuracy to detect even small movements of the user, and the behavior of the robot 100 corresponding to the movements is shown. By doing this, it is possible to express the characteristic of living creatures that ``they have a weak reaction to touching other parts of their body, but are sensitive to touch at a certain point.'' In addition, even if the user makes a slight touch, the robot 100 detects the contact and responds, so the user feels that the robot 100 responds to even the slightest movement of the user, and is able to communicate with the robot 100. You can feel the feeling of being there. Note that the detection accuracy is not limited to the detection sensitivity to the amount of change when the user changes the point of contact with the robot 100 body, but may be the ability to detect the strength of contact, the range of contact, etc., for example. Furthermore, a physically moving sensor may be used as a high-precision sensor, a capacitive touch sensor may be used as a low-precision sensor, or the types of sensors may be the same but the detection accuracy may be different. Various types of sensors can be employed, and in addition to the above-mentioned sensors, for example, a sensor that provides haptic feedback when the sensor detects a user's touch may be used.

In the embodiment, the eye movements of the robot 100 are shown as eye movement M1, which is a movement of rotating the eyes, eye movement M2, which is a blinking movement, and eye movement M3, which is a movement of bringing both eyes together. The eye movements exhibited by the robot 100 are not limited to these. For example, any action that can be performed by the robot 100 may be used, such as an action of averting the line of sight, an action of opening the eyes, an action of squinting the eyes, or the like.

In the embodiment, the motions of the robot 100 are shown as motion C1, which is a motion that moves the head left and right, and motion C2, which is a motion that moves the hand up and down. The motions exhibited by the robot 100 are not limited to these. For example, move your head vertically to "nod," tilt your head to "crawl," tuck your hands behind your body, lean your body forward to "bow," and move your legs to "bow." Any movement that can be realized by the robot 100 may be used, such as ``backing away'' or stopping movement and ``ignoring''.

In the embodiment, the nose 109 is smaller than the eyes 110 and the face region 107. The size of the nose 109 is not limited to this, and it may be the same size as or larger than the eyes 110. Furthermore, the size relationship of each part such as the eyes 110 and the face region 107 is not limited to the embodiment, and can be changed in design as appropriate.

In the present invention, when a user tampers with the nose 109 of the robot 100, the robot 100 responds in accordance with the way the user tampers with the nose 109. A sensor is provided that can detect changes in areas that are likely to be touched unconsciously by the user. When the user tampers with that part, if the eyes of the robot 100 move, the facial expression changes, and the body begins to move depending on how the user tampers with the area, the user will develop an attachment to the robot 100. If the reaction of the robot 100 changes depending on how the user manipulates the robot 100, the time the user spends touching the robot 100 becomes longer. If the user's contact time with the robot 100 can be extended, the user's attachment to the robot 100 can be deepened. Furthermore, there are no animals that move their eyes when their nose is touched. Even humans don't do that kind of thing. Even if you did, it would be difficult to move your eyes in the same direction as someone else's nose. It can be said that this is a form of expression that is only possible with Robot 100.

Claims (15)

an operation detection unit that detects deformation of the robot body;
an eye generation unit that generates an eye image and displays the eye image in a face area of the robot;
The robot is characterized in that, when a deformation of the robot main body is detected, the eye generation unit changes the eye image according to the deformation state. The robot according to claim 1, wherein the eye generation unit changes the eye image based on one or more of a deformation direction, a deformation amount, and a deformation speed as the deformation mode. Equipped with a motion control unit that selects the robot's motion,
1 . The motion control unit selects a motion corresponding to the deformation state of the robot body when the deformation state of the robot body satisfies a predetermined condition after the change of the eye image is started. Or the robot described in 2. The robot according to claim 3, characterized in that the motion selected when the predetermined condition is met is a motion that drives the head of the robot. Equipped with an operation history management unit that records the deformation state of the robot body over time as an operation history,
The robot according to claim 1, wherein the eye generation unit changes the eye image according to the operation history. a motion control unit that selects the motion of the robot;
an operation history management unit that records the deformation state of the robot body over time as an operation history;
6. The robot according to claim 1, wherein the motion control section changes the motion according to the operation history. The robot according to claim 5 or 6, wherein the operation history includes a duration of the deformed state. The robot according to any one of claims 1 to 7, characterized in that the operation detection unit detects deformation of a protrusion provided in a face area of the robot body. the protrusion is a protrusion provided on a portion of the robot body that corresponds to a nose,
The robot according to claim 8 , wherein the operation detection unit detects a deformation of the protrusion. a posture determination unit for determining a posture of the robot;
The robot according to any one of claims 1 to 9, characterized in that, on condition that the posture determination unit identifies that the robot is being picked up by a user, the eye generation unit changes the eye image in accordance with the deformation state. A motion control unit that selects the motion of the robot;
a drive mechanism for performing the selected motion;
An operation detection unit that detects a movement of a device provided on the robot body,
The robot is characterized in that, when movement of the device is detected, the operation control unit selects a motion corresponding to the movement of the device from among motions that are to be driven within a predetermined range of the device. The motion control unit drives a part of the robot within the predetermined range when the movement of the device is detected, and the movement of the device satisfies a predetermined condition after starting to drive the part. 12. The robot according to claim 11, wherein the robot further includes a drive target outside the predetermined range. a first sensor that is installed on a body surface of the robot and detects a touch by a user;
a second sensor provided in a face region of the robot and configured to detect a user's contact with the robot with higher accuracy than the first sensor;
an eye generating unit that generates an eye image and displays the eye image in the face area;
The robot is characterized in that the eye generation unit changes the eye image in accordance with an amount of change in the detection value of the second sensor. 14. The robot according to claim 13, wherein the second sensor is provided in a narrower area than the first sensor and detects the amount of deformation of the robot's body surface caused by a user's touch. A protruding part that resembles a nose,
An eye generating unit that expresses a line of sight by moving a pupil,
The robot is characterized in that the eye generation unit changes the pupil in accordance with the deformation state of the convex portion.