JPWO2020009098A1 - robot - Google Patents

Abstract

The robot generates an eye image, an motion control unit that selects the motion of the robot, a drive mechanism that executes the selected motion, an operation detection unit that detects deformation of the robot body, and an eye image in the face area of the robot. It is provided with an eye generation unit for displaying.

Description

The present invention relates to a robot that humans tend to attach to.

Humans keep pets in search of healing. On the other hand, many people give up on pets for various reasons, such as not having enough time to take care of them, not having a living environment to keep pets, allergies, and hard bereavement. If there is a robot that plays the role of a pet, it may be possible to provide healing that a pet gives to a person who cannot keep a pet (see Patent Documents 1 and 2).

Japanese Unexamined Patent Publication No. 2000-323219 International Publication No. 2017/169826

In recent years, robot technology has been rapidly advancing, but it has not yet realized its presence as a companion like a pet. I don't think robots have free will. By observing behaviors that seem to be the pet's free will, humans feel the existence of the pet's free will, sympathize with the pet, and are healed by the pet.

The present inventors thought that if the reaction of the robot changes depending on how it is touched, the attachment of humans to the robot may be further enhanced. As a result of diligent research, the present inventors have come up with the idea of solving the above problem by providing a sensor at a place where a human unconsciously wants to tamper with, detecting how the human tampering is detected, and reflecting it in the behavior of the robot.

The present invention is an invention completed based on the above-mentioned problem recognition, and a main object thereof is to provide a robot that humans can easily attach to.

The robot according to an aspect of the present invention generates an eye image, a motion control unit that selects a motion of the robot, a drive mechanism that executes the selected motion, an operation detection unit that detects deformation of the robot body, and the robot. It is provided with an eye generation unit for displaying an eye image in the face area of the robot.
When the deformation of the robot body is detected, the eye generation unit changes the eye image according to the deformation mode.

The robot according to another aspect of the present invention includes a motion control unit that selects the motion of the robot, a drive mechanism that executes the selected motion, and an operation detection unit that detects the movement of a device provided in the robot body.
When the movement of the device is detected, the motion control unit selects a motion corresponding to the movement of the device from the motions to be driven within a predetermined range from the device.

The robot according to another aspect of the present invention includes a motion control unit that selects a motion of the robot, a drive mechanism that executes the selected motion, and first and first devices that are installed on the body surface of the robot and detect user contact. It is equipped with two sensors.
When the user's contact is detected by both or one of the first and second sensors, the motion control unit selects a motion according to the contact mode.
The detection accuracy of the first sensor is set to be higher than the detection accuracy of the second sensor, and the first sensor is installed in the face region of the robot.

According to the present invention, it becomes easy for humans to have an attachment to a robot.

FIG. 1A is a front view of the robot. FIG. 1B is a side view of the robot. It is sectional drawing which shows the structure of a robot schematicly. It is a hardware block diagram of a robot. It is a functional block diagram of a robot system. FIG. 5A is a front view of the robot according to the present embodiment. FIG. 5B is a side view of the robot according to the present embodiment. It is a hardware block diagram of the robot in this embodiment. It is a functional block diagram of the robot system in 1st Embodiment. It is an external view of an eye image. It is an enlarged view of an eye image. It is a schematic diagram which shows the generation method of an eye image. FIG. 11A is an external view of the eyes of the robot when the analog stick is moved upward from the initial position. FIG. 11B is an external view of the eyes of the robot when the analog stick is moved to the right from the initial position. FIG. 11C is an external view of the eyes of the robot when the analog stick is moved downward from the initial position. FIG. 11D is an external view of the eyes of the robot when the analog stick is moved from the initial position to the left. It is a functional block diagram of the robot system in 2nd Embodiment. It is a flowchart which shows the eye movement and the motion of a robot. FIG. 14A is a data structure diagram of the eye movement table. FIG. 14B is a data structure diagram of the deformation pattern selection table. It is a data structure diagram of a motion selection table. It is a front view of a robot at a fixed time. It is a front view of a robot at the time of motion activation.

Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings. In the following description, for convenience, the positional relationship of each structure may be expressed with reference to the illustrated state. Further, with respect to the following embodiments and modifications thereof, the same reference numerals will be given to substantially the same constituent requirements, and the description thereof will be omitted as appropriate.

The robot of the present embodiment responds to tampering with the nose by moving the eyes and body when the user touches the nose. Hereinafter, the basic configuration of the robot will be described before the implementation of the robot in the present embodiment is described.

[Basic configuration]
FIG. 1A is a front view of the robot 100. FIG. 1B is a side view of the robot 100.
The robot 100 in this embodiment is an autonomous action type robot that determines an action based on an external environment and an internal state. The external environment is recognized by various sensors such as cameras and thermosensors. The internal state is quantified as various parameters expressing the emotions of the robot 100. The robot 100 has an action range inside the owner's home. Hereinafter, the human being involved in the robot 100 is referred to as a "user".

The body 104 of the robot 100 has an overall rounded shape and includes an exodermis formed of a soft and elastic material such as urethane, rubber, resin, or fiber. The robot 100 may be dressed. The total weight of the robot 100 is about 5 to 15 kilograms, and the height is about 0.5 to 1.2 meters. Due to various attributes such as appropriate weight, roundness, softness, and good touch, the effect that the user can easily hold the robot 100 and want to hold it is realized.

The robot 100 includes a pair of front wheels 102 (left wheel 102a, right wheel 102b) and one rear wheel 103. The front wheels 102 are the driving wheels, and the rear wheels 103 are the driven wheels. Although the front wheels 102 do not have a steering mechanism, the rotation speed and the rotation direction can be individually controlled. The rear wheel 103 is a caster and is rotatable in order to move the robot 100 back and forth and left and right. The rear wheel 103 may be an omni wheel.

The front wheels 102 and the rear wheels 103 can be completely housed in the body 104 by a drive mechanism (rotation mechanism, link mechanism). Most of each wheel is hidden in the body 104 even during traveling, but when each wheel is completely housed in the body 104, the robot 100 becomes immovable. That is, the body 104 descends with the retracting operation of the wheels and sits on the floor surface F. In this seated state, the flat seating surface 108 (ground contact bottom surface) formed on the bottom of the body 104 comes into contact with the floor surface F.

Robot 100 has two hands 106. The hand 106 does not have a function of gripping an object. The hand 106 can perform simple operations such as raising, shaking, and vibrating. The two hands 106 can also be individually controlled.

The eye 110 can display an image by a liquid crystal element or an organic EL element. The robot 100 is equipped with various sensors such as a microphone array and an ultrasonic sensor that can specify the direction of a sound source. It also has a built-in speaker and can emit simple voice.

A horn 112 is attached to the head of the robot 100. Since the robot 100 is lightweight as described above, the user can also lift the robot 100 by grasping the horn 112. An omnidirectional camera is attached to the tsuno 112, and the entire upper part of the robot 100 can be imaged at once.

FIG. 2 is a cross-sectional view schematically showing the structure of the robot 100.
As shown in FIG. 2, the body 104 of the robot 100 includes a base frame 308, a body frame 310, a pair of resin wheel covers 312 and an exodermis 314. The base frame 308 is made of metal and constitutes the axis of the body 104 and supports the internal mechanism. The base frame 308 is configured by connecting the upper plate 332 and the lower plate 334 vertically by a plurality of side plates 336. Sufficient spacing is provided between the plurality of side plates 336 to allow ventilation. A battery 118, a control circuit 342, and various actuators are housed inside the base frame 308.

The main body frame 310 is made of a resin material and includes a head frame 316 and a body frame 318. The head frame 316 has a hollow hemisphere and forms the head skeleton of the robot 100. The body frame 318 has a stepped tubular shape and forms the body skeleton of the robot 100. The body frame 318 is fixed integrally with the base frame 308. The head frame 316 is assembled to the upper end of the body frame 318 so as to be relatively displaceable.

The head frame 316 is provided with three axes, a yaw axis 320, a pitch axis 322, and a roll axis 324, and an actuator 326 for rotationally driving each axis. Actuator 326 includes a plurality of servomotors for individually driving each axis. The yaw shaft 320 is driven for the swinging motion, the pitch shaft 322 is driven for the nodding motion, and the roll shaft 324 is driven for the tilting motion.

A plate 325 that supports the yaw shaft 320 is fixed to the upper part of the head frame 316. The plate 325 is formed with a plurality of ventilation holes 327 for ensuring ventilation between the upper and lower parts.

A metal base plate 328 is provided to support the head frame 316 and its internal mechanism from below. The base plate 328 is connected to the plate 325 via a cross-link mechanism 329 (pantograph mechanism), while being connected to the upper plate 332 (base frame 308) via a joint 330.

The fuselage frame 318 accommodates the base frame 308 and the wheel drive mechanism 370. The wheel drive mechanism 370 includes a rotating shaft 378 and an actuator 379. The lower half of the body frame 318 is narrowed to form a storage space S for the front wheels 102 with the wheel cover 312.

The outer skin 314 is made of urethane rubber and covers the main body frame 310 and the wheel cover 312 from the outside. The hand 106 is integrally molded with the exodermis 314. An opening 390 for introducing outside air is provided at the upper end of the exodermis 314.

FIG. 3 is a hardware configuration diagram of the robot 100.
The robot 100 includes an internal sensor 128, a communication device 126, a storage device 124, a processor 122, a drive mechanism 120, and a battery 118. The processor 122 and the storage device 124 are included in the control circuit 342. Each unit is connected to each other by a power supply line 130 and a signal line 132. The battery 118 supplies electric power to each unit via the power supply line 130. Each unit sends and receives control signals via the signal line 132. The battery 118 is a lithium ion secondary battery and is a power source for the robot 100.

The internal sensor 128 is an assembly of various sensors built in the robot 100. Specifically, it includes a camera (omnidirectional camera), a microphone array, a ranging sensor (infrared sensor), a thermo sensor, a touch sensor, an acceleration sensor, an odor sensor, and the like. The touch sensor is installed between the outer skin 314 and the main body frame 310 to detect the user's touch. The odor sensor is a known sensor that applies the principle that the electrical resistance changes due to the adsorption of molecules that cause odors.

The communication device 126 is a communication module that performs wireless communication for various external devices. The storage device 124 is composed of a non-volatile memory and a volatile memory, and stores computer programs and various setting information. The processor 122 is a means for executing a computer program. The drive mechanism 120 includes a plurality of actuators and the wheel drive mechanism 370 described above. In addition, a display and speakers are also installed.

The drive mechanism 120 mainly controls the wheels (front wheels 102) and the head (head frame 316). The drive mechanism 120 can change the moving direction and moving speed of the robot 100, and can also raise and lower the wheels (front wheels 102 and rear wheels 103). When the wheels are raised, the wheels are completely housed in the body 104, and the robot 100 comes into contact with the floor surface F at the seating surface 108 to be in a seated state. Further, the drive mechanism 120 controls the hand 106 via the wire 134.

FIG. 4 is a functional block diagram of the robot system 300.
The robot system 300 includes a robot 100, a server 200, and a plurality of external sensors 114. Each component of the robot 100 and the server 200 includes hardware including a CPU (Central Processing Unit), a computing unit such as various coprocessors, a storage device such as memory and storage, and a wired or wireless communication line connecting them, and storage. It is realized by software that is stored in the device and supplies processing commands to the arithmetic unit. A computer program may be composed of a device driver, an operating system, various application programs located on the upper layers thereof, and a library that provides common functions to these programs. Each block described below shows a block for each function, not a configuration for each hardware.
A part of the functions of the robot 100 may be realized by the server 200, and a part or all of the functions of the server 200 may be realized by the robot 100.

A plurality of external sensors 114 are installed in advance in the house. The position coordinates of the external sensor 114 are registered in the server 200. Based on the information obtained from the internal sensor 128 of the robot 100 and the plurality of external sensors 114, the server 200 determines the basic action of the robot 100. The external sensor 114 is for reinforcing the sensory organs of the robot 100, and the server 200 is for reinforcing the brain of the robot 100. The communication device 126 of the robot 100 periodically communicates with the external sensor 114, and the server 200 identifies the position of the robot 100 by the external sensor 114 (see also Patent Document 2).

(Server 200)
The server 200 includes a communication unit 204, a data processing unit 202, and a data storage unit 206.
The communication unit 204 is in charge of communication processing with the external sensor 114 and the robot 100. The data storage unit 206 stores various data. The data processing unit 202 executes various processes based on the data acquired by the communication unit 204 and the data stored in the data storage unit 206. The data processing unit 202 also functions as an interface between the communication unit 204 and the data storage unit 206.

The data storage unit 206 includes a motion storage unit 232 and a personal data storage unit 218.
The robot 100 has a plurality of deformation patterns (motions). Various motions are defined, such as shaking the hand 106, approaching the owner while meandering, and staring at the owner with his neck bent.

The motion storage unit 232 stores a "motion file" that defines the control content of the motion. Each motion is identified by a motion ID. The motion file is also downloaded to the motion storage unit 160 of the robot 100. Which motion is executed may be determined by the server 200 or by the robot 100.

Most of the motions of the robot 100 are configured as compound motions including a plurality of unit motions. For example, when the robot 100 approaches the owner, it may be expressed as a combination of a unit motion of turning toward the owner, a unit motion of approaching while raising a hand, a unit motion of approaching while shaking the body, and a unit motion of sitting while raising both hands. .. By combining these four motions, the motion of "approaching the owner, raising his hand on the way, and finally shaking his body and sitting down" is realized. In the motion file, the rotation angle, the angular velocity, and the like of the actuator provided in the robot 100 are defined in relation to the time axis. Various motions are expressed by controlling each actuator with the passage of time according to the motion file (actuator control information).

The transition time when changing from the previous unit motion to the next unit motion is called "interval". The interval may be defined according to the time required to change the unit motion and the content of the motion. The length of the interval is adjustable.
Hereinafter, the settings related to the behavior control of the robot 100, such as when and which motion to select, and the output adjustment of each actuator in realizing the motion, are collectively referred to as "behavior characteristics". The behavioral characteristics of the robot 100 are defined by a motion selection algorithm, a motion selection probability, a motion file, and the like.

In addition to the motion file, the motion storage unit 232 stores a motion selection table that defines a motion to be executed when various events occur. In the motion selection table, one or more motions and their selection probabilities are associated with the event.

The personal data storage unit 218 stores user information. Specifically, it stores master information indicating intimacy with the user and the physical and behavioral characteristics of the user. Other attribute information such as age and gender may be stored.

The robot 100 has an internal parameter called intimacy for each user. When the robot 100 recognizes an action that shows a favor to itself, such as picking it up or calling out to it, the intimacy with the user becomes high. The intimacy with users who are not involved in the robot 100, users who work violently, and users who meet infrequently is low.

The data processing unit 202 includes a position management unit 208, a recognition unit 212, an operation control unit 222, an intimacy management unit 220, and a state management unit 244.
The position management unit 208 specifies the position coordinates of the robot 100. The state management unit 244 manages various internal parameters such as the charge rate, the internal temperature, and various physical states such as the processing load of the processor 122. In addition, the state management unit 244 manages various emotion parameters indicating the emotions (loneliness, curiosity, desire for approval, etc.) of the robot 100. These emotional parameters are constantly fluctuating. The movement target point of the robot 100 changes according to the emotional parameter. For example, when loneliness is increasing, the robot 100 sets the place where the user is as a movement target point.

Emotional parameters change over time. In addition, various emotional parameters change depending on the response action described later. For example, the emotional parameter indicating loneliness decreases when the owner "hugs", and the emotional parameter indicating loneliness gradually increases when the owner is not visually recognized for a long period of time.

The recognition unit 212 recognizes the external environment. Recognition of the external environment includes various recognitions such as recognition of weather and seasons based on temperature and humidity, and recognition of shadows (safety zones) based on the amount of light and temperature. The recognition unit 156 of the robot 100 acquires various environmental information by the internal sensor 128, performs primary processing on the information, and then transfers the information to the recognition unit 212 of the server 200.

Specifically, the recognition unit 156 of the robot 100 extracts an image area corresponding to a moving object, particularly a person or an animal, from the image, and a feature showing the physical characteristics and behavioral characteristics of the moving object from the extracted image area. Extract the "feature vector" as a set of quantities. The feature vector component (feature amount) is a numerical value that quantifies various physical and behavioral features. For example, the width of the human eye is quantified in the range 0 to 1 to form one feature vector component. The method of extracting a feature vector from a captured image of a person is an application of a known face recognition technique. The robot 100 transmits the feature vector to the server 200.

The recognition unit 212 of the server 200 is imaged by comparing the feature vector extracted from the image captured by the built-in camera of the robot 100 with the feature vector of the user (cluster) registered in advance in the personal data storage unit 218. It is determined which person the user corresponds to (user identification process). In addition, the recognition unit 212 estimates the user's emotion by recognizing the user's facial expression as an image. The recognition unit 212 also performs user identification processing on moving objects other than humans, for example, cats and dogs that are pets.

The recognition unit 212 recognizes various response actions performed by the robot 100 and classifies them into pleasant / unpleasant actions. The recognition unit 212 also classifies the robot 100 into affirmative / negative reactions by recognizing the owner's response to the action of the robot 100.
Pleasant / unpleasant behavior is determined by whether the user's response behavior is comfortable or unpleasant as a living thing. For example, being hugged is a pleasant act for the robot 100, and being kicked is an unpleasant act for the robot 100. The affirmative / negative reaction is determined by whether the user's response action indicates the user's pleasant feelings or unpleasant feelings. Being hugged is a positive reaction that indicates the user's pleasant feelings, and being kicked is a negative reaction that indicates the user's unpleasant feelings.

The motion control unit 222 of the server 200 cooperates with the motion control unit 150 of the robot 100 to determine the motion of the robot 100. The operation control unit 222 of the server 200 creates a movement target point of the robot 100 and a movement route for the movement target point. The operation control unit 222 may create a plurality of movement routes and then select one of the movement routes.

The motion control unit 222 selects the motion of the robot 100 from the plurality of motions of the motion storage unit 232. Each motion is associated with a selection probability for each situation. For example, a selection method is defined such that motion A is executed with a probability of 20% when the owner makes a pleasant act, and motion B is executed with a probability of 5% when the temperature rises above 30 degrees Celsius. ..

The intimacy management unit 220 manages the intimacy of each user. As described above, the intimacy is registered as part of the personal data in the personal data storage unit 218. When the pleasant behavior is detected, the intimacy management unit 220 raises the intimacy with the owner. Intimacy goes down when discomfort is detected. In addition, the intimacy of owners who have not been visually recognized for a long period of time gradually decreases.

(Robot 100)
The robot 100 includes a communication unit 142, a data processing unit 136, a data storage unit 148, an internal sensor 128, and a drive mechanism 120.
The communication unit 142 corresponds to the communication device 126 (see FIG. 4), and is in charge of communication processing with the external sensor 114, the server 200, and the other robot 100. The data storage unit 148 stores various data. The data storage unit 148 corresponds to the storage device 124 (see FIG. 4). The data processing unit 136 executes various processes based on the data acquired by the communication unit 142 and the data stored in the data storage unit 148. The data processing unit 136 corresponds to the processor 122 and the computer program executed by the processor 122. The data processing unit 136 also functions as an interface for the communication unit 142, the internal sensor 128, the drive mechanism 120, and the data storage unit 148.

The data storage unit 148 includes a motion storage unit 160 that defines various motions of the robot 100.
Various motion files are downloaded from the motion storage unit 232 of the server 200 to the motion storage unit 160 of the robot 100. The motion is identified by the motion ID. A state in which the front wheels 102 are housed, in which the robot 100 is rotated by accommodating and sitting on the front wheels 102, lifting the hand 106, rotating the two front wheels 102 in the reverse direction, or rotating only one of the front wheels 102. In order to express various motions such as trembling by rotating the front wheel 102, stopping once when leaving the user, and looking back, the operation timing, operation time, operation direction, etc. of various actuators (drive mechanism 120) are set. It is defined in time series in the motion file.
Various data may be downloaded to the data storage unit 148 from the personal data storage unit 218 as well.

The data processing unit 136 includes a recognition unit 156 and an operation control unit 150.
The motion control unit 150 of the robot 100 determines the motion of the robot 100 in cooperation with the motion control unit 222 of the server 200. Some motions may be determined by the server 200, and other motions may be determined by the robot 100. Further, the robot 100 determines the motion, but when the processing load of the robot 100 is high, the server 200 may determine the motion. The server 200 may determine the base motion and the robot 100 may determine additional motions. How the motion determination process is shared between the server 200 and the robot 100 may be designed according to the specifications of the robot system 300.

The motion control unit 150 of the robot 100 instructs the drive mechanism 120 to execute the selected motion. The drive mechanism 120 controls each actuator according to the motion file.

The motion control unit 150 can also perform a motion of lifting both hands 106 as a gesture to plead for "holding" when a user with high intimacy is nearby, and when tired of "holding", the left and right front wheels 102 can be used. It is also possible to express a motion that dislikes a hug by alternately repeating reverse rotation and stop while it is housed. The drive mechanism 120 causes the robot 100 to express various motions by driving the front wheels 102, the hands 106, and the neck (head frame 316) according to the instructions of the motion control unit 150.

The recognition unit 156 of the robot 100 interprets the external information obtained from the internal sensor 128. The recognition unit 156 is capable of visual recognition (visual part), odor recognition (smell part), sound recognition (auditory part), and tactile recognition (tactile part).

The recognition unit 156 extracts a feature vector from the captured image of the moving object. As described above, the feature vector is a set of parameters (features) indicating the physical features and behavioral features of a moving object. When a moving object is detected, physical characteristics and behavioral characteristics are extracted from an odor sensor, a built-in sound collecting microphone, a temperature sensor, and the like. These features are also quantified and become feature vector components. The recognition unit 156 identifies the user from the feature vector based on the known technique described in Patent Document 2 and the like.

Of a series of recognition processes including detection, analysis, and determination, the recognition unit 156 of the robot 100 selects and extracts information necessary for recognition, and interpretation processing such as determination is executed by the recognition unit 212 of the server 200. .. The recognition process may be performed only by the recognition unit 212 of the server 200, or may be performed only by the recognition unit 156 of the robot 100, or the above recognition process is executed while both parties share roles as described above. May be good.

When a strong impact is applied to the robot 100, the recognition unit 156 recognizes this by the touch sensor and the acceleration sensor, and the recognition unit 212 of the server 200 recognizes that the "rough act" has been performed by a nearby user. When the user grabs the horn 112 and lifts the robot 100, it may be recognized as a violent act. When a user facing the robot 100 speaks in a specific volume region and a specific frequency band, the recognition unit 212 of the server 200 may recognize that a "calling action" has been made to itself. Further, when a temperature of about body temperature is detected, it is recognized that a "contact act" has been performed by the user, and when an upward acceleration is detected in the state of contact recognition, it is recognized that a "hug" has been performed. The user may sense the physical contact when lifting the body 104, or may recognize the hug by reducing the load applied to the front wheels 102.
In summary, the robot 100 acquires the user's action as physical information by the internal sensor 128, and the recognition unit 212 of the server 200 determines whether it is comfortable or unpleasant. Further, the recognition unit 212 of the server 200 executes the user identification process based on the feature vector.

The recognition unit 212 of the server 200 recognizes various user responses to the robot 100. Some typical responses of the various responses are associated with pleasant or unpleasant, affirmative or negative. In general, most of the pleasant actions are positive reactions, and most of the unpleasant actions are negative reactions. Pleasure / discomfort is related to intimacy, and affirmative / negative reactions affect the action selection of the robot 100.

The intimacy management unit 220 of the server 200 changes the intimacy with respect to the user according to the response action recognized by the recognition unit 156. In principle, the intimacy with the user who performed the pleasant act increases, and the intimacy with the user who performed the unpleasant act decreases.

On the premise of the above basic configuration, next, the implementation of the robot 100 in the present embodiment will be described focusing on the features and purposes of the implementation and the differences from the basic configuration. The description will be divided into the first embodiment and the second embodiment. When the first embodiment and the second embodiment are described together, or when there is no particular distinction, they are referred to as "the present embodiment".

[First Embodiment]
FIG. 5A is a front view of the robot 100 according to the first embodiment. FIG. 5B is a side view of the robot 100 according to the first embodiment.
The robot 100 in the first embodiment is provided with a face region 107. In addition to the eyes 110, the nose 109 is provided in the face area 107. The nose 109 is provided near the center of the face region 107 and at a position lower than the eyes 110. The nose 109 is provided so as to be sandwiched between the left and right eyes, and the distance from the right eye to the nose 109 is equal to the distance from the left eye to the nose 109. Also, the nose 109 is smaller than the eyes 110 and the face area 107. The nose 109 is provided with a physical device that is deformed by the user (hereinafter sometimes referred to as a "convex" or "protrusion").

In this embodiment, an analog stick is installed on the nose 109. The user can tilt the analog stick (nose 109) in all directions, up, down, left, and right, and can also push it in. In this embodiment, tilting and pushing of a physical device such as an analog stick corresponds to "deformation of the robot body". The analog stick includes a sensor for detecting touch (hereinafter referred to as "nose sensor"). As a method for detecting the analog stick itself and the deformation of the analog stick, for example, the techniques described in JP-A-9-134251 are known.

FIG. 6 is a hardware configuration diagram of the robot 100 according to the first embodiment.
The robot 100 in the first embodiment includes a monitor 170 and an analog stick 180 in addition to the basic configuration shown in FIG. The monitor 170 is installed on the eyes 110 of the robot 100 to display an eye image (details will be described later).

FIG. 7 is a functional block diagram of the robot system 300 according to the first embodiment.
As mentioned above, the robot 100 includes a monitor 170 and an analog stick 180. The data processing unit 136 of the robot 100 in the first embodiment further includes an operation detection unit 182, an operation history management unit 184, and an eye generation unit 172. The operation detection unit 182 sequentially detects the movement of the analog stick 180, that is, the inclination direction, the inclination angle (angle from the initial position), and the pushing amount. The tilting direction in the first embodiment is called a "deformation direction", and the tilt angle and the pushing amount may be collectively called a "deformation amount".

The operation history management unit 184 stores the detection result of the operation detection unit 182 as an operation history in chronological order. The operation history also includes the duration (deformation duration) of the deformed state of the analog stick 180 (the state in which the analog stick 180 is moved from the initial position). The operation history management unit 184 also calculates the deformation speed as the amount of deformation per unit time. The degree of "deformation of the robot body" in the present embodiment, such as the deformation direction, the amount of deformation, and the deformation speed, is collectively referred to as the "deformation mode". The eye generation unit 172 generates an eye image to be displayed on the eye 110 (monitor 170).

The eye generation unit 172 changes the eye image displayed on the monitor 170 by outputting a signal to the monitor 170 according to the deformation mode of the analog stick 180. The motion control unit 150 also selects a motion according to the deformation mode of the analog stick 180. Hereinafter, changing the eye image by the eye generation unit 172 may be referred to as "the robot 100 moves the eyes" or "the eyes of the robot 100 change". Further, when the motion control unit 150 selects a motion and the drive mechanism 120 executes the selected motion, it may be referred to as "the robot 100 moves". Further, in the case of "moving the robot 100", if it is a motion to move a specific part X, it may be said that "the robot 100 moves X". For example, when the motion control unit 150 selects a motion for moving the hand of the robot 100 and the drive mechanism 120 executes the motion, it may be expressed as "the robot 100 moves the hand". When the user tampers with the nose 109 (analog stick 180), the robot 100 moves the eyes, and when the user continues to tamper with the nose 109, the robot 100 begins to move not only the eyes but also the body such as the neck and hands. The "motion" in the present embodiment means that the robot 100 physically moves the body, and the change in the eye image is not included in the "motion". Hereinafter, changes in the image and motion of the monitor 170 when the analog stick 180 is moved will be described in detail.

FIG. 8 is an external view of the eye image 171.
The eye generation unit 172 generates an eye image 171 including a pupil image 175 and a peripheral image 176. The eye generation unit 172 displays the eye image 171 as a moving image. Specifically, the change in the line of sight of the robot 100 is expressed by moving the pupil image 175.

The pupil image 175 includes a pupil region 177 and a corneal region 178. In addition, the pupil image 175 also displays a catch light 179 for expressing the reflection of external light. The catch light 179 of the eye image 171 is not shining due to the reflection of external light, but is an image region expressed as a high-luminance region by the eye generation unit 172.

The eye generation unit 172 moves the pupil image 175 up / down / left / right. When the recognition unit 156 of the robot 100 recognizes the user, the eye generation unit 172 directs the pupil image 175 in the direction in which the user exists. The eye generation unit 172 expresses a change in the virtual line of sight (hereinafter, may be referred to as “virtual line of sight”) of the robot 100 by changing the pupil image 175 of the eye image 171. Details of the control of the eye image 171 will be described later in relation to FIG.

The eye generation unit 172 may change the shape of the pupil image 175. For example, when the pupil image 175 is in the center of the monitor 170, it is changed to a perfect circular shape, and when it is in the peripheral portion, it is changed to an elliptical shape. By changing the shape of the pupil image 175 according to the position in the monitor 170, the flat monitor 170 can be made to look like a curved surface shape like an actual eyeball.

The eye generation unit 172 changes the position of the catch light 179 according to the presence direction of the external light source. FIG. 8 shows the display position of the catch light 179 when the external light source is present on the upper left side when viewed from the robot 100. By linking the position of the catch light 179 with the external light source, a more realistic eye image 171 can be expressed. The eye generation unit 172 may determine the direction of the external light source from the captured image by image recognition, or may determine from the detection data of the optical sensor (not shown).

FIG. 9 is an enlarged view of the eye image 171.
In the eye image 171, the eyelid image 190 showing the eyelids is superimposed on the pupil image 175 and the peripheral image 176. The eyelid image 190 includes eyelashes 192. Peripheral image 176 is a portion corresponding to the human conjunctiva.

The eye generation unit 172 changes the eyelid image 190, the pupil region 177, the corneal region 178, and the catch light 179 of the eye image 171. As the amount of light increases, the eye generator 172 increases the diameter of the pupil region 177 intermittently or continuously. The eye generation unit 172 may enlarge or contract not only the pupil region 177 but also the entire pupil image 175 intermittently or continuously. The eye generation unit 172 may express a "dazzling appearance" by lowering the eyelid image 190 when the amount of light is particularly large (for example, when it is larger than a predetermined threshold value).

FIG. 10 is a schematic view showing a method of generating an eye image 171.
The eyeball model 250 is three-dimensional computer graphics that imitates the eyeball of the robot 100. The eye generation unit 172 first forms a three-dimensional sphere with polygons, and attaches a texture (hereinafter, referred to as “eyeball texture”) to the three-dimensional sphere to form an eyeball model 250. The eyeball texture is an image including a pupil image 175. The eyeball texture is stored in an eye image storage unit (not shown) included in the data storage unit 148.

The first surface 252 and the second surface 254 are set in front of the eyeball model 250. The first surface 252 and the second surface 254 are virtual planes corresponding to the display surfaces of the monitor 170 of the eyes 110. The eye generation unit 172 generates a two-dimensional eyeball projection image 256 from the three-dimensional eyeball model 250 by projecting the eyeball model 250 onto the first surface 252.

The eye generation unit 172 causes the eyelid image 190 to be displayed on the second surface 254. By superimposing the eyeball projection image 256 on the first surface 252 and the eyelid image 190 on the second surface 254, the eye image 171 shown in FIG. 8 or the like is generated. The eye generation unit 172 generates two eye model 250s for the right eye and the left eye, and generates an eye image 171 for each. Hereinafter, when the first surface 252 and the second surface 254 are collectively referred to as "eye ball 258".

The eye generation unit 172 changes the eyeball projection image 256 by rotating the eyeball model 250. Since it is a method of generating a three-dimensional eyeball model 250 and projecting it on the first surface 252 while rotating it, the movement of the line of sight of the robot 100 is smoother than drawing the eye image 171 directly on the first surface 252. Can be expressed. Even the two-dimensional eye image 171 is generated and controlled based on the three-dimensional eyeball model 250, so that this method can easily express complicated movements peculiar to the eyeball of an organism.

The eye generation unit 172 superimposes the eyelid image 190 on the eyeball projection image 256 by displaying the eyelid image 190 on the second surface 254 different from the first surface 252. Humans reflexively close their eyes when they clap their hands in front of them. In order to implement such conditional reflection of the eye on the robot 100, it is necessary to change the eyelid image 190 at high speed. In the present embodiment, the image processing of the first surface 252 and the image processing of the second surface 254 are independent. When expressing the eyes closed, the eye generation unit 172 may control the image only on the second surface 254. Even when blinking, the eye generation unit 172 may perform image processing on the second surface 254. Since the eyeball model 250 (eyeball projection image 256) and the eyelid image 190 can be controlled separately, the eyelid image 190 can be controlled at high speed. With such a configuration, the eye generation unit 172 can dynamically generate the eye image 171.

The robot 100 moves its eyes in conjunction with the movement of the analog stick 180. The external views of the eyes of the robot 100 are shown in FIGS. 11 (a), 11 (b), 11 (c), and 11 (d).

FIG. 11A is an external view of the eyes of the robot 100 when the analog stick 180 is moved upward from the initial position 188.
FIG. 11B is an external view of the eyes of the robot 100 when the analog stick 180 is moved from the initial position 188 to the right.
FIG. 11C is an external view of the eyes of the robot 100 when the analog stick 180 is moved downward from the initial position 188.
FIG. 11D is an external view of the eyes of the robot 100 when the analog stick 180 is moved from the initial position 188 to the left.

When the user moves the analog stick 180, the operation detection unit 182 detects the deformation amount and the deformation direction (tilt angle and tilt direction) of the analog stick 180. The eye generation unit 172 calculates the rotation direction and the amount of rotation of the eyeball model 250 based on the detection result, and rotates the eyeball model 250. Specifically, the same direction as the tilt direction of the analog stick 180 is set as the rotation direction, and the eyeball model 250 is rotated by a rotation amount proportional to the tilt angle. The eye generation unit 172 projects the eyeball model 250 onto the first surface 252. By such control, the eye image 171 of the robot 100 moves so as to direct the line of sight in the tilting direction of the analog stick 180. The user can change the line of sight of the robot 100 by moving the analog stick 180.

As shown in FIG. 11A, when the analog stick 180 is tilted upward, the robot 100 also directs its line of sight upward. As shown in FIG. 11B, when the analog stick 180 is tilted to the right, the line of sight of the robot 100 turns to the right. When the analog stick 180 is tilted downward, the line of sight of the robot 100 faces downward (FIG. 11 (c)), and when tilted to the left, the line of sight of the robot 100 moves to the left (FIG. 11 (d)). The same applies to directions other than up, down, left, and right.

By using the eyeball model 250, the eye image 171 can be made to follow the movement of the analog stick 180 at high speed. When the user touches the nose 109 (analog stick 180), the robot 100 immediately moves the eyes, so that the user can experience a quick reaction from the robot 100.

The robot 100 may move not only the eyes but also the body (at least one of the head, hands and wheels) when the nose 109 is tampered with. When the analog stick 180 is moved quickly, if the robot 100 also shows a reaction by immediately moving the head and body, the robot 100 can express a biological behavior characteristic called "reflexive behavior".

For example, when the user tilts the analog stick 180 to the right by 30 degrees or more within 0.1 seconds, the robot 100 may turn its head to the right. According to such a control method, by tilting the nose 109 (analog stick 180) quickly and greatly, the robot 100 can express the motion of moving not only the eyes but also the head.

The robot 100 not only links the eyes to the movement of the nose 109, but also moves the body depending on the movement of the nose 109, thereby exhibiting various reactions to the tampering of the nose 109. Other movements of the nose 109 that trigger the robot 100 to move the body may be the user flipping the analog stick 180 upward or moving the analog stick 180 left and right at high speed. When the deformation speed of the analog stick 180 exceeds a predetermined reference value, it is defined as "rapid movement", and when the analog stick 180 makes a sudden movement, the motion control unit 150 selects one of a plurality of motions. You may. These motions may represent reflexive behaviors performed by living things, such as rotating the head in the direction opposite to the direction in which the nose 109 moves, driving the wheels to fly backwards, and the like.

In summary, when the nose 109 (analog stick 180) is moved, the eye generation unit 172 changes the eye image 171 by rotating the eyeball model 250 according to the movement of the analog stick 180. Therefore, the line of sight of the robot 100 can be continuously moved by the movement of the nose 109. So to speak, a user interface is realized as if the nose 109 is an eye operating device. Further, when the movement of the nose 109 satisfies a predetermined condition, the motion control unit 150 selects the motion according to the movement of the nose 109. Therefore, depending on how the nose 109 is moved, the robot 100 can respond to the user's involvement not only by the eye image 171 but also by the movement of the whole body.

The motion control unit 150 may execute a motion that follows the movement of the nose 109. In this way, the eye generation unit 172 may change the eye image 171 according to the movement of the analog stick 180. Further, the motion control unit 150 may change the motion according to the movement of the analog stick 180. This gives the user the impression that the operating direction of the analog stick 180 and the change in the eye image 171 or the motion are linked.

As described in relation to FIGS. 5A and 5B, the nose 109 is smaller than the eyes 110 and the face area 107. It is necessary to reliably detect user contact in any part of the nose 109. Therefore, it is necessary to provide sensors in the entire area or most of the nose 109. When sensors of the same size are provided for the robot 100, the number of sensors for the nose 109, which is smaller than the face area 107, can be reduced. Therefore, the cost of manufacturing the robot 100 can be suppressed.

Moreover, although the nose 109 is small, it is easily noticed by the user. The main factors are that it is provided near the center of the face region 107 and that it has a convex shape with respect to the surface of the robot 100. By providing the analog stick 180 (operation device) on the nose 109, which is easily noticed by the user, the user can easily recognize the operable part (the part for detecting the user's contact) in the robot 100.

The eye image 171 may be changed based on the internal parameters of the robot 100, for example, the emotional parameters. For example, when the value of the emotion parameter indicating the degree of irritation of the robot 100 is equal to or higher than a predetermined value, the eye generation unit 172 may rotate the eyeball model 250 in the direction opposite to the tilting direction of the analog stick 180.

A conversion function may be prepared in which the deformation mode (deformation direction, deformation amount, deformation speed) of the analog stick 180 is used as a variable, and the rotation direction and rotation amount of the eyeball model 250 are defined as outputs. The eye generation unit 172 may determine the rotation direction and the amount of rotation of the eyeball model 250 based on the deformation mode and the conversion function of the analog stick 180.

The eye generation unit 172 may select one of a plurality of types of conversion functions based on emotional parameters. The eye generation unit 172 may randomly select any one of a plurality of types of conversion functions. According to such a control method, it is possible to give diversity to the movement of the eye while maintaining the interlocking of the deformation of the analog stick 180 and the movement of the eye. Since the robot 100 responds to the same tampering with various eye movements, the user can enjoy various facial expressions of the robot 100. It is also possible to estimate the emotions of the robot 100 according to the movement of the eyes.

The robot 100 changes the motion according to the operation history of the analog stick 180. As described above, the operation detection unit 182 sequentially detects the movement of the analog stick 180, that is, the deformation direction and the deformation amount. The operation history management unit 184 stores the detection result as an operation history in chronological order. Specifically, the operation history management unit 184 stores the detection result and the detection time in association with each other. The motion control unit 150 selects a motion according to the operation history, that is, how the analog stick 180 moves.

The state management unit 244 may change the emotional parameters depending on how the analog stick 180 is tampered with (operated). For example, if the user gently tampers with the analog stick 180, the eye generation unit 172 starts to move the eyes in conjunction with the tampering. If the same tampering is continued, the state management unit 244 raises the emotional parameter indicating a sense of security. When the emotional parameter indicating a sense of security becomes equal to or higher than a predetermined value, the eye generation unit 172 expresses the drowsiness of the robot 100 by lowering (closing the eyes) the eyelid image 190. At this time, the motion control unit 150 may express a "drowsiness" by selecting a motion for moving the head up and down. The designer can arbitrarily define how to play with the analog stick 180, which is easy to play with, in other words, how to play with the analog stick 180 that reassures the robot 100. For example, the analog stick 180 may be reciprocated to the left and right at substantially the same pace, or the nose sensor may be continuously tapped a predetermined number of times or more with a strength that does not push the analog stick 180 without moving the analog stick 180. It may be.

If the user keeps messing around with the analog stick 180, the robot 100 starts to move his eyes in conjunction with the messing around. If the robot 100 continues to play with the mess, the emotional parameters indicating irritation increase and the robot 100 stops the movement of the eyes. Further, the robot 100 expresses "no" by moving the head left and right to move the user's fingertip away from the analog stick 180. The "messy tampering method" may also be arbitrarily defined by the designer.

In one embodiment, the state of being gently tampered with is a state of monotonous movement. Expressed by the output value from the analog stick 180, a monotonous movement can be said to be a movement in which the output value has periodicity and the amount of deformation is small for a longer period than a predetermined period. In one embodiment, the messed up state is a state in which non-monotonous movements are made. When expressed by the output value from the analog stick 180, it can be said that the state in which the output value has no periodicity and the amount of deformation is large continues for a longer period than a predetermined period. In this way, not only does the eye continue to move in conjunction with the movement of the analog stick 180, but when a predetermined condition is satisfied, the robot 100 changes the way in which the analog stick 180 and the eye are linked. The robot 100 expresses a "sleepy" state by closing the eyelids or moving the body, rather than simply stopping the movement of the eyes.

For example, when the analog stick 180 is moved alternately left and right within a range of 30% or less of the maximum tilt angle (hereinafter, this movement method is referred to as "alternate left and right tilt"), the robot 100 stops the movement of the eyes and the eyelids. It may be said that the state of "becoming sleepy" is expressed by closing and moving the head up and down. The operation history management unit 184 refers to the operation history and determines whether or not the movement of the analog stick 180 corresponds to "alternate left-right tilt". When it corresponds to the alternating left-right tilt, the eye generation unit 172 stops the movement of the eyeball model 250 and displays the eyelid image 190. Further, the motion control unit 150 selects a motion for moving the head of the robot 100 up and down. As a result, the robot 100 closes its eyes and moves its head up and down to express a "sleepy" state.

It is when you feel at ease that an organism becomes sleepy. If the robot 100 is tampered with and the behavioral expression that the robot 100 wants to sleep is expressed, the user can feel that the robot 100 is relieved by tampering with the robot 100. The state management unit 244 increases the emotional parameter indicating a sense of security when the alternating left-right inclination is detected. When the emotional parameter indicating a sense of security exceeds a predetermined value, the robot 100 may close its eyes and move its head up and down.

When the user holds the robot 100 sideways, the robot 100 occupies most of the user's field of view. At this time, the user can easily notice small changes in the robot 100. The user can easily gaze at the nose 109 and the eyes 110 of the robot 100 held sideways. When this state is detected, for example, when the user detects the state in which the robot 100 is held sideways by the image captured by the camera of the robot 100, when the user tampers with the nose 109, it is linked with the movement of the analog stick 180. If the eyes change, the user can immerse himself in the robot 100 without being conscious of anything other than the robot 100.

Organisms respond insensitively to short-term stimuli, but can begin to move significantly when exposed to long-term stimuli. Even in the robot 100, the same behavior can be expressed by gradually increasing the movement as the contact from the user continues. As the user continues to mess with the nose 109, the robot 100, which initially moved only the eyes, also begins to move the head. The motion control unit 150 may drive a portion far from the nose 109 as the deformation duration becomes longer. For example, when the nose 109 is tampered with, the robot 100 moves the eye image 171, and when the nose 109 is continuously tampered with, the robot 100 slowly moves the head. If the nose 109 is further tampered with, the robot 100 may move its hand, and if the nose 109 is further tampered with, the robot 100 may move away from the user. The deformation duration may be defined as the duration of the tilt of the analog stick 180 or the duration of the touch of the nasal sensor.

As another example, when the user continues to tilt the left and right alternately for 10 seconds, the robot 100 moves the head up and down to express a state of "becoming sleepy and drowsy".

Subsequently, it is assumed that the user continues the left-right alternating inclination for 20 seconds or more. In this case, the robot 100 may further move the hand 106 toward the body 104 and store the front wheels 102 and the rear wheels 103 in the body 104. When the user continues to tilt left and right alternately in this way, the robot 100 first moves its eyes, then moves its head to "drows", and then brings both hands and feet closer to the body to "sleep". ..

When the user alternately tilts left and right, the robot 100 may change the motion depending on whether the analog stick 180 is moved fast or slowly. The operation history management unit 184 records the deformation amount and the deformation direction of the analog stick 180 as the operation history. The operation history management unit 184 calculates the deformation speed (change amount per unit time) from the operation history.

When the deformation duration is 10 seconds or more and the deformation speed for 10 seconds is equal to or less than a predetermined value, the motion control unit 150 selects a motion for moving the head up and down. On the other hand, when the deformation speed exceeds a predetermined value, the motion control unit 150 selects a motion for moving the head left and right. With such control, the robot 100 shakes its head sideways to express "no".

The robot 100 of the first embodiment changes the motion depending on how the user tampers with the nose 109. The tampering method here does not simply mean the deformation direction of the analog stick 180. For example, it includes fine movements such as whether the analog stick 180 is moved small or large, that is, monotonous, and whether it is moved fast or slowly, that is, whether it is roughly touched or stroked. If the robot 100 changes the motion according to the slight difference in how the user is tampered with, the user can come into contact with the robot 100 without getting tired of it.

The time required to reach the "sleepy" state may change depending on the internal parameters of the robot 100, such as the intimacy of the robot 100 with respect to the user and the state of emotional parameters at that time.

It is assumed that the user has tampered with the nose 109 when the robot 100 has his eyes closed and the state of not performing movement or motion is continued for a predetermined time or longer (when the robot 100 is sleeping). When the movement of the analog stick 180 is detected in this state, the robot 100 may change the eye image 171 with the eyes closed to the eye image 171 with the eyes open, and execute the movement or the motion. As described above, if the contact of the user is detected when the predetermined eye image 171 is and the motion is not executed, the robot 100 may change the eye image 171 and execute the motion.

The motion of the robot 100 may be changed according to the intimacy. For example, when a user whose intimacy is equal to or higher than a predetermined value tampers with the nose 109, the robot 100 brings the head closer to the user. On the other hand, when a user whose intimacy is less than a predetermined value similarly tampers with the nose 109, the robot 100 turns its head away from the user. As described above, the motion control unit 150 may select one of a plurality of motions according to the deformation mode of the analog stick 180 and the intimacy of the user who operates the analog stick 180. Specifically, when a predetermined operation input is made by a user having an intimacy of T1 or more, the motion control unit 150 randomly selects one of motions M1 to M3, and the same operation by a user having an intimacy of less than T1. When an input is made, the motion control unit 150 may randomly select any one of the motions M4 to M6.

In order to be attached to something, it is necessary for human beings to spend as much time as possible on themselves. The behavior of the robot 100 of the first embodiment changes depending on the intimacy with the user. Since the motion of the robot 100 changes depending on the intimacy even if the user makes the same tampering, the user can continue to touch the robot 100 without getting tired of it. By continuing to be in contact with the robot, the user can be attached to the robot 100. In addition, by continuing to interact gently, the intimacy of the robot 100 with respect to the user is increased. As the intimacy changes, so does the reaction of the robot 100. The robot 100 can realize various behavioral expressions according to the situation by the complex influence of the operation on the nose 109, the intimacy (feelings toward the user of the robot 100), and the emotional parameters (feelings of the robot 100).

When the user tampers with the nose 109, the robot 100 first moves the eyes, then the head, and then the body 104. Further, in the robot 100, the function of "moving the analog stick 180 of the nose 109 to move the eyes" (hereinafter, this function may be referred to as "pupil interlocking function") is enabled by the user using the robot 100. , It may be a condition that the face is held sideways toward the user side.

Even if the user moves the analog stick 180 when the robot 100 is not held sideways, the eyes of the robot 100 may only blink. The fact that the user is holding the robot 100 sideways is detected by a touch sensor (not shown) that detects contact with the surface of the robot 100 and an acceleration sensor (not shown) that detects the inclination of the robot 100. The recognition unit 156 detects the user's contact with the robot 100 via the touch sensor and the acceleration sensor. The posture determination unit (not shown) identifies the posture in which the user is in contact with the robot 100 from the detection result detected by the recognition unit 156. When the user identifies that the robot 100 is held sideways, the posture determination unit instructs the operation history management unit 184 to enable the pupil interlocking function. The operation history management unit 184, which receives an instruction from the posture determination unit, enables the pupil interlocking function and activates the movement of the eyes according to the movement of the analog stick 180 as described above. In this way, the behavioral characteristics of the robot 100 with respect to the operation of the nose 109 may be changed according to the posture of the robot 100 such as hugging.

The posture determination unit has a pupil interlocking function on condition that the user and the robot 100 are in a predetermined positional relationship and the user's posture is detected based on the touch sensor, the acceleration sensor, or the captured image. May be enabled. For example, the pupil interlocking function is effective when the user is holding the robot 100 sideways with the face of the robot 100 facing itself. Even if the robot 100 reacts when the user is not concentrating on the robot 100, there is a high possibility that the user will miss the reaction. By linking the movement of the eyes with the analog stick 180 when the robot 100 is in this posture, it is possible to concentrate the user's consciousness on the robot 100 and notice small changes in the robot 100. Instead of detecting that the user holds the robot 100 sideways, the user's consciousness is concentrated on the robot 100, for example, sitting face to face on the user's lap, and the robot 100 is made aware of small changes. The detection that the user is in the desired posture may be a condition for enabling the pupil interlocking function. Further, not limited to the touch sensor and the acceleration sensor, it may be a condition for enabling the pupil interlocking function to detect whether the user and the robot 100 are facing each other by using the face recognition technology of the robot 100. When the robot 100 defines the state in which the wheels are completely housed in the body 104 as the "holding mode", and the user and the robot 100 are identified as facing each other by face recognition technology or the like in the "holding mode" state. , The pupil interlocking function may be enabled.

[Second Embodiment]
The robot 100 of the second embodiment changes the movement of the eyes (eye movement) according to the locus (path pattern) of the nose 109 when the nose 109 is moved. The robot 100 of the second embodiment has the same appearance as the robot 100 of the first embodiment.

FIG. 12 is a functional block diagram of the robot system 300 according to the second embodiment.
The data storage unit 148 of the robot 100 in the second embodiment further includes an eye movement storage unit 174. The eye movement storage unit 174 stores the movement of the analog stick 180 (deformation pattern) and the movement of the robot's eyes (eye movement) in association with each other. The motion storage unit 160 of the robot 100 in the second embodiment also stores a motion corresponding to the duration (deformation duration) of the deformed state of the analog stick 180 (the state in which the analog stick 180 is moved from the initial position). ing.

The operation history management unit 184 of the robot 100 in the second embodiment stores the detection result of the operation detection unit 182 as an operation history in chronological order. The operation history management unit 184 collates the stored operation history with the execution conditions of the deformation pattern (details will be described later) stored in the eye movement storage unit 174, and when the operation history matches any of the execution conditions, The eye generation unit 172 is instructed to perform an eye movement associated with the deformation pattern. The eye generation unit 172 receives an instruction from the operation history management unit 184 and executes the eye movement. The “eye movement” means a change in the eye image 171 due to the rotation of the eyeball model 250, as in the first embodiment.

The robot 100 in the second embodiment also changes the eye image 171 according to the operation history of the analog stick 180 provided on the nose 109. Further, the motion is selected according to the deformation duration to control the drive mechanism 120 of the robot 100.

FIG. 13 is a flowchart showing eye movements and motions of the robot 100 based on the operation history.
As described above, the operation history management unit 184 stores the detection result of the operation detection unit 182 as an operation history in chronological order (S10). The operation history management unit 184 determines whether or not the stored operation history matches any of the deformation patterns stored in the eye movement storage unit 174 (S12). When there is a matching deformation pattern (Y in S12), the operation history management unit 184 sequentially determines how long the deformation state continues based on the operation history (S14). If there is no matching deformation pattern (N in S12), the subsequent processing is skipped. When the deformation duration is less than 5 seconds (N in S14), the operation history management unit 184 instructs the eye generation unit 172 to execute the eye movement corresponding to the deformation pattern (S16). On the other hand, when the deformation duration is 5 seconds or more (Y in S14), the operation history management unit 184 instructs the eye generation unit 172 to execute the eye closing operation (S18), and the motion control unit 150 is instructed to execute the motion storage unit. The motion is inquired from 160 and an instruction is given to the drive mechanism 120 to execute the motion (S20).

Specifically, the operation history management unit 184 of the robot 100 selects an eye movement associated with the deformation pattern in advance, and instructs the eye generation unit 172 to execute the eye movement. For example, it is assumed that the eye movement of turning the eye is associated with the deformation pattern defined by the predetermined execution condition of rotating the analog stick 180 clockwise once. In this case, when the user makes one round of the analog stick 180 clockwise, the robot 100 turns its eyes round and round. The motion control unit 150 also selects one or more motions from one or more motions that are associated with the deformation duration in advance. When the condition that the deformation duration is 5 seconds or more is satisfied, it is assumed that the motion of moving the head of the robot 100 to the left or right is associated with the deformation duration. In this case, when the user keeps moving the analog stick 180 for, for example, 6 seconds, the robot 100 swings its head left and right.

FIG. 14A is a data structure diagram of the eye movement table 420. FIG. 14B is a data structure diagram of the deformation pattern selection table 440 of the analog stick 180.
The eye movement table 420 defines the correspondence between the deformation pattern of the analog stick 180 and the eye movement. The deformation pattern selection table 440 shows what kind of parameter each deformation pattern is defined by. The eye movement table 420 and the deformation pattern selection table 440 are stored in the eye movement storage unit 174.

In the operation history, the deformation direction of the analog stick 180 and the time corresponding to the deformation amount are recorded in chronological order. By referring to the operation history, the path (trajectory) followed by the analog stick 180 can be specified. For example, the operation of the analog stick 180 that follows the path R1 that is the locus when making one round clockwise and the movement that follows the path R1 is within 1 second is defined as the deformation pattern A1. In the second embodiment, the "modification mode" of the present embodiment includes a route. Similarly, the operation of the analog stick 180 in which the analog stick 180 is moved in the order of left and right from the initial position, the path R2 which is the trajectory when returning to the initial position is followed, and the movement following the path R2 is within 2 seconds is performed. It is defined as the deformation pattern A2. The operation of the analog stick 180 that follows the path R3, which is the locus when pushing from the initial position, and the movement of following the path R3 is performed within 0.5 seconds is defined as the deformation pattern A3.

Each deformation pattern is associated with eye movement. For example, in the case of the deformation pattern A1, the eye generation unit 172 executes an motion of turning the eye (eye motion M1). In the case of the deformation pattern A2, the eye generation unit 172 executes a blinking motion (eye motion M2). In the case of the deformation pattern A3, the eye generation unit 172 executes an operation of bringing both eyes together (eye movement M3).

When the user moves the analog stick 180, the operation detection unit 182 detects the movement of the analog stick 180 via the internal sensor 128. The movement method is transmitted from the operation detection unit 182 to the operation history management unit 184, and the operation history management unit 184 transfers the movement method and the deformation pattern stored in the eye movement storage unit 174 when the deformation duration is less than 5 seconds. (The case where the deformation duration is 5 seconds or more will be described later). If the operation is performed within 5 seconds and the movement of the analog stick 180 matches the execution condition of any of the deformation patterns, the operation history management unit 184 identifies the eye movement associated with the deformation pattern. The operation history management unit 184 instructs the eye generation unit 172 to execute the specified eye movement. The eye generator 172 performs the specified eye movement.

Specifically, for example, it is assumed that the user moves the analog stick 180 so as to follow the path R1 and completes the movement in 0.5 seconds. Since this movement matches the execution condition of the deformation pattern A1, the operation history management unit 184 identifies the eye movement M1 associated with the deformation pattern A1, and the eye generation unit 172 refers to the eye movement M1, that is, the movement of spinning the eye. I do.

In the robot 100 of the second embodiment, the behavior of the robot 100 changes according to not only the eye movement but also the deformation duration of the analog stick 180.

FIG. 15 is a data structure diagram of the motion selection table 460.
The motion selection table 460 defines the correspondence between the deformation duration and the motion. The motion selection table 460 is stored in the motion storage unit 160.

As described above, when the deformation duration is 5 seconds or more, the operation history management unit 184 instructs the execution of the eye closing operation (eye closing operation). The eye generation unit 172 executes the eye closing operation based on the instruction from the operation history management unit 184.
The motion control unit 150 identifies the motion C1 which is the corresponding motion when the deformation duration is 5 seconds or more from the motion storage unit 160. Motion C1 is a movement that moves the head of the robot 100 left and right. The robot 100 exhibits a behavior of "turning away" by closing its eyes and shaking its head from side to side.

When the deformation duration is 15 seconds or more, the motion control unit 150 selects motion C2. Motion C2 is a movement of raising and lowering the hand of the robot 100. The motion control unit 150 activates motion C1 and motion C2. Then, the robot 100 makes motions C1 and C2, and expresses a "no-no" movement in which the eyes are closed and the hands are fluttered while shaking the head.

The motion control unit 150 selects a motion for driving a portion from the nose 109 to the head of the robot 100 within a predetermined range while the deformation duration is short. When the deformation duration becomes long, the motion for driving the outside of the head of the robot 100 is also selected.

The robot 100 in the second embodiment moves its eyes in response to the movement of the analog stick 180 provided on the nose 109. As the analog stick 180 continues to move, the robot 100 begins to move its head. Even if the head starts to move, if the analog stick 180 continues to move, the robot 100 starts to move the body. In this way, as the user keeps messing with the nose 109, the robot 100 moves the eyes and then the head and body, so that the user gets tired of being touched by the robot 100, begins to feel tickled, and so on. You can feel that you are behaving and you can feel that the robot 100 is in contact with living things.

In the second embodiment, the analog stick 180 is installed on the nose 109. When communicating, humans often look at the other person's face. The same applies when touching the robot, and it is considered that the user touches the robot 100 while looking at the face area 107 of the robot 100. At this time, if there is something that the user wants to unknowingly touch, such as a face or nose, the user is likely to touch it unintentionally. In the robot 100 of the second embodiment, the nose 109 corresponds to it. When the analog stick 180 of the nose 109 that makes you want to mess with it unconsciously is moved, the robot 100 behaves according to the movement. Become.

FIG. 16 is a front view of the robot 100 in a steady state. FIG. 16 is a diagram showing a state when the robot 100 is standing in a steady state when the user is not touching the robot 100.
FIG. 17 is a front view of the robot 100 when the motion C1 is activated. FIG. 17 is a diagram showing a state when the robot 100 activates the motion C1 because the user continuously touches the analog stick 180 of the robot 100.
When the user is not moving the analog stick 180 in a steady state, the eyes 110 of the robot 100 open their eyes and stare at the front. When the user keeps moving the analog stick 180 and the deformation duration exceeds 5 seconds, the robot 100 closes his eyes 110 and turns his head away.

The robot 100 has been described above based on the embodiment.
In the robot 100 of this embodiment, an analog stick 180 is provided on the nose 109. Since the robot 100 reacts to the contact by the user of the nose 109, which is easy to tamper with unconsciously, the user feels as if the robot 100 communicates with himself / herself, and it becomes easy for the user to have an attachment to the robot 100.

The robot 100 of the present embodiment changes its eyes according to the movement of the analog stick 180. Also, if the analog stick 180 is tampered with for a long time, the robot 100 starts to move its head and body. When the user slightly tampers with the analog stick 180, the robot 100 moves his eyes to pretend to be interested in the user. If you continue to mess with it, the robot 100 will move its head and pretend to avoid the user's hand, expressing the feeling of "stop". If it is too good, the robot 100 will start flapping its hands and start expressing "no". The robot 100 of the present embodiment changes facial expressions and actions not only by the movement of the analog stick 180 but also by its duration. The user feels that the robot 100 has emotions (whispering) such as "tired", "tickling", "annoying", and "doing more", and can also feel the biological behavior of the robot 100.

When the user tampers with the analog stick 180 provided on the nose 109, the robot 100 of the present embodiment changes its eyes. Further, if the analog stick 180 is tampered with for a long time, the robot 100 starts to move its head, and if it continues to be tampered with, it starts to move its body. When the nose 109 is tampered with, the behavior of the robot 100 is first shown in the portion located in the face region 107 near the nose 109, so that the user does not overlook the reaction of the robot 100. If the tampering is continued, the inside of the head of the robot 100 that is within the predetermined range starts to move as well as the inside of the face area 107, and if the tampering is continued, the behavior is shown outside the head of the robot 100 (outside the predetermined range). The user can enjoy watching the facial expressions and actions of the robot 100 that change depending on how the nose 109 is tampered with and the time it is tampered with.

The robot 100 of the present embodiment changes the eyes according to the movement of the analog stick 180 provided on the nose 109. As a modification, the user may be imaged by the imaging device, the movement of the user may be detected from the captured image by the motion sensor, and then the eyes may be changed according to the movement of the user. However, in the case of detecting the movement of the analog stick 180, the time from the user's contact with the robot 100 to the behavior of the robot 100 is shorter than in the case of detecting the movement of the user with image processing such as a motion sensor. can. In addition, the user's contact with the robot 100 can be reliably detected.

In the present embodiment, the analog stick 180 (operation device) is directly provided on the robot 100 main body. In the modified example, an operation device such as a remote controller may be provided separately from the robot 100 main body. However, when the operation device is directly provided on the robot 100 main body, the user directly touches the robot 100. Therefore, it is easier for the user to intuitively operate and control the operation device as compared with the case where the operation device is provided separately from the robot 100 main body.

The present invention is not limited to the above-described embodiment or modification, and the components can be modified and embodied within a range that does not deviate from the gist. Various inventions may be formed by appropriately combining a plurality of components disclosed in the above embodiments and modifications. In addition, some components may be deleted from all the components shown in the above embodiments and modifications.
Hereinafter, other embodiments will be described.

In the embodiment, the operation detection unit 182 detects the movement (deformation) of the analog stick 180, and determines the behavior of the robot 100 according to the detection result. What is detected by the operation detection unit 182 is not limited to this, and may be any deformation of the robot 100 main body or movement of the device provided in the robot 100 main body. The detected deformation or movement may be, for example, a physical movement such as a movement of shaking the neck of the robot 100, a movement of holding a hand, a movement of shaking the body, or a movement of the cheek caused by the user pinching. Further, the means for detecting these movements may be any means for detecting physical movements such as a joystick or a trackball. An analog stick 180 or the like may be arranged inside the flexible exodermis so that these detecting means are not exposed on the surface of the robot 100.

In the embodiment, it is assumed that the nose 109 is where the contact is detected by the operation detection unit 182. The place where contact is detected is not limited to the nose 109. For example, any place where contact can be detected, such as a rounded part such as a paws in an animal, a protruding part like a tail, or a part where deformation is expected such as a cheek, may be used. Preferably, it is a place where the user unknowingly wants to mess with.

In the embodiment, it is assumed that the eyes move when the nose 109 is tampered with. The operating portion of the robot 100 is not limited to the eyes, and may be any movable portion such as the neck, tail, and hands. Further, the portion where the contact is detected by the operation detection unit 182 and the operation portion may be near or far, and for example, the nose may move when the tail is tampered with.

In the embodiment, the head of the robot 100 is set as a predetermined range, and if the time for the nose 109 to be tampered with is 5 seconds or more and less than 15 seconds, only the head of the robot 100 is used, and if it is 15 seconds or more, the head of the robot 100 is used. Not only that, the hand 106, which is a part outside the predetermined range, was also driven. The “predetermined range” that determines the location to be driven according to the deformation of the robot 100 body or the movement of the device provided on the robot 100 body is not limited to the head of the robot 100. For example, it may be the face area 107 of the robot 100, or it may be a part other than the legs of the robot 100. The range may be appropriately set among the parts that can be moved by the drive mechanism of the robot 100.

Further, the condition of the deformation duration for activating each motion is not limited to 5 seconds or more and 15 seconds or more, and can be set to various times such as 1 second or less and 1 minute or more.

In the embodiment, the motion is changed according to the time during which the nose 109 is continuously tampered with (deformation duration). Not limited to the deformation duration, for example, the deformation mode of the robot 100 main body such as the path and the deformation speed, the movement of the device provided in the robot 100, and the like may be adopted as conditions for selecting the motion.

The eyes of the robot 100 or the movement may be changed according to the cumulative value of the amount of deformation, the cumulative value of the duration of deformation, or the pressure applied to the robot 100. For example, when the nose 109 is tampered with with a weak force, the transition to the movement of the eyes, the movement of the head, and the movement of the whole body is gradual, so that only the change of the eyes may be performed for a long time. When tampering with a strong force, it may transition to the movement of the whole body in a short time. As a result, the behavior of the robot 100 according to the tampering method becomes richer in variation.
Unlike the touching method of simply stroking and hitting, the robot 100 of the present invention responds to the touching method accompanied by "deformation". In other words, contact is detected not only as electrical contact detection but also as physical "deformation". There are various "tampering methods" such as the strength to be tampered with, the amount of deformation, and the contact area depending on the user. If the robot 100 reacts expressively according to the "playing method" that differs depending on the user, the user becomes attached to the robot 100.

In the embodiment, the behavior of the robot 100 is determined according to the movement of the analog stick 180. The trigger for determining the behavior of the robot 100 is not limited to this, and for example, a sensor may be provided at a place where the user is expected to unknowingly want to tamper with it. Further, a plurality of sensors may be provided on the robot 100 main body. When a plurality of sensors are provided, one of them may be a sensor for detecting the movement of the analog stick 180. Further, all the sensors may have the same detection accuracy, or the sensors may have different detection accuracy. When a plurality of sensors having different detection accuracys are provided, sensors having higher detection accuracys may be provided at places where it is expected that the user will want to tamper with them. The detection accuracy in this case is the detection sensitivity with respect to the amount of change when the user changes the contact point with respect to the robot 100 main body. Hereinafter, the detection of the user's contact by any of the sensors provided in the robot 100 main body may be referred to as "the user touches the robot 100" or "the user touches the robot 100".

An example of a sensor provided other than the nose 109 is a sensor that detects a user's contact with two positions on the surface of the robot 100 that sandwich the eye 110. When the user's contact is detected by this sensor, the eye generation unit 172 lowers (closes the eye) the eyelid image 190. As a result, the behavior of the robot 100 of closing the eyes because the eyes 110 are covered can be expressed.

When a plurality of sensors are provided in the robot 100 main body, when a user's contact is detected by any of the sensors, the eye generation unit 172 generates an eye image 171 in which the virtual line of sight of the robot 100 is directed to the contact point. May be. Hereinafter, displaying the eye image 171 with the virtual line of sight directed to a predetermined location on the monitor 170 for a predetermined time may be referred to as "the robot 100 gazes at the predetermined location". For example, when the user touches the hand of the robot 100, the robot 100 gazes at the hand. Alternatively, when the user touches the stomach of the robot 100, the robot 100 gazes at the stomach. Further, when the user's contact exceeds a predetermined time, the robot 100 may move the portion including the contact portion. For example, when the user touches the robot 100's hand for more than 5 seconds, the robot 100 makes a motion to release the robot 100's hand from the user's hand. When the user touches the abdomen of the robot 100 for more than 10 seconds, the robot 100 makes a motion of shaking the abdomen from side to side. As described above, when the contact of the user is detected by any of the plurality of sensors provided at the plurality of points of the robot 100, the eye generation unit 172 generates an eye image 171 that is gazing at the contact points. Further, when the detection time of the user's contact in any of the sensors exceeds a predetermined time, the motion control unit 150 executes a motion to move the portion including the contact portion. As a result, the user can feel that the robot 100 exhibits a biological behavior of twisting the back body by staring at the touched portion.

If a high-precision sensor is attached to the entire area of the robot 100 body, power consumption will increase. Power consumption can be reduced by making only some sensors highly accurate. Further, it is assumed that only the sensor of the part where the user is expected to be tampered with is made highly accurate to detect a small movement of the user and the behavior of the robot 100 corresponding to the movement is shown. By doing this, it is possible to express the characteristic of living things that "the reaction is weak even if you touch other parts of the body, but you are sensitive to the contact at a certain point". Further, even if the user touches a little, the robot 100 detects the contact and returns a reaction, so that the user feels that the robot 100 reacts to even a little movement of himself / herself and can communicate with the robot 100. You can enjoy the feeling of being there. The detection accuracy is not limited to the detection sensitivity with respect to the amount of change when the user changes the contact point with respect to the robot 100 main body, and may be, for example, the ability to detect the strength of contact, the contact range, and the like. Further, a physically moving sensor may be used as a high-precision sensor, a capacitive touch sensor may be used as a low-precision sensor, or the same type of sensor but different detection accuracy may be used. Various types of sensors can be adopted, and in addition to the above-mentioned sensors, for example, when the sensor detects a user's contact, force feedback may be provided.

In the embodiment, as the eye movements of the robot 100, the eye movement M1 which is the movement of turning the eyes, the eye movement M2 which is the movement of blinking, and the eye movement M3 which is the movement of bringing both eyes together are shown. The eye movements shown by the robot 100 are not limited to these. For example, any movement that can be realized by the robot 100, such as a movement of turning the line of sight, a movement of opening the eyes, a movement of squinting, etc., may be used.

In the embodiment, as the motion of the robot 100, a motion C1 which is a motion of moving the head left and right and a motion C2 which is a motion of moving the hand up and down are shown. The motion shown by the robot 100 is not limited to these. For example, move your head vertically to "nod", tilt your head to "bend your neck", retract your hands behind your body, tilt your body forward to "bow", move your legs to "bow" Any movement that can be realized by the robot 100, such as "backward", stop the movement, and "ignore", is sufficient.

In the embodiment, the nose 109 is smaller than the eyes 110 and the face area 107. The size of the nose 109 is not limited to this, and may be about the same as or larger than the eyes 110. Further, the size relationship of each part such as the eyes 110 and the face area 107 is not restricted by the embodiment, and the design can be changed as appropriate.

In the present invention, when the user tampers with the nose 109 of the robot 100, the robot 100 returns a reaction according to the tampering method. Provide a sensor that can detect changes in places that are likely to be tampered with by the user. When the user tampers with the portion, the eyes of the robot 100 move, the facial expression changes, and the body begins to move, and the user becomes attached to the robot 100. If the reaction of the robot 100 changes depending on how it is tampered with, the time for the user to touch the robot 100 becomes longer. If the user's contact time with the robot 100 can be lengthened, the user's attachment to the robot 100 can be further deepened. There are no animals that move their eyes when their noses are tampered with. Even humans do not do that. Even if it were, it would be difficult for someone else to tamper with the nose and move the eyes in conjunction with the direction of the tampering. It can be said that this is an expression that is possible only with Robot 100.

Claims (15)

An operation detector that detects the deformation of the robot body and
It is provided with an eye generation unit that generates an eye image and displays the eye image on the face area of the robot.
The eye generation unit is a robot characterized in that when a deformation of the robot body is detected, the eye image is changed according to the deformation mode. The robot according to claim 1, wherein the eye generation unit changes the eye image based on one or more of a deformation direction, a deformation amount, and a deformation speed as the deformation mode. Equipped with a motion control unit that selects the motion of the robot
1. The motion control unit is characterized in that, after the change of the eye image is started, when the deformation mode of the robot body satisfies a predetermined condition, the motion control unit selects a motion according to the deformation mode. Or the robot according to 2. The robot according to claim 3, wherein the motion selected when the predetermined condition is satisfied is a motion for driving the head of the robot. Equipped with an operation history management unit that records the deformation state of the robot body with the passage of time as an operation history.
The robot according to any one of claims 1 to 4, wherein the eye generation unit changes the eye image according to the operation history. A motion control unit that selects the motion of the robot,
It is equipped with an operation history management unit that records the deformation state of the robot body with the passage of time as an operation history.
The robot according to any one of claims 1 to 5, wherein the motion control unit changes a motion according to the operation history. The robot according to claim 5 or 6, wherein the operation history includes a duration of a deformed state. The robot according to any one of claims 1 to 7, wherein the operation detection unit detects deformation of a convex portion provided in a face region of the robot body. The convex portion is a protrusion installed on a portion corresponding to the nose of the robot body.
The robot according to claim 8, wherein the operation detection unit detects deformation of the protrusion. It is equipped with a posture determination unit that identifies the posture of the robot.
The invention according to any one of claims 1 to 9, wherein the eye generation unit changes the eye image according to the deformation mode, provided that the posture determination unit identifies the robot being lifted by the user. Robot. A motion control unit that selects the motion of the robot,
A drive mechanism that executes the selected motion, and
It is equipped with an operation detection unit that detects the movement of the device provided in the robot body.
The robot is characterized in that, when the movement of the device is detected, the motion control unit selects a motion corresponding to the movement of the device from among the motions to be driven within a predetermined range from the device. When the movement of the device is detected, the motion control unit drives a part of the robot with the predetermined range as a driving target, and the movement of the device satisfies a predetermined condition after the start of driving the part. The robot according to claim 11, further comprising driving the robot outside the predetermined range. A first sensor installed on the body surface of the robot to detect user contact,
A second sensor provided in the face area of the robot and detecting a user's contact with higher accuracy than the first sensor.
It is provided with an eye generation unit that generates an eye image and displays the eye image in the face region.
The eye generation unit is a robot characterized in that the eye image is changed in association with the amount of change in the detected value of the second sensor. The robot according to claim 13, wherein the second sensor is provided in a narrower area than the first sensor, and detects the amount of deformation of the body surface of the robot that occurs upon contact with the user. A convex part that imitates the nose and
Equipped with an eye generator that expresses the line of sight by moving the eyes,
The eye generation unit is a robot characterized in that the pupil is changed in conjunction with the deformation state of the convex portion.

Images