JP2007156577A - Method of gaining color information using life support robot - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To cause a robot to smoothly provide life support services by easily, surely and rapidly specifying the articles of a user to be served. <P>SOLUTION: In S40, a user 41 asks a robot 1 about a schedule first in the morning. In S41, the robot performs face recognition through pattern matching. In S42, whether the user is standing, sitting or lying is estimated. If this estimation is uncertain, the robot 1 moves to e.g., the front of the user in S43. If necessary, the robot moves again in S44. In S45, the user's clothing area matching his attitude is determined and the color of his clothing is specified. In S46, the user 41 is recognized by the color of the clothing, and in S47 the tracking of the user 41 is started. On the next day, the series of processes are performed again to deal with the change of the clothing. <P>COPYRIGHT: (C)2007,JPO&INPIT

Description

  The present invention relates to a movable robot, by recognizing each user of a user such as a family member, responding to a call from each person, or voluntarily from the robot side to each person, actions and various services The present invention relates to a robot that can serve, for example, move freely, talk to a family's face, talk, and access the Internet to report a weather forecast.

  Conventional robots are industrial robots used in production sites such as articulated arm robots and transfer robots. However, the purpose of industrial robots is limited to welding, painting, transportation, etc., and robots that are fixed in one place, such as arm robots, and routes that are determined like transport robots are used. Most of them moved in a predetermined order. However, recently, this autonomous mobile robot has appeared and moved freely within a predetermined space, such as simple work, dangerous work, difficult work in industrial activities and production activities, such as nuclear power plants, thermal power plants It has become possible to provide assistance and substitution for maintenance work, etc., cleaning work in high-rise buildings, and rescue operations at disaster sites such as fire sites. The scope of use of autonomous mobile robots is not limited to this. For example, it is installed in the homes of elderly people living alone, cleaning the room, managing the physical condition of the residents, or checking indoor safety when the residents leave the house. There are also robots that support daily life, such as life support type.

  One of the basic actions of a life-supporting robot is to create a response sentence by referring to information related to an object viewed by the robot and speak.

  For example, in the “information processing apparatus and method, and recording medium” disclosed in Patent Document 1, the utterance of the user and the utterance of the response sentence of the robot are identified through identifying an object such as an article viewed by the robot. Done. Specifically, it is assumed that when the robot is watching “tea bowl”, the user speaks “How much hot water does the tea bowl enter?”. Then, on the robot side, an attribute in the speech recognition of “tea bowl” is detected, and the same or similar word as the attribute detected from the dictionary information is referred to. The user's question is language-recognized, and a response to it is generated. For example, if there is a color, a manufacturer, and a capacity as knowledge information for “tea bowl”, a response sentence is configured by inputting the capacity as a response to “how much”. In this way, by associating the article information identified by the robot with the linguistic analysis of the user's utterance, a response sentence is generated accurately and quickly, and the interaction between the user and the robot is performed accurately and rapidly.

Further, in the “object recognition apparatus and method and robot apparatus” disclosed in Patent Document 2, the robot itself does not require prior knowledge about the range and size of a three-dimensional object to be learned in a general home or the like. Select and learn 3D objects. Specifically, the robot places a three-dimensional article having a size and distance that can be lifted on the palm, and moves the article to a predetermined spatial position. Next, image extraction processing is performed by rotating the palm 360 degrees, for example, 45 degrees. The robot extracts the characteristics of the three-dimensional object based on the cut-out image and stores it.
As described above, the life support robot recognizes a person or an object from the camera information of the robot, and performs services such as carrying away objects, assisting with memory, and smart conversation.
JP 2001-188780 (paragraph 0063, FIG. 1) Japanese Patent Laying-Open No. 2003-346152 (paragraph 0050, FIG. 1)

  However, although recognition and classification of humans and objects by robots are limited to living spaces, there are an unspecified number of objects, and it is difficult to specify that leads to a service by a simple method. For this reason, recognition / classification using a form that is more complicated information is performed. However, in general, the system becomes complicated and is greatly influenced by observation conditions such as illumination conditions, resulting in difficulty in recognition and classification. In addition, recognition and classification may be difficult depending on the posture of the person, the place where the article is placed, and the angle.

  In addition, if recognition and classification become complicated and sophisticated, calculation processing takes time, and the invention disclosed in Patent Literature 1 and Patent Literature 2 may make it difficult to respond quickly. In addition, when trying to recognize an article itself or a face with its own information, it often falls into a misjudgment and fails in tracking.

  Accordingly, an object of the present invention is to provide a life support service smoothly to a user by specifying a user and an article to be serviced simply, surely, and quickly.

  Originally, the life support robot returns to the starting point that it should be recognized only with attention to a person, and provides a life support service using only the color of clothes of the user and the color of articles related to the user as clues. First, user recognition is performed using a face image and clothes color. Secondly, the user's belongings are recognized with reference to the belongings database (DB) based on the color. Third, the method or position pointed to by the user is detected, the color of the article in that direction / position is identified, the article placement DB is referenced, and the article pointed to by the user is recognized. Fourthly, the result of detecting the relationship between the user's motion and the motion of the article is accumulated as the motion DB, and the meaning is recognized when the user performs a certain motion.

  Based on the result of the user recognition / article recognition described above, the robot executes a life support service for the user based on the life support program.

  Specifically, a first means for solving the above-described problem is a face image database DB for registering a face image of the user in a robot that moves to provide life support services to one or more users. A face image capturing unit that captures the user's face image while facing the user, a user identifying unit that identifies the user facing the robot with reference to the face image DB, and the specific user A clothing color identification unit that identifies a color included in a predetermined part of the body as a clothing color, and provides the life support service to the specific user having the clothing color.

  The second means is to specify the clothing color in a predetermined time zone in which the clothing color in the day can be distinguished in the first means.

  The third means, in the first means, recognizes the possession database DB for registering the color of the possession for each user and the article possessed by the specific user when facing the robot, and identifies the color. By referring to the article color specifying unit and the property DB, a person who owns the article having the specified article color is specified.

  The fourth means further comprises a possession history database DB for updating the possession history of the user for each article in the third means, refers to the possession history DB, and is the user possessing the article, Identifying the user facing the robot.

  The fifth means is that in the third difference means, the property DB registers the size and / or shape of the property.

  The sixth means is to identify the article in the first means by detecting a direction and / or position of the article indicated by the specific user and referring to the property DB.

  The seventh means is the first means further comprising an action database DB for storing the action of the article and the action of the specific user, and specifies the article based on the action of the article and / or the action of the specific user. That is.

The eighth means is a method for providing life support by a mobile robot to one or more users, a user recognition step of identifying the user by face and clothing color, and human-accompanying information of the user facing the robot In other words, the feature information creating step for storing the color of clothes worn by the user, the color of an article held, the color of an article touched, or the color of an installed article, and the voice instruction waiting for the voice instruction of the user And when the voice instruction is received, referring to the feature information, the product as a target of the life support service is specified, and at the same time, the voice support is recognized to recognize the voice support for the specific product. A service identification step that determines the content of the service;
Providing a life support service; and an operation history storage step of storing the life support service as an operation history of the robot.

  A ninth means is the eighth means, wherein the feature information creating step further stores the behavior information of the user, that is, whether the user is standing, sitting or sleeping.

  By the first means, the user to be tracked by the robot can be easily and reliably recognized.

  By the second means, it is possible to provide a life support service all day long after waking up.

  By the third means, the specified article can be easily cleaned up.

  By the fourth means, the user who currently possesses the article recognized by the robot can be quickly identified.

  By the fifth means, it is possible to quickly determine the property based not only on the color but also on the size and shape.

  According to the sixth means, not only the product can be recognized by the color and shape, but also the product can be recognized based on the direction and position indicated by the user, and the product recognition can be performed by various methods.

  According to the seventh means, the movement of the article can be recognized in relation to the user's action, or conversely, the user's action can be recognized in relation to the action of the article. I can grasp it. Then, by enriching the motion database that is the result of detecting the correlation between the motion of the user and the article, the user can smoothly receive a life support service from the robot.

  By the eighth means, it is possible to specify the service target article based on the voice instruction recognition while referring to the feature information, and it is possible to specify the service.

  According to the ninth means, the color of the user's clothes is created as feature information depending on whether the user is standing, sitting, or sleeping, so the robot also stores the action information. Therefore, there is an advantage that the user can be specifically specified by the color of clothes and the behavior information.

  ADVANTAGE OF THE INVENTION According to this invention, a user can be specified easily, reliably, and quickly with a face and clothes color. Further, it is possible to quickly specify an article that is incidental to the specified user (owned and installed by the user). Since the user and the service target item can be identified quickly and reliably, the service contents can be identified quickly and reliably by voice instructions. Thereby, communication (life support) between a robot and a human can be performed smoothly.

  Hereinafter, the present invention will be described in detail with reference to the embodiments shown in the drawings. However, the dimensions, materials, shapes, relative arrangements, and the like of the component parts described in this example are not intended to limit the scope of the present invention only to specific examples unless otherwise specified. Only.

  FIG. 1 is a front view of a life support robot according to an embodiment of the present invention.

  In the head 2 of the robot, a top camera 3, a pair of left and right front cameras 5, and a pair of microphones 4 are arranged at the position of a human eyebrow at the lower part of the forehead. A pair of left and right speakers 7 are arranged on the chest 6 away from the vertical center line. The elbows and shoulders of both arms 9 have a joint structure, and a hand 10 holds an object. The neck also has a joint structure. In this robot, the lower limb is a skirt 12-shaped cover that houses an electric circuit and the like, and can be moved by the two left and right wheels 14 at the lowermost part of the lower limb. In addition, the pair of left and right obstacle sensors (infrared detection or ultrasonic detection) 13 automatically avoids obstacles and moves autonomously to the destination on the upper leg.

  FIG. 2 is a block diagram of the robot according to the present embodiment.

  Input to the CPU 21 is output from the top camera 3, front camera 5, microphone 4, obstacle sensor 13, and data input / output 20. The CPU 21 includes an accident position recognizing unit 22, a pointing point recognizing unit 23, a voice recognizing unit 24, a neck / arm / manual recognizing unit 25, a body direction recognizing unit 26, and a past motion / position database. The neck joint drive unit 28, the speaker 7, the wrist joint drive unit 29, the chest joint drive unit 30, and the vehicle drive unit 31 are driven. The communication ability of this robot by computer control is the ability to detect a person who moves or detect a face; the ability to identify a person who detects a facial feature and identifies a predetermined number of users including one owner; Speech recognition ability such as word recognition, continuous word recognition necessary for conversation scenarios, and human voice recognition by acoustic echo canceller; text-to-speech method, speech synthesis ability that can change settings such as voice volume, etc. . In addition, the moving ability is given the ability to search for and approach a person in order to support life. As for communication capability, it has a homepage dedicated to robots, and it is possible to register family names, change behavior patterns, perform remote operations, and view images. For example, it is possible to have a person come to meet you at a specified time by remote control, or to visit a home away from home and view the video. As for the video, not only the current video but also the past video can be browsed using the memory of the computer 21. The images include a color image obtained by the front camera 5 and a monochrome image obtained by panoramicizing the omnidirectional camera obtained by the top camera 3. Also, the robot itself can search the homepage on the Internet and deliver information to family members.

  FIG. 3 is a block diagram of the robot of the present embodiment centering on user recognition and article recognition.

[User recognition]
First, user recognition will be described with reference to FIG. User recognition is performed by face recognition and clothes color. Therefore, a face image DB 220 that registers one or more user face images, a face image photographing unit 21 that faces the user and photographs the user's face image, and a user who faces the robot with reference to the face image DB 220 The user specifying unit 270 for specifying the color and the clothing color specifying unit 260 for specifying the color included in the predetermined part of the specified user's body as the clothing color are used.

  The face image DB 220 stores the face image of the robot user. The user is a family member or the like.

  The face image photographing unit 210 is the front camera 5 of the robot shown in FIG. 1 and photographs a user's face (including a body other than the face) facing the robot, and the photographed image is stored in a memory (not shown). Note that even if the face image is obliquely, laterally, or backwards, it may be useful if it is simply tracked.

  The user specifying unit 270 compares the face of the user currently facing the robot and the face image in the face image DB 220 by pattern matching to specify the user. It is better to define the face mainly by front-facing contours and features. It is better not to use lighting-dependent elements such as hair color, eye color, and skin color.

  The clothing color specifying unit 270 uses the color of the body part captured by the face image capturing unit 210 as the color of the clothing. Therefore, the user's posture is classified into “standing”, “sitting”, and “sleeping” based on the height of the user. When standing, clothes under the face are dressed, and when sitting, clothes are under the face, but are judged from the front of the body in order to distinguish them from the background such as a chair. Also, when “sleeping”, the direction in which the limbs extend is detected, and one of the upper, lower, left, and right clothing regions or at least one of the clothing regions is defined as the clothing region. In order to ensure clothing color recognition, it is preferable to move to the front of the user or to move in consideration of the light source position.

  Once the user is identified by face and clothing color, when the robot faces a certain user, if both the face and clothing color are identified as that of the specific user, the robot side will follow the computer program, Provide life support through conversations and accusations.

  Further, the clothing color may be specified in a predetermined time zone in the morning. Thus, the robot life support service can be received throughout the day after the user wears clothes.

  When the robot faces the user, the instruction unit 200 instructs the face image capturing unit to capture the face image, and the control result is output from the clothing color specifying unit 260 and the user specifying unit 270 as described above. Input to 280. Thereafter, the control unit 280 receives a service request from the user, and outputs a specific operation such as moving an arm via the output unit 290.

  FIG. 4 is an illustration and flowchart for more specifically explaining the user recognition described above. FIG. 4A is an illustration in which one of two users (father) 41 calls the robot 1 and the robot attempts to specify the clothing color of the father 41 in response to the call event. FIG. 4B is a flowchart of an example of user authentication processing. Referring to FIG. 4B, first, in S40, the user 41 inquires the robot 1 about the schedule first in the morning. Next, in S41, the robot starts face authentication of the user 41. When the face authentication is completed, the whole body posture is estimated in S42 (estimation of “standing”, “sitting”, or “sleeping”). If the estimation is uncertain, in S43, the robot 1 moves (for example, when “sitting”, the robot is dressed under the face but moves to the front of the user to distinguish it from a background such as a chair). If necessary, the robot 1 moves again in S44 (for example, when “sleeping”, the direction in which the limbs are extended is detected). If the necessary movement is completed, in S45, a clothing area matching the user's posture is determined, and a clothing color is specified. Therefore, the clothing color is recorded in S46. Thus, tracking of the user 41 is started in S47. Note that, on the next day, the above-described process of the flowchart is executed again in order to cope with the change in the clothes of the user 41.

[Article recognition]
Next, article recognition will be described with reference to FIG. 3 again. The article recognition may be, for example, first recognition of an article possessed by a specific user, second recognition of an article indicated by the specific user, third recognition of an article related to the user's behavior, and the like. In order to recognize these articles, the possession database DB 300 for registering colors of the possessions of one or more users, and the article colors that recognize the articles possessed by the specific user when the robot faces the specific user and identify the color The identification unit 310 and the property DB 300 are referred to.

  The possession DB 300 is a list of articles owned by registered users, and describes at least what the color is as an attribute of the article.

  The article color specifying unit 310 specifies the color of the article of the robot.

  In response to a service request of a specific user (user specified by face and clothes color) or voluntarily accessing the possession DB 300 and possession history DB 310, the robot uses the product color determination unit 250 to determine the service target product. The color is specified, the specified result is input from the article color determination unit 250 to the control unit 280, and an output operation such as moving the arm via the output unit 290 is performed.

[Recognition of goods possessed by a specific user]
With reference to FIG. 5, recognition of an article possessed by a specific user will be described.

  FIG. 5 is an illustration for specifically explaining the article color specifying process. Referring to FIG. 5A, a scene in which the robot 1 recognizes the mother 51 holding the mobile phone 52 is shown. . The article to be specified is a thing possessed by a specific user (a user determined by a specific face and a specific clothing color). After recognizing the color of the article (in this case, the mobile phone 52), the robot 1 searches the possession DB 300 for the possession of the specific user and retrieves the article of the same color. When there are a plurality of articles as a result of the search, one is selected with reference to the dimensions and shapes stored in the possession DB 300. By comparing the shape and dimensions with the size of the hand, it can be selected quickly. Further, a possession history DB 310 may be provided for each portable article, and the owner may be updated each time the owner changes. Thereby, it is possible to estimate the possibility that the user who is currently in front of the robot has the article, screen the candidate for the article, and quickly specify the article to be serviced. Further, the property DB 300 may be updated every time the user uses the item, and the item that is not used may be deleted from the property DB 300. As a result, articles that are frequently used are stored in the possession DB 300, and thus the service target articles are quickly identified. Moreover, you may store the image of articles | goods in property DB300. Thereby, an article can be quickly identified by pattern matching.

  FIG. 5B is an illustration showing an example of a life support service provided by the robot 1 as a result of recognizing an article possessed by a specific user. The robot 1 refers to the article arrangement DB 240, and the mobile phone 52 possessed by the mother 51 is cleaned up on the sorting shelf 53.

[Authorization of goods pointed to by a specific user]
Referring to FIG. 3 again, the authorization of an article (authorization by the robot) that is pointed to obtain a life support service for the article that the specific user does not actually possess in front of the robot will be described. In order to perform this authorization, the direction and / or position of the article pointed to by the specific user is detected, the color of the article existing in the direction / position is specified, and the article arrangement DB 240 and the article specifying unit 320 are referred to. Identify the article. The identification of the article is quickly executed according to the pattern matching with the article image stored in the property DB 300 or the article arrangement DB 240. As the instruction method, for example, pointing with a finger or a laser pointer, or voice instruction can be used. At that time, if a conversation for specifying the position is performed between the robot and the user, the direction is surely specified. On the robot side, history information such as past designated locations for the article is also used. As this usage information, an operation DB 230 described later can also be used. If the specific direction / position is known, it is possible to link to the life support service without specifying the article.

  The article arrangement DB 240 stores the article arrangement in the room or in the arrangement shelf.

  The article specifying unit 320 specifies the indicated article with reference to the article arrangement DB 240 based on the user pointing operation photographed by the face image photographing unit 210.

  FIG. 6 is an illustration for explaining a specific example of pointing. FIG. 6A shows a scene in which the user 62 is pointing the heel 61 with a finger. The robot 1 refers to the article arrangement DB and identifies an article that is a contrast for the life support service. FIG. 6B shows a scene in which the user 63 points to his coffee cup on the cupboard 64 using the laser pointer 65. If the laser pointer 65 and the robot 1 can communicate with each other, the pointing position is automatically input to the robot 1, and the robot 1 refers to the article arrangement DB 240 and specifies an article that is a reference for the life support service.

[Recognition of goods related to user behavior]
Referring to FIG. 3 again, recognition of an article related to the user's action will be described. In order to perform this authorization, the operation DB 230, the article arrangement DB 240, and the article specifying unit 320 are used. That is, the service target article is specified based on the user's operation (behavior) performed in relation to the article described in the article arrangement DB 240.

  The operation DB 230 stores the operation of the article and the operation (action) of the specific user. Thereby, for example, the action of the article can be recognized in relation to the user's action, or conversely, the user's action can be recognized in relation to the action of the article. I can grasp it. Then, by enriching the motion database that is the result of detecting the correlation between the motion of the user and the article, the user can smoothly receive a life support service from the robot.

  FIG. 7 shows some examples of scenes in which the action DB 230 may be created.

  FIG. 7A shows a scene in which an article related to the action of the user sitting on the sofa is specified. Since the user is sitting but not pointing to the sofa, the sofa is not a service target article, and the left arm with the mobile phone 71 is pointed at the robot. Therefore, if this scene is captured as an image and stored in the operation DB 230, the service target article can be immediately identified by pattern matching. In addition, the services to be provided, for example the clean up, can be identified immediately.

  FIG. 7B shows a scene in which an article related to the operation of the user 73 who is operating the door 72 is specified. The user is standing and the robot 1 is looking at the door 72. Here, it is assumed that the user 73 gives an instruction to open or close the door 72 by voice. Then, if the image and voice instruction of this scene are stored in the action DB 230, the service content (for example, a door opening / closing service by a robot) can be immediately specified by pattern matching.

  FIG. 7C shows a scene in which the user 73 operates the wall switch 75 to turn on or off the illumination lamp. The robot 1 looks at the user 73, the illumination lamp 74, and the user 73. Looking at switch door 72. Here, it is assumed that the user 73 gives an instruction to turn on or off by voice. Then, if the image and voice instruction of this scene are stored in the action DB 230, the service content (for example, a door opening / closing service by a robot) can be immediately specified by pattern matching. This example is the same as FIG. 7B, but is a difficult task for the robot to operate a small article called a switch according to the scene, and the robot 1 mainly uses pattern matching without using a program. If is operated, there is an advantage that the robot 1 can operate independently only by the color of the switch without specifying the position and size of the switch.

[Human incidental information]
FIG. 8 is a block diagram showing an example of the relationship between the human attendant information and the types of life support services provided by the robot. The robot inputs human information INPUT1, INPUT2, INPUT3, and INPUT4 (respectively, human part, human posture, human action, and time zone) using a CCD, etc. Recognize what you have, what you have, what you touch, and what you install. Here, INPUTI and OUTPUTj (i, j = 1 to 4) are alternately related. If the article to be serviced is specified by the human supplementary information OUTPUTj, the robot follows the voice instruction of the user or the accumulated information of the service performed in the past, and the life support services SERVICE1, SERVICE2, SERVICE3, SASERVICE4, SERVICE5 ( (Schedule management, Internet search, tidying up, home appliance control, action record creation).

[How to provide life support services]
With reference to FIG. 9, FIG. 10, FIG. 11, the life support service providing method of the present embodiment will be described. This method is a method for providing life support by a mobile robot to one or more users. The method includes a user recognition step for recognizing a user by face and clothing color, and human-accompanying information of the user facing the robot, that is, the color of the article worn by the user, the color of the article held, A feature information creation step for storing the color of the article or the color of the installed article, and further creating feature information storing the behavior information of the user, that is, the feature information storing whether the user is standing, sitting or sleeping; and A voice instruction waiting step for waiting for a voice instruction, and when receiving the voice instruction, the service target product is identified with reference to the feature information, and a specific service for the specific article is determined by recognizing the voice command A service specifying step and an operation history storing step of storing the service as an operation history of the robot.
[One example of life support service provision method]
FIG. 9 is an illustration showing an example of a lifestyle support service providing method. The user performs utterances 91, 92, 93, and 94 to the robot in an indefinite or scheduled time zone TZ1, TZ2, TZ3, and T4, respectively. On the other hand, the robot performs face authentication in the time zone TZ1. In TZ1, face authentication (including clothing color identification) is performed, the user's schedule is heard or the user is informed of the schedule, and color feature recording (clothing color recording) is performed. Further, an instruction is received at TZ2, TZ3, and TZ4. That is, when the user instruction voice is input, the user is identified by the face and the clothes color, the service target article is recognized based on the human supplementary information (FIG. 8), and the user voice instruction or past The contents of the service are determined with reference to the service history, and the robot operation is performed. The provision of the life support service is based on the feature information database DB1 and the operation history database DB2. Although not shown, the robot voluntarily performs a service according to the situation (for example, it lights up when it gets dark) and a regular service (for example, the TV is switched on at noon).

  FIG. 10 is a flowchart for the feature information creation step (feature information database DB1 creation step shown in FIG. 9). First, in S101, a user image is input. In step S102, person recognition is performed using face pattern matching and clothing colors. Next, in S103, human-accompanying information (article information) and behavior information (for example, the user is sitting on the sofa and holding a mobile phone. See FIG. 7A): The user stands and opens the door. 7 (see FIG. 7C) is recognized, and is taken into the feature information DB 1 as an image, for example, as a still image. Next, in S104, the article color and clothes color for the life support service are recorded. Thereby, one feature is recorded in the feature information database DB1. Then, S101 to S104 are repeated to record the next feature information (feature information on the next user and / or the next article).

  FIG. 11 is a flowchart for the operation history storage step (operation history database DB2 creation step). First, when an event such as a voice instruction from the user occurs in S111, image input and voice recognition are performed simultaneously in S112a and S112b, respectively. Next, in S113a and S113b, human-accompanying information acquisition and voice instruction understanding are performed simultaneously. Next, in S115, the service to be provided is specified. Next, in S116, referring to the feature information DB1, using the color as a main clue, mobilize all the necessary databases in the database shown in FIG. 3 to start object search (article search). It is specified (S116). Then, a service is provided by the robot operation (S117). If the service provision is completed, the event ends (S118). The robot operation performed in S117 is stored in the operation history database DB2 as an operation history, and is used for the next service specifying step (S115) if necessary.

  The present invention relates to a movable robot, and in particular, by recognizing each person of a user such as a family, responding to a call from each person or voluntarily from the robot side, a life support service for each person The present invention relates to an autonomous mobile robot that provides serve.

It is a front view of the robot of this embodiment. It is a block diagram of the robot of this embodiment. It is a block diagram of the robot centering on user recognition and article recognition. It is the illustration and flowchart explaining user recognition. It is an illustration explaining recognition of the article which a user possesses. It is an illustration explaining recognition of the article which a user points out. It is an illustration explaining recognition of an article relevant to a user's action. It is a block diagram which shows the relationship between human attendant information and a life support service. It is an illustration which shows an example of the execution form of a life support service. It is a feature information database creation flowchart. It is an operation history information database creation flowchart.

Explanation of symbols

DESCRIPTION OF SYMBOLS 1 Robot 2 Head 3 Head camera 4 Microphone 5 Front camera 6 Chest 7 Speaker 9 Arm 10 Hand 12 Skirt 13 Obstacle sensor 14 Wheel 20 Data input / output 21 CPU
22 Self Position Recognizing Unit 23 Point Point Recognizing Unit 24 Voice Recognizing Unit 25 Neck / Arm / Hand Motion Recognizing Unit 26 Body Direction Recognizing Unit 27 Past Motion / Position Database 28 Neck Joint Drive Unit 29 Wrist Joint Drive Unit 30 Arm Joint Drive unit 31 Vehicle drive unit 200 Instruction unit 210 Face image photographing unit 220 Face image DB
230 Operation DB
240 article placement DB
250 article color specifying unit 260 clothing color specifying unit 270 user specifying unit 280 control unit 290 output unit 300 property DB
310 possession history DB
41, 51, 62, 63, 73 User 52, 71 Mobile phone (user's belongings)
61 ６４ 64 Cupboard 65 Laser pointer 72 Door 74 Fluorescent lamp 75 Wall switch 91, 92, 93, 94 Speech event TZ1, TZ2, TZ3, TZ4 Time zone DB1 Feature information database DB2 Operation history database

Claims (9)

In a robot that moves to provide life support services to one or more users,
A face image database DB for registering the user's face image;
A face image photographing unit that faces the user and photographs a face image of the user;
A user identifying unit that identifies the user facing the robot with reference to the face image DB;
A clothing color identification unit that identifies a color included in a predetermined part of the body of the specific user as a clothing color;
A robot that provides the life support service to the specific user having the clothing color. In claim 1,
A robot characterized in that the clothing color is specified in a predetermined time zone in which the clothing color in a day can be distinguished. In claim 1,
An possession database DB for registering the color of the possession for each user;
When facing the robot, an article color identifying unit for recognizing an article possessed by the specific user and identifying a color thereof;
A robot characterized by referring to the property DB and identifying a person who possesses the article of the particular article color. In claim 3,
For each article, further comprising a possession history database DB for updating the possession history of the user,
Refer to the possession history DB,
A robot that identifies the user who possesses the article and is facing the robot. In claim 3,
The property DB registers the size and / or shape of the property. In claim 1,
Detecting the direction and / or position of the article pointed to by the specific user;
A robot characterized by specifying the article with reference to the property DB. In claim 1,
An operation database DB for storing the operation of the article and the operation of the specific user;
A robot that identifies an article based on an action of the article and / or an action of the specific user. In a method for providing life support by a mobile robot to one or more users,
A user recognition step of identifying the user by face and clothing color;
The human attendant information of the user facing the robot, that is, the color of the clothes worn by the user, the color of the article possessed, the color of the touched article, or the color of the installed / moved article Information creation step;
A voice instruction standby step of waiting for a voice instruction of the user;
When the voice instruction is received, the feature information is referenced to identify the article that is the target of the life support service, and at the same time, the contents of the life support service for the specific article are recognized by recognizing the voice instruction. Service identification steps to determine,
Providing the life support service;
A life support providing method comprising: an operation history storage step of storing the life support service as an operation history of a robot.   The life support providing method according to claim 8, wherein the feature information creating step further stores the behavior information of the user, that is, whether the user is standing, sitting, or sleeping.

Images