JP2007152442A - Robot guiding system - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide a robot guiding system, in which a robot makes guiding services at an appropriate time by recognizing, on a robot system side, whether a person is looking at an exhibited object and a guide-board, and whether the person is waiting to look at the exhibited object and guide-board. <P>SOLUTION: The robot guiding system includes one or more cameras Cam0 to Cam4 for taking pictures of one or more persons, and a moving robot 101 for guiding the person. The robot guiding system further comprises a position calculating device 102 for calculating the position of the person, an area stayed time measuring device 103 for measuring the stayed time of the person within a required zone, an action distinguishing device 104 for distinguishing the content of the action of the person, and a robot control platform 105 for controlling the robot based on the result of the action distinguished by the action distinguishing device, wherein a zone around the object is divided into a first zone neighboring the zone where the object is arranged, and a second zone neighboring the first zone, and the cameras track the person, and the robot guides the person who is moving or stationary in the first and second zones. <P>COPYRIGHT: (C)2007,JPO&INPIT

Description

  The present invention relates to a robot guidance system, and more particularly to a robot guidance system in which a robot guides visitors at an exhibition hall, an exhibition hall, a vending machine installation place, a ticket sales hall, and the like.

  For example, Patent Document 1 discloses a robot system in which a robot guides or recommends visitors to an exhibition hall or the like. This robot system includes a plurality of autonomous mobile robot bodies and a data acquisition server, and an infrared tag and a human observation device are provided for each presenter and visitor. The human observation device observes the interaction with other people from the viewpoint of the person wearing it, and obtains identification information and position information such as infrared tags within the detection range, as well as the biometric data of the person. Is done. The robot body is a humanoid type having visual, auditory, and tactile sensations, and observes the situation of the instructors and visitors from the viewpoint of the robot. In a ubiquitous sensor environment in which sensors are provided everywhere in the venue, the observed interaction is recorded in the interaction DB via the data acquisition server. Based on the interaction DB, the robot body presents guidance or recommendation information and information on the interaction object by gesture or voice.

Further, “attention information measuring method and apparatus and various systems using the same” disclosed in Patent Document 2 is a system for obtaining a degree of attention with respect to exhibits, displayed products, and the like. Therefore, a television camera is used to extract and track humans, accumulate attention from the movement route, stay time, stop time, line-of-sight direction, etc. within the measurement range, and analyze the relationship with sales.
Japanese Patent Laying-Open No. 2005-131713 (FIG. 2, paragraph 0079) Japanese Patent Laid-Open No. 10-48008 (FIG. 5, paragraph 0008)

  However, in Patent Document 1, a system is large because sensors are provided everywhere in the venue, and a data acquisition server and an interaction DB are provided separately from the robot. In addition, it is troublesome because the instructor and the visitor carry a human observation device equipped with a portable computer and put the infrared tag on the chest.

  Moreover, in patent document 2, since the attention degree to goods is determined by a visitor's moving route and stay time, the person who just passed the place and the person who slowly hangs cannot be distinguished.

  Therefore, an object of the present invention is to recognize on the robot system side whether or not a person is looking at an exhibit or a guide board, and whether or not to wait for viewing, and the mobile robot provides a guidance service at an appropriate timing. That is.

  A first means for solving the above-mentioned problem is a position calculation device that calculates the position of a human in a robot guidance system including one or more cameras that image one or more humans and a mobile robot that guides humans, An area stay time measuring device that measures the time that a human has stayed in a predetermined area, a behavior identification device that identifies the content of human behavior, and a robot control platform that controls the robot based on the behavior identification result by the behavior identification device; A first area in contact with the area where the article is placed and a second area in contact with the first area, the camera tracks a person, and the mobile robot stays in the first and second areas Alternatively, guidance is given to a stationary human.

  The second means is that, in the first means, the position calculation device detects the human head center of gravity by stereo vision of two or more cameras and tracks the human by the head center of gravity.

  The third means is the first means, wherein the article is an exhibition, an exhibition, a display board or a device, and the first area is an area where a person views the exhibition, the exhibition, the display board or the equipment, Area 2 is an area where a human waits to see an exhibition, exhibit, display board or equipment.

  The fourth means is that, in the first means, the mobile robot provides guidance to a person whose continuous stay time or continuous stationary time in the first region exceeds a predetermined threshold.

  The fifth means is that, in the fourth means, the predetermined threshold is a minimum value of a histogram of continuous stay time or continuous rest time.

  In the fourth means, in the fourth means, the robot speaks a question as to whether or not attention has been paid to the article, and classifies the person into a group that has paid attention to the article and a group that has not paid attention to the article. For each group, a histogram of continuous stay time or continuous stationary time in the first region is created, and the continuous stay time or continuous stationary time at which the two histograms intersect is set as a predetermined threshold.

  The seventh means is to divide the surrounding area of the article into a plurality of blocks and select a block for the first and second areas from among the plurality of blocks.

The eighth means is a robot guidance method using the robot guidance system of the first means,
A first step for determining a first threshold value of a human stationary time, on which the action identification device side determines that the person is looking at an exhibition, an exhibit, a display board or a device in the first area; A second step for determining a second threshold of human rest time, which should be determined that the exhibition, exhibit, display board or device is viewed with interest in the first region; A third step in which the mobile robot provides a guidance service to a human when the stationary time exceeds a second threshold;
A fourth step in which the mobile robot provides another guidance service to another person in the second area when the stationary time of the person in the first area exceeds the second threshold value.

  The ninth means is the eighth means, wherein the action identifying device side determines a mean time during which the human is looking at an exhibition, an exhibit, a display board or a device in the first area; When the stationary time in one area exceeds the average time, the second step in which the mobile robot provides a guidance service to a human, and when the stationary time in the first area of the human exceeds the average time, the mobile robot And a third step of providing another guidance service to another person in the second area.

  According to the first means, it is possible to provide appropriate guidance to a person who is viewing an exhibition or the like and a person who is waiting to see the exhibition.

  According to the second means, it is possible to reliably track a person.

  According to the third means, it is possible to give guidance suitable for the place to the person who sees the exhibits.

  According to the fourth means, it is possible to prevent traffic jams in the venue.

  According to the fifth means, the criterion for how long to see an exhibit etc. for guidance is clarified.

  According to the sixth means, the criterion for how long to see an exhibit etc. for guidance is further clarified.

  According to the seventh means, it is possible to appropriately determine a place where guidance is required.

  According to the eighth means, it is possible to determine whether or not a person is “seeing” an exhibit or the like, and it is also possible to determine that “a person is watching with great interest”. A variety of guidance services can be provided to "waiting" people.

  According to the ninth means, it is possible to cause the robot to program in advance a guidance service when the average time is exceeded by determining an average time during which a person “sees” an exhibition or the like.

  According to the present invention, the robot system recognizes whether or not a person is looking at an exhibit or a guide board, whether it is waiting for viewing, etc., and the mobile robot provides a guidance service at an appropriate timing. Can do.

  Embodiments of the present invention will be described below with reference to the drawings. However, even if there are specific descriptions in the dimensions, materials, shapes, relative arrangements, and the like of the component parts described in the present embodiment, the present invention is not intended to be limited thereto. In particular, in the present embodiment, an exhibition will be described as an example. However, the present invention is not limited to this, and is not limited to an exhibition. It can be used for a guidance system.

  FIG. 1 is a block diagram of the robot guidance system of this embodiment. This system is a position calculation device for calculating the position of a human in a robot guidance system including two or more cameras Cam0, Cam1, Cam2, Cam3, and Cam4 that capture one or more humans and a mobile robot 101 that guides humans. 102, an area staying time measuring device 103 that measures the time during which a human stayed in a predetermined area, a behavior identifying device 104 that identifies the content of human behavior, and a robot based on a behavior identification result by the behavior identifying device A robot control platform 105 that defines a first area in contact with the area where the article is placed and a second area in contact with the first area. The cameras Cam0, Cam1, Cam2, Cam3, and Cam4 are humans. The mobile robot 101 provides guidance to a person who stays or stops in the first and second areas. Cormorant.

  FIG. 2 is a block diagram of the position calculation device 102 that calculates the position of a person. The apparatus 102 detects the center of gravity of the human head using one or more cameras, and tracks the person based on the center of gravity of the head. Therefore, a human image is extracted from the image of the camera 230, the head extraction unit 200 extracts the head from the color information and the luminance change information, the head center of gravity extraction unit 210 extracts the center of gravity of the head, and stereo The vision calculation unit 220 uses the position, orientation, and lens parameters of each camera to set the head center-of-gravity coordinates as human position coordinates (X, Y, Z). It should be noted that the present invention can be implemented even when the number of cameras is one. Further, in the case of a single unit, stereo viewing is not performed, so the stereo vision calculation unit 220 is read as the human position calculation unit 220. Cam0 is a camera of the robot, and accuracy is a problem, but it can be used.

  The human position coordinates (X, Y, Z), which is the output of the position calculation device 102, is input to the area stay time measuring device 103 to track the person, and the first area (place near the exhibit) and the second area. The staying time at (a place far from the exhibit) is measured.

  FIG. 3A shows recognition area information input to the area stay time measuring apparatus 103. The exhibition hall is divided into an area Y (first area) close to the exhibit X and an area Z (second area) far away.

  FIG. 3B shows the blocking of the first and second areas. The periphery of the exhibit X is divided into a plurality of square or round blocks. For example, the square blocks Y1 and Y2 that are the front of the exhibit may be the first area, and the remaining square blocks may be the second area. Alternatively, the round block Y3 where people are likely to concentrate may be used as the first area, and the remaining three round blocks may be used as the second area.

  The measurement result of the stay time is input to the action identification device 104, and the action recognition result (“watching”, “waiting”, etc.) is output. “Looking” can be determined by the fact that the time in which the user stays in the first area continuously exceeds a predetermined threshold value. On the other hand, “waiting” can be determined by identifying a person who stays continuously in the second area, and the continuous stay time in the second area exceeds another predetermined threshold. .

  FIG. 4A shows an example of a histogram that defines a threshold for staying time. This histogram shows the frequency of continuous stay time of a large number of people in the past, and the minimum value of the histogram can be used as a threshold value. The peak near the origin is the peak of a person who stays in the first area and does not “see” (or the peak of a person who does not “wait” in the second area). The other peak is the peak of the person who is staying in the first area and “watching” (or the peak of the person “waiting” in the second area). If a histogram is taken for each exhibit, an optimum guidance system can be constructed by changing the threshold value for each exhibit.

  FIG. 4B is an example of another histogram that defines a threshold for staying time. The intersection of the histogram of “watching” people in the area (first region) and the histogram of “not watching” people in the area can be used as a threshold value. Here, for example, whether the robot has actually been viewed may be determined by the robot interacting with a person. Thereby, the identification system is further improved as compared with FIG.

  When the measurement area is large, it is difficult to distinguish between people who are waiting, people who are watching, people who are waiting and who are not watching (people who are slowly walking around). Therefore, the threshold value of the stationary time may be obtained from a histogram of continuous stationary time (time in which the first or second region is continuously stationary) instead of the staying time.

  The action recognition result output from the action identification device 104 is transmitted to the robot 101 directly or via the robot control platform 5 by a high-frequency wireless method or the like, and a guidance service is provided to a person.

  FIG. 5 is a front view of the mobile robot 101. In the head 2 of the robot, a top camera 3, a pair of left and right front cameras 5, and a pair of microphones 4 are arranged at the position of a human eyebrow at the lower part of the forehead. A pair of left and right speakers 7 are arranged on the chest 6 away from the vertical center line. The elbows and shoulders of both arms 9 have a joint structure, and a hand 10 holds an object. The neck also has a joint structure. In this robot, the lower limb is a cover having a skirt 12 shape, houses an electric circuit, etc., and can be moved by the left and right wheels 14 at the lowermost part of the lower limb. In addition, the pair of left and right obstacle sensors (infrared detection or ultrasonic detection) 13 automatically avoids obstacles and moves autonomously to the destination on the upper leg.

  FIG. 6 is a block diagram of the mobile robot 101 of this embodiment. Input to the CPU 21 is output from the top camera 3, front camera 5, microphone 4, obstacle sensor 13, and data input / output 20. The CPU 21 includes an accident position recognizing unit 22, a pointing point recognizing unit 23, a voice recognizing unit 24, a neck / arm / manual recognizing unit 25, a body direction recognizing unit 26, and a past motion / position database. The neck joint drive unit 28, the speaker 7, the wrist joint drive unit 29, the chest joint drive unit 30, and the vehicle drive unit 31 are driven. The communication ability of this robot by computer control is the ability to detect a person who moves or detect a face; the ability to identify a person who detects a facial feature and identifies a predetermined number of users including one owner; Speech recognition ability such as word recognition, continuous word recognition necessary for conversation scenarios, and human voice recognition by acoustic echo canceller; text-to-speech method, speech synthesis ability that can change settings such as voice volume, etc. . In addition, the moving ability is given the ability to search for and approach a person in order to support life.

[Guidance for a single exhibit]
FIG. 7 shows an example of guidance by the mobile robot 1 for one exhibit. First, in S <b> 70, it is assumed that the action identification device 4 recognizes the person A who is standing still in the first area and looking at the exhibit. Since the resting time of the person A exceeds a threshold (for example, 2 minutes), the action identification device 4 recognizes that the person A is “looking”. Next, in S <b> 71, it is assumed that the action identification device 4 recognizes that the person A is “waiting” in the second area after recognizing that the person A is “looking”. In this state, in S72, it is assumed that the person A is “waiting” while the person A is “looking” with a stationary time of 10 minutes or more. Therefore, in S73, the mobile robot 101 receives the action recognition result of the person A, determines that the person A is very interested in the exhibit, and utters "I sell replicas there". To encourage movement. Further, in S74, when the mobile robot 101 recognizes the movement of the person A, the mobile robot 101 speaks to the person B “Please come closer”.

  As described above, in the first embodiment, the threshold value indicating whether or not the user is “looking” in the first region is about 1 to 2 minutes. The reason for this is that the person who is “not looking” but is still standing is a person who is bowing, so the threshold is considered to be about 1-2 minutes. The continuous stationary time of the person A is 10 minutes or more, because the “watching” time is long, the threshold value for judging that he / she is very interested is 10 minutes. It is thought that it is the peak of the distribution of stationary time of people.

[Guidance for multiple exhibits]
FIG. 8 is an example of guidance by the mobile robot 1 for a plurality of exhibits. First, in S60, it is assumed that the person A recognizes that he / she is “looking” because the resting time of the person A exceeds a threshold value (for example, 2 minutes) in the first region on the behavior identification device 4 side. Next, in S61, it is assumed that the person B who is “waiting” in the second area is recognized 5 minutes after the person A is recognized as “looking”. It is assumed that the person A is still “looking”. In this state, in S62, the mobile robot A101 checks the status of other exhibits (whether or not it is vacant) by itself or camera information via the robot control platform 5. If it is free, the process proceeds to S66.

  In S66, the mobile robot 101 speaks to the person B "How about other exhibitions?" If the person B does not move, the process proceeds to S63 described later. On the other hand, if the person B moves, the entire process is terminated.

  On the other hand, if no other exhibits are available in S62, the process proceeds to S63.

  In S63, the mobile robot 101 speaks to the person B "Please wait for about 5 minutes". Therefore, suppose that person B waits for another 5 minutes, but person A is still watching and person B is still waiting. In S64, it is assumed that the mobile robot 101 recognizes that by the camera information via the robot control platform 5 itself. Therefore, in S65, the mobile robot 101 utters the chest that the person A is about to change in S65.

  In the second embodiment described above, the average time is assumed to be 10 minutes for the exhibits that the person A “sees”. Therefore, in S61, if the action identification device 4 recognizes the person B who is "waiting" in the second area after 7 minutes, the robot will say "Please wait for about 3 minutes" in S63. I should have told person B.

[Setting of first and second areas]
FIG. 9A shows an example of setting the first region Y and the second region Z when the guide plate X is the target of attention. The region Y has a radius of 1 m from the front center of the guide plate X, and the region Z surrounds the region Y and has a radius of 2 m from the front center of the guide plate X.

  FIG. 9B shows a setting example of the first region Y and the second region Z when the device X operated in front is the target of attention. The area Y is a rectangular area from the front width of the device X to a location 50 cm away from the front of the device X, and the region Z surrounds the region Y from the front of the device X up to 1 m to the left and right. A rectangular area up to 1.5m away from the front.

  FIG. 9C shows a setting example of the first region Y and the second region Z when the device X operated from the front and the side is the target of attention. As an initial setting of the area Y, three rectangular areas Y2, Y3, Y4 are provided in front of the device X and one rectangular area Y1, Y5 is provided on the left and right of the apparatus X according to the installation location of the operation switch or the like.

  FIG. 9D shows a setting example of the first region Y and the second region Z in the case where the device X operated on the front and side is the target of attention. Since it turned out that people gathered in the areas Y1 and Y3 with respect to the initial setting in FIG. 9C, the areas Y1 and Y3 were excluded from the recognition area of the area stay time measuring device 3 thereafter. is there.

  INDUSTRIAL APPLICABILITY The present invention can be used for guidance by a mobile robot in general places where people gather such as exhibition halls, exhibition halls, near display boards, in the vicinity of equipment such as vending machines, ticket sales halls, and the like.

It is a block diagram of the robot guidance system of the present invention. It is a block diagram of the apparatus which calculates a person's position. It is an example of two area | regions set around the exhibit. It is a histogram which determines the threshold value which determines whether it is "watching" to see an exhibit or "waiting" to see. It is a front view of a mobile robot. It is a block diagram of a mobile robot. 3 is a flowchart of guidance according to the first embodiment. 10 is a flowchart of guidance according to the second embodiment. 12 is an example of area setting according to the third embodiment.

Explanation of symbols

2 Head 3 Head Camera 4 Microphone 5 Front Camera 6 Chest 7 Speaker 9 Arm 10 Hand 12 Skirt 13 Obstacle Sensor 14 Wheel 20 Data Input / Output 21 CPU
22 Self Position Recognizing Unit 23 Point Point Recognizing Unit 24 Voice Recognizing Unit 25 Neck / Arm / Hand Motion Recognizing Unit 26 Body Direction Recognizing Unit 27 Past Motion / Position Database 28 Neck Joint Drive Unit 29 Wrist Joint Drive Unit 30 Arm Joint Drive unit 31 Vehicle drive unit 101 Mobile robot 102 Position calculation device 103 Area stay time measuring device 104 Action identification device 105 Robot control platform Cam0-Cam4 camera

Claims (9)

In a robot guidance system including one or more cameras that image one or more humans and a mobile robot that guides the humans,
A position calculation device for calculating the position of the person;
An area stay time measuring device for measuring the time during which the person stays in a predetermined area;
An action identification device for identifying the human action content;
A robot control platform for controlling the robot based on a behavior identification result by the behavior identification device,
In the article peripheral area, a first area in contact with the area where the article is placed and a second area in contact with the first area are defined,
2. The robot guidance system according to claim 1, wherein the camera tracks the person, and the mobile robot performs the guidance for the person staying or standing still in the first and second areas. In claim 1,
The position calculation device detects the center of gravity of the human head by stereo viewing of the two or more cameras, and tracks the human by the center of gravity of the head. In claim 1,
The article is an exhibition, an exhibit, a display board or a device,
The first region is a region where the person views the exhibition, exhibit, display board or device,
The robot guidance system according to claim 2, wherein the second area is an area where the person waits to see the exhibit, display board, or device. In claim 1,
The robot guidance system according to claim 1, wherein the mobile robot performs the guidance to the person whose continuous stay time or continuous stationary time in the first region exceeds a predetermined threshold. In claim 4,
The robot guidance system according to claim 1, wherein the predetermined threshold value is a minimum value of a histogram of the continuous stay time or continuous stationary time. In claim 4,
The robot speaks a question as to whether or not the person has focused on the article,
Classifying the human into a group that focused on the article and a group that did not focus on the article;
For each group, create a histogram of the continuous stay time or continuous rest time in the first region,
The robot guidance system characterized in that the continuous stay time or continuous stationary time at which the two histograms intersect is set as the predetermined threshold value. In claim 1,
Dividing the surrounding area of the article into a plurality of blocks;
A robot guidance system, wherein the block is selected for the first and second areas. A robot guidance method using the robot guidance system according to claim 1,
A first step in which the action identification device side determines a first threshold value of the human stationary time, which should be determined that the human is looking at an exhibition, an exhibit, a display board or a device in the first area; ,
A second step of determining a second threshold for the human stationary time to determine that the person is looking at the exhibition, exhibit, display board or equipment with interest in the first region;
A third step in which the mobile robot provides a guidance service to the human when the stationary time of the human in the first region exceeds the second threshold;
A fourth step in which the mobile robot provides another guidance service to another person in the second area when the stationary time of the person in the first area exceeds the second threshold value. Robot guidance method characterized. In claim 8,
A first step in which the behavior identification device side determines an average time during which the human is looking at an exhibition, an exhibit, a display board or a device in the first area;
A second step in which the mobile robot provides a guidance service to the human when the stationary time of the human in the first area exceeds the average time;
A third step in which the mobile robot provides another guidance service to another person in the second area when the stationary time of the person in the first area exceeds the average time. Robot guidance method characterized.

Images