JP2007229817A - Autonomous mobile robot - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide an autonomous mobile robot capable of fetching information on an object, in which a specified person is interested, when the robot moves together with the person. <P>SOLUTION: This mobile robot moves together with a specified person while taking images of a peripheral condition of a specified person by CCD cameras 6A, 6B and controlling driving wheels of a traveling device based on the taken image data by a wheel-in-motor control part 5E. At that time, when a gaze condition determination part 5h detects a gazing condition of a vision of a specified person, an interested-object storing part 5J stores the taken images of the interested object, which the person gazes. Thereafter, an interested-object informing part 5K compares the image of the peripheral condition created by the image creating part 5A with the image including the interested-object stored in the interested object storing part 5J, and informs existence of the interested object of the person by a loudspeaker 7 and a display 2 when the image of the peripheral condition includes the image similar to the interested-object. <P>COPYRIGHT: (C)2007,JPO&INPIT

Description

  The present invention relates to an autonomous mobile robot that can move along with a specific person.

  In recent years, research and development of robots that perform various operations by moving autonomously with a traveling device or a walking device have been actively conducted, and various autonomous mobile robots have been proposed as a result (for example, Patent Document 1). 2).

Here, Patent Document 1 proposes a robot that guides a specific guided person to a predetermined guidance destination by autonomously moving according to the walking state of the specific guided person. Patent Document 2 proposes a robot that provides appropriate information to a driver of a vehicle in a parking lot and guides the vehicle to a parking section.
Japanese Patent Laid-Open No. 2003-340764 JP-A-2005-290813

  By the way, in the conventional autonomous mobile robot described in Patent Document 1, even though it is possible to identify the guided person, it is not possible to identify the object that the specific person has shown an interest in. There is a problem that it is not possible to acquire information on an object of interest that has shown interest. The autonomous mobile robot described in Patent Document 2 has the same problem.

  Accordingly, an object of the present invention is to provide an autonomous mobile robot that can acquire information on an object of interest that a specific person has shown an interest when moving with the specific person.

  An autonomous mobile robot according to the present invention detects a gaze state of a specific person's line of sight in an autonomous mobile robot that can move along with a specific person based on observation information while observing a surrounding situation including the specific person. It is characterized by comprising gaze state detection means and interest object storage means for storing observation information of the object of interest as the object of interest of the person with the object gazeed by the specific person.

  In the autonomous mobile robot according to the present invention, when the gaze state detecting means detects the gaze state of the gaze of the specific person while moving along with the specific person based on the observation information while observing the surrounding situation including the specific person. The interest object storage means stores observation information of the object of interest that the specific person gazes at.

  In the autonomous mobile robot of the present invention, the observation information of the object of interest stored in the object of interest storage means is compared with the observation information of the surrounding situation, and the observation information of the surrounding situation is similar to the observation information of the object of interest. An interest object notifying means for notifying the presence of the object of interest when the information to be included is included can be provided.

  In the autonomous mobile robot of the present invention provided with this interesting object notification means, the interesting object notification means informs the existence of the interesting object of the specific person, so that the specific person has the interesting object in the vicinity. Can be recognized.

  In the autonomous mobile robot according to the present invention, when the gaze state detection unit detects the gaze state of the gaze of the specific person, the interest object storage unit stores observation information of the object of interest that the specific person gazes at. That is, according to the autonomous mobile robot of the present invention, it is possible to acquire information on an object of interest that the specific person has shown interest when moving with the specific person.

  Further, according to the autonomous mobile robot of the present invention provided with the interest object notification means, the interest object notification means notifies the presence of the interest object of the specific person, so that the specific person has the interest object in the vicinity. It can be recognized that it exists.

  Hereinafter, embodiments of an autonomous mobile robot according to the present invention will be described with reference to the drawings. In the drawings to be referred to, FIG. 1 is a plan view schematically showing a mobile robot as an autonomous mobile robot according to an embodiment and a specific person as a master, and FIG. 2 shows a structure of the mobile robot shown in FIG. FIG. 3 is a side view, FIG. 3 is a block diagram showing a configuration of the control device shown in FIG. 2, and FIG. 4 is a flowchart showing a part of a processing procedure by the control device shown in FIG.

  As shown in FIG. 1, the autonomous mobile robot according to an embodiment is configured as a mobile robot R that can move along with a specific person M as a master. The mobile robot R leads the specific person M or follows the specific person M based on the observation information while observing the specific person M and its surroundings in the vicinity of the specific person M.

  As shown in FIG. 2, the mobile robot R includes a body R1 that can be moved by the traveling device 1 and a head R2 that is rotatably connected to the body R1. A display 2 capable of displaying various messages such as alarms and guidance with images and characters is installed. The body R1 includes a battery 3 as a power source, a head drive motor 4 for rotationally driving the head R2, and a control device 5 for controlling the travel device 1, the display 2, the head drive motor 4, and the like. Built in.

  On the other hand, in the head R2 of the mobile robot R, in order to observe a surrounding situation including a specific person M and obstacles (another person, animal, structure, installation, vehicle, etc.) existing around it as a stereo image. A pair of left and right CCD cameras (or infrared cameras) 6A and 6B and a speaker 7 capable of producing voices of various messages such as notifications, warnings, and guidance are installed.

  As shown in FIGS. 1 and 2, the traveling device 1 includes, for example, a pair of left and right drive wheels 1C and 1D whose rotation direction and rotation speed are independently controlled by wheel-in motors 1A and 1B, and a 360-degree turn. It is possible to have caster wheels 1E.

  The traveling device 1 moves the mobile robot R forward or backward by driving the pair of left and right drive wheels 1C and 1D forward or backward in the same direction. At that time, if the rotational speeds of the pair of left and right drive wheels 1C, 1D are the same, the mobile robot R moves straight, and if there is a difference between the rotational speeds of the pair of left and right drive wheels 1C, 1D, The mobile robot R is turned to the slow rotation side. When the pair of left and right drive wheels 1C and 1D are driven to rotate in the opposite directions at the same speed, the traveling device 1 rotates the mobile robot R to the left or right on the spot.

  Here, the control device 5 is configured by using microcomputer hardware and software such as an ECU (Electric Control Unit). The control device 5 includes a drive circuit for the wheel-in motors 1A and 1B and the head drive motor 4, an input / output interface I / O between the display 2, the CCD cameras 6A and 6B, the speaker 7, and the A / D converter. A ROM (Read Only Memory) that stores programs and data, a RAM (Random Access Memory) that temporarily stores input data and the like, a CPU (Central Processing Unit) that executes programs, and the like are provided as hardware.

  As shown in FIG. 3, the control device 5 includes an image generation unit 5A, a specific person detection unit 5B, a head drive motor control unit 5C, an obstacle detection unit 5D, a wheel-in motor control unit 5E, and a gaze point detection. The unit 5F, the gaze point locus recording unit 5G, the gaze state determination unit 5H, the interesting object storage unit 5J, and the interesting object notification unit 5K are configured as software.

  Left and right image data are input to the image generation unit 5A from the CCD cameras 6A and 6B together with measurement time information at predetermined intervals. The image generation unit 5A sequentially generates a stereo image at each measurement time based on left and right photographing data input every predetermined time, and the image data together with the measurement time information is a specific person detection unit 5B, an obstacle detection unit. 5D, the gazing point detection unit 5F, the interesting object storage unit 5J, and the interesting object notification unit 5K are repeatedly output every predetermined time.

  The specific person detection unit 5B performs edge processing of the captured image based on the image data sequentially input from the image generation unit 5A every predetermined time, and specifies by matching processing with the contour model of the specific person M stored in advance. A person M is detected. And this specific person detection part 5B presumes the three-dimensional position of the specific person M for every predetermined time by the principle of triangulation by carrying out the stereo matching process of the stereo image of the detected specific person M.

  The head drive motor controller 5C receives the three-dimensional position data of the specific person M from the specific person detection unit 5B every predetermined time. The head drive motor control unit 5C controls the rotation of the head drive motor 4 so that the specific person M is located at the center of the image, for example, based on the input three-dimensional position data of the specific person M, and the head of the mobile robot R Rotate R2.

  The obstacle detection unit 5D performs edge processing of the photographed image based on the image data sequentially input from the image generation unit 5A every predetermined time, so that the obstacle (other person or animal, Detect structures, installations, vehicles, etc.). And this obstacle detection part 5D presumes the three-dimensional position of an obstacle for every predetermined time by the principle of a triangulation by carrying out the stereo matching process of the stereo image of the detected obstacle.

  The wheel-in motor control unit 5E receives the three-dimensional position data of the specific person M from the specific person detection unit 5B every predetermined time and the obstacle three-dimensional position data from the obstacle detection unit 5D every predetermined time. Is input. The wheel-in motor control unit 5E guides or identifies the specific person M while avoiding the obstacle in the vicinity of the specific person M by the mobile robot R based on the input specific person M and the three-dimensional position data of the obstacle. In order to follow the person M, the rotation of the wheel-in motors 1A and 1B is individually controlled to individually control the rotation direction and the rotation speed of the drive wheels 1C and 1D of the traveling device 1.

  Here, the CCD camera 6A, 6B, the image generation unit 5A, the gazing point detection unit 5F, the gazing point locus recording unit 5G, and the gazing state determination unit 5H constitute the gazing state detection unit of the present invention, and the object of interest storage unit 5J. Thus, the interested object storage means of the present invention is configured, and the interested object notification unit 5K, the display 2 and the speaker 7 constitute the interesting object notification means of the present invention.

  The gazing point detection unit 5F extracts the binocular image of the specific person M from the photographed image based on the image data sequentially input from the image generation unit 5A every predetermined time, and performs edge processing on the image. The position of the black eye of both eyes is detected. The gazing point detection unit 5F detects the position of the detected black eye with a pre-stored eyeball model, thereby detecting the gazing point of both eyes as three-dimensional position data at predetermined time intervals.

  The gazing point locus recording unit 5G receives detection data of the gazing point of both eyes of the specific person M from the gazing point detection unit 5F for a predetermined time (for example, about 5 seconds). The gazing point locus recording unit 5G records the locus of the gazing point of both eyes of the specific person M based on the detection data input as the three-dimensional position data.

  The gaze state determination unit 5H receives record data of the gazing point locus of both eyes of the specific person M from the gazing point locus recording unit 5G. Then, the gaze state determination unit 5H determines that the gaze point is stopped for a predetermined time (for example, about 3 seconds) or the movement range of the gaze point can be regarded as the gaze state based on the input record data of the gaze point locus. If it is narrow, it is determined that both eyes of the specific person M are in the gaze state, and a gaze state determination signal is output to the object-of-interest storage unit 5J.

  When the interest state storage unit 5J receives a gaze state determination signal from the gaze state determination unit 5H, the object O gazeed by the specific person M is included as an interest object of the specific person M and includes the interest object Memorize situation observation information. That is, the interested object storage unit 5J provides an edge to the captured image based on the image data at the measurement time when the specific person M gazes at the object O among the image data sequentially input from the image generation unit 5A every predetermined time. Processing is performed, and the data of the captured image subjected to the edge processing is stored.

  The interested object notifying unit 5K performs edge processing on the captured image based on the image data sequentially input from the image generating unit 5A every predetermined time, and the captured image data of the peripheral situation subjected to the edge processing, and the object of interest The captured image data including the object of interest stored in the object storage unit 5J after edge processing is compared. At that time, the arrangement pattern of the luminance values in the photographed image is also compared.

  The interested object notifying unit 5K includes an object of interest in the vicinity of the specific person M when image data similar to the object of interest is included in the captured image data of the surrounding situation to be compared. This is notified by the speaker 7 and the display 2. That is, the object-of-interest notification unit 5K outputs a predetermined sound signal to the speaker 7 to utter a notification sound or a notification message, and outputs a predetermined image signal to the display 2 to output a notification image or text message. Is displayed.

  Here, the processing procedure by the image generation unit 5A, the gazing point detection unit 5F, the gazing point locus recording unit 5G, the gazing state determination unit 5H, and the object of interest notification unit 5K constituting the control device 5 shown in FIG. It demonstrates along the flowchart shown in FIG.

  First, in step S1, the image generation unit 5A generates a stereo image of the surrounding situation including the specific person M at each measurement time based on the left and right photographing data input together with the measurement time information from the CCD cameras 6A and 6B every predetermined time. Generate sequentially.

  In the next step S2, the gazing point detection unit 5F detects the positions of the black eyes of both eyes of the specific person M based on the image data sequentially input from the image generation unit 5A every predetermined time. Then, by comparing the detected position of the black eye with a pre-stored eyeball model, the gazing point detection unit 5F detects the gazing point of both eyes as three-dimensional position data at predetermined time intervals.

  In subsequent step S3, the gazing point locus recording unit 5G records the gazing point locus of both eyes of the specific person M based on the gazing point detection data input from the gazing point detection unit 5F every predetermined time.

  In the next step S4, the gaze state determination unit 5H determines whether or not both eyes of the specific person M are in the gaze state based on the data of the gaze point locus input from the gaze point locus recording unit 5G. In this determination, when the gazing point is stopped for a predetermined time (for example, about 3 seconds) or when the moving range of the gazing point is narrow enough to be regarded as the gazing state, the gazing state is determined. Is determined not to be in a gaze state.

  If the determination result in step S4 is NO and not the gaze state, the process returns to step S1, but if the determination result is YES and the gaze state is established, the process proceeds to the next step S5.

  In the next step S5, the image data input from the image generation unit 5A as the image data of the surrounding situation including the object of interest that the specific person M watched in accordance with the YES determination signal input from the gaze state determination unit 5H. The object of interest storage unit 5J stores it. At that time, the interested object storage unit 5J performs edge processing on the captured image based on the image data at the measurement time when the specific person M gazes at the object O, and stores the data of the captured image subjected to the edge processing.

  In the subsequent step S6, the captured image data of the peripheral situation that is sequentially input from the image generating unit 5A to the object of interest storage unit 5J at every predetermined time and subjected to edge processing, and edge processing of the object of interest storage unit 5J is stored. The target object notification unit determines whether or not the image data similar to the target object is included in the captured image data of the peripheral situation to be compared by comparing the captured image data including the target object of interest. 5K determines.

  If the determination result in step S6 is NO and the image data similar to the object of interest is not included in the captured image data in the surrounding situation, the series of processing ends, but the determination result is YES. If image data similar to the object of interest is included, the process proceeds to the next step S7.

  In the next step S7, the object-of-interest notification unit 5K outputs a predetermined sound signal to the speaker 7 to utter a notification sound or a notification message, and outputs a predetermined image signal to the display 2 to output an image for notification or By displaying a text message, it is notified that there is an object of interest around the specific person M.

  As described above, the mobile robot R (see FIGS. 1 and 2) configured as the autonomous mobile robot according to the embodiment captures the surrounding situation including the specific person M with the CCD cameras 6A and 6B, and the captured image. Based on this data, the rotation direction and the rotation speed of the pair of left and right drive wheels 1C, 1D of the traveling device 1 are independently controlled so that the specific person M can be avoided while avoiding obstacles in the vicinity of the specific person M. Lead or follow a specific person M.

  At that time, when the mobile robot R detects the gaze state of the line of sight of the specific person M by the gaze state determination unit 5H of the control device 5, the captured image of the peripheral situation including the object of interest that the specific person M has watched is edge-processed. Is stored in the object storage unit 5J.

  After that, the mobile robot R compares the image of the surrounding situation generated by the image generating unit 5A with the image including the target object stored in the target object storage unit 5J by the target object notifying unit 5K of the control device 5. To do. Then, as a result of this comparison processing, if the image of the surrounding situation generated by the image generation unit 5A includes an image similar to the object of interest, the mobile robot R sends the interest object notification unit 5K to the speaker 7. A predetermined sound signal is output and a notification sound or a notification message is uttered, and a predetermined image signal is output from the interested object notification unit 5K to the display 2 to display a notification image or a text message. Notify that there is an object of interest around M.

  Therefore, according to the mobile robot R configured as the autonomous mobile robot according to the embodiment, it is possible to acquire the information on the object of interest that the specific person M gazes at as the image data stored in the object of interest storage unit 5J. It is possible to recognize the object of interest that the specific person M has shown interest from the image data stored in the object of interest storage unit 5J.

  Further, according to such a mobile robot R, the specific person M is interested in himself / herself by the notification sound or notification message uttered from the speaker 7 and the notification image or character displayed on the display 2. Recognize that the object of interest exists around you.

  The autonomous mobile robot according to the present invention is not limited to the above-described embodiment. For example, the head R2 of the mobile robot R shown in FIG. 2 is equipped with a millimeter wave radar (or laser radar) for observing the distance to a specific person and obstacles around the specific person and the surface roughness of the person. May be installed. Also, an infrared sensor for observing other people and animals around the specific person, a sound collecting microphone for observing sounds around the specific person, or a temperature sensor for observing the temperature around the specific person Etc. may be installed.

  In the mobile robot R having a millimeter wave radar (or laser radar), an infrared sensor, a sound collecting microphone, a temperature sensor and the like in the head R2, a CCD camera is used as observation information of an object of interest when a specific person is in a gaze state. Therefore, in addition to the observation information, observation information from the millimeter wave radar (or laser radar), infrared sensor, sound collecting microphone, and temperature sensor can be stored in the object storage unit.

It is a top view which shows typically the mobile robot as an autonomous mobile robot which concerns on one Embodiment of this invention, and the specific person as its master. It is a side view which shows the structure of the mobile robot shown in FIG. It is a block diagram which shows the structure of the control apparatus shown in FIG. It is a flowchart which shows a part of process sequence by the control apparatus shown in FIG.

Explanation of symbols

  DESCRIPTION OF SYMBOLS 1 ... Traveling apparatus, 2 ... Display, 3 ... Battery, 4 ... Head drive motor, 5 ... Control apparatus, 5A ... Image generation part, 5B ... Specific person detection part, 5C ... Head drive motor control part, 5D ... Obstacle detection , 5E ... wheel in motor control unit, 5F ... gazing point detection unit, 5G ... gazing point locus recording unit, 5H ... gaze state determination unit, 5J ... interest object storage unit, 5K ... interest object notification unit, 6A, 6B ... CCD camera, 7 ... Speaker, R ... Mobile robot, M ... Specific person.

Claims (2)

In an autonomous mobile robot that can move along with a specific person while observing the surrounding situation including the specific person,
Gaze state detection means for detecting a gaze state of the gaze of a specific person;
An autonomous mobile robot characterized by comprising an object-of-interest storage means for storing observation information of an object of interest as an object of interest of the specific person, with the object of interest of the specific person as the object of interest.   2. The autonomous mobile robot according to claim 1, wherein observation information of an object of interest stored in the object of interest storage unit is compared with observation information of a surrounding situation, and the interest information is included in the observation information of the surrounding situation. An autonomous mobile robot characterized by comprising interest object notifying means for notifying the presence of an object of interest when information similar to the observation information of the object is included.

Images