JP2009217456A - Landmark device and control system for mobile robot - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide a landmark device that allows position control with high precision over a wide range and that is structurally very simple. <P>SOLUTION: The landmark device includes a short-range landmark set 400 having two marks Mn spaced a predetermined distance apart, and a long-range landmark set 500 having two marks Mf spaced a wider distance apart than the short-range landmark set 400. When a mobile robot 200 is located away from the landmark device 300, the marks Mf of the long-range landmark set 500 where two points are at a distance from each other are used to perform position recognition for the distance between the landmark device 300 and the mobile robot 200, their directions and the like. As the moving robot 200 approaches the landmark device 300, position recognition is performed using the marks Mn of the short-range landmark set 400. <P>COPYRIGHT: (C)2009,JPO&INPIT

Description

  The present invention relates to a landmark apparatus and a mobile robot control system using the landmark apparatus. Specifically, the present invention relates to a landmark apparatus used for position control of a mobile robot that moves autonomously.

As a self-localization method for a mobile robot that moves autonomously in a work environment, a self-localization method based on external observation is known.
In such a self-position estimation method, first, the position of the landmark existing in the work space is stored in advance in the mobile robot as map information. Then, the landmark is observed, and the self position is calculated from the relative positional relationship with the landmark whose position is known.
In estimating the self position, two landmarks are sufficient if the two-dimensional position is specified, and three landmarks are required to specify the three-dimensional position or the position and orientation.

  As means for observing landmarks, it is known to use image data captured by a camera. That is, the surrounding environment is imaged by a camera having an imaging element such as a CCD, and landmarks included in the obtained image data are extracted. Based on the extracted landmark, the distance between the landmark and the imaging position is calculated by geometric calculation to estimate the self position.

  For example, Patent Documents 1 to 6 listed below are cited as documents disclosing methods for performing self-position recognition and movement direction control using a camera image obtained by imaging a landmark.

JP 2004-216552 A JP 2002-132346 A JP 2002-268746 A JP 2006-346767 A Japanese Patent Laid-Open No. 03-166072 JP 2004-303137 A

Here, the position estimation accuracy of the robot depends on the distance resolution per pixel in the captured image. Generally, if a sufficiently separated landmark image can be obtained in the captured image, highly accurate position estimation is performed. Can do.
Therefore, when the mobile robot is close to the landmarks to such an extent that a landmark image sufficiently separated in the imaging region can be obtained with respect to the landmarks spaced apart with a predetermined interval, the accuracy is high. Positioning can be performed.
On the other hand, when the mobile robot is away from the landmark, the distance between the landmarks in the captured image becomes short. Then, the position estimation accuracy becomes low, and in the moving direction control based on the position estimation, overshoot is repeated with respect to the target position, and stable control along the ideal target trajectory becomes difficult ( This problem applies to Patent Document 6, for example).

Here, there is also an idea that landmarks are installed everywhere in the work environment so that any landmark image that is appropriately separated can be obtained regardless of the position of the mobile robot.
However, installing many landmarks in the work environment, giving all these landmarks positions on the three-dimensional coordinates and teaching them to the mobile robot, and also locating multiple landmarks on the mobile robot. It is necessary to make them recognize one by one, and the system becomes very complicated and expensive (this problem applies to Patent Document 1, for example).

  An object of the present invention is to provide a landmark device that enables highly accurate self-position estimation regardless of the distance from the landmark, and a control system for a mobile robot using the landmark device. There is to do.

  A landmark apparatus according to the present invention is an landmark apparatus installed in a working environment of a mobile robot that has an imaging unit that images the surrounding environment and autonomously performs self-position estimation and movement direction control, and is separated by a predetermined distance. A short distance landmark set having at least two marks spaced apart from each other, and a long distance landmark set having at least two marks spaced apart from each other by a distance larger than the short distance landmark set. The midpoint between the marks of the short distance landmark set and the midpoint between the marks of the long distance landmark set are imaged by the imaging means from a position away from the landmark by a predetermined distance. In this case, they are included in the same imaging region.

In such a configuration, the mobile robot performs position control using a landmark device.
When the mobile robot is located far from the landmark device, the distance and direction between the landmark device and the mobile robot are determined using the marks of the long-range landmark set where the distance between the two points is long. Perform position recognition.
By providing the long-distance landmark in which the distance between the two points is long, two marks sufficiently separated in the captured image can be obtained even when the mobile robot is away from the landmark. Therefore, highly accurate position control can be performed even when the mobile robot is away from the landmark device.
On the other hand, when the mobile robot approaches the landmark device and, for example, two marks of the long-distance landmark set cannot be imaged in one imaging area, position recognition is performed using the marks of the short-distance landmark set. Since the midpoints of the short-range landmark set and the long-range landmark set are arranged within a range that is included in the same imaging area, the position using the long-range landmark set It is possible to easily switch from the control to the position control using the short distance landmark set.
In this way, depending on the distance between the mobile robot and the landmark device, the landmark to be used can be used separately for long distance and short distance, so position control can be performed with high accuracy regardless of the distance from the landmark device. It can be carried out.
In addition, the present invention has a very simple configuration as compared with the conventional configuration in which many marks are arranged everywhere in the work environment, and it is a revolutionary in that high-accuracy position control can be easily realized over a wide range. Has an effect.

  In the present invention, it is preferable that the short distance landmark set and the long distance landmark set are disposed at different positions in the height direction.

In such a configuration, even when the long-distance landmark set and the short-distance landmark set are simultaneously imaged in one captured image, the two can be distinguished by the difference in height. Therefore, even when there are two sets of the short distance landmark set and the long distance landmark set, each can be accurately identified and recognized, and correct position control can be performed.
Here, when many marks are arranged everywhere in the work environment as in the prior art, a complicated configuration for identifying each mark is required. In the present invention, the landmark set for long distance and the object for short distance are used. Since the landmark set can be simply identified by the difference in height, the control method of the mobile robot using the landmark device of the present invention can also have a simple configuration.

Here, in the landmark device, one set of the short distance landmark set and the long distance landmark set is arranged on the upper side with the height of the optical axis of the imaging means in between. The other set is preferably arranged on the lower side.
According to such a configuration, it is possible to easily and accurately identify whether the captured image data is a long-distance landmark set or a short-distance landmark set by vertically dividing the image data into two. .

  For example, each mark is formed by a light emitter and the blinking frequency of the light emitters is different from each other without distinguishing the landmark set for long distance and the landmark set for short distance in the height direction. May be identified.

  In the present invention, it is preferable that a midpoint between marks of the short distance landmark set and a midpoint between marks of the long distance landmark set are on the same vertical line.

  In such a configuration, when the mobile robot is away from the landmark device, the long distance landmark set is used for position control, and when the mobile robot approaches the landmark device, the short distance landmark set is used. Switch to the used position control. Then, when switching from position control using the long-range landmark set to position control using the short-range landmark set, if the center point of each landmark set is on the same vertical line, such switching is performed. In this case, the control structure of the mobile robot can be simplified by searching up and down in the image.

  In the present invention, it is preferable that the short distance landmark set and the long distance landmark set are disposed in the same plane.

  In such a configuration, when the short-distance landmark set and the long-distance landmark set are arranged in one plane, for example, each mark may be arranged on one board, so that it can be easily manufactured. .

  It is preferable that the short distance landmark set and the long distance landmark set are in the same plane, and that the midpoints are on the same vertical line.

  The mobile robot control system of the present invention preferably includes the landmark device and a mobile robot that moves autonomously by performing position control based on image data obtained by imaging the landmark device.

  According to such a configuration, it is possible to obtain a mobile robot control system that exhibits the same effects as the above-described invention. In other words, the mobile robot control system can perform position control with high accuracy regardless of the distance from the landmark device.

  According to the present invention, the landmarks to be used according to the distance between the mobile robot and the landmark device are selectively used for the long distance and the short distance, so that the high accuracy can be obtained over a wide range regardless of the distance from the landmark device. Position control can be performed.

Embodiments of the present invention will be described with reference to the drawings.
A first embodiment according to the mobile robot control system 100 of the present invention will be described.
FIG. 1 is a diagram showing an external configuration of a control system 100 for a mobile robot.
The mobile robot control system 100 includes a mobile robot 200 that performs position control autonomously and a landmark device 300.
The mobile robot 200 includes a main body unit 210, a moving unit 220, and an imaging unit 230.
The structure of the moving means 220 is not particularly limited as long as the main body 210 can be moved, such as a wheel type, a crawler type, and a bipedal walking type, but FIG. 1 shows a case of a wheel type.
The imaging unit 230 includes an image sensor such as a CCD, and is provided in the main body 210.
Mobile robot 200 uses, for example, a rechargeable secondary battery as a power source, and autonomously charges when the remaining battery level is low. That is, when the remaining amount of the battery is low, the battery autonomously moves toward the charging device 700 installed at a predetermined position, and receives power from the charging device 700 to charge the battery.

  The main body unit 210 includes a control unit (not shown). The control unit performs a process of processing image data captured by the imaging unit 230, a process of performing position control using the landmark device 300, and the like. The control operation by such a control unit will be described later.

Next, the landmark apparatus 300 will be described.
FIG. 2 is a diagram illustrating a configuration of the landmark device 300.
The landmark device 300 includes a short-range landmark set 400 and a long-range landmark set 500.
The short-range landmark set 400 includes two marks Mn that are spaced apart from each other by a predetermined distance. The distance between the two marks Mn in the short-range landmark set 400 is such that when the mobile robot 200 approaches the landmark device 300 (for example, in the range of 10 cm to 1 m), the two marks enter the imaging area of the imaging means 230. In addition, it is a distance that is sufficiently separated.
Such an interval between the landmark set 400 for short distances cannot be generally specified because it depends on the configuration of the lens of the imaging unit 230 and the image processing unit, but for example, about 3 to 30 cm is given as an example. The two marks Mn in the short-range landmark set 400 are arranged on one horizontal line L _Hn ( _note that the horizontal line here means a line parallel to the horizontal plane).

The long-range landmark set 500 includes two marks Mf that are spaced apart from each other by a distance greater than that of the short-range landmark set 400.
The distance between the two marks Mf of the long distance landmark set 500 is set wider than the distance between the two marks of the short distance landmark set 400.
The distance between the two marks Mf in the long-range landmark set 500 is set so that the distance between the two marks Mf is within the imaging area of the imaging means 230 when the mobile robot 200 is considerably separated from the landmark device 300 (for example, a range of 1 m to 10 m). The distance is such that the mark Mf enters and is sufficiently separated.
Such a distance between the landmark set 500 for long distance depends on the configuration of the lens of the imaging unit 230 and the image processing unit, and thus cannot be specified in general. For example, about 30 to 300 cm is given as an example.
The two marks Mf of the long distance landmark set 500 are arranged on one horizontal line L _Hf (the horizontal line here means a line parallel to the horizontal plane).

The mark Mn of the short-range landmark set 400 and the mark Mf of the long-range landmark set 500 are composed of, for example, a light emitter.
Although it does not specifically limit as a light-emitting body, For example, LED (light emitting diode) is mentioned as an example. The emission color of the illuminant is not limited. For example, the illuminant may emit light in the infrared region. The illuminator may be lit constantly or may be blinked at a specific cycle.

The short distance landmark set 400 and the long distance landmark set 500 are disposed on the same plane. For example, it is provided on the wall surface of the wall 600.
The midpoints P _mn and P _mf of the short-range landmark set 400 and the long-range landmark set 500 are arranged on the same vertical line Lv.
At this time, the short distance landmark set 400 is disposed on the upper side, and the long distance landmark set 500 is disposed on the lower side.
In the present embodiment, the optical axis height Ha of the imaging means 230 of the mobile robot 200 is fixed, and the short distance landmark set 400 is placed on the upper side with the optical axis height Ha in between. A long-distance landmark set 500 is disposed in the center.
In addition, such arrangement | positioning is not specifically limited, The upper and lower sides may be reversed.
The short distance landmark set 400 and the long distance landmark set 500 are provided relatively close to each other. For example, when the mobile robot 200 is close to the landmark device 300 to some extent, the midpoints P _mn and P _mf for the short distance and the long distance respectively enter the same imaging region at the same time.
Although not specified as a specific distance, for example, the distance between the midpoint of the short-range landmark set 400 and the midpoint of the long-range landmark is about 1 cm to 15 cm.

Here, the coordinates of each mark of the landmark apparatus 300 are set and input in advance in the control unit of the mobile robot 200.
In the present embodiment, the charging device 700 is provided on the vertical line Lv where the midpoint _Pmn of the short-range landmark set 400 and the midpoint _Pmf of the long-range landmark set 500 exist. .

Next, a position control method for the mobile robot 200 using the landmark apparatus 300 having such a configuration will be described.
In particular, a method for estimating the self position Ps by the mobile robot 200 will be described.
FIG. 3 is a flowchart showing the procedure of the position control method of the mobile robot 200.
In estimating the self position Ps, the mobile robot 200 first images the surrounding environment by the imaging means 230 (imaging process ST100). Then, the image data obtained in the imaging step (ST100) is subjected to image processing to extract mark candidates (mark candidate extraction step ST200).

The procedure of the mark candidate extraction step (ST200) is shown in FIG.
In the mark candidate extraction step (ST200), first, threshold processing is performed on the image data to search and identify pixels having a luminance equal to or higher than the threshold. If there is a pixel having a luminance equal to or higher than the threshold (ST220: YES), it is further determined whether or not pixels having a luminance equal to or higher than the threshold are continuous (ST230). (ST230: YES), these pixels are combined for each region (ST240).
Further, each region of the combined pixels is labeled (ST250), the area and roundness are calculated for each region (ST260), and each region is identified as a mark candidate (ST270).

  If the pixel brightness does not exceed the threshold brightness in ST220 (ST220: NO), the mark is not captured, and the process returns to the imaging step (ST100) and starts again. Also, in ST230, when pixels having a luminance equal to or higher than the threshold value are not continuous (ST230: NO), the marks Mn and Mf are not captured, so the process returns to the imaging step (ST100) and starts again.

After the mark candidate extraction (ST200) is thus completed, the process proceeds to the mark extraction step (ST300).
The procedure of the mark extraction process (ST300) is shown in FIG.
In the mark extraction step (ST300), first, the number of mark candidates in the image data is counted (ST310).
When the number of mark candidates is 2 or more and 4 or less, it is determined that the marks Mn and Mf of the landmark apparatus 300 have been imaged and identified as the marks Mn and Mf.
The process proceeds to a mark determination step (ST400) for determining whether the marks Mn and Mf thus identified are for short-distance use or long-distance use.

  In ST320, when there are one mark candidate or five or more mark candidates (ST320: NO), the number of marks is insufficient for position control, or something other than marks is erroneously detected. Therefore, the procedure returns to the imaging step (ST100).

Next, the mark determination step (ST400) will be described.
FIG. 6 is a flowchart showing the procedure of the mark determination step (ST400).
In the mark determination step (ST400), first, ellipse approximation processing is performed on the mark extracted in the mark extraction step (ST300) (ST410), and then the center of gravity for each mark is calculated (ST420).
Then, the image data is vertically divided into two (ST430). That is, the image data is divided into two at a horizontal line Ha passing through the center of the image data.
Next, the mark position in the two-divided image data is confirmed using the previously calculated center-of-gravity position (ST440).
Here, when both of the long-range landmark sets 500 can be imaged, the position control using the long-range landmark set 500 is more accurate, so is the long-range landmark set 500 captured? First, determine whether or not.
That is, in ST450, it is determined whether there are two marks below the horizontal line Ha that divides the image data into two (ST450).
If there are two marks below the bisector (ST450: YES), it is determined that these two marks Mf are the long-range landmark set 500 (ST460).

  On the other hand, if there are no two marks Mf below the bisector (ST450: NO), it is then determined whether there are two marks above the bisector (ST470). If there are two marks on the upper side of the dividing line (ST470: YES), it is determined that these two marks Mn are the short distance landmark set 400 (ST480).

  In ST470, when there are no two marks above the two-part dividing line Ha of the image data, the process returns to the imaging step (ST100) and starts again.

  After determining whether the mark in the captured image data is the long-range landmark set 500 or the short-range landmark set 400, the process proceeds to self-position estimation (ST500).

In the self-position estimation step (ST500), the self-position Ps of the mobile robot 200 is estimated using the position of the center of gravity of the imaged mark.
Since the mark position of the landmark apparatus 300 is set and input in advance in the control unit, the self position of the mobile robot 200 in the real space can be obtained from the mark position in the image data and the camera parameters. The direction vector to the point in the real space as viewed from the imaging means 230 is obtained from the relationship of the transformation matrix for converting the point in the U-V coordinate system set on the image plane including the mark into the coordinate system in the real space (world coordinate system). obtain.
A position Ps (point of self-location) in the real space of the mobile robot 200 is uniquely determined by the direction vectors for the two marks (see FIG. 7).

Thus, when the self-position Ps in the real space is specified by self-position estimation, various position controls are executed, and movement control to a predetermined destination is performed.
Such movement control includes, for example, an operation of autonomously approaching the charging device 700 installed at a predetermined position and further connecting a charging plug (not shown) of the mobile robot 200 to the charging device 700. It is done.
In such a charging operation, there are two stages: approaching the charging device 700 from a long distance and further positioning the charging device 700 with high accuracy after approaching the charging device 700. .
In this case, when approaching the charging device 700 from a long distance, position control is performed using the long-range landmark set 500, and when approaching the charging device 700, the short-range landmark set 400 is used at a short distance. Perform high-precision alignment.

Here, since the two marks Mf of the long-distance landmark set 500 do not both enter the imaging area when the charging device 700 is short-distance, position control by the long-distance landmark set 500 is performed. Can not.
Even in that case, since the midpoint P _{mn of the} long-range landmark set 500, the mid-point _Pmn of the short-range landmark set 400, and the charging device 700 are also on the same vertical line Lv, the long-range landmark set When the position control is performed with the charging device 700 as a target using 500, when the long-distance landmark set 500 does not enter the imaging region, the short-distance landmark set 400 is surely in the imaging region.
Therefore, the position control is continued without interruption.
Finally, by performing high-precision alignment using the short-range landmark set 400, charging can be performed by connecting the charging plug of the mobile robot 200 to the charging device 700.

According to the present embodiment having such a configuration, the following effects can be obtained.
(1) Since the landmark apparatus 300 is provided with a long distance landmark set 500 in which the distance between the two points is widely separated and a short distance landmark set 400 in which the distance between the two points is small, By using different landmarks for long distance and short distance depending on the distance from the landmark apparatus 300, position control can be performed with high accuracy regardless of the distance from the landmark apparatus 300.

(2) Compared to a complicated configuration in which a large number of marks are arranged everywhere in the work environment as in the prior art, this embodiment is provided with a long distance landmark set 500 and a short distance landmark set 400. It has a simple configuration and has an epoch-making effect of simply realizing highly accurate position control over a wide range.

(3) Since the long distance landmark set 500 and the short distance landmark set 400 are provided at different heights, each can be easily identified and determined in the captured image.
Therefore, even when two sets of the short distance landmark set 400 and the long distance landmark set 500 are provided, each can be accurately identified and recognized, and correct position control can be performed.

(4) Since the midpoint between the two marks of the short distance landmark set 400 and the midpoint between the two marks of the long distance landmark set 500 are on the same vertical line, the long distance landmark set 500 If it is directed to the charging device 700 (or the middle point of the long-range landmark set 500) by position control using the short-range landmark set 500 when the long-range landmark set 500 does not enter the imaging region, 400 surely falls within the imaging area. Therefore, the position control using the long distance landmark set 500 can be shifted to the position control using the short distance landmark set 400 without interruption, and the position control can be continued.

(5) Since the landmark set 400 for short distance and the landmark set 500 for long distance are in the same plane, the landmark apparatus 300 of this embodiment can be easily installed by installing it on the wall surface.

In addition, this invention is not limited to the said embodiment, The deformation | transformation in the range which can achieve the objective of this invention, improvement, etc. are included in this invention.
For example, the short distance landmark set and the long distance landmark set may not be on the same plane. If the respective mark coordinates are set and input to the mobile robot, position control using these is possible even if the landmark set for long distance and the landmark set for short distance are not on the same plane.
In addition, although the midpoint between the two marks of the short-range landmark set and the midpoint between the two marks of the long-range landmark set are on the same vertical line, If they are not within the same vertical line but within a close range, the position control using the long-range landmark set and the position control using the short-range landmark set can be continued without interruption. .
The mark of the landmark set may be colored with a predetermined color instead of the light emitter.
In the above description, the short distance landmark set and the long distance landmark set each have two marks. However, the number is not limited to two, and for example, three marks may be provided. If there are three marks each, the three-dimensional position and posture can be estimated.

The figure which shows the external appearance structure of the control system of a mobile robot. The figure which shows the structure of a landmark apparatus. The flowchart which shows the procedure of the position control method of a mobile robot. The flowchart which shows the procedure of a mark candidate extraction process. The flowchart which shows the procedure of a mark extraction process. The flowchart which shows the procedure of a mark determination process. The figure which shows the relationship of coordinate transformation in a self-position estimation process.

Explanation of symbols

DESCRIPTION OF SYMBOLS 100 ... Control system, 200 ... Mobile robot, 210 ... Main part, 220 ... Moving means, 230 ... Imaging means, 300 ... Landmark apparatus, 400 ... Landmark set for short distance, 500 ... Landmark set for long distance, 600 ... Wall, 700 ... Charging device, L _Hf ... Horizontal line, L _Hn ... Horizontal line, Lv ... Vertical line, Mf ... Mark of long-range landmark set, Mn ... Mark of landmark set for short distance, P _mf ... Long-distance Land mark set midpoint, P _mn ... Short distance landmark set midpoint, Ps ... self position.

Claims (5)

A landmark apparatus installed in a working environment of a mobile robot having an imaging means for imaging the surrounding environment and autonomously performing self-position estimation and movement direction control,
A short distance landmark set having at least two marks spaced apart by a predetermined distance;
A long distance landmark set having at least two marks spaced apart from each other by a distance larger than the short distance landmark set;
The midpoint between the two marks in the short-range landmark set and the midpoint between the two marks in the long-range landmark set were imaged by the imaging unit from a position away from the landmark by a predetermined distance. The landmark device is characterized in that it is included in the same imaging area. The landmark device according to claim 1,
The landmark apparatus, wherein the short distance landmark set and the long distance landmark set are arranged at different positions in the height direction. In the landmark apparatus according to claim 1 or 2,
The landmark device, wherein a midpoint between marks of the short distance landmark set and a midpoint between marks of the long distance landmark set are on the same vertical line. In the landmark apparatus in any one of Claims 1-3,
The landmark apparatus, wherein the short-range landmark set and the long-range landmark set are disposed in the same plane. The landmark device according to any one of claims 1 to 4,
A mobile robot that autonomously moves by performing position control based on image data obtained by imaging the landmark device.

Images