KR20120078339A - Image-based simultaneous localization and mapping for moving robot - Google Patents

Abstract

PURPOSE: An image-based simultaneous localization and mapping for moving robot are provided to simplify data handling by expressing a vertical straight line composing three dimensions and a floor straight line in a two dimensional plane. CONSTITUTION: A first candidate straight line is determined from extracted lines of photographing images. A corresponding estimated straight line expected in the next photographing images according to the movement of a robot is calculated. A second candidate straight line is determined from the photographing images after the robots moved. The identity of the estimated straight line and the second candidate straight line is determined. A map is made based on the second candidate straight line admitted to have an identity with an estimated straight line.

Description

Image-based simultaneous location recognition and mapping of mobile robots {IMAGE-BASED SIMULTANEOUS LOCALIZATION AND MAPPING FOR MOVING ROBOT}

The present invention relates to a simultaneous location recognition and mapping method of a robot (SLAM: Simultaneous Localization And Mapping), and more particularly, location recognition by extracting a linear component from an image input through a camera and using it as feature data And how to do mapping at the same time.

Simultaneous location recognition and mapping (hereinafter referred to as "SLAM") is one of the most basic functions for robots requiring autonomous driving. When the robot enters the first encounter environment, it is necessary to understand the environment by creating a map of the environment. In order to do this, it is necessary to be able to recognize the current position of the robot on the map in order to create an accurate map. have.

Depending on the type of sensor used in the robot, the characteristic data used in the SLAM algorithm is also different. For example, feature information may include distance information obtained through a distance sensor such as an ultrasonic wave or a laser, coordinate information obtained directly from a GPS device, acceleration information obtained through an IMU device, and vision information using a stereo camera or a mono camera. Can be.

Mono cameras offer the advantages of lower cost and smaller volume than other types of sensors. In addition, since the vision information obtained from the camera can be used to extract other useful information such as object identification and human facial recognition, in addition to SLAM, it is more useful than other sensors and effective in providing various services.

SLAM using vision information (hereinafter, referred to as "vSLAM") has been mainly studied how to use feature points as feature data. While feature points have the advantage of being easy to process with vision algorithms and applicable to various environments, the number of required feature points must be large and the created maps are difficult to expand further.

On the other hand, there is a problem in that uncertainty increases when a linear component is used as feature data, and research is relatively insufficient. In other words, there is an evaluation that the linear component is suitable for an environment with many artificial structures, but there is an advantage that the number of maps required for constructing the map is small, and that the prepared map is easy to add semantic labels. Existing researches for applying linear components to SLAMs have a drawback to increase the uncertainty of feature data by approaching general modeling methods.

Accordingly, an object of the present invention is to improve a vSLAM algorithm using an existing linear component, and to provide a SLAM having simple data processing and reduced uncertainty.

According to one aspect of the present invention, in the image-based simultaneous position recognition and mapping method of a mobile robot, a first line estimated from a line extracted from a photographed image is a line located on the floor on which the mobile robot travels. Determining a candidate straight line; Calculating a corresponding estimated straight line of the candidate straight line predicted in a next captured image according to the movement of the mobile robot; Determining a second candidate straight line estimated as a line located on a floor on which the mobile robot travels among the images photographed after the movement of the mobile robot; Determining the identity of the estimated straight line and the second candidate straight line; And generating a map by using the second candidate straight line whose identity is recognized as the estimated straight line as a result of the determination.

The determining of the identity between the estimated straight line and the second candidate straight line may include determining whether the difference between the coordinates specifying the estimated straight line and the coordinates specifying the second candidate straight line is within a predetermined threshold range. It may include the step of determining. The coordinates can be specified to be obtained from a distance from an arbitrary point in the image to a straight line and an angle formed by the horizontal straight line and the straight line.

The present invention may further include extracting a vertical straight line perpendicular to the floor from the photographed image, wherein the first line is estimated to be a line located on the floor on which the mobile robot travels from the lines extracted from the photographed image. The determining of the candidate straight line may be selected to include determining a first candidate straight line among straight lines formed at a position lower than a midpoint of the vertical straight line.

The method may further include extracting a vertical straight line perpendicular to the floor from the photographed image, wherein the vertical straight line is formed on a map created by using the second candidate straight line recognized as the estimated straight line. By adding a point projected to the floor to include a three-dimensional map information to represent a two-dimensional map, it is possible to simplify the data processing and reduce the uncertainty.

In the series of the present invention, calculating the estimated straight line may determine the estimated straight line by the following equation.

Where I is the first candidate straight line, I 'is the estimated straight line, H is the homography matrix, and is defined as follows:

은 카메라의 각도 변화를 나타내는 회전 행렬, 0 는 3×1의 영 행렬,

는 카메라의 위치 변화를 나타내는 행렬 , h는 이미지 촬영에 사용된 카메라의 높이를 나타낸다.
Where K 'is the camera property matrix after the movement of the mobile robot,

Is the rotation matrix representing the angle change of the camera, 0 is the zero matrix of 3 × 1,

Is a matrix representing the change in the position of the camera, and h is the height of the camera used to capture the image.

According to the present invention in which a SLAM is performed using a linear component as feature data, there is an advantage of better representation of the surrounding environment than a map created as a feature point.

In addition, the uncertainty can be greatly improved by determining the observation model for the floor straight line by the novel method provided by the present invention. Data processing can be simplified.

1 is a flowchart illustrating a method for image-based simultaneous location recognition and mapping of a mobile robot according to an embodiment of the present invention.
2 is a conceptual diagram for explaining a linear transformation relationship by a homography matrix;
3 is a conceptual diagram for explaining a spherical coordinate system for specifying a straight line on an image;
4 is a conceptual diagram for explaining an observation model for the determined floor straight line;
5 is a conceptual diagram for explaining the concept of a projection point of a vertical straight line;
6 is a conceptual diagram for explaining an observation model for the determined projection point;
7 is an example of the final map completed by applying the observation model of the floor straight line and the vertical straight line to the EFK algorithm; And
8 is a diagram for comparing a map created by a feature point with a map created by a straight line.

Since the present invention relates to a SLAM algorithm of a mobile robot, a detailed description of the hardware part of the mobile robot is omitted, and an embodiment of the present invention is assumed to describe a hardware structure of a mobile robot that is movable and is equipped with a mono camera. do.

Hereinafter, embodiments of the present invention will be described in detail with reference to the accompanying drawings.

1 is a flowchart illustrating a method of image-based simultaneous location recognition and mapping of a mobile robot according to an exemplary embodiment of the present invention.

First, the mobile robot captures a photographed image by using a mono camera, and extracts linear components from the photographed image. Well-known Canny edge detection and Hough transform are utilized as a method of extracting a straight component from an image.

Next, among the extracted linear components, a first candidate straight line estimated as a line located on the floor on which the mobile robot travels is determined.

At this time, based on the rotation information of the mono camera and the directionality of the straight lines can be distinguished whether the first straight line is located on the floor, and the straight line higher than the middle point of the vertical straight component is subtracted from the candidate of the bottom line, etc. It is also possible to determine the first candidate straight line with various direct and indirect criteria. When the camera is horizontal, it can also be first-order estimated as the bottom straight line if the difference between the vertical coordinates of the two endpoints of the extracted straight lines represents a pixel difference within the acceptable range.

Next, when the mobile robot has a motion, the camera photographs again with a mono camera to obtain an image after the motion. Here, the movement of the mobile robot includes individual movements such as driving, rotation, and rotation of the camera, or simultaneous movements of the mobile robot. In either case, the movement or control information of the robot may be accumulated and utilized.

Next, a straight line component on the new image corresponding to the first candidate straight line due to the motion is estimated. Here, the front, rear, left, and right movements of the robot, the amount of rotation of the camera, the change of camera properties, the amount of rotation of the robot, and the like may be used as a basis for predicting and estimating the first candidate straight line.

It is possible to use the following formula as an estimation method:

[Equation 1]

Where I is the first candidate straight line, I 'is the estimated straight line, H is the homography matrix, and is defined as follows:

[Equation 2]

은 카메라의 각도 변화를 나타내는 회전 행렬, 0 는 3×1의 영 행렬,

는 카메라의 위치 변화를 나타내는 행렬, h는 이미지 촬영에 사용된 카메라의 높이를 나타낸다.Where K 'is the camera property matrix after the movement of the mobile robot,

Is the rotation matrix representing the angle change of the camera, 0 is the zero matrix of 3 × 1,

Is a matrix representing a change in the position of the camera, and h is the height of the camera used to capture an image.

Equation 1 can be easily understood from the conceptual diagram shown in FIG.

That is, in the case where the camera of the mobile robot moves from the C position to C ', the projection straight line l of the bottom straight line L is on the image π obtained at the C position, and the image obtained at the C' position ( It is determined by the homography matrix (H) how it will appear on π '), ie the transformation of straight lines on the two image planes when moving the camera from C to C'.

은 도2에서 개시된 카메라의 위치 변화(C->C')에 따른 각도 변화를 나타내는 것으로서, 따라서 로봇의 회전에 따른 카메라 촬영 방향의 각도 변화를 포함하는 것이다. 또한,

는 로봇의 이동을 포함한 카메라의 위치 이동을 반영하는 것으로서 역시 카메라의 위치 변화(C->C')를 반영하기 위한 것이다.

2 represents an angle change according to the position change (C-> C ′) of the camera disclosed in FIG. 2, and thus includes an angle change in the camera photographing direction according to the rotation of the robot. Also,

Is to reflect the positional movement of the camera including the movement of the robot, and also to reflect the change in position of the camera (C-> C ').

The inventor newly proposes a homography matrix such as Equation 2 to provide a method for calculating an accurate estimated straight line according to the present invention.

Now, as in the case where the first candidate straight line is determined, a second candidate straight line estimated as the bottom line is determined from the straight line components extracted from the image captured after the movement.

The sameness of the calculated estimated straight line and the second candidate straight line is determined. In the case where there are a large number of estimated straight lines and second candidate straight lines, equality judgment is performed separately.

In this case, the determination of the identity of the straight lines may be based on a coordinate comparison specifying a straight line on the image. For example, referring to Fig. 3, the left corner of the image is referred to as the origin, and the coordinates are specified by the shortest distance d from the origin to the straight line and the angle (θ) between the horizontal direction and the straight line of the image. Can do it.

Now, in order to determine the identity between the straight lines, weights α and β are given to the difference between the distance d and the angle θ to calculate the magnitude of the estimated error ε.

[Equation 3]

ε = α (d-d ') + β (θ-θ') <Ζ

When the calculated error is smaller than the predetermined threshold value Z, it is determined that there is an identity between the estimated straight line and the second candidate straight line.

Now, the second candidate straight line whose identity is recognized as the estimated straight line is determined as the bottom straight line, and thus the observation model for the bottom straight line is determined. That is, as shown in Figure 4, it is possible to define the change in the coordinate (x, y) of the mobile robot and the distance change between the floor straight line (L) and the angle change (θ _k ) to the driving direction, etc. .

Once the observation model for the floor line is in place, the map is created according to the usual procedures using standard EFK algorithms.

Embodiments of the present invention may further comprise a map comprising a vertical straight line perpendicular to the floor.

As shown in Fig. 5, the vertical straight line may be expressed as one point when projected to the floor plane. Since the vertical line extracted from the image from the moto camera has only angular information and no distance information, it can be modeled by Inverse Depth Parameterization (IDP) method, and an observation model for the projection point as shown in FIG. 6 can be prepared.

The final map obtained using the vertical straight line and the bottom straight line by applying the observation model of the floor straight line and the vertical straight line thus prepared to the standard Extended Kalman Filter (EFK) is shown in FIG. 7.

The three-dimensional map of FIG. 7 can be substantially created with two-dimensional information of the floor straight line and the projection point, thus providing a significant advantage in data processing such as easy data representation and simple operation.

As can be seen from FIG. 8, it can be seen that the maps by the floor straight line and the vertical straight line can better express the surrounding environment as compared with the map created by the feature points.

Although the embodiments of the present invention have been described so far, those skilled in the art will understand that variations of the present invention are possible without departing from the spirit of the present invention.

Therefore, the embodiments of the present invention described above should be understood as illustrative, and the protection scope of the present invention should be recognized to the equivalent range based on the technical spirit of the present invention described in the claims below.

Claims (6)

In the image-based simultaneous location recognition and mapping method of a mobile robot,
Determining a first candidate straight line estimated from a line extracted from a photographed image as a line located on a floor on which the mobile robot travels;
Calculating a corresponding estimated straight line of the first candidate straight line predicted in a next captured image according to the movement of the mobile robot;
Determining a second candidate straight line estimated as a line located on a floor on which the mobile robot travels among the images photographed after the movement of the mobile robot;
Determining the identity of the estimated straight line and the second candidate straight line; And
And generating a map by utilizing the second candidate straight line whose determination result is the same as the estimated straight line.
The method of claim 1,
Determining the identity of the estimated straight line and the second candidate straight line,
And determining the identity according to whether a difference between the coordinates specifying the estimated straight line and the coordinates specifying the second candidate straight line is within a predetermined threshold range.
The method of claim 2,
And the coordinates are obtained at a distance from an arbitrary point in the image to a straight line and at an angle formed by a horizontal straight line and the straight line.
The method of claim 1,
Extracting a vertical straight line perpendicular to the floor from the photographed image;
Determining a first candidate straight line estimated from a line extracted from a photographed image as a line located on a floor on which the mobile robot travels, determines a first candidate straight line among straight lines formed at a position lower than a middle point of the vertical straight line. Simultaneous location recognition and mapping method comprising the steps of.
The method of claim 1,
Extracting a vertical straight line perpendicular to the floor from the photographed image;
And adding a point at which the vertical straight line is projected on the floor to a map created by using the second candidate straight line that has been identified to be equal to the estimated straight line so as to express three-dimensional map information in a two-dimensional map. Simultaneous location recognition and mapping method.
The method according to any one of claims 1 to 5,
The calculating of the estimated straight line may include determining the estimated straight line by the following equation.

Where I is the first candidate straight line, I 'is the estimated straight line, and H is a homography matrix, defined as:

Where K 'is the camera property matrix after the movement of the mobile robot,

Is the rotation matrix representing the angle change of the camera, 0 is the zero matrix of 3 × 1,

Is a matrix representing the change in position of the camera, and h is the height of the camera used to capture the image.

Images