KR20130022918A - Cleaning robot navigationg method using the repeating image patterns of working environment - Google Patents

Abstract

PURPOSE: A self-control movement method of a cleaning robot using repetitive image patterns around a workplace is provided to predict location information using a sum vector in which an optical flow is accumulated in a video frame. CONSTITUTION: A self-control movement method of a cleaning robot using repetitive image patterns around a workplace comprises; a step of receiving repetitive image patterns by installing a camera to the direction of the repetitive image around a workplace at the center of the cleaning robot and a step of calculating the angle with respect to a Y axis from a standard starting point of the cleaning robot at the workplace and set of angles. [Reference numerals] (AA) Initializing a robot imaging device; (BB) Processing a ceiling image with repetitive patterns; (CC) Straight lines are extracted?; (DD) Calculating the direction of a robot; (EE) Calculating the direction using an optical flow; (FF) Crossing point of the straight lines is extracted?; (GG) Calculating the position of the robot; (HH) Calculating the position using an optical flow; (II) Using the position/direction

Description

Autonomous movement of cleaning robot using repeating pattern around robot workplace {Cleaning Robot Navigationg method using the Repeating Image Patterns of Working Environment}

The present invention relates to a method for moving a robot, and more particularly, to a method for autonomous movement of a robot using a repeating pattern around a robot workplace.

In order to calculate the position of a mobile robot, a lot of dead reckoning methods have been used in the past, based on the initial position of the robot. This method is simple, but there is a problem that an error caused by the sliding of the wheel is accumulated. In order to overcome this problem, many ultrasonic sensors have been used. This method uses ultrasonic waves to calculate the time-of-flight (TOF) of an object and measures the distance. It is not very accurate and is heavily influenced by the environment.

Recently, many ways of using landmarks have been studied. One method for this is to use RFID (Radio technology). After placing a large number of RFID tags in the work environment, the robot is equipped with an RFID reader to calculate the position of the robot by finding the location of the RFID tag. Since each RFID tag has its own location, it can be used to locate the robot, and another method is to use visual landmarks. For this purpose, a landmark made in a special shape is used, and an effective method of using the landmark is to attach the landmark to the ceiling. Color-based ceiling-attached lands developed to display and include a lot of code information in small landmarks In this method, each landmark represents a unique relative position, and the robot calculates the position by accumulating the relative position of the landmark. Although there is an advantage of increasing the utilization of space, there is a problem that it impairs the effort and aesthetics of attaching workplace landmarks.

  The present invention is to solve the above problems, the present invention is a new and effective way to find the direction and location of the robot using only a repeating pattern without attaching a landmark to the workplace. In the case of lattice repeating patterns, errors accumulated in position or direction information can be reduced by using straight lines or intersection points. In an image frame in which a repeating pattern is not detected, position information may be predicted by using a sum vector obtained by accumulating optical flow vectors, which are motion vectors between pixels, for each image frame. The present invention relates to techniques for achieving this goal.

To this end, the present invention, by installing the camera in the direction of the repeating pattern of the workplace in the center of the mobile robot to obtain a repeating pattern image; Calculating the angle set with respect to the y-axis at the reference origin of the robot workshop for the repeated linear components among the repeated patterns of the robot workshop to obtain and prepare the angle set; Prepare an intersection set consisting of positional information for all intersections in the repeating pattern; In order to calculate the direction during the autonomous movement of the robot, a straight line is detected from the image taken during the movement, the angle between the straight line and the current direction of the robot is obtained from the image, and then the respective straight lines based on the y axis of the workplace prepared in advance. In addition to each individual angle in the set of angles for, it is called the set of orientation angle candidates of the robot; Selecting an angle closest to the robot direction angle obtained from the robot position of the previous image among the angles in the direction angle candidate set to determine the current robot angle; In order to calculate the position during the robot autonomous movement, only the position coordinates of the intersections around the previous image position of the robot on the intersection set prepared at the initialization are called robot reference position candidate sets; Selecting the most distinct intersection point among the workplace images photographed at the current position to display pixel coordinates of the intersection point as a relative coordinate from the image center and display it as the transformed actual distance relative position vector described in claim 2; Adding the transformed actual distance relative position vector to the reference robot position candidate set and calling it a candidate set for the actual position of the robot; And among the elements of the candidate set of the actual position of the robot, determining the position point closest to the position of the robot obtained from the previous image as the current robot position point.

The present invention is a technology for recognizing the position and direction of the robot with only a repeating pattern, when the initial direction and position is given, the direction and position of the robot can be gradually calculated through the recognition of the characteristic of the repeating pattern, Errors can be corrected each time a feature of each repeating pattern is found and continue to move without accumulating. In particular, since the present invention can use the image of the workplace as it is, it is possible to accurately measure the position without additional landmarks or encoders, which are essential in existing products, and thus it is judged to be a technology having a high price competitiveness. Since most offices have a lattice pattern to which the present invention can be applied, the present invention can be said to be useful for an office robot.

1 is a diagram illustrating ceilings with repeating lattice patterns;
2 is a main flow diagram of the present invention for determining the direction and position of a robot using repeated repeating patterns;
3 is a flowchart of a routine for determining the direction of the robot using the detected straight line;
4 (a) is a diagram illustrating origin reference coordinates and robot coordinates for straight lines and intersections in a general repeating pattern;
4 (b) shows the origin reference coordinate system (xy) and the robot reference coordinate system (x'-y ') superimposed;
5 is a flowchart of a routine for calculating the position of the robot using the intersection of straight lines in the image;
FIG. 6 is an auxiliary picture used to calculate the actual position of the robot in the routine of FIG. 5; FIG.
Fig. 7 is a routine describing a method of converting into a real distance relative position vector according to claim 2;
8 illustrates a method of predicting a robot direction using an optical flow or cumulative vector between image frames in case of an image in which a repeating pattern is not found, and using a straight line when a straight line is found. A routine showing how to correct the direction of the robot predicted from the cumulative vector;
FIG. 9 illustrates a method of predicting a robot position using an optical flow or cumulative vector between image frames in an image in which a repeating pattern is not found, and using the pattern when a repeating pattern is found. This routine shows how to correct the robot position predicted from the cumulative vector.

In order to solve the above problems, the present invention intends to recognize the position and orientation of the robot by using only the image without ancillary devices such as landmarks by using a grid-like image feature as shown in FIG. 1. To do this, a camera is installed at the center of the mobile robot with a repeating pattern to obtain a continuous digital image of the workplace.

In the case of lattice repeating pattern, the direction is detected from the straight line of the image as shown in FIG. 2 and the position is recognized from the intersection point. If a straight line or an intersection point is not detected in the image, the direction and position of the robot are predicted using the motion vector between the image frames. In more detail, the direction of the robot in the present invention is calculated using the angle between the straight line and the robot detected from the image, the position of the robot is the center of the robot from the intersection and the number of repetitions from the origin of the workplace Calculate using vector up to. In an image frame in which a repeating pattern is not detected, a negative value of a sum vector accumulated for each image frame is used by adding a negative value of a sum vector accumulated in each image frame to a previous image direction or position. However, when straight lines or intersection points are detected in the image, errors are accumulated by correcting the direction and position information of the robot by the detected straight lines or intersection points information.

To this end, the camera is installed at the center of the mobile robot toward the repeat pattern to obtain a set of angles based on the origin of the robot's workplace for the repeated linear components among the repeat patterns, and the positions of all intersection points in the grid pattern. Prepare the assembly.

Robot direction recognition

The direction of the moving robot detects a straight line from the image as shown in FIG. The direction closest to the robot direction in the image is determined by the current robot angle.

In the case of a lattice repeating pattern, there are two straight lines that are parallel or intersect. As shown in FIG. 4 (a), the direction of the straight line parallel to the x axis is −π / 2 based on the origin of the real space, and the straight line parallel to the y axis defines the direction to be zero. In this way, the direction parallel to the -x axis is π / 2, and the direction parallel to the -y axis is π.

In the spatial coordinate system as shown in FIG. 4B, the directions of the straight lines have four directions such as {−π / 2, 0, π / 2, π}. In the video coordinate system, the upper direction of the image is defined as y 'as the front direction of the robot on the arrangement of the robot and the camera system, and the angle between the y axis and θ is assumed. If one straight line is found and θ, which is the angle to the front face of the linear robot, is obtained, the actual robot direction is one of {θ-π / 2, θ, θ + π / 2, θ + π}. First, a straight line closest to the vertical line connecting the upper and lower sides of the image is first obtained, and the angle formed by the vertical line and the straight line is called θ. However, the straight line obtained from the image should be distinguished from which line is θ in the 0 degree direction, π / 2, π, or -π / 2 based on the origin.

로 한다면, 현재의 방향은 {θ-π/2, θ, θ+π/2, θ+π} 중

에 가까운 값이다. If we assume that the rotational speed of the robot is low enough, that is, if the direction change of the robot is very small, it will be a direction similar to the direction of the most recently captured image of the current direction of the current robot. This direction

Current direction is {θ-π / 2, θ, θ + π / 2, θ + π}

Is close to.

를 현재의 방향이라고 하면,
for this,

If is the current direction,

[Equation 1]

It will be.

By the way, it is not necessary to calculate the initial direction very accurately, and if only the rough direction is known, it is only necessary to determine which angle is close to {θ-π / 2, θ, θ + π / 2, θ + π}. It is because it becomes. Therefore, there is a big advantage that this error does not accumulate. The next section calculates the range of errors allowed.

 Robot position recognition

In order to calculate the position of the robot, as shown in FIG. 5, the position coordinates of the intersections around the robot are taken from the previous position of the robot. In addition, by selecting the most distinct intersection point in the current image, the position of the image center is expressed as a relative vector with respect to the intersection point, and the rotation is converted into the coordinates of the reference origin, and then the position closest to the robot position in the previous image is displayed. Determined by location.

 In the case of the lattice repeating pattern, as shown in FIG. 4, the target lattice repeating pattern has many intersection points formed by two straight lines crossing each other. Let d be the distance between two adjacent intersections of a grid pattern. Thus, the positions of these intersections are known and, if the coordinate axis is defined, the positions of the intersections can be calculated.

If we calculate the distance from this intersection to the center of the image, we can also calculate the current position of the robot.

라고 할 때, 현재 영상 프레임에서의 교차점의 실제 위치는 다음과 같은 점 중의 하나이다.
The actual position of the intersection detected in the image of the previous frame.

In this case, the actual position of the intersection point in the current video frame is one of the following points.

&Quot; (2) "

Where and are the distances between the x and y axes.

의 방향은 로봇의 방향과 같다. The position of the robot can be calculated using FIG. 6 after calculating the position of the intersection using the above method. That is, since the position of the robot is the same as the position of the center point of the image, it is called C. If the position point of the detected intersection point is A and B as the intersection point of the adjacent position, the vector

The direction of is the same as that of the robot.

로 계산되어 있으므로, 각

를 계산할 수 있다. 만약, 영상 평면과 반복 무늬의 크기비율 관계를 이용하면, 실제위치점 A로부터

점의 위치는 [수학식 3]과 같이 계산 할 수 있다.
The current robot direction is

Is calculated as

Can be calculated. If the relationship between the image plane and the size of the repeating pattern is used,

The position of the point can be calculated as shown in [Equation 3].

&Quot; (3) "

은 9개의 후보 점을 가질 수 있으므로,

a점 역시 9 점을 갖는다. 이와 같은 변환의 절차는 도 7의 실제거리 상대위치 벡터로 변환방법에서 을 기술하고 있다. point

Can have 9 candidate points,

point a also has 9 points. This conversion procedure is described in the conversion method to the actual distance relative position vector of FIG.

However, since the speed of the robot is not fast, the point is located close to the point. So the current position of the robot

[Equation 4]

Is calculated as

[Equation 5]

와

는 {-1, 0, 1} 중에서 [수학식 5] 식이 최소값을 갖는 값이다. [수학식 2]에 의하면, 현재의 화면상에 나타나는 교차점

의 위치도 구할 수 있다. As a result, the position point of the robot can be obtained.

Wow

Is a value in which Equation 5 has a minimum value among {-1, 0, 1}. According to [Equation 2], the intersection point appearing on the current screen

The position of can also be obtained.

과 이전 프레임에서의 로봇 위치점

를 알면, 로봇의 현재 위치와 현 영상에 나타나는 교차점의 위치는 [수학식 5] 및 [수학식 2]를 이용하면 계산할 수 있다. 따라서, 로봇의 초기 위치가 주어지면, [수학식 2]에 의해, 로봇의 위치를 점차적으로 계산할 수 있는 것이며, [수학식 5]를 이용하므로,

점에 대한 에러가 누적되지 않는다는 중요한 특징이 있다.
The intersection that we know in conclusion

And robot position points in the previous frame

The current position of the robot and the position of the intersection point appearing on the current image can be calculated by using Equations 5 and 2 below. Therefore, given the initial position of the robot, it is possible to gradually calculate the position of the robot by [Equation 2], because [Equation 5] is used,

An important feature is that errors for points are not cumulative.

Optical Flow The robot's direction and position

In the case of an image in which a repeating pattern is not detected, a motion vector between pixels is calculated as shown in FIG. 8 or 9, and when the image without a straight line or an intersection point is continued, a sum vector between the motion vectors is obtained. The robot predicts the direction and position of the robot. If the image is obtained by detecting a straight line or intersection point, the robot predicts the direction and position of the robot.

는
Optical flow is a motion vector of pixels between adjacent image frames, and when the robot moves, the pixels move in the opposite direction to the robot. In this case, since the motion vectors of the pixels of many images may be slightly different, a representative motion vector between image frames is used. Let be a representative motion vector between image m and n frames. When the optical flow of k image frames is accumulated from the m th frame, the accumulated optical flow,

The

&Quot; (6) "

=

+

++...+

=

+

++ ... +

Can be calculated as

이었다면, k프레임 후의 각도는 는
Therefore, the direction angle of the robot at time m is based on the origin.

If, the angle after k frames is

[Equation 7]

=

-

=

-

It is obtained by adding negative values as follows.

If an image with a straight line is obtained from the image, the angle obtained from the straight line is obtained by adding the angle obtained from the straight line to the individual linear angles in the directional angle candidate set as in the method of obtaining the robot direction based on the straight line. The direction of the plurality of robot direction candidates having the smallest difference from the robot direction determined by the sum of the motion vectors is determined as the current robot direction.

In addition, when the intersection point does not exist in the image, as shown in FIG. 9, the sum of the motion vectors is obtained by accumulating the vector of the optical flow as shown in Equation 6, and the negative value of the sum of the motion vector is obtained at the position of the robot obtained from the image having the intersection point. In addition, the robot predicted position is obtained.

If the intersection point (s) is detected in the image, the candidate set for the actual position of the robot is obtained as in the robot position recognition method, and the position point closest to the robot predicted position is determined from the candidate set for the actual position of the robot. Correct it to the robot position.

Claims (5)

Installing a camera in the direction of the repeating pattern of the workplace at the center of the mobile robot to obtain a repeating pattern image;
Calculating the angle set with respect to the y-axis at the reference origin of the robot workshop for the repeated linear components among the repeated patterns of the robot workshop to obtain and prepare the angle set;
Prepare an intersection set consisting of positional information for all intersections in the repeating pattern;
In order to calculate the direction during the autonomous movement of the robot, a straight line is detected from the image taken during the movement, the angle between the straight line and the current direction of the robot is obtained from the image, and then the respective straight lines based on the y axis of the workplace prepared in advance. In addition to each individual angle in the set of angles for, it is called the set of orientation angle candidates of the robot;
Selecting an angle closest to the robot direction angle obtained from the robot position of the previous image among the angles in the direction angle candidate set to determine the current robot angle;
In order to calculate the position during the robot autonomous movement, only the position coordinates of the intersections around the previous image position of the robot on the intersection set prepared at the initialization are called robot reference position candidate sets;
Selecting the most distinct intersection point among the workplace images photographed at the current position to display pixel coordinates of the intersection point as a relative coordinate from the image center and display it as the transformed actual distance relative position vector described in claim 2;
Adding the transformed actual distance relative position vector to the reference robot position candidate set and calling it a candidate set for the actual position of the robot;
Method of calculating the direction and position of the robot using a repeating pattern of the workplace image among the elements of the candidate set of the actual position of the robot to determine the position point closest to the position of the robot obtained from the previous image as the current robot position point. The method according to claim 1,
The transformed relative position vector is
Find a relative vector from the image center for the selected intersection;
Converting the intersection by rotating the vector in the reverse direction with respect to the center of the image by the direction of the robot obtained in claim 1;
Multiplying the rotation-converted intersection vector by the actual distance with respect to the unit pixel spacing to convert the position of the intersection into the actual distance relative vector from the center of the robot;
Calculate the direction and position of the robot using a repeating pattern of the workplace image, wherein the robot center is displayed as the actual distance relative vector to the intersection by multiplying a negative sign by the actual distance relative vector to the robot center of the intersection. Way.
The method according to claim 1,
In the method of obtaining the robot direction, if the repetitive pattern is not found in the image and it is difficult to calculate the direction of the robot, the vector of the flow vector sum is accumulated by accumulating the vector of the optical flow until the repetitive pattern feature is found in the image. In addition, it predicts and uses in the direction of the robot;
When the image having a straight line is obtained, the angle of the robot is obtained from the straight line, and the direction angle candidate set is obtained as in the method of claim 1, and the difference between the robot direction determined by the cumulative sum of the individual angles and the motion vector in the set is most different. A method for calculating the direction and position of a robot using a repeating pattern of a workplace image, further comprising a method for supplementing a direction determination of a robot using an optical flow for determining a smaller direction as the current robot direction.
The method according to claim 1 or 2,
If there is no feature of the repeating pattern in the image, the sum of the motion vectors is obtained by accumulating the vector of the optical flow until the next image where the repeating pattern is found, and moving to the previous position of the robot obtained from the image with the repeating pattern. Obtain the robot prediction position by adding the negative values of the sum of
If a repeating pattern is detected in the image, it is the same as the method of calculating the robot position in claim 1 or claim 2, but the candidate for the enlarged robot actual position is expanded by using a 2 to 3 times enlargement of the range of the reference robot position candidate set. Obtain a set;
And a robot position point calculation complementary method using an optical flow for correcting a position point closest to the robot predicted position from the candidate set for the actual position of the robot to the current robot position. Method of calculating the direction and position of the robot using patterns.
The method according to claim 1 or 4,
It is not limited to a gap pattern consisting of only straight lines, but applies to general or repeating patterns composed of curves or circles.In this case, the size and direction of the pattern are measured in advance to utilize the information to adjust the position and direction of the robot. Method for calculating the direction and position of the robot using a repeating pattern of the workplace image, characterized in that the calculation.

Images