KR20110010422A - Computation of robot location and orientation using repeating feature of ceiling textures and optical flow vectors - Google Patents

Abstract

PURPOSE: A method for computing location and orientation of a robot by using repeated patterns of the ceiling and optical flow vectors is provided to reduce error accumulated on the location or direction information using the straight line or intersecting points of lattice-shaped repeated patterns. CONSTITUTION: A method for computing location and orientation of a robot by using repeated patterns of the ceiling and optical flow vectors comprises following steps. If initial direction and location are set, the location and direction of the robot is gradually calculated through the recognition of straight lines and latticed points. Every time the robot finds out each intersecting point of patterns of the ceiling, small errors are corrected and not accumulated, so the robot can continuously move.

Description

BACKGROUND OF THE INVENTION Field of the Invention [0001] The present invention relates to a method and apparatus for calculating a position and a direction of a robot using a repeating pattern of a ceiling and a pixel motion vector,

In the case of an image frame in which no straight line or intersection point is detected, Optical Flow Vectors, which are motion vectors between pixels, are displayed on the image The position information can be predicted by utilizing the value of the sum vector accumulated for each frame. The present invention relates to an effective method of finding out the position and direction of a robot by utilizing only a repeating pattern of a ceiling.

In order to calculate the position of the mobile robot, a dead reckoning method has been used in the past to calculate the position based on the initial position of the robot. Although this method is simple, there has been a problem that errors caused by sliding of the wheels are accumulated. In order to overcome this problem, many ultrasonic sensors have been used. This method is a method of measuring the distance by calculating the time-of-flight (TOF) of the ultrasound to the object, which is not very accurate and is highly influenced by the environment.

Recently, many methods of using landmarks have been studied. One of the methods for this is to use RFID technology to locate the robot by locating RFID tags in the work environment and then locating the RFID tags while the robot equipped with the RFID reader moves The RFID tag can be used to locate the robot because each RFID tag has its own position, and another method is to use a visual landmark. A landmark is a method of attaching a landmark to a ceiling, and an effective method of using the landmark is to attach the landmark to the ceiling. Color-based ceiling land developed to include a large amount of code information in small-sized landmarks. In this method, the robot calculates the position by accumulating the relative positions of the landmarks, allowing each landmark to express its own relative position. This method involves attaching a landmark to an unused ceiling There is an advantage to increase the utilization of space, but there is a problem that the efforts to attach the landmark to the ceiling and the aesthetics are damaged.

SUMMARY OF THE INVENTION The present invention has been made to solve the above-mentioned problems, and it is a new and effective method for determining the direction and position of a robot by utilizing only ceiling repeated patterns without attaching landmarks to the ceiling. In the case of an image frame in which a straight line or an intersection point is not detected, Optical Flow Vectors, which are motion vectors between pixels, are used for each image frame The location information can be predicted by utilizing the accumulated sum vector. The present invention relates to a technique for achieving such a target.

In order to solve the above-described problems, the present invention aims at recognizing the position and direction of a robot by using only a video image without a side device such as a landmark by using the feature of a ceiling with a grid pattern as shown in Fig. To do this, a camera is installed facing the ceiling at the center of the mobile robot to obtain a continuous digital image of the ceiling.

The configuration of the present invention detects the direction from the straight line of the image and recognizes the position from the intersection as shown in Fig. If a straight line or an intersection is not detected in the image, the direction and position of the robot are predicted using the motion vector between the image frames. More specifically, the direction of the robot in the present invention is calculated by using the angle between the straight line detected from the image and the robot, and the position of the robot is determined based on the number of repetitions from the origin point of the work site and the intersection point, Calculate using the vector up to the center. In an image frame in which no straight line or intersection point is detected, the negative value of the sum vector of the accumulated optical flow vectors, which are motion vectors between pixels, for each image frame is added to the previous image direction or position. Then, when straight lines or intersections are detected in the image, correcting the direction and position information of the robot by the detected straight line or crossing point information avoids accumulation of errors.

For this purpose, a camera is installed toward the ceiling at the center of the mobile robot, and an angle set is prepared based on the origin of the robot work site for the repeated linear components of the grid-like ceiling pattern, Prepare a location set.

Direction recognition of robot

As shown in FIG. 3, the direction of the moving robot is such that an angle between the straight line and the current direction of the robot is calculated and then added to the individual angles of the reference straight lines of the workplace prepared in advance, The direction closest to the direction of the robot in the image is determined as the current robot angle.

이고, y 축에 평행한 직선이 방향을 0로 정의한다. 이렇게 하면, -x 축에 평행한 방향은

/2이고, -y축에 평행한 방향은

가 되는 것이다. There are two parallel straight lines intersecting the ceiling image of the grid pattern. As shown in FIG. 4 (a), the direction of a straight line parallel to the x-axis with respect to the origin of the actual space is represented by -

, And a straight line parallel to the y-axis defines the direction as zero. In this way, the direction parallel to the -x axis is

/ 2, and the direction parallel to the -y axis is

.

와 같은 4 개의 방향을 갖게 된다. 또, 영상 좌표 계에서는 로봇과 카메라 시스템의 배치상, 영상의 윗 쪽 방향을 로봇의 전면 방향으로서, y'로 정의하고, y 축과의 사이 각을

라 하자. 한 개의 직선을 구해서 그 직선 로봇의 앞면과의 각도인

를 구했다고 하면 실제의 로봇의 방향은

중의 하나가 된다.In the spatial coordinate system shown in Fig. 4 (b), the direction of each straight line is

As shown in FIG. In the image coordinate system, the position of the robot and the camera system and the direction of the upper side of the image are defined as y 'as the front direction of the robot, and the angle between the y-

Let's say. One straight line is obtained and the angle between the straight line and the front surface of the linear robot

, The direction of the actual robot is

.

라고 한다. 그런데, 영상에서 구한 직선은 원점을 기준으로 0도 방향,

,

혹은 -

중 어느 직선과

인지를 구별해야 한다. In other words, a straight line closest to the vertical line connecting the top and bottom of the image is first obtained, and the angle formed by the vertical line and the straight line is

. However, the straight line obtained from the image is oriented at 0 degree with respect to the origin,

,

or -

Which of the straight lines

It should be distinguished.

로 한다면, 현재의 방향은

중

에 가까운 값이다. Assuming that the rotation speed of the robot is sufficiently low, that is, if the change in the direction of the robot is very small, the direction will be similar to the direction of the most recent captured image in the current direction of the current robot. This direction

, The current direction is

medium

.

를 현재의 방향이라고 하면,for this,

Is the current direction,

Equation 1

.

중 어느 각도에 가까운지 결정만 하면 되는 것이기 때문이다. 따라서, 이 에러는 누적이 되지 않는다는 큰 장점이 있다. 다음 절에서는 허용되는 에러의 범위를 계산한다. However, it is not necessary to calculate the initial direction very accurately, and if only the rough direction is known,

It is only necessary to determine which angle is close to the angle. Therefore, there is a great advantage that this error does not accumulate. The next section calculates the range of allowed errors.

 Robot position recognition

In order to calculate the position of the robot, as shown in FIG. 5, the position coordinates of the intersections around the previous position point of the robot are taken among the positions of the intersections of the workplace. The position of the center of the image is represented as a relative vector to the intersection by selecting the most distinct intersection point in the current image, and the rotation is converted into the coordinates of the reference origin. Then, Position.

As shown in Fig. 4, there are many intersections formed by intersection of two straight lines in the target ceiling pattern. Let d be the distance between two adjacent intersections of the ceiling pattern. Thus, the positions of these intersections can be known and, if a coordinate axis is defined, the positions of the intersections can be calculated.

If the distance from this intersection to the center of the image is calculated, the current position of the robot can also be calculated.

라고 할 때, 현재 영상 프레임에서의 교차점의 실제 위치는 다음과 같은 점 중의 하나이다.The actual position of the intersection recognized in the image of the previous frame is

, The actual position of the intersection point in the current image frame is one of the following points.

Equation 2

.

.

과

는 x 축과 y축 간의 거리이다. here

and

Is the distance between the x and y axes.

의 방향은 로봇의 방향과 같다. The position of the robot can be obtained by calculating the position of the intersection using the method as described above and referring to FIG. That is, since the position of the robot is the same as the position of the center point of the image, this is C, and if the detected point of intersection is A and B is the intersection point of the adjacent position,

The direction of the robot is the same as that of the robot.

로 계산되어 있으므로, 각

=

.를 계산할 수 있다. 만약, 영상 평면과 천정평면의 크기비율 관계를 이용하면, 실제위치점 A로부터 C =

점의 위치는 수학식 3과 같이 계산 할 수 있다. The current robot direction

Respectively,

=

Can be calculated. If the magnitude ratio relationship between the image plane and the ceiling plane is used,

The position of the point can be calculated as shown in Equation (3).

Equation 3

은 9개의 후보 점을 가질 수 있으므로,

a점 역시 9 점을 갖는다. 이와 같은 변환의 절차는 도 7의 실제거리 상대위치 벡터로 변환방법에서 을 기술하고 있다. point

Can have nine candidate points,

The a point also has 9 points. Such a conversion procedure is described in the conversion method to the actual distance relative position vector in FIG.

점은

점에 가까이 위치하게 된다. 따라서 로봇의 현재 위치 점은However, since the speed of the robot is not fast,

The point is

It is located close to the point. Therefore, the current position point of the robot is

Equation 4

Lt; / RTI >

Equation 5

와

는 {-1, 0, 1} 중에서 수학식 5식이 최소 값을 갖는 값이다. 수학식 2에 의하면, 현재의 화면상에 나타나는 교차점

의 위치도 구할 수 있다. The position point of the robot can be obtained.

Wow

Is a value having the minimum value of the expression (5) in {-1, 0, 1}. According to Equation (2), the intersection point

Can also be found.

과 이전 프레임에서의 로봇 위치점

를 알면, 로봇의 현재 위치과 현 영상에 나타나는 교차점의 위치는 수학식 5 및 수학식 2를 이용하면 계산할 수 있다. 따라서, 로봇의 초기 위치가 주어지면, 수학식 2에 의해, 로봇의 위치를 점차적으로 계산할 수 있는 것이며, 식 수학식 5를 이용하므로,

점에 대한 에러가 누적되지 않는다는 중요한 특징이 있다. In conclusion,

And the robot position point in the previous frame

The current position of the robot and the position of the intersection point appearing in the current image can be calculated using Equation (5) and Equation (2). Therefore, given the initial position of the robot, the position of the robot can be gradually calculated using Equation (2), and since Equation (5) is used,

There is an important feature that errors for points are not accumulated.

Complementing robot direction and position using vector of optical flow

In the case of an image in which no straight line or intersection points are detected, an optical flow between pixels is calculated as shown in FIG. 8 or FIG. 9. If a straight line or an image without an intersection continues, a sum vector between the motion vectors is calculated The direction and the position of the robot are predicted and used. When the robot obtains an image in which a straight line or an intersection is detected, the predicted direction and position of the used robot are predicted.

을 영상 m과 n프레임 간의 대표 이동벡터라고 하자. m번째 프레임으로부터 k개의 영상프레임의 Optical Flow가 누적되는 경우, 누적된 optical flow,

는 When the robot moves as a motion vector of a pixel between adjacent image frames, the optical flow moves in a direction opposite to the robot. At this time, since the motion vectors of pixels of many images may be slightly different, a typical motion vector between image frames is used.

Let be a representative motion vector between m and n frames. When the optical flows of k image frames from the m-th frame are accumulated, accumulated optical flow,

The

Equation 6

=

+

+++...+

=

=

+

+++ ... +

.

이었다면, k프레임 후의 각도는

는 Therefore, when the direction angle of the robot at the time point m is based on the origin

, Then the angle after k frames is

The

Equation 7

=

-

=

-

As shown in Fig.

If an image having a straight line is obtained from the image, the angle of the robot is obtained on the basis of the straight line, and the angle obtained from the straight line is added to the individual straight line angles in the direction angle candidate set, The direction of the plurality of robot direction candidates having the smallest difference from the robot direction determined by the sum of the motion vectors is determined as the current robot direction.

If there is no intersection in the image, the sum of the motion vectors is obtained by accumulating the vectors of the optical flow as shown in Equation (6) as shown in FIG. 9, and the sum of the motion vectors is added to the position of the robot obtained by zero- And the robot predicted position is obtained.

If the intersection point (s) is detected in the image, a candidate set for the robot actual position is obtained as in the robot position recognition method, and a position point closest to the robot predicted position from the candidate set for the robot actual position is determined as the current Correct the robot position.

The present invention is a technique for recognizing the position and direction of a robot even if the grid pattern shown in FIG. 1 is a ceiling pattern only. If the initial direction and position are given, the direction and position of the robot are gradually , And small errors occurring at this time are corrected each time the intersection of each ceiling pattern is found, and the movement can be continued without accumulation. Particularly, since the present invention utilizes the pattern of the ceiling as it is, it is possible to measure an accurate position without using an additional device such as a landmark or an encoder, which is essential in existing products. Since most offices have a lattice pattern to which the present invention can be applied, it can be said that the present invention is useful for office robots.

In order to implement the recognition method of position and orientation based on the developed ceiling pattern recognition robot, it was installed in an indoor mobile robot to perform an errand function. The task the robot should perform is

 a. At a certain location in the office where several people work, if a person randomly designates a person to deliver after making the coffee,

 b. The robot will deliver coffee to the designated person avoiding obstacles,

 c. Once delivery is complete, return to the original coffee shop and wait for another delivery order

.

For this purpose, a small indoor robot as shown in Fig. 10 was installed with a camera toward the ceiling and recognized the grid pattern of the office ceiling as shown in Fig. 4 (a).

Figure 11 shows the layout of the office and the people performing the above scenario. At the top of the picture is a table for making coffee, with a destination desk 1 for delivering coffee to the left of the first third. In order to carry out the task of the scenario, the robot loads coffee from the coffee table position, arrives at the destination desk1 along the first and second path, and then rings the bell to notify that the coffee delivery has arrived and when the person sitting on the desk 1 takes the coffee , The robot is going back to the coffee table via the overpass.

Fig. 12 shows the process of the robot performing the above-mentioned tasks in order. As shown in FIG. 12 (a), the robot that receives and receives coffee from the coffee table rotates the curve in the images of FIGS. 12 (b), 12 (c), and 12 (B), (o), and (p), the robot returns to the position of the coffee table for the first time after completing the delivery mission as shown in FIG. 12 (r) .

As a result of implementing the present invention on an office coffee delivery robot as described above, the robot has always reached the destination precisely along the correct path and has accomplished the errand mission excellently. The results showed that the position error was less than 2 cm and the direction error was less than 2 degrees.

FIG. 1 is a view illustrating ceilings having repeated grid patterns; FIG.

FIG. 2 is a main flow chart of the present invention for determining the direction and position of a robot using a repeated ceiling pattern; FIG.

3 is a flowchart of a routine for determining the direction of the robot using the detected straight line;

FIG. 4 (a) is a diagram illustrating an origin-based coordinate and a robot coordinate for a straight line and an intersection point in a typical ceiling pattern;

4 (b) shows the origin-based coordinate system (x-y) and the robot reference coordinate system (x'-y ') superimposed;

5 is a flowchart of a routine for calculating the position of a robot using an intersection of straight lines in an image;

Figure 6 is an auxiliary figure used to calculate the actual position of the robot in the routine of Figure 5;

Fig. 7 is a routine describing a method of converting to an actual distance relative position vector according to claim 2; Fig.

In the case of an image in which a straight line is not found, a method of predicting the robot direction by using a pixel motion vector (optical flow) between the image frames or an accumulated vector thereof, and a method of estimating the robot direction from a cumulative vector A routine showing a method of correcting the predicted direction of the robot;

In the case of an image in which an intersection of straight lines is not found, a method of predicting a robot position by using a pixel motion vector or an accumulated vector between image frames, and a method of estimating a robot position using an intersection, And correcting the predicted robot position.

10 is a view of a mobile robot for applying the present invention to a robot for delivering coffee in an office;

11 is a view of an indoor room for performing an office coffee errand function;

12 shows a scene in which the robot performs a coffee delivery errand mission. here

(a) is a scene where coffee is taken from a coffee table, (j) (k) (l) is a scene where coffee is delivered to a destination, It is a scene that returns to the position.

Claims (4)

A camera is installed at the center of the mobile robot toward the ceiling to obtain continuous digital images of the ceiling; Calculating an angle with respect to the y-axis at the reference origin of the robot workplace for the repeated linear components of the repeated ceiling pattern of the robot workstation, preparing and preparing the angle set; Preparing and initializing an intersection set composed of positional information on all intersections in the ceiling pattern; In order to calculate the direction during the autonomous movement of the robot, a straight line is detected from the image taken during the movement and the angle between the straight line and the current direction of the robot is obtained from the image. Then, In addition to each individual angle in the set of angles for the robot, is referred to as a direction angle candidate set of the robot; Selecting an angle closest to the robot direction angle obtained from the robot position of the previous image among the angles in the direction angle candidate set as a current robot angle; In order to calculate the position during robot autonomous movement, only the positional coordinates of the intersections around the previous image position of the robot on the intersection set prepared at the initialization are taken as a robot reference position candidate set;  Selecting the most significant intersection point among the ceiling images photographed at the current position, displaying the pixel coordinates of the intersection point as relative coordinates from the center of the image, and displaying the coordinates as the converted actual distance relative position vector described in claim 2; Adding the transformed actual distance relative position vector to the reference robot position candidate set and calling it a candidate set for the actual position of the robot; Among the elements of the candidate set of the robot actual position, a position point closest to the position of the robot obtained from the previous image is determined as the current robot position point Method of calculating direction and position of robot using ceiling pattern. The transformed relative position vector according to claim 1, Obtaining a relative vector from the image center for the selected intersection; Wherein the intersection rotates and converts the vector in a direction opposite to the center of the image by the direction of the robot obtained in claim 1; Transforming the rotation-converted intersection vector by the actual distance to the unit pixel interval to convert the position of the intersection point to the actual distance relative vector from the center of the robot; Calculating an actual distance relative position vector by multiplying an actual distance relative vector to the center of the robot at the intersection by a negative sign so that the center of the robot is represented by an actual distance relative vector to the intersection. In the method of finding the direction of the robot according to claim 1, when it is difficult to calculate the direction of the robot because one linear component is not found in the image, the optical flow vector is accumulated until a linear component is found in the image, In addition to the direction of the robot in the direction of the robot; When an image having a straight line is obtained, the angle of the robot is obtained from a straight line, and a direction angle candidate set is obtained as in the method of finding the direction of claim 1, and the difference between the individual angles in the set and the cumulative sum of the motion vectors Determine the less direction as the current robot direction Directional Correction Method of Robot Using Optical Flow The method according to claim 1 or claim 2, wherein when there is no intersection point in the image, the sum of the motion vectors is accumulated by accumulating the vector of the optical flow until the next image where the intersection point is found, Adding a negative value of a sum of motion vectors to a position to obtain a robot predicted position; If the intersection point is detected in the image, the method is the same as the robot position calculation method according to claim 1 or claim 2, wherein the range of the reference robot position candidate set is enlarged by 2 to 3 times, ≪ / RTI > A position point closest to the robot predicted position is corrected from the candidate set for the robot actual position to the current robot position Robot position point calculation and complementary method using optical flow.

Images