KR101092002B1 - Artificial landmark, appratus and method for navigation using artificial landmark and natural landmark - Google Patents

Abstract

According to an embodiment of the present invention, a navigation apparatus using a artificial marker and a natural marker may include a camera for photographing an illumination image irradiated with a light source and a natural image without irradiating the light source; A difference image extracting unit calculating a difference image between the illumination image and the natural image; And a position detector for estimating the position information from the natural marker extracted from the natural image and correcting the position information from the artificial marker extracted from the difference image. Compared to the method using only natural markers, errors in position and direction can be reduced, and the position of the robot can be detected and corrected even in an environment where it is difficult to detect natural markers when the robot moves. In addition, since the use of natural markers is mixed, there is an advantage that the robot can be accurately moved and traveled in a large indoor space using a much smaller number of artificial markers than the method using only artificial markers.

Description

ARTIFICIAL LANDMARK, APPRATUS AND METHOD FOR NAVIGATION USING ARTIFICIAL LANDMARK AND NATURAL LANDMARK}

The present invention relates to a navigation device and a method of a mobile robot using artificial markers, artificial markers, and natural markers for location recognition of a mobile robot, and more particularly, from an artificial marker and a camera capable of correcting an accurate position of a mobile robot. The present invention relates to a navigation device and a method for acquiring artificial marks and natural marks to move a mobile robot to a destination more accurately.

The navigation device that controls the robot to move a relatively large indoor space such as an inpatient ward of an office or a hospital requires a function of recognizing where the robot is located in the indoor space. The technology that the robot recognizes the position in the indoor space can be largely divided into the method of using artificial markers and the method of using natural markers.

Artificial markers can be configured in a variety of ways, such as the form of a simple pattern and the method using a light emitting source. Forming a simple pattern can be easily and simply implemented, but there is a problem that it is difficult to guarantee the performance because it is sensitive to the ambient lighting conditions. The method using a light emitting source is more robust to lighting conditions than the method of forming a simple pattern, but there is a problem in that a separate power source and wiring are required because the light emitting source must be used.

The natural marker is not an artificially created marker, but extracts information that the robot can distinguish from the natural image patterns reflected on the camera image, and uses this information as a marker. A representative example used as a natural marker is a corner point, which uses a region in which the brightness level varies greatly, mainly a boundary region, in the black and white image as a natural marker.

The method of using an artificial mark is a method of comparing the artificial mark read by the camera attached to the robot with the mark previously stored in the memory, and using the position information of the previously stored mark to determine the current position of the robot.

The method of using the natural marker is an image that is in its natural state without attaching a separate marker when the artificial marker cannot be used due to an environment in which it is difficult to apply the artificial marker, that is, the place where the marker can be attached or cannot be easily seen from the robot's point of view. It extracts feature points on information and analyzes the extracted feature points as if they were artificial markers. In order to recognize natural marks, an algorithm called Scale-invariant Feature Transform (SIFT) disclosed in US Patent US6711293 is used.

The method of using artificial markers has to attach artificial markers to the spaces to which the robot must move at appropriate intervals. In particular, there is a problem that too many artificial markers are required to use such methods in a large indoor space. In addition, there may be a place where it is difficult to attach the artificial marker in the indoor space.

The method of using natural markers has the advantage of not requiring artificial markers, but there is a problem in that the positional error is too large in a place where feature information on the robot's moving environment is similar. In other words, when viewed from a camera mounted on a robot in a room, similar features are extracted even in different environments, and similar features are extracted to detect a wrong position. In addition, other feature points may be detected due to a change in lighting conditions or an object that has not been placed before, and a large error in position may occur. In this way, the position error is large in a large indoor space due to the accumulated error when detecting the position using natural markers.

Therefore, there is a need for a method for reducing an error rate for a location while minimizing artificial markers in a large indoor space and an apparatus for implementing the same.

An object of the present invention is to provide an artificial marker that enables the navigation device to quickly and accurately detect the current location.

In addition, it is an object of the present invention to provide a navigation apparatus and method capable of reliably confirming the current position by using artificial markers and natural markers and guiding the route to the destination accurately.

Artificial marker according to an aspect of the present invention is a central body representing the center of the artificial marker; A direction indicator indicating a reference axis using the center point as the origin; And a code part including figures arranged at predetermined intervals on the circumference of the center point, wherein the figures constituting the code part represent an ID of the artificial mark.

According to another aspect of the present invention, a navigation device includes: a camera for photographing an illumination image irradiated with a light source and a natural image not irradiated with a light source; A difference image extracting unit calculating a difference image between the illumination image and the natural image; And a position detector for estimating the position information from the natural marker extracted from the natural image and correcting the position information from the artificial marker extracted from the difference image.

According to still another aspect of the present invention, there is provided a navigation method, comprising: calculating a starting position by detecting an artificial mark from an image input from a camera; Retrieving a next artificial marker located at a path to a destination and spaced the shortest distance from the calculated starting position; Estimating a position by detecting a natural marker while moving from the start position to the next artificial marker; And correcting the position when the next artificial marker is detected.

Compared to the method using only natural markers, errors in position and direction can be reduced, and the position of the robot can be detected and corrected even in an environment where it is difficult to detect natural markers when the robot moves. In addition, since the use of natural markers is mixed, there is an advantage that the robot can be accurately moved and traveled in a large indoor space using a much smaller number of artificial markers than the method using only artificial markers.

1 is a block diagram illustrating a navigation device according to an embodiment of the present invention.

Referring to FIG. 1, the navigation device 500 includes a central controller 200, a travel encoder 300, and a location calculation module 100.

The central control unit 200 receives position information from the position calculating module 100 and the driving encoder 300, and a power unit (not shown) such as a motor to move an object equipped with a navigation device, for example, a robot from a starting point to a destination. It is a device to control.

The driving encoder 300 is connected to a power unit such as a motor to provide information about the rotation angle and the rotation speed according to the rotation of the power unit. The driving encoder 300 provides the central controller 200 with information about the rotation angle and the rotation speed.

Position calculation module 100 is a module for calculating the current position of the robot. The position calculation module 100 calculates the current position of the robot using artificial marks and / or natural marks, and provides information on the current position to the central controller 200.

2 is a block diagram showing a position calculation module.

The position calculation module 100 may include a camera 10, an illumination unit 20, a synchronization signal controller 30, a difference image extractor 40, a memory 50, and a position detector 60.

When a specific signal is given from the synchronization signal controller 30, the camera 10 may capture an image and adjust a time for capturing the image. The camera 10 may be integrated with the lighting unit 20 or may be spaced apart from each other. The camera 10 may take a visible light picture or an infrared picture.

The lighting unit 20 may use, for example, an infrared light emitting diode (LED) as a light source that shines light on an artificial mark. The lighting unit 20 may be turned on or off by a specific signal given from the outside and the light emission time is adjustable.

The synchronization signal controller 30 is connected to the lighting unit 20 and the camera 10 to control the operation of the lighting unit 20 and the camera 10. The synchronization signal controller 30 may acquire an image at a desired speed while the robot moves. The image taken by the camera 10 by turning off the light source of the lighting unit 20 by the synchronization signal controller 30 is called a natural image, and the image taken by the camera 10 by turning on the light source of the lighting unit 20 is called an illumination image. It is called.

The difference image extractor 40 performs a function of extracting a difference between the natural image and the illumination image. For example, the difference image may be extracted by performing an exclusive OR operation for each pixel of the natural image and the illumination image. The difference image detected by the difference image extraction unit 40 is easily represented by an artificial mark. The difference image extractor 40 may be configured of, for example, a field programmable gate array (FPGA).

The memory 50 stores location information of artificial marks and natural marks, and updates and stores the current position. The memory 50 is connected to the position detector 60 to provide position information of artificial marks and natural marks. The memory 50 may be integrated into the location calculation module 100 or may be separately configured.

The position detector 60 identifies the natural marker from the natural image input from the camera 10, identifies the artificial marker from the difference image input from the difference image extractor 40, and matches the position information stored in the memory 50. Determine the current position of the position calculation module 100 by determining whether or not.

3 is an exemplary view illustrating an operation process of a synchronization signal controller, an illumination unit, and a camera in time. 4 shows an example of obtaining a difference image.

The synchronization signal controller 30 may provide the first synchronization signal 31 for operating the camera 10 and the illumination unit 20 simultaneously, and the second synchronization signal 32 for operating only the camera 10. The first synchronization signal 31 and the second synchronization signal 32 may be signals that are separated from each other, or may be signals that are distinguished according to the same signal or sequence. The illumination image may be photographed by the first sync signal 31, and the natural image may be photographed by the second sync signal 32.

5 is an exemplary view showing an artificial mark.

Artificial landmarks (ALs) are artificial markers that are distinct from the background and can contain coded information that the robot can recognize.

Referring to FIG. 5, the artificial marker may be composed of a central body, a direction indicator, and a code part.

The centrosome is located in the center of the artificial marker and can be configured in a circle. The central body may obtain a center point of the artificial mark, that is, a reference position, and may have a larger area than other parts of the artificial mark, that is, the direction indicator or the code part, so that the camera of the robot can be easily identified.

The direction indicator indicates the reference axis of the artificial marker with the center point as the origin. The direction indicator is preferably composed of different geometric shapes so as to be distinguishable from the code part and the center body. For example, the direction indicator may be configured in the form of an ellipse.

The code part is a part containing information for identifying the unique ID of the artificial mark. The code part may be arranged at predetermined intervals on the same circumference with respect to the center point of the artificial mark. The code part may be composed of a plurality of circles having the same area. When a circle is disposed at a position where such circles may be arranged, the bit string may be represented by a bit value '1' and a bit string '0' when the circle is not disposed. For example, if there are seven locations where circles can be placed, we can make 2 ⁷ = 128 codes.

The plurality of circles constituting the code part are each one bit in order from Most Significant Bit (MSB) to Least Significant Bit (LSB) in the clockwise (or counterclockwise) direction on the reference axis 51 determined by the center body and the direction indicator. In FIG. 5, MSB is p ₀ , and LSB is _{shown in} the order p ₆ . This bit string code provides a unique number for each artificial marker so that each artificial marker can be identified. Each artificial marker can exist at a known location, so the mobile robot can know the current location by identifying the artificial marker.

Equation 1 shows a code identified in the code part as an equation.

In Equation 1, ρ _i denotes a pixel value of a region corresponding to the i-th bit in the difference image, and T is a threshold value that determines whether a circle included in the code part exists. T can be given experimentally predetermined.

Artificial markers can be made of reflective surfaces made of a retro-reflective material. Then, even when there is no general lighting at night when the illumination unit is irradiated with infrared, the camera has no problem shooting artificial marks.

6 shows an actual image of the artificial marker acquired from the difference image.

In FIG. 6, it is determined whether there is a circle included in the code part in the clockwise direction with the center point as the center point and the direction with the direction indicator as the reference axis. That is, the pixel value of the area where the circle included in the code part is to be located is compared with the boundary value T to determine. In FIG. 6, p ₀ , p ₂ , p ₃ , and p ₆ are smaller than the boundary value so that the corresponding bit becomes '0', and p ₁ , p ₄ and p ₅ become equal to or greater than the boundary value and the corresponding bit becomes '1'. . Therefore, the code is '0100110' in binary.

Natural Landmark (NL) means a marker for location identification except artificial marker. That is, it is not a marker that artificially includes information, but a marker that uses information that is easy to identify among the image patterns reflected on the image as if it is an artificial marker. A representative example used as a natural marker is a corner point, which uses a region where a change in brightness degree is severe, mainly a boundary region, in a black and white image as a natural marker.

7 illustrates a method for identifying natural marks according to an embodiment of the present invention.

The position detector acquires a natural image from the camera (S10). The position detector removes noise by applying a Gaussian filter to the natural image (S20). The position detector detects a corner point by analyzing the eigenvalue in the image from which the noise is removed (S30). The corner point can be detected by the following equation.

In Equation 2, I _x , I _y are the daily partial differential values of the pixel values in the x and y directions for each pixel included in the image region of a specific size, and Z is I _x , I in the image region of the specific size. _This is the matrix for the covariance of _y . From these covariance matrices, the eigenvalues λ ₁ and λ ₂ are obtained. When only one of λ ₁ and λ ₂ has a value larger than λ _t , which is a boundary value, it is determined that only one line exists in the image area of the specific size. When both λ ₁ and λ ₂ have a value larger than the boundary value λ _t , it is determined that two lines exist within the image area of the specific size. Here, the line represents a boundary area in which a change in brightness value is severe when each pixel has only a brightness value in a black and white image, for example. Λ _{t, which} is a boundary value, may be predetermined as a value determined experimentally. As described above, the method of identifying natural marks is a simple method and a small amount of calculation.

Feature points such as the corner points described above are stored in the memory. Feature points can store general three-dimensional coordinates using three parameters (x, y, z) relative to a particular reference point in the room to be moved when stored in memory, but to speed up the convergence of the filter when mapping It can also be stored using five parameters.

8 shows parameters used when storing feature points in a memory.

(x _i , y _i ) is a coordinate value of a two-dimensional plane in which the robot first observes a feature point based on a specific reference point W in an indoor space, and is called an observation point. The position of the feature point is represented by a new polar coordinate system (1 / ρ _i , α _i , β _i ) based on the coordinate values (x _i , y _i ) of the observation point. Where ρ _i is the inverse of the distance between the observation point and the feature point, α _i represents the azimuth, and β _i represents the elevation.

The position of the feature point represented by the polar coordinate system with respect to the observation point may be converted into the Cartesian coordinate system of the reference point W as shown in Equation 3 below.

In Equation 3, X ^W _F is a vector representing a feature point based on the reference point W. As the robot moves, feature points moving in the camera image are continuously updated using the Extended Kalman Filter (EKF). The difference between the position of the observed feature point and the position of the feature point stored in the memory is calculated and the correction is performed when there is an error. When the position of the observed feature point is out of a predetermined range and the position of the feature point stored in the memory, it is determined as a new feature point and stored in the memory. In this case, in order to prevent the problem of misrecognizing the feature point, it may be stored in the report memory as a valid feature point only for the feature points continuously observed for a predetermined time or more.

9 is a flowchart illustrating a navigation method according to an embodiment of the present invention.

When a destination is input from the outside (S80), the robot including the above-described navigation device (S80) grasps the current position (S90). In order to determine the current position, first, an artificial marker is detected. If no artificial marker is detected, only the natural marker identifies the current position. If the current location is identified, a path to the destination is generated (S100).

The mobile robot searches for the next artificial marker located on the path to the destination and is spaced apart from the determined starting position and moves to the next artificial marker (S200). While moving from the starting position to the next artificial marker, the mobile robot continuously photographs the ceiling to acquire the natural image, identifies the natural marker in the natural image, and estimates the current position. When the mobile robot detects the next artificial marker (S300), the mobile robot accurately corrects the current position (S400). If the mobile robot does not detect the next artificial marker, it attempts to detect the natural marker. If a natural marker is detected (S600), the current position is estimated based on the natural marker. If neither the artificial mark nor the natural mark is detected, the current position is estimated based on the encoding information provided by the driving encoder (S800).

The mobile robot determines whether it has arrived at the destination (S500), and if it does not arrive at the destination, moves to the artificial marker spaced apart from the shortest distance on the path (S200).

Hereinafter, specific embodiments and effects of the present invention will be illustrated through a photograph and a drawing.

10 is a picture of implementing the position calculation module according to an embodiment of the present invention.

An infrared illuminating unit was installed around the camera, and was composed of one module including a synchronization signal controller.

11 illustrates a natural image and an illumination image photographed by a camera included in a location calculation module. In FIG. 11, the left picture shows a natural image, but the artificial mark does not appear well, while the right picture shows the artificial image clearly visible in the illumination image.

12 shows an example of arranging natural marks and artificial marks in an indoor space.

As shown in FIG. 12, there are a large number of natural markers in an indoor space, whereas only a few artificial markers exist according to an embodiment of the present invention. In other words, it is possible to use the minimum artificial marking required.

13 is a diagram illustrating a result of a conventional navigation method and a navigation method according to the present invention in a graph.

In FIG. 13, the red line represents the movement path of the mobile robot when the navigation is performed using only the encoding information of the driving encoder of the mobile robot, and the blue line represents the movement route of the mobile robot when the navigation is performed using only the natural marker. . That is, the result according to the conventional navigation method. On the other hand, the green line represents the movement path of the mobile robot when the navigation is performed using both the natural mark and the artificial mark according to the embodiment of the present invention. As shown in FIG. 13, in the case of the present invention, the mobile robot moves to the destination along the corridor, but the other methods show that the location measurement error is large and navigation is not performed properly.

The navigation method according to the present invention can estimate the current position of the robot by using a natural marker, and correct the current position by using a reliable artificial marker. Since both artificial and natural markers can be used, it can be applied to areas where artificial markers are difficult to install.

Although the embodiments of the present invention have been described in detail as described above, those skilled in the art will appreciate that the present invention may be made without departing from the scope of the present invention as defined in the appended claims. It may be modified in various ways. Therefore, modifications of the embodiments of the present invention will not depart from the scope of the present invention.

1 is a block diagram illustrating a navigation device according to an embodiment of the present invention.

2 is a block diagram showing a position calculation module.

3 is an exemplary view illustrating an operation process of a synchronization signal controller, an illumination unit, and a camera in time.

4 shows an example of obtaining a difference image.

5 is an exemplary view showing an artificial mark.

6 shows an actual image of the artificial marker acquired from the difference image.

7 illustrates a method for identifying natural marks according to an embodiment of the present invention.

8 shows parameters used when storing feature points in a memory.

9 is a flowchart illustrating a navigation method according to an embodiment of the present invention.

10 is a picture of implementing the position calculation module according to an embodiment of the present invention.

11 illustrates a natural image and an illumination image photographed by a camera included in a location calculation module.

12 shows an example of arranging natural marks and artificial marks in an indoor space.

13 is a graph illustrating results of various navigation methods.

Claims (12)

In the artificial marker to inform the navigation device of the location information, A centroid representing a centroid of the artificial marker; A direction indicator indicating a reference axis using the center point as the origin; And And a code part including figures arranged at predetermined intervals on a circumference of the center point, wherein the figures constituting the code part represent an ID of the artificial mark. The method of claim 1, wherein each of the figures of the code part represents one bit in order of most significant bit (MSB) to least significant bit (LSB) according to the order arranged clockwise or counterclockwise on the reference axis. Artificial marker characterized by. The artificial marker according to claim 1, wherein the cord part comprises circles arranged at equal intervals on a circumference of the center point. The method of claim 1, The artificial marker is an artificial marker, characterized in that the reflective surface is composed of a material that totally reflects light. A camera for photographing an illumination image irradiated with a light source and a natural image not irradiated with the light source; A difference image extracting unit calculating a difference image between the illumination image and the natural image; And A position detector for estimating the position information from the natural marker extracted from the natural image and correcting the position information from the artificial marker extracted from the difference image, The artificial marker is A centroid representing a centroid of the artificial marker; A direction indicator indicating a reference axis using the center point as the origin; And A navigation unit comprising an artificial marker and a natural marker including a code part including figures arranged at predetermined intervals on a circumference of the center point, wherein the figures constituting the code part represent an ID of the artificial mark. Device. The navigation apparatus as claimed in claim 5, wherein the natural marker is identified by obtaining an eigenvalue from the covariance of each pixel included in the natural image. The method of claim 6, The covariance is a navigation apparatus using a combination of artificial and natural markers, characterized in that determined by the following equation. [Equation]

In the above equation, Ix and Iy are daily partial differential values in the x and y directions of respective pixel values of the predetermined region included in the natural image, and Z is a covariance matrix of Ix and Iy in the predetermined region. Calculating a starting position by detecting an artificial mark from an image input from a camera; Retrieving a next artificial marker located at a path to a destination and spaced the shortest distance from the calculated starting position; Estimating a position by detecting a natural marker while moving from the start position to the next artificial marker; And Compensating the position when the next artificial marker is detected, The artificial marker is A centroid representing a centroid of the artificial marker; A direction indicator indicating a reference axis using the center point as the origin; And A navigation unit comprising an artificial marker and a natural marker including a code part including figures arranged at predetermined intervals on a circumference of the center point, wherein the figures constituting the code part represent an ID of the artificial mark. Way. delete delete delete delete

Images