KR101980677B1 - Apparatus and method of feature registration for image based localization and robot cleaner comprising the same apparatus - Google Patents

Abstract

The present invention relates to a feature point registration apparatus and a registration method for image-based position recognition. One aspect of an embodiment of a feature point registration apparatus for image-based position recognition according to the present invention is a feature point registration apparatus for inputting a 2D image and a depth image; An image processor for extracting feature points and depth information from 2D images and depth images input by the image input device, and for generating descriptors by matching the extracted feature points and depth information; An image register for registering the feature points including the descriptor generated in the image processor; . Therefore, according to the embodiment of the present invention, it is possible to expect a simpler and more accurate feature point extraction and registration.

Description

TECHNICAL FIELD The present invention relates to a feature point registration device for image-based position recognition, a registration method using the feature point registration device, and a robot cleaner including the feature point registration device.

The present invention relates to a feature point registration apparatus for image-based position recognition, a registration method using the feature point registration apparatus, and a robot cleaner including the same.

A mobile robot is a robot that performs a predetermined operation while traveling, for example, a robot-type vacuum cleaner or a robot-type medical device. In order to perform such an operation of the mobile robot, it is necessary to create a map and recognize an absolute position on the map. Simultaneous localization and mapping (SLAM) technology is a representative example of such a technique for creating a map and recognizing an absolute position.

SLAM technology is an image-based location recognition technology in which a mobile robot creates a map based on acquired images and recognizes its position. In such SLAM, an image is captured using a StereoVision camera or a TOF (Time of Flight) camera, and a feature is extracted from the captured image. Then, three-dimensional coordinate values corresponding to the feature points are calculated and registered as three-dimensional feature points.

However, the following problems arise in the technique of position recognition and mapping according to the related art.

Conventionally, a three-dimensional feature point is registered as a three-dimensional coordinate value calculated through a plurality of calculations from an image taken by the camera. For example, a feature point and a depth value may be extracted from a plurality of images obtained by down-sampling an image taken by the camera, and three-dimensional coordinate values may be calculated therefrom. Alternatively, a background region and a foreground region may be extracted from the image captured by the camera, and a three-dimensional coordinate value may be calculated from a depth value of a foreground region having a relatively small depth value. However, since the above-described methods require a relatively large number of calculations, a separate processor for processing the same, that is, a graphics processing unit (GPU) or the like is required, It can take.

In addition, conventionally, it may become impossible to calculate the depth value of a feature point from an image taken by the camera due to external conditions, for example, a change in shade due to an obstacle or illumination. In such a case, since the feature points can not be extracted substantially, accurate three-dimensional information can not be registered.

It is an object of the present invention to provide a feature point registration device for image-based location recognition that enables simpler and more accurate extraction and registration of feature points, and a registration method using the same And a robot cleaner including the same.

According to another aspect of the present invention, there is provided a feature point registration apparatus for image-based location recognition, the feature point registration apparatus comprising: a video input unit for inputting a 2D image and a depth image; An image processor for extracting feature points and depth information from 2D images and depth images input by the image input device, and for generating descriptors by matching the extracted feature points and depth information; An image register for registering the feature points including the descriptor generated in the image processor; .

One aspect of the feature point registration method for image-based location recognition according to the present invention includes: inputting first and second image input units, a 2D image and a depth image; The feature point extracting unit and the depth information extracting unit extracting feature points and depth information from the 2D image and the depth image, respectively; Matching the feature point matching unit with the extracted feature point and depth information; Generating a descriptor from the extracted feature point and depth information; And registering the minutiae point including the descriptor; .

In the embodiment of the feature point registration device for image-based position recognition, the registration method using the same, and the robot cleaner including the same, the feature point and depth information are extracted from the 2D image and the depth image, respectively, . In the embodiment of the present invention, when there is no depth information corresponding to the extracted feature points, the minimum value among the depth values of other pixels adjacent to the feature points is estimated as the depth value of the feature points. Therefore, according to the embodiment of the present invention, it is possible to expect a simpler and more accurate feature point extraction and registration.

1 is a block diagram showing an SLAM apparatus constituting an embodiment of a feature point registration apparatus for image-based position recognition according to the present invention;
2 is a control flowchart showing a minutiae point extraction process according to an embodiment of the present invention.
FIGS. 3 to 7 are conceptual diagrams showing a minutiae point extraction process in an embodiment of the present invention. FIG.
FIG. 8 is a perspective view illustrating an embodiment of a robot cleaner including a feature point registration device for image-based position recognition according to the present invention. FIG.
9 is a block diagram showing an embodiment of the present invention.

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS Hereinafter, embodiments of a feature point registration apparatus for image-based position recognition according to the present invention will be described in detail with reference to the accompanying drawings.

1 is a block diagram showing an SLAM apparatus constituting an embodiment of a feature point registration apparatus for image-based position recognition according to the present invention.

Referring to FIG. 1, the minutiae registration apparatus according to the present embodiment includes a video input device 100, an image processor 200, and an image register 300. The image input device 100 is a part for inputting images of a subject for extracting and registering minutiae points. The image processor 200 extracts Feature and Depth information from the image input by the image input device 100 and generates a descriptor for each feature point. The feature points extracted from the image processor 200 are registered in the image register 300.

In more detail, the image input device 100 includes a first image input unit 110 and a second image input unit 120. [ The first image input unit 110 inputs a 2D image of a subject and the second image input unit 120 inputs a depth image of a subject. As the first image input unit 110, a CMOS (Complementary Metal Oxide Semiconductor) camera or a CCD (Charge-Coupled Device) camera capable of capturing a color of a general subject can be used. The second image input unit 120 calculates a depth value by measuring a time difference between the irradiation and reflection of light to a subject, and expresses the depth value as an image. For example, the second image input unit 120 irradiates infrared rays to a subject, detects reflected infrared rays, measures a time difference between them, and can express the calculated depth value as an image.

The image processor 200 includes a feature point extraction unit 210, a depth information extraction unit 220, a feature point matching unit 230, and a descriptor generation unit 240. The feature point extraction unit 210 extracts feature points from the 2D image input by the first image input unit 110. For example, the corners existing in the 2D image may be extracted as feature points. The depth information extracting unit 220 extracts depth information, that is, a depth value, from the depth image input by the second image input unit 120. The feature point matching unit 230 matches the feature points and the depth information corresponding to each other. In this case, when the depth information corresponding to the feature point does not exist, the feature point matching unit 230 estimates a minimum value among the depth values of other pixels adjacent to the feature point as a depth value of the feature point. For example, if the depth value of the feature point is represented as 0 from the depth image by surrounding shadows or obstacles, the actual depth value can not be extracted. In such a case, the feature point matching unit 230 estimates the minimum value among the depth values of other pixels adjacent to the feature point as the depth value of the feature point. For example, a minimum value among the depth values of pixels belonging to the mxn area around the feature point can be estimated as the depth value of the feature point. This is because the feature points are usually extracted in the foreground relative to the background. The descriptor generator 240 generates a descriptor from the minutiae extracted by the minutiae extraction unit 210 and the extracted depth information from the depth information extraction unit 220. [ For example, the descriptor generating unit 240 may match a feature point extracted by the feature point extracting unit 210 and the depth information extracted by the depth information extracting unit 220 to form a SIFT (Scale Invariant Feature Transform) descriptor Can be generated.

The image register 300 includes a minutia point register 310 and a data base 320. The minutia point registering unit 310 registers the minutiae pointing to the descriptor received from the minutia matching unit 230 in the database 320.

Hereinafter, embodiments of a feature point registration method for image-based position recognition according to the present invention will be described in detail with reference to the accompanying drawings.

FIG. 2 is a control flowchart showing a minutiae point extraction process according to an embodiment of the present invention, and FIGS. 3 to 7 are conceptual diagrams showing a minutiae point extraction process in an embodiment of the present invention.

2, the image input device 100, that is, the first and second image input units 110 and 120, respectively, inputs a 2D image and a depth image of a subject (S10). At this time, The 2D image input by the first image input unit 110 and the depth image input by the second image input unit 120 can be easily understood with reference to FIG. 3 and FIG. 4, respectively.

Next, the image processor 200 extracts feature points and depth information from the images input by the image input device 100, and generates descriptors for the feature points (S20) (S30). Then, the image processor 200 determines whether or not the depth information corresponding to the minutia is present, and then matches the minutiae and corresponding depth information (S50) (S60)

More specifically, in step 20, the feature point extracting unit 210 and the depth information extracting unit 220 extract the 2D image input by the first image input unit 110 and the 2D image input by the second image input unit 120, respectively (Refer to FIG. 3 and FIG. 4). In step 30, the feature point matching unit 230 extracts feature points and depth information corresponding to the feature points extracted from the feature point extraction unit 210 That is, a depth value. In other words, in step 30, the depth information extracting unit 220 may determine whether or not the depth information corresponding to the feature point can be extracted from the depth image. If it is determined in step 30 that the depth information corresponding to the feature point exists, in step 40, the descriptor generating unit 240 generates a descriptor. The descriptor may be SIFT descriptor.

Meanwhile, if it is determined in step 30 that the depth information corresponding to the feature point does not exist, the feature point matching unit 230 estimates the minimum depth value of the pixels adjacent to the feature point as the depth value of the feature point (S50) For example, as shown in Fig. 6, the minutiae point extracting unit 210 can extract the minutiae point (C). However, as shown in FIG. 7, depth information corresponding to the feature point C is displayed as 0 in the depth image. For example, the depth information corresponding to the feature point C may not be expressed in the depth image due to surrounding shadows or obstacles. In this case, referring to Table 1 below, when the feature point matching unit 230 refers to a plurality of pixels adjacent to the feature point C, 976, which is the minimum value among the depth values of the pixel A, is estimated as the depth value of the minutiae point (C). However, the region for estimating the depth value of the feature point C is not limited to the 5x5 region. If the depth value of the feature point is estimated in step 50, step 40 is performed.

1278 1270 1284 1273 1275 0 0 0 1034 1027 0 0 0 (feature point) 0 1088 0 0 0 0 1070 0 0 976 978 987

In operation 40, when the descriptor for the feature point is generated, the image register 300, that is, the minutia point registration unit 310 registers the minutiae including the descriptor in the database 320 5)

Hereinafter, an embodiment of a robot cleaner including a feature point registration device for image-based position recognition according to the present invention will be described in detail with reference to the accompanying drawings.

FIG. 8 is a perspective view illustrating an embodiment of a robot cleaner including a feature point registration device for image-based position recognition according to the present invention, and FIG. 9 is a block diagram illustrating an embodiment of the present invention. The constituent elements of this embodiment that are the same as the constituent elements of the above-described embodiment of the present invention are referred to with reference to Figs. 1 to 7, and a detailed description thereof will be omitted.

Referring to FIGS. 7 and 8, the robot cleaner according to the present embodiment carries out an operation of traveling while moving in a predetermined area. The robot cleaner includes a casing 10, a traveling unit 20, a suction unit 30, an image input device 100, an image processor 200, an image registration device 300, and a control unit 40.

The casing (10) forms the appearance of the robot cleaner. In the casing 10, parts constituting the robot cleaner, that is, the traveling unit 20, the suction unit 30, the image input device 100, the image processor 200, the image recorder 300, A control unit 40 is provided.

The running section 20 provides a driving force for running the casing 10. [ For example, the traveling section 20 may include a traveling motor (not shown) and a traveling wheel (not shown) rotated by the traveling motor.

The suction unit 30 provides a driving force for sucking foreign matter into the casing 10. [ For example, the suction unit 30 may include a suction motor (not shown) and a suction fan (not shown) rotated by the suction motor.

The image input device 100 and the image processor 200 may respectively input a 2D image and a depth image of a predetermined area corresponding to the front of the casing 10 in the traveling direction of the casing 10, Extracts feature points and depth information from 2D images and depth images input by the image processor 100, and generates descriptors by matching the extracted feature points and depth information. The image register 300 registers the minutiae including the descriptor generated in the image processor 200. Practically, the detailed configuration and operation of the image input device 100, the image processor 200, and the image recorder 300 are the same as those of the first embodiment of the present invention, and a description thereof will be omitted.

The control unit 40 controls the operations of the components constituting the robot cleaner, that is, the running unit 20, the suction unit 30, the image input device 100, the image processor 200, do. The control unit 40 creates a map based on the information on the minutiae registered in the video registrar 300 and compares the coordinates of the minutiae extracted in real time in the image processor 200 on the created map, On the map, the position of the casing 10, i.e., the robot cleaner. The control unit 40 controls the operation of the travel unit 20 and the suction unit 30 so that the robot cleaner can clean the entire area while traveling without being interfered with by the obstacle according to the position of the robot cleaner on the map .

It will be understood by those skilled in the art that various changes in form and details may be made therein without departing from the spirit and scope of the present invention as defined by the appended claims and their equivalents. .

Claims (11)

A video input device for inputting a 2D image and a depth image;
An image processor for extracting feature points and depth information from 2D images and depth images input by the image input device, and for generating descriptors by matching the extracted feature points and depth information; And
An image register for registering the feature points including the descriptors generated in the image processor; Lt; / RTI >
Wherein the image processor determines whether or not the depth information corresponding to the feature point can be extracted from the depth image and if the depth information corresponding to the feature point can not be extracted from the depth image, Wherein the minimum value of the depth values of the pixels belonging to the mxn area is estimated as the depth value of the feature point. The method according to claim 1,
The image input device includes:
A first image input unit for inputting the 2D image; And
A second image input unit for inputting the depth image; And a feature point registration device for image-based location recognition.
delete delete The method according to claim 1,
Wherein the image processor comprises:
A feature point extracting unit for extracting the feature points from the 2D image;
A depth information extracting unit for extracting the depth information from the depth image; And
A feature point matching unit for matching the feature points extracted by the feature point extraction unit and the depth information extracted by the depth information extraction unit; And a feature point registration device for image-based location recognition. 6. The method of claim 5,
Wherein the feature point extracting unit extracts a minimum value of depth values of pixels belonging to an mxn area centered on the feature point as a depth value of the feature point when the depth information corresponding to the feature point is displayed as 0 in the depth image A feature point registration device for image - based location recognition. Inputting first and second image input units, a 2D image and a depth image;
The feature point extracting unit and the depth information extracting unit extracting feature points and depth information from the 2D image and the depth image, respectively;
Matching the feature point matching unit with the extracted feature point and depth information;
Generating a descriptor from the extracted feature point and depth information; And
The minutia point registering unit registers the minutiae including the descriptor; Lt; / RTI >
Extracting the feature points and the depth information, respectively,
Determining whether or not the depth information corresponding to the feature point can be extracted from the depth image, and if it is not possible to extract the depth information corresponding to the feature point from the depth image, And estimating a minimum value among the depth values of the pixels belonging to the region as a depth value of the feature point. delete 8. The method of claim 7,
The feature point matching unit may further include an image-based position recognition unit that estimates a minimum value among depth values of pixels belonging to an mxn area around the feature point as a depth value of the feature point, when the depth value extracted by the depth information extraction unit is 0 A minutiae registration method for.
delete delete

Images