KR20210066091A - A robot cleaner using lidar member and image photographing member and SLAM-map generation method thereof - Google Patents

Abstract

The present invention provides a robot cleaner using a LiDAR unit and an image capturing unit to implement a cleaning map close to a real map and a simultaneous localization and map-building (SLAM)-map generation method thereof. According to the present invention, the robot cleaner comprises: a housing unit forming an inner space, having one side part made of a light-transmitting material, and having a plurality of moving wheels formed therein to rotate while moving; a LiDAR unit disposed inside the housing unit adjacent to eth one side part of the housing unit and generating a SLAM map for a front area viewed through the one side part among external environments; an image capturing unit disposed inside the housing unit adjacent to the inner surface of the housing unit to capture an image for the rear area opposite to the front area and generate feature information about the image; an information correction unit using the SLAM map and the feature information to estimate the location of the housing unit and correct the SLAM map; a positioning unit using the image to estimate the current position and receiving final position information on the final position of the housing unit corrected by the information correction unit; a mapping unit receiving a cleaning map acquired by correcting the SLAM map in the information correcting unit; and a control unit controlling of the LiDAR unit, the image capturing unit, and the information correction unit.

Description

A robot cleaner using lidar member and image photographing member and SLAM-map generation method thereof}

The present invention relates to a robot cleaner using a lidar unit and an image capturing unit and a slam map generation method thereof, and more particularly, by correcting the slam map based on the slam map generated by the lidar unit and the characteristic information generated by the image capturing unit. A robot cleaner using a lidar unit and an image capturing unit that implements a cleaning map close to reality, and a slam map generation method thereof.

In general, robots have been developed for industrial use and have been used to perform repetitive tasks as part of factory automation. Recently, not only industrial robots but also various types of robots have appeared, and in particular, human-friendly robots that perform human tasks while moving on their own in homes or offices are being put to practical use. Representative examples corresponding to this include a cleaning robot, a security robot, a guide robot, a service robot, and the like.

Mobile robots such as double cleaning robots use SLAM (Simultaneous Localization And Map-Building) technology to measure and measure their location during the current time and create a map of the external environment at the same time. For this purpose, it is common to apply a lidar sensor to the cleaning robot. This lidar sensor is mounted on the outside of the housing of the cleaning robot to easily sense the external environment in a 360° direction.

However, the cleaning robot equipped with the above-described external lidar sensor has a problem in that foreign substances are frequently accumulated between the lidar sensor and the housing, thereby reducing the mechanical performance and protruding outwardly in appearance, thereby failing to give an aesthetic feeling.

To solve this problem, there are cleaning robots equipped with a built-in lidar sensor, but the cleaning robot equipped with such a built-in lidar sensor has a limited angle sensing the front area as the lidar sensor is located inside the cleaning map. There was a problem in that there was a lot of difference from reality in mapping or estimating the current location.

(Patent Document 1) Registered Patent Publication No. 10-0791384 (2007.12.27.)

(Patent Document 2) Registered Patent Publication No. 10-1575597 (2015.12.02.)

An object of the present invention for solving the above problems is the first slam map, the second slam for the first front area, the second front area, and the third front area, respectively, in the rider part by the housing part that rotates by 120 degrees. Create a map and a third slam map, and use the feature information obtained from the image capturing unit to correct the overlapping or spaced parts between the first slam map, the second slam map, and the third slam map to create a cleaning map close to the real one It is to provide a robot cleaner using a lidar unit and an image capturing unit to implement and a method for generating a slam map thereof.

The technical problems to be achieved by the present invention are not limited to the technical problems mentioned above, and other technical problems not mentioned can be clearly understood by those of ordinary skill in the art to which the present invention belongs from the description below. There will be.

The configuration of the present invention for achieving the above object is a housing portion that forms an inner space, one side is formed of a light-transmitting material, and a plurality of moving wheels are formed and are rotatable while moving; a lidar unit located inside the housing unit adjacent to one side of the housing unit, rotatable and generating a slam map for a front area viewed through the side unit in an external environment; an image capturing unit located inside the housing unit adjacent to the inner surface of the housing unit to capture an image of a rear area opposite to the front area and to generate characteristic information about the image; an information correction unit for estimating a position of the housing unit using the slam map and the characteristic information and correcting the slam map; a positioning unit estimating a current position using the image and receiving final position information on the final position of the housing unit corrected by the information correction unit; a mapping unit for receiving a cleaning map obtained by correcting the slam map from the information correction unit; and a control unit for controlling the operations of the lidar unit, the image capturing unit, and the information correcting unit; provides a robot cleaner using a lidar unit and an image capturing unit, comprising: a.

In an embodiment of the present invention, the image capturing unit, a first camera located at a location rotated by 120° in the lidar unit with respect to the center of the housing unit; and a second camera positioned at a location rotated by 120° from the first camera with respect to the center of the housing part.

In an embodiment of the present invention, the front area includes a first front area, a second front area, and a third front area, and the slam map includes a first slam map for the first front area and the second front area. a second slam map for the region and a third slam map for the third front region, wherein when the position of the housing part is requested, the control unit controls the moving wheel to rotate the housing part by 120°, , the lidar unit may be characterized in that it controls to generate the first to third slam maps according to the rotation of the housing unit.

In an embodiment of the present invention, the information correction unit may generate the entire slim map through an operation of summing the first to third slim maps.

In an embodiment of the present invention, between the first slam map and the second slam map, the second slam map and the third slam map, and the first slam map and the third slam map in the entire slim map When a spaced apart section or an overlapping overlapping section is formed, the image capturing unit may generate the feature information using a feature selection and feature extraction algorithm based on PCA (Principal Component Analysis).

In an embodiment of the present invention, the feature information includes first feature information on the first front region, second feature information on the second front region, and third feature information on the third front region It is information about the geometrical position of the feature, and the information correction unit may generate the cleaning map by matching the first to third feature information.

In an embodiment of the present invention, the information correction unit may be characterized in that the final position is estimated from the entire slim map by generating a cleaning area mark based on the image of the fixed object in the external environment in the image. have.

In an embodiment of the present invention, the lidar unit irradiates light of a laser wavelength to an external environment through one side of the housing unit and receives reflected light reflected from the external environment to generate location information and mapping information of the housing unit can be characterized.

In addition, in the configuration of the present invention for achieving the above object, in the method for generating a slam map of a robot cleaner according to the above-mentioned bar, (a) generating a slam map for at least a part of the front area using the lidar unit to do; (b) generating the slim map while the housing unit rotates when a position indication is requested; and (c) correcting and completing the slam map using the image capturing unit.

In an embodiment of the present invention, the step (a) comprises: (a1) rotating the lidar unit by the control unit; and (a2) generating the slam map by the lidar unit.

In an embodiment of the present invention, the step (b) comprises: (b1) rotating the housing part by 120° by the control unit; (b2) determining whether the housing part of the robot cleaner is rotated; (b3) estimating the current location in the location check unit; (b4) determining whether the current location is correct; and (b5) performing step (c).

In an embodiment of the present invention, the step (b) may further include, after the step (b2), (b6) returning to the step (a1) when the housing part is not rotated. have.

In an embodiment of the present invention, the step (b) further comprises, after the step (b4), (b7) returning to the step (a1) when the current location is incorrect. can do.

In an embodiment of the present invention, the step (c) comprises: (c1) moving the housing unit by the control unit; (c2) capturing and acquiring the image by the image capturing unit; (c3) estimating the final position from the slim map by generating, by the information correction unit, a cleaning area mark based on an image of an object fixed in the external environment in the image; (c4) generating the slam map by the lidar unit; and (c5) completing, by the information correction unit, a cleaning map in which the slam map is corrected.

The effect of the present invention according to the above configuration is, the first slam map, the second slam map for the first front area, the second front area and the third front area in the rider part by the housing part rotated by 120 degrees, respectively and a third slam map is generated and a cleaning map close to reality is implemented by correcting overlapping or spaced portions between the first slam map, the second slam map, and the third slam map using the feature information obtained from the image capturing unit. can

In addition, the effect of the present invention according to the configuration as described above is to reduce the phase error for mapping the cleaning map from the slam map and feature information obtained from the lidar unit and the image capturing unit to realize the cleaning map in an approximate approximation to the real one. have.

It should be understood that the effects of the present invention are not limited to the above-described effects, and include all effects that can be inferred from the configuration of the invention described in the detailed description or claims of the present invention.

1 is a block diagram illustrating a robot cleaner using a lidar unit and an image capturing unit according to an embodiment of the present invention.
2 is a plan view showing a housing unit, a lidar unit, and an image capturing unit of a robot cleaner using a lidar unit and an image capturing unit according to an embodiment of the present invention.
3 is a plan view illustrating that the robot cleaner using the lidar unit and the image capturing unit according to an embodiment of the present invention rotates while moving to photograph the first to third front regions.
4 (a), (b), and (c) are diagrams illustrating geometric positions of feature information extracted from images obtained by photographing the first to third front regions by the image capturing unit of FIG. 3 .
5 is a plan view for explaining correction according to overlapping or spacing of the first to third front areas of the robot cleaner using the lidar unit and the image capturing unit according to an embodiment of the present invention.
6 is a flowchart illustrating a method of generating a slam map of a robot cleaner using a lidar unit and an image capturing unit according to an embodiment of the present invention.
7 is a flowchart illustrating detailed steps of each step of FIG. 6 .

Hereinafter, the present invention will be described with reference to the accompanying drawings. However, the present invention may be embodied in several different forms, and thus is not limited to the embodiments described herein. And in order to clearly explain the present invention in the drawings, parts irrelevant to the description are omitted, and similar reference numerals are attached to similar parts throughout the specification.

Throughout the specification, when a part is said to be “connected (connected, contacted, coupled)” with another part, it is not only “directly connected” but also “indirectly connected” with another member interposed therebetween. "Including cases where In addition, when a part "includes" a certain component, this means that other components may be further provided without excluding other components unless otherwise stated.

The terms used herein are used only to describe specific embodiments, and are not intended to limit the present invention. The singular expression includes the plural expression unless the context clearly dictates otherwise. In this specification, terms such as "comprise" or "have" are intended to designate that a feature, number, step, operation, component, part, or a combination thereof described in the specification exists, but one or more other features It should be understood that this does not preclude the existence or addition of numbers, steps, operations, components, parts, or combinations thereof.

Hereinafter, embodiments of the present invention will be described in detail with reference to the accompanying drawings.

1. Robot cleaner using lidar unit and video recording unit

1 is a block diagram illustrating a robot cleaner using a lidar unit and an image capturing unit according to an embodiment of the present invention. 2 is a plan view showing a housing unit, a lidar unit, and an image capturing unit of a robot cleaner using a lidar unit and an image capturing unit according to an embodiment of the present invention. 3 is a plan view illustrating that the robot cleaner using the lidar unit and the image capturing unit according to an embodiment of the present invention rotates while moving to photograph the first to third front regions. 4 (a), (b), and (c) are diagrams illustrating geometric positions of feature information extracted from images obtained by photographing the first to third front regions by the image capturing unit of FIG. 3 . 5 is a plan view for explaining correction according to overlapping or spacing of the first to third front areas of the robot cleaner using the lidar unit and the image capturing unit according to an embodiment of the present invention.

Referring to FIG. 1 , a robot cleaner 100 using a lidar unit and an image capturing unit according to an embodiment of the present invention includes a housing unit 110 , a lidar unit 120 , an image capturing unit 130 , and an information correction unit ( 140 ), a positioning unit 150 , a mapping unit 160 , and a control unit 170 .

Referring to FIG. 1 , as the housing unit 110 forms an internal space, the lidar unit 120 , the image capturing unit 130 , the information correcting unit 140 , the positioning unit 150 , and the mapping unit 160 . and the control unit 170 may be accommodated.

In addition, as the one side portion (the upper part of FIG. 2 ) of the housing unit 110 is formed of a translucent material, the laser wavelength light irradiated from the lidar unit 120 passes through the one side portion to reach the object. In addition, one side of the housing 110 may pass reflected light reflected from the object.

In addition, the housing 110 is rotatable while moving as a plurality of moving wheels are formed in the lower portion. In particular, since the housing 110 cannot rotate in place, it rotates while moving in the moving direction as shown in FIG. 3 . Here, the housing 110 may be a rotating body widely used in a general robot cleaner.

The lidar unit 120 is positioned inside the housing unit 110 adjacent to one side of the housing unit 110 and is rotatable, and the front regions S1 and S2 seen through one side of the housing unit 110 in the external environment. , S3) to generate a slam map. Here, the front region includes the first front region S1 , the second front region S2 , and the third front region S3 shown in FIG. 3 . Specifically, the first front area S1, the second front area S2, and the third front area S3 are the front areas S1, S2, and S3 that are the entire area sensed by the lidar 120 in the external environment. It means an area sensed whenever the housing part 110 rotates at an angle of 120°.

That is, since the lidar unit 120 senses the front only within an angle of 120° due to a structural problem as it is located inside the housing unit 110, the housing unit 110 has 120 to sense all areas of 360°. It rotates at an angle of °.

In addition, the slam map includes a first slam map for the first front region S1, a second slam map for the second front region S2, and a third slam map for the third front region.

The lidar unit 120 generates position information and mapping information of the housing unit 110 by irradiating light of a laser wavelength to the external environment through one side of the housing unit 110 and receiving reflected light reflected from the external environment. For this purpose, the lidar unit 120 may include a laser transmitter for irradiating light of a laser wavelength and a laser receiver for receiving reflected light reflected after the light of the laser wavelength reaches an object.

In addition, the lidar unit 120 may further include a rotating plate on which the laser transmitting unit and the laser receiving unit are mounted, and a rotating shaft passing through the center of the rotating plate and fixed to the upper and lower surfaces of the housing unit 110 .

Here, the location information may be information on a location at which the current location of the housing unit 110 can be estimated, and the mapping information may be information for implementing a slam map.

The image capturing unit 130 is located inside the housing unit 110 adjacent to the inner surface of the housing unit 110 to capture an image for the rear region opposite to the front regions S1, S2, and S3, and Generate feature information. Here, the characteristic information includes first characteristic information on the first front region S1, second characteristic information on the second front region S2, and third characteristic information on the third front region S3. information about the geometrical position of In this regard, in FIG. 4 (a), geometric position information of the first characteristic information, in FIG. 4 (b), geometric position information of the second characteristic information, and in FIG. 4 (c), geometric position information of the third characteristic information is shown.

When the geometric positions of the first characteristic information, the second characteristic information, and the third characteristic information shown in (a), (b) and (c) of FIG. 4 are matched, as shown in FIG. 4(d), One complete area is formed.

The feature information generated using the above principle is transmitted to the information correction unit 140, and the cleaning map is completed by interpolating and correcting the overlapping or spaced portions between the first slim map, the second slim map, and the third slim map. make it According to this process, the present invention can generate a cleaning map that is the final position and the final slim map of the housing unit without measurement error.

The above characteristic information may be extracted and applied to the following situations.

When a spaced apart or overlapping overlapping section is formed between the first slam map and the second slam map, the second slam map and the third slam map, and the first slam map and the third slam map in the entire slim map The photographing unit 130 generates characteristic information using a characteristic selection and characteristic extraction algorithm based on PCA (Principal Component Analysis). The characteristic information generated in this way is received by the information correction unit 140 and interpolates and corrects overlapped or spaced portions between the first slam map, the second slam map, and the third slam map to help complete the cleaning map. gives The image capturing unit 130 includes a first camera 131 and a second camera 132 .

As shown in FIG. 2 , the first camera 131 is positioned at a location rotated by 120° in the lidar unit 120 with respect to the center of the housing unit 110 .

As shown in FIG. 2 , the second camera 132 is positioned at a location rotated by 120° from the first camera 131 with respect to the center of the housing 110 .

That is, the first camera 131 , the second camera 132 , and the lidar unit 120 are positioned at a position rotated at an angle of 120° with respect to the center of the housing unit 110 , and from the center of the housing unit 110 . located radially.

The information correction unit 140 estimates the position of the housing unit 110 using the slam map and the characteristic information and corrects the slam map. Specifically, the information correction unit 140 generates the entire slim map through an operation of summing the first to third slim maps.

Specifically, referring to (a), (b), (c), and (d) of FIG. 4 , the information correction unit 140 generates a cleaning map in which the slam map is corrected by matching the first to third characteristic information. complete it

Additionally, the information correction unit 140 estimates the final position from the entire slim map by generating a cleaning area mark based on an image of an object fixed in the external environment in the image.

The positioning unit 150 estimates the current position using the image and receives final position information on the final position of the housing unit 110 corrected by the information correction unit. In addition, the location check unit 150 may transmit the final location information to the display unit (not shown) or the user terminal (not shown), and accordingly, the user may check the final location information output through the display unit or the user terminal. have.

In addition, the positioning unit 150 transmits and reflects the final position information to the lidar unit 120 to minimize measurement errors.

The mapping unit 160 receives the cleaning map from the information correcting unit 140 . In addition, the mapping unit 160 may transmit the cleaning map to the display unit or a user terminal (not shown), and accordingly, the user may check the final location information output through the display unit or the user terminal.

In addition, the mapping unit 160 transmits the cleaning map to the lidar unit 120 and reflects it, thereby minimizing measurement errors.

The control unit 170 controls operations of the lidar unit 120 , the image capturing unit 130 , and the information correcting unit 140 .

Specifically, when the position display of the housing unit 110 is requested, the control unit 170 controls the moving wheel so that the housing unit 110 rotates by 120°, and the lidar unit 120 rotates the housing unit 110 . control to generate the first to third slam maps according to the

2. How to create a slam map of a robot cleaner using a lidar unit and an image capturing unit

6 is a flowchart illustrating a method of generating a slam map of a robot cleaner using a lidar unit and an image capturing unit according to an embodiment of the present invention. 7 is a flowchart illustrating detailed steps of each step of FIG. 6 .

Referring to FIG. 6 , in the method for generating a slam map of a robot cleaner using a lidar unit and an image capturing unit according to an embodiment of the present invention, (a) the front regions S1, S2, and S3 using the lidar unit 120 are Generating a slam map for at least a part (S100), (b) generating a slim map while the housing unit 110 rotates when a position indication is requested (S200) and (c) image capturing unit 130 ) to correct and complete the slam map (S300).

Referring to FIG. 7 , in step (a), (a1) the lidar unit 120 is rotated by the control unit 170 (S110) and (a2) the lidar unit 120 generates a slam map. (S120).

The step (b) includes (b1) the step of rotating the housing 110 by 120° by the control unit 170 (S210), and (b2) whether the housing 110 of the robot cleaner 100 is rotated. Determining (S220), (b3) estimating the current location in the positioning unit 150 (S230), (b4) determining whether the current location is accurate (S240) and (b5) above (c) ) to perform the steps.

In addition, the step (b) further includes, after the step (b2), (b6) returning to the step (a1) when the housing part 110 is not rotated. According to the above process, the lidar unit 120 senses the front regions S1, S2, and S3 until the housing unit 110 rotates 360°.

Meanwhile, step (b) may further include, after step (b4), (b7) returning to step (a1) when the current location is incorrect.

The step (c) is, (c1) the housing unit 110 is moved by the control unit 170 (S310), (c2) the image capturing unit 130 takes and acquires an image (S320), (c3) the step of estimating the final position of the housing unit 110 from the slam map by generating the cleaning area mark based on the image of the object fixed in the external environment by the information correction unit 140 in the image (S330), ( c4) the lidar unit 120 generates a slam map (S340) and (c5) the information correction unit 140 completes the cleaning map corrected for the slam map (S350).

According to the present invention as described above, it is possible to obtain an accurate cleaning area even when the built-in lidar unit located inside the robot cleaner is applied.

In addition, the present invention can implement a cleaning map to an approximate approximation to the real by reducing the phase error for mapping the cleaning map from the slam map and feature information obtained from the lidar unit and the image capturing unit.

The above description of the present invention is for illustration, and those of ordinary skill in the art to which the present invention pertains can understand that it can be easily modified into other specific forms without changing the technical spirit or essential features of the present invention. will be. Therefore, it should be understood that the embodiments described above are illustrative in all respects and not restrictive. For example, each component described as a single type may be implemented in a dispersed form, and likewise components described as distributed may be implemented in a combined form.

The scope of the present invention is indicated by the following claims, and all changes or modifications derived from the meaning and scope of the claims and their equivalents should be construed as being included in the scope of the present invention.

100: robot vacuum cleaner
110: housing unit
120: Riderbu
130: video recording unit
131: first camera
132: second camera
140: information correction department
150: positioning unit
160: mapping unit
170: control unit

Claims (13)

a housing part that forms an inner space, one side part is formed of a light-transmitting material, and a plurality of moving wheels are formed and are rotatable while moving;
a lidar unit located inside the housing unit adjacent to one side of the housing unit, rotatable and generating a slam map for a front area viewed through the side unit in an external environment;
an image capturing unit located inside the housing unit adjacent to the inner surface of the housing unit to capture an image of a rear area opposite to the front area and to generate characteristic information about the image;
an information correction unit for estimating a position of the housing unit using the slam map and the characteristic information and correcting the slam map;
a positioning unit estimating a current position using the image and receiving final position information on the final position of the housing unit corrected by the information correction unit;
a mapping unit for receiving a cleaning map obtained by correcting the slam map from the information correcting unit; and
A robot cleaner using a lidar unit and an image capturing unit, comprising: a control unit for controlling the operations of the lidar unit, the image capturing unit, and the information correcting unit.
According to claim 1,
The video recording unit,
a first camera positioned at a position rotated by 120° in the lidar part with respect to the center of the housing part; and
A robot cleaner using a lidar unit and an image capturing unit, comprising a; a second camera positioned at a position rotated by 120° from the first camera with respect to the center of the housing unit.
According to claim 1,
The front area includes a first front area, a second front area and a third front area,
The slam map includes a first slam map for the first front area, a second slam map for the second front area, and a third slam map for the third front area,
When the position indication of the housing part is requested,
The control unit controls the moving wheel to rotate the housing unit by 120°, and controls the lidar unit to generate the first to third slam maps according to the rotation of the housing unit. Robot vacuum cleaner used.
4. The method of claim 3,
The information correcting unit is a robot cleaner using a lidar unit and an image capturing unit, characterized in that generating the entire slim map through the operation of summing the first to third slim maps.
5. The method of claim 4,
In the entire slim map, a spaced apart interval or overlapping overlapping between the first slam map and the second slam map, the second slam map and the third slam map, and the first slam map and the third slam map When a section is formed,
The image capturing unit is a robot cleaner using a lidar unit and an image capturing unit, characterized in that it generates the feature information using a feature selection and feature extraction algorithm based on PCA (Principal Component Analysis).
6. The method of claim 5,
The characteristic information is information on a geometric position of a feature including first characteristic information for the first front region, second characteristic information for the second front region, and third characteristic information for the third front region, and ,
The information correction unit is a robot cleaner using a lidar unit and an image capturing unit, characterized in that by matching the first to third characteristic information to generate the cleaning map.
5. The method of claim 4,
The information correction unit generates a cleaning area mark based on an image of an object fixed in the external environment in the image and estimates the final position from the overall slim map. A robot cleaner using a lidar unit and an image capturing unit .
According to claim 1,
The lidar unit irradiates laser wavelength light to an external environment through one side of the housing unit and receives reflected light reflected from the external environment to generate location information and mapping information of the housing unit. Robot vacuum cleaner using wealth.
In the method for generating a slam map of the robot cleaner according to claim 1,
(a) generating a slam map for at least a portion of the front area using the lidar unit;
(b) generating the slim map while the housing unit rotates when a position indication is requested; and
(c) correcting and completing the slam map using the image capturing unit;
10. The method of claim 9,
The step (a) is,
(a1) rotating the lidar unit by the control unit; and
(a2) generating the slam map by the lidar unit;
11. The method of claim 10,
Step (b) is,
(b1) rotating the housing by 120° by the control unit;
(b2) determining whether the housing part of the robot cleaner is rotated;
(b3) estimating the current location in the location check unit;
(b4) determining whether the current location is correct; and
(b5) performing the step (c);
12. The method of claim 11,
Step (b) is,
After step (b2),
(b6) returning to the step (a1) when the housing unit does not rotate.
10. The method of claim 9,
The step (c) is,
(c1) moving the housing unit by the control unit;
(c2) capturing and acquiring the image by the image capturing unit;
(c3) estimating the final position from the slim map by generating, by the information correction unit, a cleaning area mark based on an image of an object fixed in the external environment in the image;
(c4) generating the slam map by the lidar unit; and
(c5) completing, by the information correction unit, a cleaning map obtained by correcting the slam map;

Images