KR101400400B1 - Robot cleaner and control method of the same - Google Patents

Abstract

The robot cleaner of the present invention comprises: a first camera and a second camera which are spaced apart from each other at a predetermined distance to look ahead; And a common feature point corresponding to a predetermined pic- ture point in the cleaning zone through comparison between a first image and a second image respectively photographed by the first camera and the second camera, And a control unit for calculating a distance to the subject point based on a positional difference between the feature points in the second image.

Description

BACKGROUND OF THE INVENTION 1. Field of the Invention [0001] The present invention relates to a robot cleaner and a robot cleaner,

The present invention relates to a robot cleaner and a control method of the robot cleaner.

The robot cleaner is a device for automatically cleaning foreign objects such as dust from the floor surface while traveling in a self-cleaning area without user's operation.

Generally, such a robot cleaner senses distances to obstacles such as furniture, office supplies, and walls installed in the cleaning area, maps the cleaning area accordingly, or controls the driving of the left and right wheels to perform an obstacle avoidance operation. Conventionally, the distance that the robot cleaner moves through the sensor that observes the ceiling or the floor is measured, and the distance to the obstacle is calculated based on the distance. However, this method calculates the distance to the obstacle based on the moving distance of the robot cleaner The distance to the obstacle is also inevitably inevitable if the moving distance of the robot cleaner can not be accurately measured due to the bending of the floor or the like.

SUMMARY OF THE INVENTION It is an object of the present invention to provide a robot cleaner and a method of controlling the robot cleaner that can accurately measure distances to an obstacle or a picocentric point in a cleaning area through an image captured by a camera.

Another object of the present invention is to provide a robot cleaner and a robot cleaner control method capable of measuring the distance from the current position to an obstacle in real time.

Another object of the present invention is to provide a robot cleaner and a robot cleaner control method capable of three-dimensionally mapping positions of obstacles in a cleaning area.

The robot cleaner of the present invention comprises: a first camera and a second camera which are spaced apart from each other at a predetermined distance to look ahead; And a common feature point corresponding to a predetermined pic- ture point in the cleaning zone through comparison between a first image and a second image respectively photographed by the first camera and the second camera, And a control unit for calculating a distance to the subject point based on a positional difference between the feature points in the second image.

The control method of the robot cleaner according to the present invention is a control method of a robot cleaner including a first camera and a second camera which are spaced apart from each other by a predetermined distance to look ahead, An image acquiring step of acquiring a first image and a second image; A feature point detection step of detecting common feature points corresponding to a predetermined pic- ture point in a subject area through comparison between the first image and the second image; And a distance calculating step of calculating a distance to the subject point based on a positional difference between the characteristic point in the first image and the characteristic point in the second image.

The robot cleaner and the control method of the robot cleaner of the present invention have the effect of more accurately measuring the distance to the object or the obstacle in the cleaning area.

In addition, the robot cleaner and the robot cleaner control method of the present invention have the effect of measuring the distance to the obstacle in real time.

In addition, the robot cleaner and the robot cleaner control method of the present invention are capable of grasping the state of an obstacle in the cleaning area before the robot cleaner moves.

1 is a perspective view illustrating a robot cleaner according to an embodiment of the present invention.
2 is a front view of the robot cleaner.
3 is a bottom view of the robot cleaner.
Fig. 4 shows the configuration of the main part of the robot cleaner.
Fig. 5 shows the control relationship of each part of the robot cleaner.
FIG. 6 is a flowchart illustrating a process of calculating a distance from a robot cleaner to a target point according to an embodiment of the present invention.
FIG. 7 is a flowchart illustrating the minutia detection step of FIG. 6; FIG.
FIG. 8 shows a process of detecting a corner using the FAST corner detection algorithm.
FIG. 9 shows a first image (a) and a second image (b) in which corners are detected and a result (a ') and (b') in which a feature point detection step is performed on both images.
10 is a diagram for explaining a method of calculating a distance to a feature point through a parallax between images taken by the first camera and the second camera.
11 is a diagram showing that a result of distance calculation between subject points is mapped.
12 shows the correlation between the measurement distance to the subject point and the error according to the distance (T, see Fig. 10) between the first camera and the second camera.

BRIEF DESCRIPTION OF THE DRAWINGS The advantages and features of the present invention and the manner of achieving them will become apparent with reference to the embodiments described in detail below with reference to the accompanying drawings. The present invention may, however, be embodied in many different forms and should not be construed as limited to the embodiments set forth herein. Rather, these embodiments are provided so that this disclosure will be thorough and complete, and will fully convey the scope of the invention to those skilled in the art. To fully disclose the scope of the invention to those skilled in the art, and the invention is only defined by the scope of the claims. Like reference numerals refer to like elements throughout the specification.

1 is a perspective view illustrating a robot cleaner according to an embodiment of the present invention. 2 is a front view of the robot cleaner. 3 is a bottom view of the robot cleaner.

1 to 3, a robot cleaner according to an embodiment of the present invention includes a main body 11 and a pair of cameras 20 and 30 for observing the front of the main body 11. The first camera 20 and the second camera 30 are disposed at a certain distance from each other and can be supported by the main body 11. [

The main body 11 moves a region to be cleaned (hereinafter referred to as a cleaning region) as the left and right wheels 61a and 62b rotate, and the dust and waste in the cleaning region Inhale foreign substances.

The main body 11 is moved in accordance with the rotation of the left wheel 61a and / or the right wheel 62a and may further be provided with auxiliary wheels 13 for stably supporting the main body 11. [ The left wheel 61a and the right wheel 62a are rotated by the left wheel drive portion 61 (see FIG. 5) and the right wheel drive portion 62 (see FIG. 5), respectively. The operation of the left and right wheel driving portions 61 and 62 is controlled independently by the control of the control portion 50 so that the main body 11 is advanced or retracted or turned. For example, when the left wheel is rotated in the forward direction by the left wheel drive unit 61 and the right wheel 62a is rotated in the reverse direction by the right wheel drive unit 62, the main body 11 is rotated to the left or right. The control unit 50 can control the rotational speed of the left wheel drive unit 61 and the right wheel drive unit 62 to be different from each other so as to induce the translational motion of the main body 11 serving also as a rectilinear motion and a rotational motion. The movement of the main body 11 through the control of the controller 50 enables avoidance or turning of the obstacle.

The suction unit 70 is detachably mounted on the main body 11 and includes a suction fan 71 which is rotated through control under the control of the control unit 50 to generate a suction force and an air flow generated by rotation of the suction fan 71 And may include an intake port 72 to be sucked in. The suction unit 70 may further include a filter (not shown) for collecting foreign substances in the air stream sucked through the suction port 72 and a foreign matter receiver (not shown) in which foreign substances collected by the filter are accumulated have. The suction unit 70 may be detached from the main body 11 to replace the filter or discard foreign matter collected in the foreign matter receiver.

Fig. 4 shows the configuration of the main part of the robot cleaner. Fig. 5 shows the control relationship of each part of the robot cleaner.

The first camera 20 and the second camera 30 capture images in front of the robot cleaner and include digital image sensors 21 and 31. The image sensors 21 and 31 have a plurality of image pickup elements arranged in a lattice shape in response to light passing through a lens (not shown). Each of the image pickup elements responds to light and includes information such as brightness, A digital signal is generated, and the digital image is reconstructed through an image processing process. The image sensors 21 and 31 may be CCD (Charge-Coupled Device) image sensors or CMOS (Complementary Metal-Oxide-Semiconductor) image sensors.

An embedded board 42 such as a PCB (Printed Circuit Board) on which various electric or electronic elements for controlling the robot cleaner are mounted is provided in the main body 11, and the first camera 20 and the second camera 30 send and receive signals to and from the embedded board 42 through a USB hub (Universal Serial Bus Hub) 41. The image captured by the image sensors 21 and 31 of the first camera 20 and / or the second camera 30 is transmitted as a digital data signal through the USB hub 41.

The control unit 50 may include a processor capable of processing digital data, and may be embodied as an integrated circuit of a CPU, or an electronic device, for example, embedded in the embedded board 42.

The control unit 50 controls the entire operation of the robot cleaner and processes the digital data transmitted through the USB hub 41 so that the image captured by the first camera 20 and the image captured by the second camera 30 Based on the parallax between the photographed images, the distance of common feature points in both images is calculated.

FIG. 6 is a flowchart illustrating a process of calculating a distance from a robot cleaner to a target point according to an embodiment of the present invention. FIG. 7 is a flowchart illustrating the minutia detection step of FIG. 6; FIG. FIG. 8 shows a process of detecting a corner using the FAST corner detection algorithm. FIG. 9 shows a first image (a) and a second image (b) in which corners are detected and a result (a ') and (b') in which a feature point detection step is performed on both images.

Referring to FIG. 6, a method of controlling a robot cleaner according to an exemplary embodiment of the present invention includes: acquiring a first image and a second image respectively captured by a first camera 20 and a second camera 30; A feature point detection step (S20) of detecting a common feature point corresponding to a predetermined focus point in the subject region through comparison between the first image and the second image, And a distance calculating step (S30) of calculating a distance to the subject point based on a positional difference of the characteristic points in the second image.

Hereinafter, an image photographed by the first camera 20 is referred to as a first image, and an image photographed by the second camera 30 is referred to as a second image.

In the image acquiring step (S10), images of the front of the robot cleaner are taken by the first camera (20) and the second camera (30). Preferably, it is preferable that the first camera 20 and the second camera 30 photograph at the same time.

However, according to the embodiment, when the robot cleaner is stopped while the movement of the robot cleaner is stopped, there may be a difference in the time of shooting of both the cameras 20 and 30. In this case, since the cleaning area photographed by the cameras 20 and 30 is mainly in a static state, even if there is a slight difference between the shooting times of the two cameras 20 and 30, And substantially the same image is obtained as in the case of performing.

At least a part of the angle of view of the first camera (20) and the angle of view of the second camera (30) overlap each other. Accordingly, the first image and the second image may have a common feature point corresponding to a predetermined pic- ture point (any one point in the actual cleaning area) in the subject area of the first camera 20 and the second camera 30 (Hereinafter referred to as a common feature point), a common feature point in the first image, and a common feature point in the second image. It is preferable that each lens of the first camera 20 and the second camera 30 is a wide angle lens having a square angle of 60 to 80 degrees.

The common feature point may be specified through comparison in all the regions of the first image and the second image, but first, a predetermined candidate region is selected from the first image and the second image (S21) (S22), and the same corner among the detected corners is defined as a minutiae (S23). Thus, the time required for searching for the minutiae and comparing the minutiae can be reduced.

Examples of methods for detecting a corner from an image captured through a camera include a Harris corner detection algorithm, a Susan corner detection algorithm, and a FAST (Features from Accelerated Segment Test) corner detection algorithm .

The FAST corner detection algorithm is a fast corner detection algorithm proposed by Edward Rosten, which performs detection several times to several times faster than the Harris or Susan corner detection algorithm.

The candidate region selection step S21 is a step of selecting candidates having a high possibility of corner detection in the first and second images and defining candidate regions to which the candidate points belong. For example, when using the FAST corner detection algorithm, a predetermined point P in each image and surrounding pixels (which need not be all 16 pixels compared in the FAST corner detection process described later, , And when the difference is equal to or more than the threshold value, the point P is defined as a candidate point, and a region having a predetermined size to which the candidate point P belongs is selected as a candidate region . According to the embodiment, the first image and the second image may be divided into N equally vertically by M equally horizontally, and the regions having the candidate points among the M x N regions may be selected as candidate regions.

The corner detecting step S22 is a step of detecting the corners by comparing the brightness of the pixels within the candidate region selected in the candidate region selecting step S21. Here, a corner is a pixel showing a threshold characteristic when compared with neighboring pixels, and is hereinafter referred to as a neighboring pixel and a threshold value (a threshold value in the candidate region selection step does not necessarily have to be the same value) Pixel.

 8, the FAST corner detection algorithm selects an arbitrary corner candidate point P on the candidate region a, compares the brightness difference between the candidate point P and the surrounding 16 pixels with a threshold value, Is determined. That is, when the number of pixels satisfying the following expression among 16 pixels around the candidate point P is equal to or larger than a predetermined number (for example, 7), the candidate point P is determined as a corner.

또는

or

I _{p :} brightness at the candidate point, In: brightness of the n-th pixel around the candidate point (P), K:

Here, the brightness (I _P , I _n is applied to the control unit 50 as a digitized value according to the detection result of each imaging element on the image sensors 21 and 31.

The feature point detection step S20 is a step of detecting common features, that is, the same corner (hereinafter, referred to as a minutiae point) by comparing candidate areas in which corners are detected in the first and second images, respectively.

Since the corners are detected respectively in the first and second images, the corners in each image may not completely coincide with each other. Therefore, in the minutia detection step S20, the corners in each image are compared to detect the same corner.

In this process, the brightness of each corner detected in the corner detection step S22 is compared with the surrounding pixels (for example, 9), and when the difference is equal to or greater than the threshold value K ' It is defined as minutiae. A brightness change between a corner and a surrounding pixel is defined as a horizontal direction X and a vertical direction Y and when the change value (for example, taxicab distance) is equal to or greater than the threshold value K ' (S23, minutiae point selection step).

In FIG. 9, corners or feature point candidates detected by the FAST corner detection algorithm are displayed in the first image (a) and the second image (b), wherein a certain corner in the candidate region (D) ) And exist only in the second image (b). These corners should be excluded in defining the common feature points of the first image (a) and the second image (b).

9 (a ') and 9 (b') show the first image (a ') and the second image (b') after the feature point detection step (S20) It can be seen that the corners having weaker threshold characteristics than the surrounding pixels are also erased in the first image a and / or the second image b, as well as being excluded from the image b '.

The distance calculating step S30 is a step of calculating a distance to a pic- ture point based on a parallax for at least one feature point selected in the feature point detecting step S20. Here, the parallax with respect to the minutiae indicates the parallax between the first camera 20 and the second camera 30 with respect to the picocentric point corresponding to the minutiae point (point in the actual requested area) 30 (the distance between the center of the lens and the center of the lens).

Referring to Fig. 10, the distance Z to the pic- ture point can be obtained by the following equations based on the triangulation method.

T: (the distance between the main axis of the first camera (O _l), and the major axis of the second camera (O _r)), the first camera 20, the main shaft drive is arranged on the left side of the main body 11, a second camera ( 20 are arranged on the right side, and the main axes of both cameras are parallel to each other.

c _left , x: focus of the first camera

c _right , x: focus of the second camera

f: focal length

x ^l : The x-component of the feature point coordinate value on the fixed coordinate based on the focus in the first camera

x ^r : The x-component of the feature point coordinate value on the fixed coordinate based on the focus in the second camera

Referring to FIG. 11, the controller 50 three-dimensionally maps the positions of the subject points P1, P2, and P3 based on the distance values up to the subject points obtained through the process described above . For example, the control unit 50 may control the left and right wheel driving units 61 and 62 based on the mapping result to calculate the position of the obstacle in the robot cleaner Movement or obstacle avoidance drive can be controlled.

12 shows the correlation between the measurement distance to the subject point and the error according to the distance (T, see Fig. 10) between the first camera and the second camera.

When the distance between the first camera 20 and the second camera is 7 cm and the case where the distance between the first camera 20 and the second camera is 8 cm is compared with the case where the distance between the first camera 20 and the second camera is 8 cm, And especially when the measurement distance is more than a certain distance, it can be seen that there is a considerable error with the actual distance.

Assuming that the upper limit of the error is about 5 to 10 mm in order to ensure the accuracy of the operation of the robot cleaner, when the main axis distance is 8 cm, it is found that an error of 5 mm or more occurs near 2 meters or less and approaches the upper limit of the error have. Therefore, the distance between the first camera 20 and the second camera 30 is preferably 7 to 8 cm.

Meanwhile, the robot cleaner control method of the present invention can be implemented as a processor-readable code on a processor-readable recording medium provided in the robot cleaner. Such a recording medium may be such that data of a format that can be read by the processor is stored. For example, it may be a ROM, a RAM, a CD-ROM, a magnetic tape, a floppy disk, an optical data storage device, or the like, and may be implemented as a carrier wave such as transmission over the Internet. In addition, the computer readable recording medium may be distributed over a networked computer system so that the code may be stored and executed.

Claims (16)

A first camera and a second camera spaced apart from each other at regular intervals to look ahead; And
A common feature point corresponding to a predetermined pic- ture point in the cleaning area is detected through comparison between the first image and the second image respectively photographed by the first camera and the second camera, And a control unit for calculating a distance to the subject point based on a positional difference of the feature points in the second image,
Wherein,
Wherein at least one candidate point whose brightness difference with respect to surrounding pixels is equal to or greater than a first threshold value is defined in each of the first image and the second image, a candidate region having a predetermined size to which the candidate point belongs is selected, When a number of pixels having a brightness difference between the candidate point and the candidate point is equal to or greater than a second threshold value, the candidate point is detected as a corner, and a brightness change between the corner and surrounding pixels And detects the corresponding corner as the minutiae when the threshold value is greater than the threshold value. delete delete The method according to claim 1,
Wherein the corner is selected by a FAST corner detection algorithm. The method according to claim 1,
Wherein the distance between the first camera and the second camera has a constant value. 6. The method of claim 5,
Wherein the first camera and the second camera are disposed in the left and right directions. 6. The method of claim 5,
And the distance between the first camera and the second camera is 7 to 9 cm. The method according to claim 1,
Wherein the main axis of the first camera and the main axis of the second camera are parallel to each other. The method according to claim 1,
Wherein the first camera and the second camera respectively comprise:
And a lens having a square angle of 60 to 80 degrees. The method according to claim 1,
Wherein,
And selects the corner as the feature point when the change in brightness along the horizontal direction and the vertical direction between the corner and surrounding pixels is equal to or greater than the threshold value. A method for controlling a robot cleaner comprising a first camera and a second camera which are spaced apart from each other at regular intervals to look ahead,
An image acquiring step of acquiring a first image and a second image respectively photographed by the first camera and the second camera;
A feature point detecting step of detecting a common feature point corresponding to a predetermined point of interest in the subject region through comparison between the first image and the second image; And
And a distance calculating step of calculating a distance to the subject point based on a positional difference between the characteristic point in the first image and the characteristic point in the second image,
The feature point detection step may include:
A candidate region selection step of defining at least one candidate point in each of the first image and the second image having a difference in brightness with respect to neighboring pixels equal to or greater than a first threshold value and selecting a candidate region having a predetermined size to which the candidate point belongs;
A corner detecting step of detecting a candidate point as a corner when the number of pixels whose difference in brightness from the candidate point among the n pixels around the candidate point in the candidate region is equal to or greater than a second threshold value; And
And a feature point selecting step of selecting the corner as the feature point when the brightness change between the corner and surrounding pixels is equal to or greater than a threshold value. 12. The method of claim 11,
Wherein the corner detecting step comprises:
And the corner is detected by the FAST corner detection algorithm. 12. The method of claim 11,
Wherein the distance calculating step includes:
A distance between the center of the lens of the first camera and the center of the lens of the second camera and a distance between the center of the lens of the first camera and the center of the lens of the second camera, Wherein the distance between the feature point coordinate value and the feature point coordinate value of the second image is calculated as a variable on a fixed coordinate centered on the focus of the second camera. 12. The method of claim 11,
Further comprising mapping the positions of the pause points in the cleaning area based on the distances calculated in the distance calculating step. 15. The method of claim 14,
And an obstacle avoidance driving step of controlling the driving of the right wheel and the left wheel according to the mapped result. 12. The method of claim 11,
The feature point selection step may include:
And selecting the corner as the feature point when the change in brightness along the horizontal direction and the vertical direction between the corner and surrounding pixels is equal to or greater than the threshold value.

Images