KR100871115B1 - Moving robot and operating method for same - Google Patents

Abstract

The present invention relates to a mobile robot and a method of operating the same, by removing unnecessary feature points from the image data photographed during the movement and determining the position of the mobile robot based on the image data photographed. There is one effect.

Mobile Robot, Location, Camera, Image, Feature Point

Description

Mobile robot and operating method for same

1 is a view showing a mobile robot according to an embodiment of the present invention,

2 is a block diagram showing the configuration of a mobile robot according to an embodiment of the present invention;

3 is a diagram showing an example of feature point extraction according to an embodiment of the present invention;

4 is a view showing a moving distance of the feature point of the mobile robot according to an embodiment of the present invention,

5 is a view showing a moving distance and a moving rate of the feature point of the mobile robot according to an embodiment of the present invention,

6 is a view showing an operation method according to the position calculation of the mobile robot according to an embodiment of the present invention,

7 is a diagram illustrating a method for determining unnecessary feature points based on a feature point moving distance of a mobile robot according to an embodiment of the present invention; and

8 is a diagram illustrating a method for determining unnecessary feature points based on the height of feature points of a mobile robot according to one embodiment of the present invention.

<Explanation of symbols on main parts of the drawings>

100: mobile robot 10: camera

11: lens 12: image sensor

20: image control unit 30: control unit

40: display unit 50: input unit

60: data portion 70: running portion

80: cleaning unit 90: battery

The present invention relates to a mobile robot and a method of operating the same, receiving a picture of an upward image, and determining a moving position based on the image.

Recently, robots used for industrial use have been developed for home use, and robots can be used in general homes, and the field of such home robots is gradually expanding.

A typical example of such a home robot is a cleaning robot. The cleaning robot, which is a mobile robot used in the home, is a mobile robot, and is a device that cleans a certain area by inhaling dust or foreign matter while driving around a certain area by itself.

The mobile robot sucks dust and foreign substances while driving a certain area, and is equipped with a rechargeable battery, which is free to move and can move by itself using a battery's operating power, and a plurality of sensors are provided to avoid obstacles while driving. So you can drive around obstacles yourself.

Such a mobile robot has a problem in that it is difficult to check a position during movement and an error may occur due to slippage, and the like, and an error may occur in determining a current position of the mobile robot while performing a predetermined cleaning instruction or returning to a charging station.

An object of the present invention is to provide a mobile robot and its operation method by reducing the load according to the image recognition and improve the accuracy of position determination by extracting and removing unnecessary feature points for the image data input during the movement of the mobile robot.

The mobile robot according to the present invention includes a camera capturing an image of a ceiling area, an image controller converting an image signal input from the camera and extracting feature points from the photographed data, and first data input at predetermined time intervals from the image controller. And comparing the second data to match a plurality of feature points, determining and removing unnecessary feature points according to at least one of a height of the feature point and a moving ratio of the feature points calculated according to the movement of the feature points, and removing the unnecessary feature points. It includes a control unit for calculating the current position in response to the movement of.

In addition, the operation method of the mobile robot according to the present invention is to extract the feature point by taking an image including the ceiling area through the camera, matching the feature point of the first and second data photographed at a predetermined time interval, matching Calculating the moving distance of the feature points, removing unnecessary feature points according to at least one of the height of the feature point and the moving ratio of the feature points calculated according to the moving distance of the feature points, and responding to the movement of the remaining feature points except the unnecessary feature points. Calculating a position of the mobile robot.

Hereinafter, exemplary embodiments of the present invention will be described with reference to the accompanying drawings.

1 is a perspective view showing a mobile robot according to an embodiment of the present invention.

In the mobile robot 100 according to the present invention, a cleaning or driving mode is set according to a button operation provided, and while cleaning a predetermined area while inhaling dust and foreign substances while moving a predetermined area. The mobile robot 100 is equipped with a battery is supplied with operating power from the battery, when the current of the battery is insufficient to return to the charging station to charge the battery and re-perform the set cleaning when the charging is completed.

As shown in FIG. 1, the mobile robot 100 includes an input unit 50 including a plurality of buttons for inputting data, and a display unit 40 for displaying information on a current operation state and a set mode. do. The mobile robot 100 includes a plurality of sensors, and is configured to travel while avoiding obstacles during movement. The mobile robot 100 includes a traveling part provided with a plurality of wheels as a moving means for moving a predetermined area on the rear surface, and includes a cleaning part that sucks dust and foreign substances from the surrounding and bottom surfaces during the movement. The mobile robot 100 moves a predetermined region as the wheel rotates according to the operation of the motor provided therein, and performs cleaning by sucking dust around during movement.

In addition, the mobile robot 100 is provided with a camera 10 for capturing an image of the upper surface, that is, for example, the ceiling surface, the upper surface of the mobile robot 100 during the movement or stop, the image of the ceiling surface Shoot. Therefore, the mobile robot 100 calculates the moving distance of the mobile robot 100 by comparing the image data photographed at predetermined time intervals through the camera 10, and calculates the position of the mobile robot 100 accordingly. .

2 is a block diagram showing the configuration of a mobile robot according to an embodiment of the present invention.

As shown in FIG. 2, the camera 10 capturing an image of an upper portion, that is, an upper surface including a ceiling surface, includes a lens 11 for condensing ambient light and a lens 11 through the lens 11. It includes an image sensor 12 for converting the incident light into an electrical signal to take an image. At this time, the light collected by the lens 11 is input to the image sensor 12, the image sensor 12 converts the input light into an electrical signal.

The mobile robot 100 according to the image control unit 20 for converting the image data by the image sensor 12 according to a predetermined format, a plurality of sensors for detecting the obstacle, the operation control and operation setting of the mobile robot 100 Data unit 60 in which data is stored, a traveling unit 70 for rotating the plurality of wheels provided to move the mobile robot, a cleaning unit 80 for sucking and cleaning dust on the floor and surroundings, and operating power The battery 90 to supply, and a control unit 30 for controlling the mobile robot 100 in response to the input data.

Here, the camera 10 may be a lens having a wide angle of view capable of photographing all areas of the ceiling even at a predetermined position. For example, it includes a fisheye lens having an angle of view of 160 degrees or more. At this time, the image controller 20 converts the image data photographed by the camera 10 and corrects the distortion caused by the use of the fisheye lens. In particular, the distorted portion of the outer portion of the image data is removed. In addition, the image controller 20 extracts feature points from the image data.

In addition, the controller 30 classifies and matches the extracted feature points with the same feature points. If the matching feature points do not exist or if the matching feature points are less than or equal to a predetermined number, it is determined that distance calculation using the feature points is difficult, and control is performed to retake the image. At this time, by controlling the shooting interval of the camera 10 during the movement of the mobile robot 100 to be retaken. The shooting interval of the camera according to the moving distance of the mobile robot 100 is to set the shooting interval within the range that the mobile robot is moved to 1m or less, preferably to set the shooting interval to move 30cm to 70cm do.

3 is a diagram illustrating extracted feature points according to an embodiment of the present invention.

First, FIG. 3A is a view showing an image of the upper part including the ceiling surface by the camera 10. At this time, since the lens 11 is a fisheye lens, distortion occurs in the peripheral portion.

As illustrated in (b) and (c) of FIG. 3, the image controller 20 extracts feature points using a predetermined algorithm. The image processing environment of the mobile robot 100 is an embedded environment, and the feature point is extracted using a Susan corner detector in consideration of using a fisheye lens. Here, the image controller 20 may extract the feature point from an arbitrary mark, and may extract the feature point using a natural mark existing in the upper region including the ceiling surface. That is, it is possible to use what is provided in the fluorescent lamp or the ceiling surface as a mark.

The controller 30 compares the image data photographed at a predetermined time interval with respect to a plurality of feature points of the image data extracted by the image controller 20, and matches the feature points, and calculates a moving distance between the matched feature points.

At this time, the controller 30 calculates the height of the feature point based on the moving distance of the feature point, and determines that the feature point having a height less than or equal to the predetermined reference height is an unnecessary feature point. In addition, the controller 30 determines unnecessary feature points by comparing the movement amount and the movement ratio with respect to the movement distance of the feature points corresponding to the distances from the center point of the photographed image data to the feature points.

Figure 4 is a diagram showing the moving distance of the feature point of the mobile robot according to an embodiment of the present invention.

As shown in FIG. 4, when comparing the first mark C1 and the second mark N1, the mobile robot 100 moves, through the camera 10, the first time k and the second time. Each of the upper surfaces including the ceiling surface is photographed at (k + 1). At this time, the first mark C1 and the second mark N1 in the image data photographed at the first time k and the second time k + 1 are the first feature point M1 and the second feature point M2. Will be displayed as).

At this time, the first mark C1 and the second mark N1 at the first time k are the first feature points M1 (k) and the second feature points M2 (k, as shown in FIG. Appears as)). Meanwhile, the first mark C1 and the second mark N1 at the second time k + 1 have the first feature point M1 (k + 1) and the second feature point as shown in FIG. Appears as (M2 (k + 1)).

The control unit 30 has a first feature point M1 (k) and a second feature point M2 (k) of the first time k, which are the feature points of the image data for the first mark C1 and the second mark N1. ) And the first feature point M1 (k + 1) and the second feature point M2 (k + 1) of the second time k + 1, respectively, are matched and compared to calculate the moving distance between the feature points. . That is, as shown in (c) of FIG. 4, the feature points of the image data photographed at the first time k and the second time k + 1 are compared with each other, and the moving distance PL1 and the first feature point M1 are first compared. The moving distance PL2 of the two feature points M2 is calculated.

At this time, the moving distance (L1) between the first mark (C1) and the second mark (N1) and the mobile robot 100 according to the movement of the mobile robot 100 is constant and the actual moving distance of the mobile robot 100 same. However, PL1 and PL2, which are moving distances of the feature points of the captured image data, are different from each other. This is because the moving distance between the mobile robot 100, the first mark C1, and the second mark N1 is different, and distortion by the fisheye lens occurs. The farther the mark is, the smaller the moving distance of the feature point relative to the mark, while the closer the mark is to the mobile robot, that is, the lower the ceiling, the greater the moving distance of the feature point.

Accordingly, when the moving ratio of the feature point according to the movement of the mobile robot 100 is greater than or equal to the predetermined ratio as in the second mark N1, the controller 30 determines that it is an unnecessary feature point and removes it.

In addition, since the first mark C1 of the first mark C1 and the second mark N1 is located on the ceiling, the controller 30 adjusts the distance from the mobile robot 100 to the ceiling. The height of the second mark N1 is calculated. If the calculated height is less than or equal to the reference height, the controller 30 determines that it is an unnecessary feature point and removes it.

At this time, when comparing the first feature point M1 and the second feature point M2, for each movement distance, the movement distance PL1 of the first feature point M1 and the movement distance PL2 of the second feature point M2 are [ 1 and the second mark, the actual moving distance / height of the mobile robot. In this case, since the actual moving distance is the same and the height of the first feature point M1 is the ceiling height, the height of the second feature point M2, that is, the second mark N1, can be calculated. Here, it is preferable to determine that the feature point located in the center portion of the captured image data is located on the ceiling surface, and the height of the feature point is calculated based on the feature point located in the center portion.

5 is a diagram illustrating a moving distance and a moving ratio of a feature point of a mobile robot according to an embodiment of the present invention.

5 (a) and 5 (b) show only a part of the captured image data, wherein (a) and (b) are image data captured at predetermined time intervals.

FIG. 5C is a graph showing the moving distance of the feature point in proportion to the distance from the center point of the photographed image data to the feature point.

As the distance from the center point of the image data to the feature point increases, the moving distance of the feature point increases. Distortion becomes more severe toward the outside of the image of the image data, and the distance of the feature point is increased due to the distance from the mobile robot 100 when the feature point is located on the wall.

Accordingly, the controller 30 determines that the feature point is located below the reference height, or removes the feature point if it is determined to be an unnecessary feature point when the moving distance of the feature point is greater than or equal to the reference ratio. After removing unnecessary feature points, the controller 30 determines the position and the movement amount of the actual mobile robot 100 by comparing the image data. Here, since the moving distance of the feature point is rapidly increased with respect to the distance away from the center of the image, it is preferable to calculate the change rate with respect to the moving distance and set it as the reference ratio.

In other words, if the moving distance of the mobile robot 100 is based on 30cm, when the height of the mark located on the actual ceiling or wall surface is 273cm, 207cm, 73cm as shown in [Table 1], The moving distances of the feature points are 19 pixels, 26 pixels, and 106 pixels. The ratio is 1.62, 1.15, 0.28fh. At this time, since the mark and the feature point of 273cm height is a normal point displayed on the ceiling surface, the reference ratio is set within 1.62 or less and over 1.15. At this time, the reference height is preferably set within the range of 273 or less and more than 207.

Height Pixels difference in the image Cm / Pixel 273 Cm 19 Pixels 1.62 Cm 207cm 26 Pixels 1.15 Cm 73cm 106 Pixels 0.28 Cm

Therefore, it is preferable that the controller 30 sets the reference value to the minimum moving distance + (minimum moving distance x α) of the feature points. In this case, the alpha is 0.3, for example, but may vary according to the angle of view of the lens. That is, the reference ratio is preferably set to 16 + (16 * 0.3) = 20.8.

6 is a view showing an operation method according to the position calculation of the mobile robot according to an embodiment of the present invention.

The mobile robot 100 drives the camera 10 to capture an image of an upper portion including the ceiling surface (S310). In this case, the image data photographed by the camera 10 is generated by being converted by the image controller 20 by the light collected by the lens 11 is incident to the image sensor 12 and converted into an electrical signal. The image controller 20 corrects the distortion of the peripheral portion due to the wide angle of view of the lens 11 (S320) and extracts the feature points (S330). The feature point extraction method uses a Susan corner detector as an example.

The controller 30 compares the image data photographed at predetermined intervals, compares the feature points of the image data of the first time with the feature points of the image data of the second time, and matches the same feature points (S340). The controller 30 calculates a moving distance according to the time difference between the matching feature points (S350), calculates a moving ratio based on the moving distance of the feature point with respect to the distance from the center point of the image data to the feature point, and compares it with the reference ratio. (S360).

In this case, when the moving ratio of the feature point is larger than the reference ratio, it is determined that the feature point is unnecessary and the feature point is removed (S370). Here, in the case of unnecessary feature points, a reference value for unnecessary feature points may be changed and set based on the feature points (S380).

When unnecessary feature points are removed by repeating the comparison of the image data and the feature points as described above, the actual moving distance and position of the mobile robot 100 is calculated based on the moving distance between the remaining feature points. At this time, in the case of the moving distance according to the wheel rotation of the mobile robot 100, an error may occur due to the sliding, and calculates the moving distance and position of the mobile robot 100 based on the moving distance of the feature point.

7 is a diagram illustrating a method for determining unnecessary feature points based on a moving distance of feature points of a mobile robot according to one embodiment of the present invention.

As described above, the image of the ceiling surface is photographed to extract a feature point (S410), and the feature point of the image data of the first time is compared with the feature point of the image data of the second time (S420). In this case, when there is no feature point that matches each other or when the matching feature point is less than or equal to a predetermined number (S430), an error is generated to be photographed again (S480). At this time, the image is taken so that the moving distance of the mobile robot 100 is less than 1M. Preferably, the shooting interval is set to move 30cm to 70cm.

On the other hand, when there is a matching specific point, the moving distance between the feature points is calculated (S440), and the moving distance of the feature points is compared with the reference moving distance (S450). If the moving distance of the feature point exceeds the reference moving distance, it is determined that the feature point is unnecessary and the feature point is removed (S460). On the other hand, if the moving distance of the feature point is less than the reference moving distance it is determined as a feature point necessary for the position determination of the mobile robot (100) (S470).

When the unnecessary feature point is removed as described above, the position of the mobile robot 100 is calculated as shown in FIG. 6.

8 is a diagram illustrating a method for determining unnecessary feature points based on the feature point height of a mobile robot according to an embodiment of the present invention.

As shown in FIG. 7, the image of the ceiling surface is photographed to extract a feature point (S510), and the feature point of the image data of the first time is compared with the feature point of the image data of the second time (S520). In this case, when there is no feature point to be matched with each other or when the number of feature points to be matched is equal to or less than a predetermined number (S530), an error is generated to be photographed again (S580). At this time, the image is taken so that the moving distance of the mobile robot 100 is less than 1M. Preferably, the shooting interval is set to move 30cm to 70cm.

On the other hand, if there is a matching specific point, the moving distance between the feature points is calculated, and the height of the feature point is calculated through comparison with the feature points located on the ceiling surface (S540). At this time, it is determined that the feature point located at the center of the captured image data is located at the ceiling surface. In addition, the height of the feature point is compared with the reference height (S550). In this case, when the height of the feature point is less than the reference height, it is determined that the feature point is unnecessary and the feature point is removed (S560). On the other hand, if the height of the feature point is more than the reference height is determined as the feature point necessary for the position determination of the mobile robot (100) (S570). In other words, it is determined to be a feature point located on the ceiling surface.

When the unnecessary feature point is removed as described above, the position of the mobile robot 100 is calculated as shown in FIG. 6.

Since the mobile robot and its operation method calculate the position of the mobile robot after removing the unnecessary feature points as described above, the operation of unnecessary data is reduced and the load according to the image processing of the mobile robot is reduced and the processing speed accordingly. Is improved to enable fast position calculation.

Although the mobile robot and its operation method according to the present invention have been described with reference to the illustrated drawings as described above, the present invention is not limited by the embodiments and drawings disclosed herein, and may be applied within the scope of the technical idea. Can be.

The mobile robot and its operation method according to the present invention configured as described above extract the feature points from the photographed image data and calculate the position of the mobile robot after removing the unnecessary feature points, thereby improving the accuracy and position calculation speed of the position calculation. It is easy to determine the position of the mobile robot.

Claims (10)

A camera for taking an image of the ceiling area; An image controller converting an image signal input from the camera and extracting feature points from the photographed data; Matching a plurality of feature points by comparing the first data and the second data input from the image controller at predetermined time intervals, and unnecessary according to at least one of the height of the feature point and the moving ratio of the feature point calculated according to the movement of the feature point. And a controller for determining and removing a feature point and calculating a current position in response to movement of the feature point except for the unnecessary feature point. The method of claim 1, The camera comprises a lens unit forming an angle of view of 160 degrees or more; And an image sensor provided at an upper portion of the mobile robot and configured to capture an image of an upper portion of the mobile robot including a ceiling area. The method of claim 1, The controller calculates a height according to a moving distance of a feature point separated by a predetermined distance or more from the center portion based on the movement distance and the ceiling height of the feature point located in the central portion of the first and second data, and the calculated height is a reference height. A mobile robot, characterized in that the following feature points are determined as unnecessary feature points and removed. The method of claim 1, The control unit may determine that the feature point having a moving ratio greater than or equal to a certain ratio according to the moving distance of the feature point is removed as an unnecessary feature point corresponding to the distance from the center to the feature point among the plurality of feature points included in the first data and the second data. Mobile robot. The method of claim 1, The control unit determines that there is no matching feature point between the first and second data or when the matching feature point is less than or equal to a predetermined number, it is determined that the data is an error, and the next photographed data is compared with each other. Extracting a feature point by capturing an image including a ceiling area through a camera; Matching feature points of the first and second data photographed at predetermined time intervals; Calculating a moving distance of the matched feature point and removing unnecessary feature points according to at least one of a height of the feature point and a moving ratio of the feature point calculated according to the movement distance of the feature point; And Calculating a position of the mobile robot in response to the movement of the remaining feature points, except for unnecessary feature points. The method of claim 6, The removing of the unnecessary feature points may be performed based on a moving distance of the feature points located in the center portion of the first and second data and a ceiling height, and calculating a height according to the movement distance of the feature point away from the center portion by a predetermined distance. If the height is less than the reference height operating method of the mobile robot, characterized in that it is determined as an unnecessary feature point. The method of claim 6, The removing of the unnecessary feature points corresponds to the distance from the center to the feature point among the plurality of feature points included in the first and second data, and determines that the feature points are unnecessary when the moving ratio of the feature points is greater than or equal to a certain ratio. Mobile robot operating method characterized in that. The method of claim 6, Determining that an error has occurred when the feature points of the first and second data do not match or the matching feature points are less than or equal to a predetermined number, and calculating a position based on the next photographed data. Mobile robot operation method. The method of claim 9, If an error occurs, the method of operation of the mobile robot further comprises the step of changing the shooting interval.

Images