WO2019237433A1

WO2019237433A1 - Method and system for calibrating coordinates of camera and odometer of cleaning robot

Info

Publication number: WO2019237433A1
Application number: PCT/CN2018/093454
Authority: WO
Inventors: 王声平; 张立新
Original assignee: 深圳市沃特沃德股份有限公司
Priority date: 2018-06-15
Filing date: 2018-06-28
Publication date: 2019-12-19
Also published as: CN108827342A

Abstract

A method and system (1) for calibrating coordinates of a camera and an odometer of a cleaning robot. The method establishes a coordinate system with the center of a QR code as a coordinate origin. A central axis of a cleaning robot is located on a yz-axis plane of the QR code. The method comprises: obtaining a relative position relationship between a camera and the center of a QR code (S101); obtaining a relative position relationship between an odometer and the center of the QR code (S102); and obtaining a relative position relationship between the camera and the odometer according to the relative position relationship between the camera and the center of the QR code and the relative position relationship between the odometer and the center of the QR code (S103). The calibration of the relative position relationship between the camera and the odometer aims at resolving the problem of large deviation of fused position information due to that an existing cleaning robot does not take the relative position relationship between a camera and an odometer into consideration.

Description

Coordinate calibration method and system for camera and odometer of sweeping robot

Technical field

The present invention relates to the technical field of cleaning robots, and in particular, to a method and a system for calibrating a camera and an odometer coordinate of a cleaning robot.

Background technique

With the maturity of vision SLAM technology, monocular-based vision positioning technology is gradually applied to sweepers due to its low price and high positioning accuracy. In order to ensure the stability and positioning accuracy of the cleaning robot positioning system, the odometry information of the cleaning robot itself is generally integrated with the vision. Because the position of the camera and the center of the odometer are often not together, if the relative position relationship between them is not considered, the position information obtained by the fusion will have a large deviation.

technical problem

In view of the shortcomings of the prior art, the present invention proposes a method and system for calibrating the camera and odometer coordinates of a cleaning robot, according to the relative position relationship between the camera and the two-dimensional code center, and the relative position between the odometer and the two-dimensional code center. The relationship is to obtain the relative position relationship between the camera and the odometer, in order to solve the problem that the existing cleaning robot does not consider the relative position relationship between the camera and the odometer, resulting in a large deviation in the position information obtained by the fusion.

Technical solutions

The technical solution proposed by the present invention is:

A method for calibrating a camera and an odometer coordinate of a cleaning robot. A coordinate system is established by taking a center of a two-dimensional code as a coordinate origin, and a center axis of the cleaning robot is located on a yz-axis plane of the two-dimensional code. The method includes:

Obtaining a relative position relationship between a camera and the center of the two-dimensional code;

Obtaining a relative position relationship between the odometer and the two-dimensional code center;

Obtaining the relative position between the camera and the odometer according to the relative position relationship between the camera and the two-dimensional code center and the relative position relationship between the odometer and the two-dimensional code center relationship.

Further, the step of obtaining a relative position relationship between the camera and the center of the two-dimensional code includes:

The relative position relationship between the camera and the center of the two-dimensional code is obtained through two-dimensional code positioning. The two-dimensional code positioning includes line detection, quadrilateral detection, calculation of a homography matrix, and external parameters.

Further, before the step of obtaining a relative position relationship between a camera and the center of the two-dimensional code, the method includes:

Detecting whether the two-dimensional code exists in an image captured by the camera;

If it does not exist, rotate the cleaning robot in place until the two-dimensional code exists in the image captured by the camera.

Further, the step of rotating the cleaning robot in place until the two-dimensional code exists in an image captured by the camera includes:

Rotate the cleaning robot in place at a preset rotation angle until the two-dimensional code exists in an image captured by the camera.

Further, before the step of obtaining a relative position relationship between the camera and the center of the two-dimensional code, the method includes:

Transforming the coordinate system of the cleaning robot so that the three-axis direction of the coordinate system of the cleaning robot is the same as the three-axis direction of the center of the two-dimensional code.

The present invention also provides a camera and odometer coordinate calibration system of a sweeping robot. A coordinate system is established by taking the center of a two-dimensional code as a coordinate origin. The center axis of the sweeping robot is located on the yz-axis plane of the two-dimensional code. The system includes :

A first acquisition module, configured to acquire a relative position relationship between a camera and the center of the two-dimensional code;

A second acquisition module, configured to acquire a relative position relationship between the odometer and the two-dimensional code center;

A third obtaining module, configured to obtain the camera and the two-dimensional code according to a relative positional relationship between the camera and the two-dimensional code center and a relative positional relationship between the odometer and the two-dimensional code center Relative position relationship between odometers.

Further, the two-dimensional code positioning includes line detection, quadrilateral detection, calculation of a homography matrix and external parameters, and the first acquisition module includes:

The first sub-acquisition module is configured to obtain a relative position relationship between a camera and a center of the two-dimensional code by using two-dimensional code positioning.

Further, the system includes:

A detection module, configured to detect whether the two-dimensional code exists in an image captured by the camera;

A rotation module is configured to rotate the cleaning robot in place if there is no two-dimensional code in an image captured by the camera.

Further, the rotation module includes:

A sub-rotation module is configured to rotate the cleaning robot in place at a preset rotation angle until the two-dimensional code exists in an image captured by the camera.

Further, the system includes:

The coordinate transformation module is configured to transform the coordinate system of the cleaning robot so that the three-axis direction of the coordinate system of the cleaning robot is the same as the three-axis direction of the center of the two-dimensional code.

Beneficial effect

According to the above technical solution, the present invention has the beneficial effects of obtaining the relative positional relationship between the camera and the odometer according to the relative positional relationship between the camera and the two-dimensional code center and the relative positional relationship between the odometer and the two-dimensional code center. The calibration of the relative position relationship between the camera and the odometer aims to solve the problem that the existing cleaning robot does not consider the relative position relationship between the camera and the odometer, resulting in a large deviation in the position information obtained by the fusion.

BRIEF DESCRIPTION OF THE DRAWINGS

1 is a flowchart of a method for calibrating a camera and an odometer coordinate of a cleaning robot according to an embodiment of the present invention;

2 is a functional module diagram of a camera and an odometer coordinate calibration system of a cleaning robot according to an embodiment of the present invention;

3 is a schematic diagram of a coordinate system of a camera and a two-dimensional code of a cleaning robot according to an embodiment of the present invention.

Best Mode of the Invention

In order to make the objectives, technical solutions, and advantages of the present invention clearer, the present invention is further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not intended to limit the present invention.

As shown in FIG. 1 and FIG. 3, an embodiment of the present invention proposes a method for calibrating a camera and an odometer coordinate of a cleaning robot. A coordinate system is established with a center of a two-dimensional code as a coordinate origin, and a center axis of the cleaning robot is located at yz of the two-dimensional code On the axis plane, the method includes the following steps:

Step S101: Obtain a relative position relationship between a camera and a center of a two-dimensional code.

Place the QR code perpendicular to the ground, and establish a coordinate system with the center of the QR code as the origin of the coordinates. The sweeping robot is placed on the ground and is facing the QR code. The central axis of the sweeping robot is on the yz axis plane of the QR code. The shooting port of the camera faces the two-dimensional code, and the center of the two-dimensional code is used as the coordinate origin to obtain the relative position relationship between the camera and the two-dimensional code center.

In some embodiments, before step S101, the method includes:

Detect whether there is a QR code in the image captured by the camera;

If it does not exist, rotate the cleaning robot in place until a QR code exists in the image captured by the camera.

Turn on the camera and use the camera to capture the image. After the camera captures the image, check whether there is a QR code in the image captured by the camera. If there is no QR code in the image captured by the camera, it means that the camera's shooting port is not facing the QR code. , Then rotate the sweeping robot in place and change the orientation of the camera's shooting port until a QR code exists in the image captured by the camera, that is, rotate the sweeping robot in place. After rotating a certain angle in place, stop the in place rotation and shoot. Detect whether there is a QR code in the image captured by the camera. If it does not exist, rotate the sweeping robot in place again until the presence of the QR code in the image captured by the camera does not rotate in place.

The steps of rotating the sweeping robot in situ until a QR code exists in the image captured by the camera include:

Rotate the sweeping robot in place at a preset rotation angle until a QR code exists in the image captured by the camera.

The rotation angle of the cleaning robot that rotates in place every time is a preset rotation angle. Specifically, the preset rotation angle is 15 °.

In this embodiment, step S101 includes:

The relative position relationship between the camera and the center of the two-dimensional code is obtained through two-dimensional code positioning.

The camera captures the two-dimensional code, and obtains the relative position relationship between the camera and the center of the two-dimensional code through the two-dimensional code positioning.

Two-dimensional code positioning includes line detection, quadrilateral detection, calculation of homography matrix and external parameters.

Specifically, line detection: Calculate the gradient direction and gradient size of each pixel in the image, and then use the similarity measure of the pixel point gradient, and adjacent pixel points with similar gradient information are merged into a whole. Using a method similar to graph cut, the nodes of the graph are one pixel, and the edges are weighted by the gradient similarity of two pixels (regions). For pixel (area) n, use D (n) to represent its gradient direction, and M (n) to represent the magnitude of the gradient value. The conditions for merging pixel (area) n and pixel (area) m are:

D (n∪m) ≤min (D (n), D (m)) + KD / | n∪m |

M (n∪m) ≤min (M (n), M (m)) + KM / | n∪m |

Quadrilateral detection: Connect the detected lines to form a polygon through the spatial neighbor criterion. Limit the number of polygons and the number of corner points formed by the polygon to limit the number of polygons to obtain the quadrilateral and spatially adjacent quadrilaterals. Then merge into a new quadrilateral, and finally get a large quadrilateral that contains many 0,1 codes (0,1 represents a small quadrilateral). After the quadrilateral is detected, the distance is calculated by comparing the encoding of the large quadrilateral with a preset encoding type to obtain a more accurate detection target.

Calculate the homography matrix and external parameters: The homography matrix represents the second transformation of the 2D point projection onto the camera coordinate system on the two-dimensional code coordinate system, which can be obtained by the Direct Linear Transform algorithm. Camera internal parameters are represented by P, including camera focal length and center deviation. External parameters are indicated by E. The homography matrix can be written as follows:

Among them, Rij (i, j = 0,1,2) represents the rotation parameter, and Tk (k = x, y, z) represents the translation parameter.

Because the columns of the rotation matrix must be unit size, and according to the corresponding direction information of the two-dimensional code and the camera, where the two-dimensional code appears in front of the camera, the size and direction of s can be obtained. The third column of the rotation matrix can be recovered by calculating the cross product of two known columns, because the rotated column matrix must be orthogonal. In this way, the relative position relationship between the two-dimensional code and the camera can be obtained.

The relative position relationship T between the camera and the center of the two-dimensional code is obtained through the two-dimensional code positioning, which is denoted as (x, 0, z). Among them, the cleaning robot only moves on the horizontal plane, the y-axis does not need to be calibrated, and the y-axis coordinate information Set to 0.

After the step of obtaining the relative position relationship between the camera and the center of the two-dimensional code through two-dimensional code positioning, the method includes:

Obtain the relative rotation matrix between the camera and the center of the cleaning robot.

As the cleaning robot moves on the xz axis, the y axis is constant. According to the two-dimensional code positioning to obtain the relative position relationship between the camera and the center of the two-dimensional code, the relative rotation matrix between the camera and the center of the cleaning robot can be obtained as R .

Before step S101, the method includes:

The coordinate system of the cleaning robot is transformed so that the three-axis direction of the coordinate system of the cleaning robot is the same as the three-axis direction of the center of the two-dimensional code.

In this embodiment, the coordinate system of the cleaning robot is transformed so that the three-axis directions of the xyz coordinate system of the cleaning robot and the xyz three-axis of the two-dimensional code center are the same, which is convenient for subsequent calculation of the position information of the cleaning robot.

Step S102: Obtain a relative position relationship between the odometer and the two-dimensional code center.

In this embodiment, the center of the two-dimensional code is used as the coordinate origin, the odometer is on the yz-axis plane of the two-dimensional code, the cleaning robot moves only on the horizontal plane, the y-axis is not calibrated, and the coordinate information of the y-axis is set to zero. For this purpose, the odometer coordinates are (0, 0, s).

Step S103: Obtain the relative positional relationship between the camera and the odometer according to the relative positional relationship between the camera and the two-dimensional code center and the relative positional relationship between the odometer and the two-dimensional code center.

According to the relative position relationship T (x, 0, z) between the camera and the center of the two-dimensional code, the relative position relationship between the odometer and the center of the two-dimensional code (0,0, s). To this end, it can be obtained by calculation. Relative position relationship between camera and odometer (x, 0, zs).

In summary, according to the relative position relationship between the camera and the QR code center, and the relative position relationship between the odometer and the QR code center, the relative position relationship between the camera and the odometer is obtained, and the camera and the odometer are calibrated. The relative positional relationship between them is to solve the problem that the existing cleaning robot does not consider the relative positional relationship between the camera and the odometer, resulting in a large deviation in the position information obtained by the fusion.

As shown in FIG. 2 and FIG. 3, an embodiment of the present invention further provides a camera and odometer coordinate calibration system 1 of a sweeping robot, which establishes a coordinate system with a QR code center as a coordinate origin, and a central axis of the sweeping robot is located in the QR code On the plane of the yz axis, the system 1 includes a first acquisition module 11, a second acquisition module 12, and a third acquisition module 13.

The first obtaining module 11 is configured to obtain a relative position relationship between a camera and a center of a two-dimensional code.

In some embodiments, system 1 includes:

A detection module, configured to detect whether a two-dimensional code exists in an image captured by a camera;

A rotation module is used to rotate the cleaning robot in place if there is no QR code in the image captured by the camera.

The rotation module includes:

The sub-rotation module is used to rotate the sweeping robot in place at a preset rotation angle until a QR code exists in the image captured by the camera.

In this embodiment, the first obtaining module 11 includes:

The first sub-acquisition module is configured to obtain the relative position relationship between the camera and the center of the two-dimensional code through two-dimensional code positioning.

D (n∪m) ≤min (D (n), D (m)) + KD / | n∪m |

M (n∪m) ≤min (M (n), M (m)) + KM / | n∪m |

System 1 includes:

A fourth acquisition module is configured to acquire a relative rotation matrix between the camera and the center of the cleaning robot.

System 1 includes:

A coordinate transformation module is used to transform the coordinate system of the cleaning robot so that the three-axis direction of the coordinate system of the cleaning robot is the same as the three-axis direction of the center of the two-dimensional code.

The second acquisition module 12 is configured to acquire a relative position relationship between the odometer and the center of the two-dimensional code.

In this embodiment, the center of the two-dimensional code is used as the origin of the coordinates, the odometer is on the yz-axis plane of the two-dimensional code, and the cleaning robot moves only on the horizontal plane. For this purpose, the odometer coordinates are (0, 0, s).

The third obtaining module 13 is configured to obtain the relative positional relationship between the camera and the odometer according to the relative positional relationship between the camera and the two-dimensional code center and the relative positional relationship between the odometer and the two-dimensional code center.

The above description is only the preferred embodiments of the present invention and is not intended to limit the present invention. Any modification, equivalent replacement, and improvement made within the spirit and principle of the present invention shall be included in the protection of the present invention. Within range.

Claims

A method for calibrating a camera and an odometer coordinate of a cleaning robot. A coordinate system is established by taking a center of a two-dimensional code as a coordinate origin, and a center axis of the cleaning robot is located on a yz-axis plane of the two-dimensional code. :

Obtaining a relative position relationship between a camera and the center of the two-dimensional code;

Obtaining a relative position relationship between the odometer and the two-dimensional code center;

Obtaining the relative position between the camera and the odometer according to the relative position relationship between the camera and the two-dimensional code center and the relative position relationship between the odometer and the two-dimensional code center relationship.
The method for calibrating a camera and an odometer coordinate of a cleaning robot according to claim 1, wherein the step of acquiring a relative position relationship between the camera and the center of the two-dimensional code comprises:

The relative position relationship between the camera and the center of the two-dimensional code is obtained through two-dimensional code positioning. The two-dimensional code positioning includes line detection, quadrilateral detection, calculation of a homography matrix, and external parameters.
The method for calibrating a camera and an odometer coordinate of a cleaning robot according to claim 1, wherein before the step of obtaining a relative position relationship between the camera and the center of the two-dimensional code, the method comprises:

Detecting whether the two-dimensional code exists in an image captured by the camera;

If it does not exist, rotate the cleaning robot in place until the two-dimensional code exists in the image captured by the camera.
The method for calibrating the camera and odometer coordinate of the cleaning robot according to claim 3, wherein in the step of rotating the cleaning robot in place until the two-dimensional code exists in the image captured by the camera ,include:

Rotate the cleaning robot in place at a preset rotation angle until the two-dimensional code exists in an image captured by the camera.
The method for calibrating a camera and an odometer coordinate of a cleaning robot according to claim 1, before the step of obtaining a relative position relationship between the camera and the center of the two-dimensional code, comprising:

Transforming the coordinate system of the cleaning robot so that the three-axis direction of the coordinate system of the cleaning robot is the same as the three-axis direction of the center of the two-dimensional code.
A camera and odometer coordinate calibration system of a sweeping robot establishes a coordinate system with a center of a two-dimensional code as a coordinate origin, and a central axis of the sweeping robot is located on a yz-axis plane of the two-dimensional code. The system includes: :

A first acquisition module, configured to acquire a relative position relationship between a camera and the center of the two-dimensional code;

A second acquisition module, configured to acquire a relative position relationship between the odometer and the two-dimensional code center;

A third obtaining module, configured to obtain the camera and the two-dimensional code according to a relative positional relationship between the camera and the two-dimensional code center and a relative positional relationship between the odometer and the two-dimensional code center Relative position relationship between odometers.
The camera and odometer coordinate calibration system of a cleaning robot according to claim 6, wherein the two-dimensional code positioning includes line detection, quadrilateral detection, calculation of a homography matrix and external parameters, and the first acquisition module include:

The first sub-acquisition module is configured to obtain a relative position relationship between a camera and a center of the two-dimensional code by using two-dimensional code positioning.
The camera and odometer coordinate calibration system of the cleaning robot according to claim 6, wherein the system comprises:

A detection module, configured to detect whether the two-dimensional code exists in an image captured by the camera;

A rotation module is configured to rotate the cleaning robot in place if there is no two-dimensional code in an image captured by the camera.
The camera and odometer coordinate calibration system of the cleaning robot according to claim 8, wherein the rotation module comprises:

A sub-rotation module is configured to rotate the cleaning robot in place at a preset rotation angle until the two-dimensional code exists in an image captured by the camera.
The camera and odometer coordinate calibration system of the cleaning robot according to claim 6, wherein the system comprises:

The coordinate transformation module is configured to transform the coordinate system of the cleaning robot so that the three-axis direction of the coordinate system of the cleaning robot is the same as the three-axis direction of the center of the two-dimensional code.