WO2022160760A1

WO2022160760A1 - Calibration method and apparatus for multiple stereo cameras

Info

Publication number: WO2022160760A1
Application number: PCT/CN2021/121462
Authority: WO
Inventors: 刘锋
Original assignee: 北京罗克维尔斯科技有限公司
Priority date: 2021-01-29
Filing date: 2021-09-28
Publication date: 2022-08-04
Also published as: CN112802124B; CN112802124A

Abstract

The present application relates to a calibration method and apparatus for multiple stereo cameras, an electronic device, and a storage medium. The calibration method comprises: extracting point clouds to which a first calibration object belongs in point clouds obtained by stereo cameras when the first calibration object is at different poses, fitting planes according to the point clouds to which the first calibration object belongs obtained by the stereo cameras, and obtaining normal vectors of the planes; obtaining, according to normal vectors corresponding to two stereo cameras, a rotation matrix between coordinate systems where the two stereo cameras are located; obtaining photographing coordinates of first and second calibration points, and fitting a straight line according to the photographing coordinates of the first calibration point; obtaining calculated coordinates of the second calibration point according to the photographing coordinates of the first calibration point and distribution positions of all calibration points; and according to the calculated coordinates and the photographing coordinates of the same second calibration point and the rotation matrix between the coordinate systems where the two stereo cameras are located, obtaining a translation matrix between the coordinate systems where the two stereo cameras are located.

Description

Calibration method and device for multi-stereo camera

CROSS-REFERENCE TO RELATED APPLICATIONS

This application is based on the Chinese patent application with the application number of 202110125395.8 and the filing date of January 29, 2021, and claims the priority of the Chinese patent application. The entire content of the Chinese patent application is incorporated herein by reference.

technical field

The present disclosure relates to the technical field of computer vision, and in particular, to a method and device for calibrating a multi-stereo camera, an electronic device, and a storage medium.

Background technique

The stereo camera can obtain the 3D point cloud of the target object, and the point cloud can be used to realize 3D reconstruction, detection and measurement of the target. The number of stereo cameras can be increased, and multiple stereo cameras can obtain point clouds at the same time and stitch them together to expand the field of view and obtain point clouds of larger targets. To complete the splicing of point clouds obtained by multiple stereo cameras, the most important thing is to unify the point clouds obtained by multiple stereo cameras into the same coordinate system, that is, to find the transformation matrix of the coordinate system where multiple cameras are located to complete multiple stereo cameras. Camera extrinsic calibration.

Among the existing stereo camera calibration solutions, one solution is to complete the calibration of the stereo camera by shooting two-dimensional images and three-dimensional point clouds of the calibration chessboard fixed on the calibration plate in different postures. Another solution is to use wrinkled paper regardless of thickness, and use the iterative closest point algorithm to calibrate the external parameters of multiple stereo cameras. Although the matching of multiple stereo cameras can be achieved, both calibration processes require iterative transition calculation, and the calibration efficiency It is extremely low, and with the progress of the iterative process, the calibration error becomes larger and larger, and the calibration accuracy is poor. In addition, a global coordinate system can also be established in various ways, and marker points can be prepared in advance in the global coordinate system, and the marker points are known in the global coordinate system, but a total station or complex calibration objects are often required to cooperate, increasing the The cost of completing stereo camera calibration.

SUMMARY OF THE INVENTION

In order to solve the above technical problems or at least partially solve the above technical problems, the present disclosure provides a multi-stereo camera calibration method and device, an electronic device and a storage medium, which improve the calibration efficiency and accuracy of the multi-stereo camera.

In a first aspect, an embodiment of the present disclosure provides a method for calibrating a multi-stereo camera, including:

The first calibration object is placed in the field of view of the plurality of stereo cameras and the second calibration object is placed in the field of view of the plurality of stereo cameras; wherein, the first calibration object includes a calibration plate, and the second calibration object Including a plurality of calibration points on a straight line, at least two of the calibration points can be seen in each of the stereo cameras;

Extracting the point cloud to which the first calibration object belongs in the point cloud obtained by the stereo camera when the first calibration object is in different poses, and fitting a plane according to the point cloud to which the first calibration object belongs to obtained by the stereo camera , and obtain the normal vector of the fitted plane;

Obtain the rotation matrix between the coordinate systems where the two stereo cameras are located according to the normal vectors corresponding to the two stereo cameras;

Obtain the shooting coordinates of the first calibration point and the second calibration point, and fit a straight line according to the shooting coordinates of the first calibration point; wherein, the first calibration point and the second calibration point are located in different a calibration point in the field of view of the stereo camera, the second calibration point is located on the fitted straight line;

Obtain the calculated coordinates of the second calibration point according to the shooting coordinates of the first calibration point and the distribution positions of all the calibration points;

The translation matrix between the coordinate systems where the two stereo cameras are located is obtained according to the calculated coordinates and the shooting coordinates of the same second calibration point and the rotation matrix between the coordinate systems where the two stereo cameras are located.

Optionally, before the point cloud fitting plane to which the first calibration object belongs, obtained according to the stereo camera, the method further includes:

The point cloud to which the first calibration object belongs in the point cloud obtained by the stereo camera is extracted by using a clustering algorithm and/or a filtering algorithm.

Optionally, a least squares algorithm or a random sampling consensus algorithm is used to fit the plane according to the point cloud to which the first calibration object belongs, obtained by the stereo camera.

Optionally, obtaining the rotation matrix between the coordinate systems where the two stereo cameras are located according to the normal vectors corresponding to the two stereo cameras, including:

According to the normal vectors corresponding to the two stereo cameras, the relationship equation between the normal vector and the rotation matrix is listed, and the relationship equation is:

Wherein, (N _1x , N _1y , N _1z ) are the three-dimensional coordinates of the normal vector corresponding to one of the two stereo cameras, and (N _2x , N _2y , N _2z ) are the two stereo cameras the three-dimensional coordinates of the normal vector corresponding to another stereo camera in the stereo camera, and R21 is the rotation matrix between the coordinate systems where the two stereo cameras are located;

The rotation matrix between the coordinate systems where the two stereo cameras are located is obtained according to the relational equation.

Optionally, a singular value decomposition algorithm is used to obtain the rotation matrix between the coordinate systems where the two stereo cameras are located according to the normal vectors corresponding to the two stereo cameras.

Optionally, fitting a straight line according to the shooting coordinates of the first calibration point, including:

A straight line is fitted according to the shooting coordinates of at least two of the first calibration points, so that the at least two first calibration points are located on the fitted straight line.

Optionally, obtaining the calculated coordinates of the second calibration point according to the shooting coordinates of the first calibration point and the distribution positions of all the calibration points, including:

Determine the distance between the first calibration point and the second calibration point on the fitted straight line according to the distribution positions of all the calibration points;

The calculated coordinates of the second calibration point are obtained according to the shooting coordinates of the first calibration point, the fitted equation of the straight line, and the distance.

Optionally, the calibration points are arranged at equal intervals;

Determine the distance between the first calibration point and the second calibration point on the fitted straight line according to the distribution positions of all the calibration points, including:

Numbering all the calibration points in sequence;

The product of the difference between the label of the second calibration point and the label of the first calibration point located on the fitted straight line and the distance between two adjacent calibration points is the The distance between the first calibration point and the second calibration point on the straight line.

Optionally, obtain the translation matrix between the coordinate systems where the two stereo cameras are located according to the calculated coordinates of the same second calibration point, the shooting coordinates and the rotation matrix between the coordinate systems where the two stereo cameras are located, including:

According to the different second calibration points, list the calculated coordinates of the second calibration points, the shooting coordinates, the rotation matrix between the coordinate systems where the two stereo cameras are located, and the coordinates where the two stereo cameras are located. The relationship equation between the translation matrices between the systems;

The relational equation is:

Wherein, R is the rotation matrix between the coordinate systems where the two stereo cameras are located, T is the translation matrix between the coordinate systems where the two stereo cameras are located, and (Px', Py', Pz') is the first The calculated coordinates of the two calibration points, (Px, Py, Pz) are the shooting coordinates of the second calibration point;

The translation matrix between the coordinate systems where the two stereo cameras are located is obtained according to the relational equation.

Optionally, the average value algorithm is used to obtain the coordinates between the coordinate systems where the two stereo cameras are located according to the calculated coordinates of the same second calibration point, the shooting coordinates, and the rotation matrix between the coordinate systems where the two stereo cameras are located. the translation matrix.

Optionally, acquiring the shooting coordinates of the first calibration point includes:

The shooting coordinates of the first calibration point where the second calibration object is in different poses are acquired.

Optionally, the pose of the first calibration object includes a distance from the first calibration object to the stereo camera, and/or an inclination angle of the first calibration object relative to the stereo camera;

The pose of the second calibration object includes a distance from the second calibration object to the stereo camera, and/or an inclination angle of the second calibration object relative to the stereo camera.

Optionally, the first calibration object and the second calibration object are set integrally or independently.

In a second aspect, an embodiment of the present disclosure also provides a multi-stereo camera calibration device, including:

A calibration object setting module, used for placing a first calibration object in the field of view of a plurality of stereo cameras and placing a second calibration object in the field of view of the plurality of stereo cameras; wherein the first calibration object includes a calibration plate , the second calibration object includes a plurality of calibration points located on a straight line, and at least two of the calibration points can be seen in each of the stereo cameras;

A normal vector acquisition module, configured to extract the point cloud to which the first calibration object belongs in the point cloud obtained by the stereo camera when the first calibration object is in different poses, and according to the first calibration obtained by the stereo camera The point cloud fitting plane to which the object belongs, and the normal vector of the fitted plane is obtained;

a rotation matrix acquisition module, configured to acquire the rotation matrix between the coordinate systems where the two stereo cameras are located according to the normal vectors corresponding to the two stereo cameras;

a straight line fitting module, used for acquiring the shooting coordinates of the first calibration point and the second calibration point, and fitting a straight line according to the shooting coordinates of the first calibration point; wherein, the first calibration point and the second calibration point The fixed points are calibration points located in different fields of view of the stereo cameras, and the second calibration point is located on the fitted straight line;

a calculation coordinate obtaining module, configured to obtain the calculation coordinates of the second calibration point according to the shooting coordinates of the first calibration point and the distribution positions of all the calibration points;

The translation matrix acquisition module is used for acquiring the translation matrix between the coordinate systems where the two stereo cameras are located according to the calculated coordinates of the same second calibration point, the shooting coordinates and the rotation matrix between the coordinate systems where the two stereo cameras are located .

In a third aspect, an embodiment of the present disclosure further provides an electronic device, including a processor and a memory, where the processor executes the steps of the calibration method for a multi-stereo camera according to the first aspect by invoking a program or an instruction stored in the memory.

In a fourth aspect, an embodiment of the present disclosure further provides a storage medium, where the storage medium stores a program or an instruction, where the program or instruction causes a computer to execute the steps of the calibration method for a multi-stereo camera according to the first aspect.

Compared with the prior art, the technical solutions provided by the embodiments of the present disclosure have the following advantages:

The technical solutions of the embodiments of the present disclosure provide a method for calibrating multiple stereo cameras. The coordinate system where one stereo camera is located is used as the basic coordinate system, and the rotation matrix between each other stereo cameras and the coordinate system where the basic stereo camera is located is calibrated and translation matrix, that is, the external parameter matrix between each other stereo camera and the coordinate system of the basic stereo camera is calibrated, which realizes the calibration of multi-stereo cameras, and the step-by-step calibration of the rotation matrix and translation matrix is conducive to the The theorem analysis of the calibration accuracy of the rotation matrix and the translation matrix is carried out to reduce the calibration error, so as to improve the calibration accuracy of the rotation matrix or translation matrix of a stereo camera in different ways. In addition, the production of the first calibration object and the second calibration object is relatively simple, no iterative calculation is required, the calibration efficiency and accuracy are high, and third-party tools such as total stations are not required, which effectively reduces the multi-stereo camera calibration work. cost.

Description of drawings

The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the disclosure and together with the description serve to explain the principles of the disclosure.

In order to more clearly illustrate the embodiments of the present disclosure or the technical solutions in the prior art, the accompanying drawings that are required to be used in the description of the embodiments or the prior art will be briefly introduced below. In other words, on the premise of no creative labor, other drawings can also be obtained from these drawings.

FIG. 1 is a schematic flowchart of a method for calibrating a multi-stereo camera according to an embodiment of the present disclosure;

FIG. 2 is a schematic diagram of a position structure of a multi-stereo camera according to an embodiment of the present disclosure;

3 is a schematic top-view structure diagram of a first calibration object provided by an embodiment of the present disclosure;

4 is a schematic structural diagram of a multi-stereo camera calibration device according to an embodiment of the present disclosure;

FIG. 5 is a schematic structural diagram of an electronic device according to an embodiment of the present invention.

Detailed ways

In order to more clearly understand the above objects, features and advantages of the present disclosure, the solutions of the present disclosure will be further described below. It should be noted that the embodiments of the present disclosure and the features in the embodiments may be combined with each other under the condition of no conflict.

Many specific details are set forth in the following description to facilitate a full understanding of the present disclosure, but the present disclosure can also be implemented in other ways different from those described herein; obviously, the embodiments in the specification are only a part of the embodiments of the present disclosure, and Not all examples.

FIG. 1 is a schematic flowchart of a method for calibrating a multi-stereo camera according to an embodiment of the present disclosure. The method for calibrating a multi-stereo camera can be applied to an application scenario where multiple stereo cameras need to be calibrated, and can be performed by the multi-stereo camera calibration device provided by the embodiment of the present disclosure, and the multi-stereo camera calibration device can use software and/or hardware. way to achieve. As shown in Figure 1, the calibration method of the multi-stereo camera includes the following steps:

S101. Place the first calibration object in the field of view of the plurality of stereo cameras and place the second calibration object in the field of view of the plurality of stereo cameras.

FIG. 2 is a schematic diagram of a position structure of a multi-stereo camera according to an embodiment of the present disclosure. As shown in FIG. 2 , multiple stereo cameras 10 are located at the same level. FIG. 2 exemplarily sets the fields of view of different stereo cameras 10 to overlap, and it is also possible to set some or all of the fields of view of the stereo cameras 10 . There is no overlap.

FIG. 3 is a schematic top-view structural diagram of a first calibration object provided by an embodiment of the present disclosure. 2 and 3, a first calibration object 1 and a second calibration object 2 can be seen in each stereo camera 10. The first calibration object 1 includes a calibration plate. The calibration plate can be, for example, a solid color calibration plate, and a first calibration can be set. Object 1 , that is, the area of the calibration plate is large enough to ensure that the first calibration object 1 can be seen in each stereo camera 10 . The second calibration object 2 includes a plurality of calibration points A located on a straight line. At least two calibration points A can be seen in each stereo camera 10 , that is, at least two calibration points A are located in the field of view of the same stereo camera 10 . The fabrication of the first calibration object 1 and the second calibration object 2 used in the disclosed embodiment is relatively simple.

S102: Extract the point cloud to which the first calibration object belongs in the point cloud obtained by the stereo camera when the first calibration object is in different poses, fit the plane according to the point cloud to which the first calibration object belongs, and obtain the fitted plane normal vector.

Specifically, with reference to FIGS. 2 and 3 , two-dimensional images and corresponding three-dimensional point clouds captured by each stereo camera 10 are obtained, and the point cloud to which the first calibration object 1 belongs is extracted from the point cloud obtained by each stereo camera 10 . Exemplarily, a clustering algorithm and/or a filtering algorithm may be used to extract the point cloud to which the first calibration object 1 belongs in the point cloud obtained by each stereo camera 10 . Specifically, the clustering algorithm is to cluster the point clouds that may be located on the same plane in the point cloud to which the first calibration object 1 belongs, so as to facilitate the subsequent work of fitting the plane, and the filtering algorithm is to filter out the points in the acquired point cloud. Noise, in order to improve the accuracy of the point cloud extraction to which the first calibration object 1 belongs, the clustering algorithm and the filtering algorithm are both commonly used processing algorithms for 3D point clouds, and will not be discussed in detail here.

Fitting the plane corresponding to the stereo camera 10 according to the point cloud of the first calibration object 1 obtained by the stereo camera 10 , exemplarily, the least squares algorithm or the random sampling consensus algorithm can be used according to the first calibration object 1 obtained by the stereo camera 10 . The corresponding point cloud fitting plane, that is, fitting the plane corresponding to the stereo camera 10 , and the fitting plane may satisfy the equation Ax+By+Cz+D=0, for example. The normal vector of the plane corresponding to the stereo camera 10 is obtained according to the equation of the fitted plane. For the plane equation Ax+By+Cz+D=0, the vector N(A, B, C) is the unit normal vector of the fitted plane .

Before fitting the plane corresponding to the stereo camera 10 according to the point cloud obtained by the stereo camera 10 to which the first calibration object 1 belongs, the first calibration object 1 can be extracted in different poses, and the first calibration object 1 in the point cloud obtained by each stereo camera 10 is the first The point cloud to which calibration object 1 belongs. Exemplarily, the pose of the first calibration object 1 includes the distance from the first calibration object 1 to the stereo camera 10 , and/or the inclination angle of the first calibration object 1 relative to the stereo camera 10 .

Specifically, referring to FIG. 2 and FIG. 3 , when acquiring the point cloud of the first calibration object 1 in the point cloud obtained by each stereo camera 10 , the pose of the first calibration object 1 is continuously changed, that is, the first calibration object 1 is continuously adjusted. The distance to the different stereo cameras 10 and the inclination angle of the first calibration object 1 relative to the different stereo cameras 10, each time the pose of the first calibration object 1 is adjusted, obtain each camera corresponding to the first calibration object 1 in one pose The point cloud to which the first calibration object 1 belongs in the point cloud obtained by the stereo camera 10 . Exemplarily, it may be set to acquire the point cloud to which the first calibration object 1 belongs in the point cloud obtained by each stereo camera 10 in which the first calibration object 1 is in at least eight different poses.

Thus, each time the pose of the first calibration object 1 is adjusted, a set of samplings of the point cloud to which the first calibration object 1 belongs in the point cloud obtained by each stereo camera 10 is formed, and the first calibration object 1 corresponding to each group of samples is sampled. If the pose is different, that is, the distance between the corresponding first calibration object 1 and the stereo camera 10, or the inclination angle of the first calibration object 1 relative to the stereo camera 10 is different, add the first calibration object 1 in the point cloud obtained by each stereo camera 10. The sampling times of the point cloud to which it belongs is beneficial to improve the accuracy and precision of the camera calibration result.

S103: Acquire a rotation matrix between the coordinate systems where the two stereo cameras are located according to the normal vectors corresponding to the two stereo cameras.

Specifically, the rotation matrix between the coordinate systems where the two stereo cameras 10 are located is obtained according to the normal vectors corresponding to the two stereo cameras 10 , and the relationship between the normal vector and the rotation matrix can be listed according to the normal vectors corresponding to the two stereo cameras 10 equation, the relational equation is:

Among them, (N _1x , N _1y , N _1z ) are the three-dimensional coordinates of the normal vector corresponding to one of the two stereo cameras 10 , and (N _2x , N _2y , N _2z ) are the two stereo cameras 10 . The three-dimensional coordinates of the normal vector corresponding to another stereo camera 10, R21 is the rotation matrix between the coordinate systems where the two stereo cameras 10 are located, and the rotation matrix between the coordinate systems where the two stereo cameras 10 are located is obtained according to the relational equation.

Exemplarily, taking the first stereo camera 101 and the second stereo camera 102 in FIG. 2 as an example, the plane corresponding to the stereo camera 101 is fitted according to the point cloud obtained by the stereo camera 101 to which the first calibration object 1 belongs, such as: For the first fitting plane, the normal vector of the first fitting plane is obtained, for example, the first normal vector N1. Similarly, a plane corresponding to the stereo camera 102 is fitted according to the point cloud obtained by the stereo camera 102 to which the first calibration object 1 belongs, such as the second fitting plane, and the normal vector of the second fitting plane is obtained, such as the second normal vector N2.

The first normal vector N1, the second normal vector N2, and the rotation matrix between the coordinate system where the stereo camera 101 and the stereo camera 102 are located satisfy the above equation relationship. Taking the coordinate system where the stereo camera 101 is located as the basic coordinate system, the first method The coordinates of the vector N1 are (N _1x , N _1y , N _1z ), the coordinates of the second normal vector N2 are (N _2x , N _2y , N _2z ), and R21 is the coordinate system where the stereo camera 102 is located and wants to be unified to the stereo camera The rotation matrix corresponding to the coordinate system where 101 is located.

It should be noted that, in order to realize the calibration of all the stereo cameras 10, it is necessary to convert the coordinate systems where all the stereo cameras 10 are located to the coordinate system where one stereo camera 10 is located. Take the coordinate system where the camera 101 is located as the basic coordinate system as an example, since the normal vectors corresponding to each of the other stereo cameras 10 are different from each other, each of the other stereo cameras 10 converts their respective coordinate systems to the coordinate system where the stereo camera 101 is located according to the above method. The corresponding rotation matrices of the process are different.

For every two stereo cameras 10 , for example, for the stereo cameras 101 and 102 in FIG. 2 , each time the first calibration object 1 is in a pose, the normal vector corresponding to the stereo camera 101 and the corresponding normal vector of the stereo camera 102 can be obtained. A set of normal vector relationship equations described in the above-mentioned embodiments that are satisfied by the normal vector and the rotation matrix between the coordinate systems of the two stereo cameras 10, the number of times to adjust the pose of the first calibration object 1 can be set to N, then N groups of normal vector relationship equations described in the above embodiments can be obtained corresponding to the stereo camera 101 and the stereo camera 102 . Exemplarily, for the N groups of normal vector relationship equations described in the above embodiments corresponding to the stereo cameras 101 and 102 , a singular value decomposition algorithm can be used to obtain the two stereo cameras according to the normal vectors corresponding to the two stereo cameras 10 . 10 The rotation matrix between the coordinate systems where the stereo camera 101 is located, that is, the rotation matrix between the coordinate systems where the stereo camera 101 and the stereo camera 102 are located can be obtained by using the singular value decomposition algorithm according to the normal vectors corresponding to the stereo camera 101 and the stereo camera 102 .

The planes fitted according to the point clouds of the first calibration object 1 obtained by different stereo cameras 10 are different. To achieve the unification of the coordinate systems where all stereo cameras 10 are located, it is necessary to make the fitting corresponding to different stereo cameras 10 through the calibration process. planes coincide. Therefore, the embodiment of the present disclosure first obtains the rotation matrices between the coordinate systems where the two stereo cameras 10 are located, and through the rotation matrices between the coordinate systems where the different stereo cameras 10 are located, the fitting planes corresponding to the different stereo cameras 10 can be realized to be parallel to each other, Then calculate the translation matrix between the coordinate systems where the different stereo cameras 10 are located, and use the translation matrix to translate the fitting planes that are parallel to each other, so that the fitting planes corresponding to the different stereo cameras 10 can be overlapped, and then the coordinate systems where all the stereo cameras 10 are located can be realized. The unification of multiple stereo cameras 10 completes the calibration work.

S104: Obtain the shooting coordinates of the first calibration point and the second calibration point, and fit a straight line according to the shooting coordinates of the first calibration point; wherein, the first calibration point and the second calibration point are located in different fields of view of the stereo camera Calibration point, the second calibration point is located on the fitted straight line.

Specifically, as shown in FIG. 2 , a stereo camera can be set and set, for example, the stereo camera 101 shoots the second calibration object 2 to obtain a set stereo camera, such as a two-dimensional image of the second calibration object 2 in the field of view of the stereo camera 101 . The image and the corresponding three-dimensional point cloud, extract and set the stereo camera, for example, the calibration point A in the two-dimensional image obtained by the stereo camera 101, that is, the first calibration point A1 and determine the shooting coordinates of the first calibration point A1. The shooting coordinates of the fixed point A1 are the three-dimensional coordinates corresponding to the first calibration point A1 in the point cloud.

Optionally, a straight line is fitted according to the shooting coordinates of the first calibration point A1, and a straight line can be fitted according to the shooting coordinates of the at least two first calibration points A1 so that the at least two first calibration points A1 are located on the fitted straight line. superior. Exemplarily, a straight line can be fitted according to the shooting coordinates of the two first calibration points A1 so that the two first calibration points A1 are located on the fitted straight line, and the stereo camera 101 in FIG. 2 is set as the stereo camera. For example, the first calibration point A1 for fitting the straight line can be, for example, the calibration point A11 and the calibration point A12, and a straight line can be fitted according to the shooting coordinates of the calibration point A11 and the calibration point A12, and the equation of the fitted straight line can be, for example, (x-x0)/l=(y-y0)/m=(z-z0)/n, that is, the fitted straight line passes through the point (x0, y0, z0) and the direction vector of the fitted straight line is (l, m, n).

Optionally, acquiring the shooting coordinates of the first calibration point A1 may be acquiring the shooting coordinates of the first calibration point A1 where the second calibration object 2 is in a different pose. Exemplarily, the pose of the second calibration object 2 may include the second calibration object 2 to the stereo camera, for example, the distance from the stereo camera 101 is set, and/or the second calibration object 2 is relative to the stereo camera, for example, the stereo camera is set. The tilt angle of the camera 101 .

Specifically, when acquiring the shooting coordinates of the first calibration point A1, the pose of the second calibration object 2 is continuously changed, that is, the second calibration object 2 is continuously adjusted to the stereo camera, for example, the distance of the stereo camera 101 is set, and the second calibration object 2 is continuously adjusted. The calibration object 2 is relative to the stereo camera, for example, the inclination angle of the stereo camera 101 is set, and each time the pose of the second calibration object 2 is adjusted, the shooting coordinates of the first calibration point A1 corresponding to a pose of the second calibration object 2 are obtained. . Exemplarily, it may be set to acquire the shooting coordinates of the first calibration point A1 where the second calibration object 2 is in at least eight different poses.

Therefore, each time the pose of the second calibration object 2 is adjusted, a set of samples of the shooting coordinates of the first calibration point A1 is formed, and the poses of the second calibration object 2 corresponding to each set of samples are different, that is, the corresponding second calibration The distance between the object 2 and the stereo camera, for example, setting the distance of the stereo camera 101, or the second calibration object 2 relative to the stereo camera, for example, setting the inclination angle of the stereo camera 101 to be different, increasing the sampling times of the shooting coordinates of the first calibration point A1 is beneficial to Improve the accuracy and precision of camera calibration results.

The shooting coordinates of the second calibration point are acquired, the first calibration point and the second calibration point are calibration points located in different fields of view of the stereo camera, and the second calibration point is located on the fitted straight line. Specifically, as shown in FIG. 2 , taking the stereo camera 102 as an example, the stereo camera 102 shoots the second calibration object 2 to obtain a two-dimensional image of the second calibration object 2 in the field of view of the stereo camera 102 and the corresponding three-dimensional point cloud , extract the calibration point A in the two-dimensional image obtained by the stereo camera 102, that is, the second calibration point A2 and determine the shooting coordinates of the second calibration point A2, the shooting coordinates of the second calibration point A2 are the second calibration point A2 at the point The corresponding three-dimensional coordinates in the cloud, and the shooting coordinates of the second calibration point A2 are the coordinates of the second calibration point A2 in the coordinate system where the stereo camera 102 is located, and the process of obtaining the shooting coordinates of the first calibration point A1 and the second calibration point can be set. The acquisition process of the shooting coordinates of A2 is performed simultaneously.

Similarly, as shown in FIG. 2 , by obtaining the shooting coordinates of the second calibration point A2 , the shooting coordinates of the second calibration point A2 where the second calibration object 2 is in different poses can be obtained. Specifically, when acquiring the shooting coordinates of the second calibration point A2, the pose of the second calibration object 2 is constantly changed, that is, the distance from the second calibration object 2 to the stereo camera 10 is continuously adjusted, and the relative position of the second calibration object 2 to the stereo camera 10 is continuously adjusted. The tilt angle of the camera 10 is adjusted every time the pose of the second calibration object 2 is adjusted, and the shooting coordinates of the second calibration point A2 corresponding to one pose of the second calibration object 2 are obtained. Exemplarily, it may be set to acquire the shooting coordinates of the second calibration point A2 where the second calibration object 2 is in at least eight different poses.

Thus, each time the pose of the second calibration object 2 is adjusted, a set of samples of the shooting coordinates of the second calibration point A2 are formed, and the poses of the second calibration object 2 corresponding to each set of samples are different, that is, the corresponding second calibration The distance from the object 2 to the stereo camera 10 or the inclination angle of the second calibration object 2 relative to the stereo camera 10 is different. Increasing the sampling times of the shooting coordinates of the second calibration point A2 is beneficial to improve the accuracy and precision of the camera calibration result.

S105: Acquire the calculated coordinates of the second calibration point according to the shooting coordinates of the first calibration point and the distribution positions of all the calibration points.

Specifically, according to the shooting coordinates of the first calibration point A1 and the distribution positions of all the calibration points A, the calculated coordinates of the second calibration point A2 located on the fitted straight line are obtained, and the second calibration point A2 is located in the stereo camera except the setting, For example, the calibration point A in the field of view of the other stereo cameras other than the stereo camera 101 is taken as an example here that the second calibration point A2 is the calibration point A located in the field of view of the stereo camera 102 . The distance between the first calibration point A1 and the second calibration point A2 on the fitted straight line can be determined according to the distribution positions of all the calibration points A, according to the shooting coordinates of the first calibration point A1, the equation of the fitted straight line and The distance obtains the calculated coordinates of the second calibration point A2 located on the fitted straight line.

Exemplarily, as shown in Figure 2, the calibration points A can be set at equal intervals, and the distance between the first calibration point A1 and the second calibration point A2 on the fitted straight line is determined according to the distribution positions of all the calibration points A, All calibration points A can be numbered in sequence, and the product of the difference between the label of the second calibration point A2 on the fitted straight line and the label of the first calibration point A1 and the distance d between the adjacent two calibration points A is: The distance between the first calibration point A1 and the second calibration point A2 on the fitted straight line.

Specifically, the marking point may be, for example, a circular marking point. As shown in FIG. 2 , the first calibration point A1 may be, for example, the calibration point A11 with the label 2, and the second calibration point A2 may be, for example, the calibration point A21 with the label 7. Then five times the distance d between two adjacent calibration points A is the distance between the first calibration point A11 and the second calibration point A21 on the fitted straight line. The equation of the fitted straight line is known, and then combined Knowing the shooting coordinates of the first calibration point A11 on the straight line, that is, the three-dimensional coordinates, the calculated coordinates of the second calibration point A21 on the fitted straight line can be obtained by calculation, and the calculated coordinates of the second calibration point A21 are the second calibration point A21. The fixed point A21 is setting the stereo camera, for example, the coordinates in the coordinate system where the stereo camera 101 is located.

It should be noted that, the above embodiments are only described by taking the stereo camera 101 as the setting stereo camera as an example, and other stereo cameras may also be used as the setting stereo camera, which is not limited in the embodiments of the present disclosure.

S106: Obtain a translation matrix between the coordinate systems where the two stereo cameras are located according to the calculated coordinates of the same second calibration point, the shooting coordinates, and the rotation matrix between the coordinate systems where the two stereo cameras are located.

Specifically, the stereo camera 10 can acquire the three-dimensional point cloud of the target object, and the point cloud can be used to realize the three-dimensional reconstruction, detection and measurement of the target. To meet actual needs, the number of stereo cameras 10 can be increased, and multiple stereo cameras 10 can acquire point clouds at the same time and use them for splicing to expand the field of view and acquire point clouds of larger targets. To complete the splicing of point clouds obtained by multiple stereo cameras 10, the most important thing is to unify the point clouds obtained by multiple stereo cameras 10 into one coordinate system, that is, to find the transformation matrix of the coordinate system where the multiple cameras are located to complete the multiple cameras. External parameter calibration of the stereo camera 10 .

Exemplarily, the transformation matrix B can be represented as a 4×4 matrix, and the transformation matrix B satisfies the following calculation formula:

Among them, the transformation matrix B, that is, the external parameter calibration matrix, can be marked as a 4×4 matrix, or it can be composed of a 3×3 rotation matrix R and a 3×1 translation matrix T, and the coordinate system is transformed by some rigid bodies. , for example, it can be converted to another coordinate system by doing some translation and rotation along the x direction, y direction or z direction respectively. To realize the calibration of the two stereo cameras 10, it is necessary to obtain the translation quantities x, y and z and the rotation quantities rx, ry and rz, and then the translation matrix T and the rotation matrix R can be calculated. In the above formula, for the convenience of calculation, both the rotation matrix R and the translation matrix T are expanded into 4×4 matrices. The first three matrices in the second row of the equation correspond to the rotation matrix R, and the latter matrix corresponds to the translation matrix T. According to the linear Algebraic related theory, the expansion of the above matrix does not affect the calculation results.

For multiple stereo cameras 10 , as shown in FIG. 2 , the stereo camera, for example, the coordinate system where the stereo camera 101 is located can be used as the basic coordinate system, as long as the transformation matrix B of the remaining stereo cameras 10 and the stereo cameras 101 is calibrated, that is, the calibration By obtaining the rotation matrices R and translation matrices T of the remaining stereo cameras 10 and the stereo cameras 101 , the calibration of multiple stereo cameras 10 can be completed, and the coordinate systems of all the stereo cameras 10 can be unified. It should be noted that the coordinate system where other stereo cameras 10 are located may also be set as the base coordinate system, and the coordinate system where the stereo camera 101 is located is not limited to be the base coordinate system.

Specifically, according to the calculated coordinates of the same second calibration point A2, the shooting coordinates and the rotation matrix between the coordinate systems where the two stereo cameras are located, the translation matrix between the coordinate systems where the two stereo cameras are located can be obtained according to different second Calibration point A2, which lists the calculated coordinates, shooting coordinates, rotation matrix between the coordinate systems where the two stereo cameras are located, and translation matrices between the coordinate systems where the two stereo cameras are located. Mode.

Specifically, a stereo camera can be set according to the calculated coordinates and shooting coordinates of the same second calibration point A2, for example, the stereo camera 101 corresponding to the second calibration point A2, such as the rotation between the coordinate systems where the stereo camera 102 is located. Matrix, to obtain and set the stereo camera, for example, the stereo camera corresponding to the stereo camera 101 and the second calibration point A2, such as the translation matrix between the coordinate systems where the stereo camera 102 is located, can list many different second calibration points A2. The calculated coordinates of the second calibration point A2, the shooting coordinates, the rotation matrix between the stereo camera 101 and the coordinate system of the stereo camera 102 corresponding to the second calibration point A2, and the stereo camera 101 corresponding to the second calibration point A2. The relational equation between the translation matrices between the coordinate systems where the camera 102 is located is:

Among them, R is the rotation matrix between the coordinate systems where the two stereo cameras are located, and T is the translation matrix between the coordinate systems where the two stereo cameras are located. For example, R can be the setting of the stereo camera, such as the stereo camera 101 and the second standard The stereo camera corresponding to the fixed point A2, such as the rotation matrix between the coordinate systems where the stereo camera 102 is located, T can be the set stereo camera, such as the stereo camera corresponding to the stereo camera 101 and the second calibration point A2, such as the coordinates of the stereo camera 102 The translation matrix between the systems, (Px', Py', Pz') are the calculated coordinates of the second calibration point A2, (Px, Py, Pz) are the shooting coordinates of the second calibration point A2, according to the relational equation to obtain the setting A stereo camera, for example, a stereo camera corresponding to the stereo camera 101 and the second calibration point A2, for example, a translation matrix between the coordinate systems where the stereo camera 102 is located.

Specifically, the calibration transformation matrix includes a rotation matrix and a translation matrix, and the three-dimensional coordinate point P' corresponding to the calculated coordinates of the second calibration point A2 obtained according to the distribution positions of all the calibration points A and the three-dimensional coordinates corresponding to the shooting coordinates of the second calibration point A2 The point P belongs to the same point in the physical world, which is called the point of the same name. The coordinates of P' are (Px', Py', Pz'), and the coordinates of P are (Px, Py, Pz). The formula P'=(RT)P, that is, the above RT relational equation is satisfied. When the calculated coordinates (Px', Py', Pz') of the second calibration point A2 and the shooting coordinates (Px, Py, Pz) of the second calibration point A2 are known, substitute the set stereo obtained by solving in step 103 A camera, such as the stereo camera corresponding to the stereo camera 101 and the second calibration point A2, such as the rotation matrix between the stereo cameras 102, can obtain and set the stereo camera, such as the stereo camera 101 corresponding to the second calibration point A2. Cameras, such as the translation matrix between the coordinate systems where the stereo camera 102 is located.

Optionally, an average value algorithm can be used to obtain the translation matrix between the coordinate systems where the two stereo cameras are located according to the calculated coordinates of the same second calibration point, the shooting coordinates, and the rotation matrix between the coordinate systems where the two stereo cameras are located, for example The average value algorithm can be used to calculate the coordinates, shooting coordinates and set the stereo camera according to the same second calibration point A2, for example, the stereo camera 101 corresponding to the second calibration point A2, for example, the stereo camera 102 is located between the coordinate systems. For the rotation matrix, obtain and set the stereo camera, for example, the stereo camera corresponding to the stereo camera 101 and the second calibration point A2, for example, the translation matrix between the coordinate systems where the stereo camera 102 is located.

Specifically, how many second calibration points A2 are in the field of view of the stereo camera 102, and how many P'=(RT) in the process of converting the coordinate system where the stereo camera 102 is located to the coordinate system where the stereo camera 101 is located The relationship equation of P, Fig. 2 exemplarily sets four second calibration points A2 in the field of view of the stereo camera 102. In the process of realizing the transformation from the coordinate system where the stereo camera 102 is located to the coordinate system where the stereo camera 101 is located, the There are four groups of points with the same name, that is, there are four relational equations of P'=(RT)P, which can be set to obtain the shooting coordinates of the second calibration point A of the second calibration object 2 in eight different poses, then there are co-existence There are 32 groups of points with the same name, and all the 32 groups of points with the same name satisfy the relational equation of P'=(RT)P, that is, 32 relational equations of P'=(RT)P can be listed.

For the relational equation of 32 P'=(RT)P, the average value algorithm can be used to obtain and set the stereo camera, for example, the stereo camera corresponding to the stereo camera 101 and the second calibration point A2, such as the coordinate system where the stereo camera 102 is located. That is, the average value of the translation matrix T calculated by each P'=(RT)P relational equation is calculated, and the stereo camera is finally determined, for example, the stereo camera 101 corresponds to the second calibration point A2. Stereo cameras, such as translation matrices between coordinate systems where the stereo camera 102 is located.

It should be noted that, in the above-mentioned embodiments, S101 to S106 are only used to represent each step in the calibration method of the multi-stereo camera 10, and do not represent a limitation on the execution order of each step. Unless it is absolutely necessary to make one of the two steps There is a strict sequential execution order between the other steps, and the other steps may be performed simultaneously or the sequential execution order may be adjusted, which is not specifically limited in this embodiment of the present disclosure.

In addition, the translation matrix obtained by the solution in the above-mentioned embodiment is the solution result of the average value, so the translation matrix obtained from the sampling group can be compared with the average value of the translation matrix. If it is large, the sampling group can be deleted to improve the accuracy of the translation matrix obtained by the solution. In addition, the process of solving the rotation matrix in the above-mentioned embodiment is an approximate solving process of singular value decomposition. The difference between the normal vectors is large, indicating that the rotation matrix obtained by the sampling group has a large error, and the sampling group can also be deleted to improve the accuracy of the rotation matrix obtained by the solution. In this way, it is possible to use the calibrated rotation matrix to detect the error that the plane normal vectors under each coordinate system are unified into one coordinate system, and to use the calibrated translation matrix to detect the converted error of the points with the same name in each coordinate system, and the rotation matrix The solution and translation matrix are solved separately, and the error analysis is carried out separately, so as to determine the problem of a single matrix of a certain stereo camera 10 in the calibration, so that the calibration accuracy of each rotation matrix and translation matrix can be quantitatively analyzed separately. , so as to improve the precision of the rotation matrix and the translation matrix calibrated by a certain stereo camera 10 in different ways, for example, a certain group of samples can be deleted.

Optionally, in conjunction with FIG. 2 and FIG. 3 , the first calibration object 1 and the second calibration object 2 can be set integrally or independently. Correspondingly, the first calibration object 1 and the second calibration object 2 can be sampled at the same time or can be sampled at the same time. sample separately. Specifically, the first calibration object 1 and the second calibration object 2 can be integrated, the first calibration object 1 includes a calibration plate, and the second calibration object 2 includes a plurality of calibration points A located on a straight line, which can be drawn on the calibration plate A straight line and multiple calibration points A are marked on the straight line. The first calibration object 1 and the second calibration object 2 can also be set independently. For example, the first calibration object 1 can be a block-shaped steel plate, and the second calibration object 2 can be a straight rod, and multiple targets are marked on the straight rod. Fixed point A.

In addition, if the number of stereo cameras 10 to be calibrated is large, and the size of the first calibration object 1 or the second calibration object 2 is not enough, the adjacent stereo cameras 10 can be calibrated first, and then the first calibration object 1 can be moved. and the second calibration object 2 to be calibrated by transfer. For example, the first to fourth stereo cameras 10 can be calibrated by using the first calibration object 1 and the second calibration object 2, and the first to fourth stereo cameras 10 are unified to the location where the first stereo camera 10 is located. Coordinate system, then move the first calibration object 1 and the second calibration object 2, calibrate the fifth to eighth stereo cameras 10, and unify the fifth to eighth stereo cameras 10 to the fifth stereo camera The coordinate system where 10 is located is finally unified to the coordinate system where the first stereo camera 10 is located, so as to prevent the first calibration object 1 or the second calibration object 2 from being too large in size, resulting in heavy calibration of multiple stereo cameras 10 .

An embodiment of the present disclosure further provides a multi-stereo camera calibration device, and FIG. 4 is a schematic structural diagram of a multi-stereo camera calibration device according to an embodiment of the present disclosure. As shown in FIG. 4 , the multi-stereo camera calibration device includes a calibration object setting module 201 , a normal vector obtaining module 202 , a rotation matrix obtaining module 203 , a line fitting module 204 , a calculation coordinate obtaining module 205 and a translation matrix obtaining module 206 .

The calibration object setting module 201 is used for placing the first calibration object in the field of view of the plurality of stereo cameras and placing the second calibration object in the field of view of the plurality of stereo cameras; wherein the first calibration object includes a calibration plate, and the first calibration object includes a calibration plate. The second calibration object includes a plurality of calibration points located on a straight line, and at least two calibration points are visible in each stereo camera. The normal vector acquisition module 202 is used to extract the point cloud to which the first calibration object belongs in the point cloud obtained by the stereo camera under different poses of the first calibration object, and to fit the plane according to the point cloud of the first calibration object obtained by the stereo camera, and obtain The normal vector of the fitted plane. The rotation matrix obtaining module 203 is configured to obtain the rotation moment between the coordinate systems where the two stereo cameras are located according to the normal vectors corresponding to the two stereo cameras. The straight line fitting module 204 is used to obtain the shooting coordinates of the first calibration point and the second calibration point, and fit a straight line according to the shooting coordinates of the first calibration point; wherein, the first calibration point and the second calibration point are located in different three-dimensional The calibration point in the camera's field of view, the second calibration point is located on the fitted straight line. The calculation coordinate obtaining module 205 is configured to obtain the calculation coordinates of the second calibration point according to the shooting coordinates of the first calibration point and the distribution positions of all the calibration points. The translation matrix obtaining module 206 is configured to obtain the translation matrix between the coordinate systems where the two stereo cameras are located according to the calculated coordinates of the same second calibration point, the shooting coordinates and the rotation matrix between the coordinate systems where the two stereo cameras are located.

An embodiment of the present invention further provides an electronic device, and FIG. 5 is a schematic structural diagram of an electronic device provided by an embodiment of the present invention. As shown in FIG. 5 , the electronic device includes a processor and a memory. The processor executes the steps of the method for calibrating a multi-stereo camera according to the above-mentioned embodiment by invoking a program or an instruction stored in the memory. Therefore, it has the beneficial effects of the above-mentioned embodiment. No longer.

As shown in FIG. 5 , the electronic device may be arranged to include at least one processor 301 , at least one memory 302 and at least one communication interface 303 . The various components in the electronic device are coupled together by a bus system 304 . The communication interface 303 is used for information transmission with external devices. It will be appreciated that the bus system 304 is used to implement the connection communication between these components. In addition to the data bus, the bus system 304 also includes a power bus, a control bus and a status signal bus. However, for clarity of illustration, the various buses are labeled as bus system 304 in FIG. 5 .

It is understood that the memory 302 in this embodiment may be a volatile memory or a non-volatile memory, or may include both volatile and non-volatile memory. In some embodiments, the memory 302 stores the following elements: executable units or data structures, or a subset thereof, or an extended set of operating systems and applications. In the embodiment of the present invention, the processor 301 executes the steps of the various embodiments of the calibration method for a multi-stereo camera provided by the embodiment of the present invention by calling a program or an instruction stored in the memory 302 .

The multi-stereo camera calibration method provided in the embodiment of the present invention may be applied to the processor 301 or implemented by the processor 301 . The processor 301 may be an integrated circuit chip, which has signal processing capability. In the implementation process, each step of the above-mentioned method can be completed by an integrated logic circuit of hardware in the processor 301 or an instruction in the form of software. The above-mentioned processor 301 can be a general-purpose processor, a digital signal processor (Digital Signal Processor, DSP), an application specific integrated circuit (Application Specific Integrated Circuit, ASIC), a ready-made programmable gate array (Field Programmable Gate Array, FPGA) or other Programmable logic devices, discrete gate or transistor logic devices, discrete hardware components. A general purpose processor may be a microprocessor or the processor may be any conventional processor or the like.

The steps of the method for calibrating a multi-stereo camera provided by the embodiment of the present invention may be directly embodied as being executed by a hardware decoding processor, or executed by a combination of hardware and software units in the decoding processor. The software unit may be located in random access memory, flash memory, read-only memory, programmable read-only memory or electrically erasable programmable memory, registers and other storage media mature in the art. The storage medium is located in the memory 302, and the processor 301 reads the information in the memory 302, and completes the steps of the method in combination with its hardware.

The electronic device may further include one entity component, or multiple entity components, according to the instructions generated by the processor 301 when executing the calibration method for a multi-stereo camera provided by the embodiment of the present application. Each entity component cooperates with the processor 301 and the memory 302 to realize the functions of the electronic device in this embodiment.

An embodiment of the present invention also provides a storage medium, such as a computer-readable storage medium, the storage medium stores a program or an instruction, the program or instruction enables a computer to execute a multi-stereo camera calibration method when running, and the method includes:

Optionally, when executed by a computer processor, the computer-executable instructions may also be used to execute the technical solution of the calibration method for a multi-stereo camera provided by any embodiment of the present invention.

From the above description of the embodiments, those skilled in the art can clearly understand that the present application can be implemented by software and necessary general-purpose hardware, and of course can also be implemented by hardware, but in many cases, the former is a better implementation. . Based on this understanding, the technical solutions of the present application can be embodied in the form of software products in essence or the parts that make contributions to the prior art, and the computer software products can be stored in a computer-readable storage medium, such as a floppy disk of a computer , read-only memory (Read-Only Memory, ROM), random access memory (Random Access Memory, RAM), flash memory (FLASH), hard disk or optical disk, etc., including several instructions to make a computer device (which can be a personal computer , server, or network device, etc.) to execute the methods of various embodiments of the present invention.

It should be noted that, in this document, relational terms such as "first" and "second" etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply these There is no such actual relationship or sequence between entities or operations. Moreover, the terms "comprising", "comprising" or any other variation thereof are intended to encompass non-exclusive inclusion such that a process, method, article or device comprising a list of elements includes not only those elements, but also includes not explicitly listed or other elements inherent to such a process, method, article or apparatus. Without further limitation, an element qualified by the phrase "comprising a..." does not preclude the presence of additional identical elements in the process, method, article, or device that includes the element.

The above are only specific embodiments of the present disclosure, so that those skilled in the art can understand or implement the present disclosure. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the generic principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the present disclosure. Therefore, the present disclosure is not to be limited to the embodiments herein, but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.

Claims

A method for calibrating a multi-stereo camera, comprising:

The first calibration object is placed in the field of view of a plurality of stereo cameras and the second calibration object is placed in the field of view of the plurality of stereo cameras; wherein, the first calibration object includes a calibration plate, and the second calibration object The calibration object includes a plurality of calibration points located on a straight line, and at least two of the calibration points can be seen in each of the stereo cameras;

Extracting the point cloud to which the first calibration object belongs in the point cloud obtained by the stereo camera when the first calibration object is in different poses, and fitting the plane according to the point cloud to which the first calibration object belongs, obtained by the stereo camera , and obtain the normal vector of the fitted plane;

Obtain the rotation matrix between the coordinate systems where the two stereo cameras are located according to the normal vectors corresponding to the two stereo cameras;

Obtain the shooting coordinates of the first calibration point and the second calibration point, and fit a straight line according to the shooting coordinates of the first calibration point; wherein, the first calibration point and the second calibration point are located at different calibration points in the stereo camera field of view, the second calibration point is located on the fitted straight line;

Obtain the calculated coordinates of the second calibration point according to the shooting coordinates of the first calibration point and the distribution positions of all the calibration points;

The translation matrix between the coordinate systems where the two stereo cameras are located is obtained according to the calculated coordinates and the shooting coordinates of the same second calibration point and the rotation matrix between the coordinate systems where the two stereo cameras are located.
The method for calibrating a multi-stereo camera according to claim 1, wherein before the point cloud fitting plane to which the first calibration object obtained according to the stereo camera belongs, the method further comprises:

The point cloud to which the first calibration object belongs in the point cloud obtained by the stereo camera is extracted by using a clustering algorithm and/or a filtering algorithm.
The method for calibrating a multi-stereo camera according to claim 1 , wherein a least squares algorithm or a random sampling consensus algorithm is used to fit the plane of the point cloud to which the first calibration object obtained by the stereo camera belongs.
The method for calibrating multi-stereo cameras according to claim 1, wherein the acquiring a rotation matrix between the coordinate systems where the two stereo cameras are located according to the normal vectors corresponding to the two stereo cameras, comprises:

According to the normal vectors corresponding to the two stereo cameras, the relationship equation between the normal vector and the rotation matrix is listed, and the relationship equation is:

Wherein, (N 1x , N 1y , N 1z ) are the three-dimensional coordinates of the normal vector corresponding to one of the two stereo cameras, and (N 2x , N 2y , N 2z ) are the two stereo cameras the three-dimensional coordinates of the normal vector corresponding to another stereo camera in the stereo camera, and R21 is the rotation matrix between the coordinate systems where the two stereo cameras are located;

The rotation matrix between the coordinate systems where the two stereo cameras are located is obtained according to the relational equation.
The method for calibrating multiple stereo cameras according to claim 4, wherein a singular value decomposition algorithm is used to obtain the rotation matrix between the coordinate systems where the two stereo cameras are located according to the normal vectors corresponding to the two stereo cameras.
The method for calibrating a multi-stereo camera according to claim 1, wherein the fitting a straight line according to the shooting coordinates of the first calibration point comprises:

A straight line is fitted according to the shooting coordinates of at least two of the first calibration points, so that the at least two first calibration points are located on the fitted straight line.
The method for calibrating a multi-stereo camera according to claim 1, wherein the obtaining the calculated coordinates of the second calibration point according to the shooting coordinates of the first calibration point and the distribution positions of all the calibration points comprises:

Determine the distance between the first calibration point and the second calibration point on the fitted straight line according to the distribution positions of all the calibration points;

The calculated coordinates of the second calibration point are obtained according to the shooting coordinates of the first calibration point, the fitted equation of the straight line, and the distance.
The method for calibrating a multi-stereo camera according to claim 7, wherein the calibration points are arranged at equal intervals;

The determining the distance between the first calibration point and the second calibration point on the fitted straight line according to the distribution positions of all the calibration points includes:

Number all the calibration points in sequence;

The product of the difference between the label of the second calibration point and the label of the first calibration point located on the fitted straight line and the distance between two adjacent calibration points is the The distance between the first calibration point and the second calibration point on the straight line.
The method for calibrating multi-stereo cameras according to claim 1 , wherein said obtaining the all the data is based on the calculated coordinates of the same second calibration point, the shooting coordinates and the rotation matrix between the coordinate systems where the two stereo cameras are located. The translation matrix between the coordinate systems where the two stereo cameras are located, including:

According to the different second calibration points, list the calculated coordinates of the second calibration points, the shooting coordinates, the rotation matrix between the coordinate systems where the two stereo cameras are located, and the coordinates where the two stereo cameras are located. The relationship equation between the translation matrices between the systems;

The relational equation is:

Wherein, R is the rotation matrix between the coordinate systems where the two stereo cameras are located, T is the translation matrix between the coordinate systems where the two stereo cameras are located, and (Px', Py', Pz') is the first The calculated coordinates of the two calibration points, (Px, Py, Pz) are the shooting coordinates of the second calibration point;

The translation matrix between the coordinate systems where the two stereo cameras are located is obtained according to the relational equation.
The method for calibrating multiple stereo cameras according to claim 9, wherein an average value algorithm is used according to the calculated coordinates of the same second calibration point, the shooting coordinates and the rotation matrix between the coordinate systems where the two stereo cameras are located, Acquire the translation matrix between the coordinate systems where the two stereo cameras are located.
The method for calibrating a multi-stereo camera according to claim 1, wherein acquiring the shooting coordinates of the first calibration point comprises:

The shooting coordinates of the first calibration point where the second calibration object is in different poses are acquired.
The multi-stereo camera calibration method according to claim 11, wherein the pose of the first calibration object includes a distance from the first calibration object to the stereo camera, and/or the first calibration object an angle of inclination relative to the stereo camera;

The pose of the second calibration object includes a distance from the second calibration object to the stereo camera, and/or an inclination angle of the second calibration object relative to the stereo camera.
A multi-stereo camera calibration device, comprising:

A calibration object setting module for placing a first calibration object in the field of view of a plurality of stereo cameras and placing a second calibration object in the field of view of the plurality of stereo cameras; wherein the first calibration object includes a calibration plate, the second calibration object includes a plurality of calibration points located on a straight line, and at least two of the calibration points can be seen in each of the stereo cameras;

A normal vector acquisition module, configured to extract the point cloud to which the first calibration object belongs in the point cloud obtained by the stereo camera when the first calibration object is in different poses, and according to the first calibration obtained by the stereo camera The point cloud fitting plane to which the object belongs, and the normal vector of the fitted plane is obtained;

a rotation matrix acquisition module, configured to acquire rotation matrices between the coordinate systems where the two stereo cameras are located according to the normal vectors corresponding to the two stereo cameras;

A straight line fitting module, configured to obtain the shooting coordinates of the first calibration point and the second calibration point, and fit a straight line according to the shooting coordinates of the first calibration point; wherein, the first calibration point and the The second calibration point is a calibration point located in a different field of view of the stereo camera, and the second calibration point is located on the fitted straight line;

a calculation coordinate obtaining module, configured to obtain the calculation coordinates of the second calibration point according to the shooting coordinates of the first calibration point and the distribution positions of all the calibration points;

The translation matrix acquisition module is used to acquire the coordinate system between the coordinate systems where the two stereo cameras are located according to the calculated coordinates of the same second calibration point, the shooting coordinates and the rotation matrix between the coordinate systems where the two stereo cameras are located. Translation matrix.
An electronic device comprising:

memory; and

A processor, wherein the processor executes the steps of the calibration method for a multi-stereo camera according to any one of claims 1-12 by invoking a program or an instruction stored in the memory.
A storage medium, wherein the storage medium stores a program or an instruction, and the program or the instruction causes a computer to execute the steps of the calibration method for a multi-stereo camera according to any one of claims 1-12.