WO2023284349A1

WO2023284349A1 - 3d camera calibration method and apparatus, and calibration system

Info

Publication number: WO2023284349A1
Application number: PCT/CN2022/087716
Authority: WO
Inventors: 胡显东; 缪丰; 贾仕勇
Original assignee: 无锡先导智能装备股份有限公司
Priority date: 2021-07-15
Filing date: 2022-04-19
Publication date: 2023-01-19
Also published as: CN113436277A

Abstract

The present application relates to a 3D camera calibration method and apparatus, and a calibration system. The method comprises: acquiring a scanning point cloud image of a first camera and a scanning point cloud image of a second camera to be calibrated, the scanning point cloud image being a point cloud image generated by scanning a calibration hole on a calibration object; extracting hole center coordinates of a calibration hole in the scanning point cloud image of the first camera to obtain first coordinates, and extracting hole center coordinates of a calibration hole in the scanning point cloud image of the second camera to obtain second coordinates; deriving a normal vector of a plane in which the calibration hole scanned by the first camera is located; calculating extension coordinates of the calibration hole scanned by the second camera in the first camera according to the first coordinates, the normal vector, and preset calibration hole distance data; and computing coordinate conversion relationship data of the first camera and the second camera on the basis of the second coordinates and the extension coordinates. Using the present application, calibration of two cameras can be easily and efficiently accomplished without relying on a common field of view.

Description

3D camera calibration method, device and calibration system

technical field

The present application relates to the technical field of machine vision, in particular to a 3D camera calibration method, device, and calibration system.

Background technique

With the development of technology, 3D cameras that can take stereoscopic images have appeared. In some occasions, two 3D cameras are sometimes used for image processing after shooting. At this time, the cameras need to be calibrated to obtain the coordinate transformation relationship of the two cameras.

In traditionally used calibration methods, a common field of view is usually required, and camera calibration is performed based on the common field of view. For the calibration of two cameras that do not have a common field of view, other electronic tools, such as a laser rangefinder, are usually required for calibration, and the operation is complicated.

Contents of the invention

Based on this, it is necessary to provide a simple and efficient 3D camera calibration method, device and calibration system that does not require a common field of view to address the above technical problems.

A 3D camera calibration method, comprising:

Obtain the scanned point cloud of the first camera to be calibrated and the second camera, the scanned point cloud is a point cloud generated by scanning the calibration hole on the calibration object;

Extracting the center coordinates of the calibration hole in the scanned point cloud image of the first camera to obtain a first coordinate, and extracting the center coordinates of the calibration hole in the scanned point cloud image of the second camera to obtain a second coordinate;

Obtain the normal vector of the plane where the calibration hole scanned by the first camera is located;

calculating the extension coordinates of the calibration hole scanned by the second camera in the first camera according to the first coordinates, the normal vector and the preset calibration hole distance data;

According to the second coordinates and the extension coordinates, coordinate conversion relationship data between the first camera and the second camera is obtained by calculation.

In one of the embodiments, the extracting the center coordinates of the calibration hole in the scanned point cloud image of the first camera to obtain the first coordinates includes:

Extracting edge data points of the calibration hole in the scanned point cloud image of the first camera;

Extracting the center coordinates of the calibration holes according to the edge data points to obtain the first coordinates.

In one of the embodiments, the obtaining the normal vector of the plane where the calibration hole scanned by the first camera includes:

Selecting at least 3 non-collinear points on the surface of the calibration hole scanned by the first camera in the calibration object;

Determine the plane according to the selected points;

Find the normal vector of the plane.

In one of the embodiments, at least 3 non-collinear calibration holes are provided on the opposite surfaces of the calibration object, and the positions of the calibration holes on the opposite surfaces correspond; the scanning point cloud image of the first camera is the first A point cloud image generated by a camera scanning a calibration hole on a surface of the calibration object, and a point cloud image scanned by the second camera is a point cloud image generated by the second camera scanning a calibration hole on the other surface of the calibration object.

In one of the embodiments, the calibration hole distance data includes the calibration object width, and the calibration object width is the width between the two opposite surfaces of the calibration object on which the calibration hole is set; according to the first coordinates, the The normal vector and the preset calibration hole distance data are used to calculate the extension coordinates of the calibration hole scanned by the second camera in the first camera, including:

According to the first coordinates, the normal vector and the width of the calibration object, the extension coordinates of the calibration hole scanned by the second camera in the first camera are estimated.

In one of the embodiments, two rows of calibration holes are provided on the calibration surface of the calibration object, the calibration holes of the same row are in a straight line, and the positions of the calibration holes of the two rows correspond one by one, and the straight line where the calibration holes of the same row are located is the same as that of the two rows. The line between the two calibration holes corresponding to the middle position is vertical;

The scanning point cloud image of the first camera is the point cloud image generated by the first camera scanning a row of calibration holes on the calibration surface, and the scanning point cloud image of the second camera is the scanning point cloud image of the calibration hole scanned by the second camera. The point cloud image generated by another row of calibration holes on the surface.

In one of the embodiments, the calibration hole distance data includes the hole spacing between the two calibration holes corresponding to the positions in the two rows of calibration holes; Hole distance data, calculating the extension coordinates of the calibration hole scanned by the second camera in the first camera, including:

According to the first coordinates of the plurality of calibration holes scanned by the first camera, the linear equation of the straight line where the calibration holes scanned by the first camera is fitted;

According to the first coordinates, the normal vector, the straight line equation and the hole spacing, the extension coordinates of the calibration holes scanned by the second camera in the first camera are calculated.

A 3D camera calibration device, comprising:

The image acquisition module is used to obtain the scanned point cloud images of the first camera and the second camera to be calibrated, and the scanned point cloud images are point cloud images generated by scanning the calibration holes on the calibration object;

A coordinate extraction module, configured to extract the center coordinates of the calibration hole in the scanned point cloud image of the first camera to obtain first coordinates, and extract the center coordinates of the calibration hole in the scanned point cloud image of the second camera to obtain the second coordinate;

A normal vector obtaining module, configured to obtain the normal vector of the plane where the calibration hole scanned by the first camera is located;

A coordinate mapping module, configured to calculate the extension coordinates of the calibration hole scanned by the second camera in the first camera according to the first coordinate, the normal vector and the preset calibration hole distance data;

A relationship acquiring module, configured to calculate coordinate conversion relationship data between the first camera and the second camera according to the second coordinates and the extended coordinates.

A calibration system comprising:

The first camera is used to scan the calibration hole on the calibration object to generate a point cloud image to obtain the scanned point cloud image of the first camera;

The second camera is used to scan the calibration hole on the calibration object to generate a point cloud image to obtain the scanned point cloud image of the second camera;

A controller, the controller is connected to the first camera and the second camera, and includes a memory and a processor, the memory stores a computer program, and the processor implements the following steps when executing the computer program:

In one of the embodiments, the above-mentioned calibration system also includes the calibration object;

At least 3 non-collinear calibration holes are provided on the opposite two surfaces of the calibration object, and the positions of the calibration holes on the opposite two surfaces correspond; or two rows of calibration holes are set on the calibration surface of the calibration object, and the calibration holes in the same row Form a straight line, and the positions of the calibration holes in the two rows correspond one by one, and the line where the calibration holes in the same row are located is perpendicular to the line between the two calibration holes corresponding to the positions in the two rows.

The above-mentioned 3D camera calibration method, device and calibration system extract the first coordinates of the calibration hole scanned by the first camera in the first camera through the scanned point cloud image generated by scanning the calibration hole on the calibration object with the first camera, and according to the second The camera scans the scanned point cloud image generated by the calibration hole on the calibration object, extracts the second coordinate of the calibration hole scanned by the second camera in the second camera, and then according to the first coordinate and the normal vector of the plane where the calibration hole scanned by the first camera And the calibration hole distance data, calculate the extension coordinates of the calibration hole scanned by the second camera in the first camera, and finally calculate the corresponding relationship based on the second coordinates and the extension coordinates, and obtain the two coordinate systems of the first camera and the second camera Convert relational data; in this way, it can be applied to the calibration of two cameras that do not have a common field of view without relying on the common field of view. Only one calibration object can be used to calibrate two cameras simply and efficiently.

Description of drawings

In order to more clearly illustrate the technical solutions in the embodiments of the present application or the prior art, the following will briefly introduce the drawings that need to be used in the description of the embodiments or the prior art. Obviously, the accompanying drawings in the following description are only These are some embodiments of the present application. Those skilled in the art can also obtain other drawings based on these drawings without creative work.

Fig. 1 is a schematic flow chart of a 3D camera calibration method in an embodiment;

Fig. 2 is a schematic diagram of the situation of two cameras scanning in one embodiment;

Fig. 3 is a schematic diagram of two points on opposite sides on the calibration object in one embodiment;

Figure 4 is a schematic diagram of the calibration process of two cameras scanning;

Fig. 5 is a schematic diagram of the situation of scanning stitching of two cameras in one embodiment;

Figure 6 is a schematic diagram of the calibration process of two-camera splicing scanning;

Fig. 7 is a structural block diagram of a 3D camera calibration device in an embodiment.

detailed description

In order to facilitate the understanding of the present application, the present application will be described more fully below with reference to the relevant drawings. Embodiments of the application are given in the drawings. However, the present application can be embodied in many different forms and is not limited to the embodiments described herein. On the contrary, the purpose of providing these embodiments is to make the disclosure of this application more thorough and comprehensive.

Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the technical field to which this application belongs. The terms used herein in the specification of the application are only for the purpose of describing specific embodiments, and are not intended to limit the application.

It can be understood that the terms "first", "second" and the like used in this application may be used to describe various elements herein, but these elements are not limited by these terms. These terms are only used to distinguish one element from another element.

It should be noted that when an element is considered to be "connected" to another element, it may be directly connected to the other element, or connected to the other element through an intervening element. In addition, "connection" in the following embodiments should be understood as "electrical connection", "communication connection" and the like if there is transmission of electrical signals or data between the connected objects.

When used herein, the singular forms "a", "an" and "the/the" may also include the plural forms unless the context clearly dictates otherwise. It should also be understood that the terms "comprising/comprising" or "having" etc. specify the presence of stated features, integers, steps, operations, components, parts or combinations thereof, but do not exclude the presence or addition of one or more The possibility of other features, integers, steps, operations, components, parts or combinations thereof.

In one embodiment, as shown in FIG. 1 , a 3D camera calibration method is provided, which can be applied to a device used for calibration processing, such as a controller. Taking the application to the controller as an example, the method includes the following steps:

S110: Obtain scanned point cloud images of the first camera and the second camera to be calibrated, where the scanned point cloud images are point cloud images generated by scanning the calibration holes on the calibration object.

The first camera and the second camera are 3D cameras that need to be calibrated. Wherein, the calibration object is an object used for calibrating the first camera and the second camera, such as a wooden block, and holes are opened on the calibration object as calibration holes, and the number of calibration holes is multiple. Specifically, the first camera scans a part of the calibration holes on the calibration object, and the second camera scans another part of the calibration holes on the calibration object, that is, the calibration holes scanned by the first camera and the second camera are different, and the first camera and the second camera have no public view. Specifically, the calibration hole may be a round hole, or a hole of other shapes. Before the calibration process, the first camera and the second camera are used to scan the calibration hole on the calibration object to generate scanned point cloud images; the controller acquires the scanned point cloud images of the first camera and the scanned point cloud images of the second camera.

S130: Extracting the coordinates of the center of the calibration hole in the scanned point cloud image of the first camera to obtain a first coordinate, and extracting the coordinates of the center of the calibration hole in the scanned point cloud image of the second camera to obtain a second coordinate.

The coordinates of the center of the calibration hole are the coordinates of the center of the indexed hole. For example, if the calibration hole is a circular hole, the coordinates of the center of the calibration hole are the coordinates of the center of the circle. The first coordinate is the coordinate value of the calibration hole scanned by the first camera in the coordinate system of the first camera, and the second coordinate is the coordinate value of the calibration hole scanned by the second camera in the coordinate system of the second camera. Specifically, the first coordinate and the second coordinate are three-dimensional coordinate values.

Specifically, the first camera scans a plurality of calibration holes, and for each calibration hole in the scanned point cloud image of the first camera, the coordinates of the center of the hole are obtained to obtain the corresponding first coordinates, that is, based on the scanned point cloud image of the first camera, it can be obtained Multiple first coordinates. Similarly, multiple second coordinates can be obtained based on the point cloud image scanned by the second camera.

S150: Calculate the normal vector of the plane where the calibration hole scanned by the first camera is located.

The calibration object has multiple surfaces, and the calibration holes scanned by the same camera are located on the same surface of the calibration object, for example, the calibration holes scanned by the first camera are located on the same surface of the calibration object.

S170: Estimate the extension coordinates of the calibration hole scanned by the second camera in the first camera according to the first coordinate, the normal vector and the preset calibration hole distance data.

Wherein, the calibration hole distance data represents the distance between the calibration hole scanned by the first camera and the calibration hole scanned by the second camera. The extended coordinate refers to the coordinate value of the position of the calibration hole scanned by the second camera in the coordinate system of the first camera. According to the first coordinates, normal vector and calibration hole distance data corresponding to the first camera, the extended coordinates of the calibration hole scanned by the second camera in the first camera are calculated, that is, the calibration hole scanned by the second camera is mapped to the first Coordinate values in the camera's coordinate system.

S190: According to the second coordinates and the extended coordinates, calculate and obtain the coordinate transformation relationship data between the first camera and the second camera.

The second coordinate is the coordinate value of the calibration hole scanned by the second camera in the coordinate system of the second camera, and the extended coordinate is the coordinate value of the calibration hole scanned by the second camera in the coordinate system of the first camera; then, the same The second coordinate and the extension coordinate corresponding to a calibration hole are coordinate values in different coordinate systems. By performing correspondence calculation according to the second coordinates and the extended coordinates, the transformation relation data of the two coordinate systems are obtained, thereby obtaining the coordinate system transformation relation data of the first camera and the second camera.

Wherein, the transformation relation data is data representing a transformation relation between the coordinates of the first camera and the coordinates of the second camera, and may specifically be a matrix. For example, taking the second camera scanning more than 3 calibration holes as an example, the second coordinates and extension coordinates are three-dimensional coordinate values, and the second coordinates of each calibration hole correspond to a row of the matrix, based on the second coordinates of multiple calibration holes Obtain an original matrix; the extension coordinates of each calibration hole correspond to a row of another matrix, and obtain an extension matrix based on the extension coordinates of multiple calibration holes, and then use mathematical algorithms to calculate the transformation from the original matrix to the extension matrix or from the extension matrix Transform the data needed to transform to the original matrix to get the transformation matrix.

The above-mentioned 3D camera calibration method extracts the first coordinates of the calibration hole scanned by the first camera in the first camera from the scanned point cloud image generated by scanning the calibration hole on the calibration object with the first camera, and scans the calibration hole on the calibration object according to the second camera. From the scanned point cloud image generated by the calibration hole, extract the second coordinate of the calibration hole scanned by the second camera in the second camera, and then according to the first coordinate, the normal vector of the plane where the calibration hole is scanned by the first camera, and the distance data of the calibration hole , to calculate the extension coordinates of the calibration hole scanned by the second camera in the first camera, and finally calculate the corresponding relationship based on the second coordinates and the extension coordinates, and obtain the conversion relationship data of the two coordinate systems of the first camera and the second camera; , does not depend on the common field of view, and can be applied to the calibration of two cameras that do not have a common field of view. Only one calibration object can be used to calibrate two cameras simply and efficiently.

In one of the embodiments, in step S130, extracting the coordinates of the center of the calibration hole in the scanning point cloud image of the first camera to obtain the first coordinates includes: extracting the edge data points of the calibration hole in the scanning point cloud image of the first camera; Data points are extracted from the center coordinates of the calibration holes to obtain the first coordinates.

Extract the edge data points of the calibration hole, that is, determine the edge of the calibration hole. By first determining the edge of the calibration hole, and then determining the coordinates of the center point of the calibration hole based on the edge, the accuracy is high. Specifically, the gradient method is used to extract the edge data points of the calibration hole, and the edge can be accurately determined; the center point coordinates of the calibration hole can be accurately obtained by using the space center fitting algorithm to extract the center point of the calibration hole. For example, taking the calibration hole as a circular hole as an example, the edge data points of the circular hole are extracted using the gradient method, and the center coordinates of the circular hole are extracted using the spatial center fitting algorithm.

Specifically, in step S130, the step of extracting the hole center coordinates of the calibration hole in the scanned point cloud image of the second camera to obtain the second coordinates is the same as the specific step of extracting the hole center coordinates of the calibration hole in the scanned point cloud image of the first camera to obtain the first coordinates The same, will not be repeated here.

In one of the embodiments, step S150 includes: selecting at least 3 non-collinear points on the surface of the calibration hole scanned by the first camera in the calibration object; determining the plane according to the selected points; calculating the normal vector of the plane .

On the surface where the calibration hole scanned by the first camera is located, select at least 3 non-collinear points, and the 3 non-collinear points can determine a plane, and then calculate the normal vector of the selected plane. Specifically, the determined at least three non-collinear points are points outside the calibration hole, which are on the same plane as the calibration hole. Compared with directly using the calibration hole to determine the plane, since the coordinates of the calibration hole are obtained by calculating the position of the center of the hole, there is a possibility of error, and the accuracy of determining the plane by taking another point alone is higher. For example, the first camera scans the calibration hole on the left side of the calibration object, and selects three non-collinear points besides the calibration hole on the left side of the calibration object to determine the plane and obtain the normal vector. Specifically, the normal vector of the plane where the calibration hole is located can be extracted through a plane fitting algorithm.

In one of the embodiments, at least 3 non-collinear calibration holes are respectively provided on the opposite surfaces of the calibration object, and the positions of the calibration holes on the opposite surfaces correspond; The point cloud image generated by the surface calibration hole, the point cloud image scanned by the second camera is the point cloud image generated by the second camera scanning the calibration hole on the other surface of the calibration object.

The calibration object is a standard part, preferably an ideal cuboid. Taking the calibration object as a cuboid as an example, the opposite sides of the calibration object include the left side and the right side, the front side and the rear side, the upper side and the lower side. For example, at least 3 The number of calibration holes on both sides is equal, and the positions correspond to each other. The first camera scans the left side and the second camera scans the right side. In this way, this embodiment is applicable to the situation where two cameras scan each other.

As shown in Figure 2, the two cameras are set on the left and right sides of the calibration object, and on the left and right sides of the calibration object, several high-precision round holes (at least 3, and not on a line) are punched, and ensure that the planes on both sides The number of round holes is equal, and the positions of the round holes correspond to each other. Specifically, in order to ensure a one-to-one correspondence between the positions of the round holes, they can be pierced. The first camera (camera 1) scans each calibration hole on the left plane, and the second camera (camera 2) scans each calibration hole on the right plane. In theory, the scanning direction does not need to be parallel to the surface of the calibration object.

In one embodiment, the distance data of the calibration hole includes the width of the calibration object, wherein the width of the calibration object is the width between two opposite surfaces of the calibration object on which the calibration hole is formed. Step S170 includes: calculating the extension coordinates of the calibration hole scanned by the second camera in the first camera according to the first coordinates, the normal vector and the width of the calibration object.

For the situation where two cameras scan against each other, through the first coordinate in the coordinate system of the first camera, the normal vector of the plane scanned by the first camera, and the intrinsic width between the two surfaces marked with the calibration hole of the calibration object, the second The coordinates of the calibration hole scanned by the camera in the coordinate system of the first camera. Specifically, the coordinates can be calculated according to a geometric algorithm. As shown in Figure 3, the field of view of the first camera on the left side of the scanning calibration object is taken as the reference system, the coordinates of point A on the left side of the calibration object are known, and the left and right sides of the calibration object The thickness d between them is known, and the normal vector is known, and the straight line equation passing through point A can be determined according to the normal vector and the coordinates of point A, and then according to the straight line equation, the coordinates of point A and the width of the calibration object (point A and point B The distance between) calculate the coordinates of point B on the right side to obtain the extended coordinates.

As shown in Figure 4, it is a schematic diagram of the calibration process of two-camera scanning. After the calibration program is started, camera 1 and camera 2 scan the calibration object to generate a point cloud image, and then use the obtained point cloud image to fit N circles on the left plane respectively. The three-dimensional coordinate position (circle center coordinate) of the center of the hole in camera 1 and the three-dimensional coordinate position of the center of the N circular holes corresponding to the right plane in camera 2, and then use the normal vector of the left plane in camera 1 and the intrinsic width of the calibration object, Calculate the corresponding three-dimensional coordinates of the centers of the N circular holes in the right plane in camera 1, and finally, after obtaining the three-dimensional coordinates of the centers of the N circular holes in the right plane in camera 2, calculate the coordinate system transformation of camera 1 and camera 2 The relationship obtains the transformation matrix, thus completing the calibration of the two cameras. The coordinate processing in this application is completed in three-dimensional space, and is not sensitive to factors such as camera position and scanning direction, so this application has better robustness and higher precision in practical applications.

In one of the embodiments, two rows of calibration holes are provided on the calibration surface of the calibration object, the calibration holes in the same row are in a straight line, and the positions of the calibration holes in the two rows correspond one by one, and the straight line where the calibration holes are in the same row is in line with the position in the two rows. The connecting line between the corresponding two calibration holes is vertical. Specifically, the positions of the two rows of calibration holes are opposite, one-to-one, and equal in number, that is, the first calibration hole in the first row corresponds to the position of the first calibration hole in the second row, and the second calibration hole in the first row corresponds to the position of the first calibration hole in the second row. The hole corresponds to the position of the second calibration hole in the second row, and so on. Taking two calibration holes in each row as an example, the corresponding two calibration holes in the two rows include the first calibration hole in the first row, the first calibration hole in the second row, and the second calibration hole in the first row. Calibration hole and the second calibration hole in the second row. Taking the calibration surface as an example, two rows of calibration holes can be arranged on two opposite edges of the calibration surface.

Specifically, the scanning point cloud image of the first camera is the point cloud image generated by the first camera scanning a row of calibration holes on the calibration surface, and the scanning point cloud image of the second camera is generated by the second camera scanning another row of calibration holes on the calibration surface point cloud image.

This embodiment is applicable to the case where two cameras are spliced and scanned, that is, the two cameras scan the two sides of the same surface of the object, and the two sides cannot be directly spliced because there is no common field of view. It should be noted that this application is used for 3D cameras, and the above-mentioned splicing is not only the splicing of the left and right, but also the case that the heights may not be equal.

In this embodiment, one surface of the calibration object is used as the calibration surface, and the two sides of the calibration surface are used as the area to be detected. A row of calibration holes is made respectively. The center of each row of calibration holes must be on the same straight line. The number of calibration holes in the two rows Equal, one-to-one correspondence of positions, and it is necessary to ensure that the straight line where the same row of calibration holes is located is perpendicular to the two pairs of calibration holes corresponding to the positions in different rows. For example, as shown in Figure 5, two rows of round holes are set on the upper surface of the calibration object. The first The camera scans the first row of calibration holes, the second camera scans the second row of calibration holes, and finally calculates the transformation matrix of the two camera coordinate systems, which can be used to stitch images or other image joint processing results. In this embodiment, the splicing implementation does not require the field of view of the camera, that is, different images do not need to contain the common field of view, and the field of view of the 3D camera used only needs to cover the detection area. There is no need to select a large field of view camera to include the public field of view, which can ensure Detection accuracy.

In one of the embodiments, the calibration hole distance data includes the hole spacing between the two calibration holes corresponding to the positions in the two rows of calibration holes; for example, the hole spacing can be equal to the first calibration hole in the first row and the second row. The spacing between the first calibration holes. Step S170 includes: according to the first coordinates of the plurality of calibration holes scanned by the first camera, fitting the straight line equation of the straight line where the calibration holes scanned by the first camera are located; according to the first coordinates, the normal vector, the straight line equation and the hole spacing, Estimate the extended coordinates of the calibration hole scanned by the second camera in the first camera.

For the case where two cameras are scanned for splicing, the first coordinate is used to fit the straight line equation, and the first coordinate, normal vector, straight line equation and the hole spacing between the corresponding two calibration holes are used to obtain the calibration of the second camera scan The coordinates of the hole in the first camera's coordinate system. Specifically, the coordinates can be calculated according to geometrical algorithms. For example, in Figure 5, point C is located in the first row, and point D is located in the second row. According to the normal vectors of the straight lines and planes where the multiple calibration holes in the same row as point C are located, the straight line passing through point D can be uniquely determined, so that it can be obtained Obtain the equation of the straight line where the two points of CD are located, and then calculate the coordinates of point D to obtain the extension coordinates according to the equation of the straight line of the two points CD and the coordinates of point C; similarly, the extension coordinates of other calibration holes in the second row can be calculated. As shown in FIG. 6 , it is a schematic diagram of the calibration process of spliced scanning in one embodiment.

It should be understood that although the steps in the flow charts of FIG. 1 , FIG. 4 , and FIG. 6 are shown sequentially as indicated by the arrows, these steps are not necessarily executed sequentially in the order indicated by the arrows. Unless otherwise specified herein, there is no strict order restriction on the execution of these steps, and these steps can be executed in other orders. Moreover, at least some of the steps in FIG. 1, FIG. 4, and FIG. 6 may include multiple steps or multiple stages, and these steps or stages are not necessarily performed at the same time, but may be performed at different times. These steps Or the execution sequence of the stages is not necessarily performed sequentially, but may be executed in turn or alternately with other steps or at least a part of steps or stages in other steps.

In one embodiment, as shown in FIG. 7 , a 3D camera calibration device is provided, including: a map acquisition module 710, a coordinate extraction module 730, a normal vector acquisition module 750, a coordinate mapping module 770 and a relationship acquisition module 790, wherein :

The image acquisition module 710 is used to obtain the scanned point cloud images of the first camera to be calibrated and the second camera, and the scanned point cloud image is a point cloud image generated by scanning the calibration hole on the calibration object; the coordinate extraction module 730 is used to extract the first camera's Scanning the center coordinates of the calibration hole in the point cloud image to obtain the first coordinates, extracting the center coordinates of the calibration hole in the scanning point cloud image of the second camera to obtain the second coordinates; The normal vector of the plane where the calibration hole is located; the coordinate mapping module 770 is used to calculate the extended coordinates of the calibration hole scanned by the second camera in the first camera according to the first coordinates, the normal vector and the preset calibration hole distance data; relationship acquisition The module 790 is used to calculate the coordinate transformation relationship data between the first camera and the second camera according to the second coordinates and the extended coordinates.

The above-mentioned 3D camera calibration device extracts the first coordinates of the calibration hole scanned by the first camera in the first camera by scanning the point cloud image generated by scanning the calibration hole on the calibration object with the first camera, and scans the calibration hole on the calibration object with the second camera. From the scanned point cloud image generated by the calibration hole, extract the second coordinate of the calibration hole scanned by the second camera in the second camera, and then according to the first coordinate, the normal vector of the plane where the calibration hole is scanned by the first camera, and the distance data of the calibration hole , to calculate the extension coordinates of the calibration hole scanned by the second camera in the first camera, and finally calculate the corresponding relationship based on the second coordinates and the extension coordinates, and obtain the conversion relationship data of the two coordinate systems of the first camera and the second camera; , does not depend on the common field of view, and can be applied to the calibration of two cameras that do not have a common field of view. Only one calibration object can be used to calibrate two cameras simply and efficiently.

In one of the embodiments, the coordinate extraction module 730 is used to extract the edge data points of the calibration hole in the scanned point cloud image of the first camera; extract the center of the calibration hole according to the edge data points of the calibration hole in the scanned point cloud image of the first camera Coordinates to obtain the first coordinates; extract the edge data points of the calibration holes in the scanned point cloud image of the second camera; extract the center coordinates of the calibration holes according to the edge data points of the calibration holes in the scanned point cloud image of the second camera to obtain the second coordinates.

In one of the embodiments, the normal vector acquisition module 750 selects at least 3 non-collinear points on the surface where the calibration hole scanned by the first camera in the calibration object is located; the plane is determined according to the selected points; the method for obtaining the plane vector.

Correspondingly, the calibration hole distance data includes the calibration object width, wherein the calibration object width is the width between two opposite surfaces of the calibration object on which the calibration hole is set. The coordinate mapping module 770 calculates the extended coordinates of the calibration hole scanned by the second camera in the first camera according to the first coordinate, the normal vector and the width of the calibration object.

In one of the embodiments, two rows of calibration holes are provided on the calibration surface of the calibration object, the calibration holes in the same row are in a straight line, and the positions of the calibration holes in the two rows correspond one by one, and the straight line where the calibration holes are in the same row is in line with the position in the two rows. The line between the corresponding two calibration holes is vertical; the scanning point cloud image of the first camera is the point cloud image generated by the first camera scanning a row of calibration holes on the calibration surface, and the scanning point cloud image of the second camera is the scanning calibration image of the second camera. The point cloud image generated by another row of calibration holes on the surface.

Correspondingly, the calibration hole distance data includes the hole spacing between the two calibration holes corresponding to the positions in the two rows of calibration holes; the coordinate mapping module 770 fits the first coordinates of the multiple calibration holes scanned by the first camera. The linear equation of the straight line where the calibration hole scanned by the camera is located; according to the first coordinate, the normal vector, the linear equation and the hole spacing, the extended coordinates of the calibration hole scanned by the second camera in the first camera are calculated.

For specific limitations on the 3D camera calibration device, refer to the above-mentioned limitations on the 3D camera calibration method, which will not be repeated here. Each module in the above-mentioned 3D camera calibration device can be fully or partially realized by software, hardware and a combination thereof. The above-mentioned modules can be embedded in or independent of the processor in the controller in the form of hardware, and can also be stored in the memory of the controller in the form of software, so that the processor can invoke and execute the corresponding operations of the above-mentioned modules. It should be noted that the division of modules in the embodiment of the present application is schematic, and is only a logical function division, and there may be other division methods in actual implementation.

In one embodiment, a calibration system is provided, including a first camera, a second camera and a controller. Wherein, the first camera is used to scan the calibration hole on the calibration object to generate a point cloud image to obtain the scanned point cloud image of the first camera; the second camera is used to scan the calibration hole on the calibration object to generate a point cloud image to obtain the scanned point cloud image of the second camera; The controller is connected to the first camera and the second camera, and includes a memory and a processor, the memory stores a computer program, and the processor implements the steps of the above method embodiments when executing the computer program. Specifically, the calibration holes scanned by the first camera and the second camera are different.

Since the above-mentioned calibration system includes a controller capable of implementing the steps of the above-mentioned method embodiments, similarly, it does not need to rely on the public field of view, and can simply and efficiently realize the calibration of two cameras.

In one of the embodiments, the above-mentioned calibration system further includes a calibration object; at least three non-collinear calibration holes are provided on the opposite surfaces of the calibration object, and the positions of the calibration holes on the opposite surfaces correspond to each other.

In one of the embodiments, the above-mentioned calibration system also includes a calibration object; two rows of calibration holes are arranged on the calibration surface of the calibration object, the calibration holes in the same row are in a straight line, and the positions of the calibration holes in the two rows correspond to each other, and the calibration holes in the same row are aligned. The straight line where the hole is located is perpendicular to the line between the two calibration holes corresponding to the positions in the two rows.

Regarding the computer program executed by the processor of the controller in the calibration system, for specific limitations, please refer to the above-mentioned limitations on the 3D camera calibration method, which will not be repeated here.

Those of ordinary skill in the art can understand that all or part of the processes in the methods of the above embodiments can be implemented through computer programs to instruct related hardware, and the computer programs can be stored in a non-volatile computer-readable memory In the medium, when the computer program is executed, it may include the processes of the embodiments of the above-mentioned methods. Wherein, any references to memory, storage, database or other media used in the various embodiments provided in the present application may include at least one of non-volatile memory and volatile memory. Non-volatile memory may include read-only memory (Read-Only Memory, ROM), magnetic tape, floppy disk, flash memory or optical memory, etc. Volatile memory can include Random Access Memory (RAM) or external cache memory. By way of illustration and not limitation, RAM can take many forms, such as Static Random Access Memory (SRAM) or Dynamic Random Access Memory (DRAM).

In the description of this specification, descriptions referring to the terms "some embodiments", "other embodiments", "ideal embodiments" and the like mean that specific features, structures, materials, or characteristics described in connection with the embodiments or examples are included in this specification. In at least one embodiment or example of the invention. In this specification, schematic descriptions of the above terms do not necessarily refer to the same embodiment or example.

The technical features of the above embodiments can be combined arbitrarily. To make the description concise, all possible combinations of the technical features in the above embodiments are not described. However, as long as there is no contradiction in the combination of these technical features, they should be It is considered to be within the range described in this specification.

The above-mentioned embodiments only represent several implementation modes of the present application, and the description thereof is relatively specific and detailed, but it should not be construed as limiting the scope of the patent for the invention. It should be noted that those skilled in the art can make several modifications and improvements without departing from the concept of the present application, and these all belong to the protection scope of the present application. Therefore, the scope of protection of the patent application should be based on the appended claims.

Claims

A 3D camera calibration method, characterized in that, comprising:

Obtain the scanned point cloud of the first camera to be calibrated and the second camera, the scanned point cloud is a point cloud generated by scanning the calibration hole on the calibration object;

Extracting the center coordinates of the calibration hole in the scanned point cloud image of the first camera to obtain a first coordinate, and extracting the center coordinates of the calibration hole in the scanned point cloud image of the second camera to obtain a second coordinate;

Obtain the normal vector of the plane where the calibration hole scanned by the first camera is located;

calculating the extension coordinates of the calibration hole scanned by the second camera in the first camera according to the first coordinates, the normal vector and the preset calibration hole distance data;

According to the second coordinates and the extension coordinates, coordinate conversion relationship data between the first camera and the second camera is obtained by calculation.
The method according to claim 1, wherein said extracting the coordinates of the center of the calibration hole in the scanned point cloud image of the first camera to obtain the first coordinates comprises:

Extracting edge data points of the calibration hole in the scanned point cloud image of the first camera;

Extracting the center coordinates of the calibration holes according to the edge data points to obtain the first coordinates.
The method according to claim 1, wherein the obtaining the normal vector of the plane where the calibration hole scanned by the first camera comprises:

Selecting at least 3 non-collinear points on the surface of the calibration hole scanned by the first camera in the calibration object;

Determine the plane according to the selected points;

Find the normal vector of the plane.
The method according to any one of claims 1-3, wherein at least 3 non-collinear calibration holes are provided on the opposite surfaces of the calibration object, and the positions of the calibration holes on the opposite surfaces correspond; The scanning point cloud image of the first camera is the point cloud image generated by the first camera scanning the calibration hole on the surface of the calibration object, and the scanning point cloud image of the second camera is the scanning point cloud image of the calibration object scanned by the second camera. A point cloud generated by calibration holes on a surface.
The method according to claim 4, wherein the calibration hole distance data includes the calibration object width, and the calibration object width is the width between the two opposite surfaces of the calibration hole on which the calibration object is set; The first coordinate, the normal vector and the preset calibration hole distance data are used to calculate the extension coordinates of the calibration hole scanned by the second camera in the first camera, including:

According to the first coordinates, the normal vector and the width of the calibration object, the extension coordinates of the calibration hole scanned by the second camera in the first camera are estimated.
According to the method described in any one of claims 1-3, it is characterized in that two rows of calibration holes are provided on the calibration surface of the calibration object, the calibration holes of the same row are in a straight line, and the calibration holes of the two rows are in the same position. One corresponding, the straight line where the calibration holes are located in the same row is perpendicular to the line between the two calibration holes corresponding to the positions in the two rows;

The scanning point cloud image of the first camera is the point cloud image generated by the first camera scanning a row of calibration holes on the calibration surface, and the scanning point cloud image of the second camera is the scanning point cloud image of the calibration hole scanned by the second camera. The point cloud image generated by another row of calibration holes on the surface.
The method according to claim 6, wherein the calibration hole distance data includes the hole spacing between the two calibration holes corresponding to the positions in the two rows of calibration holes; Vector and preset calibration hole distance data to calculate the extended coordinates of the calibration hole scanned by the second camera in the first camera, including:

According to the first coordinates of the plurality of calibration holes scanned by the first camera, the linear equation of the straight line where the calibration holes scanned by the first camera is fitted;

According to the first coordinates, the normal vector, the straight line equation and the hole spacing, the extension coordinates of the calibration holes scanned by the second camera in the first camera are calculated.
A 3D camera calibration device is characterized in that it comprises:

The image acquisition module is used to obtain the scanned point cloud images of the first camera to be calibrated and the second camera, and the scanned point cloud images are point cloud images generated by scanning the calibration holes on the calibration object;

A coordinate extraction module, configured to extract the center coordinates of the calibration hole in the scanned point cloud image of the first camera to obtain first coordinates, and extract the center coordinates of the calibration hole in the scanned point cloud image of the second camera to obtain the second coordinate;

A normal vector obtaining module, configured to obtain the normal vector of the plane where the calibration hole scanned by the first camera is located;

A coordinate mapping module, configured to calculate the extension coordinates of the calibration hole scanned by the second camera in the first camera according to the first coordinate, the normal vector and the preset calibration hole distance data;

A relationship acquiring module, configured to calculate coordinate transformation relationship data between the first camera and the second camera according to the second coordinates and the extended coordinates.
A calibration system, characterized in that it comprises:

The first camera is used to scan the calibration hole on the calibration object to generate a point cloud image to obtain the scanned point cloud image of the first camera;

The second camera is used to scan the calibration hole on the calibration object to generate a point cloud image to obtain the scanned point cloud image of the second camera;

A controller, the controller is connected to the first camera and the second camera, and includes a memory and a processor, the memory stores a computer program, and the processor implements claims 1-7 when executing the computer program The steps of any one of the described methods.
The calibration system according to claim 9, further comprising the calibration object;

At least 3 non-collinear calibration holes are respectively provided on the opposite surfaces of the calibration object, and the positions of the calibration holes on the opposite two surfaces correspond; or two rows of calibration holes are set on the calibration surface of the calibration object, and the calibration holes in the same row Form a straight line, and the positions of the calibration holes in the two rows correspond one by one, and the line where the calibration holes in the same row are located is perpendicular to the line between the two calibration holes corresponding to the positions in the two rows.