CN115471562A

CN115471562A - TOF module calibration method and device and electronic equipment

Info

Publication number: CN115471562A
Application number: CN202110656575.9A
Authority: CN
Inventors: 吴勇辉
Original assignee: Shenzhen Goodix Technology Co Ltd
Current assignee: Shenzhen Goodix Technology Co Ltd
Priority date: 2021-06-11
Filing date: 2021-06-11
Publication date: 2022-12-13

Abstract

The application provides a calibration method and a calibration device of a TOF module, and electronic equipment, wherein the method comprises the following steps: acquiring a calibration plate image, wherein the calibration plate image comprises at least three calibration plates with different poses, and each calibration plate comprises a plurality of target points; determining internal and external parameters of the TOF module according to the image coordinate of each target point and the corresponding world coordinate; determining the real optical path of each pixel point in the calibration plate image according to the internal and external parameters of the TOF module; and determining the optical path error of the TOF module according to the real optical path of each pixel point and the measured optical path detected by the TOF module. Therefore, the calibration efficiency of the TOF module is improved.

Description

TOF module calibration method and device and electronic equipment

Technical Field

The present disclosure relates to depth sensing technologies, and in particular, to a method and an apparatus for calibrating a Time of flight (TOF) depth measurement module, and an electronic device.

Background

The TOF depth measurement module (referred to as TOF module for short) is a device for acquiring depth (i.e. three-dimensional) information of an object, and is a very critical link for calibration of the TOF module, and whether the calibration result of the TOF module is accurate or not directly affects the accuracy of the depth obtained in the subsequent working process of the TOF module.

The calibration of the TOF module comprises two parts of parameter calibration and error calibration: parameter calibration, namely determining internal and external parameters of a sensing assembly of the TOF module, such as a focal length, an imaging center, distortion, a rotation matrix, a translation matrix and the like; and error calibration is to determine the measurement error of the TOF module, and is used for carrying out error compensation in the process of acquiring the depth, so that the depth precision is improved.

In the traditional method, the parameter calibration and the error calibration of the TOF module are calibrated in two stations on a production line. And at the first station, calibrating internal and external parameters of the TOF module by means of a three-dimensional calibration plate or a two-dimensional calibration plate and target points with known world coordinates on the calibration plate. And at the second station, measuring the measured depth of the calibration plate by the TOF module by virtue of the calibration plate with the known real depth, and calibrating the error of the TOF module by virtue of the difference between the known real depth and the measured depth. However, this calibration method using two stations is inefficient.

Disclosure of Invention

The application provides a calibration method and device of a TOF module and electronic equipment, and improves calibration efficiency of the TOF module.

In a first aspect, the present application provides a calibration method for a TOF module, including:

acquiring a calibration plate image, wherein the calibration plate image comprises at least three calibration plates with different poses, and each calibration plate comprises a plurality of target points;

determining internal and external parameters of the TOF module according to the image coordinate of each target point and the corresponding world coordinate;

determining the real optical path of each pixel point in the calibration plate image according to the internal and external parameters of the TOF module;

and determining the optical path error of the TOF module according to the real optical path of each pixel point and the measured optical path detected by the TOF module.

In an embodiment, the determining the true optical path of each pixel point in the calibration plate image according to the internal and external parameters of the TOF module includes:

determining the world coordinate of each pixel point according to the internal and external parameters of the TOF module and the image coordinate of each pixel point;

determining the coordinate of each pixel point in a TOF module coordinate system according to the world coordinate of each pixel point and the external parameters of the TOF module;

and determining the real optical path of each pixel point according to the coordinate of each pixel point in a TOF module coordinate system.

In one embodiment, the optical path error of the TOF module includes the optical path error of each pixel point;

the determining the optical path error of the TOF module according to the real optical path of each pixel point and the measured optical path detected by the TOF module comprises:

and respectively determining the optical path error of each pixel point according to the real optical path of each pixel point and the measured optical path detected by the TOF module.

In an embodiment, the determining an optical path error of the TOF module according to the real optical path of each pixel and the measured optical path detected by the TOF module includes:

obtaining an optical path error of each pixel point according to the real optical path of each pixel point and the measured optical path detected by the TOF module;

and determining the average value of the optical path errors of each pixel point as the optical path error of the TOF module, wherein the optical path error of the TOF module is used for correcting the measured optical path of each pixel point.

In one embodiment, the optical path error of the TOF module comprises optical path errors of a plurality of pixel regions;

dividing the calibration plate image into a plurality of pixel regions, and obtaining the optical path error of each pixel point in each pixel region according to the real optical path of the pixel point in each pixel region and the measured optical path detected by the TOF module;

and determining the average value of the optical path errors of the pixels in each pixel area as the optical path error of each pixel area, wherein the optical path error of each pixel area is used for correcting the measured optical path of each pixel in each pixel area.

In one embodiment, each calibration plate includes a plurality of circular patterns thereon, and the target point is a center point of the circular patterns;

before determining internal and external parameters of the TOF module according to the image coordinates and the corresponding world coordinates of each target point, the method further comprises:

determining an inner edge and an outer edge of each annular pattern;

and determining the image coordinates of each target point according to the inner edge and the outer edge of each annular pattern.

In one embodiment, said determining the inner and outer edges of each of said annular patterns comprises:

and determining the inner edge and the outer edge of each annular pattern at the pixel level according to the gray value of each pixel point in the calibration plate image, and determining the inner edge and the outer edge of each annular pattern at the pixel level as the inner edge and the outer edge of each annular pattern.

In one embodiment, the acquiring the inner edge and the outer edge of each annular pattern in the calibration plate image comprises:

determining the inner edge and the outer edge of each annular pattern at the pixel level according to the gray value of each pixel point in the calibration plate image;

and performing smoothing processing on the inner edge and the outer edge of the pixel level to obtain the inner edge and the outer edge of the sub-pixel level, and determining the inner edge and the outer edge of the sub-pixel level as the inner edge and the outer edge of each annular pattern.

In one embodiment, said determining image coordinates of said each target point from said inner and outer edges of said each annular pattern comprises:

dividing the region between the inner edge and the outer edge of each annular pattern to obtain a plurality of divided blocks;

and calculating the gravity center of each segmentation block, fitting the gravity centers of the segmentation blocks, and determining the image coordinates of each target point according to the fitting result.

In a second aspect, an embodiment of the present application provides a calibration apparatus for a TOF module, including:

the system comprises an acquisition module, a display module and a display module, wherein the acquisition module is used for acquiring a calibration plate image which comprises a plurality of target points;

the first determining module is used for determining internal and external parameters of the TOF module according to the image coordinate of each target point and the corresponding world coordinate;

the second determining module is used for determining the real optical path of each pixel point in the calibration board image according to the internal and external parameters;

and the calibration module is used for determining the optical path error of the TOF module according to the real optical path of each pixel point and the measured optical path detected by the TOF module.

In one embodiment, the second determining module is configured to:

and determining the real optical path of each pixel point according to the coordinate of each pixel point in the TOF module coordinate system.

the calibration module is used for:

In one embodiment, the calibration module is configured to:

obtaining the optical path error of each pixel point according to the real optical path of each pixel point and the measured optical path detected by the TOF module;

the calibration module is used for:

calibration device of TOF module still includes:

a third determining module for determining an inner edge and an outer edge of each of the annular patterns; and determining the image coordinates of each target point according to the inner edge and the outer edge of each annular pattern.

In one embodiment, the third determining module is configured to:

and smoothing the inner edge and the outer edge of the pixel level to obtain the inner edge and the outer edge of the sub-pixel level, and determining the inner edge and the outer edge of the sub-pixel level as the inner edge and the outer edge of each annular pattern.

In one embodiment, the third determining module is configured to:

and calculating the gravity center of each segmentation block, fitting the gravity centers of the plurality of segmentation blocks, and determining the image coordinates of each target point according to the fitting result.

In a third aspect, the present application provides an electronic device comprising a memory and a processor, the memory and the processor being connected;

the memory is used for storing a computer program;

the processor is adapted to carry out the method according to the first aspect and embodiments thereof when the computer program is executed.

In a fourth aspect, the present application provides a computer-readable storage medium having stored thereon a computer program which, when executed by a processor, implements the method according to the first aspect and embodiments thereof.

In a fifth aspect, the present application provides a computer program product comprising a computer program which, when executed by a processor, implements the method of the first aspect and embodiments thereof.

The application provides a calibration method, a calibration device and electronic equipment of a TOF module, which are characterized in that a one-stop calibration method is adopted to calibrate parameters and errors of the TOF module, after internal and external parameters of the TOF module are calibrated by calibration plates with different poses, the internal and external parameters of the TOF module are directly utilized to calculate the real optical path of each pixel point, and then according to the difference between the measured optical path and the real optical path detected by the TOF module, the optical path errors of the TOF module are calibrated, so that two kinds of calibration of the TOF module are completed at one time, and the calibration efficiency is improved.

Drawings

In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings used in the description of the embodiments or the prior art will be briefly described below, and it is obvious that the drawings in the description below are some embodiments of the present application, and those skilled in the art can obtain other drawings based on the drawings without inventive labor.

Fig. 1 is a schematic diagram illustrating a relationship between a depth and an optical path according to an embodiment of the present disclosure;

fig. 2 is a first schematic flowchart of a calibration method of a TOF module according to an embodiment of the present disclosure;

FIG. 3 is a schematic diagram of a calibration plate image provided in an embodiment of the present application;

fig. 4 is a second schematic flowchart of a calibration method of a TOF module according to an embodiment of the present disclosure;

FIG. 5 is a schematic diagram of a ring pattern segmentation provided in an embodiment of the present application;

FIG. 6 is a schematic diagram of an optical path error curve according to an embodiment of the present application;

fig. 7 is a schematic structural diagram of a calibration apparatus of a TOF module according to an embodiment of the present disclosure;

fig. 8 is a schematic structural diagram of an electronic device according to an embodiment of the present application.

Detailed Description

In order to make the objects, technical solutions and advantages of the embodiments of the present application clearer, the technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the drawings in the embodiments of the present application, and it is obvious that the described embodiments are some embodiments of the present application, but not all embodiments. All other embodiments obtained by a person of ordinary skill in the art based on the embodiments in the present application without making any creative effort belong to the protection scope of the present application.

In various applications such as image measurement and machine vision, it is often necessary to calculate geometric information of an object in a three-dimensional space from acquired image information, for example, to perform three-dimensional object reconstruction, to identify depth information of the object, and the like. One of the methods commonly used today is to perform depth measurement of a target object by using the time interval t from transmission to reception of an optical signal emitted by a TOF module (e.g., a TOF camera) or the phase difference generated by light traveling to and from the target object. Before the TOF module is put into practical use, the TOF module is usually required to be calibrated, so that the accuracy of the depth obtained in the subsequent working process of the TOF module is ensured.

Because the calibration plates adopted for the parameter calibration and the error calibration of the TOF module are different, the two calibration procedures are performed on a production line in two stations in the conventional practice in the industry. And at the first station, calibrating internal and external parameters of the TOF module by means of calibration plate images of a plurality of calibration plates with different poses and target points with known world coordinates on the calibration plates. In the second station, a calibration plate with known real depth (distance) is fixedly placed, the measured depth of the calibration plate is measured through the TOF module, and then the depth error of the TOF module is calibrated according to the difference between the known real depth and the measured depth. However, this calibration method using two stations is inefficient.

In the method, the internal and external parameters of the TOF module are calibrated by using the calibration plate image, then the depth of each point on the calibration plate is calculated by using the internal and external parameters of the TOF module and is used as the real depth of each point, and then the depth error of the TOF module is calibrated according to the measured depth detected by the TOF module and the real depth obtained by using the internal and external parameters of the TOF module. By the one-stop calibration method, the calibration efficiency is improved.

Before describing the calibration method of the TOF module in detail in the embodiments of the present application, first, the concepts of depth and optical path are described. As shown in FIG. 1, X _w Y _w Z _w The coordinate system is a world coordinate system; o is _c XYZ coordinate system being the TOF module coordinate system, i.e. the camera coordinate system, the Z axis being the TOF moduleAn optical axis; o is _I The uv coordinate system is an image coordinate system, i.e. an image pixel coordinate system, for describing the position of the pixel points in the image. Wherein, the world coordinate system is used for describing the position of the calibration plate in the three-dimensional world; the coordinates of the world coordinate system, the camera coordinate system and the image coordinate system are all expressed by Cartesian coordinates. Point P is a point on the calibration plate and I (u, v) is the image point corresponding to point P. Point in the figure (C) _x, C _y ) Is the optical center and is the origin O of the camera coordinate system _c Projected position in the image, f _x The focal length is the optical center and the focal length are all internal parameters of the TOF module. Point P in the figure and origin O of the camera coordinate system _c The distance between the point P and the point Z is the optical path of the point P, and the point Q and the point O of the intersection of the point P and the vertical line of the Z axis _c Is the depth of the point P. When the TOF module calculates the depth, firstly, the TOF module calculates the optical path according to the phase information, and then determines the depth according to the included angle θ between the optical path and the depth, wherein θ can be determined according to the formula tan (θ) = | I (u, v) - (C) _x, C _y, )‖/f _x And (4) determining.

Based on the above relationship between the depth and the optical path, in the embodiment of the present application, the TOF module is subjected to depth error calibration, and actually, the optical path error is calibrated, that is, the optical path of each point on the calibration plate is calculated by using the internal and external parameters of the TOF module, the optical path calculated by using the internal and external parameters is used as the true optical path of each point, and then the optical path error of the TOF module is calibrated according to the measured optical path detected by the TOF module and the true optical path obtained by using the internal and external parameters of the TOF module. It can be understood that, based on the above correspondence between the depth and the optical path, calibrating the optical path error of the TOF module is equivalent to calibrating the depth error, which improves the depth accuracy.

The calibration method of the TOF module provided by the present application will be described in detail through specific embodiments. It is to be understood that the following detailed description may be combined with other embodiments, and that the same or similar concepts or processes may not be repeated in some embodiments.

Fig. 2 is a first schematic flow chart of a calibration method of a TOF module according to an embodiment of the present disclosure.

As shown in fig. 2, the method includes:

s201, obtaining a calibration plate image.

The calibration plate image comprises at least three calibration plates with different poses, and each calibration plate comprises a plurality of target points.

In order to realize parameter calibration of the TOF module, the calibration plate image needs to include at least three viewing angles of the calibration plate, so that at least three calibration plates with different poses need to be photographed, each calibration plate includes more than four target points, and the world coordinates of the target points are predetermined, that is, the world coordinates of the target points are known. The calibration plate may include patterns such as a checkerboard, a solid circle, or a circular ring, and the target point may be an angular point of the checkerboard, a center of the solid circle, or a center of the circular ring, which is not limited in the embodiment of the present application.

It should be noted that the calibration plate image in the embodiment of the present application may include an amplitude image and a phase image. Specifically, when the calibration plate is shot by the TOF module, multiple sampling, for example, four or eight sampling, is performed to obtain a sequence diagram, and an amplitude image and a phase image can be calculated through the sequence diagram, where the amplitude image, that is, a grayscale image, can be used for parameter calibration; the phase image can be used for calculating depth information and error calibration. The following description of the embodiments of the present application will not be specifically explained in a differentiation manner. For example, as shown in fig. 3, 4 calibration plates with different poses, a calibration plate 301, a calibration plate 302, a calibration plate 303, and a calibration plate 304, are photographed to obtain calibration plate images, each calibration plate includes a plurality of circular rings (e.g., circular ring a in the figure), and the target point is a central point of a circular ring. It is understood that, since the calibration plate is placed obliquely in different poses, the circular ring appearing in the captured image may not be a perfect circular ring but an elliptical circular ring.

S202, determining internal and external parameters of the TOF module according to the image coordinates and the corresponding world coordinates of each target point.

The image coordinates of each target point are the coordinates of the image point of the target point in the image coordinate system, and the world coordinates of each target point are the coordinates of the target point in the world coordinate system of the calibration board. The world coordinate of each target point is known in the embodiment of the application, and the image coordinate of each target point can be determined according to the calibration plate image, so that the internal and external parameters of the TOF module can be determined according to the image coordinate of the target point and the corresponding world coordinate.

And S203, determining the real optical path of each pixel point in the calibration plate image according to the internal and external parameters of the TOF module.

The real optical path of each pixel point in the calibration plate image refers to the real distance between a point on the calibration plate corresponding to each pixel point and the TOF module, and specifically refers to the real distance between a point on the calibration plate corresponding to each pixel point and the origin of the coordinate system of the TOF module. The real distance can be determined according to coordinates of points on the calibration plate under a TOF module coordinate system, and coordinates of points on the calibration plate corresponding to each pixel point under the TOF module coordinate system can be determined based on internal and external parameters of the TOF module and image coordinates of each pixel point.

And S204, determining the optical path error of the TOF module according to the real optical path of each pixel point and the measured optical path detected by the TOF module.

The measured optical path detected by the TOF module is the measured distance between a point on the calibration plate corresponding to each pixel point determined by the TOF module based on the phase image and the origin of the coordinate system of the TOF module, and the measured distance is calculated and obtained by the TOF module based on the algorithm of the TOF module. Specifically, the TOF module includes optical transmission module and light receiving module, and optical transmission module can be infrared laser emitter, and light receiving module can be CMOS image sensor, transmits specific wavelength infrared light to calibration board through optical transmission module, receives the infrared light that calibration board reflects by light receiving module under the shutter control, forms the light sensing image (phase image) of a plurality of different phases, determines the distance between calibration board and the TOF module according to the phase image, namely obtains measuring distance. In this step, the real optical path obtained in S203 is used as a standard value, so as to determine the error of the measured optical path of the TOF module, and further, the optical path can be corrected and compensated in the subsequent measurement.

According to the embodiment of the application, the TOF module is subjected to parameter calibration and error calibration by adopting a one-stop calibration method, after internal and external parameters of the TOF module are calibrated by virtue of calibration plates with different poses, the real optical path of each pixel point is calculated by directly utilizing the internal and external parameters of the TOF module, and then the optical path error of the TOF module is calibrated according to the difference between the measured optical path and the real optical path detected by the TOF module, so that two kinds of calibration of the TOF module are completed at one time, the calibration efficiency is improved, the time is saved, and the cost is also saved.

The above embodiments are further described below with reference to specific examples. Fig. 4 is a second flowchart illustrating a calibration method of a TOF module according to an embodiment of the present disclosure. As shown in fig. 4, the method includes:

s401, obtaining a calibration plate image.

The calibration plate used in this embodiment has a circular pattern as shown in fig. 2, and the target point is the center point of the circular pattern. The background of the calibration plate may be white and the annular ring portion may be black or gray. The size of each annular pattern on the calibration plate can be the same or different, and the interval between the annular patterns can also be the same or different, which is not limited in the embodiments of the present application. The calibration plate substrate is made of reflective material, and the ring pattern of the target points is also made of reflective material, but because it is black or gray, the reflectivity is slightly different from that of the substrate.

S402, determining the inner edge and the outer edge of each annular pattern.

Because the annular pattern in the calibration plate has obvious differentiation with the color of the background part of calibration plate, consequently can acquire the edge of every annular pattern through carrying out edge detection to the calibration plate image, the edge of annular pattern includes inside edge and outward flange, circle and excircle promptly.

In one embodiment, the inner and outer edges at the pixel level of each annular pattern are determined based on the gray value of each pixel point in the calibration plate image, and the inner and outer edges at the pixel level of each annular pattern are determined as the inner and outer edges of each annular pattern.

Because the colors of the annular pattern in the calibration plate and the background part of the calibration plate are obviously distinguished, the edge and the margin of the annular pattern can be determined according to the change of the gray value by detecting the gray value of each pixel point in the image of the calibration plate. For example, by setting a gray-scale variation threshold, when the gray-scale variation of the adjacent pixels is larger than the gray-scale variation threshold, the pixels with lower gray-scale values are determined as the pixels at the edge of the ring-shaped pattern. The inner edge and the outer edge of each annular pattern at the pixel level can be determined by detecting the gray value of the pixel point in the whole calibration plate image.

In another embodiment, determining the inner edge and the outer edge of each annular pattern at the pixel level according to the gray value of each pixel point in the calibration plate image; and smoothing the inner edge and the outer edge of the pixel level to obtain the inner edge and the outer edge of the sub-pixel level, and determining the inner edge and the outer edge of the sub-pixel level as the inner edge and the outer edge of each annular pattern.

For the TOF module with low resolution, that is, with a low pixel, subsequent processing is performed based on the edge at the pixel level, which may result in low accuracy, in this embodiment of the application, after the inner edge and the outer edge at the pixel level of each annular pattern are determined, the inner edge and the outer edge at the pixel level may be further smoothed, for example, the gray values of the pixel points at the inner edge and the outer edge at the pixel level are interpolated by using an interpolation method, so as to obtain the inner edge and the outer edge at the sub-pixel level. For example, the inner edge and the outer edge at the sub-pixel level can be obtained by fitting the gray values of the pixel points at the inner edge and the outer edge at the pixel level.

And S403, determining the image coordinates of each target point according to the inner edge and the outer edge of each annular pattern.

The inner and outer edges of the annular pattern together define the extent of the annulus of the annular pattern, so that the image coordinates of the target point can be determined together from the inner and outer edges.

As an example, a region between the inner edge and the outer edge of each annular pattern (annular region) is divided to obtain a plurality of divided blocks; and calculating the gravity center of each segmentation block, fitting the gravity centers of the segmentation blocks, and determining the image coordinates of each target point according to the fitting result.

After the inner edge and the outer edge of each annular pattern are determined, the annular region between the inner edge and the outer edge is divided, and optionally, the region between the inner edge and the outer edge of each annular pattern is divided according to the normals of the inner edge and/or the outer edge of each annular pattern by determining the normals of the inner edge and/or the outer edge at a plurality of positions. Taking the normal of the inner edge as an example for explanation, the inner edge is divided into a plurality of arc segments, an arc AB illustrated in fig. 5 is one of the arc segments, normals (shown by dotted lines in the figure) of the arc are respectively determined at points a and B, and the two normals intersect with the outer edge, so that a region illustrated in a grid shape and formed between the two normals and the inner and outer edges is a divided block (for clarity, the annular region in fig. 5 is not filled with color). It should be noted that, when the circular ring is divided, the size of the divided block may be set as needed, for example, the inner edge is divided into a plurality of arc segments by taking two pixels as a unit, that is, each arc segment occupies two pixels, and then the divided block is divided according to the foregoing method.

For each of the divided blocks obtained by the division, the image coordinates of the barycentric position thereof are calculated. For example, the center of gravity of the segment may be determined from the gray value of each pixel in the segment by a gray center method. The three points shown in fig. 5 are the centers of gravity of the three divided blocks, respectively, and the other divided blocks and the centers of gravity thereof are not illustrated in the figure.

And then, fitting the centers of gravity of the plurality of segmentation blocks, and determining the image coordinates of the target point according to a fitting curve. Because the calibration plate is obliquely arranged, a fitting curve obtained by fitting the centers of the plurality of segmentation blocks is usually an ellipse, and the image coordinate of the center of the ellipse is calculated to be the image coordinate of the target point. For example, the center of gravity of the plurality of segments may be fitted by an average value method, a least square method, a gaussian fitting method, or the like, so as to determine the image coordinates of the target point.

S404, determining the relative position relation between the image coordinates of each target point in the calibration board image.

Since the parameter calibration depends on the conversion between the image coordinates of the target points and the world coordinates, after the image coordinates of each target point are extracted, the relative position relationship between the target points needs to be determined according to the size of the horizontal axis and the vertical axis of the coordinate values, so as to determine the corresponding relationship between the image coordinates and the world coordinates.

S405, determining the corresponding relation between the image coordinate and the world coordinate of each target point according to the relative position relation and the world coordinate of the predetermined target point.

According to the image coordinates of each target point, the relative position relationship between each target point can be determined, and then the one-to-one corresponding relationship between the image coordinates and the world coordinates of each target point in the predetermined calibration plate image can be further determined.

S406, determining internal and external parameters of the TOF module according to the corresponding relation between the image coordinate and the world coordinate of each target point and the mapping relation between the image coordinate system and the world coordinate system.

When parameter calibration is carried out, a homography matrix corresponding to each calibration plate is determined according to the image coordinate and the world coordinate of a target point in each calibration plate, internal parameters of the TOF module are determined according to the homography matrices of the calibration plates, and further, external parameters of the TOF module are determined according to the homography matrices and the internal parameters.

And S407, determining the world coordinate of each pixel point according to the internal and external parameters of the TOF module and the image coordinate of each pixel point.

During the parameter calibration, a homography matrix of each calibration plate is determined, and the homography matrix comprises internal and external parameters of the TOF module and is used for describing the mapping relation between the world coordinate system and the image coordinate system of the calibration plate. The homography matrix is used for converting the image coordinates of each pixel point, and the world coordinates of each pixel point can be obtained, namely the world coordinate of the point of the calibration plate corresponding to each pixel point in a world coordinate systemAnd (4) coordinates. With reference to FIG. 1, i.e. according to I (u, v) at O _i -coordinate determination of the uv coordinate system of point P corresponding to I (u, v) at X _w Y _w ,Z _w Coordinates of a coordinate system.

And S408, determining the coordinate of each pixel point in the TOF module coordinate system according to the world coordinate of each pixel point and the external parameters of the TOF module.

The external parameters of the TOF module are used for describing the mapping relation of the calibration plate between the world coordinate system and the TOF module coordinate system, the world coordinate of each pixel point is converted by using the external parameters of the TOF module, the coordinate of each pixel point in the TOF module coordinate system can be obtained, namely the coordinate of the calibration plate point corresponding to each pixel point in the TOF module coordinate system, and reference is made to figure 1, namely the coordinate of the calibration plate point corresponding to each pixel point in the X point in the TOF module coordinate system _w Y _w ,Z _w The coordinate of the coordinate system determines that the point P is at the point O _c -coordinates of an XYZ coordinate system.

And S409, determining the real optical path of each pixel point according to the coordinate of each pixel point in the TOF module coordinate system.

Referring to fig. 1, the real optical path of each pixel point, i.e. the distance between the point of the calibration plate corresponding to each pixel point and the origin of the TOF module coordinate system, taking P point as an example, i.e. P point and O point _c According to P point is at O _c Calculating the coordinates of XYZ coordinate system to obtain P point and O point _c The distance between the points, that is, the true optical path of the point P.

And S410, determining the optical path error of the TOF module according to the real optical path of each pixel point and the measured optical path detected by the TOF module.

In one embodiment, the optical path error of the TOF module includes the optical path error of each pixel; specifically, the optical path error of each pixel point is determined according to the real optical path of each pixel point and the measured optical path detected by the TOF module. In the application of the TOF module, for each pixel point, the optical path of the pixel point is corrected and compensated according to the optical path error corresponding to the pixel point, so that the error compensation precision is improved, and the depth precision is improved.

It should be noted that, when the pattern of the calibration board is black and white, since the reflectivity of the black area is low, the optical path error of the pixels in the white area around the pixels in the black area can be used as the optical path error.

In another embodiment, the optical path error of each pixel point is determined according to the real optical path of each pixel point and the measured optical path detected by the TOF module; and determining the average value of the optical path errors of each pixel point as the optical path error of the TOF module, wherein the optical path error of the TOF module is used for correcting the measured optical path of each pixel point. In the application of the TOF module later, the optical path of each pixel point is corrected and compensated according to the optical path error of the TOF module, so that the compensation is carried out after the optical path error of each pixel point is determined without looking up tables one by one, and the processing efficiency can be improved.

In yet another embodiment, the optical path length error of the TOF module comprises optical path length errors of a plurality of pixel regions; specifically, dividing a calibration plate image into a plurality of pixel regions, and determining an optical path error of each pixel point in each pixel region according to a real optical path of the pixel point in each pixel region and a measured optical path detected by a TOF module; and determining the average value of the optical path errors of the pixels in each pixel area as the optical path error of each pixel area, wherein the optical path error of each pixel area is used for correcting the measured optical path of each pixel in each pixel area. In the application of the TOF module, for each pixel point, the optical path of the pixel point is corrected and compensated according to the optical path error of the pixel point region to which the pixel point belongs, so that the compensation precision and the processing efficiency can be considered.

In the embodiment of the application, firstly, the parameter calibration is carried out by adopting the calibration plate with the annular pattern, the annular pattern is provided with the inner circle edge and the outer circle edge, the central point of the annular pattern is jointly determined by utilizing the inner circle edge and the outer circle edge, the obtained image coordinate of the central point is more accurate, and when the edge detection is carried out, the inner edge and the outer edge at the sub-pixel level can be obtained by carrying out optimization processing on the inner edge and the outer edge at the pixel level, so that the edge precision is further improved. Therefore, the calibration result of the internal and external parameters can be more accurate, and on the basis, the world coordinates of the pixel points calculated according to the internal and external parameters can be more accurate, namely the real optical distance is more accurate, so that the accuracy of error calibration can be improved, and the depth precision is improved.

It should be noted that, in the above manner, when the distance between the calibration plate and the TOF module is fixed, the error of the pixel point under a certain optical path can be obtained, and the calibration is repeatedly performed on different distances, so that the error of the pixel point under different optical paths can be obtained. In practical application, the TOF module can remove the optical path error after obtaining the measured optical path, and then further calculate the depth, thereby improving the depth accuracy.

Fig. 7 is a schematic structural diagram of a calibration apparatus of a TOF module according to an embodiment of the present disclosure. As shown in fig. 7, the calibration apparatus 700 of the TOF module includes:

an obtaining module 701, configured to obtain a calibration plate image, where the calibration plate image includes a plurality of target points;

a first determining module 702, configured to determine internal and external parameters of the TOF module according to the image coordinate of each target point and the corresponding world coordinate;

a second determining module 703, configured to determine a true optical path of each pixel in the calibration board image according to the internal and external parameters;

and the calibration module 704 is used for determining the optical path error of the TOF module according to the real optical path of each pixel point and the measured optical path detected by the TOF module.

In one embodiment, the second determining module 703 is configured to:

In one embodiment, the optical path error of the TOF module includes the optical path error of each pixel;

the calibration module 704 is configured to:

and determining the optical path error of each pixel point according to the real optical path of each pixel point and the measured optical path detected by the TOF module.

In one embodiment, the calibration module 704 is configured to:

determining the optical path error of each pixel point according to the real optical path of each pixel point and the measured optical path detected by the TOF module;

the calibration module 704 is configured to:

dividing the calibration plate image into a plurality of pixel regions, and determining the optical path error of each pixel point in each pixel region according to the real optical path of the pixel point in each pixel region and the measured optical path detected by the TOF module;

and determining the average value of the optical path errors of all the pixels in each pixel area as the optical path error of each pixel area, wherein the optical path error of each pixel area is used for correcting the measured optical path of each pixel in each pixel area.

In one embodiment, each calibration plate comprises a plurality of annular patterns, and the target point is the central point of the annular patterns;

the calibration apparatus 700 of the TOF module further includes:

a third determining module for determining an inner edge and an outer edge of each annular pattern; image coordinates for each target point are determined from the inner and outer edges of each annular pattern.

In one embodiment, the third determining module is configured to:

The device provided in the embodiment of the present application can be used to execute the calibration method of the TOF module in any one of the method embodiments, and the implementation principle and the calculation effect are similar, which are not described herein again.

Fig. 8 is a schematic structural diagram of an electronic device according to an embodiment of the present application. As shown in fig. 8, the electronic device 800 includes a memory 801 and a processor 802, and the memory 801 and the processor 802 may be connected by a bus 803.

The memory 801 is used to store computer programs.

The processor 802 is configured to implement the calibration method of the TOF module in the above-described method embodiment when the computer program is executed.

Optionally, the electronic device may be a computer device, a server, or the like for processing an image captured by the TOF module and performing parameter calibration and error calibration. Or, the electronic device may also be an electronic device with a TOF module, and the electronic device may calibrate the TOF module itself.

The embodiment of the application further provides a computer-readable storage medium, on which a computer program is stored, and when the computer program is executed by a processor, the calibration method of the TOF module in the above method embodiment is implemented.

The embodiment of the present application further provides a computer program product, which includes a computer program, and when the computer program is executed by a processor, the calibration method for the TOF module in the above method embodiment is implemented.

Those of ordinary skill in the art will understand that: all or a portion of the steps of implementing the above-described method embodiments may be performed by hardware associated with program instructions. The foregoing program may be stored in a computer-readable storage medium. When executed, the program performs steps comprising the method embodiments described above; and the aforementioned storage medium includes: various media that can store program codes, such as ROM, RAM, magnetic or optical disks.

Finally, it should be noted that: the above embodiments are only used to illustrate the technical solutions of the present application, and not to limit the same; although the present application has been described in detail with reference to the foregoing embodiments, it should be understood by those of ordinary skill in the art that: the technical solutions described in the foregoing embodiments may still be modified, or some or all of the technical features may be equivalently replaced; and the modifications or the substitutions do not make the essence of the corresponding technical solutions depart from the scope of the technical solutions of the embodiments of the present application.

Claims

1. A calibration method of a TOF module is characterized by comprising the following steps:

2. The method of claim 1, wherein said determining the true optical path length of each pixel point in the calibration plate image according to the internal and external parameters of the TOF module comprises:

3. The method according to claim 1 or 2, wherein the optical path error of the TOF module comprises the optical path error of each pixel point;

4. The method according to claim 1 or 2, wherein the determining the optical path error of the TOF module according to the true optical path of each pixel point and the measured optical path detected by the TOF module comprises:

5. The method according to claim 1 or 2, wherein the optical path length error of the TOF module comprises optical path length errors of a plurality of pixel regions;

dividing the calibration plate image into a plurality of pixel areas, and obtaining the optical path error of each pixel point in each pixel area according to the real optical path of the pixel point in each pixel area and the measured optical path detected by the TOF module;

6. The method of claim 1 or 2, wherein each calibration plate includes a plurality of circular patterns thereon, the target point being a center point of the circular patterns;

determining an inner edge and an outer edge of each of the annular patterns;

7. The method of claim 6, wherein said determining the inner and outer edges of each of said annular patterns comprises:

8. The method of claim 6, wherein said obtaining the inner and outer edges of each annular pattern in the calibration plate image comprises:

9. The method of claim 6, wherein said determining image coordinates of said each target point from the inner and outer edges of said each annular pattern comprises:

10. The utility model provides a calibration device of TOF module which characterized in that includes:

the second determining module is used for determining the real optical path of each pixel point in the calibration plate image according to the internal and external parameters;

11. An electronic device comprising a memory and a processor, the memory and the processor being connected;

the memory is used for storing a computer program;

the processor is adapted to implement the method of any of claims 1-9 when the computer program is executed.