CN113506351A - Calibration method and device for ToF camera, electronic equipment and storage medium - Google Patents
Calibration method and device for ToF camera, electronic equipment and storage medium Download PDFInfo
- Publication number
- CN113506351A CN113506351A CN202110859955.2A CN202110859955A CN113506351A CN 113506351 A CN113506351 A CN 113506351A CN 202110859955 A CN202110859955 A CN 202110859955A CN 113506351 A CN113506351 A CN 113506351A
- Authority
- CN
- China
- Prior art keywords
- tof camera
- calibration
- depth value
- raw data
- distances
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000000034 method Methods 0.000 title claims abstract description 98
- 238000005259 measurement Methods 0.000 claims description 60
- 238000012545 processing Methods 0.000 claims description 19
- 230000003287 optical effect Effects 0.000 claims description 9
- 230000009466 transformation Effects 0.000 claims description 5
- 238000006243 chemical reaction Methods 0.000 claims description 4
- 230000008569 process Effects 0.000 description 14
- 238000010586 diagram Methods 0.000 description 7
- 230000006870 function Effects 0.000 description 6
- 238000004891 communication Methods 0.000 description 5
- 238000003384 imaging method Methods 0.000 description 5
- 238000012360 testing method Methods 0.000 description 5
- 230000000694 effects Effects 0.000 description 4
- 238000004441 surface measurement Methods 0.000 description 4
- 230000007613 environmental effect Effects 0.000 description 3
- 230000004044 response Effects 0.000 description 3
- 238000004458 analytical method Methods 0.000 description 1
- 230000008859 change Effects 0.000 description 1
- 230000000295 complement effect Effects 0.000 description 1
- 238000012937 correction Methods 0.000 description 1
- 238000001514 detection method Methods 0.000 description 1
- 238000007599 discharging Methods 0.000 description 1
- 238000005516 engineering process Methods 0.000 description 1
- 238000012886 linear function Methods 0.000 description 1
- 239000004973 liquid crystal related substance Substances 0.000 description 1
- 229910044991 metal oxide Inorganic materials 0.000 description 1
- 150000004706 metal oxides Chemical class 0.000 description 1
- 239000004065 semiconductor Substances 0.000 description 1
- 230000009897 systematic effect Effects 0.000 description 1
- 230000001960 triggered effect Effects 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/80—Analysis of captured images to determine intrinsic or extrinsic camera parameters, i.e. camera calibration
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/50—Depth or shape recovery
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/10—Image acquisition modality
- G06T2207/10028—Range image; Depth image; 3D point clouds
Landscapes
- Engineering & Computer Science (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Theoretical Computer Science (AREA)
- Length Measuring Devices By Optical Means (AREA)
Abstract
The application discloses a calibration method and device of a ToF camera, electronic equipment and a storage medium, and belongs to the technical field of computer vision. The calibration method comprises the following steps: calculating the depth value of real-time Raw data acquired by a ToF camera; compensating the depth value by utilizing a pre-established distance error model to obtain a calibration result; the distance error model is obtained by performing model parameter fitting in advance based on measured depth values of Raw data samples acquired by the ToF camera at different measured distances and real depth values of the Raw data samples at the different measured distances, and the real depth values are obtained by performing curved surface acquisition on the reference surfaces at the different measured distances based on internal parameters of the ToF camera.
Description
Technical Field
The application belongs to the technical field of computer vision, and particularly relates to a calibration method and device for a ToF camera, an electronic device and a storage medium.
Background
A Time-of-Flight (ToF) camera is one of the most preferred choices for obtaining a distance depth map of a three-dimensional scene because of its compact structure, fast response speed, simple algorithm, capability of providing a three-dimensional image at a high frame rate, capability of providing intensity data and distance information for each pixel, and the like.
However, due to the existence of systematic errors and random errors, the measurement result and the measurement precision of the depth information of Raw image (Raw) data collected by the ToF camera are affected by many factors such as the internal environment and the external environment of the camera system, and finally the accuracy of the acquired depth information is not high.
Disclosure of Invention
An object of the embodiments of the present application is to provide a calibration method and apparatus for a ToF camera, an electronic device, and a storage medium, which can solve the problem that the existing ToF camera is not high in accuracy in measuring depth information.
In a first aspect, an embodiment of the present application provides a calibration method for a ToF camera, where the method includes:
calculating the depth value of real-time Raw data acquired by a ToF camera;
compensating the depth value by utilizing a pre-established distance error model to obtain a calibration result;
the distance error model is obtained by performing model parameter fitting in advance based on measured depth values of Raw data samples acquired by the ToF camera at different measured distances and real depth values of the Raw data samples at the different measured distances, wherein the real depth values are obtained by performing curved surface acquisition on reference surfaces at the different measured distances based on internal parameters of the ToF camera.
In a second aspect, an embodiment of the present application provides a calibration apparatus for a ToF camera, the apparatus including:
the first processing module is used for calculating the depth value of real-time Raw data acquired by the ToF camera;
compensating the depth value by utilizing a pre-established distance error model to obtain a calibration result;
the distance error model is obtained by performing model parameter fitting in advance based on measured depth values of Raw data samples acquired by the ToF camera at different measured distances and real depth values of the Raw data measured distances at the different measured distances, and the real depth values are obtained by performing curved surface acquisition on reference surfaces at the different measured distances based on internal parameters of the ToF camera.
In a third aspect, an embodiment of the present application provides an electronic device, which includes a processor, a memory, and a program or instructions stored on the memory and executable on the processor, and when executed by the processor, the program or instructions implement the steps of the method according to the first aspect.
In a fourth aspect, embodiments of the present application provide a readable storage medium, on which a program or instructions are stored, which when executed by a processor implement the steps of the method according to the first aspect.
In a fifth aspect, an embodiment of the present application provides a chip, where the chip includes a processor and a communication interface, where the communication interface is coupled to the processor, and the processor is configured to execute a program or instructions to implement the method according to the first aspect.
In the embodiment of the application, the distance error model is established in advance according to the measured depth value measured by the ToF camera at the test position and the real depth value of the position acquired by adopting different methods, so that when the ToF camera is actually used, the distance error model is utilized to carry out depth compensation calibration on the Raw data acquired by the ToF camera to eliminate errors, and the measurement accuracy is higher.
Drawings
Fig. 1 is a flowchart of a calibration method of a ToF camera according to an embodiment of the present application;
fig. 2 is a second flowchart of a calibration method of a ToF camera according to an embodiment of the present application;
fig. 3 is a schematic flowchart illustrating a process of constructing a distance error model in a calibration method of a ToF camera according to an embodiment of the present application;
fig. 4 is a schematic flow chart illustrating internal reference calibration of a ToF camera in the calibration method of the ToF camera according to the embodiment of the present application;
fig. 5 is a schematic diagram of an incompletely photographed calibration plate image in the calibration method of the ToF camera according to the embodiment of the application;
fig. 6 is a schematic structural diagram of a calibration apparatus of a ToF camera according to an embodiment of the present application;
fig. 7 is a schematic structural diagram of an electronic device according to an embodiment of the present application;
fig. 8 is a schematic diagram of a hardware structure of an electronic device implementing an embodiment of the present application.
Detailed Description
The technical solutions in the embodiments of the present application will be described clearly below with reference to the drawings in the embodiments of the present application, and it is obvious that the described embodiments are some, but not all, embodiments of the present application. All other embodiments that can be derived by one of ordinary skill in the art from the embodiments given herein are intended to be within the scope of the present disclosure.
The terms first, second and the like in the description and in the claims of the present application are used for distinguishing between similar elements and not necessarily for describing a particular sequential or chronological order. It will be appreciated that the data so used may be interchanged under appropriate circumstances such that embodiments of the application may be practiced in sequences other than those illustrated or described herein, and that the terms "first," "second," and the like are generally used herein in a generic sense and do not limit the number of terms, e.g., the first term can be one or more than one. In addition, "and/or" in the specification and claims means at least one of connected objects, a character "/" generally means that a preceding and succeeding related objects are in an "or" relationship.
Aiming at the problem of low measurement accuracy in the prior art, the distance error model is established in advance according to the measured depth value measured by the ToF camera at the test position and the real depth value of the position acquired by adopting different methods, so that when the ToF camera is actually used, the distance error model is utilized to carry out depth compensation calibration on the Raw data acquired by the ToF camera to eliminate errors, and the measurement accuracy is higher. The ToF camera calibration method, the ToF camera calibration device, the ToF camera calibration electronic device, and the ToF camera calibration storage medium provided by the embodiments of the present application are described in detail below with reference to the accompanying drawings.
Fig. 1 is a flowchart of a calibration method for a ToF camera according to an embodiment of the present application, where the method may be executed by a terminal having a ToF function, as shown in fig. 1, and the method includes:
It can be understood that, when the ToF camera is actually used to collect data, the ToF camera may be used to collect real-time Raw data of the target three-dimensional scene, and then a depth value corresponding to the real-time Raw data may be calculated. For example, a four-step phase method may be used to calculate depth values for the real-time Raw data.
And 102, compensating the depth value by using a pre-established distance error model to obtain a calibration result.
The distance error model is obtained by performing model parameter fitting in advance based on measured depth values of Raw data samples acquired by the ToF camera at different measured distances and real depth values of the Raw data samples at the different measured distances, and the real depth values are obtained by performing curved surface acquisition on reference surfaces at the different measured distances based on internal parameters of the ToF camera.
It can be understood that, in the embodiment of the present application, the measurement error of the ToF camera may be analyzed in advance through a test, and a distance error model of the ToF camera is obtained through fitting, so that the distance depth map of the target scene can be obtained quickly and accurately when the ToF camera is subsequently used for data acquisition.
Specifically, when the distance error model of the ToF camera is fitted, Raw data of a preset plane may be acquired at a plurality of measurement distances of the preset plane by the ToF camera respectively as Raw data samples, and measurement depth values corresponding to the Raw data samples are calculated. And simultaneously, respectively determining reference surfaces at each measuring distance, and performing curved surface on the reference surfaces by using internal reference values of the ToF camera to obtain real depth values of Raw data samples at each measuring distance. And then, fitting to obtain parameters of a distance error model by calculating the error between the measured depth value and the real depth value at each measured distance, thereby determining the distance error model of the ToF camera.
On the basis of pre-establishing the distance error model of the ToF camera, the distance error model of the ToF camera can be utilized to quickly and accurately compensate and correct the depth value calculated in the step 101, and the depth value obtained after correction becomes the calibration result of the measurement result of the ToF camera.
That is to say, the embodiment of the application may adopt different methods to obtain the true depth value of the acquired data in advance, and establish the distance error model by measuring the depth value and the true depth value. In practical application, as shown in fig. 2, a second flowchart of the calibration method for the ToF camera provided in the embodiment of the present application is shown, where the calibration of the ToF camera in the embodiment of the present application refers to that, for real-time Raw data acquired by the ToF camera, a distance error model is used to correct a distance error and then obtain a correct depth value. Specifically, the depth value of the real-time Raw data acquired by the ToF camera is calculated by a four-step phase method, and the distance error of the embodiment of the application is added to compensate the distance measurement value of each pixel point of the image, so that the calibrated depth value can be obtained.
The ToF camera provides a distance depth map of a three-dimensional scene by using a low-cost Complementary Metal Oxide Semiconductor (CMOS) pixel array and an active modulation light source technology, and each pixel in the obtained image can measure the brightness of a corresponding target and the arrival time of reflected modulated light, so that the distance depth corresponding to the point can be calculated.
RAW data is RAW data in which a CMOS or Charge Coupled Device (CCD) image sensor converts a captured light source signal into a digital signal. The RAW file is a file in which RAW information of a digital camera sensor is recorded, and at the same time, some Metadata (Metadata such as ISO setting, shutter speed, aperture value, white balance, etc.) generated by camera shooting is recorded. RAW is in an unprocessed, also uncompressed, format.
The different measurement distances refer to positions of the ToF camera at a plurality of different distances from the preset plane, and these positions are referred to as different measurement distances since the measurement values from the preset plane are directly measured and acquired by the ToF camera at these positions. Correspondingly, the depth value directly measured by the ToF camera is referred to as a measured depth value, and the compensated and corrected depth values of the Raw data samples at different measured distances are referred to as real depth values.
According to the calibration method of the ToF camera, the distance error model is established in advance according to the measured depth value of the ToF camera at the test position and the real depth value of the position acquired by adopting different methods, when the ToF camera is actually used, the distance error model is used for carrying out depth compensation calibration on real-time Raw data acquired by the ToF camera to eliminate errors, and the measurement accuracy is higher.
Optionally, the calibration method of the ToF camera further includes a processing step of constructing a distance error model, as shown in fig. 3, which is a schematic flow chart of constructing the distance error model in the calibration method of the ToF camera provided by the embodiment of the present application, and the method includes:
It can be understood that, when performing measurement error analysis of the ToF camera, in the embodiment of the application, first, Raw data of the white plane may be collected at different distances from the white plane as Raw data samples, and then depth values at different distances are respectively calculated as measured depth values according to an algorithm such as a four-step phase method.
For example, the ToF camera may be fixed to a tripod that is fixed to a slidable rail to ensure that the relative position of the camera with respect to the vertical white plane being photographed does not change. Meanwhile, the distance measuring instrument is fixed on the sliding guide rail, and the distance between the distance measuring instrument and the distance measuring ToF camera to the shot white plane is kept consistent, so that the measuring position is positioned. ToF cameras use a camera resolution of 180 x 240, fixed frequencies of 20MHZ and 100MHZ, or other camera resolutions and frequencies.
On the basis of parameter setting, Raw maps at different distances at a fixed frequency can be acquired by using a ToF camera. The measuring distance can be 600mm to 1000mm, the interval is 100mm, the specific distance can be measured by a distance meter, the same distance can be ensured to be kept consistent as much as possible in multiple measurements, different measuring distances are fixed through a sliding guide rail, 5 frames are collected in total, and the depth value is calculated in each frame through a four-step phase method and used as the measuring depth value of different distances.
it is understood that, to obtain the true depth values at different measurement distances from the white plane, different methods may be used to determine the reference surfaces at the different measurement distances according to the pose of the ToF camera. Then, because the ToF camera adopts the principle of pinhole imaging, the measured depth value obtained in step 301 is a curved surface, and according to the principle of triangle similarity, on the basis of the known depth values of the internal reference value and the reference surface of the ToF camera, the depth value of the reference surface can be curved, and the curved result is used as the real depth values at different measuring distances.
Optionally, the determining the reference surfaces at the different measurement distances respectively includes: taking the planes at the different measurement distances as the reference plane if the center of the optical axis of the ToF camera is perpendicular to the target white plane; or controlling the ToF camera to collect calibration Raw data of the target white plane at different calibration distances respectively, and calculating the depth values of the calibration Raw data at the different calibration distances respectively by adopting a four-step phase method; superposing the depth difference values from the different calibration distances to the different measurement distances on the depth value of the calibration Raw data, and taking the superposed depth value as the depth value of the reference surface; and fitting the depth value of the reference surface into a plane by adopting a least square method, and taking the fitted plane as the reference surface.
It is understood that the embodiment of the present application may acquire Raw data of the white plane at a plurality of calibration distances different from the measurement distance of step 301 by using the ToF camera, and respectively calculate depth values of the Raw data at the calibration distances according to a four-step phase method. To obtain the true depth value of the measured depth values in step 301, the depth values of the Raw data at the calibrated distances are added/subtracted with the depth value of the difference between the calibrated distances and the measured distances to obtain the reference depth value, and a plane is fitted by a least square method based on the reference depth value to obtain the reference plane. Then, according to the internal reference value of the ToF camera, the depth value of the reference surface obtained by the whole fitting is curved, and the real depth values at different measurement distances can be obtained.
Or before data acquisition, the pose of the ToF camera can be adjusted to make the optical axis center of the ToF camera perpendicular to the target white plane, and then the plane where each measurement distance where the ToF camera is located is directly used as a reference plane.
For example, the ToF camera and rangefinder may be fixed in the same manner as step 301 and Raw data at different calibration distances may be acquired in the same manner. It should be understood that in the embodiments of the present application, the positions at different calibration distances are different from the positions at different measurement distances, and have determined distances from the corresponding measurement distances, that is, determined depth differences exist. The specific distance from the target white plane at different calibration distances can be 650mm to 1050mm, and the distance is 100mm, and 5 frames are acquired in total. The depth value is calculated by a four-step phase method every frame.
To obtain the true depth values at 600mm to 1000mm in step 301, the depth value calculated for each frame is subtracted by 50mm to be used as the depth value of the reference plane, and the depth value d of the reference plane is fitted to a plane ax + by + c-d by a least square method to be used as the reference plane at different measuring distances. On this basis, because the ToF camera adopts the pinhole imaging principle, the measurement value obtained in step 301 is a curved surface, and according to the triangle similarity principle, if the depth value d of the camera reference plane and the plane is known, the plane can be curved, and the result is used as the true depth value of the data acquired by the ToF camera.
In addition, it can also be assumed that the center of the optical axis of the camera is perpendicular to the shooting plane, and the plane where the distance 600mm to 1000mm is measured in step 301 can be directly taken as an absolute plane. On the basis, according to the camera pinhole imaging and the triangle similarity principle, when the camera is known to be involved, the absolute plane is curved, and the curved depth value is used as the real depth value.
The formula for performing the faceting can be expressed as:
wherein d isquAnd dpingRespectively representing the depth value after the surface is curved and the depth value of the reference surface before the surface is curved, f is the focal length of the camera, i and j are certain pixel coordinates, x and y are x-axis and y-axis coordinate variables respectively, and c is the intercept parameter of the reference surface plane.
The curved surface depth value is obtained by performing surface curving on the depth value of the reference surface plane, the curved surface measurement method and the curved surface measurement device can effectively fit the measurement curved surface of the ToF camera, and compensate errors caused by small hole imaging, so that the measurement accuracy is further improved.
It can be understood that, on the basis of measuring the measured depth values and the true depth values at different measured distances, the embodiment of the application may construct an initial distance error model based on the measured depth values and the true depth values. Specifically, when the measured depth value and the true depth value are known, the distance error Δ d is the measured depth value minus the true depth value. Assuming d is the measured depth value, the initial distance error model may be constructed including: Δ d ═ p (1) × d2+ p (2) × d + p (3), where p (1), p (2), p (3) are the three parameters to be fitted for the range error model. Further alternatively, the distance error model may also be expressed as a linear function.
Optionally, the constructing an initial distance error model includes: and acquiring an influence factor of the temperature on a ToF sensor in the ToF camera, and constructing the initial distance error model based on the measured depth value, the real depth value and the influence factor.
That is, the influence of the ambient temperature on the ToF sensor in the ToF camera may be converted into an influence factor of the mathematical quantity forming temperature, the influence factor is also added into the model, and the primary or secondary initial distance error model constructed above is modified to obtain the final initial distance error model. By adding the environmental temperature influence factor, the influence of the environmental temperature on the measurement result can be effectively compensated, so that more reliable measurement results can be obtained at different environmental temperatures.
On the basis of constructing the initial distance error model, the secondary initial distance error model or the primary initial distance error model can be fitted through a least square method, model parameters are calculated, and the distance error model with known model parameters is finally constructed.
According to the embodiment of the application, the measured depth values of the Raw data acquired by the ToF camera at different measuring distances are acquired, the corrected real depth values at the positions are acquired, the measuring error of the ToF camera is fitted, the distance error model measured by the ToF camera is finally obtained, the measuring error of the ToF camera can be quickly and accurately compensated in subsequent application, and therefore the measuring efficiency and the measuring accuracy are effectively improved.
Optionally, the calibration method of the ToF camera further includes: and controlling the ToF camera to collect multi-frame calibration plate images under different camera poses, and carrying out internal reference calibration on the ToF camera by utilizing a camera calibration tool kit based on the calibration plate images to obtain the internal reference value of the ToF camera.
The multi-frame calibration plate images are generated by storing confidence maps of calibration plate Raw data conversion or pictures directly acquired through the ToF camera, the multi-frame calibration plate images are the same in size, and the selection sequence of the multi-frame calibration plate images is consistent.
It can be understood that, in the embodiment of the present application, before the distance error model of the ToF camera is constructed, the internal reference calibration may be performed on the ToF camera to obtain the internal reference value of the ToF camera. Specifically, calibration plate images can be acquired at different poses of the ToF camera, and calibration can be performed based on these calibration plate images by using the existing calibration toolkit. For example, the calibration board images may be placed in a calibration kit directory, and a camera calibration process in the calibration kit may be triggered, so as to obtain camera internal parameters. When the calibration plate image is collected, the calibration plate image can be incomplete, free selection can be carried out according to the size of the shot calibration plate, the size selection of the calibration plate of all the images needs to be consistent in the selection process, and the selection sequence is consistent (if the size selection is clockwise or anticlockwise).
For example, as shown in fig. 4, an internal reference calibration process for a ToF camera is a schematic flow chart of performing internal reference calibration on the ToF camera in the calibration method for the ToF camera provided by the embodiment of the present application, and mainly includes the following processing flows:
firstly, a calibration plate is pasted on a vertical white plane, a ToF camera is fixed on a tripod, and multi-frame calibration plate images in different ToF camera poses are obtained by rotating the tripod. An incomplete calibration plate image, such as the shape shown in fig. 5, may be acquired, fig. 5 is a schematic diagram of the incomplete calibration plate image captured in the calibration method of the ToF camera provided in the embodiment of the present application, and 15-20 frames of calibration plate images with similar shapes may be acquired. In the process of acquiring the calibration board image, the distance from the ToF camera to the calibration board may be within the range of the acquisition frequency, for example, a distance from 500mm to 800mm at a frequency of 20MHZ may be adopted, and a distance may also be within a corresponding range at other frequencies, such as 80MHZ, 100MHZ, and 120 MHZ.
Then, the TOOLBOX _ calib kit of Matlab was selected for calibration. Converting the collected Raw data of the calibration plate into a confidence map, and storing the confidence map as pictures in a fixed format, such as jpg and bmp formats; alternatively, the fixed format picture can be used directly if it is taken directly by the camera. And placing the obtained picture in the calibration kit catalog, and setting the size of the calibration board and the length of the grid of the calibration board through a Matlab command terminal to calibrate so as to obtain the camera internal parameters cx, cy, fx and fy.
In one embodiment, the dimensions of the calibration plate are selected to be 7 x 6 rectangular areas as shown in fig. 5, the boundary points are selected in a counter-clockwise direction, and the length of the calibration plate squares is 25 mm. The embodiment of the application can also select calibration plates with other sizes, as long as all the images of the calibration plates can be ensured to be capable of selecting rectangular areas with fixed sizes, and all the rectangular areas of the images of the calibration plates are ensured to be consistent in selection direction, and the length of the grids of the calibration plates can be 20mm to 50mm, for example.
The confidence map is calculated according to the following formula:
temp1＝x0-x2temp2＝x3-x1；
confidence＝abs(temp1)+abs(temp)。
wherein confidence represents confidence, x0、x1、x2、x3Optical signals respectively representing phases of 0 °, 90 °, 180 °, and 270 ° can be directly read from Raw data, and abs represents an absolute value.
According to the embodiment of the application, the calibration plate images of the ToF camera under different poses are collected, and the internal reference calibration operation is performed on the ToF camera by using the existing calibration tool kit, so that the operation is simple, and the response speed is high.
It should be noted that, in the calibration method of the ToF camera provided in the embodiment of the present application, the execution subject may be a calibration device of the ToF camera, or a control module in the calibration device of the ToF camera for executing the calibration method of the ToF camera. In the embodiment of the present application, a calibration method for executing a calibration method of a ToF camera by using a calibration device of the ToF camera is taken as an example, and the calibration device of the ToF camera provided in the embodiment of the present application is described.
Fig. 6 shows a structure of the calibration device of the ToF camera according to the embodiment of the present application, which is a schematic structural diagram of the calibration device of the ToF camera according to the embodiment of the present application, and the calibration device may be used to implement the calibration process of the ToF camera in the calibration method embodiments of the ToF camera, where the calibration device includes:
the first processing module 601 is configured to calculate a depth value of real-time Raw data acquired by the ToF camera;
and the second processing module is used for compensating the depth value by utilizing a pre-established distance error model to obtain a calibration result.
The distance error model is obtained by performing model parameter fitting in advance based on measured depth values of Raw data samples acquired by the ToF camera at different measured distances and real depth values of the Raw data samples at the different measured distances, wherein the real depth values are obtained by performing curved surface acquisition on reference surfaces at the different measured distances based on internal parameters of the ToF camera.
Optionally, the calibration apparatus of the ToF camera further includes:
a third processing module, configured to control the ToF camera, collect Raw data of a target white plane at the different measurement distances, respectively, to form Raw data samples, and calculate the measurement depth values of the Raw data samples at the different measurement distances, respectively;
a fourth processing module, configured to determine the reference surfaces at the different measurement distances, respectively, and perform surface transformation on the depth value of the reference surface by using the internal reference value of the ToF camera, so as to obtain a surface depth value as the true depth value of the Raw data sample at the different measurement distances;
and the fifth processing module is used for constructing an initial distance error model based on the measured depth value and the real depth value, fitting parameters of the initial distance error model through a least square method and obtaining the distance error model.
Optionally, the fourth processing module, when configured to determine the reference surfaces at the different measurement distances respectively, is configured to:
controlling the ToF camera, respectively collecting calibration Raw data of the target white plane at different calibration distances, and respectively calculating depth values of the calibration Raw data at the different calibration distances by adopting a four-step phase method; superposing the depth difference values from the different calibration distances to the different measurement distances on the depth value of the calibration Raw data, and taking the superposed depth value as the depth value of the reference surface; fitting the depth value of the reference surface into a plane by adopting a least square method, and taking the fitted plane as the reference surface;
or,
taking the plane at the different measurement distances as the reference plane in a case where the center of the optical axis of the ToF camera is perpendicular to the target white plane.
Optionally, the fifth processing module, when configured to construct the initial distance error model, is configured to:
and acquiring an influence factor of the temperature on a ToF sensor in the ToF camera, and constructing the initial distance error model based on the measured depth value, the real depth value and the influence factor.
Optionally, the calibration apparatus of the ToF camera further includes:
the sixth processing module is used for controlling the ToF camera to acquire multi-frame calibration plate images under different camera poses, and performing internal reference calibration on the ToF camera by using a camera calibration kit based on the calibration plate images to acquire the internal reference values of the ToF camera;
the multi-frame calibration plate images are generated by storing confidence maps of calibration plate Raw data conversion or pictures directly acquired through the ToF camera, the multi-frame calibration plate images are the same in size, and the selection sequence of the multi-frame calibration plate images is consistent.
The calibration device of the ToF camera in the embodiment of the present application may be a device, or may be a component, an integrated circuit, or a chip in a terminal. The device can be mobile electronic equipment or non-mobile electronic equipment. By way of example, the mobile electronic device may be a mobile phone, a tablet computer, a notebook computer, a palm top computer, a vehicle-mounted electronic device, a wearable device, an ultra-mobile personal computer (UMPC), a netbook or a Personal Digital Assistant (PDA), and the like, and the non-mobile electronic device may be a server, a Network Attached Storage (NAS), a Personal Computer (PC), a Television (TV), a teller machine or a self-service machine, and the like, and the embodiments of the present application are not particularly limited.
The calibration device of the ToF camera in the embodiment of the present application may be a device having an operating system. The operating system may be an Android (Android) operating system, an IOS operating system, or other possible operating systems, which is not specifically limited in the embodiments of the present application.
The calibration device for the ToF camera provided in the embodiment of the present application can implement each process implemented by the method embodiments of fig. 1 to 5, and can achieve the same technical effect, and is not described herein again to avoid repetition.
Optionally, as shown in fig. 7, an electronic device 700 is further provided in an embodiment of the present application, and includes a processor 701, a memory 702, and a program or an instruction stored on the memory 702 and executable on the processor 701, where the program or the instruction is executed by the processor 701 to implement each process of the above-mentioned calibration method for a ToF camera, and can achieve the same technical effect, and in order to avoid repetition, the description is omitted here.
It should be noted that the electronic device in the embodiment of the present application includes the mobile electronic device and the non-mobile electronic device described above.
Fig. 8 is a schematic diagram of a hardware structure of an electronic device implementing an embodiment of the present application. As shown in fig. 8, the electronic device 800 includes, but is not limited to: a radio frequency unit 801, a network module 802, an audio output unit 803, an input unit 804, a sensor 805, a display unit 806, a user input unit 807, an interface unit 808, a memory 809, and a processor 810.
Those skilled in the art will appreciate that the electronic device 800 may further comprise a power source (e.g., a battery) for supplying power to the various components, and the power source may be logically connected to the processor 810 via a power management system, so as to manage charging, discharging, and power consumption management functions via the power management system. The electronic device structure shown in fig. 8 does not constitute a limitation of the electronic device, and the electronic device may include more or less components than those shown, or combine some components, or arrange different components, and thus, the description is omitted here.
The processor 810 is configured to calculate a depth value of real-time Raw data acquired by the ToF camera, and compensate the depth value by using a pre-established distance error model to obtain a calibration result;
the distance error model is obtained by performing model parameter fitting in advance based on measured depth values of Raw data samples acquired by the ToF camera at different measured distances and real depth values of the Raw data samples at the different measured distances, wherein the real depth values are obtained by performing curved surface acquisition on reference surfaces at the different measured distances based on internal parameters of the ToF camera.
In the embodiment of the application, the distance error model is established in advance according to the measured depth value measured by the ToF camera at the test position and the real depth value of the position acquired by adopting different methods, so that when the ToF camera is actually used, the distance error model is utilized to carry out depth compensation calibration on the Raw data acquired by the ToF camera to eliminate errors, and the measurement accuracy is higher.
Optionally, the input unit 804 is a ToF camera in this embodiment of the application, and is configured to acquire Raw data of a target white plane at different measurement distances, respectively, to form the Raw data sample;
a processor 810 further for calculating the measured depth values of the Raw data samples at the different measured distances, respectively; determining the reference surfaces at different measuring distances respectively, and performing surface transformation on the depth value of the reference surface by using the internal reference value of the ToF camera to obtain a surface depth value as the real depth value of the Raw data sample at the different measuring distances; and constructing an initial distance error model based on the measured depth value and the real depth value, and fitting parameters of the initial distance error model through a least square method to obtain the distance error model.
According to the embodiment of the application, the measured depth values of the Raw data acquired by the ToF camera at different measuring distances are acquired, the corrected real depth values at the positions are acquired, the measuring error of the ToF camera is fitted, the distance error model measured by the ToF camera is finally obtained, the measuring error of the ToF camera can be quickly and accurately compensated in subsequent application, and therefore the measuring efficiency and the measuring accuracy are effectively improved.
Optionally, the input unit 804 is a ToF camera in this embodiment of the application, and is configured to acquire calibration Raw data of the target white plane at different calibration distances respectively;
the processor 810 is further configured to calculate depth values of the calibrated Raw data at different calibration distances by using a four-step phase method; superposing the depth difference values from the different calibration distances to the different measurement distances on the depth value of the calibration Raw data, and taking the superposed depth value as the depth value of the reference surface; fitting the depth value of the reference surface into a plane by adopting a least square method, and taking the fitted plane as the reference surface;
or,
for taking the planes at the different measurement distances as the reference plane if the center of the optical axis of the ToF camera is perpendicular to the target white plane.
The curved surface depth value is obtained by performing surface curving on the depth value of the reference surface plane, the curved surface measurement method and the curved surface measurement device can effectively fit the measurement curved surface of the ToF camera, and compensate errors caused by small hole imaging, so that the measurement accuracy is further improved.
Optionally, the processor 810 is further configured to obtain an influence factor of temperature on a ToF sensor in the ToF camera, and construct the initial distance error model based on the measured depth value, the true depth value and the influence factor.
According to the embodiment of the application, the influence of the ambient temperature on the measurement result can be effectively compensated by adding the ambient temperature influence factor, so that more reliable measurement results can be obtained at different ambient temperatures.
Optionally, the input unit 804 is a ToF camera in this embodiment of the application, and is configured to acquire multiple frames of calibration plate images in different camera poses;
a processor 810, further configured to perform internal reference calibration on the ToF camera by using a camera calibration kit based on the calibration board image, and obtain the internal reference value of the ToF camera;
the multi-frame calibration plate images are generated by storing confidence maps of calibration plate Raw data conversion or pictures directly acquired through the ToF camera, the multi-frame calibration plate images are the same in size, and the selection sequence of the multi-frame calibration plate images is consistent.
According to the embodiment of the application, the calibration plate images of the ToF camera under different poses are collected, and the internal reference calibration operation is performed on the ToF camera by using the existing calibration tool kit, so that the operation is simple, and the response speed is high.
It should be understood that in the embodiment of the present application, the input Unit 804 may include a Graphics Processing Unit (GPU) 8041 and a microphone 8042, and the Graphics Processing Unit 8041 processes image data of a still picture or a video obtained by an image capturing device (such as a camera) in a video capturing mode or an image capturing mode. The display unit 806 may include a display panel 8061, and the display panel 8061 may be configured in the form of a liquid crystal display, an organic light emitting diode, or the like. The user input unit 807 includes a touch panel 8071 and other input devices 8072. A touch panel 8071, also referred to as a touch screen. The touch panel 8071 may include two portions of a touch detection device and a touch controller. Other input devices 8072 may include, but are not limited to, a physical keyboard, function keys (e.g., volume control keys, switch keys, etc.), a trackball, a mouse, and a joystick, which are not described in detail herein. The memory 809 may be used to store software programs as well as various data including, but not limited to, application programs and operating systems. The processor 810 may integrate an application processor, which mainly handles operating systems, user interfaces, application programs, etc., and a modem processor, which mainly handles wireless communications. It will be appreciated that the modem processor described above may not be integrated into processor 810.
The embodiments of the present application further provide a readable storage medium, where a program or an instruction is stored on the readable storage medium, and when the program or the instruction is executed by a processor, the program or the instruction implements each process of the above calibration method for a ToF camera, and can achieve the same technical effect, and in order to avoid repetition, details are not repeated here.
The processor is the processor in the electronic device described in the above embodiment. The readable storage medium includes a computer readable storage medium, such as a Read-Only Memory (ROM), a Random Access Memory (RAM), a magnetic disk or an optical disk, and so on.
The embodiment of the present application further provides a chip, where the chip includes a processor and a communication interface, the communication interface is coupled to the processor, and the processor is configured to run a program or an instruction to implement each process of the above embodiment of the calibration method for a ToF camera, and can achieve the same technical effect, and in order to avoid repetition, the details are not repeated here.
It should be understood that the chips mentioned in the embodiments of the present application may also be referred to as system-on-chip, system-on-chip or system-on-chip, etc.
It should be noted that, in this document, the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising an … …" does not exclude the presence of other like elements in a process, method, article, or apparatus that comprises the element. Further, it should be noted that the scope of the methods and apparatus of the embodiments of the present application is not limited to performing the functions in the order illustrated or discussed, but may include performing the functions in a substantially simultaneous manner or in a reverse order based on the functions involved, e.g., the methods described may be performed in an order different than that described, and various steps may be added, omitted, or combined. In addition, features described with reference to certain examples may be combined in other examples.
Through the above description of the embodiments, those skilled in the art will clearly understand that the method of the above embodiments can be implemented by software plus a necessary general hardware platform, and certainly can also be implemented by hardware, but in many cases, the former is a better implementation manner. Based on such understanding, the technical solutions of the present application may be embodied in the form of a computer software product, which is stored in a storage medium (such as ROM/RAM, magnetic disk, optical disk) and includes instructions for enabling a terminal (such as a mobile phone, a computer, a server, or a network device) to execute the method according to the embodiments of the present application.
While the present embodiments have been described with reference to the accompanying drawings, it is to be understood that the invention is not limited to the precise embodiments described above, which are meant to be illustrative and not restrictive, and that various changes may be made therein by those skilled in the art without departing from the spirit and scope of the invention as defined by the appended claims.
Claims (10)
1. A calibration method of a ToF camera, the method comprising:
calculating the depth value of real-time Raw data acquired by a ToF camera;
compensating the depth value by utilizing a pre-established distance error model to obtain a calibration result;
the distance error model is obtained by performing model parameter fitting in advance based on measured depth values of Raw data samples acquired by the ToF camera at different measured distances and real depth values of the Raw data samples at the different measured distances, wherein the real depth values are obtained by performing curved surface acquisition on reference surfaces at the different measured distances based on internal parameters of the ToF camera.
2. The calibration method of the ToF camera according to claim 1, further comprising:
controlling the ToF camera, respectively collecting Raw data of a target white plane at different measuring distances to form Raw data samples, and respectively calculating the measuring depth values of the Raw data samples at the different measuring distances;
respectively determining the reference surfaces at different measuring distances, and performing surface transformation on the depth value of the reference surface by using the internal reference value of the ToF camera to obtain a surface depth value as the real depth value of the Raw data sample at the different measuring distances;
and constructing an initial distance error model based on the measured depth value and the real depth value, and fitting parameters of the initial distance error model by a least square method to obtain the distance error model.
3. The calibration method of the ToF camera according to claim 2, wherein said determining the reference surfaces at the different measurement distances respectively comprises:
controlling the ToF camera, respectively collecting calibration Raw data of the target white plane at different calibration distances, and respectively calculating depth values of the calibration Raw data at the different calibration distances by adopting a four-step phase method; superposing the depth difference values from the different calibration distances to the different measurement distances on the depth value of the calibration Raw data, and taking the superposed depth value as the depth value of the reference surface; fitting the depth value of the reference surface into a plane by adopting a least square method, and taking the fitted plane as the reference surface;
or,
taking the plane at the different measurement distances as the reference plane in a case where the center of the optical axis of the ToF camera is perpendicular to the target white plane.
4. The calibration method of the ToF camera according to claim 2 or 3, wherein the constructing the initial distance error model comprises:
and acquiring an influence factor of the temperature on a ToF sensor in the ToF camera, and constructing the initial distance error model based on the measured depth value, the real depth value and the influence factor.
5. The calibration method of the ToF camera according to claim 2 or 3, further comprising:
controlling the ToF camera to collect multi-frame calibration plate images under different camera poses, and carrying out internal reference calibration on the ToF camera by using a camera calibration kit based on the calibration plate images to obtain the internal reference values of the ToF camera;
the multi-frame calibration plate images are generated by storing confidence maps of calibration plate Raw data conversion or pictures directly acquired through the ToF camera, the multi-frame calibration plate images are the same in size, and the selection sequence of the multi-frame calibration plate images is consistent.
6. Calibration apparatus for a ToF camera, the apparatus comprising:
the first processing module is used for calculating the depth value of real-time Raw data acquired by the ToF camera;
the second processing module is used for compensating the depth value by utilizing a pre-established distance error model to obtain a calibration result;
the distance error model is obtained by performing model parameter fitting in advance based on measured depth values of Raw data samples acquired by the ToF camera at different measured distances and real depth values of the Raw data samples at the different measured distances, wherein the real depth values are obtained by performing curved surface acquisition on reference surfaces at the different measured distances based on internal parameters of the ToF camera.
7. The ToF camera calibration apparatus according to claim 6, further comprising:
a third processing module, configured to control the ToF camera, collect Raw data of a target white plane at the different measurement distances, respectively, to form Raw data samples, and calculate the measurement depth values of the Raw data samples at the different measurement distances, respectively;
a fourth processing module, configured to determine the reference surfaces at the different measurement distances, respectively, and perform surface transformation on the depth value of the reference surface by using the internal reference value of the ToF camera, so as to obtain a surface depth value as the true depth value of the Raw data sample at the different measurement distances;
and the fifth processing module is used for constructing an initial distance error model based on the measured depth value and the real depth value, fitting parameters of the initial distance error model through a least square method and obtaining the distance error model.
8. Calibration apparatus for ToF camera according to claim 7, wherein said fourth processing module, when being configured to determine said reference surfaces at said different measurement distances respectively, is configured to:
controlling the ToF camera, respectively collecting calibration Raw data of the target white plane at different calibration distances, and respectively calculating depth values of the calibration Raw data at the different calibration distances by adopting a four-step phase method; superposing the depth difference values from the different calibration distances to the different measurement distances on the depth value of the calibration Raw data, and taking the superposed depth value as the depth value of the reference surface; fitting the depth value of the reference surface into a plane by adopting a least square method, and taking the fitted plane as the reference surface;
or,
taking the plane at the different measurement distances as the reference plane in a case where the center of the optical axis of the ToF camera is perpendicular to the target white plane.
9. An electronic device, comprising a processor, a memory and a program or instructions stored on the memory and executable on the processor, which when executed by the processor, implement the steps of the calibration method for a ToF camera according to any one of claims 1 to 5.
10. A readable storage medium, characterized in that it stores thereon a program or instructions which, when executed by a processor, implement the steps of the calibration method of a ToF camera according to any one of claims 1 to 5.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110859955.2A CN113506351A (en) | 2021-07-28 | 2021-07-28 | Calibration method and device for ToF camera, electronic equipment and storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110859955.2A CN113506351A (en) | 2021-07-28 | 2021-07-28 | Calibration method and device for ToF camera, electronic equipment and storage medium |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN113506351A true CN113506351A (en) | 2021-10-15 |
Family
ID=78015017
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202110859955.2A Pending CN113506351A (en) | 2021-07-28 | 2021-07-28 | Calibration method and device for ToF camera, electronic equipment and storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN113506351A (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN114263352A (en) * | 2022-01-07 | 2022-04-01 | 中国建筑第八工程局有限公司 | Reinforcing steel bar binding robot and identification method of reinforcing steel bar intersection point |
Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN109102546A (en) * | 2018-07-16 | 2018-12-28 | 珠海市微半导体有限公司 | A kind of scaling method of the robot camera based on more scaling boards |
| CN112116666A (en) * | 2020-09-21 | 2020-12-22 | 华中科技大学鄂州工业技术研究院 | Pose adjusting method and device for camera calibration |
| CN113096189A (en) * | 2021-03-22 | 2021-07-09 | 西安交通大学 | ITOF depth camera calibration and depth optimization method |
-
2021
- 2021-07-28 CN CN202110859955.2A patent/CN113506351A/en active Pending
Patent Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN109102546A (en) * | 2018-07-16 | 2018-12-28 | 珠海市微半导体有限公司 | A kind of scaling method of the robot camera based on more scaling boards |
| CN112116666A (en) * | 2020-09-21 | 2020-12-22 | 华中科技大学鄂州工业技术研究院 | Pose adjusting method and device for camera calibration |
| CN113096189A (en) * | 2021-03-22 | 2021-07-09 | 西安交通大学 | ITOF depth camera calibration and depth optimization method |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN114263352A (en) * | 2022-01-07 | 2022-04-01 | 中国建筑第八工程局有限公司 | Reinforcing steel bar binding robot and identification method of reinforcing steel bar intersection point |
| CN114263352B (en) * | 2022-01-07 | 2023-10-03 | 中国建筑第八工程局有限公司 | Reinforcement binding robot and identification method of reinforcement intersection |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN110021046B (en) | External parameter calibration method and system for camera and laser radar combined sensor | |
| CN111354042B (en) | Feature extraction method and device of robot visual image, robot and medium | |
| US10334151B2 (en) | Phase detection autofocus using subaperture images | |
| US10277889B2 (en) | Method and system for depth estimation based upon object magnification | |
| WO2018140107A1 (en) | System for 3d image filtering | |
| CN110300292A (en) | Projection distortion bearing calibration, device, system and storage medium | |
| US20180245909A1 (en) | Object measurement apparatus and object measurement method | |
| CN102980556A (en) | Distance measuring method and device | |
| CN111986250B (en) | Object volume measuring method, device, measuring equipment and storage medium | |
| JP2012058076A (en) | Three-dimensional measurement device and three-dimensional measurement method | |
| US11042984B2 (en) | Systems and methods for providing image depth information | |
| CN114494388B (en) | Three-dimensional image reconstruction method, device, equipment and medium in large-view-field environment | |
| CN111307046B (en) | Tree height measuring method based on hemispherical image | |
| JPWO2015068470A1 (en) | 3D shape measuring apparatus, 3D shape measuring method, and 3D shape measuring program | |
| CN104864810A (en) | Digital measuring method and device thereof | |
| CN114578329A (en) | Multi-sensor joint calibration method, device, storage medium and program product | |
| CN103940344A (en) | High-precision remote displacement sensor | |
| CN116188591A (en) | Multi-camera global calibration method and device and electronic equipment | |
| CN113506351A (en) | Calibration method and device for ToF camera, electronic equipment and storage medium | |
| WO2022025283A1 (en) | Measurement processing device, method, and program | |
| CN109087341B (en) | Fusion method of close-range hyperspectral camera and ranging sensor | |
| JP2019100969A (en) | Distance measuring device, distance measurement method, and distance measurement program | |
| CN112146589A (en) | Three-dimensional morphology measurement system and method based on ZYNQ platform | |
| Ataiwe et al. | Digital model in close-range photogrammetry using a smartphone camera | |
| CN116228974A (en) | Three-dimensional model construction method, three-dimensional model construction device, computer equipment and storage medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |