CN115601445A - Method and device for acquiring true distance of calibration plate of ToF camera and electronic equipment - Google Patents
Method and device for acquiring true distance of calibration plate of ToF camera and electronic equipment Download PDFInfo
- Publication number
- CN115601445A CN115601445A CN202211286197.0A CN202211286197A CN115601445A CN 115601445 A CN115601445 A CN 115601445A CN 202211286197 A CN202211286197 A CN 202211286197A CN 115601445 A CN115601445 A CN 115601445A
- Authority
- CN
- China
- Prior art keywords
- distance
- diagonal
- reference matrix
- internal reference
- tof camera
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000000034 method Methods 0.000 title claims abstract description 35
- 239000011159 matrix material Substances 0.000 claims abstract description 77
- 238000006243 chemical reaction Methods 0.000 claims description 4
- 230000014509 gene expression Effects 0.000 claims description 4
- 238000005457 optimization Methods 0.000 claims description 4
- 238000001514 detection method Methods 0.000 claims description 3
- 230000006870 function Effects 0.000 description 6
- 238000004519 manufacturing process Methods 0.000 description 6
- 238000009434 installation Methods 0.000 description 5
- 238000005259 measurement Methods 0.000 description 5
- 238000004364 calculation method Methods 0.000 description 3
- 238000010586 diagram Methods 0.000 description 3
- 238000005034 decoration Methods 0.000 description 2
- 230000003247 decreasing effect Effects 0.000 description 2
- 238000005516 engineering process Methods 0.000 description 2
- 230000007246 mechanism Effects 0.000 description 2
- 238000012986 modification Methods 0.000 description 2
- 230000004048 modification Effects 0.000 description 2
- 230000003287 optical effect Effects 0.000 description 2
- 238000003384 imaging method Methods 0.000 description 1
- 230000002452 interceptive effect Effects 0.000 description 1
- 238000000691 measurement method Methods 0.000 description 1
- 239000004065 semiconductor Substances 0.000 description 1
- 230000000007 visual effect Effects 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/80—Analysis of captured images to determine intrinsic or extrinsic camera parameters, i.e. camera calibration
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/70—Determining position or orientation of objects or cameras
Landscapes
- Engineering & Computer Science (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Theoretical Computer Science (AREA)
- Measurement Of Optical Distance (AREA)
Abstract
The invention discloses a method and a device for acquiring a real distance of a calibration plate of a ToF camera and electronic equipment. The method comprises the following steps: acquiring vertexes in each corner point of a gray scale image of a calibration plate in a field of view of a TOF camera, wherein a chess board used for calibrating a lens is used as the calibration plate; taking a lens-calibrated internal reference matrix as an initial value, and acquiring a corresponding diagonal Euclidean distance according to the vertex; acquiring a corresponding diagonal true distance according to the actual length and width of the checkerboard; optimizing the internal reference matrix according to the difference value between the European diagonal distance and the real diagonal distance; and calculating the real distance from each pixel point in the gray-scale image to an image sensor of the TOF camera according to the optimized internal reference matrix. The chessboard used for calibrating the lenses is used as the FPPN calibration plate, so that the calibration time and the cost are saved; the Euclidean distance of the diagonal line is corrected based on the real distance of the diagonal line of the checkerboard, the reference matrix is optimized, and the accuracy of calculating the real distance is improved.
Description
Technical Field
The invention relates to the technical field of ToF ranging, in particular to a method and a device for acquiring a real distance of a calibration plate of a ToF camera and electronic equipment.
Background
Binocular ranging, structured light and Time-of-Flight (ToF for short) are three major 3D imaging technologies at present, wherein ToF has been gradually applied to the fields of gesture recognition, 3D modeling, unmanned driving, machine vision and the like due to the advantages of simple principle, simple and stable structure, long measurement distance and the like. The working principle of the ToF technology is as follows: the method comprises the steps of utilizing an external light source (VCSEL or LED or the like) to emit continuously modulated emitted light, enabling the emitted light to irradiate the surface of an object to be measured and then be reflected back, enabling the reflected light to be captured by an image sensor (sensor) of a ToF camera, and calculating the time difference or phase difference between the emitted light and the reflected light to obtain the depth/distance of the object from the camera. Among them, a method of calculating a distance by a time difference is called a pulse method (Pulsed ToF), and a method of calculating a distance by a phase difference is called a Continuous Wave method (Continuous-Wave ToF).
Due to the manufacturing process of the image sensor of the ToF camera and some optical characteristics, a series of calibrations, such as lens calibration, wiggling calibration, FPPN calibration, etc., need to be performed first on the ToF camera under the requirement of ensuring the measurement accuracy. The FPPN calibration is to compensate for the difference of each pixel point, so a white board covering the whole view field (fov) of the image sensor is needed to be used as a calibration board, the installation inclination angle of the calibration board is also accurately given, the true distance from each pixel point to the image sensor in the corresponding image of the calibration board is needed to be calculated, and otherwise, the FPPN calibration is inaccurate due to the errors. However, since the discharge position of the whiteboard on the production line is not determined, a certain difficulty is brought to the solution of the real distance, and lens calibration usually adopts a checkerboard with alternate black and white as a calibration board, so that the calibration board switching is required to be performed between different calibrations on the production line, which increases the calibration cost.
In addition, there are generally two methods for calculating the true distance from each pixel point to the image sensor: the first method is that the installation of the white board is ensured through a mechanism, and the real distance from each pixel point to the image sensor can be calculated on the basis of knowing the included angle between the white board and the ground and the distance from the central point of the image sensor to the white board by utilizing the trigonometric function relation; however, the mechanism ensures that the whiteboard is installed, and related hardware facilities are additionally added to increase calibration cost, so that the existing production line cannot be well utilized. The second method is that the world coordinate of each pixel point is calculated by an internal reference matrix and an external reference matrix through a monocular distance measurement method and through the conversion relation from the world coordinate to the pixel point coordinate, so that the real distance from each pixel point to an image sensor is calculated; however, the image sensor has low resolution and the transmitted modulated light is not uniform, which makes accurate internal reference calibration difficult.
Therefore, how to accurately calculate the real distance on the basis of saving the calibration cost of the ToF camera is a technical problem to be solved at present.
Disclosure of Invention
The invention aims to provide a method and a device for acquiring a true distance of a calibration plate of a ToF camera and electronic equipment, which are used for solving the technical problems that the calibration cost is increased or the true distance cannot be accurately calculated when the true distance of the calibration plate is calculated by the conventional ToF camera.
In order to achieve the above object, the present invention provides a method for obtaining a true distance of a calibration plate of a ToF camera, comprising the following steps: obtaining vertexes in each angular point of a gray scale image of a calibration plate in a field of view of a TOF camera, wherein a checkerboard used for calibrating lens is used as the calibration plate; taking a lens-calibrated internal reference matrix as an initial value, and acquiring a corresponding diagonal Euclidean distance according to the vertex; acquiring a corresponding diagonal true distance according to the actual length and width of the checkerboard; optimizing the internal parameter matrix according to the difference value between the Euclidean diagonal distance and the real diagonal distance; and calculating the real distance from each pixel point in the gray-scale image to an image sensor of the TOF camera according to the optimized internal reference matrix.
Optionally, the step of acquiring a vertex in each corner point of the gray scale image of the calibration plate in the field of view of the TOF camera further includes: calculating the brightness value of each pixel point in a gray scale image according to the original data of the gray scale image of a calibration plate in the field of view of the ToF camera, which is output by an image sensor of the ToF camera; acquiring each corner of the gray-scale image according to the brightness value; and selecting the vertex in each corner point according to the coordinates of each corner point.
Optionally, the step of iteratively optimizing the internal reference matrix according to the difference between the euclidean distance of the diagonal line and the true distance of the diagonal line further includes: and taking the lens-calibrated internal reference matrix as an initial value, and calculating a target value by iterative search, wherein the difference value between the Euclidean distance of the diagonal line and the real distance of the diagonal line is smaller than or equal to a preset difference value range, so that the internal reference matrix is optimized.
In order to achieve the above object, the present invention further provides a device for obtaining a true distance of a calibration plate of a ToF camera, including: the vertex acquisition module is used for acquiring vertexes in all corner points of a gray scale image of a calibration plate in a field of view of a TOF camera, wherein a chess board used for calibrating a lens is used as the calibration plate; the system comprises a diagonal Euclidean distance acquisition module, a diagonal Euclidean distance acquisition module and a parameter calculation module, wherein the diagonal Euclidean distance acquisition module is used for taking an internal reference matrix marked by a lens as an initial value and acquiring a corresponding diagonal Euclidean distance according to the vertex; the diagonal real distance acquisition module is used for acquiring the corresponding diagonal real distance according to the actual length and width of the checkerboard; the optimization module is used for iteratively optimizing the internal parameter matrix according to the difference value of the Euclidean distance of the diagonal line and the real distance of the diagonal line; and the real distance acquisition module is used for calculating the real distance from each pixel point in the gray-scale image to an image sensor of the TOF camera according to the optimized internal reference matrix.
To achieve the above object, the present invention further provides an electronic device, which includes a memory, a processor and a computer executable program stored in the memory and running on the processor, wherein the processor executes the computer executable program to implement the steps of the method for obtaining the true distance of the ToF camera calibration board according to the present invention.
According to the method and the device for acquiring the true distance of the calibration plate of the ToF camera, provided by the invention, the chessboard used by lens calibration is used as the calibration plate to calibrate the FPPN, and a white plate covering the whole field of view is not required to be additionally provided to calibrate the FPPN, so that different calibration plate patterns are not required to be switched during calibration on a production line; by acquiring vertexes in each corner point of the checkerboard gray-scale image in the field of view and acquiring corresponding diagonal Euclidean distances, the checkerboard can be basically placed at any position in the field of view of the TOF camera, the condition that the TOF camera is parallel to the calibration board or the installation inclination angle of the calibration board and the like are accurately given is not required to be limited, the distance from the central point of the image sensor to the calibration board is not required to be known, and calibration time and cost are saved; correcting a diagonal Euclidean distance based on the real diagonal distance of the checkerboard, and optimizing an internal reference matrix calibrated by the lens, so that the accuracy of calculating the real distance is improved; and through utilizing the lens to mark used check board as the FPPN calibration board, utilize monocular distance measurement principle can calculate the true distance, on the basis of saving calibration cost of ToF camera, can calculate the true distance of calibration board accurately.
Drawings
In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the drawings that are required to be used in the description of the embodiments will be briefly described below. It is obvious that the drawings in the following description are only some embodiments of the invention, and that for a person skilled in the art, other drawings can be derived from them without inventive effort.
Fig. 1 is a flowchart of a method for obtaining a true distance of a ToF camera calibration plate according to an embodiment of the present invention;
FIG. 2 is a checkerboard gray scale graph according to an embodiment of the present invention;
FIG. 3 is a schematic diagram of a checkerboard provided in accordance with an embodiment of the present invention;
fig. 4 is a block diagram of a real distance obtaining device of a ToF camera calibration board according to an embodiment of the invention.
Detailed Description
The technical solution in the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. It is to be understood that the disclosed embodiments are merely exemplary of the invention, and are not intended to be exhaustive or exhaustive. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.
An embodiment of the invention provides a method for acquiring a true distance of a calibration plate of a ToF camera.
Please refer to fig. 1, which is a flowchart illustrating a method for obtaining a true distance of a calibration plate of a ToF camera according to an embodiment of the invention. As shown in fig. 1, the method of this embodiment includes the following steps: s11, obtaining vertexes in each corner point of a gray scale image of a calibration plate in a TOF camera view field, wherein a chess board used for calibrating a lens is used as the calibration plate; s12, taking the lens-calibrated internal reference matrix as an initial value, and acquiring a corresponding diagonal Euclidean distance according to the vertex; s13, acquiring a corresponding real diagonal distance according to the actual length and width of the checkerboard; s14, optimizing the internal reference matrix according to the difference value between the Euclidean distance of the diagonal line and the real distance of the diagonal line; and S15, calculating the real distance from each pixel point in the gray-scale image to an image sensor of the TOF camera according to the optimized internal reference matrix.
Regarding step S11, the vertex in each corner point of the gray scale image of the calibration plate in the field of view of the TOF camera is acquired, wherein the checkerboard used for lens calibration is used as the calibration plate. Specifically, a checkerboard used for calibrating a lens as a calibration plate is shot by the ToF camera, a gray scale image of the checkerboard in a field of view (fov) of the ToF camera is acquired, and each corner point of the gray scale image is further acquired, so that a corresponding vertex is acquired. For example, for an n × m checkerboard, the detection obtains each vertex corner _11, corner _1n, corner _ m1, corner _ mn, as shown in fig. 2.
The calibration plane of the calibration plate is a surface of the checkerboard facing the lens of the ToF camera, i.e. a plane in the checkerboard for reflecting light emitted by the ToF camera. In this embodiment, the checkerboard as the calibration board can be placed at substantially any position in the field of view of the TOF camera without knowing the distance from the center point of the image sensor to the calibration board. Through adopting the alternate checkerboard of black and white that the lens was used to mark as the calibration board and carry out FPPN demarcation, need not additionally to provide a blank that covers whole visual field again and carry out FPPN demarcation for need not to switch different calibration board patterns when the mark on producing the line. By means of obtaining vertexes of all corner points of the checkerboard gray-scale image in the view field, the checkerboard can be basically placed at any position in the view field of the TOF camera, and the TOF camera and the calibration plate do not need to be limited to be parallel or the installation inclination angle of the calibration plate can be accurately given; and the distance from the central point of the image sensor to the calibration plate is not required to be known, so that calibration time and cost are saved.
In some embodiments, the step of acquiring the vertex in each corner point of the gray scale map of the calibration plate in the field of view of the TOF camera further includes: calculating the brightness value (amplitude) of each pixel point in a gray scale image according to raw data (raw data) of the gray scale image of a calibration plate in a field of view of the ToF camera, which is output by an image sensor of the ToF camera; acquiring each corner of the gray-scale image according to the brightness value; and selecting the vertex in each corner point according to the coordinates of each corner point.
In some embodiments, the following formula is used to calculate the brightness value amplitude of each pixel point in the gray scale map: amplitude = abs (I) + abs (Q); wherein, I = A0-B0- (a 180-B180), Q = a90-B90- (a 270-B270), and a and B are raw data corresponding to the phases of 0, 90, 180, and 270 output of the image sensor. Wherein, the modulation wave emitted by the TOF camera is a square wave.
In some embodiments, the step of obtaining corner points of the gray-scale map according to the brightness values further includes: and acquiring each corner of the gray-scale image by adopting an opencv corner detection mode according to the brightness value. For example, the Harris algorithm, shi-Tomasi algorithm, etc. can be used to find, screen, and mark each corner point in the gray scale map based on the brightness value of the gray scale map.
Regarding step S12, the lens-calibrated internal reference matrix is used as an initial value, and the corresponding diagonal euclidean distance is obtained according to the vertex. Specifically, after each vertex in the gray-scale image is obtained, based on the lens-calibrated internal reference matrix, the euclidean distance of the diagonal line corresponding to each vertex can be calculated. The obtained diagonal Euclidean distance is the initial diagonal Euclidean distance of the lens-calibrated internal reference matrix, and the diagonal Euclidean distance is corrected based on the real diagonal distance subsequently, so that the lens-calibrated internal reference matrix is optimized, and the accuracy of calculating the real distance is improved.
In some embodiments, the step of obtaining the corresponding diagonal euclidean distance according to the vertex further comprises: (1) Calculating the world coordinates of each vertex in the gray-scale image by adopting the following conversion relation between the pixel coordinates and the world coordinates:
wherein z is c Representing the scaling coefficient, u and v are coordinate values of a pixel point in the pixel coordinate system, the matrix corresponding to M1 is an internal reference matrix, u 0 And v 0 Is the center point of the internal reference matrix, f x And f y Is the focal length of the internal reference matrix, the matrix corresponding to M2 is the external reference matrix, r is the direction vector of the coordinate axis of the pixel coordinate system in the coordinate axis of the world coordinate system, t is the translation vector from the origin of the world coordinate system to the origin of the pixel coordinate system, x w 、y w 、z w Coordinate values of corresponding points in a world coordinate system; and (2) according to the world coordinates of each vertex, calculating the corresponding diagonal Euclidean distances d1 and d2 by adopting the following formula:
d1＝sqrt((x w11 -x wmn ) 2 +(y w11 -y wmn ) 2 +(z w11 -z wmn ) 2 )，
d2＝sqrt((x w1n -x wm1 ) 2 +(y w1n -y wm1 ) 2 +(z w1n -z wm1 ) 2 )。
after the vertex in each corner point in the gray-scale image is obtained, the pixel coordinate value of each vertex is converted into a world coordinate value based on the lens-calibrated reference matrix, and then the corresponding diagonal Euclidean distances d1 and d2 are calculated based on the world coordinate of each vertex, as shown in FIG. 2. The obtained European diagonal distance is the initial European diagonal distance of the lens-calibrated internal reference matrix, and the European diagonal distance is corrected based on the real diagonal distance to optimize the lens-calibrated internal reference matrix, so that the accuracy of calculating the real distance is improved.
And S13, acquiring the corresponding real diagonal distance according to the actual length and width of the checkerboard. Specifically, for the selected checkerboard, the actual length and width of the checkerboard are known, and the true distance of the diagonal can be obtained according to the Pythagorean theorem.
Specifically, the step of obtaining the corresponding real diagonal distance according to the actual length and width of the checkerboard further includes: obtaining corresponding diagonal real distances drel1 and drel2 according to the Pythagorean theorem: drel1= drel2= sqrt (x) 2 +y 2 ) (ii) a Where x is the actual length of the checkerboard (the horizontal length of the checkerboard is shown) and y is the actual width of the checkerboard (the vertical length of the checkerboard is shown), as shown in fig. 3.
And S14, optimizing the internal reference matrix according to the difference value of the Euclidean diagonal distance and the real diagonal distance. Specifically, due to the low resolution of the image sensor of the ToF camera and the non-uniformity of the modulated light, there may be a difference between the euclidean distance of the diagonal line obtained based on the lens-calibrated internal reference matrix and the true distance of the diagonal line; if the true distance is calculated based on the lens-calibrated internal reference matrix, a certain deviation may exist. Therefore, a set of appropriate target internal reference matrixes is calculated by iterative search by taking the lens calibrated internal reference matrixes as initial values, so that the accuracy of real distance calculation is improved.
In some embodiments, the lens-calibrated internal reference matrix is used as an initial value (i.e., an initial internal reference matrix), and a target value that makes the difference between the euclidean distance of the diagonal line and the true distance of the diagonal line smaller than or equal to a preset difference range is calculated through iterative search, so as to optimize the internal reference matrix. Specifically, the optimal solution of the internal parameter matrix is iteratively searched by changing the value of the variable in the initial internal parameter matrix.
Specifically, the iteration constraint added according to the diagonal true distance and the diagonal Euclidean distance is argmin (abs (drel 1-d 1) + abs (drel 2-d 2)). Considering that the focal lengths fx and fy of the internal reference matrix are relatively close, it is further required to satisfy: abs (f) x -f y )<thr, where thr is a user defined maximum threshold for focus difference. By varying f x And/or f y The values of (b) are such that the diagonal Euclidean distances d1, d2 calculated based on the corrected reference matrix satisfy argmin (abs (drel 1-d 1) + abs (drel 2-d 2)). Assume, initially, f x ＝100、f y =100 by increasing or decreasing f x (e.g. f) x Is changed within 90-110, and satisfies the following conditions: abs (f) x -f y )<thr), obtaining a new internal reference matrix, and recalculating to obtain diagonal Euclidean distances d1 and d2; or by increasing or decreasing f y (e.g. f) y Varies from 90 to 110 and satisfies: abs (f) x -f y )<thr) to obtain a new internal reference matrix, and recalculating to obtain diagonal Euclidean distances d1 and d2; or simultaneously changing f x And f y Value of (e.g. f) x Varies from 90 to 110, f y Also varies from 90 to 110 and satisfies: abs (f) x -f y )<thr), a new internal reference matrix is obtained, and diagonal Euclidean distances d1 and d2 are obtained through recalculation.
In some embodiments, the step of iteratively optimizing the reference matrix according to the difference between the euclidean distance of the diagonal line and the true distance of the diagonal line further comprises: judging whether the following relation is satisfied: argmin (abs (drel 1-d 1) + abs (drel 2-d 2)) & & abs (fx-fy) < thr, wherein drel1 and drel2 are diagonal true distances, d1 and d2 are diagonal Euclidean distances, and thr is a maximum threshold value of a predetermined focus difference; if the relation is established, acquiring the current internal reference matrix as the optimized internal reference matrix; and if the relation is not established, changing the values of fx and/or fy, and iteratively searching an internal reference matrix meeting the establishment of the relation as the optimized internal reference matrix. Wherein argmin is the value of d1 and d2 when the target function abs (drel 1-d 1) + abs (drel 2-d 2) is minimum; and & represents the meaning of logical AND, namely AND, when the results of expressions on both sides of an operator are both true, the whole operation result is true, otherwise, as long as one side is false, the result is false; and & also has the function of short circuit, i.e. if the first expression is false, the second expression is not computed anymore.
And S15, calculating the real distance from each pixel point in the gray-scale image to an image sensor of the TOF camera according to the optimized internal reference matrix. Specifically, according to the optimized internal reference matrix, the real distance from each pixel point in the gray-scale image to the image sensor of the TOF camera can be calculated through the conversion relation between the pixel coordinate and the world coordinate. And because the lens-calibrated internal reference matrix is used as an initial value, an iteration constraint condition is added according to the diagonal true distance and the diagonal Euclidean distance, a group of proper target internal reference matrices are calculated through iterative search, and the accuracy of true distance calculation is improved. The invention can calculate the real distance by using the monocular distance measurement principle by using the chessboard grids used by the lens calibration as the FPPN calibration plate, and can accurately calculate the real distance of the calibration plate on the basis of saving the calibration cost of the ToF camera.
Based on the same inventive concept, the invention also provides a device for acquiring the true distance of the calibration plate of the ToF camera. The provided device for acquiring the real distance of the calibration plate of the ToF camera can adopt the method for acquiring the real distance of the calibration plate of the ToF camera shown in fig. 1 to perform FPPN calibration on the ToF camera so as to acquire the real distance of the calibration plate.
Please refer to fig. 4, which is a block diagram illustrating an apparatus for obtaining a true distance of a calibration plate of a ToF camera according to an embodiment of the invention. As shown in fig. 4, the real distance acquiring apparatus of the ToF camera calibration plate includes: a vertex acquisition module 41, a diagonal euclidean distance acquisition module 42, a diagonal true distance acquisition module 43, an optimization module 44, and a true distance acquisition module 45.
Specifically, the vertex acquisition module 41 is configured to acquire a vertex in each corner point of a gray scale image of a calibration plate in a field of view of the TOF camera; wherein, a chess board used by the lens calibration is used as the calibration board. The diagonal Euclidean distance obtaining module 42 is configured to obtain a corresponding diagonal Euclidean distance according to the vertex by using the lens-calibrated internal reference matrix as an initial value. The diagonal true distance obtaining module 43 is configured to obtain a corresponding diagonal true distance according to the actual length and width of the checkerboard. The optimization module 44 is configured to iteratively optimize the reference matrix according to a difference between the euclidean distance of the diagonal line and the true distance of the diagonal line. The real distance obtaining module 45 is configured to calculate a real distance from each pixel point in the gray-scale map to an image sensor of the TOF camera according to the optimized internal reference matrix. The working modes of the modules can refer to the description of the corresponding steps in the method for obtaining the actual distance of the ToF camera calibration plate shown in fig. 1, and are not described herein again.
According to the method and the device for acquiring the true distance of the calibration plate of the ToF camera, provided by the invention, the chessboard used by lens calibration is used as the calibration plate to calibrate the FPPN, and a white plate covering the whole field of view is not required to be additionally provided to calibrate the FPPN, so that different calibration plate patterns are not required to be switched during calibration on a production line; by acquiring vertexes in each corner point of the checkerboard gray-scale image in the field of view and acquiring corresponding diagonal Euclidean distances, the checkerboard can be basically placed at any position in the field of view of the TOF camera, the condition that the TOF camera is parallel to the calibration board or the installation inclination angle of the calibration board and the like are accurately given is not required to be limited, the distance from the central point of the image sensor to the calibration board is not required to be known, and calibration time and cost are saved; correcting a diagonal Euclidean distance based on the real diagonal distance of the checkerboard, and optimizing an internal reference matrix calibrated by the lens, so that the accuracy of calculating the real distance is improved; and through utilizing the lens to mark used check board as the FPPN calibration board, utilize monocular distance measurement principle can calculate the true distance, on the basis of saving calibration cost of ToF camera, can calculate the true distance of calibration board accurately.
Based on the same inventive concept, the invention also provides an electronic device, which comprises a memory, a processor and a computer executable program, wherein the computer executable program is stored on the memory and can run on the processor; the processor, when executing the computer executable program, implements the steps of the method for obtaining the true distance of the ToF camera calibration plate as shown in fig. 1.
It is within the scope of the inventive concept that embodiments may be described and illustrated in terms of modules performing one or more of the described functions. These modules (which may also be referred to herein as cells, etc.) may be physically implemented by analog and/or digital circuitry, such as logic gates, integrated circuits, microprocessors, microcontrollers, memory circuits, passive electronic components, active electronic components, optical components, hardwired circuitry, etc., and may optionally be driven by firmware and/or software. The circuitry may be implemented in one or more semiconductor chips, for example. The circuitry making up the modules may be implemented by dedicated hardware, or by a processor (e.g., one or more programmed microprocessors and associated circuitry), or by a combination of dedicated hardware to perform some of the functions of the module and a processor to perform other functions of the module. Each module of an embodiment may be physically separated into two or more interactive and discrete modules without departing from the scope of the inventive concept. Likewise, the modules of the embodiments may be physically combined into more complex modules without departing from the scope of the inventive concept.
In general, terms may be understood, at least in part, from their usage in context. For example, the term "one or more" as used herein may be used in a singular sense to describe a feature, structure, or characteristic, or may be used in a plural sense to describe a feature, structure, or combination of features, at least in part, depending on the context. Additionally, the term "based on" may be understood as not necessarily intended to convey an exclusive set of factors, but may instead allow for the presence of other factors not necessarily expressly described, again depending at least in part on the context.
It should be noted that the terms "comprising" and "having," and variations thereof, as used in the context of the present invention, are intended to cover a non-exclusive inclusion. The terms "first," "second," and the like are used for distinguishing between similar elements and not necessarily for describing a particular sequential or chronological order, unless otherwise indicated by the context, it being understood that the data so used may be interchanged where appropriate. In addition, the embodiments and features of the embodiments in the present invention may be combined with each other without conflict. Moreover, in the foregoing description, descriptions of well-known components and techniques are omitted so as to not unnecessarily obscure the concepts of the present invention. In the above embodiments, each embodiment is described with emphasis on differences from other embodiments, and the same/similar parts among the embodiments may be referred to each other.
The foregoing is only a preferred embodiment of the present invention, and it should be noted that, for those skilled in the art, various modifications and decorations can be made without departing from the principle of the present invention, and these modifications and decorations should also be regarded as the protection scope of the present invention.
Claims (10)
1. A method for acquiring the real distance of a calibration plate of a ToF camera is characterized by comprising the following steps:
obtaining vertexes in each angular point of a gray scale image of a calibration plate in a field of view of a TOF camera, wherein a checkerboard used for calibrating lens is used as the calibration plate;
taking a lens-calibrated internal reference matrix as an initial value, and acquiring a corresponding diagonal Euclidean distance according to the vertex;
acquiring a corresponding diagonal true distance according to the actual length and width of the checkerboard;
optimizing the internal reference matrix according to the difference value between the European diagonal distance and the real diagonal distance; and
and calculating the real distance from each pixel point in the gray-scale image to an image sensor of the TOF camera according to the optimized internal reference matrix.
2. The method of claim 1, wherein the step of obtaining the vertex of each corner point of the gray scale image of the calibration plate in the field of view of the TOF camera further comprises:
calculating the brightness value of each pixel point in a gray scale image according to the original data of the gray scale image of a calibration plate in the field of view of the ToF camera, which is output by an image sensor of the ToF camera;
acquiring each corner of the gray-scale image according to the brightness value; and
and selecting the vertex in each corner point according to the coordinates of each corner point.
3. The method of claim 2, wherein the luminance value amplitude of each pixel in the gray-scale map is calculated by the following formula:
amplitude＝abs(I)+abs(Q)，
wherein, I = A0-B0- (a 180-B180), Q = a90-B90- (a 270-B270), and a and B are raw data corresponding to the phase of 0, 90, 180, and 270 output of the image sensor.
4. The method of claim 2, wherein the step of obtaining the corner points of the gray-scale map according to the brightness values further comprises: and acquiring each corner of the gray-scale image by adopting an opencv corner detection mode according to the brightness value.
5. The method of claim 1, wherein the step of iteratively optimizing the reference matrix based on the difference between the Euclidean diagonal distance and the true diagonal distance further comprises:
and taking the lens-calibrated internal reference matrix as an initial value, and calculating a target value by iterative search so that the difference value between the Euclidean distance of the diagonal line and the real distance of the diagonal line is less than or equal to a preset difference value range, thereby optimizing the internal reference matrix.
6. The method of claim 1, wherein said step of obtaining a corresponding diagonal euclidean distance from said vertex further comprises:
and calculating the world coordinates of each vertex by adopting the following conversion relation between the pixel coordinates and the world coordinates:
wherein z is c Representing the scaling coefficient, u and v are coordinate values of a pixel point in the pixel coordinate system, the matrix corresponding to M1 is an internal reference matrix, u 0 And v 0 Is the center point of the internal reference matrix, f x And f y Is the focal length of the internal reference matrix, the matrix corresponding to M2 is the external reference matrix, r is the direction vector of the coordinate axis of the pixel coordinate system in the coordinate axis of the world coordinate system, t is the translation vector from the origin of the world coordinate system to the origin of the pixel coordinate system, x w 、y w 、z w Coordinate values of corresponding points in a world coordinate system; and
according to the world coordinates of each vertex, calculating corresponding diagonal Euclidean distances d1 and d2 by adopting the following formulas:
d1＝sqrt((x w11 -x wmn ) 2 +(y w11 -y wmn ) 2 +(z w11 -z wmn ) 2 )，
d2＝sqrt((x w1n -x wm1 ) 2 +(y w1n -y wm1 ) 2 +(z w1n -z wm1 ) 2 )。
7. the method of claim 6, wherein said step of obtaining the corresponding diagonal true distance according to the actual length and width of said checkerboard further comprises:
obtaining corresponding diagonal real distances drel1 and drel2 according to the Pythagorean theorem:
drel1＝drel2＝sqrt(x 2 +y 2 )；
wherein, x is the actual length of the checkerboard, and y is the actual width of the checkerboard.
8. The method of claim 7, wherein the step of iteratively optimizing the reference matrix based on the difference between the Euclidean diagonal distance and the true diagonal distance further comprises:
judging whether the following relation is satisfied:
argmin(abs(drel1-d1)+abs(drel2-d2))&&abs(f x -f y )<thr，
wherein, drel1 and drel2 are diagonal real distances, d1 and d2 are diagonal Euclidean distances, and thr is a maximum threshold value of the preset focal length difference;
if the relation is established, acquiring the current internal reference matrix as the optimized internal reference matrix;
if the relation is not satisfied, change f x And/or f y And (4) iteratively searching an internal reference matrix satisfying the establishment of the relational expression as the optimized internal reference matrix.
9. A real distance acquisition apparatus of a ToF camera calibration plate, comprising:
the vertex acquisition module is used for acquiring vertexes in all corner points of a gray scale image of a calibration plate in a field of view of the TOF camera, wherein a checkerboard used for calibrating lens is used as the calibration plate;
the diagonal Euclidean distance acquisition module is used for acquiring a corresponding diagonal Euclidean distance according to the vertex by taking the lens-calibrated internal reference matrix as an initial value;
the diagonal real distance acquisition module is used for acquiring the corresponding diagonal real distance according to the actual length and width of the checkerboard;
the optimization module is used for iteratively optimizing the internal parameter matrix according to the difference value of the Euclidean distance of the diagonal line and the real distance of the diagonal line; and
and the real distance acquisition module is used for calculating the real distance from each pixel point in the gray-scale image to an image sensor of the TOF camera according to the optimized internal reference matrix.
10. An electronic device comprising a memory, a processor and a computer-executable program stored on the memory and executable on the processor, wherein the processor implements the steps of the method for obtaining the true distance of the ToF camera calibration plate according to any one of claims 1 to 8 when executing the computer-executable program.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202211286197.0A CN115601445A (en) | 2022-10-20 | 2022-10-20 | Method and device for acquiring true distance of calibration plate of ToF camera and electronic equipment |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202211286197.0A CN115601445A (en) | 2022-10-20 | 2022-10-20 | Method and device for acquiring true distance of calibration plate of ToF camera and electronic equipment |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN115601445A true CN115601445A (en) | 2023-01-13 |
Family
ID=84849636
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202211286197.0A Pending CN115601445A (en) | 2022-10-20 | 2022-10-20 | Method and device for acquiring true distance of calibration plate of ToF camera and electronic equipment |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN115601445A (en) |
-
2022
- 2022-10-20 CN CN202211286197.0A patent/CN115601445A/en active Pending
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN108377380B (en) | Image scanning system and method thereof | |
| US7321839B2 (en) | Method and apparatus for calibration of camera system, and method of manufacturing camera system | |
| JP5503578B2 (en) | Object detection apparatus and object detection method | |
| CN112634374B (en) | Stereoscopic calibration method, device and system for binocular camera and binocular camera | |
| CN111045029B (en) | Fused depth measuring device and measuring method | |
| WO2021259151A1 (en) | Calibration method and apparatus for laser calibration system, and laser calibration system | |
| CN108700662A (en) | Range image acquisition device and its application | |
| US10107899B1 (en) | System and method for calibrating light intensity | |
| CN113658241B (en) | Monocular structured light depth recovery method, electronic device and storage medium | |
| KR20180018736A (en) | Method for acquiring photographing device and depth information | |
| CN112017248B (en) | 2D laser radar camera multi-frame single-step calibration method based on dotted line characteristics | |
| CN102401901B (en) | Distance measurement system and distance measurement method | |
| CN112419427A (en) | Method for improving time-of-flight camera accuracy | |
| CN115601445A (en) | Method and device for acquiring true distance of calibration plate of ToF camera and electronic equipment | |
| JP2018066579A (en) | In-vehicle camera calibration device and in-vehicle camera calibration method | |
| US20230009071A1 (en) | Control method for light sources of vision machine, and vision machine | |
| KR101818104B1 (en) | Camera and camera calibration method | |
| JP7152506B2 (en) | Imaging device | |
| KR101684095B1 (en) | Apparatus and method for estimating distance to a pedestrian | |
| WO2021232247A1 (en) | Point cloud coloring method, point cloud coloring system, and computer storage medium | |
| EP3001141B1 (en) | Information processing system and information processing method | |
| CN109239727B (en) | Distance measurement method combining solid-state area array laser radar and double CCD cameras | |
| CN111373222A (en) | Light projection system | |
| CN113643337B (en) | Camera line deviation calibration method and device, electronic equipment and storage medium | |
| CN117893588A (en) | Depth reconstruction method under extremely close range and electronic equipment |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| TA01 | Transfer of patent application right | ||
| TA01 | Transfer of patent application right |
Effective date of registration: 20231103 Address after: Room 308 and Room 309, No. 268 Xiangke Road, China (Shanghai) Pilot Free Trade Zone, Pudong New Area, Shanghai, 201210 Applicant after: Shanghai Jieming Technology Co.,Ltd. Address before: 351100 705-706, building a, Jiulong community, No. 999, lihuadong Avenue, Xialin street, Chengxiang District, Putian City, Fujian Province Applicant before: Fujian Jiemu Technology Co.,Ltd. |