WO2018153374A1 - Étalonnage de caméras - Google Patents

Étalonnage de caméras Download PDF

Info

Publication number
WO2018153374A1
WO2018153374A1 PCT/CN2018/077341 CN2018077341W WO2018153374A1 WO 2018153374 A1 WO2018153374 A1 WO 2018153374A1 CN 2018077341 W CN2018077341 W CN 2018077341W WO 2018153374 A1 WO2018153374 A1 WO 2018153374A1
Authority
WO
WIPO (PCT)
Prior art keywords
camera
mapping
adjustment
parameter values
external parameter
Prior art date
Application number
PCT/CN2018/077341
Other languages
English (en)
Chinese (zh)
Inventor
汪孔桥
赵威
Original Assignee
安徽华米信息科技有限公司
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 安徽华米信息科技有限公司 filed Critical 安徽华米信息科技有限公司
Priority to US16/034,083 priority Critical patent/US10586352B2/en
Publication of WO2018153374A1 publication Critical patent/WO2018153374A1/fr

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/80—Analysis of captured images to determine intrinsic or extrinsic camera parameters, i.e. camera calibration
    • G06T7/85—Stereo camera calibration
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N13/00—Stereoscopic video systems; Multi-view video systems; Details thereof
    • H04N13/20—Image signal generators
    • H04N13/204—Image signal generators using stereoscopic image cameras
    • H04N13/239—Image signal generators using stereoscopic image cameras using two 2D image sensors having a relative position equal to or related to the interocular distance
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N13/00—Stereoscopic video systems; Multi-view video systems; Details thereof
    • H04N13/20—Image signal generators
    • H04N13/204—Image signal generators using stereoscopic image cameras
    • H04N13/246—Calibration of cameras
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/10—Image acquisition modality
    • G06T2207/10004—Still image; Photographic image
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/10—Image acquisition modality
    • G06T2207/10004—Still image; Photographic image
    • G06T2207/10012—Stereo images

Definitions

  • the present disclosure relates to the field of computer vision technology, and in particular, to a camera calibration method, device, and electronic device.
  • a standard calibration board (such as a standard chessboard) can be used to simultaneously shoot the calibration plate multiple times with a binocular/multi-head camera, and change the posture and position of the calibration plate as much as possible. Collect enough sample images. Therefore, this calibration method takes a long time.
  • the present disclosure provides a new technical solution, which can avoid manual intervention in the calibration process and improve the efficiency of camera calibration.
  • a camera calibration method comprising:
  • each The adjustment method has a preset step size
  • a camera calibration apparatus comprising:
  • a first determining module configured to determine a first feature point set and a second feature point set that match between the first original image captured by the first camera and the second original image captured by the second camera;
  • a first mapping module configured to map, according to the first set of external parameter values of the first camera, the first set of feature points determined by the first determining module to a reference mapping surface, to obtain a first mapping point set;
  • the parameter adjustment module is configured to adjust the second set of external parameter values used by the second camera according to the preset M adjustment manners, to obtain M intermediate second set external parameter values, where M is greater than or equal to 2 An integer, each of the adjustment methods having a preset step size;
  • a second mapping module configured to map, according to the intermediate second set of external parameter values obtained by the parameter adjustment module by using each of the adjustment manners, the second feature point set determined by the first determining module to the Referring to the mapping surface, obtaining a second mapping point set corresponding to the adjusting manner;
  • a first calculation module configured to calculate, between the second mapping point set corresponding to each of the adjustment modes obtained by the second mapping module, and the first mapping point set obtained by the first mapping module Distance, the distance corresponding to the adjustment method
  • An external parameter calibration module configured to adjust a second set of external parameter values adopted by the second camera based on the distance corresponding to each of the adjustment modes and the step size, corresponding to each of the adjustment modes If the distance is determined to satisfy the iterative termination condition, the calibration of the external parameter value of the second camera is completed, otherwise the parameter adjustment module continues to use the second group of the second camera by the M adjustment modes.
  • the outer parameter values are adjusted to continue the iteration.
  • an electronic device comprising:
  • a storage medium for storing the processor-executable instructions
  • the processor is configured to perform the camera calibration method provided by the first aspect above.
  • the present disclosure can extract image feature points on the original images captured by the first camera and the second camera, and iterate the second set of external parameter values according to the preset iteration manner according to the matched feature points. Since the shooting of the original image does not require manual adjustment and intervenes in the posture and position of the first camera and the second camera, the external content of the second camera can be calibrated using the scene content in the original image captured by the first camera and the second camera. The calibration process of the external parameter value of the second camera is greatly shortened, the efficiency of the camera calibration is improved, and the mass production of binocular and multi-camera imaging equipment is facilitated.
  • FIG. 1 is a scene diagram of a first camera and a second camera in a camera calibration method provided by the present disclosure
  • FIG. 2A is a schematic flowchart of a camera calibration method according to an exemplary embodiment of the present disclosure
  • FIG. 2B is a schematic diagram showing a first original image and a second original image according to the embodiment shown in FIG. 2A;
  • 2C is a schematic diagram showing mapping of P points in the first feature point set to D points on the reference mapping surface according to the embodiment shown in FIG. 2A;
  • FIG. 3 is a schematic flowchart diagram of a camera calibration method according to still another exemplary embodiment of the present disclosure
  • FIG. 4A is a schematic flowchart diagram of a camera calibration method according to another exemplary embodiment of the present disclosure.
  • FIG. 4B is a schematic diagram of a correspondence relationship between a reference mapping surface and an imaging plane in the embodiment shown in FIG. 4A;
  • 5A is a schematic diagram of a projection image obtained by projecting a second original image onto a reference mapping surface by an initial value of an outer parameter of the second camera;
  • 5B is a schematic diagram of a projected image obtained by projecting a second original image onto a reference mapping surface by an external parameter after calibration of the present disclosure
  • FIG. 6 is a schematic structural diagram of a camera calibration apparatus according to an exemplary embodiment of the present disclosure.
  • FIG. 7 is a schematic structural diagram of a camera calibration apparatus according to another exemplary embodiment of the present disclosure.
  • FIG. 8 is a schematic structural diagram of a camera calibration apparatus according to another exemplary embodiment of the present disclosure.
  • FIG. 9 shows a schematic structural diagram of an electronic device according to an exemplary embodiment of the present disclosure.
  • first, second, third, etc. may be used in the present disclosure to describe various information, such information should not be limited to these terms. These terms are only used to distinguish the same type of information from each other.
  • first information may also be referred to as second information without departing from the scope of the present disclosure.
  • second information may also be referred to as first information.
  • word "if” as used herein may be interpreted as "when” or “when” or “in response to a determination.”
  • the camera calibration in the present disclosure refers to determining that the second camera 12 is in space with respect to the first camera 11
  • the relationship may include, for example, the distance between the optical centers of the first camera 11 and the second camera 12 and the angle of rotation between the optical axes of the first camera 11 and the second camera 12.
  • the first camera 11 and the second camera 12 each have an in-camera parameter and an out-of-camera parameter, and the following may be simply referred to as an inner parameter and an outer parameter, respectively.
  • the internal parameters may characterize the camera itself, and may include the focal length of the camera, the pixel size, and the like.
  • the outer parameter may characterize the positional relationship of the optical center or optical axis of the camera with respect to the reference coordinate system, for example, may be expressed as [ ⁇ , ⁇ , ⁇ , t 1 , t 2 , t 3 ] T .
  • ⁇ , ⁇ , ⁇ represent the rotation angles of the optical axis of the camera with respect to the X-axis, the Y-axis, and the Z-axis of the three-dimensional reference coordinate system
  • t 1 , t 2 , and t 3 represent the optical center of the camera with respect to the three-dimensional reference coordinates.
  • the value of the internal parameter of the first camera 11 and the second camera 12 has been determined and the value of the external parameter of the first camera 11 is known (for example, the external parameter of the first camera 11 may be set to [0, 90 by default). , 0, 0, 0, 0] T ), the value of the external parameter of the second camera 12 can be determined, hereinafter also referred to as calibrating the external parameters of the second camera 12.
  • the spatial coordinate C(x, y, z) of the pixel P in the camera coordinate system may be obtained based on the values of the internal parameter and the external parameter of the camera, and the specific method may be See any calibration technique well known to those skilled in the art and will not be described in detail herein.
  • the coordinates of the pixel point P in the camera coordinate system C in the world coordinate system W can be obtained, that is, the camera coordinate system is determined.
  • the pixel point P under C is mapped to the mapping point D in the world coordinate system.
  • R represents the rotation matrix
  • T represents the translation vector
  • the world coordinate system W may include three parameters of the radius r, the longitude ⁇ , and the latitude ⁇ of the mapping spherical surface, and the coordinates of the mapping point D corresponding to the pixel P point on the reference mapping surface It can be expressed as (r ⁇ , r ⁇ ). Since the radius of the mapping sphere can be obtained by calibrating the camera lens in advance, it can be regarded as a known amount. Therefore, under the premise that the value of the internal parameter of the camera is fixed, mapping the pixel point P on the original image to the mapping point D on the mapping sphere may mainly depend on the external parameters ⁇ , ⁇ , ⁇ , t 1 , t 2 of the camera. The value of t 3 .
  • the value of the external parameter of the first camera 11 has been determined, that is, [0, 90, 0, 0, 0, 0] T , outside the second camera
  • the values of the parameters [ ⁇ , ⁇ , ⁇ , t 1 , t 2 , t 3 ] T are to be calibrated.
  • mapping point of the pixel point P in the first original image captured by the first camera on the reference mapping surface is the first mapping point D
  • the mapping point on the reference mapping surface is the second mapping point D'
  • the first mapping point D and the second mapping point D' may be referred to as mapping point pairs, and the distance between the pair of mapping points may be the second
  • the values of the external parameters ⁇ , ⁇ , ⁇ , t 1 , t 2 , t 3 of the camera 12 are related.
  • FIG. 2A is a schematic flowchart of a camera calibration method according to an exemplary embodiment of the present disclosure
  • FIG. 2B is a schematic diagram of a first original image and a second original image according to the embodiment shown in FIG. 2A
  • FIG. 2C shows The first feature point P according to the embodiment shown in FIG. 2A is mapped to the first mapping point D on the reference mapping surface; the embodiment can be applied to a binocular camera or a multi-view camera, and the embodiment is combined with FIG. 1 .
  • An exemplary illustration, as shown in FIG. 2A includes the following steps:
  • Step 201 Determine a first feature point set and a second feature point set that match between the first original image captured by the first camera and the second original image captured by the second camera.
  • the first feature point set and the second feature point set may each include N first feature points and N second feature points, and each of the first feature points is and only one of the The second feature points match, and N is an integer greater than or equal to 2.
  • Step 202 Map the first feature point set to the reference mapping surface based on the first set of external parameter values adopted by the first camera to obtain a first mapping point set.
  • the mapping the first feature point set to the reference mapping surface based on the first set of outer parameter values may include: determining, according to the first set of outer parameter values, each first feature point in the first feature point set to map to the reference mapping surface a first mapping point, and the determined first mapping point is formed into the first mapping point set.
  • Step 203 Adjust the second set of external parameter values used by the second camera according to the preset M adjustment manners to obtain M intermediate second set external parameter values.
  • M is an integer greater than or equal to 2
  • each adjustment mode has a preset step size.
  • Step 204 Map the second feature point set to the reference mapping surface based on the intermediate second set of external parameter values obtained by using each adjustment mode, and obtain a second mapping point set corresponding to the adjustment mode.
  • the mapping the second feature point set to the reference mapping surface based on the intermediate second set of outer parameter values may include: determining, according to the intermediate second set of outer parameter values, each second feature point mapping in the second feature point set to the reference mapping A second mapping point on the surface, and the determined second mapping point is formed into a second mapping point set.
  • Step 205 Calculate a distance between the second mapping point set corresponding to each adjustment mode and the first mapping point set, as the distance corresponding to the adjustment mode.
  • the calculating the distance between the second mapping point set and the first mapping point set may include: calculating, by the second mapping point set, the second mapping point and the first mapping point set to match the first mapping point of the first mapping point. The distance between the points, and the sum of the calculated distances is taken as the distance between the second set of mapping points and the first set of mapping points.
  • Step 206 Adjust a second set of external parameter values used by the second iteration of the second camera based on the distance and the step size corresponding to each adjustment mode, to obtain a second set of external parameters used by the second camera in this iteration.
  • the value which is the external parameter of the second camera.
  • Step 207 If it is determined that the iteration termination condition is satisfied based on the distance corresponding to each adjustment mode, the process ends, otherwise return to step 203 to continue the iteration.
  • the first camera 11 and the second camera 12 may have the same type of lens.
  • the lenses of the first camera 11 and the second camera 12 may both be fisheye lenses or both are wide-angle lenses.
  • the first original image 21 and the second original image 22 as shown in Fig. 2B can be obtained.
  • feature points in the first original image 21 and the second original image 22 may be obtained based on any feature point extraction algorithm well known to those skilled in the art, such as may be SIFT feature points.
  • the first feature point set identified in the first original image 21 is (P 1 , P 2 , ..., P N )
  • the second feature point set identified in the second original image 22 is (P' 1 , P' 2 , ..., P' N ).
  • N represents the number of feature points included in the first feature point set and the second feature point set.
  • the first feature point P and the second feature point P' matched between the first feature point set and the second feature point set may be referred to as feature point pairs, and may be labeled as (P 1 -P' 1 , P 2 - P' 2 , ..., P N - P' N ).
  • 2B may represent the first feature points and the second feature points that match in the first original image 21 and the second original image 22.
  • the feature points included in each of the first original image 21 and the second original image 22 can be represented by pixel coordinates.
  • the first camera 11 is set as the referenced camera, and the first set of external parameter values adopted by the first camera 11 can be set to [0, 90, 0, 0, 0. , 0] T .
  • the first set of feature points may be mapped to the reference map based on the first set of outer parameter values to obtain a first set of map points (D 1 , D 2 , . . . , D N ).
  • the first feature point P 1 in the first feature point set corresponds to the first mapping point D 1 in the first mapping point set, and so on, the first feature point P N in the first feature point set corresponds to the first mapping point set.
  • the first mapping point D N .
  • the reference mapping surface may be a mapping spherical surface corresponding to the first camera, and the radius corresponding to the mapping spherical surface may be obtained by calibrating the lens of the first camera.
  • the reference mapping surface may be a plane obtained by plane mapping the mapping spherical surface corresponding to the first camera according to latitude and longitude.
  • each first feature point in the first feature point set can be mapped to the reference mapping surface based on a mathematical conversion relationship between the world coordinate system, the camera coordinate system, and the image coordinate system.
  • the feature points 211 and 221 in the original images 21 and 22 on the left side are mapped by the step 202, and the mapping points 241 and 251 in the mapping plane on the right side are obtained, thereby obtaining a camera shooting.
  • the second camera 12 acts as a camera to be calibrated, and the second set of external parameter values adopted by the second camera 12 can be set to an initial value [0, 90, 0, 0, 0, 0] T , and The initial second group external parameter values are adjusted according to a preset adjustment manner to obtain an intermediate second group external parameter value corresponding to each of the M adjustment modes.
  • external parameters having the same physical meaning can be set to have the same iteration step size.
  • the iteration step size ⁇ s can be obtained experimentally; for the external parameters t 1 , t 2 , t 3 representing the amount of translation, the iteration step size ⁇ m can also be passed The test was obtained.
  • an iteration step size can be set for each outer parameter so that the iterative process of each outer parameter can be adjusted based on different step sizes. In this way, while ensuring that the entire iterative process converges faster, the calibration accuracy of the second set of external parameter values can be maximized.
  • the adjustment manner may include adjusting external parameters of the pair of second cameras one by one, and may increase or decrease the value of the parameter by a preset step size for the parameter. For example, suppose the external parameter value of the second group is [0.11, 90.2, 0.13, 4.1, 5.3, 4.7] T , and the value of the parameter ⁇ is increased by a step size corresponding to the parameter ⁇ to obtain one intermediate second group external parameter.
  • stepwise accumulation of the parameters t 1 , t 2 , and t 3 respectively can also obtain three intermediate and second sets of external parameter values. In this way, there are a total of 6 adjustment methods, and 6 intermediate and second sets of external parameter values can be obtained.
  • the angle parameters ⁇ , ⁇ , and ⁇ may be synchronously added with a corresponding step size of 0.1, to obtain an intermediate second group of external parameter values, and then to the translation parameters t 1 , t 2 , and t 3 .
  • the step size is accumulated synchronously to obtain another intermediate second group external parameter value.
  • the number M of the intermediate external parameter values obtained in each iteration is related to the manner in which the external parameters are adjusted. For example, if the six external parameters are adjusted one by one, six intermediate and second external parameter values can be obtained for each iteration. If the external parameters with the same physical meaning are adjusted synchronously, two can be obtained for each iteration. The second set of external parameter values in the middle.
  • the second feature point set (P' 1 , P' 2 , ..., P' N ) may be mapped based on the intermediate second set of external parameter values corresponding to the six adjustment modes.
  • a second mapping point set corresponding to each of the six adjustment modes is obtained (D' 1 ( ⁇ ), D' 2 ( ⁇ ), ..., D' N ( ⁇ )), (D' 1 ( ⁇ ), D' 2 ( ⁇ ), ..., D' N ( ⁇ )), (D' 1 ( ⁇ ), D' 2 ( ⁇ ), ..., D' N ( ⁇ )), (D' 1 (t 1 ), D' 2 (t 1 ), ..., D' N (t 1 )), (D' 1 (t 2 ), D' 2 (t 2 ), ... D' N (t 2 )), (D' 1 (t 3 ), D' 2 (t 3 ), ..., D' N (t 3 )).
  • the reference mapping surface may be a mapping sphere obtained based on the lens calibration of the first camera.
  • the first original image 21 and the second original image 22 are mapped onto the mapping spherical surface 23, and the first mapping can be obtained respectively.
  • the first feature point 211 on the first original image 21 corresponds to the first mapping point 241 on the mapping spherical surface 23, and the second feature point 221 on the second original image 22 corresponds to the second mapping point 251 on the mapping spherical surface 23.
  • a plurality of different feature points may correspond to a plurality of different mapping points.
  • the rectangular shapes of the first original image 21 and the second original image 22 in FIG. 2C are merely exemplary descriptions, and the first original image 21 and the second original image 22 may be images of any shape, and the present disclosure is This is not a limitation.
  • the reference mapping surface may be a plane obtained by performing plane mapping according to latitude and longitude of the mapping sphere.
  • the distance between the second mapping point set corresponding to each of the six adjustment modes and the first mapping point set is calculated as the distance corresponding to each iteration by the six adjustment methods.
  • the distances of the six adjustment methods for this iteration can be expressed as:
  • f ⁇ (i) represents the distance corresponding to the i-th adjustment of the parameter ⁇
  • f ⁇ (i) represents the distance corresponding to the i-th adjustment of the parameter ⁇
  • f ⁇ (i) represents the i-th of the parameter ⁇ Adjust the corresponding distance
  • the absolute value of the difference between the respective distances of the current iterations and the distances corresponding to the previous iterations may be determined based on the six adjustment methods to determine whether the iteration termination condition is satisfied. For example, when the adjustment mode is to adjust the value of the parameter ⁇ by a preset step size, the distance f ⁇ (i) corresponding to the ith adjustment of the parameter ⁇ can be calculated and the i-1th adjustment corresponding to the parameter ⁇ is performed. The absolute value of the difference between the distances f ⁇ (i-1)
  • the iterative termination condition can be expressed as Equation 7 below:
  • ⁇ 2 represents a second threshold preset for adjusting the parameter ⁇ according to a specific step size
  • ⁇ 3 represents a second threshold preset for adjusting the parameter ⁇ according to a specific step size
  • ⁇ 4 represents a specific step for the specific step
  • the second adjustment threshold is preset by the adjustment parameter ⁇
  • ⁇ 5 represents a second threshold preset for adjusting the parameter t 1 according to a specific step size
  • ⁇ 6 represents that the parameter t 2 is adjusted for a specific step size.
  • the adjustment mode presets a second threshold
  • ⁇ 7 represents a second threshold preset for adjusting the parameter t 3 according to a specific step size.
  • the respective second threshold values ⁇ 2- ⁇ 7 may be the same or different, and may be set according to actual needs.
  • the iterative termination condition may also include the total number of iterations being greater than or equal to a predetermined third threshold.
  • the third threshold is 5, on the premise that the distance corresponding to each of the M adjustment modes determines that the iteration does not need to be terminated, if the total number of current iterations is 4, the step 203 may be continued. However, if the total number of iterations to the current iteration is 5, the iterative process can be terminated and the calibration result for the second camera can be obtained based on step 206. Limiting the iterative process by constraining the total number of iterations improves the efficiency of the calibration camera.
  • the first feature point and the second feature point may be respectively extracted by using the original image captured by the first camera and the second camera, based on the phase
  • the distance between the matched first feature point and the second feature point on the mapping reference surface is iteratively adjusted on the outer parameter value of the second camera.
  • FIG. 3 is a schematic flowchart of a camera calibration method according to another exemplary embodiment of the present disclosure. This embodiment is based on the embodiment shown in FIG. 2A, and how to calibrate the external parameters of the second camera as an example. An exemplary description is made in conjunction with FIG. 1, as shown in FIG. 3, including the following steps:
  • Step 301 For any one of the M adjustment modes, determine an absolute value of the difference between the distance corresponding to the adjustment mode and the distance corresponding to the adjustment mode in the previous iteration, and obtain M adjustment modes. Corresponding difference
  • Step 302 Determine a ratio between a difference between each of the M adjustment modes and a step size of each of the M adjustment modes, and obtain a ratio corresponding to each of the M adjustment modes.
  • Step 303 determining weights corresponding to each of the M adjustment modes
  • Step 304 Adjust, according to the weights and ratios corresponding to the M adjustment modes, the second set of external parameter values used by the second camera in the previous iteration to obtain the second set of external parameter values used by the second camera in this iteration, that is, Calibrate the external parameters of the second camera.
  • step 301 for example, for adjusting the adjustment mode of the parameter ⁇ by the step size ⁇ s, the distance corresponding to the i-th iteration is as shown in the following formula 9-1:
  • D' 1 ( ⁇ ), D' 1 ( ⁇ ), ..., D' 1 ( ⁇ ) represents the mapping of the second feature point to the reference map based on the intermediate second set of outer parameter values obtained by the i-th adjustment parameter ⁇ The second mapping point on the face.
  • D' 1 ( ⁇ , ⁇ s), D' 1 ( ⁇ , ⁇ s), ..., D' 1 ( ⁇ , ⁇ s) represents the intermediate second set of external parameter values obtained based on the i-1th adjustment parameter ⁇
  • the second feature point is mapped to the second mapping point on the reference mapping surface.
  • the adjustment method of adjusting the parameter ⁇ by the step size ⁇ s can be obtained.
  • the difference between the corresponding distance in the current iteration and the corresponding distance in the previous iteration is f ⁇ (i)-f ⁇ (i-1) ).
  • the available difference between the parameter ⁇ , the parameter ⁇ , the parameter t 1 , the parameter t 2 , and the parameter t 3 can be obtained by the difference f ⁇ (i)-f ⁇ (i-1), f ⁇ (i )-f ⁇ (i-1),
  • step 302 for example, for adjusting the adjustment of the parameter ⁇ by the step size ⁇ s, the ratio between the difference value and the step size corresponding to the adjustment mode may be calculated as shown in the following formula 10-1:
  • the calculated ratios corresponding to the specific step adjustment parameter ⁇ , the parameter ⁇ , the parameter t 1 , the parameter t 2 , and the parameter t 3 can be calculated as shown in the following formulas 10-2 to 10-6:
  • step sizes ⁇ s and ⁇ m of different adjustment modes may be the same or different, and the specific values of the step size are not limited in the present disclosure.
  • the weights of the different adjustment modes may be the same or different.
  • the specific value of the weight corresponding to each adjustment mode is not limited.
  • step 304 the outer parameter value of the second camera can be calculated as follows:
  • k denotes a weight vector
  • the number of weights is the same as the number of ratios.
  • the speed of the iterative process can be adjusted by the weight, and the iterative process can be flexibly controlled according to the specific situation of the camera to ensure that the external parameters of the finally calibrated second camera can make the captured image have a better visual effect.
  • FIG. 4A is a schematic flowchart of a camera calibration method according to another exemplary embodiment of the present disclosure
  • FIG. 4B is a schematic diagram of a correspondence between a reference mapping surface and an imaging plane in the embodiment shown in FIG. 4A
  • FIG. 2A On the basis of the embodiment shown in FIG. 2A, an example of how to obtain a second mapping table based on the external parameters of the calibrated second camera is exemplified in conjunction with FIG. 2C.
  • FIG. 4A the following steps are included:
  • Step 401 Determine, according to a preset sampling interval, a plurality of second sampling points on the reference mapping surface;
  • Step 402 Determine, according to the external parameter value of the calibrated second camera, a second pixel point corresponding to each of the plurality of second sampling points on the imaging plane of the second camera;
  • Step 403 Record coordinate correspondence between the plurality of second sampling points and the plurality of second pixel points in the second mapping table of the second camera.
  • the preset sampling interval may be determined according to the accuracy requirement for image processing.
  • the preset sampling interval may be set larger, and when the precision requirement is high, The sampling interval can be set smaller.
  • the size is 1000*1000, and the preset sampling interval can be 50 pixels, and the number of the second sampling points is 20*20.
  • step 402 for any one of the sampling points 411 on the reference mapping surface 41, based on the external parameters after the second camera calibration and the internal parameters of the second camera, the pixel mapping on the captured image may be referenced in the reference map.
  • the opposite mapping process on the surface results in the corresponding pixel point 421 of the sampling point 411 on the imaging plane (also referred to as image plane) 42 of the second camera, correspondingly, for all sampling points on the reference mapping surface 41.
  • the corresponding pixel points on the imaging plane 42 can be calculated.
  • a plurality of second sampling points P' 1 , P' 2 , ..., P' N
  • a plurality of second pixel points D" 1 , D" 2 , ...,
  • the coordinate correspondence relationship between D′′ N is recorded in the second mapping table.
  • N is 400
  • the reference mapping surface and the imaging plane of the second camera have 400 coordinate positions, and thus may be based on the 400
  • the correspondence between the coordinate positions directly projects the image of the corresponding position on the image acquired by the second camera onto the reference mapping surface, thereby saving the coordinate calculation by the coordinate conversion of the world coordinate system, the camera coordinate system and the image coordinate system.
  • the mapping points corresponding to the feature points simplify the calculation process.
  • a plurality of first sampling points may be determined according to a preset sampling interval on the first mapping plane; and the plurality of first parameters are determined based on the first set of external parameter values adopted by the first camera.
  • a sampling point corresponding to the plurality of first pixel points on the imaging plane of the first camera; recording a coordinate correspondence relationship between the plurality of first sampling points and the plurality of first pixel points in the first mapping table of the first camera in.
  • the first mapping table and the second mapping table may be used to record the coordinate correspondence between the reference mapping surface and the imaging plane of the camera.
  • the panoramic image of the binocular or multi-head camera needs to be spliced, The original image captured by the camera is directly mapped to the reference mapping surface, and the calculation process of the image processing is simplified without repeatedly calculating the mapping relationship between the feature points and the reference mapping surface.
  • FIG. 5A is a schematic diagram of a projection image obtained by projecting an original image captured by a second camera onto a reference mapping surface based on an initial value of an external parameter of the second camera
  • FIG. 5B is an external parameter value calibrated by the second camera based on the method of the present disclosure.
  • a schematic diagram of a projected image obtained by projecting an original image taken by a second camera onto a reference mapping surface; for the second original image 22 shown in FIG. 2B, based on an external parameter [ ⁇ , ⁇ , ⁇ , t 1 of the second camera 12, t 2, t 3] is an initial value of T [0,90,0,0,0,0] T, for the second original image 22 mapped to a mapping surface of the reference, to obtain the reference image.
  • the image of the first original image 21 captured by the first camera 11 and the reference image are superimposed to obtain the image shown in FIG. 5A. It is apparent from the image shown in FIG. 5A that the distant building is compared with the nearby small tree. Blurring, that is, there is parallax, and the image is not effective.
  • the images 21 are superimposed to obtain the image shown in Fig. 5B. It is apparent from the image shown in Fig. 5B that the distant buildings are aligned in position, indicating that the parallax is reduced.
  • FIG. 6 is a schematic structural diagram of a camera calibration apparatus according to an exemplary embodiment of the present disclosure.
  • the camera calibration apparatus may include: a first determining module 61, a first mapping module 62, and a parameter adjustment module 63.
  • a first determining module 61 configured to determine a first feature point set and a second feature point set that match between the first original image captured by the first camera and the second original image captured by the second camera;
  • the first mapping module 62 is configured to map, according to the first set of external parameter values of the first camera, the first feature point set determined by the first determining module 61 to the reference mapping surface, to obtain a first mapping point set;
  • the parameter adjustment module 63 is configured to adjust the second set of external parameter values used by the second camera according to the preset M adjustment manners to obtain M intermediate second set external parameter values, where M is greater than or equal to 2.
  • M is greater than or equal to 2.
  • the second mapping module 64 is configured to map the second feature point set determined by the first determining module 61 to the reference mapping surface based on the intermediate second set of external parameter values obtained by the parameter adjusting module 63 by using each of the adjusting manners, to obtain a second set of mapping points corresponding to the adjustment manner;
  • the first calculation module 65 is configured to calculate a distance between the second mapping point set corresponding to each of the adjustment modes obtained by the second mapping module 64 and the first mapping point set obtained by the first mapping module 62, as the adjustment The distance corresponding to the method;
  • the external parameter calibration module 66 is configured to adjust the second set of external parameter values used by the second camera based on the distance and the step size corresponding to each adjustment mode, and determine that the iterative termination condition is satisfied based on the distance corresponding to each adjustment mode. The calibration of the external parameter value of the second camera is completed. Otherwise, the parameter adjustment module 63 continues to adjust the second set of external parameter values adopted by the second camera in the M adjustment manner to continue the iteration.
  • FIG. 7 is a schematic structural diagram of a camera calibration apparatus according to another exemplary embodiment of the present disclosure.
  • the external parameter calibration module 66 includes:
  • the first determining unit 661 is configured to determine, according to any one of the M adjustment modes, a difference between a distance corresponding to the adjustment mode and a distance corresponding to the adjustment mode in the previous iteration, to obtain an M.
  • the second determining unit 662 is configured to determine a ratio between the difference between the M adjustment manners determined by the first determining unit 661 and the step size, and obtain a ratio corresponding to each of the M adjustment modes;
  • the calibration unit 663 is configured to adjust a weight corresponding to each of the M adjustment modes and the ratio, and adjust a second set of external parameter values used by the second camera in the previous iteration to obtain the second group of the second camera used in the current iteration. Parameter value.
  • the M adjustment manners include: adjusting a value of any one of the second set of external parameter values adopted by the second camera by a preset step size, and ensuring other parameters of the second set of external parameter values. The value remains unchanged, and the intermediate second set of external parameter values corresponding to the parameter adjustment is obtained; or the value of the parameter having the same physical meaning among the second set of external parameter values used by the second camera is adjusted by a preset step size, and Ensure that the values of the other parameters in the second set of external parameter values remain unchanged, and obtain the intermediate second set of external parameter values corresponding to the set of parameter adjustments.
  • the iterative termination condition includes any one or more of the following: a distance corresponding to each of the M adjustment modes obtained by the current iteration and a distance corresponding to the M adjustment modes obtained in the previous iteration.
  • the sum of the absolute values of the difference is less than or equal to the first preset threshold; the distance corresponding to each of the adjustment modes is less than or equal to the second preset threshold corresponding to the adjustment mode.
  • the iterative termination condition further includes: the total number of iterations of the second set of external parameter values employed by the second camera is greater than or equal to a third predetermined threshold.
  • the reference mapping surface is a mapping spherical surface obtained by calibrating the lens of the first camera; or the reference mapping surface is obtained by performing plane mapping on the mapping spherical surface obtained by calibrating the lens based on the first camera according to latitude and longitude.
  • the camera calibration device may further include:
  • a second determining module 67 configured to determine, according to a preset sampling interval, a plurality of first sampling points on the reference mapping surface
  • the third determining module 68 is configured to determine, according to the first set of external parameter values adopted by the first camera, the plurality of first sampling points determined by the second determining module 67 corresponding to the plurality of first images on the imaging plane where the first camera is located pixel;
  • the first recording module 69 is configured to record the coordinate correspondence between the plurality of first sampling points determined by the second determining module 67 and the plurality of first pixel points determined by the third determining module 68 in the first camera In the mapping table.
  • the camera calibration device may further include:
  • a fourth determining module 70 configured to determine, according to a preset sampling interval, a plurality of second sampling points on the reference mapping surface
  • a fifth determining module 71 configured to determine, according to the second set of external parameter values after the calibration of the second camera, the plurality of second sampling points determined by the fourth determining module 70 are corresponding to the plurality of imaging planes where the second camera is located Second pixel
  • the second recording module 72 is configured to record the coordinate correspondence between the plurality of second sampling points determined by the fourth determining module 70 and the plurality of second pixel point coordinates determined by the fifth determining module 71 in the second camera In the second mapping table.
  • FIG. 9 is a schematic structural diagram of an electronic device according to an exemplary embodiment of the present disclosure; corresponding to the camera calibration method of FIGS. 2A-4A described above, as shown in FIG. 9, at the hardware level, the electronic device includes a processor. 901, internal bus 902, network interface 903, memory 904, and non-volatile storage medium 905, of course, may also include hardware required for other services.
  • the processor reads the corresponding machine executable instructions from the non-volatile storage medium into the memory 904 and then operates to form the camera calibration apparatus provided by the embodiment shown in any of Figures 6-8 above at a logical level.
  • the present disclosure does not exclude other implementations, such as a logic device or a combination of software and hardware, etc., that is, the execution body of the above processing flow is not limited to each logical unit, and may be Hardware or logic device.

Landscapes

  • Engineering & Computer Science (AREA)
  • Multimedia (AREA)
  • Signal Processing (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Theoretical Computer Science (AREA)
  • Studio Devices (AREA)

Abstract

L'invention concerne un procédé, et un dispositif électronique Lorsqu'une première caméra est utilisée comme référence, et qu'une seconde caméra doit être étalonnée, le procédé consiste à : déterminer, sur la base de paramètres externes de la première caméra, un premier ensemble de points caractéristiques sur une image originale photographiée par la première caméra étant dans un premier ensemble de points de mappage sur un plan de mappage de référence; régler, selon M procédés de réglage prédéfinis, des paramètres externes de la seconde caméra pour obtenir M valeurs de paramètres externes intermédiaires; déterminer, sur la base de la valeur de paramètre externe intermédiaire correspondant à chaque procédé de réglage, un second ensemble de points caractéristiques, mis en correspondance avec le premier ensemble de points caractéristiques, sur une image originale photographiée par la seconde caméra se trouvant dans un second ensemble de points de mappage sur un plan de mappage de référence, et calculer la distance entre le second ensemble de points de mappage et le premier ensemble de points de mappage et l'utiliser comme la distance correspondant au procédé de réglage; et régler, sur la base de la distance correspondant à chaque procédé de réglage et à une longueur de pas, des paramètres externes de la seconde caméra.
PCT/CN2018/077341 2017-02-27 2018-02-27 Étalonnage de caméras WO2018153374A1 (fr)

Priority Applications (1)

Application Number Priority Date Filing Date Title
US16/034,083 US10586352B2 (en) 2017-02-27 2018-07-12 Camera calibration

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN201710108686.X 2017-02-27
CN201710108686.XA CN106875451B (zh) 2017-02-27 2017-02-27 相机标定方法、装置及电子设备

Related Child Applications (1)

Application Number Title Priority Date Filing Date
US16/034,083 Continuation-In-Part US10586352B2 (en) 2017-02-27 2018-07-12 Camera calibration

Publications (1)

Publication Number Publication Date
WO2018153374A1 true WO2018153374A1 (fr) 2018-08-30

Family

ID=59167906

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2018/077341 WO2018153374A1 (fr) 2017-02-27 2018-02-27 Étalonnage de caméras

Country Status (3)

Country Link
US (1) US10586352B2 (fr)
CN (1) CN106875451B (fr)
WO (1) WO2018153374A1 (fr)

Cited By (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN112102414A (zh) * 2020-08-27 2020-12-18 江苏师范大学 基于改进遗传算法和神经网络的双目远心镜头标定方法
CN112308931A (zh) * 2020-11-02 2021-02-02 深圳市泰沃德技术有限公司 相机的标定方法、装置、计算机设备和存储介质
CN112562013A (zh) * 2020-12-25 2021-03-26 深圳看到科技有限公司 多镜头相机标定方法、装置及存储介质
CN113628283A (zh) * 2021-08-10 2021-11-09 地平线征程(杭州)人工智能科技有限公司 摄像装置的参数标定方法、装置、介质以及电子设备

Families Citing this family (16)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN106875451B (zh) * 2017-02-27 2020-09-08 安徽华米智能科技有限公司 相机标定方法、装置及电子设备
CN110363818A (zh) * 2018-04-11 2019-10-22 杭州海康机器人技术有限公司 双目视觉系统的异常检测方法及装置
CN110689580B (zh) * 2018-07-05 2022-04-15 杭州海康机器人技术有限公司 多相机标定方法及装置
CN109300163B (zh) * 2018-09-14 2021-09-24 高新兴科技集团股份有限公司 室内全景相机的空间标定方法、存储介质及电子设备
CN109410283B (zh) * 2018-09-14 2021-09-24 高新兴科技集团股份有限公司 室内全景相机的空间标定装置和具有其的定位装置
CN109754432B (zh) * 2018-12-27 2020-09-22 深圳市瑞立视多媒体科技有限公司 一种相机自动标定方法及光学动作捕捉系统
CN109859254B (zh) * 2019-02-28 2022-08-30 北京百度网讯科技有限公司 自动驾驶中用于发送信息的方法和装置
CN113643378B (zh) * 2019-09-30 2023-06-09 深圳市瑞立视多媒体科技有限公司 多相机环境中主动式刚体的位姿定位方法及相关设备
CN110766761B (zh) * 2019-10-21 2023-09-26 北京百度网讯科技有限公司 用于相机标定的方法、装置、设备和存储介质
CN111862236B (zh) * 2020-07-22 2023-10-20 中国科学院上海微系统与信息技术研究所 定焦双目相机自标定方法及系统
CN114170302A (zh) * 2020-08-20 2022-03-11 北京达佳互联信息技术有限公司 相机外参标定方法、装置、电子设备及存储介质
CN112489240B (zh) * 2020-11-09 2021-08-13 上海汉时信息科技有限公司 一种商品陈列巡检方法、巡检机器人以及存储介质
CN112541951A (zh) * 2020-11-13 2021-03-23 国网浙江省电力有限公司舟山供电公司 预防船舶钩断跨海架空电力线路的监测系统及其监测方法
CN112489111B (zh) * 2020-11-25 2024-01-30 深圳地平线机器人科技有限公司 相机外参标定方法、装置及相机外参标定系统
CN114067002B (zh) * 2022-01-17 2023-02-17 江苏中云筑智慧运维研究院有限公司 一种双目相机外部参数确定方法和系统
CN116993830A (zh) * 2023-08-17 2023-11-03 广州赋安数字科技有限公司 一种动态摄像头坐标映射的自动校准方法

Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN101826206A (zh) * 2010-03-31 2010-09-08 北京交通大学 一种相机自定标的方法
CN102436660A (zh) * 2011-11-08 2012-05-02 北京新岸线网络技术有限公司 3d相机图像自动校正方法及装置
CN104019799A (zh) * 2014-05-23 2014-09-03 北京信息科技大学 一种利用局部参数优化计算基础矩阵的相对定向方法
CN105678740A (zh) * 2015-12-30 2016-06-15 完美幻境(北京)科技有限公司 一种相机几何标定处理方法及装置
WO2017008516A1 (fr) * 2015-07-15 2017-01-19 华为技术有限公司 Système, dispositif et appareil de calcul de position relative à deux caméras
CN106875451A (zh) * 2017-02-27 2017-06-20 安徽华米信息科技有限公司 相机标定方法、装置及电子设备

Family Cites Families (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US5598515A (en) * 1994-01-10 1997-01-28 Gen Tech Corp. System and method for reconstructing surface elements of solid objects in a three-dimensional scene from a plurality of two dimensional images of the scene
JP5285619B2 (ja) * 2006-12-18 2013-09-11 コーニンクレッカ フィリップス エレクトロニクス エヌ ヴィ カメラシステムのキャリブレーション
JP2008187564A (ja) * 2007-01-31 2008-08-14 Sanyo Electric Co Ltd カメラ校正装置及び方法並びに車両
US9915857B2 (en) * 2013-12-09 2018-03-13 Geo Semiconductor Inc. System and method for automated test-pattern-free projection calibration
US9532031B1 (en) * 2014-04-08 2016-12-27 The United States Of America As Represented By The Secretary Of The Navy Method for extrinsic camera calibration using a laser beam
EP2942951A1 (fr) * 2014-05-06 2015-11-11 Application Solutions (Electronics and Vision) Limited Calibrage d'image
CN104182982B (zh) * 2014-08-27 2017-02-15 大连理工大学 双目立体视觉摄像机标定参数的整体优化方法

Patent Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN101826206A (zh) * 2010-03-31 2010-09-08 北京交通大学 一种相机自定标的方法
CN102436660A (zh) * 2011-11-08 2012-05-02 北京新岸线网络技术有限公司 3d相机图像自动校正方法及装置
CN104019799A (zh) * 2014-05-23 2014-09-03 北京信息科技大学 一种利用局部参数优化计算基础矩阵的相对定向方法
WO2017008516A1 (fr) * 2015-07-15 2017-01-19 华为技术有限公司 Système, dispositif et appareil de calcul de position relative à deux caméras
CN105678740A (zh) * 2015-12-30 2016-06-15 完美幻境(北京)科技有限公司 一种相机几何标定处理方法及装置
CN106875451A (zh) * 2017-02-27 2017-06-20 安徽华米信息科技有限公司 相机标定方法、装置及电子设备

Cited By (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN112102414A (zh) * 2020-08-27 2020-12-18 江苏师范大学 基于改进遗传算法和神经网络的双目远心镜头标定方法
CN112308931A (zh) * 2020-11-02 2021-02-02 深圳市泰沃德技术有限公司 相机的标定方法、装置、计算机设备和存储介质
CN112562013A (zh) * 2020-12-25 2021-03-26 深圳看到科技有限公司 多镜头相机标定方法、装置及存储介质
CN112562013B (zh) * 2020-12-25 2022-11-01 深圳看到科技有限公司 多镜头相机标定方法、装置及存储介质
CN113628283A (zh) * 2021-08-10 2021-11-09 地平线征程(杭州)人工智能科技有限公司 摄像装置的参数标定方法、装置、介质以及电子设备
CN113628283B (zh) * 2021-08-10 2024-05-17 地平线征程(杭州)人工智能科技有限公司 摄像装置的参数标定方法、装置、介质以及电子设备

Also Published As

Publication number Publication date
CN106875451B (zh) 2020-09-08
CN106875451A (zh) 2017-06-20
US10586352B2 (en) 2020-03-10
US20180322658A1 (en) 2018-11-08

Similar Documents

Publication Publication Date Title
WO2018153374A1 (fr) Étalonnage de caméras
WO2021036353A1 (fr) Système et procédé de modélisation 3d basée sur la photographie, et appareil et procédé de modélisation 3d automatique
KR101666959B1 (ko) 카메라로부터 획득한 영상에 대한 자동보정기능을 구비한 영상처리장치 및 그 방법
WO2018068678A1 (fr) Procédé et dispositif pour déterminer un paramètre externe d'un appareil stéréoscopique
WO2020014909A1 (fr) Procédé et dispositif de photographie, et véhicule aérien sans pilote
WO2017076106A1 (fr) Procédé et dispositif d'assemblage d'images
WO2018068719A1 (fr) Procédé et appareil de collage d'image
WO2021196548A1 (fr) Procédé, appareil, et système de détermination de la distance
JP2017112602A (ja) パノラマ魚眼カメラの画像較正、スティッチ、および深さ再構成方法、ならびにそのシステム
WO2017020150A1 (fr) Procédé de traitement d'image, dispositif et appareil photographique
CN108257183A (zh) 一种相机镜头光轴校准方法和装置
WO2018153313A1 (fr) Caméra stéréoscopique et procédé d'acquisition de hauteur associé et système d'acquisition de hauteur
US11282232B2 (en) Camera calibration using depth data
CN111127524A (zh) 一种轨迹跟踪与三维重建方法、系统及装置
WO2021189804A1 (fr) Procédé et dispositif de rectification d'image, et système électronique
WO2019232793A1 (fr) Procédé d'étalonnage de deux caméras, dispositif électronique et support de stockage lisible par ordinateur
TW201635242A (zh) 室內二維平面圖的生成方法、裝置和系統
WO2021104308A1 (fr) Procédé de mesure de profondeur panoramique, caméra à œil de poisson à quatre yeux et caméra fisheye binoculaire
US8509522B2 (en) Camera translation using rotation from device
WO2023236508A1 (fr) Procédé et système d'assemblage d'images basés sur une caméra ayant un réseau d'un milliard de pixels
KR20060056050A (ko) 자동화된 360°파노라마 이미지 생성 방법
CN113436267B (zh) 视觉惯导标定方法、装置、计算机设备和存储介质
Corke et al. Image Formation
CN111353945B (zh) 鱼眼图像校正方法、装置及存储介质
CN109389645B (zh) 相机自校准方法、系统、相机、机器人及云端服务器

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 18756676

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 18756676

Country of ref document: EP

Kind code of ref document: A1