WO2020063708A1 - 鱼眼摄像机内参标定方法、装置和系统、标定装置控制器以及标定器具 - Google Patents
鱼眼摄像机内参标定方法、装置和系统、标定装置控制器以及标定器具 Download PDFInfo
- Publication number
- WO2020063708A1 WO2020063708A1 PCT/CN2019/108017 CN2019108017W WO2020063708A1 WO 2020063708 A1 WO2020063708 A1 WO 2020063708A1 CN 2019108017 W CN2019108017 W CN 2019108017W WO 2020063708 A1 WO2020063708 A1 WO 2020063708A1
- Authority
- WO
- WIPO (PCT)
- Prior art keywords
- camera
- marker
- target
- calibration
- internal
- Prior art date
Links
- 238000000034 method Methods 0.000 title claims abstract description 43
- 239000003550 marker Substances 0.000 claims abstract description 171
- 238000004590 computer program Methods 0.000 claims description 11
- 238000012545 processing Methods 0.000 claims description 9
- 238000013507 mapping Methods 0.000 claims description 7
- 230000003287 optical effect Effects 0.000 claims description 7
- 230000000007 visual effect Effects 0.000 claims description 3
- 238000010586 diagram Methods 0.000 description 10
- 238000003384 imaging method Methods 0.000 description 9
- 230000008569 process Effects 0.000 description 6
- 239000011159 matrix material Substances 0.000 description 3
- 238000012937 correction Methods 0.000 description 2
- 238000005516 engineering process Methods 0.000 description 2
- 238000004364 calculation method Methods 0.000 description 1
- 238000011550 data transformation method Methods 0.000 description 1
- 230000000694 effects Effects 0.000 description 1
- 230000006870 function Effects 0.000 description 1
- 230000006872 improvement Effects 0.000 description 1
- 238000009434 installation Methods 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 238000012544 monitoring process Methods 0.000 description 1
- 239000004065 semiconductor Substances 0.000 description 1
- 238000013519 translation Methods 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T3/00—Geometric image transformation in the plane of the image
-
- G06T3/047—
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/70—Determining position or orientation of objects or cameras
- G06T7/73—Determining position or orientation of objects or cameras using feature-based methods
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/80—Analysis of captured images to determine intrinsic or extrinsic camera parameters, i.e. camera calibration
Definitions
- the present application relates to the technical field of cameras, and in particular, to internal calibration of fisheye cameras.
- the process of obtaining camera geometric model parameters is called camera calibration. It is an essential step in the field of image processing and computer vision to extract three-dimensional spatial information from two-dimensional images. It is widely used in three-dimensional reconstruction, navigation, visual monitoring and other fields. . Therefore, how to calibrate the camera to obtain the camera geometric model parameters is very important.
- Camera calibration involves processing the image under a certain camera model, and using a series of data transformation and calculation methods to obtain the camera geometric model parameters.
- the traditional calibration method multiple images of the same marker at different positions are obtained, and then internal parameters of the camera are obtained based on the multiple images. In this way, it is necessary to manually place the markers at different positions, and then collect multiple images of the markers at different positions, which is time-consuming and has a low calibration efficiency.
- the present application provides a method, a device, and a system for calibrating internal parameters of a fisheye camera, a calibration device controller, and a calibration instrument to improve calibration efficiency.
- a first aspect of the present application provides a method for calibrating a fisheye camera internal reference, the method including:
- the internal parameters of the camera are calibrated according to the pixel coordinates of the designated point on each marker in the target image and the position information in the physical coordinate system of the marker.
- a second aspect of the present application provides a camera internal reference calibration device, wherein the calibration device includes a plurality of calibration frames, each of which is fixed with a plurality of markers, and the internal reference calibration device includes:
- a determining module configured to determine a target placement angle of each marker relative to the camera to be calibrated to adjust a placement angle of each marker relative to the camera to the target placement angle, wherein the calibration instrument Covering the entire field of view of the camera, and the respective markers are evenly distributed in the height direction and the width direction of the field of view of the camera;
- An acquisition module configured to acquire a target image collected by the camera under the current field of view, the target image including images of the respective markers at respective target positions, and the target position of each marker is related to the marker A physical position where the target placement angle of the object matches;
- a processing module is configured to calibrate the internal parameters of the camera according to the pixel coordinates of the designated point on each marker in the target image and the position information in the physical coordinate system of the marker.
- a third aspect of the present application provides a computer storage medium on which a computer program is stored, and when the computer program is executed by a processor, the method provided by the first aspect of the present application is implemented.
- a fourth aspect of the present application provides a calibration device controller including a memory, a processor, and a computer program stored on the memory and executable by the processor.
- the processor implements the application when the computer program is executed.
- the first aspect provides the method.
- a fifth aspect of the present application provides a fisheye camera internal reference calibration system.
- the system includes a calibration device and a camera.
- the camera includes a calibration device controller.
- the system includes a calibration device, a camera, and a calibration device controller. among them,
- the calibration apparatus includes a plurality of calibration frames, and each of the calibration frames is fixed with a plurality of markers;
- the camera is used to collect a target image containing the respective markers in the current field of view, the target images include the images of the respective markers at the respective target positions, and the target position of each marker is related to The physical position of the target where the marker is placed matches the angle;
- the calibration device controller is used for:
- a sixth aspect of the present application provides a calibration instrument for internal reference calibration of a fish-eye camera.
- the calibration instrument includes a plurality of calibration frames, and each of the calibration frames is fixed with a plurality of markers.
- the placement angle of each marker relative to the camera is adjusted to its respective target placement angle, the calibration instrument covers the entire field of view of the camera, and each marker is in the field of view of the camera Evenly distributed in the height and width directions.
- camera calibration can be completed based on one image, and the calibration efficiency is improved.
- the target image collected by the camera is clear, and the camera is calibrated based on the clear target image, which can improve the obtained internal parameters. Precision.
- FIG. 1 is a flowchart of a fisheye camera internal reference calibration method according to an exemplary embodiment of the present application
- FIG. 2 is a schematic diagram of a calibration instrument according to an exemplary embodiment of the present application.
- FIG. 3 is an imaging principle diagram of a fisheye camera according to an exemplary embodiment of the present application.
- FIG. 4 is a flowchart of a method for calibrating a fisheye camera internal reference according to an exemplary embodiment of the present application
- FIG. 5 is a schematic diagram of re-projection according to an exemplary embodiment of the present application.
- FIG. 6 is a schematic structural diagram of a calibration device controller in which a fisheye camera internal reference calibration device is located according to an exemplary embodiment of the present application;
- FIG. 7 is a schematic structural diagram of a fisheye camera internal reference calibration device according to an exemplary embodiment of the present application.
- first, second, third, etc. may be used in this application to describe various information, such information should not be limited by these terms. These terms are only used to distinguish the same type of information from each other.
- first information may also be referred to as the second information, and similarly, the second information may also be referred to as the first information.
- word “if” as used herein can be interpreted as “at” or “when” or “responding to”.
- the present application provides a method, a device, and a system for calibrating internal parameters of a fisheye camera, a calibration device controller, and a calibration instrument to improve calibration efficiency.
- the method and device for internal reference calibration of a fisheye camera provided in the present application can be applied to a calibration device controller.
- the calibration device controller can be integrated into a fisheye camera.
- FIG. 1 is a flowchart of a fisheye camera internal reference calibration method according to an exemplary embodiment of the present application.
- the internal calibration method for a fisheye camera may include:
- S101 Determine the target placement angle of each marker with respect to the camera to be calibrated, so as to adjust the placement angle of each marker with respect to the camera to the target placement angle, wherein a calibration instrument composed of each marker covers the above The entire field of view of the camera, and each of the markers is uniformly distributed in the height direction and the width direction of the field of view of the camera.
- FIG. 2 is a schematic diagram of a calibration instrument according to an exemplary embodiment of the present application.
- the calibration apparatus may include a plurality of calibration frames, each of which has a plurality of markers fixed on it.
- the position of the calibration frame can be flexibly set, and the installation angle and height of the markers on the calibration frame can be adjusted.
- the appliance covers the entire field of view of the camera.
- each marker is uniformly distributed in the height direction and the width direction of the field of view of the camera.
- the plurality of markers includes 15 markers disposed on different calibration racks.
- the marker may be a checkerboard calibration board.
- step S101 may include:
- the internal parameters of the camera are estimated according to a reference image collected by the camera, the reference image includes an image of the respective markers at their respective initial positions, and the initial position of each marker is related to the marker The physical position of the object's initial placement angle matches.
- the reference image is an image that was acquired before the placement angle of the marker relative to the camera was adjusted.
- FIG. 3 is an imaging principle diagram of a fisheye camera according to an exemplary embodiment of the present application.
- O c is the origin of the camera coordinate system
- Xc, Yc, and Zc are the coordinate axes of the camera coordinate system
- P is a three-dimensional point in space.
- point P is the center point of the marker
- P1 is the projection point of the point on the camera coordinate system XcOcYc plane
- ⁇ in FIG. 3 is the first included angle
- ⁇ in FIG. 3 is the second included angle .
- first angle and the second angle can be calculated according to the following formula:
- ⁇ d ⁇ (1 + k 1 ⁇ 2 + k 2 ⁇ 4 + k 3 ⁇ 6 + k 4 ⁇ 8 )
- x w and y w are the normalized distortion coordinates
- x distort and y distort are the pixel coordinates of the center point in the reference image
- c x0 and c y0 are the estimated initial internal reference principal point coordinates
- f x0 and f y0 is the estimated initial internal reference focal length
- k 1 -k 4 is the internal reference distortion coefficient
- ⁇ is the first included angle
- ⁇ is the second included angle.
- ⁇ can be calculated by Gauss-Newton iteration. It should be noted that according to the geometric relationship, ⁇ is equal to that in FIG. 3
- the calibrator can adjust the placement angle of the marker relative to the camera based on the first and second included angles, so that the connection between the center point of the marker and the origin of the camera coordinate system and the positive direction of the camera coordinate system Z axis
- the included angle is equal to the first included angle
- the included angle between the projection point of the center point of the marker on the camera coordinate system XOY plane and the origin of the camera coordinate system and the positive direction of the camera coordinate system X axis is equal to the Two angles.
- the acquired target image can be ensured.
- the imaging is clear, and then the camera calibration based on the clear target image can improve the accuracy of the internal parameters obtained.
- the camera is controlled to collect the target image, and then the target image collected by the camera is obtained.
- the internal reference refers to parameters related to the characteristics of the camera itself, including the focal length of the camera, the coordinates of the principal point, and the distortion coefficient.
- the internal parameters include the main point coordinates (2), the focal length (2), and the distortion coefficient (4), a total of 8 parameters.
- step S103 may include:
- an initial value of an internal parameter is used as a target reference internal parameter
- an initial value of an external parameter is used as a target reference external parameter.
- step S301 designated points on each marker in the target image may be detected, and pixel coordinates of the designated points in the target image may be obtained.
- the designated point on the marker may be set according to actual needs.
- the designated point on the marker may be a corner point of the marker.
- the checkerboard-like template operator can be used to convolve with the target image.
- the energy voting method is used to calculate the similarity checkerboard position.
- the region growth and subpixelized checkerboard corner points are used to extract Each checkerboard corner point in the target image is detected. After the corner point is detected, the pixel coordinates of the corner point in the target image can be obtained.
- the physical coordinate system of the marker can use the upper left corner of the marker as the coordinate origin, the width direction of the marker is the X axis (the side away from the upper left corner is the positive direction), and the height direction of the marker is the Y axis (away from the upper left corner) Side is positive). In this way, for any designated point on a marker, the position information of the designated point in the physical coordinate system of the marker can be determined in advance.
- each marker Based on the mapping relationship between the target reference internal reference and target reference external reference, and the pixel coordinates of the designated point on each marker in the target image, based on the mapping relationship between the pre-built image coordinate system and the marker physical coordinate system, each marker is The specified points on the projection are projected from the target image to the marker physical coordinate system, and projection position information of the specified points in the marker physical coordinate system is obtained.
- O c is the origin of the camera coordinate system
- Xc, Yc, and Zc are the coordinate axes of the camera coordinate system
- o is the origin of the image coordinate system
- x and y are the coordinate axes of the image coordinate system
- a three-dimensional point P whose imaging point under the fisheye camera is p, and its imaging point under the pinhole camera is q
- ⁇ is the angle between the incident light from point P and the optical axis of the camera
- r is The radial distance from the fisheye camera imaging point p to the origin of the image coordinate system
- Is the angle between the radial distance r and the positive direction of the x-axis of the image coordinate system
- f is the focal length of the fisheye camera.
- ⁇ d ⁇ (1 + k 1 ⁇ 2 + k 2 ⁇ 4 + k 3 ⁇ 6 + k 4 ⁇ 8 ) (9)
- X, Y, and Z are the position information of a three-dimensional point in the space under the physical coordinate system of the marker
- R 1 and T 1 are the amount of rotation and translation of the physical coordinate system of the marker to the camera coordinate system, that is, External parameters of the camera
- k 1 -k 4 are the internal parameter distortion coefficients
- c x and c y are the coordinates of the main points of the internal parameters
- f x and f y are the internal focal lengths
- u and v are the imaging points corresponding to the three-dimensional points in the image coordinate system Pixel coordinates.
- formulas (13)-(1) can be used to calculate the specified point obtained by projecting the specified point from the target image to the physical coordinate system of the marker in the physical coordinate system of the marker.
- Projection position information That is, according to formulas (13)-(2), the target image can be inferred to the distortion correction map (that is, Xc, Yc, and Zc are obtained according to u and v), and then the specified point is calculated from the distortion correction map according to formula (1)
- Projection position information in the physical physical coordinate system that is, X, Y, and Z are calculated).
- FIG. 5 is a schematic diagram of re-projection according to an exemplary embodiment of the present application.
- a marker physical coordinate system is established on the marker plane, and the Z direction coordinate is 0, which is equivalent to placing the marker on the ground on.
- the projection position information of the designated point in the physical coordinate system of the marker can be calculated in the forward direction.
- the reprojection error can be calculated according to the following formula:
- X i , Y i are the coordinates corresponding to the projection position information of the i-th designated point in the physical coordinate system of the marker;
- X i0 and Y i0 are the coordinates corresponding to the position information of the i-th designated point in the physical coordinate system of the marker;
- n is the number of designated points on all markers.
- a reprojection error is calculated by projecting a designated point from a physical coordinate system of a marker onto a target image.
- the area where the marker is located occupies a smaller proportion on the target image.
- a reprojection error is calculated by projecting a designated point from a target image to a physical coordinate system of a marker, which is independent of the distance of the marker, and thus the accuracy of the internal reference can be measured more accurately.
- the internal and external parameters can be iteratively updated based on the Gauss iterative method.
- the internal and external parameters can be updated iteratively according to the following formula:
- ⁇ (k + 1) ⁇ k + (J T J) -1 J T w
- ⁇ (k + 1) is an updated internal or external reference
- ⁇ k is the internal or external reference before the update
- w is the calculated reprojection error
- J is the Jacobian matrix
- Jacobian matrix J can be obtained by the following methods:
- X f1 (u, f x , f y , c x , c y , k x1 , k x2 , k x3 , k x4 , r 1 , r 2 , r 3 , t 1 , t 2 , t 3 )
- Y f2 (v, f x , f y , c x , c y , k x1 , k x2 , k x3 , k x4 , r 1 , r 2 , r 3 , t 1 , t 2 , t 3 )
- u, v are the pixel coordinates of the specified point in the target image
- X, Y are the position information of the specified point in the physical coordinate system of the marker
- f x , f y , c x , c y , k x1 , k x2 , k x3 , k x4 are internal parameters
- r 1 , r 2 , r 3 , t 1 , t 2 , and t 3 are external parameters. Among them, the internal and external parameters are unknown.
- step S304 Determine whether the internal reference calibration conditions are met. If yes, execute step S305; otherwise, return to step S302.
- step S304 may include:
- the preset number of times and the preset threshold may be set according to actual needs, which is not limited in this embodiment.
- the positioning angle of each marker relative to the camera to be calibrated is determined to adjust the positioning angle of each marker relative to the camera to the target positioning angle, where: A calibration instrument composed of each marker covers the entire field of view of the camera, and each marker is evenly distributed in the height direction and the width direction of the field of view of the camera; acquiring a target image collected by the camera in the current field of view, The target images include images of the respective markers at their respective target positions, and the target position of each marker is a physical position matching the target placement angle of the marker; according to each marker The pixel coordinates of the specified point in the target image and the position information in the physical coordinate system of the marker mark the internal parameters of the camera. In this way, camera calibration can be completed based on one image, and the calibration efficiency is high.
- the embodiment of the present application may further provide a fisheye camera internal reference calibration device corresponding to the fisheye camera internal reference calibration method according to the foregoing embodiment.
- the internal reference calibration device can be implemented by software, or by hardware or a combination of software and hardware.
- the fisheye camera internal reference calibration device may be applied to a calibration device controller.
- the calibration device controller can be integrated on a fisheye camera.
- FIG. 6 is a schematic structural diagram of a calibration device controller in which a fisheye camera internal reference calibration device is located according to an exemplary embodiment of the present application.
- the calibration device controller may include a memory 610, a processor 620, a memory 630, and a network interface 640.
- the fisheye camera internal reference calibration device can be used as a device in a logical sense. It can be formed by reading the corresponding computer program instructions in the memory 610 into the memory 630 through the processor 620.
- the controller of the calibration device may also include other hardware, which will not be described again.
- FIG. 7 is a schematic structural diagram of a fisheye camera internal reference calibration device according to an exemplary embodiment of the present application.
- the internal reference calibration apparatus may include a determination module 710, an acquisition module 720, and a processing module 730.
- the determining module 710 may be configured to determine a target placement angle of each marker relative to the camera to be calibrated to adjust a placement angle of each marker relative to the camera to the target placement angle, where:
- the calibration instrument composed of each marker covers the entire field of view of the camera, and each marker is evenly distributed in the height direction and the width direction of the field of view of the camera.
- the obtaining module 720 may be configured to obtain a target image collected by the camera under the current field of view, where the target image includes an image of the respective markers at their respective target positions, and the target position of each marker Is the physical position that matches the target placement angle of the marker.
- the processing module 730 may be configured to calibrate internal parameters of the camera according to the pixel coordinates of the designated point on each marker in the target image and the position information in the physical coordinate system of the marker.
- the internal reference calibration device of this embodiment may be used to execute the method shown in FIG. 1, and its implementation principles and technical effects are similar, which are not described herein again.
- the determination module 710 may be configured to:
- An internal parameter of the camera is estimated according to a reference image collected by the camera, the reference image includes an image of the respective markers at a respective initial position, and the initial position of each marker is related to an initial value of the marker Place the physical position that matches the angle;
- a first between the incident light from the center point and the optical axis of the camera is calculated based on the estimated internal reference and the pixel coordinates of the center point of the marker in the reference image.
- the included angle and the second included angle between the line connecting the projection point of the center point on the camera coordinate system XOY plane and the origin of the camera coordinate system and the positive direction of the camera coordinate system X axis;
- the processing module 730 may perform the following steps:
- Step 1 Initially, use the initial value of the internal parameter as the target reference internal parameter and the initial value of the external parameter as the target reference external parameter;
- Step 2 According to the target reference internal reference and target reference external reference, and the pixel coordinates of the designated point on each marker in the target image, based on the mapping relationship between the pre-built image coordinate system and the marker physical coordinate system, Projecting a designated point on the marker from the target image to the marker physical coordinate system to obtain projection position information of the designated point in the marker physical coordinate system;
- Step 3 calculating a reprojection error according to the position information and projection position information of a designated point on each marker in the physical coordinate system of the marker, and updating the target reference internal reference and the target reference according to the reprojection error External reference
- Step 4 Determine whether the internal calibration conditions are met. If yes, the updated target reference internal parameter is used as the internal parameter of the calibrated camera; otherwise, return to step 2 above.
- processing module 730 may be configured to:
- the present application may also provide a computer storage medium on which a computer program is stored, and when the program is executed by a processor, the camera internal reference calibration method according to any of the foregoing embodiments of the present application is implemented.
- computer-readable media suitable for storing computer program instructions and data include all forms of non-volatile memory, media, and memory devices, including, for example, semiconductor memory devices (such as erasable programmable read-only memory (EPROM), Erasable Programmable Read Only Memory (EEPROM) and Flash devices), magnetic disks (such as internal hard disks or removable disks), magneto-optical disks, and compact disc read-only memory (CD-ROM) and digital versatile disc read-only memory (DVD-ROM ).
- semiconductor memory devices such as erasable programmable read-only memory (EPROM), Erasable Programmable Read Only Memory (EEPROM) and Flash devices
- magnetic disks such as internal hard disks or removable disks
- magneto-optical disks and compact disc read-only memory (CD-ROM) and digital versatile disc read-only memory (DVD-ROM ).
- CD-ROM compact disc read-only memory
- DVD-ROM digital versatile disc read-only memory
- the present application may also provide a calibration device controller, which may include a memory 610, a processor 620, and a computer program stored on the memory 610 and executable by the processor 620.
- a calibration device controller which may include a memory 610, a processor 620, and a computer program stored on the memory 610 and executable by the processor 620.
- the processor 620 executes the program, A camera internal reference calibration method according to any one of the above embodiments of the present application is implemented.
- the present application can also provide a fisheye camera internal reference calibration system.
- the system includes a calibration instrument and a camera, and the camera includes a calibration device controller.
- the system includes a calibration appliance, a camera, and a calibration device controller.
- the calibration instrument may include a plurality of calibration frames, and each of the calibration frames is fixed with a plurality of markers.
- the camera may acquire a target image including the respective markers in a current field of view.
- the target images include images of the respective markers at their respective target positions, and the target position of each marker is a physical position that matches the target placement angle of the marker.
- the calibration device controller may be used for:
- the internal parameters of the camera are calibrated according to the pixel coordinates of the designated point on each marker in the target image and the position information in the physical coordinate system of the marker.
- the calibration device controller can be integrated into a fisheye camera.
- the camera may acquire a reference image including the plurality of markers.
- the reference images include images of the respective markers at their respective initial positions, and the initial position of each marker is a physical position that matches the initial placement angle of the marker.
- the calibration device controller may be configured to:
- a first between the incident light from the center point and the optical axis of the camera is calculated based on the estimated internal reference and the pixel coordinates of the center point of the marker in the reference image.
- the included angle and the second included angle between the line connecting the projection point of the center point on the camera coordinate system XOY plane and the origin of the camera coordinate system and the positive direction of the camera coordinate system X axis;
- the calibration device controller may perform the following steps:
- Step 1 Initially, use the initial value of the internal parameter as the target reference internal parameter and the initial value of the external parameter as the target reference external parameter;
- Step 2 According to the target reference internal reference and target reference external reference, and the pixel coordinates of the designated point on each marker in the target image, based on the mapping relationship between the pre-built image coordinate system and the marker physical coordinate system, Projecting a designated point on the marker from the target image to the marker physical coordinate system to obtain projection position information of the designated point in the marker physical coordinate system;
- Step 3 calculating a reprojection error according to the position information and projection position information of a designated point on each marker in the physical coordinate system of the marker, and updating the target reference internal reference and the target reference according to the reprojection error External reference
- step 4 it is judged whether the internal reference calibration condition is satisfied, and if it is, the updated target reference internal reference is used as the internal reference of the camera to be calibrated; otherwise, return to step 2.
- the calibration device controller may be used to:
Abstract
一种鱼眼摄像机内参标定方法包括：确定各个标记物相对于待标定的摄像机的目标摆放角度，以将各个标记物相对于所述摄像机的摆放角度调整至所述目标摆放角度，其中，各个标记物组成的标定器具覆盖所述摄像机的全部视野，且所述各个标记物在所述摄像机的视野的高度方向和宽度方向上均匀分布；获取所述摄像机在当前视野下采集的目标图像，所述目标图像包含处于各自目标位置时的所述各个标记物的图像，每一个标记物的所述目标位置是与该标记物的所述目标摆放角度匹配的物理位置；依据各个标记物上的指定点在所述目标图像中的像素坐标以及在标记物物理坐标系下的位置信息，标定所述摄像机的内参。
Description
本申请涉及摄像机技术领域，尤其涉及鱼眼摄像机内参标定。
摄像机几何模型参数的获取过程被称为摄像机的标定，它是图像处理和计算机视觉领域中从二维图像提取三维空间信息必不可少的步骤，被广泛应用于三维重建、导航、视觉监控等领域。因此，如何进行摄像机的标定以获取摄像机几何模型参数很重要。
摄像机标定包括在一定的摄像机模型下，经过对图像进行处理，利用一系列数据变换和计算方法，求取摄像机几何模型参数。传统的标定方法中，通过获取同一标记物在不同位置的多张图像，进而基于这多张图像获取摄像机的内参。这样，需要人工将标记物放置在不同位置，进而采集标记物在不同位置的多张图像，比较耗时，标定效率较低。
发明内容
有鉴于此，本申请提供一种鱼眼摄像机内参标定方法、装置和系统、标定装置控制器以及标定器具，以提高标定效率。
本申请的第一方面提供一种鱼眼摄像机内参标定方法，所述方法包括：
确定各个标记物相对于待标定的摄像机的目标摆放角度，以将各个标记物相对于所述摄像机的摆放角度调整至所述目标摆放角度，其中，各个标记物组成的标定器具覆盖所述摄像机的全部视野，且所述各个标记物在所述摄像机的视野的高度方向和宽度方向上均匀分布；
获取所述摄像机在当前视野下采集的目标图像，所述目标图像包含处于各自目标位置时的所述各个标记物的图像，每一个标记物的所述目标位置是与该标记物的所述目标摆放角度匹配的物理位置；
依据各个标记物上的指定点在所述目标图像中的像素坐标以及在标记物物理坐标系下的位置信息，标定所述摄像机的内参。
本申请的第二方面提供一种摄像机内参标定装置，其中，标定器具包括多个标定架，每一个所述标定架上固定有多个标记物，所述内参标定装置包括：
确定模块，用于确定各个标记物相对于待标定的摄像机的目标摆放角度，以将各个标记物相对于所述摄像机的摆放角度调整至所述目标摆放角度，其中，所述标定器具覆盖所述摄像机的全部视野，且所述各个标记物在所述摄像机的视野的高度方向和宽度方向上均匀分布；
获取模块，用于获取所述摄像机在当前视野下采集的目标图像，所述目标图像包含处于各自目标位置时的所述各个标记物的图像，每一个标记物的所述目标位置是与该标记物的所述目标摆放角度匹配的物理位置；
处理模块，用于依据各个标记物上的指定点在所述目标图像中的像素坐标以及在标记物物理坐标系下的位置信息，标定所述摄像机的内参。
本申请的第三方面提供一种计算机存储介质，其上存储有计算机程序，所述计算机程序被处理器执行时实现本申请的第一方面提供的方法。
本申请的第四方面提供一种标定装置控制器，包括存储器、处理器及存储在所述存储器上并可由所述处理器执行的计算机程序，所述处理器执行所述计算机程序时实现本申请的第一方面提供的方法。
本申请的第五方面提供一种鱼眼摄像机内参标定系统，所述系统包括标定器具和摄像机，所述摄像机中包括标定装置控制器，或者，所述系统包括标定器具、摄像机和标定装置控制器，其中，
所述标定器具包括多个标定架，每一个所述标定架上固定有多个标记物；
所述摄像机用于在当前视野下采集包含所述各个标记物的目标图像，所述目标图像包含处于各自目标位置时的所述各个标记物的图像，每一个标记物的所述目标位置是与该标记物的目标摆放角度匹配的物理位置；
所述标定装置控制器用于：
确定各个标记物相对于所述摄像机的所述目标摆放角度，以将各个标记物相对于所述摄像机的摆放角度调整至所述目标摆放角度，所述标定器具覆盖所述摄像机的全部视野，且所述各个标记物在所述摄像机的视野的高度方向和宽度方向上均匀分布；
获取所述摄像机采集的所述目标图像，并依据各个标记物上的指定点在所述目 标图像中的像素坐标以及在标记物物理坐标系下的位置信息，标定所述摄像机的内参。
本申请的第六方面提供一种用于鱼眼摄像机内参标定的标定器具，所述标定器具包括多个标定架，每一个所述标定架上固定有多个标记物，其中，在进行鱼眼摄像机内参标定时，各个标记物相对于所述摄像机的摆放角度被调整至各自目标摆放角度，所述标定器具覆盖所述摄像机的全部视野，且所述各个标记物在所述摄像机的视野的高度方向和宽度方向上均匀分布。
根据本申请的上述各个方面，基于一张图像即可完成摄像机标定，标定效率得到提高。此外，通过将各个标记物相对于所述摄像机的摆放角度调整至目标摆放角度，使得摄像机采集到的目标图像清晰，进而基于清晰的目标图像进行摄像机的标定，可提高获取到的内参的精度。
图1为根据本申请一示例性实施例的鱼眼摄像机内参标定方法的流程图；
图2为根据本申请一示例性实施例的标定器具的示意图；
图3为根据本申请一示例性实施例的鱼眼摄像机的成像原理图；
图4为根据本申请一示例性实施例的鱼眼摄像机内参标定方法的流程图；
图5为根据本申请一示例性实施例的重投影的示意图；
图6为根据本申请一示例性实施例的鱼眼摄像机内参标定装置所在的标定装置控制器的结构示意图；
图7为根据本申请一示例性实施例的鱼眼摄像机内参标定装置的结构示意图。
这里将详细地对示例性实施例进行说明，其示例表示在附图中。下面的描述涉及附图时，除非另有表示，不同附图中的相同参考数字表示相同或相似的要素。以下示例性实施例中所描述的实施方式并不代表与本申请相一致的所有实施方式。相反，它们仅是与如所附权利要求书中所述的、本申请的一些方面相一致的装置和方法的例子。
在本申请使用的术语是仅仅出于描述特定实施例的目的，而非旨在限制本申请。在本申请和所附权利要求书中所使用的单数形式的“一种”、“所述”和“该”也旨在包括复 数形式，除非上下文清楚地表示其他含义。还应当理解，本文中使用的术语“和/或”是指并包含一个或多个相关联的列出项目中的任何一个或其所有可能组合。
应当理解，尽管在本申请可能采用术语第一、第二、第三等来描述各种信息，但这些信息不应受这些术语限制。这些术语仅用来将同一类型的信息彼此区分开。例如，在不脱离本申请范围的情况下，第一信息也可以被称为第二信息，类似地，第二信息也可以被称为第一信息。取决于语境，如在此所使用的词语“如果”可以被解释成为“在……时”或“当……时”或“响应于”。
本申请提供一种鱼眼摄像机内参标定方法、装置和系统、标定装置控制器以及标定器具，以提高标定效率。
本申请提供的鱼眼摄像机内参标定方法和装置，可应用于标定装置控制器中。该标定装置控制器可以集成在鱼眼摄像机中。
下面描述本申请的几个具体的实施例。这些实施例可以相互结合，对于相同或相似的概念或过程可能在某些实施例中不再赘述。
图1为根据本申请一示例性实施例的鱼眼摄像机内参标定方法的流程图。参照图1，该鱼眼摄像机内参标定方法可以包括：
S101、确定各个标记物相对于待标定的摄像机的目标摆放角度，以将各个标记物相对于上述摄像机的摆放角度调整至上述目标摆放角度，其中，各个标记物组成的标定器具覆盖上述摄像机的全部视野，且上述各个标记物在上述摄像机的视野的高度方向和宽度方向上均匀分布。
具体的，图2为根据本申请一示例性实施例的标定器具的示意图。参照图2，标定器具可以包括多个标定架，每一个所述标定架上固定有多个标记物，标定架的位置可灵活设置，标定架上的标记物的安装角度和高度可调整，标定器具覆盖摄像机的全部视野。可选的，各个标记物在摄像机的视野的高度方向和宽度方向上均匀分布。例如，在图2所示实施例中，多个标记物包含设置在不同标定架上的15个标记物。需要说明的是，如图2所示，标记物可以为棋盘格标定板。
步骤S101的具体实现过程可以包括：
(1)依据所述摄像机采集的参考图像估算所述摄像机的内参，所述参考图像包含处于各自初始位置时的所述各个标记物的图像，每一个标记物的所述初始位置是与该标记物的初始摆放角度匹配的物理位置。
参考图像是在未调整标记物相对于摄像机的摆放角度之前采集到的图像。有关依据所述摄像机采集的参考图像估算所述摄像机的内参的具体实现原理和实现过程可以参照相关技术中的描述，此处不再赘述。
(2)针对每一所述标记物，依据估算的内参、该标记物的中心点在所述参考图像中的像素坐标，计算来自所述中心点的入射光线与所述摄像机的光轴之间的第一夹角以及所述中心点在摄像机坐标系XOY平面的投影点与摄像机坐标系原点的连线与摄像机坐标系X轴正方向之间的第二夹角。
图3为根据本申请一示例性实施例的鱼眼摄像机的成像原理图。参照图3，O
c为摄像机坐标系原点，Xc、Yc、Zc为摄像机坐标系的坐标轴；P为空间中的一个三维点。本例中，假设P点为标记物的中心点，P1为该点在摄像机坐标系XcOcYc平面的投影点，则图3中的θ为第一夹角，图3中的α为第二夹角。需要说明的是，有关鱼眼摄像机的成像原理将在下面的实施例中详细介绍，此处不再赘述。
具体的，可按照如下公式计算第一夹角和第二夹角：
θ
d＝θ(1+k
1θ
2+k
2θ
4+k
3θ
6+k
4θ
8)
其中，x
w，y
w为归一化后的畸变坐标，x
distort，y
distort为中心点在参考图像中的像素坐标，c
x0和c
y0为估算的初始内参主点坐标，f
x0和f
y0为估算的初始内参焦距，k
1-k
4为内参畸变系数，θ为第一夹角，α为第二夹角。θ可以通过高斯牛顿迭代法计算出来。需要说明的是，根据几何关系可知，α等于图3中的
(3)将所述第一夹角和所述第二夹角作为所述目标摆放角度。
具体的，当计算得到第一夹角和第二夹角后，此时，就将第一夹角和第二夹角作为所述目标摆放角度。这样，标定人员可基于该第一夹角和第二夹角调整标记物相对于摄像机的摆放角度，以使标记物的中心点与摄像机坐标系原点的连线与摄像机坐标系Z轴正方向的夹角等于第一夹角，并使该标记物的中心点在摄像机坐标系XOY平面上 的投影点与摄像机坐标系原点的连线与摄像机坐标系X轴正方向之间的夹角等于第二夹角。
这样，通过确定各个标记物相对于待标定的摄像机的目标摆放角度，进而将各个标记物相对于所述摄像机的摆放角度调整至所述目标摆放角度，可保证采集到的目标图像的成像清晰，进而基于清晰的目标图像进行摄像机的标定，可提高获取到的内参的精度。
S102、获取上述摄像机在当前视野下采集的目标图像，上述目标图像包含处于各自目标位置时的所述各个标记物的图像，每一个标记物的上述目标位置是与该标记物的上述目标摆放角度匹配的物理位置。
具体的，当调整好各个标记物的摆放角度后，就控制摄像机采集目标图像，进而获取摄像机采集到的目标图像。
这样，不需要像现有技术那样采集同一个标记物在不同位置的多个图像，因而可提高标定的效率。
S103、依据各个标记物上的指定点在上述目标图像中的像素坐标以及在标记物物理坐标系下的位置信息，标定上述摄像机的内参。
具体的，内参指的是与摄像机自身特性相关的参数，包括摄像机的焦距、主点坐标和畸变系数等。例如，针对鱼眼摄像机来说，内参包括主点坐标(2个)、焦距(2个)和畸变系数(4个)，共8个参数。
参照图4，步骤S103的具体实现过程可以包括：
S301、在初始，将内参初始值作为目标参考内参，将外参初始值作为目标参考外参。
具体的，步骤S301之前，可以检测上述目标图像中的各个标记物上的指定点，并获取上述指定点在目标图像中的像素坐标。
具体的，标记物上的指定点可以是根据实际需要设定的，例如，一实施例中，标记物上的指定点可以是标记物的角点。以棋盘格标定板为例，可利用类棋盘格模板算子与目标图像进行卷积，利用能量投票方法计算相似度高的棋盘格位置，再利用区域生长和亚像素化棋盘格角点提取来检测出目标图像中的各个棋盘格角点。检测到角点后，可得到该角点在上述目标图像中的像素坐标。
此外，标记物物理坐标系可以以标记物的左上角为坐标原点，标记物的宽度方向为X轴(远离左上角的一侧为正方向)，标记物的高度方向为Y轴(远离左上角的一侧为正方向)。这样，针对一标记物上的任一指定点，可以预先确定该指定点在标记物物理坐标系下的位置信息。
S302、依据目标参考内参和目标参考外参、以及各个标记物上的指定点在上述目标图像中的像素坐标，基于预先构建的图像坐标系与标记物物理坐标系的映射关系，将各个标记物上的指定点从上述目标图像投影至上述标记物物理坐标系，得到上述指定点在上述标记物物理坐标系下的投影位置信息。
再次参照图3，其中，O
c为摄像机坐标系原点，Xc、Yc、Zc为摄像机坐标系的坐标轴；o为图像坐标系原点，x、y为图像坐标系的坐标轴；针对空间中的一个三维点P，其在该鱼眼摄像机下的成像点为p，其在针孔摄像机下的成像点为q；θ为来自点P的入射光线与摄像机光轴之间的夹角；r为鱼眼摄像机成像点p距离图像坐标系原点的径向距离；
为径向距离r与图像坐标系x轴正方向之间的夹角；f为鱼眼摄像机的焦距。基于鱼眼摄像机的成像原理可知，预先构建的图像坐标系与标记物物理坐标系的映射关系可用下列公式(1)-(13)表示：
x＝X
c (2)
y＝Y
c (3)
z＝Z
c (4)
a＝x/z (5)
b＝y/z (6)
r
2＝a
2+b
2 (7)
θ＝atan(r) (8)
θ
d＝θ(1+k
1θ
2+k
2θ
4+k
3θ
6+k
4θ
8) (9)
x′＝(θ
d/r)a (10)
y′＝(θ
d/r)b (11)
u＝f
x(x′+ay′)+c
x (12)
v＝f
yy′+c
y (13)
其中，X，Y，Z为空间中的一个三维点在标记物物理坐标系下的位置信息，R
1和T
1分别为标记物物理坐标系转换到摄像机坐标系的旋转量和平移量，即摄像机的外参，k
1-k
4为内参畸变系数，c
x和c
y为内参主点坐标，f
x和f
y为内参焦距，u和v为该三维点对应的成像点在图像坐标系下的像素坐标。
具体的，可依据上述指定点在目标图像中的像素坐标，利用公式(13)-(1)计算将指定点从目标图像投影至标记物物理坐标系而得到的指定点在标记物物理坐标系下的投影位置信息。即，可根据公式(13)-(2)，反推目标图像到畸变矫正图(即根据u、v得到Xc、Yc、Zc)，进而根据公式(1)由畸变矫正图计算指定点在标记物物理坐标系下的投影位置信息(即计算得到X、Y、Z)。
例如，图5为根据本申请一示例性实施例的重投影的示意图。参照图5，参见前面的介绍，以标记物的左上角为标记物物理坐标系原点，在标记物平面上建立标记物物理坐标系，Z方向坐标为0，等价于标记物摆放在地面上。此时，可依据指定点在目标图像中的像素坐标，正向计算该指定点在标记物物理坐标系下的投影位置信息。
S303、依据各个标记物上的指定点在标记物物理坐标系下的位置信息和投影位置信息计算重投影误差，并依据上述重投影误差更新上述目标参考内参和上述目标参考外参。
具体的，可按照如下公式计算重投影误差：
其中，w为重投影误差；
X
i，Y
i为第i指定点在标记物物理坐标系下的投影位置信息相应的坐标；
X
i0，Y
i0为第i指定点在标记物物理坐标系下的位置信息相应的坐标；
m为所有标记物上的指定点的数量。
需要说明的是，现有技术中，通过将指定点从标记物物理坐标系投影到目标图像来计算重投影误差。这样，当标记物摆放得距摄像机较远时，该标记物所在区域在目标图像上所占比例较少。当通过将指定点从标记物物理坐标系投影到目标图像计算重投影误差时，无法排除小的重投影误差是否是因标记物所在区域在目标图像上所占比例较 少而引起的，即无法准确度量内参的精度。
根据本实施例提供的方法，通过将指定点从目标图像投影到标记物物理坐标系来计算重投影误差，与标记物的摆放距离无关，因而可以更准确地度量内参的精度。
进一步地，可基于高斯迭代法迭代更新内参和外参。例如，可按照如下公式迭代更新内参和外参：
β
(k+1)＝β
k+(J
TJ)
-1J
Tw
其中，β
(k+1)为更新后的内参或外参；
β
k为更新前的内参或外参；
w为计算得到的重投影误差；
J为雅克比矩阵。
需要说明的是，雅克比矩阵J可通过如下方法获得：
具体的，针对标记物上的指定点，有：
X＝f1(u，f
x，f
y，c
x，c
y，k
x1，k
x2，k
x3，k
x4，r
1，r
2，r
3，t
1，t
2，t
3)
Y＝f2(v，f
x，f
y，c
x，c
y，k
x1，k
x2，k
x3，k
x4，r
1，r
2，r
3，t
1，t
2，t
3)
其中，u，v为指定点在目标图像中的像素坐标，X，Y为该指定点在标记物物理坐标系下的位置信息；
f
x，f
y，c
x，c
y，k
x1，k
x2，k
x3，k
x4为内参，r
1，r
2，r
3，t
1，t
2，t
3为外参。其中，内参和外参均为未知数。
这样，针对所有标记物，若共有m个指定点，则有如下方程：
进一步地，对上述方程求偏导，即可得到雅克比矩阵J：
具体的，有关基于高斯迭代法迭代更新内参和外参的具体实现原理和实现方式可以参照相关技术中的描述，此处不再赘述。
S304、判断是否满足内参标定条件。如果是，则执行步骤S305；否则，返回到步骤S302。
具体的，步骤S304的实现过程可以包括：
(1)判断目标参考内参的更新次数是否达到预设次数，或者，判断所述重投影误差是否小于或等于预设阈值；
(2)如果是，则确定满足内参标定条件；否则，确定不满足内参标定条件。
具体的，预设次数和预设阈值可以是根据实际需要设定的，本实施例中，不对此进行限定。
S305、将更新后的目标参考内参作为标定的上述摄像机的内参。
根据本实施例提供的方法，通过确定各个标记物相对于待标定的摄像机的目标摆放角度，以将各个标记物相对于所述摄像机的摆放角度调整至所述目标摆放角度，其中，各个标记物组成的标定器具覆盖所述摄像机的全部视野，且所述各个标记物在所述摄像机的视野的高度方向和宽度方向上均匀分布；获取所述摄像机在当前视野下采集的目标图像，所述目标图像包含处于各自目标位置时的所述各个标记物的图像，每一个标记物的所述目标位置是与该标记物的所述目标摆放角度匹配的物理位置；依据各个标记物上的指定点在所述目标图像中的像素坐标以及在标记物物理坐标系下的位置信息，标定所述摄像机的内参。这样，基于一张图像即可完成摄像机标定，标定效率较高。
本申请的实施例还可以提供与根据前述实施例的鱼眼摄像机内参标定方法相对应的鱼眼摄像机内参标定装置。该内参标定装置可以通过软件实现，也可以通过硬件或者软硬件结合的方式实现。
根据本申请的实施例的鱼眼摄像机内参标定装置可以应用在标定装置控制器上。该标定装置控制器可以集成在鱼眼摄像机上。
图6为根据本申请一示例性实施例的鱼眼摄像机内参标定装置所在的标定装置控制器的结构示意图。参照图6，该标定装置控制器可以包括存储器610、处理器620、内存630和网络接口640。鱼眼摄像机内参标定装置可以作为逻辑意义上的装置，通过处理器620将存储器610中对应的计算机程序指令读取到内存630中运行而形成。此外，根据该鱼眼摄像机内参标定装置的实际功能，该标定装置控制器还可以包括其他硬件，对此不再赘述。
图7为根据本申请一示例性实施例的鱼眼摄像机内参标定装置的结构示意图。参照图7，该内参标定装置可以包括确定模块710、获取模块720和处理模块730。
所述确定模块710可以用于：确定各个标记物相对于待标定的摄像机的目标摆放角度，以将各个标记物相对于所述摄像机的摆放角度调整至所述目标摆放角度，其中，各个标记物组成的标定器具覆盖所述摄像机的全部视野，且所述各个标记物在所述摄像机的视野的高度方向和宽度方向上均匀分布。
所述获取模块720可以用于：获取所述摄像机在当前视野下采集的目标图像，所述目标图像包含处于各自目标位置时的所述各个标记物的图像，每一个标记物的所述目标位置是与该标记物的所述目标摆放角度匹配的物理位置。
所述处理模块730可以用于：依据各个标记物上的指定点在所述目标图像中的像素坐标以及在标记物物理坐标系下的位置信息，标定所述摄像机的内参。
本实施例的内参标定装置可用于执行图1所示的方法，其实现原理和技术效果类似，此处不再赘述。
为了确定各个标记物相对于待标定的摄像机的目标摆放角度，所述确定模块710可以用于：
依据所述摄像机采集的参考图像估算所述摄像机的内参，所述参考图像包含处于各自初始位置时的所述各个标记物的图像，每一个标记物的所述初始位置是与该标记物的初始摆放角度匹配的物理位置；
针对每一所述标记物，依据估算的内参、该标记物的中心点在所述参考图像中的像素坐标，计算来自所述中心点的入射光线与所述摄像机的光轴之间的第一夹角以及所述中心点在摄像机坐标系XOY平面的投影点与摄像机坐标系原点的连线与摄像机坐标系X轴正方向之间的第二夹角；
将所述第一夹角和所述第二夹角作为所述目标摆放角度。
为了标定所述摄像机的内参，所述处理模块730可以执行以下步骤：
步骤1，在初始，将内参初始值作为目标参考内参，将外参初始值作为目标参考外参；
步骤2，依据目标参考内参和目标参考外参、以及各个标记物上的指定点在所述目标图像中的像素坐标，基于预先构建的图像坐标系与标记物物理坐标系的映射关系，将各个标记物上的指定点从所述目标图像投影至所述标记物物理坐标系，得到所述指定点在所述标记物物理坐标系下的投影位置信息；
步骤3，依据各个标记物上的指定点在所述标记物物理坐标系下的位置信息和投影位置信息计算重投影误差，并依据所述重投影误差更新所述目标参考内参和所述目标参考外参；
步骤4，判断是否满足内参标定条件。如果是，则将更新后的目标参考内参作为标定的上述摄像机的内参；否则，返回到上述步骤2。
具体地，所述处理模块730可以用于：
判断目标参考内参的更新次数是否达到预设次数，或者，判断所述重投影误差是否小于或等于预设阈值；
如果是，则确定满足内参标定条件；否则，确定不满足内参标定条件。
本申请还可以提供一种计算机存储介质，其上存储有计算机程序，所述程序被处理器执行时实现根据本申请的上述任一实施例的摄像机内参标定方法。
具体的，适合于存储计算机程序指令和数据的计算机可读介质包括所有形式的非易失性存储器、媒介和存储器设备，例如包括半导体存储器设备(例如可擦可编程只读存储器(EPROM)、电可擦可编程只读存储器(EEPROM)和闪存设备)、磁盘(例如内部硬盘或可移动盘)、磁光盘以及光盘只读存储器(CD-ROM)和数字多功能光盘只读存储器(DVD-ROM)。
再次参照图6，本申请还可以提供一种标定装置控制器，其可以包括存储器610、处理器620及存储在存储器610上并可由处理器620执行的计算机程序，处理器620执行所述程序时实现根据本申请的上述任一实施例的摄像机内参标定方法。
本申请还可以提供一种鱼眼摄像机内参标定系统。所述系统包括标定器具和摄像机，所述摄像机中包括标定装置控制器。或者，所述系统包括标定器具、摄像机和标 定装置控制器。
所述标定器具可以包括多个标定架，每一个所述标定架上固定有多个标记物。
所述摄像机可以在当前视野下采集包含所述各个标记物的目标图像。所述目标图像包含处于各自目标位置时的所述各个标记物的图像，每一个标记物的所述目标位置是与该标记物的目标摆放角度匹配的物理位置。
所述标定装置控制器可以用于：
确定各个标记物相对于所述摄像机的所述目标摆放角度，以将各个标记物相对于所述摄像机的摆放角度调整至所述目标摆放角度，所述标定器具覆盖所述摄像机的全部视野，且所述各个标记物在所述摄像机的视野的高度方向和宽度方向上均匀分布；
获取所述摄像机采集的所述目标图像；
依据各个标记物上的指定点在所述目标图像中的像素坐标以及在标记物物理坐标系下的位置信息，标定所述摄像机的内参。
具体的，标定器具的具体结构可以参见图2，此处不再赘述。此外，标定装置控制器可以集成在鱼眼摄像机中。
此外，所述摄像机还可以采集包含所述多个标记物的参考图像。所述参考图像包含处于各自初始位置时的所述各个标记物的图像，每一个标记物的所述初始位置是与该标记物的初始摆放角度匹配的物理位置。
为了确定各个标记物相对于所述摄像机的目标摆放角度，所述标定装置控制器可以用于：
获取所述摄像机采集的所述参考图像，并依据所述参考图像估算所述摄像机的内参；
针对每一所述标记物，依据估算的内参、该标记物的中心点在所述参考图像中的像素坐标，计算来自所述中心点的入射光线与所述摄像机的光轴之间的第一夹角以及所述中心点在摄像机坐标系XOY平面的投影点与摄像机坐标系原点的连线与摄像机坐标系X轴正方向之间的第二夹角；
将所述第一夹角和所述第二夹角作为所述目标摆放角度。
为了标定所述摄像机的内参，所述标定装置控制器可以执行以下步骤：
步骤1，在初始，将内参初始值作为目标参考内参，将外参初始值作为目标参考外参；
步骤2，依据目标参考内参和目标参考外参、以及各个标记物上的指定点在所述目标图像中的像素坐标，基于预先构建的图像坐标系与标记物物理坐标系的映射关系，将各个标记物上的指定点从所述目标图像投影至所述标记物物理坐标系，得到所述指定点在所述标记物物理坐标系下的投影位置信息；
步骤3，依据各个标记物上的指定点在所述标记物物理坐标系下的位置信息和投影位置信息计算重投影误差，并依据所述重投影误差更新所述目标参考内参和所述目标参考外参；
步骤4，判断是否满足内参标定条件，如果是，则将更新后的目标参考内参作为标定的所述摄像机的内参；否则，返回到步骤2。
为了判断是否满足内参标定条件，所述标定装置控制器可以用于：
判断目标参考内参的更新次数是否达到预设次数，或者，判断所述重投影误差是否小于或等于预设阈值；
如果是，则确定满足内参标定条件；否则，确定不满足内参标定条件。
以上所述仅为本申请的一些实施例，并不用以限制本申请。凡在本申请的精神和原则之内所做的任何修改、等同替换、改进等，均应包含在本申请保护的范围之内。
Claims (15)
- 一种鱼眼摄像机内参标定方法，其中，标定器具包括多个标定架，每一个所述标定架上固定有多个标记物，所述方法包括：确定各个标记物相对于待标定的摄像机的目标摆放角度，以将各个标记物相对于所述摄像机的摆放角度调整至所述目标摆放角度，其中，所述标定器具覆盖所述摄像机的全部视野，且所述各个标记物在所述摄像机的视野的高度方向和宽度方向上均匀分布；获取所述摄像机在当前视野下采集的目标图像，所述目标图像包含处于各自目标位置时的所述各个标记物的图像，每一个标记物的所述目标位置是与该标记物的所述目标摆放角度匹配的物理位置；依据各个标记物上的指定点在所述目标图像中的像素坐标以及在标记物物理坐标系下的位置信息，标定所述摄像机的内参。
- 根据权利要求1所述的方法，其中，所述确定各个标记物相对于待标定的摄像机的目标摆放角度包括：依据所述摄像机采集的参考图像估算所述摄像机的内参，所述参考图像包含处于各自初始位置时的所述各个标记物的图像，每一个标记物的所述初始位置是与该标记物的初始摆放角度匹配的物理位置；针对每一所述标记物，依据估算的内参、该标记物的中心点在所述参考图像中的像素坐标，计算来自所述中心点的入射光线与所述摄像机的光轴之间的第一夹角以及所述中心点在摄像机坐标系XOY平面的投影点与摄像机坐标系原点的连线与摄像机坐标系X轴正方向之间的第二夹角；将所述第一夹角和所述第二夹角作为所述目标摆放角度。
- 根据权利要求1或2所述的方法，其中，所述标定所述摄像机的内参包括：步骤1，在初始，将内参初始值作为目标参考内参，将外参初始值作为目标参考外参；步骤2，依据所述目标参考内参和所述目标参考外参、以及各个标记物上的指定点在所述目标图像中的像素坐标，基于预先构建的图像坐标系与标记物物理坐标系的映射关系，将各个标记物上的指定点从所述目标图像投影至所述标记物物理坐标系，得到所述指定点在所述标记物物理坐标系下的投影位置信息；步骤3，依据各个标记物上的指定点在所述标记物物理坐标系下的位置信息和所述投影位置信息计算重投影误差，并依据所述重投影误差更新所述目标参考内参和所述目标参考外参；步骤4，判断是否满足内参标定条件，如果满足内参标定条件，则将更新后的所述目标参考内参作为标定的所述摄像机的内参，而如果不满足内参标定条件，则返回到步骤2。
- 根据权利要求3所述的方法，其中，所述判断是否满足内参标定条件包括：判断所述目标参考内参的更新次数是否达到预设次数；如果所述更新次数达到所述预设次数，则确定满足内参标定条件，而如果所述更新次数没达到所述预设次数，则确定不满足内参标定条件。
- 根据权利要求3所述的方法，其中，所述判断是否满足内参标定条件包括：判断所述重投影误差是否小于或等于预设阈值；如果所述重投影误差小于或等于所述预设阈值，则确定满足内参标定条件，而如果所述重投影误差大于所述预设阈值，则确定不满足内参标定条件。
- 根据权利要求3所述的方法，其中，所述依据所述重投影误差更新所述目标参考内参和所述目标参考外参包括：依据所述重投影误差，基于高斯迭代法迭代更新所述目标参考内参和所述目标参考外参。
- 一种鱼眼摄像机内参标定装置，其中，标定器具包括多个标定架，每一个所述标定架上固定有多个标记物，所述内参标定装置包括：确定模块，用于确定各个标记物相对于待标定的摄像机的目标摆放角度，以将各个标记物相对于所述摄像机的摆放角度调整至所述目标摆放角度，其中，所述标定器具覆盖所述摄像机的全部视野，且所述各个标记物在所述摄像机的视野的高度方向和宽度方向上均匀分布；获取模块，用于获取所述摄像机在当前视野下采集的目标图像，所述目标图像包含处于各自目标位置时的所述各个标记物的图像，每一个标记物的所述目标位置是与该标记物的所述目标摆放角度匹配的物理位置；处理模块，用于依据各个标记物上的指定点在所述目标图像中的像素坐标以及在标记物物理坐标系下的位置信息，标定所述摄像机的内参。
- 根据权利要求7所述的装置，其中，所述确定模块被配置成：依据所述摄像机采集的参考图像估算所述摄像机的内参，所述参考图像包含处于各自初始位置时的所述各个标记物的图像，每一个标记物的所述初始位置是与该标记物的初始摆放角度匹配的物理位置；针对每一所述标记物，依据估算的内参、该标记物的中心点在所述参考图像中的像 素坐标，计算来自所述中心点的入射光线与所述摄像机的光轴之间的第一夹角以及所述中心点在摄像机坐标系XOY平面的投影点与摄像机坐标系原点的连线与摄像机坐标系X轴正方向之间的第二夹角；将所述第一夹角和所述第二夹角作为所述目标摆放角度。
- 根据权利要求7或8所述的装置，其中，所述处理模块被配置成执行以下步骤：步骤1，在初始，将内参初始值作为目标参考内参，将外参初始值作为目标参考外参；步骤2，依据所述目标参考内参和所述目标参考外参、以及各个标记物上的指定点在所述目标图像中的像素坐标，基于预先构建的图像坐标系与标记物物理坐标系的映射关系，将各个标记物上的指定点从所述目标图像投影至所述标记物物理坐标系，得到所述指定点在所述标记物物理坐标系下的投影位置信息；步骤3，依据各个标记物上的指定点在所述标记物物理坐标系下的位置信息和所述投影位置信息计算重投影误差，并依据所述重投影误差更新所述目标参考内参和所述目标参考外参；步骤4，判断是否满足内参标定条件，如果满足内参标定条件，则将更新后的所述目标参考内参作为标定的所述摄像机的内参，而如果不满足内参标定条件，则返回到步骤2。
- 根据权利要求9所述的装置，其中，所述依据所述重投影误差更新所述目标参考内参和所述目标参考外参包括：依据所述重投影误差，基于高斯迭代法迭代更新所述目标参考内参和所述目标参考外参。
- 一种计算机存储介质，其上存储有计算机程序，所述计算机程序被处理器执行时实现根据权利要求1-6中任一项所述的方法。
- 一种标定装置控制器，包括存储器、处理器及存储在所述存储器上并可由所述处理器执行的计算机程序，所述处理器执行所述计算机程序时实现根据权利要求1-6中任一项所述的方法。
- 一种鱼眼摄像机内参标定系统，所述系统包括标定器具和摄像机，所述摄像机中包括标定装置控制器，或者，所述系统包括标定器具、摄像机和标定装置控制器，其中，所述标定器具包括多个标定架，每一个所述标定架上固定有多个标记物；所述摄像机用于在当前视野下采集包含所述各个标记物的目标图像，所述目标图像 包含处于各自目标位置时的所述各个标记物的图像，每一个标记物的所述目标位置是与该标记物的目标摆放角度匹配的物理位置；所述标定装置控制器用于：确定各个标记物相对于所述摄像机的所述目标摆放角度，以将各个标记物相对于所述摄像机的摆放角度调整至所述目标摆放角度，所述标定器具覆盖所述摄像机的全部视野，且所述各个标记物在所述摄像机的视野的高度方向和宽度方向上均匀分布；获取所述摄像机采集的所述目标图像，并依据各个标记物上的指定点在所述目标图像中的像素坐标以及在标记物物理坐标系下的位置信息，标定所述摄像机的内参。
- 一种用于鱼眼摄像机内参标定的标定器具，包括多个标定架，每一个所述标定架上固定有多个标记物，其中，在进行鱼眼摄像机内参标定时，各个标记物相对于所述摄像机的摆放角度被调整至各自目标摆放角度，所述标定器具覆盖所述摄像机的全部视野，且所述各个标记物在所述摄像机的视野的高度方向和宽度方向上均匀分布。
- 根据权利要求14所述的标定器具，其中，所述目标摆放角度是按以下方式确定的：依据所述摄像机采集的参考图像估算所述摄像机的内参，所述参考图像包含处于各自初始位置时的所述各个标记物的图像，每一个标记物的所述初始位置是与该标记物的初始摆放角度匹配的物理位置；针对每一所述标记物，依据估算的内参、该标记物的中心点在所述参考图像中的像素坐标，计算来自所述中心点的入射光线与所述摄像机的光轴之间的第一夹角以及所述中心点在摄像机坐标系XOY平面的投影点与摄像机坐标系原点的连线与摄像机坐标系X轴正方向之间的第二夹角；将所述第一夹角和所述第二夹角作为所述目标摆放角度。
Applications Claiming Priority (2)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201811143432.2A CN110969662B (zh) | 2018-09-28 | 2018-09-28 | 鱼眼摄像机内参标定方法、装置、标定装置控制器和系统 |
| CN201811143432.2 | 2018-09-28 |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| WO2020063708A1 true WO2020063708A1 (zh) | 2020-04-02 |
Family
ID=69952797
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| PCT/CN2019/108017 WO2020063708A1 (zh) | 2018-09-28 | 2019-09-26 | 鱼眼摄像机内参标定方法、装置和系统、标定装置控制器以及标定器具 |
Country Status (2)
| Country | Link |
|---|---|
| CN (1) | CN110969662B (zh) |
| WO (1) | WO2020063708A1 (zh) |
Cited By (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112102417A (zh) * | 2020-09-15 | 2020-12-18 | 北京百度网讯科技有限公司 | 确定世界坐标的方法和装置及用于车路协同路侧相机的外参标定方法 |
| CN112509034A (zh) * | 2020-11-26 | 2021-03-16 | 江苏集萃未来城市应用技术研究所有限公司 | 一种基于图像像素点匹配的大范围行人体温精准检测方法 |
| CN112509064A (zh) * | 2020-12-22 | 2021-03-16 | 北京百度网讯科技有限公司 | 显示摄像机标定进度的方法、装置、设备和存储介质 |
| CN113077524A (zh) * | 2021-04-22 | 2021-07-06 | 中德（珠海）人工智能研究院有限公司 | 一种双目鱼眼相机自动标定方法、装置、设备及存储介质 |
| CN113489945A (zh) * | 2020-12-18 | 2021-10-08 | 深圳市卫飞科技有限公司 | 一种目标定位方法、装置、系统及计算机可读存储介质 |
| CN113538588A (zh) * | 2020-04-17 | 2021-10-22 | 虹软科技股份有限公司 | 标定方法、标定装置及应用其的电子设备 |
| CN115401689A (zh) * | 2022-08-01 | 2022-11-29 | 北京市商汤科技开发有限公司 | 基于单目相机的距离测量方法、装置以及计算机存储介质 |
| CN117226853A (zh) * | 2023-11-13 | 2023-12-15 | 之江实验室 | 一种机器人运动学标定的方法、装置、存储介质、设备 |
Families Citing this family (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113379853B (zh) * | 2021-08-13 | 2021-11-23 | 腾讯科技（深圳）有限公司 | 相机内参的获取方法、装置、设备及可读存储介质 |
| CN117152257B (zh) * | 2023-10-31 | 2024-02-27 | 罗普特科技集团股份有限公司 | 一种用于地面监控摄像机多维角度计算的方法和装置 |
Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20070196016A1 (en) * | 2006-02-21 | 2007-08-23 | I-Hsien Chen | Calibration system for image capture apparatus and method thereof |
| CN206441238U (zh) * | 2017-02-09 | 2017-08-25 | 杭州零智科技有限公司 | 标定板和标定系统 |
| CN107610185A (zh) * | 2017-10-12 | 2018-01-19 | 长沙全度影像科技有限公司 | 一种鱼眼相机快速标定装置及标定方法 |
| CN108230396A (zh) * | 2017-12-08 | 2018-06-29 | 深圳市商汤科技有限公司 | 标定装置和标定设备 |
Family Cites Families (13)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| FR2946444B1 (fr) * | 2009-06-08 | 2012-03-30 | Total Immersion | Procede et dispositif de calibration d'un capteur d'images utilisant un systeme temps reel de suivi d'objets dans une sequence d'images |
| CN102034238B (zh) * | 2010-12-13 | 2012-07-18 | 西安交通大学 | 基于光学成像测头和视觉图结构的多摄像机系统标定方法 |
| US8970349B2 (en) * | 2011-06-13 | 2015-03-03 | Tyco Integrated Security, LLC | System to provide a security technology and management portal |
| JP2013137760A (ja) * | 2011-12-16 | 2013-07-11 | Cognex Corp | 複数のカメラに対する複数部品コレスポンダ |
| CN102968794B (zh) * | 2012-11-22 | 2015-04-22 | 上海交通大学 | 小型二维靶大视场双目立体视觉系统结构参数在位标定法 |
| US9641830B2 (en) * | 2014-04-08 | 2017-05-02 | Lucasfilm Entertainment Company Ltd. | Automated camera calibration methods and systems |
| JP6554873B2 (ja) * | 2015-03-31 | 2019-08-07 | 株式会社リコー | 投影システム、画像処理装置、校正方法およびプログラム |
| CN105844624B (zh) * | 2016-03-18 | 2018-11-16 | 上海欧菲智能车联科技有限公司 | 动态标定系统、动态标定系统中的联合优化方法及装置 |
| CN105913439B (zh) * | 2016-04-22 | 2018-12-11 | 清华大学 | 一种基于激光跟踪仪的大视场摄像机标定方法 |
| CN106251334B (zh) * | 2016-07-18 | 2019-03-01 | 华为技术有限公司 | 一种摄像机参数调整方法、导播摄像机及系统 |
| CN107784672B (zh) * | 2016-08-26 | 2021-07-20 | 百度在线网络技术（北京）有限公司 | 用于获取车载相机的外部参数的方法和装置 |
| CN106803273B (zh) * | 2017-01-17 | 2019-11-22 | 湖南优象科技有限公司 | 一种全景摄像机标定方法 |
| CN108010086A (zh) * | 2017-12-04 | 2018-05-08 | 深圳市赢世体育科技有限公司 | 基于网球场标志线交点的摄像机标定方法、装置和介质 |
-
2018
- 2018-09-28 CN CN201811143432.2A patent/CN110969662B/zh active Active
-
2019
- 2019-09-26 WO PCT/CN2019/108017 patent/WO2020063708A1/zh active Application Filing
Patent Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20070196016A1 (en) * | 2006-02-21 | 2007-08-23 | I-Hsien Chen | Calibration system for image capture apparatus and method thereof |
| CN206441238U (zh) * | 2017-02-09 | 2017-08-25 | 杭州零智科技有限公司 | 标定板和标定系统 |
| CN107610185A (zh) * | 2017-10-12 | 2018-01-19 | 长沙全度影像科技有限公司 | 一种鱼眼相机快速标定装置及标定方法 |
| CN108230396A (zh) * | 2017-12-08 | 2018-06-29 | 深圳市商汤科技有限公司 | 标定装置和标定设备 |
Cited By (12)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113538588A (zh) * | 2020-04-17 | 2021-10-22 | 虹软科技股份有限公司 | 标定方法、标定装置及应用其的电子设备 |
| CN112102417A (zh) * | 2020-09-15 | 2020-12-18 | 北京百度网讯科技有限公司 | 确定世界坐标的方法和装置及用于车路协同路侧相机的外参标定方法 |
| CN112102417B (zh) * | 2020-09-15 | 2024-04-19 | 阿波罗智联(北京)科技有限公司 | 确定世界坐标的方法和装置 |
| CN112509034A (zh) * | 2020-11-26 | 2021-03-16 | 江苏集萃未来城市应用技术研究所有限公司 | 一种基于图像像素点匹配的大范围行人体温精准检测方法 |
| CN113489945A (zh) * | 2020-12-18 | 2021-10-08 | 深圳市卫飞科技有限公司 | 一种目标定位方法、装置、系统及计算机可读存储介质 |
| CN112509064A (zh) * | 2020-12-22 | 2021-03-16 | 北京百度网讯科技有限公司 | 显示摄像机标定进度的方法、装置、设备和存储介质 |
| CN113077524A (zh) * | 2021-04-22 | 2021-07-06 | 中德（珠海）人工智能研究院有限公司 | 一种双目鱼眼相机自动标定方法、装置、设备及存储介质 |
| CN113077524B (zh) * | 2021-04-22 | 2023-12-08 | 中德（珠海）人工智能研究院有限公司 | 一种双目鱼眼相机自动标定方法、装置、设备及存储介质 |
| CN115401689A (zh) * | 2022-08-01 | 2022-11-29 | 北京市商汤科技开发有限公司 | 基于单目相机的距离测量方法、装置以及计算机存储介质 |
| CN115401689B (zh) * | 2022-08-01 | 2024-03-29 | 北京市商汤科技开发有限公司 | 基于单目相机的距离测量方法、装置以及计算机存储介质 |
| CN117226853A (zh) * | 2023-11-13 | 2023-12-15 | 之江实验室 | 一种机器人运动学标定的方法、装置、存储介质、设备 |
| CN117226853B (zh) * | 2023-11-13 | 2024-02-06 | 之江实验室 | 一种机器人运动学标定的方法、装置、存储介质、设备 |
Also Published As
| Publication number | Publication date |
|---|---|
| CN110969662A (zh) | 2020-04-07 |
| CN110969662B (zh) | 2023-09-26 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| WO2020063708A1 (zh) | 鱼眼摄像机内参标定方法、装置和系统、标定装置控制器以及标定器具 | |
| US11113835B2 (en) | Plant point cloud acquisition, registration and optimization method based on TOF camera | |
| US10176554B2 (en) | Camera calibration using synthetic images | |
| CN108765328B (zh) | 一种高精度多特征平面模板及其畸变优化和标定方法 | |
| US9928595B2 (en) | Devices, systems, and methods for high-resolution multi-view camera calibration | |
| WO2020038386A1 (zh) | 确定单目视觉重建中的尺度因子 | |
| CN110807809B (zh) | 基于点线特征和深度滤波器的轻量级单目视觉定位方法 | |
| Douxchamps et al. | High-accuracy and robust localization of large control markers for geometric camera calibration | |
| US10540813B1 (en) | Three-dimensional point data alignment | |
| JP2013186816A (ja) | 動画処理装置、動画処理方法および動画処理用のプログラム | |
| CN111899305A (zh) | 一种相机自动标定优化方法及相关系统、设备 | |
| CN109003309B (zh) | 一种高精度相机标定和目标姿态估计方法 | |
| WO2020168685A1 (zh) | 一种三维扫描视点规划方法、装置及计算机可读存储介质 | |
| US11212511B1 (en) | Residual error mitigation in multiview calibration | |
| CN115187658B (zh) | 多相机视觉的大目标定位方法、系统及设备 | |
| CN113324528B (zh) | 已知摄站位置的近景摄影测量目标定位方法及系统 | |
| CN107330927B (zh) | 机载可见光图像定位方法 | |
| CN115457147A (zh) | 相机标定方法、电子设备及存储介质 | |
| WO2018107427A1 (zh) | 相位映射辅助三维成像系统快速对应点匹配的方法及装置 | |
| CN113947638B (zh) | 鱼眼相机影像正射纠正方法 | |
| CN112686961A (zh) | 一种深度相机标定参数的修正方法、装置 | |
| US8406563B2 (en) | Photometric calibration method and device | |
| CN113822920B (zh) | 结构光相机获取深度信息的方法、电子设备及存储介质 | |
| CN113658279B (zh) | 相机内参和外参估算方法、装置、计算机设备及存储介质 | |
| CN109389645B (zh) | 相机自校准方法、系统、相机、机器人及云端服务器 |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| 121 | Ep: the epo has been informed by wipo that ep was designated in this application |
Ref document number: 19866044 Country of ref document: EP Kind code of ref document: A1 |
|
| NENP | Non-entry into the national phase |
Ref country code: DE |
|
| 122 | Ep: pct application non-entry in european phase |
Ref document number: 19866044 Country of ref document: EP Kind code of ref document: A1 |
|
| 122 | Ep: pct application non-entry in european phase |
Ref document number: 19866044 Country of ref document: EP Kind code of ref document: A1 |
|
| 32PN | Ep: public notification in the ep bulletin as address of the adressee cannot be established |
Free format text: NOTING OF LOSS OF RIGHTS PURSUANT TO RULE 112(1) EPC (EPO FORM 1205A DATED 29.11.2021) |
|
| 122 | Ep: pct application non-entry in european phase |
Ref document number: 19866044 Country of ref document: EP Kind code of ref document: A1 |