CN106548477B - A kind of multichannel fisheye camera caliberating device and method based on stereo calibration target - Google Patents
A kind of multichannel fisheye camera caliberating device and method based on stereo calibration target Download PDFInfo
- Publication number
- CN106548477B CN106548477B CN201710059420.0A CN201710059420A CN106548477B CN 106548477 B CN106548477 B CN 106548477B CN 201710059420 A CN201710059420 A CN 201710059420A CN 106548477 B CN106548477 B CN 106548477B
- Authority
- CN
- China
- Prior art keywords
- calibration target
- coordinate system
- camera
- camera lens
- coordinate
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/10—Image acquisition modality
- G06T2207/10004—Still image; Photographic image
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/30—Subject of image; Context of image processing
- G06T2207/30244—Camera pose
Landscapes
- Studio Devices (AREA)
Abstract
A kind of multichannel fisheye camera caliberating device and method based on stereo calibration target, belongs to camera calibration technical field.This method comprises: establishing three-dimensional coordinate system；Using two blocks of different location scaling board images in every road fisheye camera acquisition calibration target of panoramic shooting module to obtain fault image；Choose equidistant projection distortion model；Calculate the external parameter between camera lens inner parameter and camera lens coordinate system and all plane reference plate coordinate systems；Calculate the rotation and translation matrix between two plane reference plate coordinate systems and stereo calibration target coordinate system；Nonlinear optimization is carried out to obtain optimal camera lens inner parameter and relative to the external parameter of each scaling board coordinate system to the camera lens internal and external parameter of acquisition；The scaling method realizes the calibration of camera lens internal and external parameter using collapsible calibration target, the uncalibrated image of high quality can be obtained, according to the positional relationship between known stereo calibration target and camera lens, camera lens internal and external parameter is optimized, substantially increases the precision of camera calibration.
Description
Technical field
The present invention relates to camera calibration technical field more particularly to a kind of multichannel fisheye cameras based on stereo calibration target
Caliberating device and method.
Background technique
What camera model reflected is the mapping relations of object space and the plane of delineation, and camera calibration determines that reflection is taken the photograph
The inner parameter and external parameter of camera geometrical property and optical characteristics.Wherein, inner parameter is the intrinsic parameter of camera lens, will not
Change because of reasons such as lens location variations；And external parameter reflection be space where camera lens coordinate system and scaling board generation
The positional relationship of boundary's coordinate system can change because of the reasons such as lens location variation.For using camera lens as capital equipment
For vision system, such as panorama camera, camera calibration result directly affects the quality of panorama camera image mosaic quality.
Currently used camera calibration method be based on traditional camera scaling method, traditional camera scaling method need using
Size is known to demarcate object, by establish on calibration object point known to coordinate with it is corresponding between its picture point, using certain
Algorithm obtains the internal and external parameter of camera model, can be divided into three-dimensional scaling object according to the difference of calibration object and plane demarcates object.
Camera calibration method based on two-dimensional calibrations object, camera and plane reference object can move freely, and not need to know
Road kinematic parameter, calibration process is flexible, calibration result precision is higher, still, since the area of two-dimensional calibrations object should not be too large, institute
With when two-dimensional calibrations object is smaller with respect to visual field, the characteristic point quantity of extraction is inadequate, and the precision of calibration is not high.
Camera calibration based on three-dimensional scaling object can be demarcated by single image, and stated accuracy is higher.It is used at present
Stereo calibration target there are itself to block, be less susceptible to obtain the uncalibrated image of high quality, and the difficulty of processing of scaling board it is big,
Purposes compares the problems such as limitation.
Summary of the invention
The present invention is to overcome existing stereo calibration target there are difficulty of processing big, itself is blocked and the not high technology of stated accuracy
Problem, it is desirable to provide a kind of multichannel fisheye camera caliberating device based on stereo calibration target that the processing is simple blocks without itself, also
Provide a kind of high multichannel fisheye camera scaling method based on stereo calibration target of stated accuracy.
The present invention provides a kind of multichannel fisheye camera caliberating device based on stereo calibration target, comprising: panoramic shooting mould
Block, first motor drive module, the first roller group, image processing module, image display, calibration target, control module；
The panoramic shooting module includes bracket, multiple fish eye lenses that each side of bracket is arranged in and multiple images sensing
Device, the corresponding imaging sensor of each camera lens, the multiple fish eye lens are located at same vertical height, the multiple flake mirror
Head is rotatablely connected with side；
First roller group is set to first motor drive module bottom, and the first motor drive module passes through
First drive shaft is connect with the panoramic shooting module, and described image processing module connects the panoramic shooting module；The figure
As display module connects described image processing module；
The calibration target includes: the first bottom plate, the second bottom plate being flexibly connected by connecting shaft, is set to first bottom
It plate surface and the first scaling board that can be moved within its scope and is set to second backplate surface and can move within its scope
The second dynamic scaling board；The second drive shaft being fixedly connected with the connecting shaft, the second electricity with the second driving axis connection
The second roller group of second motor drive module bottom is arranged in machine drive module；First bottom plate and described second
The angle of bottom plate is (0 °, 150 °)；First calibration, the pattern in the second scaling board be chequered with black and white chessboard grid pattern or
Circular hole pattern；
The calibration target is opposite with any camera all the way of the panoramic shooting module, the control module connection described first
Motor drive module and second motor drive module.
Further, the fish eye lens and imaging sensor are 4.
Further, the caliberating device further includes the second calibration target identical with the calibration target structure, third calibration
Target, the 4th calibration target, the second calibration target, third calibration target, the 4th demarcate other three of target respectively with panoramic shooting module
The camera lens of road camera is opposite.
In addition, the present invention also provides a kind of multichannel fisheye camera scaling method based on stereo calibration target, utilization are above-mentioned
Described in any item caliberating devices demarcate fisheye camera, include the following steps:
Using connecting shaft as stereo calibration target coordinate system, the first scaling board is the first calibration coordinate system, and the second scaling board is the
Two calibration coordinate systems, the rotation and translation matrix between two plane reference plates and stereo calibration target coordinate system are expressed as (R1,
T1), (R2, T2)；
Using two blocks of different location scaling board images in every road fisheye camera acquisition calibration target of panoramic shooting module to
Obtain fault image；
Choose equidistant projection distortion model；
Equidistant projection distortion model is specifically expressed as follows:
rd=f θd； (1)
In formula (1), rdIndicate distance of the point in fault image to center of distortion, the focal length of f expression fisheye camera, θd
Indicate the angle between incident ray and fish eye lens optical axis, i.e. incidence angle；
It is as follows to choose radial distortion model:
θd=θ (1+k1θ2+k2θ4+k3θ6+k4θ8+…)； (2)
In formula (2), θ indicates distortionless incidence angle, K=(k1,k2,k3,k4...) and indicate radial distortion parameter；θdTable
Show distortion incidence angle；
By above-mentioned distortion model, the distortion point in fault image is calculated from the point in orthoscopic image, formula is as follows:
In formula (3),(xs, ys) it is to have distortion coordinate points, (xc, yc)
It is undistorted coordinate points, (cx, cy) indicate fault image center point coordinate；
Camera lens inner parameter and camera lens coordinate system are calculated separately according to collected fault image and imaging model and are owned
External parameter between plane reference plate coordinate system；
Circular is as follows: extracting characteristic point coordinate of all plane reference plates in the plane of delineation, calculates each flat
Homography matrix Homography, Dan Yingju in face scaling board coordinate system down space point and plane of delineation coordinate system between characteristic point
Battle array calculation method is as follows:
In formula (4),Indicate characteristic point coordinate in plane of delineation coordinate system,It indicates
Scaling board coordinate system down space characteristic point；S is the scale factor of any scale；W indicates the object plane for location observation
Physical conversion, including part spin matrix R relevant to the plane of delineation observed and part translation matrix T's and, W=
[RT]；M camera lens inner parameter matrix, including the direction x focal length fx, the direction y focal length fy, the central point of fault image in image
Coordinate (cx, cy)；
Homography matrix Homography is decomposed by the orthogonality of spin matrix, one is constituted and includes camera lens inner parameter and outer
The overdetermined equation of portion's parameter obtains the inner parameter and external parameter of video camera by singular value decomposition and a series of transformation, outside
Portion's parameter refers to the rotation and translation matrix between camera lens coordinate system and all plane reference plate coordinate systems；
According to the external parameter between camera lens coordinate system and all plane reference plate coordinate systems, and known stereo calibration target
Rotation and translation matrix between coordinate system and fish eye lens coordinate system calculates two plane reference plate coordinate systems and stereo calibration
Rotation and translation matrix between target coordinate system；
According to the internal and external parameter for solving obtained camera lens, the two plane reference plate coordinate systems calculated and three-dimensional mark
Rotation and translation matrix and selected equidistant projection distortion model and radial distortion model between targeting coordinate system, to acquisition
Camera lens inner parameter and external parameter carry out nonlinear optimization to obtaining optimal camera lens inner parameter and relative to each
The external parameter of scaling board coordinate system；
The specific method is as follows: carrying out camera parameters optimization using Levenberg-Marquardt algorithm, establishes to throw again
Shadow error is that the smallest optimization object function is as follows:
In formula (6),It is spatial point PjIt is converted into stereo calibration target
After coordinate under coordinate system, secondly the re-projection image with lens distortion projected by camera imaging model is sat
Mark, mijIt is the pixel coordinate point obtained using angular-point detection method, M is camera lens inner parameter matrix, and K indicates radial distortion system
Number, R1,T1,R2,T2Indicate the spin matrix and translation square between two plane reference plate coordinate systems and stereo calibration target coordinate system
Battle array；It is the spin matrix and translation matrix between stereo calibration target coordinate system and camera lens coordinate system, m indicates that shooting is three-dimensional
The number of target is demarcated, n indicates characteristic point total number in stereo calibration target.
Further, two pieces of different location marks in every road fisheye camera acquisition calibration target using panoramic shooting module
Fixed board image passes through the second driving unit being mounted below calibration target and the second roller group to obtain in fault image step
Driving calibration target moves on horizontal plane and vertical plane, to change scaling board in camera lens position within the vision.
Further, two pieces of different location marks in every road fisheye camera acquisition calibration target using panoramic shooting module
Fixed board image is to obtain in fault image step, by moving position and second calibration of first scaling board in the first bottom plate
Position of the plate in the second bottom plate, to change scaling board in fish eye lens position within the vision.
Further, two pieces of different location marks in every road fisheye camera acquisition calibration target using panoramic shooting module
Fixed board image rotates the angle that two bottom plates open to obtain in fault image step, by adjusting connecting shaft, to change
Scaling board is in fish eye lens position within the vision, and angular range is at (0 °, 150 °).
Further, two pieces of different location marks in every road fisheye camera acquisition calibration target using panoramic shooting module
Fixed board image is rotated and translatory mirror by first motor drive module and the first roller group to obtaining in fault image step
The location of head, changes the pose of camera lens, to shoot stereo calibration target image of each camera lens under different positions and pose.
Further, two pieces of different location marks in every road fisheye camera acquisition calibration target using panoramic shooting module
Fixed board image, by adjusting the angle between camera lens and side face, changes the pose of camera lens to obtain in fault image step,
To shoot stereo calibration target image of each camera lens under different positions and pose.
Further, the external parameter according between camera lens coordinate system and all plane reference plate coordinate systems, and
Rotation and translation matrix between the stereo calibration target coordinate system known and fish eye lens coordinate system calculates two plane reference plates and sits
Rotation and translation matrix between mark system and stereo calibration target coordinate system specifically: flat according to the camera lens coordinate system of acquisition and two
Relationship between the scaling board coordinate system of face finds out all plane reference plate coordinate systems and three-dimensional mark using camera lens coordinate system as intermediary
Spin matrix R between targeting coordinate systemiWith translation matrix Ti, transformation relation is as follows:
In formula (5),Indicate the spin matrix between known stereo calibration target coordinate system and camera lens coordinate system
And translation matrix,Indicate the spin matrix and translation square between i-th of plane reference plate coordinate system and camera lens coordinate system
Battle array.
Multichannel fisheye camera scaling method disclosed by the invention based on stereo calibration target is using a kind of new collapsible mark
Targeting realizes the calibration of camera lens internal and external parameter, can be realized simultaneously the acquisition of two pieces of scaling board image informations in calibration target,
And there is no images to block, and reduces uncalibrated image collecting quantity, in addition, according between known stereo calibration target and camera lens
Positional relationship optimizes camera lens internal and external parameter, substantially increases the precision of camera calibration.
Additional aspect and advantage of the invention will be set forth in part in the description, and will partially become from the following description
Obviously, or practice through the invention is recognized.
Detailed description of the invention
In order to more clearly explain the embodiment of the invention or the technical proposal in the existing technology, to embodiment or will show below
There is attached drawing needed in technical description to be briefly described, it should be apparent that, the accompanying drawings in the following description is only this
Some embodiments of invention without any creative labor, may be used also for those of ordinary skill in the art
To obtain other drawings based on these drawings.
Fig. 1 is that a kind of multichannel fisheye camera caliberating device structure based on stereo calibration target provided in an embodiment of the present invention is shown
It is intended to；
Fig. 2 is a kind of multichannel fisheye camera caliberating device calibration shape based on stereo calibration target provided in an embodiment of the present invention
State structural schematic diagram；
Fig. 3 is a kind of multichannel fisheye camera scaling method process based on stereo calibration target provided in an embodiment of the present invention
Figure；
Fig. 4 is stereo calibration target coordinate system schematic diagram provided in an embodiment of the present invention；
Fig. 5 is the stereo calibration target image of fisheye camera shooting provided in an embodiment of the present invention.
Specific embodiment
Following will be combined with the drawings in the embodiments of the present invention, and technical solution in the embodiment of the present invention carries out clear, complete
Site preparation description, it is clear that described embodiments are only a part of the embodiments of the present invention, instead of all the embodiments.It is based on
Embodiment in the present invention, it is obtained by those of ordinary skill in the art without making creative efforts every other
Embodiment shall fall within the protection scope of the present invention.
There are itself to block for the existing camera calibration based on three-dimensional scaling object, is less susceptible to obtain the calibration maps of high quality
Picture, and the difficulty of processing of scaling board is big, and purposes compares limitation, needs to research and develop a kind of convenient and practical and high stated accuracy solid
Target is demarcated, to improve panorama camera stated accuracy, is spliced convenient for panoramic image.
In order to solve the above technical problems, the present invention provides a kind of, the multichannel fisheye camera based on stereo calibration target demarcates dress
It sets, which includes: panoramic shooting module 9, first motor drive module 14, the first roller group 15, image processing module 16, figure
As display module 17, calibration target 1, control module 19；
The panoramic shooting module 9 includes bracket 18, multiple fish eye lenses 101 of each side of bracket 18 and multiple is arranged in
Imaging sensor (not shown), the corresponding imaging sensor of each camera lens 101, the multiple fish eye lens 101 are located at
Same vertical height, the multiple fish eye lens 101 are rotatablely connected with side；
First roller group 15 is set to 14 bottom of first motor drive module, the first motor drive module
14 are connect by the first drive shaft 13 with the panoramic shooting module 9, and described image processing module 16 connects the panoramic shooting
Module 9；Described image display module 17 connects described image processing module 16；
The calibration target 1 includes: the first bottom plate 11, the second bottom plate 12 being flexibly connected by connecting shaft 3, is set to described
First bottom plate, 11 surface and the first scaling board 21 that can be moved within its scope and it is set to 12 surface of the second bottom plate and can
The the second scaling board (not shown) moved within its scope；The second drive shaft 6 being fixedly connected with the connecting shaft 3, with
The second motor drive module 7 that second drive shaft 6 connects, is arranged in the second of 7 bottom of the second motor drive module
Roller group 8；The angle of first bottom plate 11 and second bottom plate 12 is (0 °, 150 °)；
The calibration target 1 is opposite with any camera all the way of the panoramic shooting module 9, and the control module 19 connects described
First motor drive module 14 and second motor drive module 7.
As the embodiment of the present invention, the present invention relates to panorama camera there are four road fish eye lenses, then fisheye camera
There are four fish eye lens and imaging sensors for tool in caliberating device.The bracket of panoramic shooting module is set as cubic shaped, branch
Four camera lenses are then set to cubic support to guarantee that each camera lens is located at same vertical height for plane by each side of frame
The center of each side.Since camera lens can be rotated relative to side, the angle of camera lens optical axis and side face vertical line is arranged
It is being less than or equal to 5 °.
Calibration while in order to realize four road fisheye cameras uses the identical calibration of four block structures in caliberating device simultaneously
Target, the caliberating device further include it is identical with the calibration target structure second calibration target, third calibration target, the 4th calibration target,
The second calibration target, third calibration target, the 4th demarcate the target camera lens phase with other No. three camera of panoramic shooting module respectively
It is right.Pass through the position and angle, panoramic shooting module of first motor drive module and the second motor drive module adjustment calibration target
Upper fish-eye position and angle, to realize four road flake scaling board image information collectings.
First motor drive module in above-mentioned apparatus can drive panoramic shooting module in three-dimensional space by the first drive shaft
Between move freely, and can in solid space arbitrarily rotation to adjust lens direction, in addition, camera lens can be with opposite flank
It rotates by a certain angle；Second motor drive module can drive calibration target to move freely in three-dimensional space by the second drive shaft, mark
The first bottom plate and the second bottom plate opening angle are also adjusted in targeting, and the first scaling board is mobile relative to the first bottom plate, the second mark
Fixed board is mobile relative to the second bottom plate, to guarantee marking timing acquiring camera lens more scaling board images within sweep of the eye.
The pattern of scaling board is set as chequered with black and white gridiron pattern in calibration target, uses calibration target, it is ensured that when calibration
Enough accuracy that is sufficient amount of, being evenly distributed can be found and be substantially at the characteristic point of same depth, without by calibration institute
Locate the constraint of actual scene.The pattern of scaling board, which can also be arranged to circular hole pattern equally, in calibration target can be realized calibration knot
Fruit.
When using calibration target, as shown in Fig. 2, angle is between 0 ° to 150 ° between the first bottom plate 11 and the second bottom plate 12, from
And realize a camera lens acquisition while having the image there are two scaling board, it not only can be reduced the quantity that image is acquired when calibration, but also energy
Improve the precision of calibration；When without using calibration target, it can be closed, memory space and scaling board quilt can be saved
Pollution, influences calibration result.
The multichannel fisheye camera caliberating device disclosed by the invention collapsible calibration target new using one kind, handling ease, no
There are itself to block, and can obtain the uncalibrated image of high quality, improve the precision of calibration result.In addition, in scaling board image
It is more flexible in acquisition, the mode that calibration target is mobile or panoramic shooting module is mobile can be taken to collect camera lens visual field model
The image for enclosing interior each region substantially increases the precision of camera calibration.
As shown in figure 3, the present invention also provides a kind of multichannel fisheye camera scaling method based on stereo calibration plate, utilizes
Above-mentioned caliberating device demarcates fisheye camera, includes the following steps:
S1: using connecting shaft as stereo calibration target coordinate system, the first scaling board is the first calibration coordinate system, and the second scaling board is
Second calibration coordinate system, the rotation and translation matrix between two plane reference plates and stereo calibration target coordinate system are expressed as (R1,
T1), (R2, T2)；
Specifically, two bottom plates are by even as shown in Figure 1, stereo calibration target is made of two bottom plates and two scaling boards
Spindle rotation connection, two scaling boards are connected on two bottom plates by magnetic connector, in addition, below stereo calibration target
Second driving unit is installed, stereo calibration target can be driven both horizontally and vertically to move.Here, between two bottom plates
Connecting shaft establish stereo calibration target coordinate system, as shown in figure 4, stereo calibration target include two plane reference plates, their mark
Position fixing system is respectively O1X1Y1Z1、O2X2Y2Z2, coordinate system OXYZ using connecting shaft as stereo calibration target, two plane references
Rotation and translation matrix between plate and stereo calibration target coordinate system is expressed as (R1, T1), (R2, T2)。
S2: using panoramic shooting module every road fisheye camera acquisition calibration target in two blocks of different location scaling board images from
And obtain fault image；
For above-mentioned calibration target, it is abnormal to obtain to acquire the scaling board image in every road fish eye lens visual field under each position
Become image, as shown in figure 5, the amount of images generally acquired is compared with single scaling board image, it is possible to reduce half, calibration knot
The accuracy of fruit is higher, and calculation amount is moderate.
When acquiring stereo calibration target image, scaling board or fish-eye position can be changed in the following way, from
And different scaling board images are shot to obtain fault image:
1, by the second driving unit and the second roller group driving calibration target that are mounted below calibration target in horizontal plane and
It is moved on vertical plane, to change scaling board in camera lens position within the vision.
2, by mobile first scaling board in the position of position and the second scaling board in the second bottom plate in the first bottom plate,
To change scaling board on fish eye lens position within the vision, the general side that two plane reference plates are moved to bottom plate
Edge, the i.e. periphery of camera lens field range can thus form bigger calibration region, to improve the precision of calibration.
3, the angle that two bottom plates open is rotated by adjusting connecting shaft, to change scaling board in fish eye lens visual field model
Interior position is enclosed, angular range covers the field range of camera lens in (0 °, 150 °), the angle for opening two bottom plates substantially.
The location of 4, rotate by first motor drive module and the first roller group and translate camera lens, change camera lens
Pose, to shoot stereo calibration target image of each camera lens under different positions and pose.
5, by adjusting the angle between camera lens and side face, change the pose of camera lens, to shoot each camera lens not
With the stereo calibration target image under pose.
Multichannel fisheye camera scaling method disclosed by the invention based on stereo calibration plate is using a kind of new collapsible mark
Targeting is demarcated to realize, handling ease, there is no itself to block, and can obtain the uncalibrated image of high quality, improves calibration knot
The precision of fruit.In addition, it is more flexible in scaling board Image Acquisition, calibration target can be taken mobile or the shifting of panoramic shooting module
Dynamic mode collects the image of camera lens each region within sweep of the eye, substantially increases the precision of camera calibration.
S3: equidistant projection distortion model is chosen；
Fish eye lens is usually to be formed by more than ten of different lens combination, and in imaging process, incident ray is not by
With the refraction of degree, it is refracted on the limited imaging plane of size.The model that panorama picture of fisheye lens follows be approximately unit ball
Face projection model, therefore using equidistant projection distortion module in the present invention, equidistant projection distortion model is specifically expressed as follows:
rd=f θd； (1)
In formula (1), rdIndicate distance of the point in fault image to center of distortion, the focal length of f expression fisheye camera, θd
Indicate the angle between incident ray and fish eye lens optical axis, i.e. incidence angle；
Since fish-eye projection model is in order to project to scene as big as possible in the limited plane of delineation,
Fish-eye distortion is very serious.The fish-eye primarily radial distortion of distortion, it is as follows to choose radial distortion model:
θd=θ (1+k1θ2+k2θ4+k3θ6+k4θ8+…)； (2)
In formula (2), θ indicates distortionless incidence angle, K=(k1,k2,k3,k4...) and indicate radial distortion parameter；θdTable
Show the angle between incident ray and fish eye lens optical axis, i.e. distortion incidence angle；
By above-mentioned distortion model, the distortion point in fault image is calculated from the point in orthoscopic image, formula is as follows:
In formula (3),(xs, ys) it is to have distortion coordinate points, (xc, yc)
It is undistorted coordinate points, (cx, cy) indicate fault image center point coordinate；
S4: camera lens inner parameter and camera lens coordinate system and institute are calculated separately according to collected fault image and imaging model
There is the external parameter between plane reference plate coordinate system；
The inner parameter that camera lens calibration is related to has optical center position, the distortion system of camera lens effective focal length and camera lens
Number, the external parameter being related to have rotation and translation matrix.The internal and external parameter for first having to initialization camera lens, i.e., by fault image
On distortion point carry out distortion and obtain metapole, then calculate and singly answer square between space coordinate point and orthoscopic image point
Battle array Homography, circular are as follows: extracting characteristic point coordinate of all plane reference plates in the plane of delineation, calculate
Homography matrix Homography in each plane reference plate coordinate system down space point and plane of delineation coordinate system between characteristic point, it is single
Answer matrix computational approach as follows:
In formula (4),Indicate characteristic point coordinate in plane of delineation coordinate system,It indicates
Scaling board coordinate system down space characteristic point；S is the scale factor of any scale；W indicates the object plane for location observation
Physical conversion, including part spin matrix R relevant to the plane of delineation observed and part translation matrix T's and, W=
[RT]；M is camera lens inner parameter matrix, including the direction x focal length fx, the direction y focal length fy, the center of fault image in image
Point coordinate (cx, cy)；
Homography matrix Homography is decomposed by the orthogonality of spin matrix, one is constituted and includes camera lens inner parameter and outer
The overdetermined equation of portion's parameter obtains the inner parameter and external parameter of video camera by singular value decomposition and a series of transformation, interior
Portion's parameter includes M, and K, external parameter includes Ri,Ti；External parameter refer to camera lens coordinate system and all plane reference plate coordinate systems it
Between rotation and translation matrix；
Imaging model includes distortion model, when calculating fish eye lens internal and external parameter, according to distortion model, and homography matrix
Internal and external parameter is sought.
S5: according to the external parameter between camera lens coordinate system and all plane reference plate coordinate systems, and known three-dimensional mark
Rotation and translation matrix between targeting coordinate system and fish eye lens coordinate system calculates each plane reference plate coordinate system and solid
Demarcate the rotation and translation matrix between target coordinate system；
In formula (5),Indicate the spin matrix between known stereo calibration target coordinate system and camera lens coordinate system
And translation matrix,Indicate the spin matrix and translation square between i-th of plane reference plate coordinate system and camera lens coordinate system
Battle array.
S6: it according to the internal and external parameter for solving obtained camera lens, each plane reference plate coordinate system calculated and stands
Body demarcates the rotation and translation matrix and selected equidistant projection distortion model and radial distortion model between target coordinate system, right
The camera lens inner parameter and external parameter of acquisition carry out nonlinear optimization to obtain optimal camera lens inner parameter and relative to
The external parameter of each scaling board coordinate system；
Fish-eye distortion is very serious, optimizes the camera shooting that above-mentioned solution obtains by using the method for nonlinear optimization
Machine parameter.Camera parameters optimization is specifically carried out using Levenberg-Marquardt algorithm, establishing with re-projection error is most
Small optimization object function is as follows:
In formula (6),It is spatial point PjIt is converted into stereo calibration target
After coordinate under coordinate system, secondly the re-projection image with lens distortion projected by camera imaging model is sat
Mark, mijIt is the pixel coordinate point obtained using angular-point detection method, M is camera lens inner parameter matrix, and K indicates radial distortion system
Number, R1,T1,R2,T2Indicate the spin matrix and translation square between two plane reference plate coordinate systems and stereo calibration target coordinate system
Battle array；It is the spin matrix and translation matrix between stereo calibration target coordinate system and camera lens coordinate system, m indicates that shooting is three-dimensional
The number of target is demarcated, n indicates characteristic point total number in stereo calibration target.
In the above-mentioned multichannel fisheye camera scaling method based on stereo calibration plate, it can be realized simultaneously two pieces of marks in calibration target
Fixed board image is to obtain the acquisition of fault image information, and there is no images to block, and reduces uncalibrated image collecting quantity, separately
Outside, according to the positional relationship between known stereo calibration target and camera lens, camera lens internal and external parameter is optimized, is greatly improved
The precision of camera calibration.
The process of multichannel fisheye camera caliberating device and method the specific implementation calibration provided through the invention is as follows:
If the video camera of visual sensor uses 1.45 millimeters of focal length of vertical ancient cooking vessel F123B0145WR1 camera lens, and image
Resolution ratio is 1225 pixel *, 1225 pixel, and the field angle of video camera is 190 ° × 120 °, and operating distance is 1500 millimeters.Calibration
The device of target is as shown in Figure 1, the angle that two bottom plates open is 125 °, and feature point number is 121 in each scaling board, characteristic point
Spacing is 50mm.
According to above-mentioned steps the method, nonlinear optimization, obtained result such as following table are carried out to intrinsic parameters of the camera
It is shown:
Multichannel fisheye camera scaling method disclosed by the invention based on stereo calibration target is using a kind of new collapsible mark
Targeting realizes the calibration of camera lens internal and external parameter, can be realized simultaneously in calibration target two blocks of scaling board images to be distorted
The acquisition of image information, and there is no images to block, and reduces uncalibrated image collecting quantity, in addition, according to known three-dimensional mark
Positional relationship between targeting and camera lens optimizes camera lens internal and external parameter, substantially increases the precision of camera calibration.
Above disclosed is only a preferred embodiment of the present invention, cannot limit the power of the present invention with this certainly
Sharp range, therefore equivalent changes made in accordance with the claims of the present invention, are still within the scope of the present invention.
Claims (10)
1. a kind of multichannel fisheye camera caliberating device based on stereo calibration target characterized by comprising panoramic shooting module,
First motor drive module, the first roller group, image processing module, image display, calibration target, control module；
The panoramic shooting module includes bracket, the multiple fish eye lenses and multiple images sensor that each side of bracket is arranged in,
The corresponding imaging sensor of each camera lens, the multiple fish eye lens are located at same vertical height, the multiple fish eye lens
It is rotatablely connected with side；
First roller group is set to first motor drive module bottom, and the first motor drive module passes through first
Drive shaft is connect with the panoramic shooting module, and described image processing module connects the panoramic shooting module；Described image is aobvious
Show that module connects described image processing module；
The calibration target includes: the first bottom plate, the second bottom plate being flexibly connected by connecting shaft, is set to the first bottom plate table
It face and the first scaling board that can be moved within its scope and is set to second backplate surface and can move within its scope
Second scaling board；The second drive shaft being fixedly connected with the connecting shaft is driven with the second motor of the second driving axis connection
The second roller group of second motor drive module bottom is arranged in dynamic model block；First bottom plate and second bottom plate
Angle be (0 °, 150 °)；Pattern in first calibration, the second scaling board is chequered with black and white chessboard grid pattern or circular hole
Pattern；
The calibration target is opposite with any camera all the way of the panoramic shooting module, and the control module connects the first motor
Drive module and second motor drive module.
2. the multichannel fisheye camera caliberating device according to claim 1 based on stereo calibration target, which is characterized in that described
Fish eye lens and imaging sensor are 4.
3. the multichannel fisheye camera caliberating device according to claim 2 based on stereo calibration target, which is characterized in that described
Caliberating device further include it is identical with the calibration target structure second calibration target, third calibration target, the 4th calibration target, described second
It is opposite with the camera lens of other No. three camera of panoramic shooting module respectively to demarcate target, third calibration target, the 4th calibration target.
4. a kind of multichannel fisheye camera scaling method based on stereo calibration target, utilization are as described in any one of claims 1 to 3
Caliberating device demarcates fisheye camera, which comprises the steps of:
Using connecting shaft as stereo calibration target coordinate system, the first scaling board is the first calibration coordinate system, and the second scaling board is the second mark
Position fixing system, the rotation and translation matrix between two plane reference plates and stereo calibration target coordinate system are expressed as (R1, T1),
(R2, T2)；
Using two blocks of different location scaling board images in every road fisheye camera acquisition calibration target of panoramic shooting module to obtain
Fault image；
Choose equidistant projection distortion model；
Equidistant projection distortion model is specifically expressed as follows:
rd=f θd； (1)
In formula (1), rdIndicate distance of the point in fault image to center of distortion, the focal length of f expression fisheye camera, θdIt indicates
Angle between incident ray and fish eye lens optical axis, i.e. incidence angle；
It is as follows to choose radial distortion model:
θd=θ (1+k1θ2+k2θ4+k3θ6+k4θ8+…)； (2)
In formula (2), θ indicates distortionless incidence angle, K=(k1,k2,k3,k4...) and indicate radial distortion parameter；θdIndicate abnormal
Variable incident angle；
By above-mentioned distortion model, the distortion point in fault image is calculated from the point in orthoscopic image, formula is as follows:
In formula (3),(xs, ys) it is to have distortion coordinate points, (xc, yc) it is without abnormal
Become coordinate points, (cx, cy) indicate fault image center point coordinate；
Camera lens inner parameter and camera lens coordinate system and all planes are calculated separately according to collected fault image and imaging model
External parameter between scaling board coordinate system；
Circular is as follows: extracting characteristic point coordinate of all plane reference plates in the plane of delineation, calculates each plane mark
Homography matrix Homography in fixed board coordinate system down space point and plane of delineation coordinate system between characteristic point, homography matrix meter
Calculation method is as follows:
In formula (4),Indicate characteristic point coordinate in plane of delineation coordinate system,Indicate calibration
Plate coordinate system down space characteristic point；S is the scale factor of any scale；W indicates the physics of the object plane for location observation
Transformation, including part spin matrix R relevant to the plane of delineation observed and part translation matrix T's and, W=[R T]；M
It is camera lens inner parameter matrix, including the direction x focal length fx, the direction y focal length fy, the center point coordinate of fault image in image
(cx, cy)；
Homography matrix Homography is decomposed by the orthogonality of spin matrix, one is constituted and includes camera lens inner parameter and external ginseng
Several overdetermined equations obtains the inner parameter and external parameter of video camera, outside ginseng by singular value decomposition and a series of transformation
Number refers to the rotation and translation matrix between camera lens coordinate system and all plane reference plate coordinate systems；
According to the external parameter between camera lens coordinate system and all plane reference plate coordinate systems, and known stereo calibration target coordinate
Rotation and translation matrix between system and fish eye lens coordinate system, calculates two plane reference plate coordinate systems and stereo calibration target is sat
Rotation and translation matrix between mark system；
According to the internal and external parameter for solving obtained camera lens, the two plane reference plate coordinate systems and stereo calibration target calculated
Rotation and translation matrix and selected equidistant projection distortion model and radial distortion model between coordinate system, to the mirror of acquisition
Head inner parameter and external parameter carry out nonlinear optimization to obtain optimal camera lens inner parameter and relative to each calibration
The external parameter of plate coordinate system；
The specific method is as follows: carrying out camera parameters optimization using Levenberg-Marquardt algorithm, foundation is missed with re-projection
Difference is that the smallest optimization object function is as follows:
In formula (6),It is spatial point PjIt is converted into stereo calibration target coordinate
After the lower coordinate of system, the re-projection image with lens distortion next coordinate projected by camera imaging model, mij
It is the pixel coordinate point obtained using angular-point detection method, M is camera lens inner parameter matrix, and K indicates coefficient of radial distortion, R1,
T1,R2,T2Indicate the spin matrix and translation matrix between two plane reference plate coordinate systems and stereo calibration target coordinate system；It is the spin matrix and translation matrix between stereo calibration target coordinate system and camera lens coordinate system, m indicates the three-dimensional mark of shooting
The number of targeting, n indicate characteristic point total number in stereo calibration target.
5. the multichannel fisheye camera scaling method according to claim 4 based on stereo calibration target, which is characterized in that described
Using two blocks of different location scaling board images in every road fisheye camera acquisition calibration target of panoramic shooting module to be distorted
In image step, by the second driving unit and the second roller group driving calibration target that are mounted below calibration target in horizontal plane and
It is moved on vertical plane, to change scaling board in camera lens position within the vision.
6. the multichannel fisheye camera scaling method according to claim 4 based on stereo calibration target, which is characterized in that described
Using two blocks of different location scaling board images in every road fisheye camera acquisition calibration target of panoramic shooting module to be distorted
In image step, by mobile first scaling board in the position of position and the second scaling board in the second bottom plate in the first bottom plate
It sets, to change scaling board in fish eye lens position within the vision.
7. the multichannel fisheye camera scaling method according to claim 4 based on stereo calibration target, which is characterized in that described
Using two blocks of different location scaling board images in every road fisheye camera acquisition calibration target of panoramic shooting module to be distorted
In image step, the angle that two bottom plates open is rotated by adjusting connecting shaft, to change scaling board in the fish eye lens visual field
Position in range, angular range is at (0 °, 150 °).
8. the multichannel fisheye camera scaling method according to claim 4 based on stereo calibration target, which is characterized in that described
Using two blocks of different location scaling board images in every road fisheye camera acquisition calibration target of panoramic shooting module to be distorted
The location of in image step, rotate by first motor drive module and the first roller group and translate camera lens, change mirror
The pose of head, to shoot stereo calibration target image of each camera lens under different positions and pose.
9. the multichannel fisheye camera scaling method according to claim 4 based on stereo calibration target, which is characterized in that described
Using two blocks of different location scaling board images in every road fisheye camera acquisition calibration target of panoramic shooting module to be distorted
In image step, by adjusting the angle between camera lens and side face, change the pose of camera lens, to shoot each camera lens not
With the stereo calibration target image under pose.
10. the multichannel fisheye camera scaling method according to claim 4 based on stereo calibration target, which is characterized in that institute
It states according to the external parameter between camera lens coordinate system and all plane reference plate coordinate systems, and known stereo calibration target coordinate system
Rotation and translation matrix between fish eye lens coordinate system calculates two plane reference plate coordinate systems and stereo calibration target coordinate
Rotation and translation matrix between system specifically: according between the camera lens coordinate system of acquisition and two plane reference plate coordinate systems
Relationship finds out the rotation between all plane reference plate coordinate systems and stereo calibration target coordinate system using camera lens coordinate system as intermediary
Matrix touchingiWith translation matrix Ti, transformation relation is as follows:
In formula (5),Indicate the spin matrix peace between known stereo calibration target coordinate system and camera lens coordinate system
The amount of shifting to,Indicate the spin matrix and translation matrix between i-th of plane reference plate coordinate system and camera lens coordinate system.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201710059420.0A CN106548477B (en) | 2017-01-24 | 2017-01-24 | A kind of multichannel fisheye camera caliberating device and method based on stereo calibration target |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201710059420.0A CN106548477B (en) | 2017-01-24 | 2017-01-24 | A kind of multichannel fisheye camera caliberating device and method based on stereo calibration target |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN106548477A CN106548477A (en) | 2017-03-29 |
| CN106548477B true CN106548477B (en) | 2019-03-29 |
Family
ID=58398778
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201710059420.0A Active CN106548477B (en) | 2017-01-24 | 2017-01-24 | A kind of multichannel fisheye camera caliberating device and method based on stereo calibration target |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN106548477B (en) |
Families Citing this family (28)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN107194972B (en) * | 2017-05-16 | 2021-04-02 | 成都通甲优博科技有限责任公司 | Camera calibration method and system |
| EP3644281A4 (en) * | 2017-06-20 | 2021-04-28 | Sony Interactive Entertainment Inc. | Calibration device, calibration chart, chart pattern generation device, and calibration method |
| CN107274439A (en) * | 2017-07-04 | 2017-10-20 | 西北工业大学 | Unmanned plane optical tracking localization method based on ground fisheye camera array |
| CN107492124B (en) * | 2017-07-27 | 2020-05-29 | 歌尔股份有限公司 | Plane calibration device of fisheye camera |
| CN109920004B (en) * | 2017-12-12 | 2023-12-19 | 广东虚拟现实科技有限公司 | Image processing method, device, calibration object combination, terminal equipment and calibration system |
| CN109064404A (en) * | 2018-08-10 | 2018-12-21 | 西安电子科技大学 | It is a kind of based on polyphaser calibration panorama mosaic method, panoramic mosaic system |
| CN109596319A (en) * | 2018-11-26 | 2019-04-09 | 歌尔股份有限公司 | The detection system and method for optics module parameter |
| CN109448063A (en) * | 2018-12-18 | 2019-03-08 | 苏州艾微视图像科技有限公司 | A kind of lens distortion center calibration facility and method |
| CN109754434B (en) * | 2018-12-27 | 2023-08-29 | 歌尔科技有限公司 | Camera calibration method, device, user equipment and storage medium |
| CN109785394A (en) * | 2018-12-31 | 2019-05-21 | 深圳积木易搭科技技术有限公司 | A kind of camera and turntable combined calibrating method, device and storage medium |
| WO2020224199A1 (en) | 2019-05-08 | 2020-11-12 | 四川深瑞视科技有限公司 | Fisheye camera calibration system, method and apparatus, electronic device, and storage medium |
| CN110136207B (en) * | 2019-05-08 | 2020-06-23 | 四川深瑞视科技有限公司 | Fisheye camera calibration system, fisheye camera calibration method, fisheye camera calibration device, electronic equipment and storage medium |
| WO2020232971A1 (en) * | 2019-05-22 | 2020-11-26 | 四川深瑞视科技有限公司 | Fisheye camera calibration system, method and apparatus, and electronic device and storage medium |
| CN110163922B (en) * | 2019-05-22 | 2020-06-23 | 四川深瑞视科技有限公司 | Fisheye camera calibration system, fisheye camera calibration method, fisheye camera calibration device, electronic equipment and storage medium |
| CN112907676B (en) * | 2019-11-19 | 2022-05-10 | 浙江商汤科技开发有限公司 | Calibration method, device and system of sensor, vehicle, equipment and storage medium |
| CN111080718A (en) * | 2019-12-23 | 2020-04-28 | 中科院微电子研究所昆山分所 | Camera module calibration method and device for 720-degree environment detection |
| CN113393529B (en) * | 2020-03-12 | 2024-05-10 | 浙江宇视科技有限公司 | Method, device, equipment and medium for calibrating camera |
| CN111652939A (en) * | 2020-04-16 | 2020-09-11 | 中国工程物理研究院应用电子学研究所 | Two-three-dimensional camera combined calibration grid plate and calibration device |
| CN111627070B (en) * | 2020-04-30 | 2024-01-19 | 如你所视(北京)科技有限公司 | Method, device and storage medium for calibrating rotation shaft |
| CN111445536A (en) * | 2020-05-13 | 2020-07-24 | 武汉夕睿光电技术有限公司 | Calibration device and method for 3D camera |
| CN111768383A (en) * | 2020-06-29 | 2020-10-13 | 易思维（杭州）科技有限公司 | Three-dimensional target and method for recovering working function of visual sensor by using same |
| CN113112549B (en) * | 2020-12-23 | 2022-08-23 | 合肥工业大学 | Monocular camera rapid calibration method based on coding stereo target |
| CN112753047B (en) * | 2020-12-30 | 2022-08-26 | 华为技术有限公司 | Method and system for in-loop calibration and target point setting of hardware of camera and related equipment |
| CN112907647B (en) * | 2021-01-26 | 2022-05-27 | 公安部交通管理科学研究所 | Three-dimensional space size measurement method based on fixed monocular camera |
| CN112837373B (en) * | 2021-03-03 | 2024-04-26 | 福州视驰科技有限公司 | Multi-camera pose estimation method without feature point matching |
| CN112991463A (en) * | 2021-03-17 | 2021-06-18 | 北京百度网讯科技有限公司 | Camera calibration method, device, equipment, storage medium and program product |
| CN113781579B (en) * | 2021-09-10 | 2022-11-11 | 浙江大学 | Geometric calibration method for panoramic infrared camera |
| CN114205483B (en) * | 2022-02-17 | 2022-07-29 | 杭州思看科技有限公司 | Scanner precision calibration method and device and computer equipment |
Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN201838036U (en) * | 2010-11-09 | 2011-05-18 | 姜光 | Bi-planar calibration plate |
| CN105654502A (en) * | 2016-03-30 | 2016-06-08 | 广州市盛光微电子有限公司 | Panorama camera calibration device and method based on multiple lenses and multiple sensors |
Family Cites Families (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| JP5491235B2 (en) * | 2010-03-02 | 2014-05-14 | 東芝アルパイン・オートモティブテクノロジー株式会社 | Camera calibration device |
-
2017
- 2017-01-24 CN CN201710059420.0A patent/CN106548477B/en active Active
Patent Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN201838036U (en) * | 2010-11-09 | 2011-05-18 | 姜光 | Bi-planar calibration plate |
| CN105654502A (en) * | 2016-03-30 | 2016-06-08 | 广州市盛光微电子有限公司 | Panorama camera calibration device and method based on multiple lenses and multiple sensors |
Also Published As
| Publication number | Publication date |
|---|---|
| CN106548477A (en) | 2017-03-29 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN106548477B (en) | A kind of multichannel fisheye camera caliberating device and method based on stereo calibration target | |
| CN106846415B (en) | A kind of multichannel fisheye camera binocular calibration device and method | |
| CN106651859A (en) | Multipath fisheye camera calibration device and method | |
| CN206460515U (en) | A kind of multichannel fisheye camera caliberating device based on stereo calibration target | |
| CN106123798B (en) | A kind of digital photography laser scanning device | |
| CN105160680B (en) | A kind of design method of the noiseless depth camera based on structure light | |
| CN105118055B (en) | Camera position amendment scaling method and system | |
| CN206460516U (en) | A kind of multichannel fisheye camera binocular calibration device | |
| CN105488810B (en) | A kind of focusing light-field camera inside and outside parameter scaling method | |
| CN106887023A (en) | For scaling board and its scaling method and calibration system that binocular camera is demarcated | |
| CN108921901A (en) | A kind of big visual field camera calibration method based on accurate two-axis platcform and laser tracker | |
| CN107610185A (en) | A kind of fisheye camera fast calibration device and scaling method | |
| JP7185821B2 (en) | Stereocalibration method for movable vision system | |
| CN108646259A (en) | A kind of three-dimensional laser scanner, which is set, stands firm to device and method | |
| CN110197466A (en) | A kind of wide-angle fish eye images antidote | |
| CN109089025A (en) | A kind of image instrument digital focus method based on optical field imaging technology | |
| CN108234989B (en) | convergent integrated imaging shooting method based on checkerboard calibration plate | |
| CN106500596A (en) | The measuring method of structure light panorama measuring system | |
| KR101222104B1 (en) | Method and Apparatus for Generating Omnidirectional 3D Image using Line Scan Camera | |
| CN106921824A (en) | Circulating type mixes light field imaging device and method | |
| CN106023193A (en) | Array camera observation method for detecting structure surface in turbid media | |
| CN105959669A (en) | Remapping-based integral imaging micro-image array rapid generation method | |
| CN107941153A (en) | A kind of vision system of laser ranging optimization calibration | |
| CN104864855A (en) | Single-camera omnidirectional stereoscopic vision sensor and design method thereof | |
| CN113763480B (en) | Combined calibration method for multi-lens panoramic camera |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |