CN108109111A - Pass through the method for the more fish eye lens panorama cameras of software and hardware combining assembly and adjustment - Google Patents
Pass through the method for the more fish eye lens panorama cameras of software and hardware combining assembly and adjustment Download PDFInfo
- Publication number
- CN108109111A CN108109111A CN201810032504.XA CN201810032504A CN108109111A CN 108109111 A CN108109111 A CN 108109111A CN 201810032504 A CN201810032504 A CN 201810032504A CN 108109111 A CN108109111 A CN 108109111A
- Authority
- CN
- China
- Prior art keywords
- fish eye
- eye lens
- software
- fish
- parameter
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000000034 method Methods 0.000 title claims abstract description 61
- 241000251468 Actinopterygii Species 0.000 title claims description 126
- 238000013461 design Methods 0.000 claims abstract description 22
- 238000001514 detection method Methods 0.000 claims description 24
- 238000012937 correction Methods 0.000 claims description 22
- 230000008569 process Effects 0.000 claims description 15
- 238000005457 optimization Methods 0.000 claims description 14
- 230000003287 optical effect Effects 0.000 claims description 12
- 230000006870 function Effects 0.000 claims description 9
- 238000011990 functional testing Methods 0.000 claims description 7
- 239000000463 material Substances 0.000 claims description 6
- 238000004806 packaging method and process Methods 0.000 claims description 4
- 230000013011 mating Effects 0.000 claims description 3
- 230000002093 peripheral effect Effects 0.000 claims description 2
- 238000005259 measurement Methods 0.000 abstract description 3
- 238000004519 manufacturing process Methods 0.000 description 12
- 239000011159 matrix material Substances 0.000 description 11
- 230000036544 posture Effects 0.000 description 10
- 239000000047 product Substances 0.000 description 10
- 238000009434 installation Methods 0.000 description 7
- 238000012986 modification Methods 0.000 description 7
- 230000004048 modification Effects 0.000 description 7
- 230000000694 effects Effects 0.000 description 6
- 230000008901 benefit Effects 0.000 description 5
- 230000009466 transformation Effects 0.000 description 5
- 238000013519 translation Methods 0.000 description 5
- 241000208340 Araliaceae Species 0.000 description 4
- 235000005035 Panax pseudoginseng ssp. pseudoginseng Nutrition 0.000 description 4
- 235000003140 Panax quinquefolius Nutrition 0.000 description 4
- 238000005516 engineering process Methods 0.000 description 4
- 235000008434 ginseng Nutrition 0.000 description 4
- 238000003384 imaging method Methods 0.000 description 4
- 238000013507 mapping Methods 0.000 description 4
- 239000000203 mixture Substances 0.000 description 4
- 238000006243 chemical reaction Methods 0.000 description 3
- 238000003556 assay Methods 0.000 description 2
- 230000000007 visual effect Effects 0.000 description 2
- 239000006227 byproduct Substances 0.000 description 1
- 238000004364 calculation method Methods 0.000 description 1
- 230000008859 change Effects 0.000 description 1
- 238000010276 construction Methods 0.000 description 1
- 230000007547 defect Effects 0.000 description 1
- 230000007812 deficiency Effects 0.000 description 1
- 238000010586 diagram Methods 0.000 description 1
- 235000013399 edible fruits Nutrition 0.000 description 1
- 239000011521 glass Substances 0.000 description 1
- 238000007689 inspection Methods 0.000 description 1
- 238000012545 processing Methods 0.000 description 1
- 230000009467 reduction Effects 0.000 description 1
- 230000001360 synchronised effect Effects 0.000 description 1
- 238000012360 testing method Methods 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T3/00—Geometric image transformation in the plane of the image
- G06T3/40—Scaling the whole image or part thereof
- G06T3/4038—Scaling the whole image or part thereof for image mosaicing, i.e. plane images composed of plane sub-images
-
- G06T5/80—
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/80—Analysis of captured images to determine intrinsic or extrinsic camera parameters, i.e. camera calibration
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/10—Image acquisition modality
- G06T2207/10004—Still image; Photographic image
Abstract
The present invention provides the method by the more camera lens panorama cameras of software and hardware combining assembly and adjustment, including：It designs and multiple hardware is provided, and assemble them into panorama camera, wherein the theoretical parameter of panorama camera is determined；Establish the fixed Calibration Field measured for panorama camera with demarcating；Calibration software is designed, and the inside and outside parameter of panorama camera is demarcated using calibration software, the error amount between the actual parameter of panorama camera and corresponding theoretical parameter is obtained, and error amount is input in the memory of panorama camera and is preserved；Design splicing software；And the error amount preserved by splicing software transfer, carry out the final debugging of panorama camera.The present invention will reduce or eliminate error and changes into measurement calibrated error, and by software and hardware combining, the error in panorama camera is effectively corrected in a manner that simple, cost are efficient.
Description
Technical field
The present invention relates to panorama camera assembling, debugging techniques.More particularly it relates to pass through software and hardware knot
It closes and carrys out the more fish eye lens panorama cameras of assembly and adjustment, so as to provide assembly and adjustment of more fish eye lens panorama cameras before dispatching from the factory
Improved method.
Background technology
Traditionally, more camera lens panorama cameras are when carrying out image mosaic processing, the optical parallax of camera lens and the position of installation
Error be easy to cause algorithm distortion, and image mosaic is caused to misplace.In order to solve this problem, usual method is to first pass through survey
Amount means are accurately positioned the position of each camera lens in panorama camera, reduce site error；Then, calculating is passed through in software is spliced
Position and the optical parallax of each camera lens are eliminated again.So current usual method be using eliminate position and optical parallax as
It is main.
At present, there are two types of the solutions that manufacturer generally uses.A solution is that camera lens is carried out physical set
Position, by laser coaxial, either camera lens is carried out physical positioning combination reduction or eliminates site error by stent.Second of solution
Scheme is that camera lens is detected by Calibration Field, then adjusts each camera lens, to reduce or eliminate site error and example
Such as optical distortion optical parallax.
The field angle of existing general camera is small, usually less than 100 degree, and observation scope is limited, limits it to week
Enclose the application that 360 environment are carried out at the same time observation.However, fish eye lens field angle can reach about 180 degree and more than, therefore,
Shooting can be synchronized using two and its above fish eye lens achieve the effect that panorama.At present, existing pattern distortion
Correcting technology usually using plane perspective Projection Constraint, is mapped to by the straight line in three dimensions in two dimensional surface, it is suitable for
Field angle is small, the small image of amount of distortion.But the image captured by fish eye lens does not satisfy the constraint rule but, because it will
The project objects observed are on spherical surface.So how to obtain fish-eye spherical projection mapping relations, and establish flake mirror
The unconventional imaging model of head, is just particularly important.Fish eye lens panorama camera is usually by the flake mirror of 2 about 180 degrees
Head form, due to this camera fields of view overlay region deficiency, can cause to occur being substantially misaligned in the stitching portion for seeing two camera lenses or
Distortion.It can solve to misplace to a certain extent or distort using the fish eye lens of more, but it is such as how simple, of high cost
The various errors that the mode of effect effectively corrected or even eliminated in fish eye lens panorama camera are always that there is a need in the art for change
Into technology.
Currently existing scheme there are it is many problem of and shortcoming, it is such as, but not limited to listed below：
1) production process is more, high to manufacturing technique requirent, particularly stringent to the installation positioning requirements of camera lens.
Following several master operations are generally comprised in the process of the prior art to the inspection of camera lens and camera：Supplied materials detects；
Shot Detection is grouped；By camera lens Selective assembly camera；Focusing；Single lens function test；The adjustment of a plurality of lenses position positioning；
Other component assembles；Finished product detection, it is therein such as detect, adjust process reduce production efficiency and product yield.
2) requirement to for example splicing software is high.The prior art needs to calculate by such as splicing software substantial amounts of various
Error can just be spliced into preferable image, and which increase the calculation amounts of software, cause the high performance hardware supported of software needs
Can normal operation, on the one hand add product cost, on the other hand reduce more fish eye lens panorama cameras the speed of service and
Its reliability.
3) existing splicing software organically combines not before dispatching from the factory with hardware and other related softwares, and algorithm is complicated, by mistake
The correction and eradicating efficacy of difference are unsatisfactory.
Included information in this background section of the specification of the present invention, including any ginseng cited herein
Document and its any description or discussion are examined, is included merely for the purpose of Technical Reference, and is not considered as that will limit
The theme of the scope of the invention processed.
The content of the invention
In view of the above and it is other it is more design and propose the present invention.Present invention seek to address that above technology
Defect and it is other the technical issues of.
The present invention provides by software and combination of hardware come the improved method of the more fish eye lens panorama cameras of assembly and adjustment,
It can solve and eliminate the technical issues of above-mentioned and other and shortcoming, enabling with higher product quality, reliability
More fish eye lens panorama cameras before dispatching from the factory are provided with lower cost, so as to simplify the life of more fish eye lens panorama cameras
Process is produced, improves production efficiency and product yield, while simplifies the software algorithm embedded in more fish eye lens panorama cameras,
This helps to reduce requirement of the software to camera hardware performance, the hardware cost so as to indirect saving.
It is characteristic of the invention that the method for reducing or eliminating error is changed into the method for measurement calibrated error, by soft
The combination of part and hardware first passes through one group of Survey Software, measure in specific environment the position of each camera, optical parameter with
These errors are saved as one group of fixed parameter by the error between design theory value, and correction splicing software is direct when calculating
This group of parameter is called to carry out correction splicing to calculate.
Specifically, according to an aspect of the present invention, one kind is provided by the more flake mirrors of software and hardware combining assembly and adjustment
The method of head panorama camera, including：It designs and multiple hardware is provided, and assemble them into more fish eye lens panorama cameras, wherein
It is described to design the theoretical parameter that more fish eye lens panorama cameras are determined；It establishes fixed for the survey of more fish eye lens panorama cameras
Amount and the Calibration Field of calibration；Calibration software is designed, and using calibration software to the actual ginseng of more fish eye lens panorama cameras
Number is demarcated, and obtains the error amount between the actual parameter of more fish eye lens panorama cameras and corresponding theoretical parameter,
And the error amount is input in the memory of more fish eye lens panorama cameras or peripheral operation platform and is preserved；Design splicing is soft
Part；The error amount with being preserved by the splicing software transfer, carries out the final debugging of more fish eye lens panorama cameras.
An embodiment according to the present invention after the debugging, carries out the packaging before finished product detection and shipment.
An embodiment according to the present invention, the splicing software is the correction mating with calibration software and splicing software, is used
It corrects and splices in real time in the video of more fish eye lens panorama cameras.
An embodiment according to the present invention, the splicing software installation is in the memory of more fish eye lens panorama cameras
In or the mobile phone beyond more fish eye lens panorama cameras, computer or other support the platform of this running software
On.
The step of embodiment according to the present invention, more fish eye lens panorama cameras that multiple hardware are provided and are assembled
Including supplied materials detection being carried out to the element of more fish eye lens panorama cameras, to the fish eye lens sheet of more fish eye lens panorama cameras
Body is assembled and Function detection and into more fish eye lens panorama cameras and each hardware composing is completed functional test.
An embodiment according to the present invention, the theoretical parameters of more fish eye lens panorama cameras include inner parameter and outer
Portion's parameter.
An embodiment according to the present invention, by the methods of the more fish eye lens panorama cameras of software and hardware combining assembly and adjustment also
Including being optimized to the error amount between the actual parameter of more fish eye lens panorama cameras and corresponding theory parameter.
An embodiment according to the present invention, the inner parameter include it is following at least one of：Fish-eye X, Y-axis
Equivalent focal length, the inclined degree of prime focus position and reference axis；
The external parameter characterizes the position relationship between the fish eye lens of more fish eye lens panoramic cameras.
An embodiment according to the present invention, the error amount include it is following at least one of：Each fish-eye optics
Error amount between parameter and design theory value；Between actual relative position parameter and design theory value between fish eye lens
Error amount；The optical distortion parameter value of the pixel collected with each fish eye lens.
An embodiment according to the present invention, the method omit the detection grouping process of fish eye lens and panorama camera, with
And fish-eye position adjustment process.
An embodiment according to the present invention, the optimization are non-linear LM optimizations.
An embodiment according to the present invention, the Calibration Field are indoor optical targets.
According to an aspect of the present invention, for example, in the indoor standardization field manually arranged, multichannel such as four tunnels is obtained and are synchronously clapped
The image taken the photograph, and the distortion parameter of camera lens is calculated using the fish eye lens projection model of the present invention, it is corrected.
Afterwards, accurate internal and external parameter is carried out in the Calibration Field of special arrangement using the four groups of images corrected
(rotation transformation between focal length and prime focus and camera lens and translation transformation wherein for example including camera lens) is estimated, from one
The 3D postures of object are estimated in the mapping of group 2D points.
In image flame detection link, due to fish-eye front lens diameter is very short and protruded in parabolical to camera lens forepart and
Cause it shoot image be finally mapped in a unit sphere, so, imaging of the fish eye lens on photo-sensitive cell with it is general
It is logical that pin-hole imaging has huge difference.So in the image flame detection link, present invention design is with a kind of side of fitting of a polynomial
Method, by automatic angle point grid, fitting of a polynomial, minimum projection's error optimization and etc. after, it is possible to will be projected
The space line for becoming circular arc on sphere is corrected by force as its reset condition, and makes its correction to meet pinhole imaging system principle
Image.
In image calibration link, four fish eye lenses of one kind shown in Fig. 1-2 according to the present invention, since its element manufactures
For precision, camera lens installation accuracy etc. there are error, the angle between the adjacent fish eye lens of any two is not many times ideal
120 degree.Therefore, posture (one kind of the external parameter) conversion between camera lens how is accurately estimated, is to avoid in the later stage splicing
Occurs the key substantially to misplace at seam.The present invention manually arranged in specific Calibration Field, using to pole constraints, merely with
Few match point can obtain accurate posture estimation.
In addition, according to the present invention, correction and the scaling method of a kind of more fish eye lens panorama cameras are disclosed, including following
Step：In step sl, establish more fish eye lens panorama cameras for the Calibration Field correcting and demarcate；In step s 2, obtain
Take each how fish-eye image in more fish eye lenses；In step s3, each fish-eye image of acquisition is carried out
Spherical projection is corrected；In step s 4, the image after being corrected to spherical projection is demarcated；In step s 5, after to correction
The inner parameter and external parameter of image optimize；In step s 6, the posture between required different fish eye lenses is obtained
Conversion.
An embodiment according to the present invention, the external parameter are the fish eye lenses for characterizing more fish eye lens panoramic cameras
Between position relationship external parameter.
An embodiment according to the present invention, the external parameter can be image Attitude estimation parameters.
An embodiment according to the present invention, the external parameter are the fish eye lenses for characterizing more fish eye lens panoramic cameras
Between position relationship external parameter.
An embodiment according to the present invention, the external parameter are fish-eye location estimation parameters, such as fish eye lens
The rotation transformation of position and translation transformation relation.
An embodiment according to the present invention, the fish-eye quantity are 2,3,4,5,6,7,8 or more.
An embodiment according to the present invention, the image after being corrected in step s 4 to spherical projection, which carries out calibration, to be included, right
Fish-eye inside and outside portion's parameter is estimated.
An embodiment according to the present invention, the optimization in the step S5 are non-linear LM optimizations.
An embodiment according to the present invention, the Calibration Field are indoor optical targets.
It can define three global coordinate systems of camera：World coordinate system, camera coordinates system and image coordinate system.Image coordinate system is
By direct coordinate system of the image upper left corner for origin foundation in units of pixel.Camera coordinates system be using camera photocentre as origin,
X, Y-axis is respectively at image coordinate system X, the parallel coordinate system of Y-axis.World coordinate system, i.e. space coordinates are to describe phase
The position of machine and be introduced into, the space coordinates under entire three dimensions.An embodiment according to the present invention is rectified described
In positive step S3：
Automation Harris Corner Detections are carried out to each fish-eye image of acquisition, detect own in gridiron pattern
X-comers and image preset value X, Y-coordinate axle and coordinate origin；
Point P (x, y, z) is provided under world coordinate system, P points are projected as P ' (u ', v ') under camera coordinates system, in image
P ' (u ', v ') is projected as under coordinate system, and is set, then have three dimensions spot projection to fisheye image coordinate
The projection mapping relation of system is：
Wherein f (u ', v ') be correction function, f (u ', v ')=a0+a1 ρ '+...+aN ρ '
Wherein ai, i=0,1,2 ... N, is distortion parameter, and ρ ' ＞ 0 are distance of the point coordinates to sensor axis；With
The centre coordinate of each image of solution, and the centre bit with originally setting are iterated to calculate with linear estimation methods
It puts and compares, carry out estimation error.
An embodiment according to the present invention, in the step S4 and S5：
Point P (x, y, z) under world coordinate system is provided, by world coordinate system structure in the plane of Z=0, then there are P points to arrive
Point P ＇'s is mapped as
Wherein A be fish-eye inner parameter, [r1 r2 r3T] it is equivalent to [R t], represent world coordinate system and image
Position transformational relation between coordinate system, R represent 3 × 3 orthogonal spin matrix, and t represents D translation vector, and H is homography
Matrix；
Inner parameter matrix A is solved, by homography matrix H=[h1 h2 h3]=A [r1 r2T], export is following two etc.
Formula
Combined by above-mentioned two equation and solved, obtain different fish-eye inner parameters；
Using the inner parameter acquired, to estimate fish-eye external parameter；With
Non-linear LM optimizations are carried out at the same time to fish-eye inner parameter and external parameter, minimize the mistake of re-projection
Difference, so that the posture changing solution of camera lens is more accurate.
An embodiment according to the present invention, the rectification step S3 include：
Automatic Corner Detection is carried out to the image to be corrected that the multiple fish eye lens obtains；
By the use of the angle point detected automatically as control point, by multinomial globular projection model to the multiple fish eye lens
It is calibrated；
The center of each image, and the center that opposite original image is set is obtained in alternative linear method iteration
Coordinate carries out further iteration and precisely corrects.
An embodiment according to the present invention, further includes, and calculates the homography matrix of image；To any two fish eye lens into
Row spatial reference point matches.
An embodiment according to the present invention, the inner parameter include it is following at least one of：Fish-eye X, Y-axis
Equivalent focal length, the inclined degree of prime focus position and reference axis.
According to an aspect of the present invention, Calibration Field and/or more fish eye lens panorama cameras are contemplated that when designing calibration software
The factors such as hardware.
According to an aspect of the present invention, design splicing software when be contemplated that Calibration Field, more fish eye lens panorama cameras it is hard
The factors such as part and/or calibration software.
More embodiments of the present invention can also realize other Advantageous techniques effects do not listed one by one, these other skills
Art effect may hereinafter have part description, and be after the present invention has been read for a person skilled in the art can
With expected and understanding.
Present invention part is intended to be introduced into simplified form and following article will further be retouched in " specific embodiment "
The design and selection stated, to help reader that the present invention is more readily understood.Present invention is not intended to identify claimed
Theme key feature or essential characteristic, be also not intended to limit the scope of theme claimed.All is upper
It states feature all to will be understood as only being exemplary, and is collected in being disclosed from the present invention on the more of structures and methods
Feature and purpose.More fully displaying to the feature of the present invention, details, practicability and advantage, will be below to this hair
It is provided in the written description of bright various embodiments, illustrates, and limit in the following claims in the accompanying drawings.Therefore, such as
Fruit does not read entire disclosure and claims and attached drawing further, then can not understand many restrictions to present invention
Property explain.
Description of the drawings
By reference to following description together with attached drawing, the features described above and advantage and other feature and advantage of these embodiments
And realize that their mode will be apparent from, and the embodiment of the present invention may be better understood, in the accompanying drawings：
Fig. 1 illustrates the side views of the geometry appearance structure of the panorama camera of four fish eye lens compositions.
Fig. 2 illustrates the top view of the geometry appearance structure of the panorama camera of four fish eye lens compositions.
Fig. 3 illustrates the particular flow sheet of correction and the calibration of panorama camera according to an embodiment of the invention.
Fig. 4 illustrates automatic straightening module specific implementation method.
Fig. 5 illustrates calibrating external parameters specific implementation method according to an embodiment of the invention.
Specific embodiment
Below in the description of the drawings and specific embodiments, will illustrate the thin of one or more embodiments of the invention
Section.From these description, attached drawing and claims, other features, objects, and advantages of the present invention can be understood.
It should be appreciated that illustrated and description embodiment is not limited to illustrate in the following description in the application or in the accompanying drawings
The construction of the component of diagram and the details of arrangement.Shown embodiment can be other embodiments, and can be with various
Mode is practiced or carried out.Each example is provided by explaining to the disclosed embodiments and unrestricted mode.It is actual
On, will be apparent to one skilled in the art is, can be in the case of without departing substantially from scope disclosed by the invention or essence
To various embodiments of the present invention, various modification can be adapted and modification.For example, it illustrates or describes as the part of one embodiment
Feature, can be used together with another embodiment, still to generate other embodiment.Therefore, the present invention, which discloses, covers category
Such modifications and variations in the range of appended claims and its equivalent elements.
Equally, it is to be understood that phrase and term used herein are in order at the purpose of description, and should not be regarded as
It is restricted.For example, " comprising " herein, "comprising" or the use of " having " and its modification, it is intended to openly include
Thereafter the item and its equivalent item and additional item listed.Unless otherwise limitation, otherwise, term herein " connection ", " connection
Connect " and its modification be widely used, and include mechanical and/or electrical connection directly or indirectly, connection.In addition, term
" inner parameter " and " external parameter " is sometimes referred to as " internal reference " and " outer ginseng ".Term " correction ", " calibration " and " correction " are
It is used interchangeably.Term " fish eye lens " and " fisheye camera " are interchangeably used in the present invention, and term " panorama
Camera " refers to that the panorama camera comprising multiple " fish eye lens (cameras) " or panoramic camera are whole, and in this application
" panoramic camera ", " panorama camera " can be used interchangeably." splicing software " sometimes can with " correction and splicing software "
To be used interchangeably.
In addition, " fish eye lens " should make broadest understanding with the angle of those skilled in the art, it includes visual angle
It can adapt to close or equal to 180 ° in any kind of wide-angle lens of the present invention.
It is more come assembly and adjustment by software and combination of hardware to the present invention below with reference to specific embodiments of the present invention
The method of fish eye lens panorama camera carries out more detailed illustrate and explanation.
An embodiment according to the present invention first, establishes a fixed Calibration Field that can be used for measuring and demarcating environment,
Calibration Field will be suitble to the performance for the more fish eye lens panorama cameras that will be produced, wherein each mirror of more fish eye lens panorama cameras
Head must be able to take clearly calibration object picture.
Panorama camera hardware is designed according to the needs of client or application scenario, wherein determining for example each in panorama camera
A lens performance parameter and placement position.
Calibration software is designed according to the Calibration Field of above-mentioned foundation and designed panorama camera hardware.Calibration software is pin
Measurement and calibration software to above-mentioned fixed Calibration Field and the design of more fish eye lens panorama cameras.The calibration software is as production
The inspection software used in the process is run in the computer used in producing detection process.
According to above-mentioned panorama camera hardware, Calibration Field and calibration software, to design splicing software.Splice software preferably with
The mating correction of calibration software and splicing software.The splicing software splices software in real time as video, preferably in panorama camera
Used in hardware platform, but can also be used as post-production software, mobile phone, computer beyond panorama camera hardware platform or
Other support to run on the platform of this running software.
Afterwards, according to design come the more fish eye lens panorama cameras of production definition, wherein, by software and combination of hardware come group
Adjustment tries more fish eye lens panorama cameras, completes for example following nonrestrictive process example：More fish eye lens panorama cameras
The assembling of element supplied materials detection-each fish eye lens itself-fish-eye focusing-fish-eye Function detection-panorama
Camera assembling, functional test and the installation for splicing software.
One or several fixed positions that a panorama camera for completing functional test is placed in Calibration Field are used
Its several camera lens separately or simultaneously shoots the calibration object in Calibration Field, and shooting picture input calibration software is counted
It calculates, so as to obtain the error between the optical parameter of each camera lens of panorama camera and relative position parameter and design theory value
Value, while calculate the distortion value for the pixel that for example each camera lens collects.After the completion of calculating, calibration software counts these
Value is input in the memory of the such as operating environment of this panorama camera and preserves, and is called to splice software.
Afterwards, this panorama camera can be opened, is allowed in running order, is drawn by means of the final output of panorama camera
Face, to detect and/or debug the splicing effect of splicing software.
It is completed in above-mentioned assembly and adjustment, after the assay was approved, it is soft that the panorama camera splicing before packaging the dispatching from the factory of shipment can be obtained
Part.
Therefore, using an exemplary embodiment of the method for the present invention, the production process of panorama camera can be reduced to wrap
It includes：Supplied materials detects；The assembling of single camera lens and Function detection；Assembling, the functional test of panorama camera；Splice the installation of software；
Panorama camera is demarcated；And finished product detection.Above-mentioned production process can omit camera lens and camera compared with traditional mode of production process
The processes such as detection grouping, the positioning adjustment of a plurality of lenses, while also avoid because the subjective errors such as grouping or positioning adjustment
Caused by product it is bad, therefore reduce production process, improve production efficiency.
In the following, by with reference to attached drawing to the present invention by software with combination of hardware come the more fish eye lens panorama phases of assembly and adjustment
The method of machine carries out more detailed illustrate and explanation.
Fig. 1 illustrates the geometry appearance structures for the panorama camera that this four field angles form for the fish eye lens of 180 degree
Side view.Fig. 2 illustrates the top view of the geometry appearance structure of the panorama camera of four fish eye lens compositions.
As depicted in figs. 1 and 2, carried out exemplified by the panorama camera that the fish eye lens by four field angles for 180 degree forms
Illustrate, in order to make to have between four fish eye lenses enough or say as far as possible more overlapping visual angles, by four fishes of panorama camera
Glasses head is arranged as shown in Fig. 1, wherein, which is arranged in four faces center of triangular pyramid, and every
Angle between two adjacent fish eye lenses is 120 degree.It, can be by interior in the panorama camera arranged with this fish eye lens
Portion's hardware controls four fish eye lenses (or for " fisheye camera ") to synchronize shooting, so as to meet to around 360
Degree environment is carried out at the same time the demand of shooting photo and photography.
Fig. 3 illustrates the flow chart of the specific embodiment of panorama camera correction and calibration according to an embodiment of the invention.
According to one embodiment of present invention, the panorama camera of four fish eye lenses composition with reference to shown in such as figure 1 above -2 is complete to this
The method that scape camera is corrected and demarcates is described below.
As shown in Figs. 1-3, in the present invention by software and combination of hardware come the more fish eye lens panorama cameras of assembly and adjustment
Method an embodiment in, complete the element supplied materials detections of more fish eye lens panorama cameras, the assembling of fish eye lens itself,
Fish-eye focusing, after fish-eye Function detection, carry out the assemblings of more fish eye lens panorama cameras, functional test and
Splice the installation of software.
It can be before the assembling, functional test and the installation of splicing software of more fish eye lens panorama cameras is completed, Zhi Houhuo
Person simultaneously, in step sl, establish panorama camera for the target field (Calibration Field) correcting and demarcate.
In step s 2, (shooting) four fish-eye images are obtained while.
After step S1, S2, in step s3, preferred spherical projection is carried out to the image of fish eye lens acquisition and is rectified
Just.
In step s 4, proving operation is carried out to the image after correction.For example, it is preferable to fish-eye inside and outside portion is joined
Number is estimated.
In step s 5, the inner parameter to the image after correction and external parameter (such as image Attitude estimation parameter) into
Row optimization is preferably carried out at the same time binding optimization.
Finally, in step s 6, the required evolution between different fish eye lenses is obtained.
Rectification step S3 is described in detail below.
Fig. 4 illustrates a specific embodiment of automatic straightening module.As shown in figure 4, for example, in rectification step S3,
Using a kind of polynomial parameters correct method, overcome fish eye lens due to spherical projection be not applied for common aperture into
As the shortcomings that model.
Furthermore, in this example, first, such as four fish-eye images to be corrected can be read in respectively,
Then can image to be corrected fish-eye to this four carry out automatic Corner Detection, this automatic detection function compensates for existing
The shortcomings that needing manually to mark in some Corner Detections, while time of the correction chart as used in can be saved.It afterwards, can be again
By the use of the angle point detected automatically as control point, fish eye lens is calibrated by multinomial globular projection model.Finally, may be used
The center of each image is obtained in alternative linear method iteration, and the center position coordinates that opposite original image is set carry out
Further iteration precisely corrects, so as to calculate four fish-eye inner parameters such as focal length, prime focus.
It can define three global coordinate systems of camera：World coordinate system, camera coordinates system and image coordinate system.Image coordinate system is
By direct coordinate system of the image upper left corner for origin foundation in units of pixel.Camera coordinates system be using camera photocentre as origin,
X, Y-axis is respectively at image coordinate system X, the parallel coordinate system of Y-axis.World coordinate system, i.e. space coordinates are to describe phase
The position of machine and be introduced into, the space coordinates under entire three dimensions.
For example, in this example, as described below：
The image to the different postures of scaling board is obtained respectively by four fish eye lenses.
All images are carried out with automation Harris Corner Detections, detects X-comers and figure all in gridiron pattern
As the X of preset value, Y-coordinate axle and coordinate origin.
In world coordinate system lower set point P (x, y, z), wherein P points are projected as P ' (u ', v ') under camera coordinates system,
P ' (u ', v ') is projected as under image coordinate system, and sets u '=[u ', v '], then three dimensions point (P ') projects to fish eye lens figure
As the projection mapping relation of coordinate system is：
Wherein f (u ', v ') be correction function, f (u', v')=a0+a1ρ'+…+aNρ'
Wherein ai, i+0.12...N is distortion parameter, and ρ ' ＞ 0 are distance of the point coordinates to sensor axis
Useable linear method of estimation solves the centre coordinate of each image, and the center with originally setting to iterate to calculate
Position compares, and carries out estimation error.
The demarcating steps S4 of the present invention is described in detail below in conjunction with specific embodiment.
Fig. 5 illustrates the external parameter according to an embodiment of the invention to the panorama camera being made of multiple fish eye lenses
The specific embodiment for the method that (such as image posture) is demarcated.As shown in figure 5, it discloses to being made of multiple fish eye lenses
A kind of method and step S4 for being demarcated of panorama camera.After fish eye lens Calibration Field is arranged, the homography of image can be calculated
Matrix；Spatial reference point matching is carried out to any two fish eye lens, and external ginseng is carried out according to the image coordinate point to match
Number (such as posture) optimization；Finally, the posture changing R and T between different fish eye lenses are calculated.
For example, it can be realized for example, by step in detail below：
By world coordinate system structure in the plane of Z=0, then there are P points to be mapped as to P '
Wherein A is fish-eye inner parameter, may include the X of camera lens, the equivalent focal length of Y-axis, prime focus position, phase
Inclined degree for reference axis etc. parameter；[r1 r2 r3T] it is equivalent to [R t], represent world coordinate system and image coordinate
Position transformational relation between system, wherein R represent 3 × 3 orthogonal spin matrix, and t represents D translation vector, and H is homography
Matrix.
Inner parameter matrix A is solved, by homography matrix H=[h1 h2 h3]=A [r1 r2T], it can export following two
Equation
Joint solves, you can obtains different fish-eye inner parameters.
Using the inner parameter acquired, to estimate fish-eye external parameter.For there is similar spatial reference
The different images of point, carry out solution homography matrix, obtain the rotation and translation transformation relation of camera lens.
Non-linear LM optimizations can be carried out at the same time to fish-eye inner parameter and external parameter, minimize the mistake of re-projection
Difference, so that the conversion solution of the posture of camera lens is more accurate.
After correction, calibration and optimization, these numerical value are input to and complete above-mentioned correction, calibration and optimization by calibration software
It preserves in memory in the such as operating environment of panorama camera, is called to splice software.Afterwards, this completion can be opened
Calibration and the panorama camera of optimization, are allowed in running order, by means of the final output picture of panorama camera, come detect and/
Or the splicing effect of debugging splicing software.
It is completed in above-mentioned assembly and adjustment, after the assay was approved, it is soft that the panorama camera splicing before packaging the dispatching from the factory of shipment can be obtained
Part.
As described above, the present invention by software and combination of hardware come the method for the more fish eye lens panorama cameras of assembly and adjustment
Advantage is such as, but not limited to as described below.
It is avoided in software and hardware combining assembly and adjustment process in the present invention because grouping or positioning adjustment etc. are subjective by mistake
Product is bad caused by difference so that product yield can reach about more than 98%.
The detection grouping of camera lens and camera, multiple cameras are avoided in software and hardware combining assembly and adjustment process in the present invention
Positioning adjustment etc. processes so that production efficiency can improve about 2-3 times.
Software and hardware combining assembly and adjustment process in the present invention makes it possible to reduce by way of software and combination of hardware
Performance requirement of the software to hardware platform, so as to reduce product cost indirectly.
Several embodiments being described above to the present invention is proposed for purposes of illustration.It is described to be described above
Exhaustion is not intended to be, also not limits the invention to disclosed exact procedure and/or form, it is clear that according to above
Introduction, can many modifications may be made and modification.The scope of the present invention and all equivalent are intended to be defined by the following claims.
It is described in detail above in association with specific embodiment of the attached drawing for the method for the present invention.But this field
It will be appreciated by the skilled person that the above is merely illustrative and describes some specific embodiments, to the scope of the present invention
And without any restrictions.The scope of the invention is only defined by the appended claims.
Claims (10)
1. a kind of method by the more fish eye lens panorama cameras of software and hardware combining assembly and adjustment, including：
It designs and multiple hardware is provided, and assemble them into more fish eye lens panorama cameras, wherein the design determines more flakes
The theoretical parameter of camera lens panorama camera；
Establish the fixed Calibration Field measured for more fish eye lens panorama cameras with demarcating；
Design calibration software, and using calibration software to the inner parameters of more fish eye lens panorama cameras and external parameter into
Rower is determined, and obtains the error amount between the actual parameter of more fish eye lens panorama cameras and corresponding theoretical parameter, and will
The error amount is input in the memory of more fish eye lens panorama cameras or peripheral operation platform and preserves；
Design splicing software；With
The error amount preserved by the splicing software transfer, carries out the final debugging of more fish eye lens panorama cameras.
2. the method according to claim 1 by the more fish eye lens panorama cameras of software and hardware combining assembly and adjustment, wherein,
After the debugging, the packaging before finished product detection and shipment is carried out.
3. pass through the more fish eye lens panorama cameras of software and hardware combining assembly and adjustment according to any one of the claims
Method, wherein, the splicing software is the correction mating with calibration software and splicing software, complete for more fish eye lenses
The correction in real time of the video of scape camera and splicing.
4. pass through the more fish eye lens panorama cameras of software and hardware combining assembly and adjustment according to any one of the claims
Method, wherein, it is described splicing software installation in the memory of more fish eye lens panorama cameras or mounted on described more
Mobile phone, computer beyond fish eye lens panorama camera or other support on the platform of this running software.
5. pass through the more fish eye lens panorama cameras of software and hardware combining assembly and adjustment according to any one of the claims
Method, wherein, the step of more fish eye lens panorama cameras for providing multiple hardware and being assembled, includes, to more fish eye lenses
The element of panorama camera carries out supplied materials detection, and the fish eye lens of more fish eye lens panorama cameras is assembled in itself and function is examined
It surveys and each hardware composing into more fish eye lens panorama cameras and is completed into functional test.
6. pass through the more fish eye lens panorama cameras of software and hardware combining assembly and adjustment according to any one of the claims
Method, wherein, the theoretical parameters of more fish eye lens panorama cameras includes inner parameter and external parameter.
7. pass through the more fish eye lens panorama cameras of software and hardware combining assembly and adjustment according to any one of the claims
Method, further include, to the error amount between the actual parameter of more fish eye lens panorama cameras and corresponding theory parameter into
Row optimization.
8. the method according to claim 6 by the more fish eye lens panorama cameras of software and hardware combining assembly and adjustment, wherein,
The inner parameter include it is following at least one of：The equivalent focal length of fish-eye X, Y-axis, prime focus position and coordinate
The inclined degree of axis；
The external parameter characterizes the position relationship between the fish eye lens of more fish eye lens panoramic cameras.
9. pass through the more fish eye lens panorama cameras of software and hardware combining assembly and adjustment according to any one of the claims
Method, wherein, the error amount include it is following at least one of：
Error amount between each fish-eye optical parameter and design theory value；
The error amount between actual relative position parameter and design theory value between fish eye lens；With
The optical distortion parameter value for the pixel that each fish eye lens collects.
10. pass through the more fish eye lens panorama cameras of software and hardware combining assembly and adjustment according to any one of the claims
Method, wherein, the method omits the detection grouping process of fish eye lens and panorama camera and fish-eye position tune
Whole process.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201810032504.XA CN108109111A (en) | 2018-01-12 | 2018-01-12 | Pass through the method for the more fish eye lens panorama cameras of software and hardware combining assembly and adjustment |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201810032504.XA CN108109111A (en) | 2018-01-12 | 2018-01-12 | Pass through the method for the more fish eye lens panorama cameras of software and hardware combining assembly and adjustment |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN108109111A true CN108109111A (en) | 2018-06-01 |
Family
ID=62218982
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201810032504.XA Pending CN108109111A (en) | 2018-01-12 | 2018-01-12 | Pass through the method for the more fish eye lens panorama cameras of software and hardware combining assembly and adjustment |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN108109111A (en) |
Cited By (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN108985213A (en) * | 2018-07-09 | 2018-12-11 | 企鹅创新（北京）科技有限公司 | Posture judges control method and system |
| CN109166152A (en) * | 2018-07-27 | 2019-01-08 | 深圳六滴科技有限公司 | Bearing calibration, system, computer equipment and the storage medium of panorama camera calibration |
| CN109166078A (en) * | 2018-10-22 | 2019-01-08 | 广州微牌智能科技有限公司 | Panoramic view joining method, device, viewing system, equipment and storage medium |
| CN111507894A (en) * | 2020-04-17 | 2020-08-07 | 浙江大华技术股份有限公司 | Image splicing processing method and device |
-
2018
- 2018-01-12 CN CN201810032504.XA patent/CN108109111A/en active Pending
Cited By (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN108985213A (en) * | 2018-07-09 | 2018-12-11 | 企鹅创新（北京）科技有限公司 | Posture judges control method and system |
| CN109166152A (en) * | 2018-07-27 | 2019-01-08 | 深圳六滴科技有限公司 | Bearing calibration, system, computer equipment and the storage medium of panorama camera calibration |
| CN109166078A (en) * | 2018-10-22 | 2019-01-08 | 广州微牌智能科技有限公司 | Panoramic view joining method, device, viewing system, equipment and storage medium |
| CN111507894A (en) * | 2020-04-17 | 2020-08-07 | 浙江大华技术股份有限公司 | Image splicing processing method and device |
| CN111507894B (en) * | 2020-04-17 | 2023-06-13 | 浙江大华技术股份有限公司 | Image stitching processing method and device |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN108364252A (en) | A kind of correction of more fish eye lens panorama cameras and scaling method | |
| CN109272478B (en) | Screen projection method and device and related equipment | |
| CN108257183B (en) | Camera lens optical axis calibration method and device | |
| WO2018076154A1 (en) | Spatial positioning calibration of fisheye camera-based panoramic video generating method | |
| CN108805801A (en) | A kind of panoramic picture bearing calibration and system | |
| Schneider et al. | Validation of geometric models for fisheye lenses | |
| CN108109111A (en) | Pass through the method for the more fish eye lens panorama cameras of software and hardware combining assembly and adjustment | |
| CN105096329B (en) | Method for accurately correcting image distortion of ultra-wide-angle camera | |
| US20170127045A1 (en) | Image calibrating, stitching and depth rebuilding method of a panoramic fish-eye camera and a system thereof | |
| CN109615663A (en) | Panoramic video bearing calibration and terminal | |
| CN109272570A (en) | A kind of spatial point three-dimensional coordinate method for solving based on stereoscopic vision mathematical model | |
| CN105759435B (en) | Overall view ring belt stereo imaging system and its scaling method and imaging method | |
| WO2021212978A1 (en) | Calibration method, calibration apparatus, and non-volatile computer-readable storage medium | |
| US7038712B1 (en) | Geometric and photometric calibration of cameras | |
| CN102143305B (en) | Image pickup method and system | |
| CN107527336B (en) | Lens relative position calibration method and device | |
| CN106612390B (en) | Camera module with double imaging modules and optical axis parallelism adjusting method thereof | |
| CN106056539A (en) | Panoramic video splicing method | |
| CN111461963B (en) | Fisheye image stitching method and device | |
| US10821911B2 (en) | Method and system of camera focus for advanced driver assistance system (ADAS) | |
| CN110505468B (en) | Test calibration and deviation correction method for augmented reality display equipment | |
| CN113920206B (en) | Calibration method of perspective tilt-shift camera | |
| CN106886976B (en) | Image generation method for correcting fisheye camera based on internal parameters | |
| CN111899305A (en) | Camera automatic calibration optimization method and related system and equipment | |
| CN108269234A (en) | A kind of lens of panoramic camera Attitude estimation method and panorama camera |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| WD01 | Invention patent application deemed withdrawn after publication |
Application publication date: 20180601 |
|
| WD01 | Invention patent application deemed withdrawn after publication |