CN105844624A - Dynamic calibration system, and combined optimization method and combined optimization device in dynamic calibration system - Google Patents

Abstract

The invention provides a dynamic calibration system, and a combined optimization method and a combined optimization device in the dynamic calibration system. The dynamic calibration system comprises a marker geometry feature extraction module, a camera parameter estimation module and a calibrate result evaluation module; the marker geometry feature extraction module detects a marker in the image shot by the camera, extracts geometry features of the marker, and traces and matches the extracted geometry features on images of follow-up frames which are provided by the camera; the camera parameter estimation module is used for obtaining the error of repeated projection according to the extracted geometry features of the marker and current poses of various cameras and obtaining a calibration result when the repeated projection error satisfies a set error threshold; all cameras are positioned in initial position states during the first operation; and the calibration result evaluation module is used for evaluating the accuracy of the calibration result and determining whether to receive parameters of the calibration result. The calibration parameter of the invention has relatively high accuracy, and the dynamic calibration system, the combined optimization method and the combined optimization device in the dynamic calibration system are applicable to the dynamic streamline calibration, high in execution efficiency, wide in application range and simple in system.

Description

Combined optimization method in dynamic calibration system, dynamic calibration system and device

Technical field

The present invention relates to automotive electronics and vehicle mounted guidance field, in particular it relates to a kind of dynamic calibration system, dynamic calibration Combined optimization method in system and device.

Background technology

Vehicle-mounted panoramic viewing system is by installing multiple fisheye cameras at vehicle's surroundings, and uses in-vehicle processor to obtaining Multi-channel video carry out image procossing, the visual field of vehicle-surroundings is provided for driver, thus improves the safety of driving.Panorama ring Viewing system is owing to comprising multiple camera (general >=4).

Before and after demarcation, geometric position and the relation of distance scale between camera are obvious on splicing result impact, therefore, and camera Between and camera and car between pose to demarcate be a key link of system, precision and the expansion of demarcating link are straight Connect effect and the convenience determining panoramic looking-around system.For different scenes, vehicle-mounted panoramic viewing system typically uses not With scaling method, such as when vehicle on a production line time, now positioner can be used on fixing station (such as four-wheel Location equipment), it is accurately determined the position of vehicle, it is ensured that the stated accuracy of viewing system, and simplifies its calibration process.

In actually used, it is contemplated that the availability under different operating environment of scaling scheme, also needing to add tool can expand The scaling scheme of property, the method is independent of positioner, can be used for the demarcation of 4S shop or dynamic pipeline.

Precision, efficiency and expansion are the subject matter that panoramic looking-around scaling scheme needs to consider.Existing scaling scheme In, segmentation scheme stated accuracy is not good enough, have impact on the overall performance of panoramic looking-around system；Segmentation scheme execution efficiency is on the low side, no It is applicable to the occasion that the requirement of real-times such as assembling line are higher；Part scaling scheme excessively relies on the auxiliary equipment of complexity, limit Make its expansion under various operating conditions.

Through retrieval, find following coordinate indexing result.

Coordinate indexing result 1:

Publication number: CN104240258A

Title: a kind of panoramic looking-around system calibrating method based on car networking, Apparatus and system

Summary: the invention discloses a kind of panoramic looking-around system calibrating method based on car networking, Apparatus and system, belong to Technical field of automotive electronics.The method includes: car networked terminals sends image capture instruction so that master controller obtains demarcates work The fault image of multiple camera collections of vehicle to be calibrated configuration on position；The fault image that car networked terminals will be gathered is logical Cross network to send to car networked platforms so that this car networked platforms, after vehicle leaves current station, is demarcated, obtain and demarcate Parameter, and then make this master controller according to calibrating parameters, this fault image is processed, obtains vehicle-surroundings panorama sketch Picture；Vehicle is left for next station by car networked terminals prompting driver.The present invention networks based on car, only need to adopt demarcating station Collection image, transmission image, demarcate on backstage.To save in the nominal time demarcating station, improve the production efficiency of depot； Save the graphic transmission equipment needed for field calibration and demarcation personnel, reduce cost

Technical essential compares:

This patent documentation is a kind of static demarcating method, needs station to have four-wheel aligner equipment accurately.And the present invention is A kind of dynamic calibrating method, completes to demarcate in vehicle travel process, need not position equipment.

Coordinate indexing result 2:

Publication number: CN104851076A

Title: for panoramic looking-around parking assisting system and the photographic head installation method of commercial car

Summary: the present invention provides a kind of panoramic looking-around parking assisting system for commercial car and method of parking, including side The position angle of visual field is at least fish-eye camera, data collection processor, memorizer and the display of 180 degree, fish-eye camera collection Video pictures carry out processing through data collection processor and obtain the image of commercial car and area-of-interest thereof and be stored in memorizer also Shown by display.The inventive method by after demarcating first by pixels all in panorama sketch with in photographic head home position Respective pixel coordinate is set up during mapping relations are stored in the memory element of system one to one, when after an image frame grabber, is System can be quickly generated panorama sketch according to the data previously generated in the memory unit by control unit, after to splicing Panorama sketch be weighted averagely merging after and show on screen, for commercial car park offer panoramic looking-around effect preferable Aid, and the process time of system can be greatly shortened, meet the requirement of system real time

Technical essential compares:

This patent documentation is also a kind of static demarcating method, needs station to have four-wheel aligner equipment accurately.And the present invention It is a kind of dynamic calibrating method, completes to demarcate in vehicle travel process, equipment need not be positioned.

Summary of the invention

For defect of the prior art, it is an object of the invention to provide the associating in a kind of calibration system, calibration system Optimization method and device.

According to the combined optimization method in the dynamic calibration system that the present invention provides, comprise the steps:

Pose figure construction step: build the pose figure of each road camera according to dynamic calibration system in real time；

Specifically, set up pose figure according to spatial point BFS, with the physical coordinates X (design of scene markers point Value) it is root node V, the camera pose located the most in the same time is Pt, Pt=Tt*P, and wherein P is the outer ginseng that camera is actual, when Tt is t Carve the camera motion relative to initial point.V is the edge e mapping formation pose figure of Pt at different positions and pose, and each of the edges e is optimizing letter Representing a constraint equation in number, therefore the quantity of constraint equation can draw according to the number of edges on pose figure, and optimizes mesh Target quantity can draw according to root node quantity, then the practical structures of majorized function can determine according to pose figure.It addition, can also Enough carry out dynamic node additions and deletions the most at any time, reflect in applying to reality and be i.e. equal to dynamically change optimization mesh Mark and constraint equation quantity.

Initial alignment step: gather the image projection point under the camera of labelling Dian Ge road by pose figure, obtain to be optimized Calibrating parameters set；

Specifically, calibrating parameters set to be optimized includes { P, H, X}.

Calibrating parameters optimization step: use single channel-single frames method or multichannel-multi-frame method combined optimization calibrating parameters, Obtain the calibrating parameters optimized.

Preferably, also include:

Image mosaic step: according to the calibrating parameters of described optimization, spliced panoramic panoramic view picture；

Label Extraction of Geometrical Features step: the label in detection image shot by camera, extracts the several of described label What feature, and the geometric properties that tracking and matching is extracted on follow-up each two field picture that camera provides；

Camera parameter estimating step: for obtaining according to the geometric properties of label extracted and the initial pose of each camera Re-projection error, and to be optimized as in calibrating parameters optimization step of re-projection error when will meet specification error threshold value Calibrating parameters.

The combined optimization device in dynamic calibration system according to present invention offer, including:

Calibrating parameters optimizes module, i.e. uses single channel-single frames method or multichannel-multi-frame method combined optimization to demarcate ginseng Number, obtains the calibrating parameters optimized.

Preferably, described employing single channel-single frames or multichannel-multi-frame method are to carry out based on pose figure；Specifically,

The computing formula of single channel-single frames optimization method is as follows:

$\{\hat{P},\hat{H},{\hat{X}}_n\}=\underset{\{P,H,X\}}{\mathrm{argmin}}\{\frac{1}{2N}\underset{n\∈N}{\mathrm{\Σ}}\[{(u_n-{\mathrm{PX}}_n)}^2+{(u_n-{\mathrm{HX}}_n)}^2\]\};$

u_n=(u_n,v_n)^T；

X_n=(x_n,y_n,z_n)^T；

P=[R, t]；

$H=R-\frac{{\mathrm{tn}}^T}{d};$

In formula:Represent the output set optimized, as the calibrating parameters of described optimization；P, H, X} represent treat excellent The calibrating parameters set changed；P represents 3D projection matrix；Represent the 3D projection matrix optimizing gained；H represents plane shock wave square Battle array；Represent the plane shock wave matrix optimizing gained；X represents labelling space of points coordinate；Symbol ∧ on letter represents optimization Target；Represent the geometric markers point coordinates optimizing gained；Subscript T represents transposition；D represents the depth of field；u_nRepresent the n-th geometry mark The characteristics of image point coordinates that note point is corresponding in a single road image；u_nRepresent that the n-th geometric markers point is single one The row-coordinate of image characteristic point corresponding in the image of road；v_nRepresent the n-th geometric markers point institute in a single road image The row coordinate of corresponding image characteristic point；X_nRepresent n-th geometric markers point coordinate in reference frame；x_nRepresent n-th Geometric markers point x-axis coordinate in reference frame；y_nRepresent that n-th geometric markers point y-axis in reference frame is sat Mark；z_nRepresent n-th geometric markers point z-axis coordinate in reference frame；R is spin matrix, represents that camera is with reference to sitting Three axles under mark system rotate；

The computing formula of multichannel-multi-frame joint optimization method is as follows:

$\{\hat{P},\hat{H},\hat{X}\}=\underset{\{P,H,X\}}{\arg \min }\{\frac{1}{2TMN}\underset{t\∈T}{\mathrm{\Σ}}\underset{m\∈M}{\mathrm{\Σ}}\underset{n\∈N}{\mathrm{\Σ}}\[{(u_{tmn}-P_m{X}_{tn})}^2+{(u_{tmn}-H_m{X}_{tn})}^2\]\};$

P=[R, t]；

$H=R-\frac{{\mathrm{tn}}^T}{d};$

In formula:Represent the labelling space of points coordinate optimizing gained；T express time nodes, M represents camera number, and N represents Geometric markers is counted；T is translation vector, represents the camera three axle translations under reference frame；M represents camera sequence number；N represents Geometric markers point sequence number；u_tmnRepresent the n-th image characteristic point in the gathered image of m-th camera under t；P_mRepresent m The 3D projection matrix at individual visual angle；X_tnRepresent the labelling point 3D coordinate of n-th image estimated under t；H_mRepresent m The plane shock wave matrix at individual visual angle.

Preferably, also include:

Image mosaic module, for the calibrating parameters according to described optimization, spliced panoramic panoramic view picture；

Label Extraction of Geometrical Features module, for detecting the label in image shot by camera, extracts described label Geometric properties, and the geometric properties that extracted of tracking and matching on follow-up each two field picture that camera provides；

Camera parameter estimation module, for obtaining according to the geometric properties of label extracted and the initial pose of each camera Obtain re-projection error, and using re-projection error when meeting specification error threshold value as calibrating parameters to be optimized.

Preferably, also include:

Calibration result evaluation module, for evaluating the accuracy of calibration result, it may be judged whether accept the parameter of calibration result.

Preferably, described label Extraction of Geometrical Features module includes:

Detection module, for detection from the image of operative scenario for the label demarcated；

Extraction of Geometrical Features module, for extracting a certain or multinomial several of this label on the label of detection gained What feature；

Matching module, for following the tracks of and mate the label geometry spy of extraction on follow-up each two field picture of camera shooting Levy.

Preferably, described camera parameter estimation module includes:

Re-projection error acquisition module, for the geometric properties extracted according to each image shot by camera, and combines each The current pose parameter of individual camera, it is thus achieved that label misses at current spatial coordinate and the image projection under Current camera pose Difference, i.e. re-projection error；

Judge module, for judging the size of re-projection error, when re-projection error is less than or equal to the error set During threshold value, estimate the locus of each label according to described re-projection error, it is thus achieved that calibration result, activate calibration result and carry Delivery block；When re-projection error is more than the error threshold set, activates current pose and update module；

Current pose updates module, is used for the estimation position according to each label and combines each image shot by camera institute The geometric properties extracted updates the current pose parameter of each camera；Activate re-projection error acquisition module；

Calibration result extraction module, for extracting re-projection error when meeting specification error threshold value.

Preferably, the calibration result evaluated in described calibration result evaluation module refers to the panoramic looking-around image of splicing, logical Cross subjective observation and objective evaluation two ways assay calibration result is the most accurate；If calibration result is accurate, then accept also Output calibrating parameters；If calibration result is inaccurate, then reactivate label Extraction of Geometrical Features module；Wherein,

Subjective observation mode refers to: observe whether panoramic mosaic figure mates；

Objective evaluation mode refers to: the pose of installing combining each camera is evaluated with the discrimination standard set up.

The dynamic calibration system provided according to the present invention, including the combined optimization device in above-mentioned dynamic calibration system.

Compared with prior art, the present invention has a following beneficial effect:

1, the combined optimization method in the dynamic calibration system that the present invention provides, is provided with re-projection error threshold value so that The calibrating parameters of output has higher precision, and is applicable to the demarcation of dynamic streamline, and execution efficiency is high.

2, the combined optimization method in the dynamic calibration system that the present invention provides, uses the dynamic calibration method looked around, in real time Adjust the pose of camera, obtain panoramic mosaic figure clearly, applied range, and system simple.

Accompanying drawing explanation

By the detailed description non-limiting example made with reference to the following drawings of reading, the further feature of the present invention, Purpose and advantage will become more apparent upon:

Pose figure plan structure signal in combined optimization method in the dynamic calibration system that Fig. 1 provides for the present invention Figure；

The schematic flow sheet of the detailed description of the invention that Fig. 2 provides for the present invention；

The dynamic calibration system that Fig. 3 provides for the present invention is at the calibration process figure of 4S streamline；

The schematic flow sheet of the combined optimization method in the dynamic calibration system that Fig. 4 provides for the present invention.

Detailed description of the invention

Below in conjunction with specific embodiment, the present invention is described in detail.Following example will assist in the technology of this area Personnel are further appreciated by the present invention, but limit the present invention the most in any form.It should be pointed out that, the ordinary skill to this area For personnel, without departing from the inventive concept of the premise, it is also possible to make some deformation and improvement.These broadly fall into the present invention Protection domain.

Under the operative scenario such as streamline or 4S shop, the dynamic calibration system of panoramic looking-around system comprises following several mould Block:

Extraction of Geometrical Features module: from collected by camera to a frame operative scenario image detection for demarcate labelling Thing, extracts the particular geometric feature of this label, subsequently then in follow-up each frame operative scenario on the label of detection gained Image is followed the tracks of and is mated the particular geometric feature of extraction.Wherein, described particular geometric feature, can be the limit of label, angle, Areas etc., particular geometric feature reflects the design size of label, and the relative position relation between label.

Camera parameter estimation module: be used for being alternately performed step a, step b, iteration update each label estimation position and The pose parameter of each road camera, i.e. re-projection error minimizes；When the re-projection error of operative scenario image is less than predetermined threshold value Time, it is believed that each road current pose parameter of camera is close to actual value.Now carry out the multichannel of camera in viewing system and multiframe Combined optimization；Wherein:

Step a: first by each road collected by camera to operative scenario image in the particular geometric feature extracted, in conjunction with The initial designs pose of camera, according to the current spatial coordinate of the label image projection error under Current camera pose, i.e. Re-projection error, estimates the locus of each label in the demarcation place in operative scenario, is designated as the estimation position of label；

Step b: estimate that position combining in the operative scenario image that each road collects is carried according to each label subsequently The particular geometric feature taken updates the current pose parameter of each road camera.

The calibration system provided according to the present invention, including: four groups of cameras, multiple labels, described four groups of cameras are arranged on Four orientation, left and right before and after automobile, and automobile surrounding is provided with multiple label；Also include: label Extraction of Geometrical Features Module, camera parameter estimation module, calibration result evaluation module；

-described label Extraction of Geometrical Features module, for detecting the label in image shot by camera, extracts described mark The geometric properties of note thing, and the geometric properties that tracking and matching is extracted on follow-up each two field picture that camera provides；

-described camera parameter estimation module, current for according to the geometric properties of label extracted and each camera Pose obtains re-projection error, and obtains calibration result when re-projection error meets specification error threshold value；Wherein each camera It is in initial position and posture when running for the first time；

-described calibration result evaluation module, for evaluating the accuracy of calibration result, it may be judged whether accept calibration result Parameter.

The geometric properties of described label includes: the phase para-position between the limit of label, angle, area parameters and label Put relation.

The initial pose of camera includes: the three axle anglecs of rotation of camera, three axle photocentre coordinates, the focal length of camera lens, principal point Skew, distortion factor；Described initial pose can obtain by the way of inquiring about camera factory-said value or carrying out off-line rectification.

According to the combined optimization method in the calibration system that the present invention provides, comprise the steps:

Label Extraction of Geometrical Features step: the label in detection image shot by camera, extracts the several of described label What feature, and the geometric properties that tracking and matching is extracted on follow-up each two field picture that camera provides；

Camera parameter estimating step: for obtaining according to the geometric properties of label extracted and the initial pose of each camera Re-projection error, and obtain calibration result when re-projection error meets specification error threshold value；

Calibration result evaluation procedure: evaluate the accuracy of calibration result, it may be judged whether accept the parameter of calibration result.

Described label Extraction of Geometrical Features step includes:

Step A1: detection is for the label demarcated from the image of operative scenario；

Step A2: extract a certain or multinomial geometric properties of this label on the label of detection gained；

Step A3: follow the tracks of and mate the label geometric properties of extraction on follow-up each two field picture of camera shooting.

Described camera parameter estimating step includes:

Step B1: use the geometric properties that each image shot by camera is extracted, and combine the current pose of each camera Parameter, it is thus achieved that label is at current spatial coordinate and the image projection error under Current camera pose, i.e. re-projection error；

Step B2: judge the size of re-projection error, when re-projection error is less than or equal to the error threshold set, Estimate the locus of each label according to described re-projection error, it is thus achieved that calibration result, perform step B4；When re-projection misses When difference is more than the error threshold set, perform step B3；

Step B3: estimating position and combining the geometric properties that each image shot by camera is extracted according to each label Update the current pose parameter of each camera；Return and perform step B1；

Step B4: extraction re-projection error meets calibration result during specification error threshold value.

Specifically, corresponding pose figure need to be constructed and look around the corresponding relation of camera multichannel and multiframe with description, and based on position Appearance figure be optimized parameter transmission, the structure of pose figure (pose graph) based on BFS under signal source shortest path, I.e. dijkstra's algorithm.As it is shown in figure 1, round dot V represents the locus of labelling point, remaining round dot represents that labelling Dian Ge road is taken the photograph Camera gathers the image projection point under image；Solid line represents the 3d space point of labelling and the corresponding relation of 2D picture point, dotted line table The corresponding relation of 2D point between the image of Shi Ge road, these corresponding relations, by participating in optimizing with the form of constraint equation, are not gone the same way many Two field picture is corresponding to the collection image under different time points.

Further, under the world coordinate system specified, by the 3d space point of its physical coordinates known, and it is corresponding 2D picture point, estimate camera pose under world coordinate system.Therefore, labelling point is derived from the 3d space point of label, Namely the geometric properties point of reflection label geometric properties.Such as: the edge of label as geometric properties, can in the picture by Extract；The edge being extracted can be a series of picture point by discrete representation, and these picture point may correspond to physical world acceptance of the bid Series of points on note thing, these points itself come from the geometric properties (edge) of label, are therefore geometric properties points, and these are several What characteristic point represents this label, namely labelling point.

The pose figure of each camera designs accordingly for viewing system, can support multichannel, the combined optimization of multiframe, Its concrete frame number (can be performed time, convergence precision) according to the actual requirements by designer and be given, to guarantee efficiency and to improve optimization The motility of process.

The optimization process of the present invention is based on " recovering pose from corresponding point " (Pose from Corresponding) former Reason, but abandoned single employing plane shock wave (homography) and single employing perspective projection (perspective Projection) outer ginseng estimation mode, and have employed the two class modes that alternately use in same renewal process.Agreement herein Spin matrix R_3×3Representing that the camera three axles under reference frame rotate, translation vector t represents that camera is under reference frame Three axle translations, R^TRepresent camera towards, c represents the photocentre position of camera, i.e. c=-R^Tt。

Now make 3D projection matrix P, P=[R, t], plane shock wave matrix H,Subscript T represents transposition；This Locate desirable unit 1, n^TRepresentation unit matrix, the d expression depth of field, N number of geometric markers point (such as angle point) of label is in ginseng The coordinate examined in coordinate system (i.e. world coordinate system) is X_n=(x_n,y_n,z_n)^T, this N number of geometric markers point is at a single road video Characteristics of image point coordinates corresponding in is u_n=(u_n,v_n)^T.Wherein, u_nRepresent that the n-th geometric markers point is at a single road figure Characteristics of image point coordinates corresponding in Xiang；u_nRepresent the figure that the n-th geometric markers point is corresponding in a single road image Row-coordinate as characteristic point；v_nRepresent the image characteristic point corresponding in a single road image of the n-th geometric markers point Row coordinate；X_nRepresent n-th geometric markers point coordinate in reference frame；x_nRepresent that the n-th geometric markers point is with reference to sitting X-axis coordinate in mark system；y_nRepresent n-th geometric markers point y-axis coordinate in reference frame；z_nRepresent the n-th geometry mark Note point z-axis coordinate in reference frame；R is spin matrix, represents that camera is under reference frame (i.e. world coordinate system) Three axles rotate；

Now optimization aim is projection matrix P, homography matrix H, geometric markers point world coordinates X_n, the present invention uses The optimization of single channel-single frames be represented by:

$\{\hat{P},\hat{H},{\hat{X}}_n\}=\underset{\{P,H,X\}}{\mathrm{argmin}}\{\frac{1}{2N}\underset{n\∈N}{\mathrm{\Σ}}\[{(u_n-{\mathrm{PX}}_n)}^2+{(u_n-{\mathrm{HX}}_n)}^2\]\}---\left(1\right)$

On the basis of above formula, the form of multichannel-multiframe can be extended to, when timing node number is T, viewing system camera number For M (M is generally 4), then the combined optimization of multichannel-multiframe is represented by:

$\{\hat{P},\hat{H},\hat{X}\}=\underset{\{P,H,X\}}{\arg \min }\{\frac{1}{2TMN}\underset{t\∈T}{\mathrm{\Σ}}\underset{m\∈M}{\mathrm{\Σ}}\underset{n\∈N}{\mathrm{\Σ}}\[{(u_{tmn}-P_{mn}{X}_{tn})}^2+{(u_{tmn}-H_{mn}{X}_{tn})}^2\]\}---\left(2\right)$

In formula:Representing the output set optimized, { P, H, X} represent the input set participating in optimizing, and P represents 3D Projection matrix,Representing the 3D projection matrix optimizing gained, H represents plane shock wave matrix,Represent the plane optimizing gained Homography matrix, X represents labelling space of points coordinate,Represent optimized labelling space of points coordinate,Represent optimized several What labelling point coordinates, the symbol ∧ on letter represents optimization aim；T express time nodes, M represents camera number, and N represents geometry Mark tally；T is translation vector, represents the camera three axle translations under reference frame；M represents camera sequence number；N represents geometry Labelling point sequence number；u_tmnRepresent the n-th image characteristic point in the gathered image of m-th video camera under t；P_mRepresent m-th The 3D projection matrix at visual angle；X_tnRepresent labelling point 3D coordinate estimated under t；H_mRepresent the plane at m-th visual angle Homography matrix.

For formula (1), (2), need to point out: the present invention adds the pact of label shape in the step optimized herein Bundle, such as square plate 4 limit is isometric, can cut down the degree of freedom of equation group to a certain extent, improves convergence efficiency.Label herein It is not limited only to square, can introduce different shape constrainings, such as the two-end-point conllinear of linear mark thing by different labels Or circular fit degree determined by the minimum circumscribed circle formed by circular label boundary point and maximum inscribed circle etc..In a word, The combined optimization function constructed by plane shock wave and perspective projection, bootable optimum results is at the 6DOF of Current camera Optimum balance is reached in the 2D splicing effect of outer ginseng precision and plane picture.

The initial value that the optimization process of the present invention uses can be at the beginning of the initial design values of camera pose or marker location Beginning design load, this kind of design load can be given by the designer of camera or demarcation place respectively, namely the optimization process of this patent can Use multiple initial trigger value, therefore can carry out initial value screening by re-projection error, filter out current under different scenes Most suitable initial value under scene；If now there is at least one rational initial value, then algorithmic statement, expand scaling scheme Scene adaptability.

Described calibration result evaluation procedure includes: demarcate knot by subjective observation and objective evaluation two ways assay Fruit is the most accurate；If calibration result is accurate, then accepts and export calibrating parameters；If calibration result is inaccurate, then re-execute mark Note thing Extraction of Geometrical Features step；Wherein:

Subjective observation mode refers to: observe whether panoramic mosaic figure mates；

Objective evaluation mode refers to: the pose of installing combining each camera is evaluated with the discrimination standard set up.

Further, as it is shown on figure 3, the grid of 4 certain length of sides is artificial label, it is placed on setting of place 4 weeks Meter position, drive (4S shop) or transport vehicles (streamline) through demarcating place with design route, are evaluated and are demarcated by rear end. Integral calibrating process can be summarized as: vehicle moves with low speed (no more than 5km/h), and through demarcating place, viewing system synchronizes to adopt Collection video, runs calibration algorithm, completes to demarcate after inspection.

Embodiment 1:

When new car assembles on streamline, principle based on the present invention, including: characteristic body pickup model, characteristic parameter Extraction module, the optimization module of calibrating parameters and calibration result inspection module；Specifically include performing step as follows:

1) characteristic body pickup model can comprise several prior design handmarking's thing and the global coordinate system specified. Label is placed in the surrounding demarcating place, and its position in the global coordinate system set known.Automobile is at global coordinate system In position be known.Automatically detect image and navigate to this handmarking's thing.

2) extraction module of characteristic parameter automatically extracts handmarking's thing position in the picture.

3) the optimization module of calibrating parameters combines handmarking's thing of obtaining position in the picture and previously known complete The position of office's coordinate system, calculation optimization calibrating parameters.

4) calibration result inspection module is according to calculated calibrating parameters spliced panoramic image, and manual or automatic inspection Its splicing effect.

Embodiment 2:

In the 4S shop of vehicle maintenance maintenance, for the demarcation of panorama system, principle based on the present invention, including: feature Thing pickup model, the extraction module of characteristic parameter, the optimization module of calibrating parameters, can use the steps:

1) characteristic body pickup model comprises handmarking's thing of a particular design.Label is placed in the surrounding of vehicle.Can To use the method for artificial or image recognition to position this label position in the picture.

2) each characteristic parameter of extraction module extraction handmarking's thing of characteristic parameter position in the picture.

3) the optimization module of calibrating parameters combines handmarking's thing of obtaining position in the picture and previously known complete The position of office's coordinate system, calculation optimization calibrating parameters.

4) calibration result inspection module is according to calculated calibrating parameters spliced panoramic image, and checks its splicing effect Really.

One skilled in the art will appreciate that except realizing, in pure computer readable program code mode, the system that the present invention provides And beyond each device, completely can by method step is carried out system that programming in logic makes the present invention provide and Each device is with the form of gate, switch, special IC, programmable logic controller (PLC) and embedded microcontroller etc. Realize identical function.So, system and every device thereof that the present invention provides are considered a kind of hardware component, and right Include in it can also be considered as the structure in hardware component for the device realizing various function；Can also realize being used for respectively The device of kind of function is considered as not only being the software module of implementation method but also can be the structure in hardware component.

Above the specific embodiment of the present invention is described.It is to be appreciated that the invention is not limited in above-mentioned Particular implementation, those skilled in the art can make various deformation or amendment within the scope of the claims, this not shadow Ring the flesh and blood of the present invention.

Claims (10)

1. the combined optimization method in a dynamic calibration system, it is characterised in that comprise the steps:

Pose figure construction step: build the pose figure of each road camera according to dynamic calibration system in real time；

Initial alignment step: gather the image projection point under the camera of labelling Dian Ge road by pose figure, obtain mark to be optimized Determine parameter sets；

Calibrating parameters optimization step: use single channel-single frames method or multichannel-multi-frame method combined optimization calibrating parameters, obtain The calibrating parameters optimized.

Combined optimization method in dynamic calibration system the most according to claim 1, it is characterised in that also include:

Image mosaic step: according to the calibrating parameters of described optimization, spliced panoramic panoramic view picture；

Label Extraction of Geometrical Features step: the label in detection image shot by camera, the geometry extracting described label is special Levy, and the geometric properties that tracking and matching is extracted on follow-up each two field picture that camera provides；

Camera parameter estimating step: the initial pose for the geometric properties according to the label extracted and each camera obtains weight Projection error, and using re-projection error when meeting specification error threshold value as the demarcation to be optimized in calibrating parameters optimization step Parameter.

3. the combined optimization device in a dynamic calibration system, it is characterised in that including:

Calibrating parameters optimizes module, i.e. uses single channel-single frames method or multichannel-multi-frame method combined optimization calibrating parameters, To the calibrating parameters optimized.

Combined optimization device in dynamic calibration system the most according to claim 3, it is characterised in that described employing list Road-single frames or multichannel-multi-frame method are to carry out based on pose figure；Specifically,

The computing formula of single channel-single frames optimization method is as follows:

$\{\hat{P},\hat{H},{\hat{X}}_n\}=\underset{\{P,H,X\}}{\arg \min }\{\frac{1}{2N}\underset{n\∈N}{\Σ}\[{(u_n-{\mathrm{PX}}_n)}^2+{(u_n-{\mathrm{HX}}_n)}^2\]\};$

u_n=(u_n,v_n)^T；

X_n=(x_n,y_n,z_n)^T；

P=[R, t]；

$H=R-\frac{{\mathrm{tn}}^T}{d};$

In formula:Represent the output set optimized, as the calibrating parameters of described optimization；{ P, H, X} represent to be optimized Calibrating parameters set；P represents 3D projection matrix；Represent the 3D projection matrix optimizing gained；H represents plane shock wave matrix； Represent the plane shock wave matrix optimizing gained；X represents labelling space of points coordinate；Symbol ∧ on letter represents optimization aim；Represent the geometric markers point coordinates optimizing gained；Subscript T represents transposition；D represents the depth of field；u_nRepresent the n-th geometric markers point Characteristics of image point coordinates corresponding in a single road image；u_nRepresent that the n-th geometric markers point is at a single road figure The row-coordinate of image characteristic point corresponding in Xiang；v_nRepresent that the n-th geometric markers point is corresponding in a single road image The row coordinate of image characteristic point；X_nRepresent n-th geometric markers point coordinate in reference frame；x_nRepresent the n-th geometry Labelling point x-axis coordinate in reference frame；y_nRepresent n-th geometric markers point y-axis coordinate in reference frame；z_n Represent n-th geometric markers point z-axis coordinate in reference frame；R is spin matrix, represents that camera is under reference frame Three axles rotate；

The computing formula of multichannel-multi-frame joint optimization method is as follows:

$\{\hat{P},\hat{H},\hat{X}\}=\underset{\{P,H,X\}}{\arg \min }\{\frac{1}{2TMN}\underset{t\∈T}{\Σ}\underset{m\∈M}{\Σ}\underset{n\∈N}{\Σ}\[{(u_{tmn}-P_m{X}_{tn})}^2+{(u_{tmn}-H_m{X}_{tn})}^2\]\};$

P=[R, t]；

$H=R-\frac{{\mathrm{tn}}^T}{d};$

In formula:Represent the labelling space of points coordinate optimizing gained；T express time nodes, M represents camera number, and N represents geometry Mark tally；T is translation vector, represents the camera three axle translations under reference frame；M represents camera sequence number；N represents geometry Labelling point sequence number；u_tmnRepresent the n-th image characteristic point in the gathered image of m-th camera under t；P_mRepresent that m-th regards The 3D projection matrix at angle；X_tnRepresent the labelling point 3D coordinate of n-th image estimated under t；H_mRepresent that m-th regards The plane shock wave matrix at angle.

Combined optimization device in dynamic calibration system the most according to claim 3, it is characterised in that also include:

Image mosaic module, for the calibrating parameters according to described optimization, spliced panoramic panoramic view picture；

Label Extraction of Geometrical Features module, for detecting the label in image shot by camera, extracts the several of described label What feature, and the geometric properties that tracking and matching is extracted on follow-up each two field picture that camera provides；

Camera parameter estimation module, the initial pose for the geometric properties according to the label extracted and each camera obtains weight Projection error, and using re-projection error when meeting specification error threshold value as calibrating parameters to be optimized.

Combined optimization device in dynamic calibration system the most according to claim 3, it is characterised in that also include:

Calibration result evaluation module, for evaluating the accuracy of calibration result, it may be judged whether accept the parameter of calibration result.

Combined optimization device in dynamic calibration system the most according to claim 5, it is characterised in that described label is several What characteristic extracting module includes:

Detection module, for detection from the image of operative scenario for the label demarcated；

Extraction of Geometrical Features module, special for extracting a certain or multinomial geometry of this label on the label of detection gained Levy；

Matching module, for following the tracks of and mate the label geometric properties of extraction on follow-up each two field picture of camera shooting.

Combined optimization device in dynamic calibration system the most according to claim 5, it is characterised in that described camera parameter Estimation module includes:

Re-projection error acquisition module, for the geometric properties extracted according to each image shot by camera, and combines each phase The current pose parameter of machine, it is thus achieved that label is in current spatial coordinate and the image projection error under Current camera pose, i.e. Re-projection error；

Judge module, for judging the size of re-projection error, when re-projection error is less than or equal to the error threshold set Time, estimate the locus of each label according to described re-projection error, it is thus achieved that calibration result, activate calibration result and extract mould Block；When re-projection error is more than the error threshold set, activates current pose and update module；

Current pose updates module, for the estimation position according to each label and combine each image shot by camera and extracted Geometric properties update the current pose parameter of each camera；Activate re-projection error acquisition module；

Calibration result extraction module, for extracting re-projection error when meeting specification error threshold value.

Combined optimization device in dynamic calibration system the most according to claim 6, it is characterised in that described calibration result The calibration result evaluated in evaluation module refers to the panoramic looking-around image of splicing, by subjective observation and objective evaluation two ways Assay calibration result is the most accurate；If calibration result is accurate, then accepts and export calibrating parameters；If calibration result is forbidden Really, then label Extraction of Geometrical Features module is reactivated；Wherein,

Subjective observation mode refers to: observe whether panoramic mosaic figure mates；

Objective evaluation mode refers to: the pose of installing combining each camera is evaluated with the discrimination standard set up.

10. a dynamic calibration system, it is characterised in that include the dynamic calibration system according to any one of claim 3 to 9 In combined optimization device.