CN103871071A - Method for camera external reference calibration for panoramic parking system - Google Patents

Abstract

The invention discloses a method for industrially-automatic external reference calibration for a panoramic parking system. The method comprises the steps that a calibrated site is arranged, an ideal world coordinate system is determined, and a reference point is placed; the position of the reference point in an image coordinate system is calculated; the position of the reference point in a sensor coordinate system is calculated; an ideal external reference of a camera is calculated; the rotating deviation and the horizontally-moving deviation of a vehicle relative to the ideal world coordinate system are calculated; according to the rotating deviation and the horizontally-moving deviation, the position of the reference point in the ideal world coordinate system determined by the actual position of the vehicle relatively is calculated again; according to an internal reference of the camera, the position of the reference point in the sensor coordinate system and the position of the reference point in the ideal world coordinate system determined by the actual position of the vehicle relatively, an actual external reference of the camera is calculated. According to the method, the calibration process can be simplified, the calculation accuracy is improved, and the method can be automatically achieved on a production line.

Description

A kind of camera for panoramic parking system is joined scaling method outward

Technical field

The present invention relates to the image of panorama in parking and process and optical technical field, more particularly, relate to a kind of camera for panoramic parking system and join scaling method outward.

Background technology

China is the fastest country of development of automobile in the world, and along with the development in China city, the recoverable amount of automobile increases progressively at a high speed, and the thing followed is that the contradiction between people and car, car and car is growing more intense.First be parking difficulty problem, in narrow region, park and become an outstanding problem of puzzlement driver trip, the security incident causing due to driver's seat blind area when low speed driving is in addition also of common occurrence, has brought huge hidden danger to pedestrian's personal safety.Although the conventional equipment such as backsight camera lens or radar for backing car has played booster action to a certain extent, coverage rate is narrow, and the information that offers driver is concrete not.Panoramic parking system is for the problems referred to above, by the fish-eye camera that has broad field angle is installed in vehicle body surrounding, cover the field range of 360 ° of vehicle body surroundings, under the support of optical calibrating and image treatment theory by vehicle arround scenery project to ground level, and the scenery that multichannel camera is captured splicing becomes a complete scene, eliminate driver's blind area, for driver provides reference information intuitively.

For general passenger car, under headstock, the tailstock and left and right rearview mirror, 4 field angle are installed and can cover at the fish-eye camera of 160 ° of left and right the scene of vehicle's surroundings, and the visual range that guarantees adjacent two fish-eye cameras has public overlap-add region, this method is also current modal mount scheme.For finally for driver provides 360 ° of complete scene images, the information that just 4 cameras need to be caught is carried out projection, splicing and fusion, the effect of these steps all will depend on the parameter such as optical characteristics and position and attitude of 4 cameras, obtains the process of these parameters and demarcates.The demarcation of camera can be divided into internal reference demarcation and outer ginseng is demarcated, wherein internal reference demarcate be mainly determine the refracting power of optical mirror slip and install in the distortion that causes due to lens light axis and CMOS/CCD sensor plane out of plumb, because this link can complete when camera is produced, so can complete with the instrument of relative complex and longer time.It is mainly that camera installation site and attitude parameter are demarcated that outer ginseng is demarcated, this process need carries out after camera is installed on car, so must complete in automobile assembly line, this just has higher requirement to time and the mode of demarcating, and typical demand is that whole calibration process completes and consuming time being limited within 3 minutes automatically.

Outer ginseng scaling scheme is mainly divided into two kinds: one is to utilize the aids such as scaling board that demarcation reference point is provided; Another kind method does not need people for reference point is provided, but as a reference point according to the identical point in the multiple image obtaining in vehicle movement process, also referred to as self-calibrating method.Because needing vehicle movement one segment distance, self-calibration scheme finds enough reference point, also uncertain according to the different required operating ranges of reconnaissance situation and time, be not adapted at using on production line, simultaneously the method complexity, calculated amount are large, be difficult to complete in real time on slave computer, so industrial employing the first scheme in general, uses the method that reference point is artificially set to carry out outer ginseng demarcation.Conventional outer ginseng scaling method often uses the aids such as scaling board respectively 4 cameras to be demarcated separately, and then go out 4 cameras position relationship each other according to the position calculation of each timing signal scaling board, it is not strong that this method arranges directiveness to the position of reference point, and the complicated precision of process is poor.

Summary of the invention

In view of this, the invention provides a kind of camera for panoramic parking system and join scaling method outward, by being set, demarcates unified ideal world coordinate system, and in overlapping public domain, four camera visuals field, reference point is set, the reference point quantity of needs setting is dropped to minimum, in unified ideal world coordinate system, calculated the outer ginseng of four cameras simultaneously, simplified calibration process, improve computational accuracy, be conducive to automatically complete on production line.

For solving the problems of the technologies described above, the technical solution used in the present invention is: a kind of camera for panoramic parking system is joined scaling method outward, comprising:

Arrange and demarcate place according to the size of vehicle, determine ideal world coordinate system, and assigned address in desirable world coordinate system is placed reference point;

Calculate the position in the image coordinate system that reference point forms on the image of camera collection according to Corner Detection Algorithm;

Position according to reference point in image coordinate system, calculates reference point and converges ray cast to sensor plane at camera, and sensor converts light intensity to the position in the sensor coordinate system forming on the image of electric signal generation;

Position according to the internal reference of camera, reference point in sensor coordinate system and the reference point position calculation in desirable world coordinate system goes out the ideal of camera and joins outward;

Calculate rotating deviation and the shifting deviation of vehicle with respect to ideal world coordinate system;

Recalculate the position of reference point in the world coordinate system determining with respect to vehicle physical location according to rotating deviation and shifting deviation;

Position according to the internal reference of camera, reference point in sensor coordinate system and the reference point position calculation in the world coordinate system determining with respect to vehicle physical location goes out the reality of camera and joins outward.

Preferably, the described size according to vehicle is arranged and is demarcated place, determines ideal world coordinate system, and assigned address in desirable world coordinate system is placed reference point and is specially:

Draw on the ground a rectangular area according to vehicle dimension, as the demarcation place of vehicle position;

X-axis using the longitudinal central axis line of rectangle as ideal world coordinate system, vehicle dead ahead when being oriented to vehicle ideal and being parked in rectangular area of X-axis;

Using the horizontal axis of rectangle as the Y-axis of ideal world coordinate system, vehicle left side when being oriented to vehicle ideal and being parked in rectangular area of Y-axis;

Vertically plane is upwards for the Z axis of ideal world coordinate system;

Initial point O using the intersection point of X-axis, Y-axis and Z axis as world coordinate system;

Place reference point, make each camera can see four reference point.

It is preferably, described that according to reference point, the position calculation in image coordinate system goes out the position of reference point in sensor coordinate system and is specially:

According to relational expression:

$\left[\begin{array}{c}{u}^{\′}\\ v^{\′}\end{array}\right]=\left[\begin{array}{cc}c& d\\ e& 1\end{array}\right]\left[\begin{array}{c}u\\ v\end{array}\right]+\left[\begin{array}{c}{x}_c\\ y_c\end{array}\right]---\left(1\right)$

Calculate a reference point point u ' in image coordinate system=[u ', v '] ^tthe position u=[u of correspondence in sensor coordinate system, v] ^t;

Wherein: image coordinate system (u ', v ') take pixel as unit, initial point is in the upper left corner of image; The initial point of sensor coordinate system (u, v) is on the intersection point of camera optical axis and sensor plane; C, d, e represent the affine transformation parameter due to camera optical axis and the generation of sensor plane out of plumb; x _cand y _crepresent camera optical axis and the actual intersection point of the sensor plane side-play amount with respect to image coordinate system initial point.

Preferably, according to the internal reference of camera, reference point, the position in sensor coordinate system and the reference point position calculation in desirable world coordinate system goes out outside the ideal of camera to join and be specially:

Set up the reference point u=[u in sensor coordinate system, v by being created as transform g] ^trelation between imaging point P corresponding to it:

λg(u)=λp,λ>0； （2）

Wherein, p is the vector that camera photocentre points to some P, and λ is constant;

Become the expression formula of transform g to be:

g(u,v)=[u,v,f(u,v)] ^T； （3）

Wherein, f (u, v)=α ₀+ α ₁ρ+α ₂ρ ²+ ... + α _nρ ⁿ,

n=4, α ₀, α ₁, α ₂, α ₃, α ₄, x _c, y _c, c, d, the e internal reference that is camera;

Suppose camera coordinate system O _cx _cy _cz _cinitial point O _cat camera photocentre, x _c, y _caxle and sensor coordinate system cocurrent and parallel, Z _caxle orientation sensor plane, ideal world coordinate system is chosen ground level as O _wx _wy _wplane, Z _wvertically upward, the coordinate of a reference point P in desirable world coordinate system that in space, camera is taken is P to axle _w=[X _w, Y _w, Z _w] ^t, in camera coordinate system, be expressed as P _c=[X _c, Y _c, Z _c] ^t, and:

P _c=R·P _w+T； （4）

Wherein, $R=[r_1,r_2,r_3]=\left[\begin{array}{ccc}{r}_{11}& r_{12}& r_{13}\\ r_{21}& r_{22}& r_{23}\\ r_{31}& r_{32}& r_{33}\end{array}\right]$ For rotation matrix, $T=t=\left[\begin{array}{c}{t}_1\\ t_2\\ t_3\end{array}\right]$ For translation matrix;

Set up equation according to the relation between the ideal world coordinate of each reference point and sensor coordinates:

$\mathrm{\λ}\left[\begin{array}{c}u\\ v\\ f\left(\mathrm{\ρ}\right)\end{array}\right]=\left[\begin{array}{cccc}{r}_{11}& r_{12}& r_{13}& t_1\\ r_{21}& r_{22}& r_{23}& t_2\\ r_{31}& r_{32}& r_{33}& t_3\end{array}\right]\left[\begin{array}{c}{X}_w\\ Y_w\\ Z_w\\ 1\end{array}\right]=\left[\begin{array}{ccc}{r}_{11}& r_{12}& t_1\\ r_{21}& r_{22}& t_2\\ r_{31}& r_{32}& t_3\end{array}\right]\left[\begin{array}{c}{X}_w\\ Y_w\\ 1\end{array}\right];---\left(5\right)$

Projection coordinate [u, v, f (ρ)] by equation (5) two ends this reference point of multiplication cross of setting up in sensor plane ^tobtain:

$\left[\begin{array}{c}u\\ v\\ f\left(\mathrm{\ρ}\right)\end{array}\right]\×\left[\begin{array}{ccc}{r}_{11}& r_{12}& t_1\\ r_{21}& r_{22}& t_2\\ t_{31}& r_{32}& t_3\end{array}\right]\left[\begin{array}{c}{X}_w\\ Y_w\\ 1\end{array}\right]=0;---\left(6\right)$

Draw three equations of a reference point according to equation (6):

v·(r ₃₁X _w+r ₃₂Y _w+t ₃)-f(ρ)·(r ₂₁X _w+r ₂₂Y _w+t ₂)=0； （7）

f(ρ)·(r ₁₁X _w+r ₁₂Y _w+t ₁)-u·(r ₃₁X _w+r ₃₂Y _w+t ₃)=0； （8）

u·(r ₂₁X _w+r ₂₂Y _w+t ₂)-v·(r ₁₁X _w+r ₁₂Y _w+t ₁)=0； （9）

Adopt said method to draw the equation of its excess-three reference point that camera photographs, show that according to the equations simultaneousness of reference point the ideal of camera is joined r outward ₁₁, r ₁₂, r ₁₃, r ₂₁, r ₂₂, r ₂₃, r ₃₁, r ₃₂, r ₃₃, t ₁, t ₂and t ₃.

Can find out from above-mentioned technical scheme, a kind of camera for panoramic parking system disclosed by the invention is joined scaling method outward, first arrange and demarcate place according to the size of vehicle, determine ideal world coordinate system, and assigned address in desirable world coordinate system is placed reference point, then calculate the position of reference point in image coordinate system according to Corner Detection Algorithm, then according to reference point, the position calculation in image coordinate system goes out the position of reference point in sensor coordinate system, then according to the internal reference of camera, the position of reference point in sensor coordinate system and the position calculation of reference point in desirable world coordinate system go out the ideal of camera and join outward, then calculate rotating deviation and the shifting deviation of vehicle with respect to ideal world coordinate system, last according to the internal reference of camera, the position of reference point in sensor coordinate system and the position calculation of reference point in the world coordinate system determining with respect to vehicle physical location go out the reality of camera and join outward.This shows, the present invention demarcates by unified ideal world coordinate system is set, and has simplified the process of demarcating, and has improved computational accuracy, is conducive to automatically complete on production line.

Accompanying drawing explanation

In order to be illustrated more clearly in the embodiment of the present invention or technical scheme of the prior art, to the accompanying drawing of required use in embodiment or description of the Prior Art be briefly described below, apparently, accompanying drawing in the following describes is only some embodiments of the present invention, for those of ordinary skills, do not paying under the prerequisite of creative work, can also obtain according to these accompanying drawings other accompanying drawing.

Fig. 1 is the schematic diagram of the reference point of the demarcation place arranged of the size according to vehicle disclosed by the invention, definite ideal world coordinate system and placement;

Fig. 2 is the process flow diagram that a kind of camera for panoramic parking system disclosed by the invention is joined scaling method outward.

Embodiment

Below in conjunction with the accompanying drawing in the embodiment of the present invention, the technical scheme in the embodiment of the present invention is clearly and completely described, obviously, described embodiment is only a part of embodiment of the present invention, rather than whole embodiment.Based on the embodiment in the present invention, those of ordinary skills, not making the every other embodiment obtaining under creative work prerequisite, belong to the scope of protection of the invention.

The embodiment of the invention discloses a kind of camera for panoramic parking system and join scaling method outward, by being set, demarcates unified ideal world coordinate system, and in overlapping public domain, four camera visuals field, reference point is set, the reference point quantity of needs setting has been dropped to minimum, in unified ideal world coordinate system, calculate the outer ginseng of four cameras simultaneously, simplify calibration process, improved computational accuracy, be conducive to automatically complete on production line.

As shown in Figure 2, join scaling method outward for a kind of camera for panoramic parking system disclosed by the invention, comprising:

S101, arrange and demarcate place according to the size of vehicle, determine ideal world coordinate system, and assigned address in desirable world coordinate system is placed reference point;

S102, calculate the position in the image coordinate system that reference point forms on the image of camera collection according to Corner Detection Algorithm;

S103, position according to reference point in image coordinate system, calculate reference point and converge ray cast to sensor plane at camera, and sensor converts light intensity to the position in the sensor coordinate system forming on the image that electric signal generates;

S104, position according to the internal reference of camera, reference point in sensor coordinate system and the position calculation of reference point in desirable world coordinate system go out the ideal of camera and join outward;

S105, calculate rotating deviation and the shifting deviation of vehicle with respect to ideal world coordinate system;

S106, recalculate the position of reference point in the world coordinate system determining with respect to vehicle physical location according to rotating deviation and shifting deviation;

S107, position according to the internal reference of camera, reference point in sensor coordinate system and the position calculation of reference point in the world coordinate system determining with respect to vehicle physical location go out the reality of camera and join outward.

Concrete, the course of work of above-mentioned method and principle are: first draw on the ground a rectangular area according to the size of vehicle, as shown in Figure 1, the position at mark vehicle place, makes length and the wide length of being a bit larger tham actual vehicle and wide of this rectangular area.Specifically arranging can be according to actual situation adjustment, to guarantee that vehicle can stop in this region, make the each camera on vehicle can see four reference point simultaneously, for example, the camera that headstock is installed can be seen a, b, c, tetra-reference point of d, the camera that the tailstock is installed can be seen e, f, g, tetra-reference point of h, the camera of installing under left-hand mirror can be seen a, b, e, tetra-reference point of f, and the camera of installing under right back visor can be seen c, d, g, tetra-reference point of h.Then follow the right-hand rule and determine ideal world coordinate system, as shown in Figure 1, the X-axis of system using the longitudinal central axis line of rectangle as ideal coordinates, is vehicle projection on the ground of longitudinal central axis line when desirable parking in rectangular area, the sensing vehicle dead ahead of X-axis; Using the horizontal axis of rectangle as the Y-axis of ideal world coordinate system, be parallel to the horizontal axis of vehicle, perpendicular to X-axis and point to vehicle left side, initial point O is the projection on the ground of intersection point of lateral direction of car, longitudinal central axis line; XOY plane is ground level; Vertically plane is upwards for Z axis.Then in the public domain covering in 4 camera visuals field, assigned address is placed reference point, and preferred reference point is positioned on the line on 4 summits corresponding to vehicle restriction rectangle, 4 summits of final visual stitching image rectangular extent.It should be noted that when reference point is set, preferably adopt the obvious angle point pattern of feature.

Then adopt the method for Corner Detection to calculate the position of reference point in image coordinate system, Corner Detection Algorithm is very ripe at present, precision can reach sub-pixel level, herein as long as select as required a kind of applicable method can obtain the position of reference point in image coordinate system in existing many algorithms, so repeat no more.In order to guarantee the accuracy of Corner Detection, except adopting the obvious angle point pattern of feature, also should keep demarcating the clean of place, avoid other objects or spot to exert an influence to this process.When placing reference point, preferably symmetrical placement between two, is so more conducive to the result of Corner Detection to be verified.

Then enter the outer ginseng computation process of camera, the concrete theory that outer ginseng is calculated is: the universal optical model that adopts Davide Scaramuzza to propose is basic as narration.First determine two coordinate systems: image coordinate system (u ', v ') and sensor coordinate system (u, v).Image coordinate system (u ', v ') take pixel as unit, initial point is in the image upper left corner; The initial point of sensor coordinate system is on the intersection point of camera optical axis and sensor plane.A reference point u ' in image coordinate system=[u ', v '] ^t, with its correspondence position u=[u in sensor coordinate system, v] ^tbetween relation can be expressed as:

$\left[\begin{array}{c}{u}^{\′}\\ v^{\′}\end{array}\right]=\left[\begin{array}{cc}c& d\\ e& 1\end{array}\right]\left[\begin{array}{c}u\\ v\end{array}\right]+\left[\begin{array}{c}{x}_c\\ y_c\end{array}\right]---\left(1\right)$

Wherein, c, d, e represent the affine transformation parameter due to camera optical axis and the generation of sensor plane out of plumb; x _cand y _crepresent camera optical axis and the actual intersection point of the sensor plane side-play amount with respect to image coordinate system initial point.Become transform g can set up 1 u=[u in sensor coordinate system, v by setting up one] ^trelation between imaging point P corresponding to it:

λg(u)=λp,λ>0 (2)

Wherein p is the vector that is pointed to some P by camera lens photocentre, and λ is constant.

Become the expression formula of transform g to be:

g(u,v)=[u,v,f(u,v)] ^T (3)

Wherein function f only with

relevant, f (u, v)=α ₀+ α ₁ρ+α ₂ρ ²+ ... + α _nρ ⁿ, while experimental results show that N=4, effect is better.The parameter alpha more than relating to ₀, α ₁, α ₂, α ₃, α ₄, x _c, y _c, c, d, the e internal reference that is camera, in the process that these parameters can be produced at camera, obtain as known parameters by demarcation.

Then suppose camera coordinate system O _cx _cy _cz _cinitial point O _cat shooting head photocentre, x _c, y _caxle and sensor coordinate system cocurrent and parallel, Z _caxle orientation sensor plane.Ideal world coordinate system free setting as required, generally chooses ground level as O _wx _wy _wplane, Z _waxle vertically upward.If the coordinate of certain some P in desirable world coordinate system is P in space _w=[X _w, Y _w, Z _w] ^t, it can be expressed as P in camera coordinate system so _c=[X _c, Y _c, Z _c] ^t, and:

P _c=R·P _w+T(4)

In above formula $R=[r_1,r_2,r_3]=\left[\begin{array}{ccc}{r}_{11}& r_{12}& r_{13}\\ r_{21}& r_{22}& r_{23}\\ r_{31}& r_{32}& r_{33}\end{array}\right]$ For rotation matrix, $T=t=\left[\begin{array}{c}{t}_1\\ t_2\\ t_3\end{array}\right]$ For translation matrix, r ₁₁, r ₁₂, r ₁₃, r ₂₁, r ₂₂, r ₂₃, r ₃₁, r ₃₂, r ₃₃, t ₁, t ₂and t ₃being camera joins outward.This shows, actual internal reference is demarcated and has been determined contacting between image coordinate system and camera coordinate system, and outer ginseng is demarcated the relation of having determined between camera coordinate system and ideal world coordinate system, after ginseng outside internal reference is all determined, point in point and ideal world coordinate system in image has just been set up corresponding relation, makes the operation such as projective transformation and splicing fusion become possibility.

If suppose ideal world coordinate system midplane O _wx _wy _wfor ground level, and on ground level, place reference point, the Z of all reference point _w=0.Relation between ideal world coordinate and the sensor coordinates of each reference point can equation of simultaneous:

$\mathrm{\λ}\left[\begin{array}{c}u\\ v\\ f\left(\mathrm{\ρ}\right)\end{array}\right]=\left[\begin{array}{cccc}{r}_{11}& r_{12}& r_{13}& t_1\\ r_{21}& r_{22}& r_{23}& t_2\\ r_{31}& r_{32}& r_{33}& t_3\end{array}\right]\left[\begin{array}{c}{X}_w\\ Y_w\\ Z_w\\ 1\end{array}\right]=\left[\begin{array}{ccc}{r}_{11}& r_{12}& t_1\\ r_{21}& r_{22}& t_2\\ r_{31}& r_{32}& t_3\end{array}\right]\left[\begin{array}{c}{X}_w\\ Y_w\\ 1\end{array}\right]---\left(5\right)$

Projection coordinate [u, v, f (ρ)] by equation (5) two ends this reference point of multiplication cross in sensor plane ^t, can obtain

$\left[\begin{array}{c}u\\ v\\ f\left(\mathrm{\ρ}\right)\end{array}\right]\×\left[\begin{array}{ccc}{r}_{11}& r_{12}& t_1\\ r_{21}& r_{22}& t_2\\ t_{31}& r_{32}& t_3\end{array}\right]\left[\begin{array}{c}{X}_w\\ Y_w\\ 1\end{array}\right]=0---\left(6\right)$

Known by formula (6), the ideal world coordinate of each group reference point and sensor coordinates can be contributed 3 equations for solving:

v·(r ₃₁X _w+r ₃₂Y _w+t ₃)-f(ρ)·(r ₂₁X _w+r ₂₂Y _w+t ₂)=0 (7)

f(ρ)·(r ₁₁X _w+r ₁₂Y _w+t ₁)-u·(r ₃₁X _w+r ₃₂Y _w+t ₃)=0 (8)

u·(r ₂₁X _w+r ₂₂Y _w+t ₂)-v·(r ₁₁X _w+r ₁₂Y _w+t ₁)=0 (9)

The unknown number that need to solve comprises r ₁₁, r ₁₂, r ₁₃, r ₂₁, r ₂₂, r ₂₃, r ₃₁, r ₃₂, r ₃₃, t ₁, t ₂and t ₃, totally 9.As long as so 3 reference point are set, in actual use, for parameter being optimized to maintenance computational accuracy, we select to arrange 4 reference point.Other 3 outer ginsengs can obtain by following formula:

$\left[\begin{array}{c}{r}_{31}\\ r_{32}\\ r_{33}\end{array}\right]=\left[\begin{array}{c}{r}_{11}\\ r_{12}\\ r_{13}\end{array}\right]\×\left[\begin{array}{c}{r}_{21}\\ r_{22}\\ r_{23}\end{array}\right]---\left(10\right)$

Can be drawn by the above-mentioned outer ginseng theory of computation, first obtain internal reference in the time calculating outer ginseng, internal reference can be stored in the storer that camera carries in the time that camera is assembled, and when ginseng calculating outside, internal reference is read in camera storer.Then prepare other parameters.Before arranging reference point, design in advance the ideal world coordinate position of reference point, obtained again the position of reference point in image coordinate system by reference to the automatic detection of point.Utilize formula (1), can obtain the position of reference point in sensor coordinate system.Afterwards by internal reference substitution system of equations and calculate, because 8 reference point have been placed on the public part in 4 camera visuals field, so each camera can be seen 4 reference point.And the corresponding formula of each reference point (8), (9) and (10) three equations.Each like this camera can be set up 12 equations and solve, and the optimization that can obtain each camera by least square method is joined result outward.Owing to having used unified ideal world coordinate system, the outer ginseng of the each camera herein obtaining is the outer ginseng with respect to same ideal world coordinate system, without further conversion.

Because reference point is prepositioned according to ideal state, and can produce deviation with ideal position unavoidably in the process of vehicle within entering restriction rectangle, cause vehicle axis and the center line of restriction rectangle to produce rotating deviation, between real world coordinate origin and ideal world coordinate origin, produce shifting deviation simultaneously.In order to eliminate these deviations, first need to calculate the rotating deviation of vehicle.Be as the criterion and (be generally as the criterion with rearview camera with certain camera position, this camera present position is difficult for variation, be convenient to guarantee its installation accuracy by frock), utilize rotation matrix information in current outer ginseng can calculate the angle of its optical axis and vehicle world coordinate system coordinate axis in the time of ideal position, this angle is the rotating deviation of vehicle.

Then calculate the shifting deviation of vehicle.Calculate forward and backward camera with respect to ideal world true origin the translational movement along X-axis translation, the half of the two difference is the directions X translation distance between vehicle physical location and ideal position; In like manner calculate left and right camera with respect to ideal world coordinate origin the translational movement along Y-axis translation, the half of the two difference is the Y-direction translation distance between vehicle physical location and ideal position.

According to rotating deviation and shifting deviation, recalculate the position in the world coordinate system that 8 reference point determine with respect to vehicle physical location, owing to not relating to the deviation of Z axis, so this adjustment process is equivalent to a two-dimentional coordinate transformation process.Can construct a rotation matrix and a translation matrix by rotating deviation and shifting deviation, utilizing coordinate transformation method is the positional information of renewable reference point in real world coordinate system.Then the position in sensor coordinate system and the reference point position in the world coordinate system determining with respect to vehicle physical location according to the internal reference of camera, reference point, the reality that calculates camera with reference to the method for the outer ginseng of above-mentioned calculating ideal is joined outward.

In this instructions, each embodiment adopts the mode of going forward one by one to describe, and what each embodiment stressed is and the difference of other embodiment, between each embodiment identical similar part mutually referring to.

To the above-mentioned explanation of the disclosed embodiments, make professional and technical personnel in the field can realize or use the present invention.To be apparent for those skilled in the art to the multiple modification of these embodiment, General Principle as defined herein can, in the situation that not departing from the spirit or scope of the present invention, realize in other embodiments.Therefore, the present invention will can not be restricted to these embodiment shown in this article, but will meet the widest scope consistent with principle disclosed herein and features of novelty.

Claims (4)

1. join a scaling method outward for the camera of panoramic parking system, it is characterized in that, comprising:

Arrange and demarcate place according to the size of vehicle, determine ideal world coordinate system, and assigned address in desirable world coordinate system is placed reference point;

Calculate the position in the image coordinate system that reference point forms on the image of camera collection according to Corner Detection Algorithm;

Position according to reference point in image coordinate system, calculates reference point and converges ray cast to sensor plane at camera, and sensor converts light intensity to the position in the sensor coordinate system forming on the image of electric signal generation;

Position according to the internal reference of camera, reference point in sensor coordinate system and the reference point position calculation in desirable world coordinate system goes out the ideal of camera and joins outward;

Calculate rotating deviation and the shifting deviation of vehicle with respect to ideal world coordinate system;

Recalculate the position of reference point in the world coordinate system determining with respect to vehicle physical location according to rotating deviation and shifting deviation;

Position according to the internal reference of camera, reference point in sensor coordinate system and the reference point position calculation in the world coordinate system determining with respect to vehicle physical location goes out the reality of camera and joins outward.

2. method according to claim 1, is characterized in that, the described size according to vehicle is arranged and demarcated place, determines ideal world coordinate system, and assigned address in desirable world coordinate system is placed reference point and is specially:

Draw on the ground a rectangular area according to vehicle dimension, as the demarcation place of vehicle position;

X-axis using the longitudinal central axis line of rectangle as ideal world coordinate system, vehicle dead ahead when being oriented to vehicle ideal and being parked in rectangular area of X-axis;

Using the horizontal axis of rectangle as the Y-axis of ideal world coordinate system, vehicle left side when being oriented to vehicle ideal and being parked in rectangular area of Y-axis;

Vertically plane is upwards for the Z axis of ideal world coordinate system;

Initial point O using the intersection point of X-axis, Y-axis and Z axis as world coordinate system;

Place reference point, make each camera can see four reference point.

3. method according to claim 2, is characterized in that, described according to reference point, the position calculation in image coordinate system goes out the position of reference point in sensor coordinate system and is specially:

According to relational expression:

$\left[\begin{array}{c}{u}^{\′}\\ v^{\′}\end{array}\right]=\left[\begin{array}{cc}c& d\\ e& 1\end{array}\right]\left[\begin{array}{c}u\\ v\end{array}\right]+\left[\begin{array}{c}{x}_c\\ y_c\end{array}\right]---\left(1\right)$

Calculate a reference point point u ' in image coordinate system=[u ', v '] ^tthe position u=[u of correspondence in sensor coordinate system, v] ^t;

Wherein: image coordinate system (u ', v ') take pixel as unit, initial point is in the upper left corner of image; The initial point of sensor coordinate system (u, v) is on the intersection point of camera optical axis and sensor plane; C, d, e represent the affine transformation parameter due to camera optical axis and the generation of sensor plane out of plumb; x _cand y _crepresent camera optical axis and the actual intersection point of the sensor plane side-play amount with respect to image coordinate system initial point.

4. method according to claim 3, is characterized in that, the position according to the internal reference of camera, reference point in sensor coordinate system and the reference point position calculation in desirable world coordinate system goes out outside the ideal of camera to join and be specially:

Set up the reference point u=[u in sensor coordinate system, v by being created as transform g] ^trelation between imaging point P corresponding to it:

λg(u)=λp,λ>0； （2）

Wherein, p is the vector that camera photocentre points to some P, and λ is constant;

Become the expression formula of transform g to be:

g(u,v)=[u,v,f(u,v)] ^T； （3）

Wherein, f (u, v)=α ₀+ α ₁ρ+α ₂ρ ²+ ... + α _nρ ⁿ,

n=4, α ₀, α ₁, α ₂, α ₃, α ₄, x _c, y _c, c, d, the e internal reference that is camera;

Suppose camera coordinate system O _cx _cy _cz _cinitial point O _cat camera photocentre, x _c, y _caxle and sensor coordinate system cocurrent and parallel, Z _caxle orientation sensor plane, ideal world coordinate system is chosen ground level as O _wx _wy _wplane, Z _wvertically upward, the coordinate of a reference point P in desirable world coordinate system that in space, camera is taken is P to axle _w=[X _w, Y _w, Z _w] ^t, in camera coordinate system, be expressed as P _c=[X _c, Y _c, Z _c] ^t, and:

P _c=R·P _w+T； （4）

Wherein, $R=[r_1,r_2,r_3]=\left[\begin{array}{ccc}{r}_{11}& r_{12}& r_{13}\\ r_{21}& r_{22}& r_{23}\\ r_{31}& r_{32}& r_{33}\end{array}\right]$ For rotation matrix, $T=t=\left[\begin{array}{c}{t}_1\\ t_2\\ t_3\end{array}\right]$ For translation matrix;

Set up equation according to the relation between the ideal world coordinate of each reference point and sensor coordinates:

$\mathrm{\λ}\left[\begin{array}{c}u\\ v\\ f\left(\mathrm{\ρ}\right)\end{array}\right]=\left[\begin{array}{cccc}{r}_{11}& r_{12}& r_{13}& t_1\\ r_{21}& r_{22}& r_{23}& t_2\\ r_{31}& r_{32}& r_{33}& t_3\end{array}\right]\left[\begin{array}{c}{X}_w\\ Y_w\\ Z_w\\ 1\end{array}\right]=\left[\begin{array}{ccc}{r}_{11}& r_{12}& t_1\\ r_{21}& r_{22}& t_2\\ r_{31}& r_{32}& t_3\end{array}\right]\left[\begin{array}{c}{X}_w\\ Y_w\\ 1\end{array}\right];---\left(5\right)$

Projection coordinate [u, v, f (ρ)] by equation (5) two ends this reference point of multiplication cross of setting up in sensor plane ^tobtain:

$\left[\begin{array}{c}u\\ v\\ f\left(\mathrm{\ρ}\right)\end{array}\right]\×\left[\begin{array}{ccc}{r}_{11}& r_{12}& t_1\\ r_{21}& r_{22}& t_2\\ t_{31}& r_{32}& t_3\end{array}\right]\left[\begin{array}{c}{X}_w\\ Y_w\\ 1\end{array}\right]=0;---\left(6\right)$

Draw three equations of a reference point according to equation (6):

v·(r ₃₁X _w+r ₃₂Y _w+t ₃)-f(ρ)·(r ₂₁X _w+r ₂₂Y _w+t ₂)=0； （7）

f(ρ)·(r ₁₁X _w+r ₁₂Y _w+t ₁)-u·(r ₃₁X _w+r ₃₂Y _w+t ₃)=0； （8）

u·(r ₂₁X _w+r ₂₂Y _w+t ₂)-v·(r ₁₁X _w+r ₁₂Y _w+t ₁)=0； （9）

Adopt said method to draw the equation of its excess-three reference point that camera photographs, show that according to the equations simultaneousness of reference point the ideal of camera is joined r outward ₁₁, r ₁₂, r ₁₃, r ₂₁, r ₂₂, r ₂₃, r ₃₁, r ₃₂, r ₃₃, t ₁, t ₂and t ₃.

Images