CN102298779A - Image registering method for panoramic assisted parking system - Google Patents

Abstract

The invention discloses an image registering method for a panoramic assisted parking system. The method comprises the following steps of: (1) acquiring image data of a reference image and an image to be registered; (2) performing histogram equalization and Gauss smooth filtering pre-processing; (3) selecting characteristic points with a Hrris comer extraction method respectively; (4) performing sub-pixel correction with a Taylor polynomial; (5) coarsely matching the reference image with the image to be registered; (6) accurately matching the reference image with an image to be matched with a Canny operator and a Hausdorff distance algorithm; (7) performing spatial transformation processing on image points to be matched with an affine transformation model; and (8) fusing an affine transformation coordinate with the reference image which is accurately matched in the step (6) so as to output a registering image, wherein each affine transformation coordinate corresponds to each image point in the to-be-matched image acquired in the step (7). The image registering method has the advantages of small calculation amount, high accuracy and high robustness in the splicing process.

Description

The method for registering images of panorama auxiliary parking system

Technical field

The present invention relates to image processing techniques, be specially a kind of method for registering images, be applied to the splicing of the fish eye images after overcorrect in the auxiliary parking system.

Background technology

Image registration is the process of determining optimum matching relation between two width of cloth taken in identical or different time, identical or different sensor or the multiple image.Registration results has provided relative position relation between two width of cloth images.Image registration is an important step of many application such as graphical analysis, image mosaic, image co-registration, virtual reality.

Traditional parking assisting system is divided into two kinds, and a kind of is at the tailstock of automobile radar for backing car to be installed, and surveys the motor vehicle environment barrier with ultrasonic sensor, with audible alarm.The shortcoming of this mode is easily reported by mistake for not directly perceived.A kind of is at automobile tail camera to be installed, and observes the distance of automobile and barrier with the pilothouse display.This mode can only cover limited zone, automobile rear, and car both sides and front region are the blind area, have potential safety hazard.Accident appears under congested situation easily.In order to enlarge driver's field range, panorama scope around the necessary perception automobile needs image registration and fusion between a plurality of cameras.

Summary of the invention

The present invention is directed to the deficiencies in the prior art, a kind of method of the image registration behind the correcting fisheye image in a kind of panorama auxiliary parking system is provided, this method calculated amount is less, and precision is higher, and robustness is higher in splicing.

For realizing above technical purpose, the present invention will take following technical scheme:

A kind of method for registering images of panorama auxiliary parking system may further comprise the steps: (1) gathers the view data of reference map and figure subject to registration; (2) reference map that respectively step (1) is collected and the view data of registration figure are carried out histogram equalization, the pre-service of Gauss's smothing filtering successively; (3) adopt Harris angle point extraction method to choose respectively, and the angular coordinate of the unique point correspondence that gets access to is stored respectively carrying out unique point through the described pretreated reference map of step (2) and arrangement plan; (4) adopt the unique point of the image subject to registration that the Taylor polynomial expression extracted step (3) to carry out sub-pix and proofread and correct, and the sub-pix correction feature point that gets access to is stored; (5) at first on reference map, choose Euclidean distance greater than the unique point of 50 pixels as candidate angular, follow position in each reference map unique point, the Euclidean distance of searching for the unique point coordinate of its corresponding match map is the sub-pix correction feature point in 50 pixels, finishes the thick coupling of reference map and figure subject to registration; (6) at first adopt the Canny operator respectively reference map and figure subject to registration through the described thick coupling of step (5) to be carried out the edge extracting binaryzation, adopt section H ausdorff distance algorithm to carry out the accurate coupling of reference map and figure to be matched then; (7) adopt the affined transformation model to handle, obtain the corresponding affined transformation coordinate of each picture point of figure to be matched carrying out spatial alternation through the picture point to be matched after the described accurate coupling of step (6); (8) carry out fusion treatment through the reference map that accurately mates in corresponding affined transformation coordinate of each picture point of figure to be matched that step (7) is got access to and the step (6), with the output registering images.

According to above technical scheme, can realize following beneficial effect:

(1) the present invention has used the sub-pix bearing calibration, image is carried out coordinate correct, and has improved matching precision; (2) section H ausdorff distance can produce effect preferably to the parts of images in the image that serious coverage or degeneration are arranged; (3) be that the center construction coupling has significantly reduced search time as Hausdorff apart from the input point set with the angle point, improved search efficiency.

Description of drawings

Fig. 1 is image acquisition of the present invention and pretreatment process figure;

Fig. 2 is an image registration algorithm process flow diagram of the present invention;

Fig. 3 is the structural representation that the present invention is applied to the panorama auxiliary parking system.

Embodiment

Accompanying drawing discloses the structural representation of the related preferred embodiment of the utility model without limitation; Explain the technical solution of the utility model below with reference to accompanying drawing.

As shown in Figures 1 to 3, the method for registering images of panorama auxiliary parking system of the present invention is by being installed in automobile four-way CCD camera images acquired all around, as shown in Figure 3.Four-way CCD camera is divided into 8 zones with 360 °, wherein needs the zone of registration to have 4, is respectively ABCD.Collect data and carry out pre-service and store zone subject to registration, as shown in Figure 1.

As shown in Figure 2, at first choose any one zone among the ABCD, extract contiguous two-way camera registering images, a width of cloth is as reference map, and a width of cloth adopts improved Harris angle point extraction algorithm respectively as match map, and the angular coordinate that gets access to is stored respectively.

Use the Taylor polynomial expression that angle point is carried out sub-pix then and proofread and correct storing coordinate.

Then adopt thick matching strategy, the Euclidean distance of the angular coordinate of benchmark figure storage, if its Euclidean distance keeps with interior angular coordinate in 50 pixels, coordinate deletion beyond 50 pixels, angular coordinate Euclidean distance relatively and between the figure angle point to be matched then, 50 pixels keep with interior coordinate, deletion in addition.After the rough matching, stayed strong angle point greater than 50 coordinate spacings.Adopt improved Hausdorff distance accurately to mate then.Earlier two images to be processed are carried out the Canny edge extracting, less threshold value gets 50, bigger threshold value gets 150, and mark the position of angle point, each angle point is the center in the taking-up match map, 25 pixels are the zone of radius, are the center at each angle point of reference map respectively, and 25 pixels are that the calculating of Hausdorff distance measure is carried out in the zone of radius.Distance is a match point less than the angle point at two figure centers of threshold value.With the angle point storage after accurately mating.Use at last least square method to the angle point of the accurate coupling that gets access to simulating the parameter of affined transformation.Utilize this parameter that match map is carried out affined transformation, and merge, the output registration results with reference map.

Wherein: Preprocessing Algorithm comprises histogram equalization and Gauss's smothing filtering, is expressed as follows:

A. establish gray level for being normalized to the continuous quantity in the scope [0,1], make p _r(r) represent gray level probability density function in certain given image, its subscript is distinguished the probability density function of input picture and output image.If input gray grade is carried out as down conversion, is obtained output gray level s:

The output gray level probability density function is that i.e. this conversion makes the gray level equalization of image uniformly as can be known, and the result is the image that gets access to a width of cloth dynamic range, and higher contrast ratio is arranged.

B. two-dimensional Gaussian function:

When r=± σ, G (r)=Ae ^1/2=0.6A; During r＜3 σ, G＜0.01 usefulness width is less than 2 σ ²Wave filter, i.e. m=2 * 2 σ ²+ 1, work as σ ²=1/2 o'clock, $G^3=\frac{1}{16}\left(\begin{array}{ccc}1& 2& 1\\ 2& 4& 2\\ 1& 2& 1\end{array}\right)$ Ask discrete template by continuous Gaussian distribution, need sampling, quantize, and make template normalization.

Step (3) feature point extraction adopts improved Harris angle point extraction algorithm, is expressed as follows:

A. use a Gaussian window or rectangular window on image, to move, by template window obtain the image second order derivative autocorrelation matrix C that original image generates 2*2 (x y), is defined as follows:

$C(x,y)=G\left(s\right)\⊗\left[\begin{array}{cc}{I}_x& I_x{I}_y\\ I_x{I}_y& I_y\end{array}\right]$

Wherein G (s) is Gauss's template, I _xBe this gradient, I in the x direction _yBe this gradient in the y direction.(x y) asks for eigenvalue to C ₁, λ ₂Set up metric function R ₁, R ₁=det (C)-k (trace (C)) ²Det (C)=λ wherein ₁λ ₂, trace (C)=λ ₁+ λ ₂

B. set up metric function R again ₂, R ₂=min (λ ₁, λ ₂), averaged R:R=(R ₁+ R ₂)/2, according to the threshold value Y of R, Y=(R _1max+ R _2max)/20, wherein R _1max, R _2maxBe metric function R ₁, R ₂Maximal value.If R＞Y then be angle point, otherwise be not angle point.

Step (4) unique point sub-pix is proofreaied and correct and is adopted the Taylor polynomial expression to approach, and is expressed as follows:

A. the initial position of supposing certain angle point is (x ₀, y ₀), its real position should be (x ₀+ s, y ₀+ t).

Wherein (s t) is error amount, and (s, t) ∈ [0.5,0.5] * [0.5,0.5] use Taylor's polynomial approximation to this angle point, obtain

$r(x_0+s,y_0+t)=r+\left(\begin{array}{cc}s& t\end{array}\right)\left(\begin{array}{c}{r}_x\\ r_y\end{array}\right)+\frac{1}{2}\left(\begin{array}{cc}s& t\end{array}\right)\left(\begin{array}{cc}{r}_{\mathrm{xx}}& r_{\mathrm{xy}}\\ r_{\mathrm{xy}}& r_{\mathrm{yy}}\end{array}\right)\left(\begin{array}{c}s\\ t\end{array}\right)$

Wherein r is handled image function, r _x, r _yBe respectively its x, the single order partial derivative of y direction, r _Xx, r _Xy, r _YyBe respectively the second-order partial differential coefficient of its respective direction.

B. the character according to angle point has: r _XxS+r _XyT+r _x=0, r _XyS+r _YyT+r _y=0

Obtain the sub-pixel location error $s=\frac{r_y{r}_{\mathrm{xy}}-r_x{r}_{\mathrm{yy}}}{r_{\mathrm{xx}}{r}_{\mathrm{yy}}-r_{\mathrm{xy}}^2},t=\frac{r_x{r}_{\mathrm{xy}}-r_y{r}_{\mathrm{xx}}}{r_{\mathrm{xx}}{r}_{\mathrm{yy}}-r_{\mathrm{xy}}^2}$

Step (5) unique point is slightly mated and has been adopted a simple and effective strategy, and this strategy is expressed as follows:

1, in the position of the strong angle point of benchmark, the Euclidean distance of searching for its corresponding match map angular coordinate is that 50 pixels are with interior angle point.

2, choose on reference map in the candidate point, the Euclidean distance between the candidate angular is outside 50 pixels.

Step (6) is carried out the edge extracting binaryzation to reference map, employing be the Canny operator, identical for reference map with the match map method, be expressed as follows:

A. choose the reference map (match map) behind the gaussian filtering, the big or small M of compute gradient and direction 0.Use the first approximation of the template of 2x2 size as x and y direction partial differential.Partial gradient

Edge direction O (x, y)=arctan (G _y/ G _x)

B. gradient is carried out non-very big inhibition: 8 neighborhoods of each pixel are divided into 4 sector regions, with gradient direction-22.5 °～22.5 ° and-157.5 °～157.5 °, 22.5 °～67.5 ° and-157.5 °～-112.5 °, 67.5 °～112.5 ° and-112.5 °～-67.5 °, 112.5 °～to be divided into these four sector numbers be 0-3 for 157.5 ° and-67.5 °～-22.5 °, if the central area pixel value is S[i, j], two pixel gradient values in itself and the 0-3 sector are compared, if S[i, j] Grad big unlike the Grad of these two pixels, then make S[i, j]=0, with the image S[i after handling, j] be stored in N (i, j) in.

C. detect and be connected the edge, use 2 threshold value T1 and T2 to do threshold process, wherein T1＜T2.Value is called strong edge pixel greater than the edge pixel of T2 and is the edge, edge pixel between T1 and the T2 is called weak edge pixel, if in the adjacent pixels of weak edge pixel point marginal point is arranged, thinks that then this weak edge pixel point also is marginal point, otherwise, think that this point is non-marginal point.The outline map picture point of reference map that gets access to and figure subject to registration deposits A respectively in, and two points of B are concentrated.

Step (7) accurately coupling adopts improved Hausdorff distance algorithm, is expressed as follows:

A. the section H ausdorff distance definition of point set A and B is as follows:

H _k，l(A，B)＝max(h _k(A，B)，h _l(B，A))，1≤k≤p，1≤l≤q

Wherein K=＜f ₁* p 〉, 0≤f ₁≤ 1, l=＜f ₂* q 〉, 0≤f ₂≤ 1,＜downward rounding operation represented.

According to from small to large series arrangement, wherein sequence number is that the distance of k is h to expression A to the distance of B point set _k(A, B).

According to from small to large series arrangement, wherein sequence number is that 1 distance is h to expression B to the distance of A point set _l(B, A), || a-b|| represents the distance of concentrated certain the some a of AB point to certain some b.Here choose f ₁, f ₂Be 0.8.

B. along the binary image edge, at each angle point is the zone at center, rectangular area with 20*20 makes up zonule subimage coordinate set, ask for reference map and figure section H ausdorff distance subject to registration respectively, get distance threshold τ, if τ=3 distance less than τ, then corresponding angle point is right for coupling, otherwise be not match point, reject.

Step (8) takes affined transformation as the spatial alternation model match map to be handled, and is expressed as follows:

A. six parameter affined transformations can be represented with following matrix form:

$\left[\begin{array}{ccc}{x}_1& y_1& 1\end{array}\right]=\left[\begin{array}{ccc}{x}_0& y_0& 1\end{array}\right]T,T=\left[\begin{array}{ccc}{a}_{11}& a_{12}& 0\\ a_{21}& a_{22}& 0\\ a_{31}& a_{32}& 1\end{array}\right]$

(x wherein ₁, y ₁), (x ₀, y ₀) be respectively two vice processing image corresponding point position coordinateses.Then have:

$\left[\begin{array}{c}{x}_1\\ y_1\end{array}\right]=\left[\begin{array}{cc}{a}_{11}& a_{21}\\ a_{12}& a_{22}\end{array}\right]\left[\begin{array}{c}{x}_0\\ y_0\end{array}\right]+\left[\begin{array}{c}{a}_{31}\\ a_{32}\end{array}\right]$

Transition form is:

$x_1=\left[\begin{array}{ccc}{a}_{11}& a_{21}& a_{31}\end{array}\right]\left[\begin{array}{c}{x}_0\\ y_0\\ 1\end{array}\right],y_1=\left[\begin{array}{ccc}{a}_{12}& a_{22}& a_{32}\end{array}\right]\left[\begin{array}{c}{x}_0\\ y_0\\ 1\end{array}\right]$

B. get access to N+1 to match point, adopt the least square fitting optimal parameter:

$\left\{\begin{array}{c}\left[\begin{array}{ccc}{x}_{10}& L& x_{1N}\end{array}\right]=\left[\begin{array}{ccc}{a}_{11}& a_{21}& a_{31}\end{array}\right]\left[\begin{array}{cccc}{x}_{00}& x_{01}& L& x_{0N}\\ y_{00}& y_{01}& L& y_{0N}\\ 1& 1& L& 1\end{array}\right]\\ \left[\begin{array}{ccc}{y}_{10}& L& y_{1N}\end{array}\right]=\begin{array}{}\end{array}\left[\begin{array}{ccc}{a}_{12}& a_{22}& a_{32}\end{array}\right]\left[\begin{array}{cccc}{x}_{00}& x_{01}& L& x_{0N}\\ y_{00}& y_{01}& L& y_{0N}\\ 1& 1& L& 1\end{array}\right]\end{array}\right.$

X wherein ₁₀L x _1NWith x ₀₀L x _0NFor N+1 to the match point coordinate.

By the least square method matrix form as can be known: the overdetermined equation of shape such as Ax=b separating of minimum meaning under least square method is A ^TAx=A ^TB, can release the approximate solution of six parameter affine transformation parameters under least square method and be:

$\left\{\begin{array}{c}\left[\begin{array}{ccc}{a}_{11}& a_{21}& a_{31}\end{array}\right]=\left[\begin{array}{ccc}{x}_{10}& L& x_{1N}\end{array}\right]A^T{\left({\mathrm{AA}}^T\right)}^{-1}\\ \left[\begin{array}{ccc}{a}_{21}& a_{22}& a_{32}\end{array}\right]=\left[\begin{array}{ccc}{y}_{10}& L& y_{1N}\end{array}\right]A^T{\left({\mathrm{AA}}^T\right)}^{-1}\end{array}\right.$

Wherein, $A=\left[\begin{array}{cccc}{x}_{00}& x_{01}& L& x_{0N}\\ y_{00}& y_{01}& L& y_{0N}\\ 1& 1& L& 1\end{array}\right].$

Claims (3)

1. the method for registering images of a panorama auxiliary parking system is characterized in that, may further comprise the steps: (1) gathers the view data of reference map and figure subject to registration; (2) reference map that respectively step (1) is collected and the view data of registration figure are carried out histogram equalization, the pre-service of Gauss's smothing filtering successively; (3) adopt Harris angle point extraction method to choose respectively, and the angular coordinate of the unique point correspondence that gets access to is stored respectively carrying out unique point through the described pretreated reference map of step (2) and arrangement plan; (4) adopt the unique point of the image subject to registration that the Taylor polynomial expression extracted step (3) to carry out sub-pix and proofread and correct, and the sub-pix correction feature point that gets access to is stored; (5) at first on reference map, choose Euclidean distance greater than the unique point of 50 pixels as candidate angular, follow position in each reference map unique point, the Euclidean distance of searching for the unique point coordinate of its corresponding match map is the sub-pix correction feature point in 50 pixels, finishes the thick coupling of reference map and figure subject to registration; (6) at first adopt the Canny operator respectively reference map and figure subject to registration through the described thick coupling of step (5) to be carried out the edge extracting binaryzation, adopt section H ausdorff distance algorithm to carry out the accurate coupling of reference map and figure to be matched then; (7) adopt the affined transformation model to handle, obtain the corresponding affined transformation coordinate of each picture point of figure to be matched carrying out spatial alternation through the picture point to be matched after the described accurate coupling of step (6); (8) carry out fusion treatment through the reference map that accurately mates in corresponding affined transformation coordinate of each picture point of figure to be matched that step (7) is got access to and the step (6), with the output registering images.

2. according to the method for registering images of the described panorama auxiliary parking system of claim 1, it is characterized in that the described Harris angle point of step (3) extraction method is:

A), at first use template window to move on original image, this template window is Gaussian window or rectangular window, and then the original image that template window is obtained generates the image second order derivative autocorrelation matrix of 2*2 $C(x,y):C(x,y)=G\left(s\right)\⊗\left[\begin{array}{cc}{I}_x& I_x{I}_y\\ I_x{I}_y& I_y\end{array}\right]$ Wherein, G (s) is Gauss's template, I _xBe this gradient, I in the x direction _yBe this gradient in the y direction;

B), ask for C (x, eigenvalue y) ₁, λ ₂, and set up metric function R ₁:

R ₁=det (C)-k (trace (C)) ²Det (C)=λ wherein ₁λ ₂, trace (C)=λ ₁+ λ ₂, k=0.04;

C), set up metric function R again ₂: R ₂=min (λ ₁, λ ₂), averaged R:R=(R ₁+ R ₂)/2

D), selected threshold Y:Y=(R _1max+ R _2max)/20, wherein R _1max, R _2maxBe respectively metric function R ₁, R ₂Maximal value;

E), judge the relation of R and Y, if R＞Y then is an angle point, otherwise be not angle point.

3. according to the method for registering images of the described panorama auxiliary parking system of claim 1, it is characterized in that, 1., the partial gradient function M and the edge direction function O of computed image the described accurate coupling of step (6) may further comprise the steps:: the template of using 2x2 is as the x of two-dimensional Gaussian function and the first approximation of y direction partial differential, then partial gradient function

Edge direction function O (x, y)=arctan (G _y/ G _x); 2., gradient is carried out non-very big inhibition: at first, 8 neighborhoods of each pixel are divided into 4 sector regions that are numbered 0-3, wherein, the gradient direction of No. 0 sector region is :-22.5 °～22.5 ° and-157.5 °～157.5 °, the gradient direction of No. 1 sector region is: 22.5 °～67.5 ° and-157.5 °～-112.5 °, the gradient direction of No. 2 sector regions is: 67.5 °～112.5 ° and-112.5 °～-67.5 °, the gradient direction of No. 3 sector regions is: 112.5 °～157.5 ° and-67.5 °～-22.5 °; Then, establishing the central area pixel is S[i, j], it is compared with two pixel gradient values of 0-3 sector region respectively, if S[i, j] Grad big unlike the Grad of these two pixels, then make S[i, j]=0, with S[i, j] be stored in N (i, j) in, (i j) then is image after the non-very big inhibition to N; 3., use two threshold value T1 and T2 to N (i, j) do threshold process, T1＜T2 wherein, value is called strong edge pixel point greater than the edge pixel of T2, then this is a marginal point, edge pixel between T1 and the T2 is called weak edge pixel point, further judges whether it is marginal point according to the edge is connective again; If in the adjacent pixels of weak edge pixel point marginal point is arranged, think that then this weak edge pixel point also is marginal point, otherwise, think that this point is non-marginal point; The outline map picture point of reference map that gets access to and figure subject to registration deposits A respectively in, and two points of B are concentrated; 4., H _{K, l}(A B) is section H ausdorff distance: H _{K, l}(A, B)=max (h _k(A, B), h _l(B, A)),

According to from small to large series arrangement, wherein sequence number is that the distance of k is h to expression A to the distance of B point set _k(A, B), same B to the A point set from small to large sequence number be that the distance of l is h _l(B, A); K=＜f wherein ₁* p 〉, 0≤f ₁≤ 1; L=＜f ₂* q 〉, 0≤f ₂≤ 1; 1≤k≤p, 1≤l≤q, p, q is respectively k, the maximal value of l, || a-b|| represents the distance of concentrated certain the some a of AB point to certain some b; 5., along the binary image edge, make up zonule subimage coordinate set, ask for section H ausdorff distance respectively, choose f ₁, f ₂Be 0.8, the angle point at part hausdorff distance two figure centers during less than threshold tau is thought match point, wherein, and τ=3.

Images