CN102298779B - Image registering method for panoramic assisted parking system - Google Patents

Abstract

The invention discloses an image registering method for a panoramic assisted parking system. The method comprises the following steps of: (1) acquiring image data of a reference image and an image to be registered; (2) performing histogram equalization and Gauss smooth filtering pre-processing; (3) selecting characteristic points with a Hrris comer extraction method respectively; (4) performing sub-pixel correction with a Taylor polynomial; (5) coarsely matching the reference image with the image to be registered; (6) accurately matching the reference image with an image to be matched with a Canny operator and a Hausdorff distance algorithm; (7) performing spatial transformation processing on image points to be matched with an affine transformation model; and (8) fusing an affine transformation coordinate with the reference image which is accurately matched in the step (6) so as to output a registering image, wherein each affine transformation coordinate corresponds to each image point in the to-be-matched image acquired in the step (7). The image registering method has the advantages of small calculation amount, high accuracy and high robustness in the splicing process.

Description

The method for registering images of panorama auxiliary parking system

Technical field

The present invention relates to image processing techniques, be specially a kind of method for registering images, be applied to the splicing of the fish eye images after overcorrect in auxiliary parking system.

Background technology

Image registration is the process of determining optimum matching relation between two width taken in identical or different time, identical or different sensor or multiple image.Registration results has provided relative position relation between two width images.Image registration is an important step of many application such as graphical analysis, Image Mosaics, image co-registration, virtual reality.

Traditional parking assisting system is divided into two kinds, and a kind of is, at the tailstock of automobile, radar for backing car is installed, with ultrasonic sensor probe vehicle peripheral obstacle, with audible alarm.The shortcoming of this mode, for not directly perceived, is easily reported by mistake.A kind of is, at automobile tail, camera is installed, and observes the distance of automobile and barrier with the pilothouse display.This mode can only cover limited zone, automobile rear, and car both sides and front region are blind area, have potential safety hazard.Accident easily appears in the situation that congested.In order to enlarge driver's field range, necessary perception automobile surrounding panorama scope, need image registration and fusion between a plurality of cameras.

Summary of the invention

The present invention is directed to the deficiencies in the prior art, a kind of method of the image registration after correcting fisheye image in a kind of panorama auxiliary parking system is provided, the method calculated amount is less, and precision is higher, and in splicing, robustness is higher.

For realizing above technical purpose, the present invention will take following technical scheme:

A kind of method for registering images of panorama auxiliary parking system comprises the following steps: (1) gathers the view data of reference map and figure subject to registration; (2) reference map respectively step (1) collected and the view data of registration figure are carried out histogram equalization, Gaussian smoothing filter preprocessing successively; (3) adopt respectively the Harris corner extraction to choose through the described pretreated reference map of step (2) and arrangement plan, carrying out unique point, and corresponding angular coordinate is stored respectively by the unique point got; (4) unique point of the image subject to registration that adopts the Taylor polynomial expression to extract step (3) is carried out the sub-pix correction, and by the sub-pix correction feature point storage got; (5) at first on reference map, choose Euclidean distance and be greater than the unique point of 50 pixels as candidate angular, follow the position in each reference map unique point, the Euclidean distance of searching for the unique point coordinate of its Corresponding matching figure is the sub-pix correction feature point in 50 pixels, completes the thick coupling of reference map and figure subject to registration; (6) at first adopt the Canny operator respectively reference map and figure subject to registration through the described thick coupling of step (5) to be carried out to the edge extracting binaryzation, then adopt part Hausdorff distance algorithm to carry out the exact matching of reference map and figure to be matched; (7) adopt affine Transform Model to carry out the spatial alternation processing to the picture point to be matched after the described exact matching of step (6), obtain the corresponding affined transformation coordinate of each picture point of figure to be matched; (8) reference map through exact matching in the affined transformation coordinate that each picture point of figure to be matched step (7) got is corresponding and step (6) carries out fusion treatment, with the output registering images.

According to above technical scheme, can realize following beneficial effect:

(1) the present invention has used the sub-pix bearing calibration, and image is carried out to coordinate adjustment, has improved matching precision; (2) part Hausdorff distance can produce effect preferably to the parts of images in the image that serious coverage or degeneration are arranged; (3) using centered by angle point and build coupling and greatly reduced search time as Hausdorff distance input point set, improved search efficiency.

The accompanying drawing explanation

Fig. 1 is image acquisition of the present invention and pretreatment process figure;

Fig. 2 is image registration algorithm process flow diagram of the present invention;

Fig. 3 is the structural representation that the present invention is applied to the panorama auxiliary parking system.

Embodiment

Accompanying drawing discloses the structural representation of the related preferred embodiment of the utility model without limitation; Explain the technical solution of the utility model below with reference to accompanying drawing.

As shown in Figures 1 to 3, the method for registering images of panorama auxiliary parking system of the present invention, by being arranged on automobile four-way CCD camera collection all around image, as shown in Figure 3.Four-way CCD camera is divided into 8 zones by 360 °, wherein needs the zone of registration to have 4, is respectively ABCD.Collect data and carry out pre-service and store zone subject to registration, as shown in Figure 1.

As shown in Figure 2, at first choose any one zone in ABCD, extract contiguous two-way camera registering images, a width is as reference map, and a width, as match map, adopts respectively improved Harris Robust Algorithm of Image Corner Extraction, and the angular coordinate got is stored respectively.

Then use the Taylor polynomial expression to carry out sub-pix correction, storing coordinate to angle point.

Then adopt thick matching strategy, the Euclidean distance of the angular coordinate of benchmark figure storage, if its Euclidean distance retains with interior angular coordinate in 50 pixels, coordinate beyond 50 pixels is deleted, then angular coordinate Euclidean distance relatively and between figure angle point to be matched, 50 pixels retain with interior coordinate, deletion in addition.After rough matching, stayed the strong angle point that is greater than 50 coordinate spacings.Then adopt improved Hausdorff distance to carry out exact matching.First two images to be processed are carried out to the Canny edge extracting, less threshold value gets 50, larger threshold value gets 150, and mark the position of angle point, take out in match map centered by each angle point, the zone that 25 pixels are radius, centered by each angle point of reference map, the calculating of Hausdorff distance measure is carried out in the zone that 25 pixels are radius respectively.The angle point that distance is less than threshold value Liang Tu center is match point.Angle point after exact matching is stored.Finally use least square method to the angle point of the exact matching that gets to simulating the parameter of affined transformation.Utilize this parameter to carry out affined transformation to match map, and merge with reference map, the output registration results.

Wherein: Preprocessing Algorithm comprises histogram equalization and Gaussian smoothing filtering, is expressed as follows:

A. establish gray level for being normalized to the continuous quantity in scope [0,1], make p _r(r) mean the gray level probability density function in certain Given Graph picture, its subscript is distinguished the probability density function of input picture and output image.If input gray grade is carried out as down conversion, is obtained output gray level s:

known output gray level probability density function is that i.e. this conversion makes the gray level equalization of image uniformly, and result is the image that gets a width dynamic range, and higher contrast is arranged.

B. two-dimensional Gaussian function: when r=± σ, G (r)=Ae ^1/2=0.6A; During r<3 σ, G<0.01 use width is less than 2 σ ²wave filter, i.e. m=2 * 2 σ ²+ 1, work as σ ²=1/2 o'clock, $G^3=\frac{1}{16}\left(\begin{array}{ccc}1& 2& 1\\ 2& 4& 2\\ 1& 2& 1\end{array}\right)$ Distribute and ask discrete template by continuous Gaussian, need sampling, quantize, and make template normalization.

Step (3) feature point extraction adopts improved Harris Robust Algorithm of Image Corner Extraction, is expressed as follows:

A. use a Gaussian window or rectangular window to move on image, by template window, obtain the image second order derivative autocorrelation matrix C (x, y) that original image generates 2*2, be defined as follows:

$C(x,y)=G\left(s\right)\&CircleTimes;\left[\begin{array}{cc}{I}_x& I_x{I}_y\\ I_x{I}_y& I_y\end{array}\right]$

Wherein G (s) is Gauss's template, I _xfor this gradient in the x direction, I _yfor this gradient in the y direction.C (x, y) is asked for to eigenvalue λ ₁, λ ₂set up metric function R ₁, R ₁=det (C)-k (trace (C)) ²det (C)=λ wherein ₁λ ₂, trace (C)=λ ₁+ λ ₂.

B. set up again metric function R ₂, R ₂=min (λ ₁, λ ₂), averaged R:R=(R ₁+ R ₂)/2, according to the threshold value Y of R, Y=(R _1max+ R _2max)/20, wherein R _1max, R _2maxfor metric function R ₁, R ₂maximal value.If R>Y be angle point, otherwise be not angle point.

Step (4) unique point sub-pix is proofreaied and correct and is adopted the Taylor polynomial expression to approach, and is expressed as follows:

A. the initial position of supposing certain angle point is (x ₀, y ₀), its real position should be (x ₀+ s, y ₀+ t).

Wherein (s, t) is error amount, and (s, t) ∈ [0.5,0.5] * [0.5,0.5], to this angle point, uses Taylor polynomial approximate, obtains

$r(x_0+s,y_0+t)=r+\left(\begin{array}{cc}s& t\end{array}\right)\left(\begin{array}{c}{r}_x\\ r_y\end{array}\right)+\frac{1}{2}\left(\begin{array}{cc}s& t\end{array}\right)\left(\begin{array}{cc}{r}_{\mathrm{xx}}& r_{\mathrm{xy}}\\ r_{\mathrm{xy}}& r_{\mathrm{yy}}\end{array}\right)\left(\begin{array}{c}s\\ t\end{array}\right)$

Wherein r is handled image function, r _x, r _ybe respectively its x, the single order partial derivative of y direction, r _xx, r _xy, r _yybe respectively the second-order partial differential coefficient of its respective direction.

B. according to the character of angle point, have: r _xxs+r _xyt+r _x=0, r _xys+r _yyt+r _y=0

Obtain the sub-pixel location error $s=\frac{r_y{r}_{\mathrm{xy}}-r_x{r}_{\mathrm{yy}}}{r_{\mathrm{xx}}{r}_{\mathrm{yy}}-r_{\mathrm{xy}}^2},t=\frac{r_x{r}_{\mathrm{xy}}-r_y{r}_{\mathrm{xx}}}{r_{\mathrm{xx}}{r}_{\mathrm{yy}}-r_{\mathrm{xy}}^2}$

Step (5) unique point is slightly mated and has been adopted a simple and effective strategy, and this strategy is expressed as follows:

1,, in the position of the strong angle point of benchmark, the Euclidean distance of searching for its Corresponding matching figure angular coordinate is that 50 pixels are with interior angle point.

2, when on reference map, choosing candidate point, the Euclidean distance between candidate angular is outside 50 pixels.

Step (6) is carried out the edge extracting binaryzation to reference map, employing be the Canny operator, identical with the match map method for reference map, be expressed as follows:

A. choose the reference map (match map) after gaussian filtering, the big or small M of compute gradient and direction 0.Use the first approximation of the template of 2x2 size as x and y direction partial differential.Partial gradient

edge direction O (x, y)=arctan (G _y/ G _x)

B. gradient is carried out to non-very big inhibition: 8 neighborhoods of each pixel are divided into to 4 sector regions, by gradient direction-22.5 °～22.5 ° and-157.5 °～157.5 °, 22.5 °～67.5 ° and-157.5 °～-112.5 °, 67.5 °～112.5 ° and-112.5 °～-67.5 °, 112.5 °～to be divided into these four sector numbers be 0-3 for 157.5 ° and-67.5 °～-22.5 °, if the central area pixel value is S[i, j], two pixel gradient values in itself and 0-3 sector are compared, if S[i, j] Grad large unlike the Grad of these two pixels, make S[i, j]=0, by the image S[i after processing, j] be stored in N (i, j) in.

C. detect and be connected edge, using 2 threshold value T1 and T2 to do threshold process, wherein T1<T2.The edge pixel that value is greater than T2 is called strong edge pixel and is edge, edge pixel between T1 and T2 is called weak edge pixel, if in the adjacent pixels of weak edge pixel point, marginal point is arranged, thinks that this weak edge pixel point is also marginal point, otherwise, think that this point is non-marginal point.The outline map picture point of the reference map got and figure subject to registration deposits respectively A in, and two points of B are concentrated.

Step (7) exact matching adopts improved Hausdorff distance algorithm, is expressed as follows:

A. the part Hausdorff distance definition of point set A and B is as follows:

H _{k, l}(A, B)=max (h _k(A, B), h _l(B, A)), 1≤k≤p, 1≤l≤q

Wherein

k=<f ₁* p>, 0≤f ₁≤ 1, l=<f ₂* q>, 0≤f ₂≤ 1,<downward rounding operation meaned.

mean that A arranges according to order from small to large to the distance of B point set, the distance that wherein sequence number is k is h _k(A, B).

mean that B arranges according to order from small to large to the distance of A point set, the distance that wherein sequence number is 1 is h _l(B, A), || a-b|| means the distance of concentrated certain the some a of AB point to certain some b.Here choose f ₁, f ₂be 0.8.

B. along the binary image edge, zone centered by each angle point, structure zonule, rectangular area subimage coordinate set with 20*20, ask for respectively reference map and figure part Hausdorff distance subject to registration, get distance threshold τ, if τ=3 distances are less than τ, corresponding angle point is right for coupling, otherwise be not match point, reject.

Step (8) takes affined transformation as the spatial alternation model, match map to be processed, and is expressed as follows:

A. six parameter affined transformations can be used following matrix representation:

$\left[\begin{array}{ccc}{x}_1& y_1& 1\end{array}\right]=\left[\begin{array}{ccc}{x}_0& y_0& 1\end{array}\right]T,T=\left[\begin{array}{ccc}{a}_{11}& a_{12}& 0\\ a_{21}& a_{22}& 0\\ a_{31}& a_{32}& 1\end{array}\right]$

(x wherein ₁, y ₁), (x ₀, y ₀) be respectively two vice processing image corresponding point position coordinateses.Have:

$\left[\begin{array}{c}{x}_1\\ y_1\end{array}\right]=\left[\begin{array}{cc}{a}_{11}& a_{21}\\ a_{12}& a_{22}\end{array}\right]\left[\begin{array}{c}{x}_0\\ y_0\end{array}\right]+\left[\begin{array}{c}{a}_{31}\\ a_{32}\end{array}\right]$

Transition form is:

$x_1=\left[\begin{array}{ccc}{a}_{11}& a_{21}& a_{31}\end{array}\right]\left[\begin{array}{c}{x}_0\\ y_0\\ 1\end{array}\right],y_1=\left[\begin{array}{ccc}{a}_{12}& a_{22}& a_{32}\end{array}\right]\left[\begin{array}{c}{x}_0\\ y_0\\ 1\end{array}\right]$

B. get N+1 to match point, adopt the least square fitting optimal parameter:

$\left\{\begin{array}{c}\left[\begin{array}{ccc}{x}_{10}& L& x_{1N}\end{array}\right]=\left[\begin{array}{ccc}{a}_{11}& a_{21}& a_{31}\end{array}\right]\left[\begin{array}{cccc}{x}_{00}& x_{01}& L& x_{0N}\\ y_{00}& y_{01}& L& y_{0N}\\ 1& 1& L& 1\end{array}\right]\\ \left[\begin{array}{ccc}{y}_{10}& L& y_{1N}\end{array}\right]=\begin{array}{}\end{array}\left[\begin{array}{ccc}{a}_{12}& a_{22}& a_{32}\end{array}\right]\left[\begin{array}{cccc}{x}_{00}& x_{01}& L& x_{0N}\\ y_{00}& y_{01}& L& y_{0N}\\ 1& 1& L& 1\end{array}\right]\end{array}\right.$

X wherein ₁₀l x _1Nwith x ₀₀l x _0Nfor N+1 to the match point coordinate.

From the least square method matrix form: shape is A as the solution of overdetermined equation minimum meaning under least square method of Ax=b ^tax=A ^tb, can release the approximate solution of six parameter affine transformation parameters under least square method and be:

$\left\{\begin{array}{c}\left[\begin{array}{ccc}{a}_{11}& a_{21}& a_{31}\end{array}\right]=\left[\begin{array}{ccc}{x}_{10}& L& x_{1N}\end{array}\right]A^T{\left({\mathrm{AA}}^T\right)}^{-1}\\ \left[\begin{array}{ccc}{a}_{21}& a_{22}& a_{32}\end{array}\right]=\left[\begin{array}{ccc}{y}_{10}& L& y_{1N}\end{array}\right]A^T{\left({\mathrm{AA}}^T\right)}^{-1}\end{array}\right.$

Wherein, $A=\left[\begin{array}{cccc}{x}_{00}& x_{01}& L& x_{0N}\\ y_{00}& y_{01}& L& y_{0N}\\ 1& 1& L& 1\end{array}\right].$

Claims (3)

1. the method for registering images of a panorama auxiliary parking system, is characterized in that, comprises the following steps: (1) gathers the view data of reference map and figure subject to registration; (2) reference map respectively step (1) collected and the view data of figure subject to registration are carried out histogram equalization, Gaussian smoothing filter preprocessing successively; (3) adopt respectively the Harris corner extraction to choose through the described pretreated reference map of step (2) and figure subject to registration, carrying out unique point, and corresponding angular coordinate is stored respectively by the unique point got; (4) unique point of the image subject to registration that adopts the Taylor polynomial expression to extract step (3) is carried out the sub-pix correction, and by the sub-pix correction feature point storage got; (5) at first in reference map, choose candidate angular, the Euclidean distance between each candidate angular is greater than 50 pixels; Then, in the position of each reference map candidate angular, search for the unique point coordinate of its corresponding figure subject to registration, and both Euclidean distances are compared, retain the sub-pix correction feature point in 50 pixels, complete the thick coupling of reference map and figure subject to registration; (6) at first adopt the Canny operator respectively reference map and figure subject to registration through the described thick coupling of step (5) to be carried out to the edge extracting binaryzation, then adopt part Hausdorff distance algorithm to carry out the exact matching of reference map and figure subject to registration; (7) adopt affine Transform Model to carry out the spatial alternation processing to the picture point subject to registration after the described exact matching of step (6), obtain the corresponding affined transformation coordinate of each picture point of figure subject to registration; (8) reference map through exact matching in the affined transformation coordinate that each picture point of figure subject to registration step (7) got is corresponding and step (6) carries out fusion treatment, with the output registering images.

2. the method for registering images of panorama auxiliary parking system according to claim 1, is characterized in that, the described Harris corner extraction of step (3) is:

A), at first use template window to move on original image, this template window is Gaussian window or rectangular window, and the original image of then template window being obtained generates the image second order derivative autocorrelation matrix of 2*2

:

Wherein,

for Gauss's template,

for this gradient in the x direction,

for this gradient in the y direction;

B), ask for

eigenwert

,

, and set up metric function

:

; Wherein

,

, k=0.04;

C), set up again metric function :

, averaged R:

;

D), selected threshold Y:

, wherein

,

be respectively metric function

,

maximal value;

E), the relation of judgement R and Y, if

, be angle point, otherwise be not angle point.

3. the method for registering images of panorama auxiliary parking system according to claim 1, it is characterized in that, 1., partial gradient function M and the edge direction function O of computed image the described exact matching of step (6) comprises the following steps:: use the template of 2x2 as the x of two-dimensional Gaussian function and the first approximation of y direction partial differential, partial gradient function

, the edge direction function

2., gradient is carried out to non-very big inhibition: at first, 8 neighborhoods of each pixel are divided into to 4 sector regions that are numbered 0-3, wherein, the gradient direction of No. 0 sector region is :-22.5 ° ~ 22.5 ° and-157.5 ° ~ 157.5 °, the gradient direction of No. 1 sector region is: 22.5 ° ~ 67.5 ° and-157.5 ° ~-112.5 °, the gradient direction of No. 2 sector regions is: 67.5 ° ~ 112.5 ° and-112.5 ° ~-67.5 °, the gradient direction of No. 3 sector regions is: 112.5 ° ~ 157.5 ° and-67.5 ° ~-22.5 °; Then, establishing the central area pixel is

, it is compared with two pixel gradient values of 0-3 sector region respectively, if

Grad large unlike the Grad of these two pixels, the order

, will

Be stored in

In,

It is the image after non-very big inhibition; 3., use two threshold value T1 and T2 couple Do threshold process, T1<T2 wherein, the edge pixel that value is greater than T2 is called strong edge pixel point, and this point is marginal point, and the edge pixel between T1 and T2 is called weak edge pixel point, more further judges according to edge connectivity whether it is marginal point; If in the adjacent pixels of weak edge pixel point, marginal point is arranged, think that this weak edge pixel point is also marginal point, otherwise, think that this point is non-marginal point; The edge graph picture point of the reference map got and figure subject to registration deposits respectively A in, and two points of B are concentrated; 4.,

For part Hausdorff distance:

,

Mean that A arranges according to order from small to large to the distance of B point set, wherein sequence number is

Distance be , same B to the A point set from small to large sequence number be

Distance be

Wherein

,

,

Be respectively

,

Maximum,

Mean the distance of concentrated certain the some a of AB point to certain some b; 5., along the binary image edge, build zonule subimage coordinate set, ask for respectively part Hausdorff distance, choose

,

Be 0.8, part hausdorff distance is less than threshold value

The time De Liangtu center angle point think match point, wherein,

.

Images