CN105957010A - Vehicle-mounted image splicing system - Google Patents

Abstract

A vehicle-mounted image splicing system disclosed by the present invention comprises obtaining an original environmental image via a camera arranged on an automobile, querying a splicing area corresponding to the original environmental image in a target image, and further constructing a direction gradient map based on the relevance between the original environmental image and the splicing area; carrying out the regularization processing on the direction gradient map to obtain a common feature space, and further constructing an energy function of the common feature space based on the relevance between the similarity distance measure and the obtained common feature space; solving the optimal solution enabling the value of the energy function to be minimum, taking the optimal solution as a final pixel value of the original environmental image, valuing the splicing area according to the final pixel value and then displaying. By implementing the present invention, the calculation amount can be reduced, the calculation speed is improved, the problems that the matching performance of the image matching feature points is low, so that the distortion rate of the spliced images is reduced, etc., are avoided.

Description

A kind of vehicle-mounted image mosaic system

Technical field

The invention belongs to technical field of automotive electronics, particularly relate to a kind of vehicle-mounted image mosaic system.

Background technology

Panorama Mosaic, as emerging technology, was developed rapidly in recent years, was also got more and more The concern of researcher.In Panoptic visualization auxiliary is parked, need the technology next life by Panorama Mosaic Become the panoramic view around vehicle body.

At present, panorama mosaic method of the prior art uses an overall homography matrix, i.e. should with a list Matrix represents the perspective transform relation between the image of input.As a example by two image mosaic, first obtain The matching characteristic point of two images, so-called matching characteristic point that is two characteristic point spatially represents same point；So Afterwards according to the matching characteristic point obtained, solve homography matrix；Finally, according to this homography matrix by a wherein width All pixels on image convert, and determine at the correspondence position of another piece image place plane, to obtain final product To the splicing result of two width figures, thus further image is carried out color blend etc. and processes, preferably spelled Map interlinking.

But, this joining method exists computationally intensive, calculates slow-footed shortcoming, and figure easily occurs As the matching of matching characteristic point is low so that the pattern distortion generation rate of splicing reduces.

Summary of the invention

The technical problem to be solved is, it is provided that a kind of vehicle-mounted image mosaic system, it is possible to reduce Amount of calculation, improves and calculates speed, it is to avoid occur that the matching of images match characteristic point is low so that the figure of splicing The problems such as image distortion generation rate reduction.

For solving above-mentioned technical problem, the embodiment of the present invention provides a kind of vehicle-mounted image mosaic system, described system System includes:

Direction gradient figure construction unit, for the photographic head acquisition primal environment image by being arranged at automobile, And inquire the splicing regions corresponding with described primal environment image in the target image, and it is based further on Relatedness between described primal environment image and described splicing regions builds direction gradient figure；

Energy function acquiring unit, for described direction gradient figure is carried out Regularization, obtains general character special Levy space, and be based further on similarity distance and estimate associating between the described common feature space obtained Property, construct the energy function in described common feature space；

Splicing unit, for solving the optimal solution minimum so that the value of described energy function, by described Excellent solution is as the final pixel value of described primal environment image, and further according to described final pixel value assignment Show to after described splicing regions.

Wherein, the target image splicing mapping table that described splicing regions is preset by inquiry realizes；Wherein, Described map information at least includes the pixel coordinate information of the sequence number of primal environment image, primal environment image.

Wherein, described energy function acquiring unit includes:

Common feature space acquisition module, is used for passing through formulaRight Described direction gradient figure carries out Regularization, obtains common feature space；Wherein, G (x, y) be described in obtain Common feature space；| I (x, y) | for the gradient modulus value of described direction gradient figure；(x y) is described direction ladder to W Du Tunei has the center window of certain size；K is fixed constant, can value be 100；

Energy function acquisition module, for based on variation principle, uses the combination of multiple distance measure to constitute institute State the energy function in common feature space so that the energy function in described common feature space can be expressed as: E (p, x, y)=-E_N(p,x,y)+λ₁E_H(p,x,y)-λ₂E_G(p,x,y)；Wherein, (p, x y) are described common feature to E The energy function in space；E_N(p,x,y),E_H(p,x,y),E_G(p, x y) represent respectively based on the normalization phase going average Close coefficient measure energy function, energy function based on Hausdorff distance and unite based on local maximum mask The energy function of metering；λ₁, λ₂Represent Lagrange multiplier weight；P represents anamorphose parameter.

Implement the embodiment of the present invention, there is following beneficial effect:

The vehicle-mounted image split-joint method that the present invention provides, single pass completes the detection of primal environment image, and Completing the estimation of target distortion parameter while detection, (same target is in difference to solve multiple target in image Imaging performance under observation geometry) quickly, the difficult problem such as high detection rate, reduce amount of calculation, improve calculating Speed, it is achieved the purpose that the coupling reliability of images match characteristic point is high and precision is high, it is to avoid the image of splicing The problems such as distortion generation rate reduction.

Accompanying drawing explanation

In order to be illustrated more clearly that the embodiment of the present invention or technical scheme of the prior art, below will be to enforcement In example or description of the prior art, the required accompanying drawing used is briefly described, it should be apparent that, describe below In accompanying drawing be only some embodiments of the present invention, for those of ordinary skill in the art, do not paying On the premise of going out creative work, it is also possible to obtain other accompanying drawing according to these accompanying drawings.

The schematic flow sheet of a kind of vehicle-mounted image mosaic system that Fig. 1 provides for the embodiment of the present invention；

To target image in the one vehicle-mounted image mosaic system application scenarios that Fig. 2 provides for the embodiment of the present invention Carry out the structural representation of region division；

Fig. 3 is the structural representation that the present invention provides a kind of vehicle-mounted image mosaic system of embodiment.

Detailed description of the invention

Below in conjunction with the accompanying drawing in the embodiment of the present invention, the technical scheme in the embodiment of the present invention is carried out clearly Chu, be fully described by, it is clear that described embodiment be only a part of embodiment of the present invention rather than Whole embodiments.Based on the embodiment in the present invention, those of ordinary skill in the art are not making creation The every other embodiment obtained under property work premise, broadly falls into the scope of protection of the invention.

As it is shown in figure 1, be in the embodiment of the present invention, it is provided that the vehicle-mounted image split-joint method of one, described side Method includes:

Step S10, obtain primal environment image by being arranged at the photographic head of automobile, and look in the target image Ask the splicing regions corresponding with described primal environment image, and be based further on described primal environment image And the relatedness between described splicing regions builds direction gradient figure；

Specifically, obtain a primal environment image by being arranged at any one photographic head of automobile, each former Beginning ambient image all to there being a sequence number, the coordinate position of the pixel of each width primal environment image, as The information such as element value can be recorded.Now, target image is needs splicing the panoramic picture that shows ( Spliced aerial view eventually), and splicing regions is that the target image splicing mapping table preset by inquiry is real Existing, in this target image splicing mapping table, storage has each pixel and primal environment image in target image Sequence number, corresponding relation between pixel coordinate.

In order to get rid of background noise and the impact on primal environment image of the imaging illumination condition, obtain pixel more smart True spliced panoramic figure, therefore devises the feature space of a kind of novelty: the feature space of regularization gradient. This feature space is realized by the gradient modulus value of calculated level with vertical direction, it is therefore desirable to based on original ring Relatedness between border image and splicing regions forms direction gradient figure, and direction gradient map is two-dimensional gradient figure, It is formed by horizontal direction gradient map and vertical gradient figure.

The structure of direction gradient figure, can pass through formula (1) and realize:

I (x, y)=I_x(x,y)+i×▽I_y(x,y) (1)；

In formula (1), I_x(x, y), I_y(x, y) respectively represent x Yu y to gradient map；Wherein, ▽I_x(x, y)=I (and x+1, y)+I (x-1, y)-2 × I (x, y), ▽I_y(x, y)=I (x, y+1)+I (x, y-1)-2 × I (x, y)；

Due to I (x, y) is a plural number, then can calculate its range value and gradient direction (gradient direction angle) thereof, Its computing formula (1) deformation is as follows:

$|\▿I(x,y)|=\sqrt{\▿I_x{(x,y)}^2+\▿I_y{(x,y)}^2}---\left(2\right);$

θ=atg (I_y(x,y)/▽I_x(x,y)) (3)；

In embodiments of the present invention, the concrete steps of target image splicing mapping table are accomplished by

(1) target image is divided into multiple target area according to world coordinate system, and will be located in two targets Region in the preset range of region intersection is defined as splicing regions, and determines each target area and spelling Connect the sequence number of primal environment image corresponding to region；

For ease of understanding, refer to shown in Fig. 2, it illustrates target image is carried out region division one The schematic diagram of embodiment, in this embodiment, in the target image, will be in automobile region all around Marking off tetra-regions of concrete F, B, L, R, wherein, point P1～P4 is four tops of the surrounding of automobile Point, and the line segment being made up of some P3, P4 and the line segment that is made up of some P5, P6 form two of target image Border, can obtain h0～h3 in target image, w1, H_CAR by measuring, and W_CAR's etc. is concrete Range information, wherein, H_CAR is the length of automobile image in target image, and W_CAR is target figure The width of automobile image in Xiang, it is to be understood that according to length and the physical length of automobile of this H_CAR Deng scaling translation relation, may be used to determine in target image pixel in pixel and primal environment image Coordinate between mapping relations.Simultaneously, it may be determined that the pixel value of the pixel in each region is from which Width primal environment image, such as, the pixel value of the pixel in F region can come from the (figure of photographic head before automobile Represent this photographic head with the constitutional diagram of small circle and little square frame in 3, lower with) captured by primal environment figure Picture, the pixel value of the pixel in Zone R territory is from the primal environment image captured by car right side photographic head, L district The pixel value of the pixel in territory is from the primal environment image captured by automobile left side photographic head, the pixel in B region The pixel value of point is from the primal environment image captured by automotive back photographic head；Captured by different photographic head Primal environment image can be made a distinction by different ambient image sequence numbers i.

(2) according to the mapping relations between primal environment image coordinate system and world coordinate system, and target figure Mapping relations between coordinate system and the world coordinate system of picture, it is thus achieved that on target image each target pixel points with The map information that original image maps mutually；Wherein, in the splicing regions of target image, each object pixel The specific pixel point on primal environment image that point is different from two width respectively maps mutually；

It is understood that in the splicing regions of target image, owing to needing to carry out the fusion of pixel, A specific pixel point on the primal environment image that each pixel is different from two width respectively maps mutually；Such as, The pixel value of the pixel in A1 region in target image, needs according to automobile front side photographic head and left side photographic head In captured picture, the pixel of the pixel of a specified point carries out calculating acquisition respectively；

Specifically, note primal environment image coordinate is (u_0i,v_0i), target image coordinate be (u, v), described target figure As the mapping relations of each pixel upper Yu original image are primal environment image coordinate (u_0i,v_0i) and environment Picture numbers i and target image coordinate (u, v) between mapping relations.Take world coordinates (x_w,y_w,z_w) as middle Amount, find out respectively (u, v) with (u_0i,v_0iWith its mapping relations both), it is hereby achieved that (u, v) with (u_0i,v_0i) it Between mapping relations.

Specifically, in one example, the coordinate mapping relations of primal environment image are determined by imaging model, Such as, the relation between photographic head coordinate system and coordinates of original image coordinates system, such as scara model can be used.

And mapping relations between target image (top view) and world coordinates are relatively simple, world coordinates arrives Only through scaling, the conversion process of translation between top view coordinate.In short, target image and body of a motor car The region around needing display is scaling relation, therefore by the coordinate of target image, can calculate corresponding vehicle body Coordinate, then joined by outside photographic head, calculate corresponding photographic head coordinate, finally according to camera internal reference, meter Calculate the coordinate of primal environment image.

During above-mentioned Coordinate Conversion, from (u v) is mapped to world coordinates (x_w,y_w,z_wAfter), can basis (x_w,y_w,z_wRegion (F, L, R, B) belonging to), determines the value of sequence number i of ambient image.

(3) by positional information and corresponding the reflecting of each target pixel points of each target pixel points of target image The information of penetrating preserves, it is thus achieved that target image splicing mapping table.

Step S20, described direction gradient figure is carried out Regularization, obtain common feature space, go forward side by side one Step estimates the relatedness between the described common feature space obtained based on similarity distance, constructs described The energy function in common feature space；

Specifically, direction gradient map is carried out Regularization by formula (4), obtain common feature space, Specific as follows；

$G(x,y)=\frac{|\▿I(x,y)|}{{\max }_{(u,v)\∈W(x,y)}\left(\right|\▿I\left(u,v\right)\left|\right)+K}---\left(4\right);$

In formula (4), G (x, y) be described in the common feature space that obtains；| I (x, y) | for described direction gradient figure Gradient modulus value；(x y) is the center window in described direction gradient figure with certain size to W；K is solid Permanent number, can value be 100；

Based on variation principle, the combination of multiple distance measure is used to constitute the energy function in common feature space, The energy function making common feature space is represented by formula (5), specific as follows:

E (p, x, y)=-E_N(p,x,y)+λ₁E_H(p,x,y)-λ₂E_G(p,x,y) (5)；

In formula (5), (p, x y) are the energy function in common feature space to E；E_N(p,x,y),E_H(p,x,y),E_G(p,x,y) Represent that the normalizated correlation coefficient based on going average measures energy function, energy based on Hausdorff distance respectively Flow function and energy function based on local maximum mask statistic；λ₁, λ₂Represent Lagrange multiplier weight； P represents anamorphose parameter.

Under the conditions of affine, the distorted pattern of image such as formula (6):

$\left(\begin{array}{c}u\\ v\end{array}\right)=\left(\begin{array}{cc}a& b\\ c& c\end{array}\right)\left(\begin{array}{c}x\\ y\end{array}\right)+\left(\begin{array}{c}\mathrm{\Δ}x\\ \mathrm{\Δ}y\end{array}\right)---\left(6\right);$

At this moment p=(a, b, c, d, Δ x, Δ y)^T。

When certain conditions are met, the distorted pattern formula (6) of image can be reduced to formula (7):

$\left(\begin{array}{c}u\\ v\end{array}\right)=-r\left(\begin{array}{cc}\cos \mathrm{\θ}& \sin \mathrm{\θ}\\ -\sin \mathrm{\θ}& \cos \mathrm{\θ}\end{array}\right)\left(\begin{array}{c}x\\ y\end{array}\right)+\left(\begin{array}{c}\mathrm{\Δ}x\\ \mathrm{\Δ}y\end{array}\right)---\left(7\right);$

Wherein p=(a, b, c, d, Δ x, Δ y)^T。

In summary, the acquisition of anamorphose parameter P, then represent the confirmation of desired value in common feature space, Each anamorphose parameter P is made to be represented as a pixel value, therefore can be by common feature space (i.e. each Local Extremum corresponds to one in the quickly location of each Local Extremum on energy function Detection target), thus realize mating between primal environment image and target image.

Step S30, solve the optimal solution minimum so that the value of described energy function, using described optimal solution as institute State the final pixel value of primal environment image, and be assigned to described splicing according to described final pixel value further Show behind region.

Specifically, use multiparticle group's algorithm that the energy function in common feature space is optimized, try to achieve altogether The optimal value of anamorphose parameter P in the energy function of property feature space, using optimal solution as primal environment figure The final pixel value of picture, and be assigned to after splicing regions show according to final pixel value further, thus have Effect that traditional algorithm is incomparable and advantage；Simultaneously in order to reach more accurately and the arithmetic result of efficiency, and And can be dynamically generated or eliminate population, reduce unnecessary amount of calculation, thus reach to realize image The purpose that the reliability joined is high and precision is high, adds convergence and the judgement of repellency, adds population Between interaction.

The energy function in common feature space use multiparticle group's algorithm to implement step as follows:

Step S301, determine the parameter of population；Wherein, parameter includes equal with the quantity of matching detection Primary group's quantity, and also include population convergence radius, population repel radius, individual study because of Son, population Studying factors and inertia weight；

Step S302, initialization population, it includes arranging maximum iteration time and primary iteration number of times is 0, And the random speed setting each population and the velocity attitude of correspondence thereof, random set in each population The locus of each particle；

Step S303, acquisition current iteration number of times, and judge whether the current iteration number of times got is less than Big iterations；If it is, perform next step S304；If it is not, then redirect execution step S305；

Step S304, the current iteration number of times got add one, and carry out each particle in each population Traversal updates, and according to described default individual Studying factors, population Studying factors and inertia weight, and According to the particle in the current optimal value traveled through in evolution track and all population of particle in single particle group Optimal value in current traversal evolution track, obtains each population speed after traversal updates；And

Each population speed after updating according to the traversal obtained, is updated the position of each population, and After sequentially carrying out the population after updating position repelling determination processing and convergence determination processing, return step S303；

Step S305, termination carry out traversal and update each particle in each population, and filter out each particle Particle optimal value in the previous evolution track terminating traversal in group, and further by each population of screening The optimal value of interior particle exports as corresponding Local Extremum.

As it is shown on figure 3, be in the embodiment of the present invention, it is provided that one vehicle-mounted image mosaic system, described system System includes:

Direction gradient figure construction unit 10, for the photographic head acquisition primal environment image by being arranged at automobile, And inquire the splicing regions corresponding with described primal environment image in the target image, and it is based further on Relatedness between described primal environment image and described splicing regions builds direction gradient figure；

Energy function acquiring unit 12, for described direction gradient figure is carried out Regularization, obtains general character Feature space, and be based further on similarity distance and estimate the pass between the described common feature space obtained Connection property, constructs the energy function in described common feature space；

Splicing unit 14, for solving the optimal solution that the value so that described energy function is minimum, by described Optimal solution is as the final pixel value of described primal environment image, and composes according to described final pixel value further It is worth and shows to after described splicing regions.

Wherein, the target image splicing mapping table that described splicing regions is preset by inquiry realizes；Wherein, Described map information at least includes the pixel coordinate information of the sequence number of primal environment image, primal environment image.

Wherein, described energy function acquiring unit 12 includes:

Common feature space acquisition module, is used for passing through formulaRight Described direction gradient figure carries out Regularization, obtains common feature space；Wherein, G (x, y) be described in obtain Common feature space；| I (x, y) | for the gradient modulus value of described direction gradient figure；(x y) is described direction ladder to W Du Tunei has the center window of certain size；K is fixed constant, can value be 100；

Energy function acquisition module, for based on variation principle, uses the combination of multiple distance measure to constitute institute State the energy function in common feature space so that the energy function in described common feature space can be expressed as: E (p, x, y)=-E_N(p,x,y)+λ₁E_H(p,x,y)-λ₂E_G(p,x,y)；Wherein, (p, x y) are described common feature to E The energy function in space；E_N(p,x,y),E_H(p,x,y),E_G(p, x y) represent respectively based on the normalization phase going average Close coefficient measure energy function, energy function based on Hausdorff distance and unite based on local maximum mask The energy function of metering；λ₁, λ₂Represent Lagrange multiplier weight；P represents anamorphose parameter.

Implement the embodiment of the present invention, there is following beneficial effect:

The vehicle-mounted image split-joint method that the present invention provides, single pass completes the detection of primal environment image, and Completing the estimation of target distortion parameter while detection, (same target is in difference to solve multiple target in image Imaging performance under observation geometry) quickly, the difficult problem such as high detection rate, reduce amount of calculation, improve calculating Speed, it is achieved the purpose that the coupling reliability of images match characteristic point is high and precision is high, it is to avoid the image of splicing The problems such as distortion generation rate reduction.

It should be noted that in said system embodiment, each included system unit is according to function Logic carries out dividing, but is not limited to above-mentioned division, as long as being capable of corresponding function； It addition, the specific name of each functional unit is also only to facilitate mutually distinguish, it is not limited to the present invention Protection domain.

One of ordinary skill in the art will appreciate that all or part of step realizing in above-described embodiment method is Can instruct relevant hardware by program to complete, described program can be stored in a computer-readable Taking in storage medium, described storage medium, such as ROM/RAM, disk, CD etc..

Above disclosed only one preferred embodiment of the present invention, can not limit this with this certainly Bright interest field, the equivalent variations therefore made according to the claims in the present invention, still belong to what the present invention was contained Scope.

Claims (3)

1. a vehicle-mounted image mosaic system, it is characterised in that described system includes:

Direction gradient figure construction unit, for the photographic head acquisition primal environment image by being arranged at automobile, And inquire the splicing regions corresponding with described primal environment image in the target image, and it is based further on Relatedness between described primal environment image and described splicing regions builds direction gradient figure；

Energy function acquiring unit, for described direction gradient figure is carried out Regularization, obtains general character special Levy space, and be based further on similarity distance and estimate associating between the described common feature space obtained Property, construct the energy function in described common feature space；

Splicing unit, for solving the optimal solution minimum so that the value of described energy function, by described Excellent solution is as the final pixel value of described primal environment image, and further according to described final pixel value assignment Show to after described splicing regions.

2. system as claimed in claim 2, it is characterised in that described splicing regions is preset by inquiry Target image splicing mapping table realizes；Wherein, described map information at least includes the sequence of primal environment image Number, the pixel coordinate information of primal environment image.

3. system as claimed in claim 2, it is characterised in that described energy function acquiring unit includes:

Common feature space acquisition module, is used for passing through formulaRight Described direction gradient figure carries out Regularization, obtains common feature space；Wherein, G (x, y) be described in obtain Common feature space；| I (x, y) | for the gradient modulus value of described direction gradient figure；(x y) is described direction ladder to W Du Tunei has the center window of certain size；K is fixed constant, can value be 100；

Energy function acquisition module, for based on variation principle, uses the combination of multiple distance measure to constitute institute State the energy function in common feature space so that the energy function in described common feature space can be expressed as: E (p, x, y)=-E_N(p,x,y)+λ₁E_H(p,x,y)-λ₂E_G(p,x,y)；Wherein, (p, x y) are described common feature to E The energy function in space；E_N(p,x,y),E_H(p,x,y),E_G(p, x y) represent respectively based on the normalization phase going average Close coefficient measure energy function, energy function based on Hausdorff distance and unite based on local maximum mask The energy function of metering；λ₁, λ₂Represent Lagrange multiplier weight；P represents anamorphose parameter.