CN107424120A - A kind of image split-joint method in panoramic looking-around system - Google Patents
A kind of image split-joint method in panoramic looking-around system Download PDFInfo
- Publication number
- CN107424120A CN107424120A CN201710237136.8A CN201710237136A CN107424120A CN 107424120 A CN107424120 A CN 107424120A CN 201710237136 A CN201710237136 A CN 201710237136A CN 107424120 A CN107424120 A CN 107424120A
- Authority
- CN
- China
- Prior art keywords
- mrow
- mtd
- msub
- mtr
- image
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000000034 method Methods 0.000 title claims abstract description 85
- 238000006243 chemical reaction Methods 0.000 claims abstract description 57
- 238000013507 mapping Methods 0.000 claims abstract description 33
- 241000251468 Actinopterygii Species 0.000 claims abstract description 25
- 238000012937 correction Methods 0.000 claims abstract description 25
- 238000012549 training Methods 0.000 claims abstract description 23
- 230000008569 process Effects 0.000 claims abstract description 14
- 238000009434 installation Methods 0.000 claims description 17
- 230000000007 visual effect Effects 0.000 claims description 17
- 230000009021 linear effect Effects 0.000 claims description 14
- 230000003287 optical effect Effects 0.000 claims description 10
- 230000008859 change Effects 0.000 claims description 6
- 238000002594 fluoroscopy Methods 0.000 claims description 6
- 239000011159 matrix material Substances 0.000 claims description 6
- 238000009877 rendering Methods 0.000 claims description 6
- 230000009466 transformation Effects 0.000 claims description 5
- 241000193935 Araneus diadematus Species 0.000 claims description 3
- 240000007594 Oryza sativa Species 0.000 claims description 3
- 235000007164 Oryza sativa Nutrition 0.000 claims description 3
- 238000004364 calculation method Methods 0.000 claims description 3
- 238000002474 experimental method Methods 0.000 claims description 3
- 238000003706 image smoothing Methods 0.000 claims description 3
- 238000012886 linear function Methods 0.000 claims description 3
- 235000009566 rice Nutrition 0.000 claims description 3
- 238000012546 transfer Methods 0.000 claims description 3
- 230000007704 transition Effects 0.000 claims description 3
- 238000013519 translation Methods 0.000 claims description 3
- XLYOFNOQVPJJNP-UHFFFAOYSA-N water Substances O XLYOFNOQVPJJNP-UHFFFAOYSA-N 0.000 claims 1
- 230000006872 improvement Effects 0.000 description 7
- 238000010586 diagram Methods 0.000 description 3
- 238000003384 imaging method Methods 0.000 description 3
- 230000000694 effects Effects 0.000 description 2
- NAWXUBYGYWOOIX-SFHVURJKSA-N (2s)-2-[[4-[2-(2,4-diaminoquinazolin-6-yl)ethyl]benzoyl]amino]-4-methylidenepentanedioic acid Chemical compound C1=CC2=NC(N)=NC(N)=C2C=C1CCC1=CC=C(C(=O)N[C@@H](CC(=C)C(O)=O)C(O)=O)C=C1 NAWXUBYGYWOOIX-SFHVURJKSA-N 0.000 description 1
- 241001212149 Cathetus Species 0.000 description 1
- 244000170916 Paeonia officinalis Species 0.000 description 1
- 235000006484 Paeonia officinalis Nutrition 0.000 description 1
- 244000131316 Panax pseudoginseng Species 0.000 description 1
- 235000005035 Panax pseudoginseng ssp. pseudoginseng Nutrition 0.000 description 1
- 235000003140 Panax quinquefolius Nutrition 0.000 description 1
- 235000004522 Pentaglottis sempervirens Nutrition 0.000 description 1
- 238000011161 development Methods 0.000 description 1
- 235000008434 ginseng Nutrition 0.000 description 1
- 238000011900 installation process Methods 0.000 description 1
- 238000012545 processing Methods 0.000 description 1
- 239000007787 solid Substances 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T3/00—Geometric image transformations in the plane of the image
- G06T3/40—Scaling of whole images or parts thereof, e.g. expanding or contracting
- G06T3/4038—Image mosaicing, e.g. composing plane images from plane sub-images
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/24—Classification techniques
- G06F18/241—Classification techniques relating to the classification model, e.g. parametric or non-parametric approaches
- G06F18/2411—Classification techniques relating to the classification model, e.g. parametric or non-parametric approaches based on the proximity to a decision surface, e.g. support vector machines
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T5/00—Image enhancement or restoration
- G06T5/73—Deblurring; Sharpening
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T5/00—Image enhancement or restoration
- G06T5/80—Geometric correction
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20212—Image combination
- G06T2207/20221—Image fusion; Image merging
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Data Mining & Analysis (AREA)
- Artificial Intelligence (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Evolutionary Biology (AREA)
- Evolutionary Computation (AREA)
- Bioinformatics & Computational Biology (AREA)
- General Engineering & Computer Science (AREA)
- Bioinformatics & Cheminformatics (AREA)
- Life Sciences & Earth Sciences (AREA)
- Image Processing (AREA)
Abstract
The present invention is the image split-joint method in a kind of panoramic looking-around system, and it is related to computer vision field.This method comprises the following steps：1) combine septal line training pattern and SVM algorithm carries out fish eye images distortion correction；2) establish image and overlook conversion look-up table to realize quickly vertical view conversion, obtain birds-eye view；3) Panorama Mosaic mapping table is generated, panoramic looking-around birds-eye view is rapidly obtained by way of searching Panorama Mosaic mapping table；4) solve the difference in brightness between stitching image two-by-two using improved brightness reconciliation process algorithm, splicing seams are further then eliminated using the image interfusion method of weighted mean method.This method effectively reduces overhead, quickly realizes seamless image splicing, obtains panoramic looking-around birds-eye view.
Description
Technical field
The present invention relates to computer vision field, refers in particular to the image split-joint method in a kind of panoramic looking-around system.
Background technology
With the fast development of national economy, automobile total quantity is continuously increased, and traffic environment also becomes increasingly congestion, causes
Automobile often travels in narrow and small environment.Automobile is run in the guide in narrow parking lot by limited road or wagon flow and stopped,
Due to the limited view of driver, easily collide, cause unnecessary loss.With digital processing unit, image first-class set
The cost matched somebody with somebody constantly reduces, based on the automobile accessory system of panoramic looking-around system by as the master of future automobile vision-aided system
Stream, and the important technical of panoramic looking-around system is to carry out the splicing of panoramic looking-around image.Pass through panoramic looking-around image mosaic
Obtain panoramic picture and be output on display so that driver can comprehensively understand the situation around vehicle body, so as to reduce
The generation of car accident.
Panoramic looking-around image mosaic step based on fish-eye camera generally comprises：Image distortion correction, perspective view are to bowing
The vertical view conversion and the splicing of multiple image of view.Fish eye lens can obtain large range of visual angle and complete spherical diagram
Picture, but fish-eye complicated optical structure causes fish eye images serious distortion to be present, so before image mosaic, first
Carry out image distortion correction.The method of utilization space coordinate system transformation realizes perspective transform, and this method is easy and effective, as long as really
Determining camera installation parameter (installation parameter such as camera position, angle) can calculate, but be missed if existed in installation process
Difference, vertical view conversion directly is carried out according to installation parameter, correct top view can not be obtained.Although the image mosaic based on region
The image split-joint method of method and feature based can complete the splicing of image, but this two kinds of joining methods require spliced map
A range of overlapping region be present as between, and image there can not be too big distortion.But panoramic looking-around image mosaic process
In vertical view conversion and the image of image difference computing a certain degree of distortion be present, therefore above two joining method is being carried out
Preferable result can not be obtained during image mosaic, or even the splicing of panoramic looking-around image can not be carried out.
The content of the invention
The technical problem to be solved in the present invention generates panorama ring to propose that it is seamless spliced that one kind can be carried out fast and effeciently
Depending on the image split-joint method in the panoramic looking-around system of birds-eye view, the birds-eye view of generation shows vehicle-surroundings letter with 360 degree of visual angles
Breath, can dead zone-eliminating and dead angle, there is good application value.
Comprise the following steps in order to solve the above technical problems, the present invention adopts the technical scheme that：
Step 1) combines septal line training pattern and SVM algorithm carries out fish eye images distortion correction, and it includes：
S1.1 builds septal line training pattern；
S1.2 carries out fish eye images correction using SVM algorithm；
Step 2) is using the relation between world coordinate system, camera coordinate system and image coordinate system, with reference to backward mapping
Method carries out conversion of the perspective view to top view, error be present for video camera installation, according to camera coordinates system
Around XC、YC、ZCThe deflection angle of axle re-calibrates vertical view changing image, then establishes image and overlooks conversion look-up table to realize
Conversion is quickly overlooked, obtains birds-eye view, it includes：
S2.1 establishes world coordinate system and camera coordinate system；
S2.2 is solved using backward mapping method and is overlooked conversion；
There is error in S2.3, construct and minimize and surrounded on camera coordinates system for video camera installation
XC、YC、ZCDeflection angle d α, d β, the d γ object functions of axle deflection obtain the optimal value of each deflection angle, then re-calibrate and bow
Depending on changing image, obtain and more accurately overlook Transformation Graphs；
S2.4 establishes image vertical view conversion look-up tables'implementation and quickly overlooks conversion, obtains birds-eye view；
Step 3) determines the gap of image mosaic, carry out image mosaic between vehicle direct picture and lateral view picture and
Panorama Mosaic, Panorama Mosaic mapping table is then generated, it is quick by way of searching Panorama Mosaic mapping table
Ground obtains panoramic looking-around birds-eye view；
S3.1 carries out the splicing between direct picture and lateral view picture, obtains panorama birds-eye view；
S3.2 establishes panoramic mosaic mapping table, completes Panorama Mosaic；
Step 4) solves the difference in brightness between stitching image two-by-two using improved brightness reconciliation process algorithm, then uses
The image interfusion method of weighted mean method further eliminates splicing seams.
As the further improvement of technical solution of the present invention, in the step 1),
The septal line training pattern includes some horizontal linears, some horizontal linears from bottom to top width with 1.3
It is incremented by again, adjacent level linear interval width is incremented by with 1.3 times from bottom to top, certain horizontal linear in some horizontal linears
It is upper to be provided with several cross spiders；
It is described to carry out distortion correction of the fish eye images correction using SVM algorithm progress fish eye images using SVM algorithm
When corresponding SVM training aids input and output be respectively in the radial distance and corresponding fish eye images of picture point in physical space
The radial distance of picture point, then input to SVM training aids and output data is classified and fit non-linear function, utilization are multigroup
Sample is repeatedly trained, and regression fit goes out transformation model, so as to construct fish eye images and distortion fish eye images after correction
The mapping relations of respective pixel coordinate.
As the further improvement of technical solution of the present invention, in the step S2.1, it is assumed that certain point P in world coordinate systemW
Coordinate be (xW,yW,zW), the point is expressed as P in camera coordinate systemC(xC,yC,zC), relation such as formula between the two
(1) shown in：
Wherein, α is the angle that camera coordinate system surrounds X-axis rotation relative to world coordinate system；H=| OCOW| it is video camera
Photocentre and the vertical range on ground；R is 3 × 3 spin matrix；T is translation matrix T=[0 0-h]T；
In the step S2.2, it is assumed that the certain point pixel overlooked on the target image after conversion is p, on its original image
Corresponding pixel points are p', and the key step for solving vertical view conversion is as follows：
(1) corresponding points P of the pixel p under world coordinate system is calculatedW：Assuming that its coordinate is (xW,yW,zW), video camera photocentre
Through picture centre, if p is u row, the pixel of v rows on target image, pixel p (u, v) is designated as, then is had：
In formula, wp、hpFor the width of target image, height, in units of pixel；Dx, dy are the horizontal, vertical of target image
The physics size in direction, l are world coordinate system origin OWTo the distance of camera optical axis and the intersection point on ground.If camera optical axis
It is θ with horizontal plane angle, the vertical range on video camera photocentre and ground is h, then l=hcot θ；
(2) point P is calculatedWCoordinate P in camera coordinate systemC(xC,yC,zC), its method such as formula (1) institute solved
Show；
(3) P is calculatedC(xC,yC,zC) subpoint P on the image planei(xi,yi), calculation formula is as follows：
F is focal length of camera in formula.
(4) subpoint P is calculatedi(xi,yi) in original image (original perspective view) to pixel p'(u', v'), meter
Calculating formula is：
In formula, wp′、hp' the width for input picture, height, in units of pixel；Dx', dy' are input picture
Horizontal, the physics size of longitudinal direction.
As the further improvement of technical solution of the present invention, in the step S2.3, the correction comprises provide that video camera
Installation is present under error condition, and camera coordinates system surrounds XC、YC、ZCAxle deflects, and deflection angle is respectively d α, d β, d
γ, then as long as the calculating point P in step S2.2WCoordinate P in camera coordinate systemC(xC,yC,zC) solution formula become
For：
PC=R3R2R1(PW+T) (5)
Wherein,
By establish on d α, d β, d γ object function, functional value is made most using linear change method to three deflection angles
The method of smallization solves the deflection angle d α of camera, d β, d γ optimal value, reuses step S2.2 vertical view conversion side
Method carries out vertical view conversion, and the solution formula (1) of second step in step S2.2 is replaced with into formula in transfer process
As the further improvement of technical solution of the present invention, in the step S2.4, the conversion look-up table of overlooking includes
The coordinate relation between all pixels point pixel corresponding with original perspective view in the top view of conversion, system can pass through
The mode that conversion look-up table is overlooked in inquiry realizes conversion of the perspective view to top view, quick by way of overlooking and converting look-up table
Realize the vertical view conversion of fluoroscopy images.
As the further improvement of technical solution of the present invention, the splicing between the direct picture and lateral view picture specifically walks
It is rapid as follows：
(1) direct picture includes forward image and rear images, and the lateral view picture includes lateral view picture and right side
Image, the length of vehicle body is C, width K, a width of H of the lateral image of vehicle body1Rice, a width of H rice of direct picture, in direct picture and
Two point P are demarcated in the public domain of lateral view picture covering1And P2, 2 points of coordinates under panorama earth axes are designated as P respectively1
(X1,Y1) and P2(X2,Y2)；
(2) obtained in the look down on the rendering of lateral view picture and point P1And P2Corresponding pixel point coordinates R '1And R'2；
(3) obtained in the look down on the rendering of direct picture and point P1And P2Corresponding pixel point coordinates R "1With R "2；
(4) according to point P1And P2It is conllinear in the image coordinate system of lateral view picture and direct picture, therefore with R '1And R'2Really
Fixed straight line R '1R'2With R "1With R "2The straight line R " of determination1R″2Respectively as direct picture and the splicing seams of lateral view picture；
(5) splicing seams position is preserved, two images to be spliced are cut further according to splicing seams according to splicing seams position
The vertical view of the top view of direct picture and lateral view picture is stitched together, it is described to be cut to direct picture and lateral view picture root
Part unnecessary outside splicing seams is cropped according to splicing seams.
As the further improvement of technical solution of the present invention, the Panorama Mosaic includes：
(1) set panoramic looking-around birds-eye view field range, set output panoramic looking-around birds-eye view width as
Width, it is highly Height.Because field range in x and y direction is proportional, the visual range in Y-direction is set
For ViewRangeY,Then the visual range in X-direction is ViewRangeX=scale
Height；
(2) panoramic mosaic mapping table is generated, according to the field range of the panoramic looking-around birds-eye view set, splicing will be included
Stitch position, the width Width of panoramic looking-around birds-eye view, height Height, visual range ViewRangeY and X in Y-direction
The parameter of visual range ViewRangeX on direction preserves the form into table, establishes Panorama Mosaic mapping table；
(3) according to the panoramic mosaic mapping table established, Panorama Mosaic is completed by the method tabled look-up.
As the further improvement of technical solution of the present invention, the step 4) specifically includes following steps：
S4.1 eliminates the difference in brightness after splicing between image using improved brightness reconciliation process algorithm, and it includes：
(1) using 1/3 common portion of two images as overlapping region；
(2) the pixel value and S of the overlapping region is calculated respectively1With S2；
(3) Differ=S is set1/S2, the pixel value of each pixel in a wherein sub-picture is multiplied with Differ
Weighting, obtains new pixel value R, if R ＞ T, original pixel value keeps constant；If R ＜ T, original pixel value assignment again
Experiment experience value 200 is taken for T, T；
S4.2 eliminates the splicing seams of splicing using average weighted image interfusion method, makes image smoothing transition.
Compared with prior art, the invention has the advantages that：
1st, flake distortion correction has been carried out by building interval training pattern and SVM algorithm, interval training pattern has utilization
Solve the problems, such as that the marginal information of the image after distortion correction is fuzzy.
2nd, using the relation between world coordinate system, camera coordinate system and image coordinate system, with reference to backward mapping method
Carry out conversion of the perspective view to top view；Error be present for video camera installation, minimize on deflection angle d α, d β, d
γ object functions obtain each deflection angle optimal value, and are realized quickly by way of establishing image and overlooking conversion look-up table
Overlook conversion.
3rd, the panoramic mosaic mapping table that panoramic looking-around birds-eye view is established is generated, panoramic picture is completed by the method tabled look-up and spelled
Connect, effectively reduce system operation time.
4th, the difference in brightness problem between stitching image two-by-two is efficiently solved using improved brightness reconciliation process algorithm, and
Splicing seams are further eliminated using the image interfusion method of weighted mean method so that panoramic looking-around birds-eye view has more preferable vision
Effect.
Brief description of the drawings
Fig. 1 is algorithm total algorithm flow chart described in embodiment；
Fig. 2 is septal line training pattern figure described in embodiment；
Fig. 3 is septal line training pattern fish eye images described in embodiment；
Fig. 4 is the camera coordinate system and world coordinate system figure established in embodiment described image splicing；
Fig. 5 is panorama earth axes schematic diagram described in embodiment；
Fig. 6 is embodiment described image splicing flow chart；
Fig. 7 is Panorama Mosaic flow chart described in embodiment.
Embodiment
The present invention is described in further details in conjunction with accompanying drawing, the present embodiment proposes the figure in a kind of panoramic looking-around system
As joining method, the image split-joint method flow is as shown in figure 1, specifically include following steps：
Step S1：Fish eye images distortion correction is carried out with reference to septal line training pattern and SVM algorithm；
Fish eye lens can obtain large range of visual angle and complete spherical diagram picture, but due to fish-eye complexity
Optical texture causes fish eye images to produce serious distortion, therefore the distortion school of image is first carried out before image mosaic is carried out
Just, herein below is specifically included：
S1.1 builds septal line training pattern；
Under normal circumstances, the marginal information of the image after distortion correction obscures, and the embodiment of the present invention builds a kind of septal line
Training pattern overcomes such a problem.As shown in Fig. 2 Fig. 2 cathetus width is incremented by with 1.3 times, while the interval between straight line
It is incremented by with 1.3 times of spacing, in order to ensure training pattern can still extract clearly intersection point at fish eye images edge, is being spaced
Several cross spiders are provided with certain horizontal linear in line training pattern.Fig. 3 is the fish eye images of septal line training pattern.
S1.2 carries out fish eye images correction using SVM algorithm；
Using SVM algorithm carry out fish eye images distortion correction when corresponding SVM training aids input and output be respectively
The radial distance of picture point in the radial distance of picture point in physical space and corresponding fish eye images, then the input to SVM training aids
Classification and fit non-linear function are carried out with output data, is repeatedly trained using multigroup sample, regression fit goes out modulus of conversion
Type, so as to construct the mapping relations of fish eye images and distortion fish eye images respective pixel coordinate after correction.
Step S2：Using the relation between world coordinate system, camera coordinate system and image coordinate system, with reference to backward mapping
Method carries out conversion of the perspective view to top view.Error be present for video camera installation, according to camera coordinates system
Around XC、YC、ZCThe deflection angle of axle re-calibrates vertical view changing image, then establishes image and overlooks conversion look-up table to realize
Conversion is quickly overlooked, birds-eye view is obtained, specifically includes herein below：
S2.1 establishes world coordinate system and camera coordinate system；
The vehicle-mounted viewing system target of panorama is the plane birds-eye view for enabling a driver to observe vehicle's surroundings, that is, requires to regard
It is wild perpendicular to ground into depression angle.Therefore need to carry out vertical view conversion to fluoroscopy images, eliminate the transparent effect of image, thoroughly
Visible image is converted into getting a bird's eye view top view.Carry out perspective conversion and obtain birds-eye view, it is necessary first to establish world coordinate system and video camera
Coordinate system.By taking front side camera as an example, as shown in figure 4, camera be arranged on Chinese herbaceous peony centre position, camera coordinate system with
Video camera photocentre is origin OC, ZCAxle is camera optical axis, XCAxle perpendicular to automobile side plane (being that vertical paper is outside in Fig. 4),
YCWith plane XCOCZCVertically., OWFor the origin of world coordinate system, world coordinate system ZWAxle is by video camera photocentre perpendicular to ground
Upwards, YWAxle points to automobile direction of advance, X on ground levelWAxle and camera coordinate system XCDirection of principal axis is identical.
World coordinate system is first along ZWAxle translates | OCOW| length (i.e. the vertical range on video camera photocentre and ground), then surround
XWAxle rotates to an angle, that is, obtains camera coordinate system.Certain point P in hypothetical world coordinate systemWCoordinate be (xW,yW,zW), should
Point is expressed as P in camera coordinate systemC(xC,yC,zC), shown in relation such as formula (1) between the two.
Wherein, α is the angle that camera coordinate system surrounds X-axis rotation relative to world coordinate system；H=| OCOW| it is video camera
Photocentre and the vertical range on ground；R is 3 × 3 spin matrix；T is translation matrix T=[0 0-h]T。
S2.2 solves image using backward mapping method and overlooks conversion；
If video camera installation is accurate, that is, ensure camera coordinate system XCAxle is parallel to ground, in camera coordinate system
YCOCZCPlane and Y in world coordinate systemWOWZWPlane overlaps, camera optical axis (ZCAxle) determined with horizontal plane angle.For this
Kind situation, the vertical view that image is carried out using backward mapping method is converted, i.e., to overlooking each on the target image after converting
Pixel, calculate the respective pixel on original fluoroscopy images.
Assuming that the certain point pixel on the target image overlooked after conversion is p, corresponding pixel points are p ' on its original image,
The key step for solving vertical view conversion is as follows：
(1) corresponding points P of the pixel p under world coordinate system is calculatedW：Assuming that its coordinate is (xW,yW,zW), video camera photocentre
Through picture centre, if p is u row, the pixel of v rows on target image, pixel p (u, v) is designated as, then is had：
In formula, wp、hpFor the width of target image, height, in units of pixel；Dx, dy are the horizontal, vertical of target image
The physics size in direction, l are world coordinate system origin OWTo the distance of camera optical axis and the intersection point on ground.If camera optical axis
It is θ with horizontal plane angle, the vertical range on video camera photocentre and ground is h, then l=hcot θ.
(2) point P is calculatedWCoordinate P in camera coordinate systemC(xC,yC,zC), its method such as formula (1) institute solved
Show.
(3) P is calculatedC(xC,yC,zC) subpoint P on the image planei(xi,yi), calculation formula is as follows：
F is focal length of camera in formula.
(4) subpoint P is calculatedi(xi,yi) in original image (original perspective view) to pixel p'(u', v'), meter
Calculating formula is：
In formula, wp'、hp' for the width of input picture, height, in units of pixel；Dx', dy' are input picture
Horizontal, the physics size of longitudinal direction.
There is error in S2.3, construct and minimize on deflection angle d α, d β, d γ targets for video camera installation
Function obtains the optimal value of each deflection angle, then re-calibrates vertical view changing image, obtains and more accurate overlooks conversion
Figure；
When the installation of video camera has error, if carrying out vertical view conversion according to step in S2.2, it is accurate to obtain
Top view, in order to ensure top view accuracy, it is necessary to S2.2 narration method on the basis of top view is corrected.Institute
State correction and comprise provide that video camera installation is present under error condition, camera coordinates system surrounds XC、YC、ZCAxle deflects, partially
Gyration is respectively d α, d β, d γ, then as long as the calculating point P in step S2.2WCoordinate P in camera coordinate systemC
(xC,yC,zC) solution formula be changed into：
PC=R3R2R1(PW+T) (5)
Wherein,
In summary, the Correction Problemss for video camera alignment error also translate into the deflection angle d for solving camera
The problem of α, d β, d γ.The embodiment of the present invention by establish on d α, d β, d γ object function, line is used to three deflection angles
Property the method for changing method that minimizes functional value solve the deflection angle d α of camera, d β, d γ.
The embodiment of the present invention determines object function using target rectangle：Error be present for video camera installation,
What is obtained using the method in S2.2 is non-accurate top view, and selection one is parallel to the visual field in the non-accurately top view
Rectangle four angle points, four angle points are designated as p respectively1(u1,v1)、p2(u2,v2)、p3(u3,v3)、p4(u4,v4), according to S2.2
In method calculate this 4 points corresponding points p ' in original perspective view1(u′1,v′1)、p'2(u'2,v'2)、p'3(u'3,v'3)、
p'4(u'4,v'4), deflection angle d α, d β, d γ influence are then added, is calculated using the Converse solved method of step S2.2 methods
Go out p '1(u′1,v′1)、p'2(u'2,v'2)、p'3(u'3,v'3)、p'4(u'4,v'4) corresponding picture in new switch target image
The position of element, above-mentioned Converse solved procedure concretely comprise the following steps：
(1) the reverse-power formula of formula (4) is utilized, tries to achieve p '1(u′1,v′1)、p'2(u'2,v'2)、p'3(u'3,v'3)、p'4
(u'4,v'4) corresponding coordinate p is distinguished in image coordinate system1(x1,y1)、p2(x2,y2)、p3(x3,y3)、p4(x4,y4), calculate
Formula is as follows：
W' in formulap、h'pThe respectively width of input picture, height, all in units of pixel；Dx', dy' are respectively to input
The horizontal stroke of image, the physics size of longitudinal direction.
(2) imaging point can be designated as in the coordinate of camera coordinate systemWherein f is Jiao of video camera
Away from.
(3) the reverse-power formula of formula (5) is utilized, imaging point coordinate in world coordinate system is：
Now add deflection angle d α, d β, d γ.
(4) intersection point of imaging point, the line of video camera photocentre and ground is exactly the coordinate of top view continuous image, then can ask
Obtain topocentric coordinates Pg(xg,yg, 0) be：
(5) P is calculated by the reverse-power formula of formula (2)gThe corresponding points in new target image (overhead view image after correction)
P " (u ", v ") position：
Respectively to point p '1(u′1,v′1)、p'2(u'2,v'2)、p'3(u'3,v'3)、p'4(u'4,v'4) carry out above-mentioned (1) extremely
(5) step, P is tried to achieve1”(u1”,v1”)、P2”(u2”,v2”)、P3”(u3”,v3") and P4″(u4″,v4"), then build below in relation to
Deflection angle d α, d β, d γ object function：
F (d α, d β, d γ)=| u " 1-u "3|+|u″2-u″4|+|v″1-v″2|+|v″3-v″4|
(13)
D α, d β, d γ are scanned for using linear change method, obtain F minimum.Generally error angle will not
It is too big, therefore make d α, d β, d γ in a small range change centered on 0.Solve deflection angle d α, d β, d γ most
The figure of merit and then vertical view conversion is carried out using step S2.2 vertical view transform method, by second in step S2.2 in transfer process
The solution formula (1) of step replaces with formula (10).
S2.4 establishes image vertical view conversion look-up tables'implementation and quickly overlooks conversion, obtains birds-eye view.
Described vertical view conversion algorithm flow and the corresponding parameter of video camera are established and overlook conversion according to embodiments of the present invention
Look-up table, overlook conversion look-up table include conversion top view in all pixels point pixel corresponding with original perspective view it
Between coordinate relation, system can inquire about overlook conversion look-up table by way of realize conversion of the perspective view to top view.It is logical
The mode for overlooking conversion look-up table is crossed, can quickly realize the vertical view conversion of fluoroscopy images.
Step S3：Determine the gap of image mosaic, carry out image mosaic between vehicle direct picture and side image with
And Panorama Mosaic, Panorama Mosaic mapping table is then generated, by inquiring about Panorama Mosaic mapping table so as to quick
Ground obtains panorama birds-eye view；Specifically include herein below：
Splicing between S3.1 direct pictures and side image；
Four video cameras are installed on vehicle body, obtain front and rear four parts of images all around, the splicing master of panoramic picture respectively
If positive and lateral view picture splicing, the step of positive and lateral image mosaic is illustrated by taking forward image and left-side images as an example
Suddenly：
(1) as shown in figure 5, the length of vehicle body is C, width K, a width of H of vehicle body side image1Rice, a width of H of forward image
Rice, two point P are demarcated in the public domain that direct picture and left-side images cover1And P2, 2 points under panorama earth axes
Coordinate is designated as P respectively1(X1,Y1) and P2(X2,Y2)；
(2) obtained in the look down on the rendering of left-side images and point P1And P2Corresponding pixel point coordinates R '1And R'2；
(3) obtained in the look down on the rendering of direct picture and point P1And P2Corresponding pixel point coordinates R "1With R "2；
(4) according to point P1And P2It is conllinear in the image coordinate system of left-side images and forward image, therefore with R '1And R'2Really
Fixed straight line R '1R'2With R "1With R "2The straight line R " of determination1R″2Respectively as forward image and the splicing seams of left-side images；
(5) splicing seams position is preserved, two images to be spliced are cut further according to splicing seams according to splicing seams position
The vertical view of the top view of forward image and left-side images is stitched together, it is described to be cut to forward image and left-side images root
Part unnecessary outside splicing seams is cropped according to splicing seams.
The splicing ginseng of rear images and left-side images, forward image and image right and rear images and image right
The splicing step of the forward image and left-side images is examined, image mosaic idiographic flow is as shown in Figure 6.
S3.2 Panorama Mosaics；
According to joining method positive in S3.1 and lateral view picture by the overhead view image of four width images around vehicle body two-by-two
It is stitched together, you can obtain panorama birds-eye view.
The step of establishing panoramic mosaic mapping table be：
(1) field range of panoramic looking-around birds-eye view is set：
The width of the panoramic looking-around birds-eye view of output is set as Width, is highly Height.Due in x and y direction
Field range is proportional, sets the visual range in Y-direction as ViewRangeY,Then X side
Upward visual range is ViewRangeX=scaleHeight.
(2) panoramic mosaic mapping table is generated：
According to the field range of the panoramic looking-around birds-eye view set, according to method in S3.1 by four width figures around vehicle body
The top view of picture is spliced two-by-two, obtains panoramic picture.But calculated because image mosaic is related to substantial amounts of coordinate transform etc.,
This will necessarily reduce system time performance.It is solid in view of mutual alignment between camera installation site, angle and camera
Fixed, and these factors will not change with the change of the content of camera collection.Image mosaic process is based on shooting
The spatial alternation process of pixel in the original fluoroscopy images that head collects, therefore can will include splicing seams position, panoramic looking-around
Width Width, height Height, the visual range ViewRangeY in Y-direction and the visual range in X-direction of birds-eye view
The parameters such as ViewRangeX preserve the form into table, establish Panorama Mosaic mapping table.According in S3.2 Panorama Mosaics
The parameter of setting in step (1), panoramic picture size is HeightWidth, i.e., is shared in Panorama Mosaic mapping table
Height rows and Width row, it is (n, i, j) to define the data in each cell, and it is represented corresponding to panoramic image pixel point
The coordinate of pixel in image to be spliced.Wherein n=1,2,3,4, the width image of front, rear, left and right four, i and j difference are represented successively
For the coordinate of pixel in the width image.
(3) according to the panoramic mosaic mapping table established, Panorama Mosaic is completed by the method tabled look-up, so as to effectively
Reduce system operation time.
Panorama Mosaic flow is as shown in Figure 7.
Step S4：Solve the difference in brightness between stitching image two-by-two using improved brightness reconciliation process algorithm, then adopt
Splicing seams are further eliminated with the image interfusion method of weighted mean method, it specifically includes following steps：
(1) luminance difference between stitching image is eliminated；
The difference for the angle installed installed in four video cameras of vehicle body surrounding causes impression of the image of shooting to light source
Degree is different, therefore luminance difference be present between the width image of front, rear, left and right four.The embodiment of the present invention is using improved bright
Degree reconciliation process algorithm eliminates the difference in brightness after splicing between image, and it is concretely comprised the following steps：
1) using 1/3 common portion of two images as overlapping region；
2) the pixel value and S of the overlapping region is calculated respectively1With S2；
3) Differ=S is set1/S2, the pixel value of each pixel in a wherein sub-picture and Differ phases is multiply-add
Power, obtains new pixel value R.If R ＞ T, original pixel value keeps constant；If R ＜ T, original pixel value are entered as again
T, T take experiment experience value 200.
(2) splicing seams of splicing are eliminated using average weighted image interfusion method, make image smoothing transition.So as to
The luminance difference and splicing seams between stitching image are effectively eliminated, realizes the seamless spliced of panoramic picture so that panorama
Panoramic view picture has more preferable visual effect.
The method proposed in the present invention can actually be embedded in FPGA realizations, applied in vehicle-mounted panoramic viewing system.More than
Embodiment only plays a part of explaining technical solution of the present invention, and protection domain of the presently claimed invention is not limited to above-mentioned implementation
System and specific implementation step are realized described in example.Therefore, only specific formula in above-described embodiment and algorithm are carried out simple
Replace, but the technical scheme that its substantive content is still consistent with the method for the invention, protection scope of the present invention all should be belonged to.
Claims (8)
1. the image split-joint method in a kind of panoramic looking-around system, it is characterised in that comprise the following steps：
Step 1) combines septal line training pattern and SVM algorithm carries out fish eye images distortion correction, and it includes：
S1.1 builds septal line training pattern；
S1.2 carries out fish eye images correction using SVM algorithm；
Step 2) is using the relation between world coordinate system, camera coordinate system and image coordinate system, with reference to backward mapping method
Conversion of the perspective view to top view is carried out, error be present for video camera installation, surrounded according to camera coordinates system
XC、YC、ZCThe deflection angle of axle re-calibrates vertical view changing image, then establishes image and overlooks conversion look-up table to realize quickly
Vertical view conversion, obtain birds-eye view, it includes：
S2.1 establishes world coordinate system and camera coordinate system；
S2.2 is solved using backward mapping method and is overlooked conversion；
There is error in S2.3, construct and minimize and surround X on camera coordinates system for video camera installationC、YC、
ZCDeflection angle d α, d β, the d γ object functions of axle deflection obtain the optimal value of each deflection angle, then re-calibrate vertical view and become
Image is changed, obtains and more accurately overlooks Transformation Graphs；
S2.4 establishes image vertical view conversion look-up tables'implementation and quickly overlooks conversion, obtains birds-eye view；
Step 3) determines the gap of image mosaic, carries out image mosaic and panorama between vehicle direct picture and lateral view picture
Image mosaic, Panorama Mosaic mapping table is then generated, is rapidly obtained by way of searching Panorama Mosaic mapping table
Obtain panoramic looking-around birds-eye view；
S3.1 carries out the splicing between direct picture and lateral view picture, obtains panorama birds-eye view；
S3.2 establishes panoramic mosaic mapping table, completes Panorama Mosaic；
Step 4) solves the difference in brightness between stitching image two-by-two using improved brightness reconciliation process algorithm, then using weighting
The image interfusion method of the method for average further eliminates splicing seams.
2. the image split-joint method in a kind of panoramic looking-around system according to claim 1, it is characterised in that the step
1) in,
The septal line training pattern includes some horizontal linears, and some horizontal linears from bottom to top with 1.3 times passed by width
Increase, adjacent level linear interval width is incremented by with 1.3 times from bottom to top, is set in some horizontal linears on certain horizontal linear
Several cross spiders are put；
It is described using SVM algorithm carry out fish eye images correction using SVM algorithm carry out fish eye images distortion correction when pair
The input and output of the SVM training aids answered are respectively the radial distance of the picture point in physical space and correspond to picture point in fish eye images
Radial distance, then input to SVM training aids and output data is classified and fit non-linear function utilizes multigroup sample
Repeatedly trained, regression fit goes out transformation model, corresponding with distortion fish eye images so as to construct fish eye images after correction
The mapping relations of pixel coordinate.
3. the image split-joint method in a kind of panoramic looking-around system according to claim 1, it is characterised in that the step
In S2.1, it is assumed that certain point P in world coordinate systemWCoordinate be (xW,yW,zW), the point is expressed as P in camera coordinate systemC
(xC,yC,zC), shown in relation such as formula (1) between the two：
<mrow>
<msub>
<mi>P</mi>
<mi>C</mi>
</msub>
<mo>=</mo>
<mfenced open = "[" close = "]">
<mtable>
<mtr>
<mtd>
<msub>
<mi>x</mi>
<mi>C</mi>
</msub>
</mtd>
</mtr>
<mtr>
<mtd>
<msub>
<mi>y</mi>
<mi>C</mi>
</msub>
</mtd>
</mtr>
<mtr>
<mtd>
<msub>
<mi>z</mi>
<mi>C</mi>
</msub>
</mtd>
</mtr>
</mtable>
</mfenced>
<mo>=</mo>
<mfenced open = "[" close = "]">
<mtable>
<mtr>
<mtd>
<mn>1</mn>
</mtd>
<mtd>
<mn>0</mn>
</mtd>
<mtd>
<mn>0</mn>
</mtd>
</mtr>
<mtr>
<mtd>
<mn>0</mn>
</mtd>
<mtd>
<mrow>
<mi>cos</mi>
<mi>&alpha;</mi>
</mrow>
</mtd>
<mtd>
<mrow>
<mi>sin</mi>
<mi>&alpha;</mi>
</mrow>
</mtd>
</mtr>
<mtr>
<mtd>
<mn>0</mn>
</mtd>
<mtd>
<mrow>
<mo>-</mo>
<mi>sin</mi>
<mi>&alpha;</mi>
</mrow>
</mtd>
<mtd>
<mrow>
<mi>cos</mi>
<mi>&alpha;</mi>
</mrow>
</mtd>
</mtr>
</mtable>
</mfenced>
<mfenced open = "[" close = "]">
<mtable>
<mtr>
<mtd>
<msub>
<mi>x</mi>
<mi>W</mi>
</msub>
</mtd>
</mtr>
<mtr>
<mtd>
<msub>
<mi>y</mi>
<mi>W</mi>
</msub>
</mtd>
</mtr>
<mtr>
<mtd>
<mrow>
<msub>
<mi>z</mi>
<mi>W</mi>
</msub>
<mo>-</mo>
<mi>h</mi>
</mrow>
</mtd>
</mtr>
</mtable>
</mfenced>
<mo>=</mo>
<mi>R</mi>
<mrow>
<mo>(</mo>
<msub>
<mi>P</mi>
<mi>W</mi>
</msub>
<mo>+</mo>
<mi>T</mi>
<mo>)</mo>
</mrow>
<mo>-</mo>
<mo>-</mo>
<mo>-</mo>
<mrow>
<mo>(</mo>
<mn>1</mn>
<mo>)</mo>
</mrow>
</mrow>
Wherein, α is the angle that camera coordinate system surrounds X-axis rotation relative to world coordinate system；H=| OCOW| it is video camera photocentre
With the vertical range on ground；R is 3 × 3 spin matrix；T is translation matrix T=[0 0-h]T；
In the step S2.2, it is assumed that the certain point pixel on target image overlooked after conversion is p, is corresponded on its original image
Pixel is p', and the key step for solving vertical view conversion is as follows：
(1) corresponding points P of the pixel p under world coordinate system is calculatedW：Assuming that its coordinate is (xW,yW,zW), video camera photocentre passes through
Picture centre, if p is u row, the pixel of v rows on target image, pixel p (u, v) is designated as, then is had：
<mrow>
<mfenced open = "{" close = "">
<mtable>
<mtr>
<mtd>
<mrow>
<msub>
<mi>x</mi>
<mi>W</mi>
</msub>
<mo>=</mo>
<mrow>
<mo>(</mo>
<mi>u</mi>
<mo>-</mo>
<mfrac>
<msub>
<mi>w</mi>
<mi>p</mi>
</msub>
<mn>2</mn>
</mfrac>
<mo>)</mo>
</mrow>
<mi>d</mi>
<mi>x</mi>
</mrow>
</mtd>
</mtr>
<mtr>
<mtd>
<mrow>
<msub>
<mi>y</mi>
<mi>W</mi>
</msub>
<mo>=</mo>
<mo>-</mo>
<mrow>
<mo>(</mo>
<mi>v</mi>
<mo>-</mo>
<mfrac>
<msub>
<mi>h</mi>
<mi>p</mi>
</msub>
<mn>2</mn>
</mfrac>
<mo>)</mo>
</mrow>
<mi>d</mi>
<mi>y</mi>
<mo>+</mo>
<mn>1</mn>
</mrow>
</mtd>
</mtr>
<mtr>
<mtd>
<mrow>
<msub>
<mi>z</mi>
<mi>W</mi>
</msub>
<mo>=</mo>
<mn>0</mn>
</mrow>
</mtd>
</mtr>
</mtable>
</mfenced>
<mo>-</mo>
<mo>-</mo>
<mo>-</mo>
<mrow>
<mo>(</mo>
<mn>2</mn>
<mo>)</mo>
</mrow>
</mrow>
In formula, wp、hpFor the width of target image, height, in units of pixel；Dx, dy are horizontal stroke, the longitudinal direction of target image
Physics size, l is world coordinate system origin OWTo the distance of camera optical axis and the intersection point on ground.If camera optical axis and water
Plane included angle is θ, and the vertical range on video camera photocentre and ground is h, then l=hcot θ；
(2) point P is calculatedWCoordinate P in camera coordinate systemC(xC,yC,zC), shown in its method such as formula (1) solved；
(3) P is calculatedC(xC,yC,zC) subpoint P on the image planei(xi,yi), calculation formula is as follows：
<mrow>
<mfenced open = "{" close = "">
<mtable>
<mtr>
<mtd>
<mrow>
<msub>
<mi>x</mi>
<mi>i</mi>
</msub>
<mo>=</mo>
<mo>-</mo>
<mi>f</mi>
<mfrac>
<msub>
<mi>x</mi>
<mi>C</mi>
</msub>
<msub>
<mi>z</mi>
<mi>C</mi>
</msub>
</mfrac>
</mrow>
</mtd>
</mtr>
<mtr>
<mtd>
<mrow>
<msub>
<mi>y</mi>
<mi>i</mi>
</msub>
<mo>=</mo>
<mo>-</mo>
<mi>f</mi>
<mfrac>
<msub>
<mi>y</mi>
<mi>C</mi>
</msub>
<msub>
<mi>z</mi>
<mi>C</mi>
</msub>
</mfrac>
</mrow>
</mtd>
</mtr>
</mtable>
</mfenced>
<mo>-</mo>
<mo>-</mo>
<mo>-</mo>
<mrow>
<mo>(</mo>
<mn>3</mn>
<mo>)</mo>
</mrow>
</mrow>
F is focal length of camera in formula.
(4) subpoint P is calculatedi(xi,yi) in original image (original perspective view) to pixel p'(u', v'), calculate public
Formula is：
<mrow>
<mfenced open = "{" close = "">
<mtable>
<mtr>
<mtd>
<mrow>
<msup>
<mi>u</mi>
<mo>&prime;</mo>
</msup>
<mo>=</mo>
<mfrac>
<msub>
<mi>x</mi>
<mi>i</mi>
</msub>
<mrow>
<msup>
<mi>dx</mi>
<mo>&prime;</mo>
</msup>
</mrow>
</mfrac>
<mo>+</mo>
<mfrac>
<mrow>
<msup>
<msub>
<mi>w</mi>
<mi>p</mi>
</msub>
<mo>&prime;</mo>
</msup>
</mrow>
<mn>2</mn>
</mfrac>
</mrow>
</mtd>
</mtr>
<mtr>
<mtd>
<mrow>
<msup>
<mi>v</mi>
<mo>&prime;</mo>
</msup>
<mo>=</mo>
<mo>-</mo>
<mfrac>
<msub>
<mi>y</mi>
<mi>i</mi>
</msub>
<mrow>
<msup>
<mi>dy</mi>
<mo>&prime;</mo>
</msup>
</mrow>
</mfrac>
<mo>+</mo>
<mfrac>
<mrow>
<msup>
<msub>
<mi>h</mi>
<mi>p</mi>
</msub>
<mo>&prime;</mo>
</msup>
</mrow>
<mn>2</mn>
</mfrac>
</mrow>
</mtd>
</mtr>
</mtable>
</mfenced>
<mo>-</mo>
<mo>-</mo>
<mo>-</mo>
<mrow>
<mo>(</mo>
<mn>4</mn>
<mo>)</mo>
</mrow>
</mrow>
In formula, wp'、hp' for the width of input picture, height, in units of pixel；Dx', dy' are the horizontal, vertical of input picture
The physics size in direction.
4. the image split-joint method in a kind of panoramic looking-around system according to claim 3, it is characterised in that the step
In S2.3, the correction comprises provide that video camera installation is present under error condition, and camera coordinates system surrounds XC、YC、ZCAxle is sent out
Raw deflection, deflection angle is respectively d α, d β, d γ, then as long as the calculating point P in step S2.2WIn camera coordinate system
Coordinate PC(xC,yC,zC) solution formula be changed into：
PC=R3R2R1(PW+T) (5)
Wherein,
<mrow>
<msub>
<mi>R</mi>
<mn>1</mn>
</msub>
<mo>=</mo>
<mfenced open = "[" close = "]">
<mtable>
<mtr>
<mtd>
<mn>1</mn>
</mtd>
<mtd>
<mn>0</mn>
</mtd>
<mtd>
<mn>0</mn>
</mtd>
</mtr>
<mtr>
<mtd>
<mn>0</mn>
</mtd>
<mtd>
<mrow>
<mi>c</mi>
<mi>o</mi>
<mi>s</mi>
<mrow>
<mo>(</mo>
<mi>&alpha;</mi>
<mo>+</mo>
<mi>d</mi>
<mi>&alpha;</mi>
<mo>)</mo>
</mrow>
</mrow>
</mtd>
<mtd>
<mrow>
<mi>sin</mi>
<mrow>
<mo>(</mo>
<mi>&alpha;</mi>
<mo>+</mo>
<mi>d</mi>
<mi>&alpha;</mi>
<mo>)</mo>
</mrow>
</mrow>
</mtd>
</mtr>
<mtr>
<mtd>
<mn>0</mn>
</mtd>
<mtd>
<mrow>
<mo>-</mo>
<mi>s</mi>
<mi>i</mi>
<mi>n</mi>
<mrow>
<mo>(</mo>
<mi>&alpha;</mi>
<mo>+</mo>
<mi>d</mi>
<mi>&alpha;</mi>
<mo>)</mo>
</mrow>
</mrow>
</mtd>
<mtd>
<mrow>
<mi>cos</mi>
<mrow>
<mo>(</mo>
<mi>&alpha;</mi>
<mo>+</mo>
<mi>d</mi>
<mi>&alpha;</mi>
<mo>)</mo>
</mrow>
</mrow>
</mtd>
</mtr>
</mtable>
</mfenced>
<mo>-</mo>
<mo>-</mo>
<mo>-</mo>
<mrow>
<mo>(</mo>
<mn>6</mn>
<mo>)</mo>
</mrow>
</mrow>
2
<mrow>
<msub>
<mi>R</mi>
<mn>2</mn>
</msub>
<mo>=</mo>
<mfenced open = "[" close = "]">
<mtable>
<mtr>
<mtd>
<mrow>
<mi>cos</mi>
<mi> </mi>
<mi>d</mi>
<mi>&beta;</mi>
</mrow>
</mtd>
<mtd>
<mn>0</mn>
</mtd>
<mtd>
<mrow>
<mo>-</mo>
<mi>sin</mi>
<mi> </mi>
<mi>d</mi>
<mi>&beta;</mi>
</mrow>
</mtd>
</mtr>
<mtr>
<mtd>
<mn>0</mn>
</mtd>
<mtd>
<mn>1</mn>
</mtd>
<mtd>
<mn>0</mn>
</mtd>
</mtr>
<mtr>
<mtd>
<mrow>
<mi>sin</mi>
<mi> </mi>
<mi>d</mi>
<mi>&beta;</mi>
</mrow>
</mtd>
<mtd>
<mn>0</mn>
</mtd>
<mtd>
<mrow>
<mi>cos</mi>
<mi> </mi>
<mi>d</mi>
<mi>&beta;</mi>
</mrow>
</mtd>
</mtr>
</mtable>
</mfenced>
<mo>-</mo>
<mo>-</mo>
<mo>-</mo>
<mrow>
<mo>(</mo>
<mn>7</mn>
<mo>)</mo>
</mrow>
</mrow>
<mrow>
<msub>
<mi>R</mi>
<mn>3</mn>
</msub>
<mo>=</mo>
<mfenced open = "[" close = "]">
<mtable>
<mtr>
<mtd>
<mrow>
<mi>cos</mi>
<mi> </mi>
<mi>d</mi>
<mi>&gamma;</mi>
</mrow>
</mtd>
<mtd>
<mrow>
<mi>sin</mi>
<mi> </mi>
<mi>d</mi>
<mi>&gamma;</mi>
</mrow>
</mtd>
<mtd>
<mn>0</mn>
</mtd>
</mtr>
<mtr>
<mtd>
<mrow>
<mo>-</mo>
<mi>sin</mi>
<mi> </mi>
<mi>d</mi>
<mi>&gamma;</mi>
</mrow>
</mtd>
<mtd>
<mrow>
<mi>cos</mi>
<mi> </mi>
<mi>d</mi>
<mi>&gamma;</mi>
</mrow>
</mtd>
<mtd>
<mn>0</mn>
</mtd>
</mtr>
<mtr>
<mtd>
<mn>0</mn>
</mtd>
<mtd>
<mn>0</mn>
</mtd>
<mtd>
<mn>1</mn>
</mtd>
</mtr>
</mtable>
</mfenced>
<mo>-</mo>
<mo>-</mo>
<mo>-</mo>
<mrow>
<mo>(</mo>
<mn>8</mn>
<mo>)</mo>
</mrow>
</mrow>
By establish on d α, d β, d γ object function, minimize functional value using linear change method three deflection angles
Method solve the deflection angle d α of camera, d β, d γ optimal value, the vertical view transform method for reusing step S2.2 enters
Row, which is overlooked, to be converted, and the solution formula (1) of second step in step S2.2 is replaced with into formula in transfer process
A kind of 5. image split-joint method in panoramic looking-around system according to claim any one of 1-4, it is characterised in that
In the step S2.4, the conversion look-up table of overlooking includes all pixels point in the top view of conversion and original perspective view pair
Coordinate relation between the pixel answered, system can realize perspective view to vertical view by way of inquiring about and overlooking conversion look-up table
The conversion of figure, quickly realize that the vertical view of fluoroscopy images converts by way of overlooking and converting look-up table.
6. the image split-joint method in a kind of panoramic looking-around system according to claim 1, it is characterised in that the forward direction
Splicing between image and lateral view picture comprises the following steps that：
(1) direct picture includes forward image and rear images, and the lateral view picture includes lateral view picture and image right,
The length of vehicle body is C, width K, a width of H of the lateral image of vehicle body1Rice, a width of H rice of direct picture, in direct picture and lateral view
Two point P are demarcated in the public domain of picture covering1And P2, 2 points of coordinates under panorama earth axes are designated as P respectively1(X1,Y1)
And P2(X2,Y2)；
(2) obtained in the look down on the rendering of lateral view picture and point P1And P2Corresponding pixel point coordinates R1' and R'2；
(3) obtained in the look down on the rendering of direct picture and point P1And P2Corresponding pixel point coordinates R1" and R'2'；
(4) according to point P1And P2It is conllinear in the image coordinate system of lateral view picture and direct picture, therefore with R1' and R'2Determine
Straight line R1'R'2With R1" and R'2' determine straight line R1”R'2' respectively as direct picture and the splicing seams of lateral view picture；
(5) splicing seams position is preserved, two images to be spliced are cut according to splicing seams position will just further according to splicing seams
The vertical view of top view and lateral view picture to image is stitched together, described to be cut to direct picture and lateral view picture according to spelling
Seam crops part unnecessary outside splicing seams.
7. the image split-joint method in a kind of panoramic looking-around system according to claim 6, it is characterised in that the panorama
Image mosaic includes：
(1) field range of panoramic looking-around birds-eye view is set, sets the width of panoramic looking-around birds-eye view of output as Width, it is high
Spend for Height.Because field range in x and y direction is proportional, set visual range in Y-direction as
ViewRangeY,Then the visual range in X-direction is ViewRangeX=scaleHeight；
(2) panoramic mosaic mapping table is generated, can by splicing seams position according to the field range of the panoramic looking-around birds-eye view set
By including the visual range in splicing seams position, the width Width of panoramic looking-around birds-eye view, height Height, Y-direction
ViewRangeY and the visual range ViewRangeX in X-direction parameter preserve the form into table, establish panoramic picture spelling
Connect mapping table；
(3) according to the panoramic mosaic mapping table established, Panorama Mosaic is completed by the method tabled look-up.
8. the image split-joint method in a kind of panoramic looking-around system according to claim 1, it is characterised in that the step
4) following steps are specifically included：
S4.1 eliminates the difference in brightness after splicing between image using improved brightness reconciliation process algorithm, and it includes：
(1) using 1/3 common portion of two images as overlapping region；
(2) the pixel value and S of the overlapping region is calculated respectively1With S2；
(3) Differ=S is set1/S2, the pixel value of each pixel in a wherein sub-picture is multiplied weighting with Differ,
New pixel value R is obtained, if R ＞ T, original pixel value keeps constant；If R ＜ T, original pixel value are entered as T, T again
Take experiment experience value 200；
S4.2 eliminates the splicing seams of splicing using average weighted image interfusion method, makes image smoothing transition.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201710237136.8A CN107424120A (en) | 2017-04-12 | 2017-04-12 | A kind of image split-joint method in panoramic looking-around system |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201710237136.8A CN107424120A (en) | 2017-04-12 | 2017-04-12 | A kind of image split-joint method in panoramic looking-around system |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN107424120A true CN107424120A (en) | 2017-12-01 |
Family
ID=60423221
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201710237136.8A Pending CN107424120A (en) | 2017-04-12 | 2017-04-12 | A kind of image split-joint method in panoramic looking-around system |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN107424120A (en) |
Cited By (21)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN107958440A (en) * | 2017-12-08 | 2018-04-24 | 合肥工业大学 | Double fish eye images real time panoramic image split-joint methods and system are realized on GPU |
| CN108198133A (en) * | 2017-12-06 | 2018-06-22 | 云南联合视觉科技有限公司 | A kind of quick joining method of vehicle panoramic image |
| CN108364333A (en) * | 2018-02-11 | 2018-08-03 | 成都康烨科技有限公司 | Method and device based on multi-direction photography fitting vertical view |
| CN108492254A (en) * | 2018-03-27 | 2018-09-04 | 西安优艾智合机器人科技有限公司 | Image capturing system and method |
| CN108638999A (en) * | 2018-05-16 | 2018-10-12 | 浙江零跑科技有限公司 | A kind of collision early warning system and method for looking around input based on 360 degree |
| CN109064397A (en) * | 2018-07-04 | 2018-12-21 | 广州希脉创新科技有限公司 | A kind of image split-joint method and system based on camera shooting earphone |
| CN110341597A (en) * | 2018-04-02 | 2019-10-18 | 杭州海康威视数字技术股份有限公司 | A kind of vehicle-mounted panoramic video display system, method and Vehicle Controller |
| CN110428361A (en) * | 2019-07-25 | 2019-11-08 | 北京麒麟智能科技有限公司 | A kind of multiplex image acquisition method based on artificial intelligence |
| CN110689512A (en) * | 2019-09-24 | 2020-01-14 | 中国科学院武汉岩土力学研究所 | Method for quickly splicing and fusing annular images of panoramic video in hole into image |
| CN111071152A (en) * | 2018-10-19 | 2020-04-28 | 图森有限公司 | Fisheye image processing system and method |
| CN111242842A (en) * | 2020-01-15 | 2020-06-05 | 深圳市中天安驰有限责任公司 | Image conversion method, terminal and storage medium |
| CN111369439A (en) * | 2020-02-29 | 2020-07-03 | 华南理工大学 | Panoramic view image real-time splicing method for automatic parking stall identification based on panoramic view |
| CN111669547A (en) * | 2020-05-29 | 2020-09-15 | 成都易瞳科技有限公司 | Panoramic video structuring method |
| CN112078538A (en) * | 2020-09-10 | 2020-12-15 | 浙江亚太机电股份有限公司 | Automatic opening system of car tail-gate based on-vehicle system of looking around |
| CN112435161A (en) * | 2020-11-12 | 2021-03-02 | 蘑菇车联信息科技有限公司 | Panoramic all-around image splicing method and system, electronic equipment and storage medium |
| CN112734639A (en) * | 2020-12-28 | 2021-04-30 | 南京欣威视通信息科技股份有限公司 | Image display splicing method and system |
| CN113012030A (en) * | 2019-12-20 | 2021-06-22 | 北京金山云网络技术有限公司 | Image splicing method, device and equipment |
| WO2021121251A1 (en) * | 2019-12-16 | 2021-06-24 | 长沙智能驾驶研究院有限公司 | Method and device for generating vehicle panoramic surround view image |
| CN113223092A (en) * | 2021-05-12 | 2021-08-06 | 天津大学 | Panoramic video generation method |
| WO2022017528A1 (en) * | 2020-07-24 | 2022-01-27 | 展讯通信(天津)有限公司 | Display method and system for vehicle-mounted avm, and electronic device and storage medium |
| WO2023272457A1 (en) * | 2021-06-28 | 2023-01-05 | 华为技术有限公司 | Apparatus and system for image splicing, and related method |
Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102045546A (en) * | 2010-12-15 | 2011-05-04 | 广州致远电子有限公司 | Panoramic parking assist system |
| US20140184737A1 (en) * | 2012-12-27 | 2014-07-03 | Hon Hai Precision Industry Co., Ltd. | Driving assistant system and method |
-
2017
- 2017-04-12 CN CN201710237136.8A patent/CN107424120A/en active Pending
Patent Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102045546A (en) * | 2010-12-15 | 2011-05-04 | 广州致远电子有限公司 | Panoramic parking assist system |
| US20140184737A1 (en) * | 2012-12-27 | 2014-07-03 | Hon Hai Precision Industry Co., Ltd. | Driving assistant system and method |
Non-Patent Citations (5)
| Title |
|---|
| 冯为嘉: "基于鱼眼镜头的全方位视觉及全景立体球视觉研究", 《中国博士学位论文全文数据库 信息科技辑》 * |
| 吴永祺: "基于全景视觉的汽车安全驾驶辅助系统的设计研究", 《中国优秀硕士学位论文全文数据库 工程科技Ⅱ辑》 * |
| 姜丽凤: "全景图拼接关键技术的研究", 《中国优秀硕士学位论文全文数据库 信息科技辑》 * |
| 王旭东: "基于环视的自动泊车方法研究与系统设计", 《中国优秀硕士学位论文全文数据库 工程科技Ⅱ辑》 * |
| 赵凯: "全景可视化辅助泊车系统研究", 《中国优秀硕士学位论文全文数据库 信息科技辑》 * |
Cited By (31)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN108198133A (en) * | 2017-12-06 | 2018-06-22 | 云南联合视觉科技有限公司 | A kind of quick joining method of vehicle panoramic image |
| CN108198133B (en) * | 2017-12-06 | 2021-09-17 | 云南联合视觉科技有限公司 | Rapid splicing method for vehicle panoramic images |
| CN107958440A (en) * | 2017-12-08 | 2018-04-24 | 合肥工业大学 | Double fish eye images real time panoramic image split-joint methods and system are realized on GPU |
| CN108364333A (en) * | 2018-02-11 | 2018-08-03 | 成都康烨科技有限公司 | Method and device based on multi-direction photography fitting vertical view |
| CN108492254A (en) * | 2018-03-27 | 2018-09-04 | 西安优艾智合机器人科技有限公司 | Image capturing system and method |
| CN110341597A (en) * | 2018-04-02 | 2019-10-18 | 杭州海康威视数字技术股份有限公司 | A kind of vehicle-mounted panoramic video display system, method and Vehicle Controller |
| CN110341597B (en) * | 2018-04-02 | 2020-11-27 | 杭州海康威视数字技术股份有限公司 | Vehicle-mounted panoramic video display system and method and vehicle-mounted controller |
| CN108638999A (en) * | 2018-05-16 | 2018-10-12 | 浙江零跑科技有限公司 | A kind of collision early warning system and method for looking around input based on 360 degree |
| CN109064397A (en) * | 2018-07-04 | 2018-12-21 | 广州希脉创新科技有限公司 | A kind of image split-joint method and system based on camera shooting earphone |
| CN111071152B (en) * | 2018-10-19 | 2023-10-03 | 图森有限公司 | Fish-eye image processing system and method |
| CN111071152A (en) * | 2018-10-19 | 2020-04-28 | 图森有限公司 | Fisheye image processing system and method |
| US11935210B2 (en) | 2018-10-19 | 2024-03-19 | Tusimple, Inc. | System and method for fisheye image processing |
| CN110428361A (en) * | 2019-07-25 | 2019-11-08 | 北京麒麟智能科技有限公司 | A kind of multiplex image acquisition method based on artificial intelligence |
| CN110689512A (en) * | 2019-09-24 | 2020-01-14 | 中国科学院武汉岩土力学研究所 | Method for quickly splicing and fusing annular images of panoramic video in hole into image |
| CN110689512B (en) * | 2019-09-24 | 2022-03-08 | 中国科学院武汉岩土力学研究所 | Method for quickly splicing and fusing annular images of panoramic video in hole into image |
| WO2021121251A1 (en) * | 2019-12-16 | 2021-06-24 | 长沙智能驾驶研究院有限公司 | Method and device for generating vehicle panoramic surround view image |
| US11843865B2 (en) | 2019-12-16 | 2023-12-12 | Changsha Intelligent Driving Institute Corp., Ltd | Method and device for generating vehicle panoramic surround view image |
| CN113012030A (en) * | 2019-12-20 | 2021-06-22 | 北京金山云网络技术有限公司 | Image splicing method, device and equipment |
| CN111242842B (en) * | 2020-01-15 | 2023-11-10 | 江苏中天安驰科技有限公司 | Image conversion method, terminal and storage medium |
| CN111242842A (en) * | 2020-01-15 | 2020-06-05 | 深圳市中天安驰有限责任公司 | Image conversion method, terminal and storage medium |
| CN111369439B (en) * | 2020-02-29 | 2023-05-23 | 华南理工大学 | Panoramic all-around image real-time splicing method for automatic parking space identification based on all-around |
| CN111369439A (en) * | 2020-02-29 | 2020-07-03 | 华南理工大学 | Panoramic view image real-time splicing method for automatic parking stall identification based on panoramic view |
| CN111669547A (en) * | 2020-05-29 | 2020-09-15 | 成都易瞳科技有限公司 | Panoramic video structuring method |
| CN111669547B (en) * | 2020-05-29 | 2022-03-11 | 成都易瞳科技有限公司 | Panoramic video structuring method |
| WO2022017528A1 (en) * | 2020-07-24 | 2022-01-27 | 展讯通信(天津)有限公司 | Display method and system for vehicle-mounted avm, and electronic device and storage medium |
| CN112078538A (en) * | 2020-09-10 | 2020-12-15 | 浙江亚太机电股份有限公司 | Automatic opening system of car tail-gate based on-vehicle system of looking around |
| CN112435161A (en) * | 2020-11-12 | 2021-03-02 | 蘑菇车联信息科技有限公司 | Panoramic all-around image splicing method and system, electronic equipment and storage medium |
| CN112734639B (en) * | 2020-12-28 | 2023-09-12 | 南京欣威视通信息科技股份有限公司 | Image display stitching method and system |
| CN112734639A (en) * | 2020-12-28 | 2021-04-30 | 南京欣威视通信息科技股份有限公司 | Image display splicing method and system |
| CN113223092A (en) * | 2021-05-12 | 2021-08-06 | 天津大学 | Panoramic video generation method |
| WO2023272457A1 (en) * | 2021-06-28 | 2023-01-05 | 华为技术有限公司 | Apparatus and system for image splicing, and related method |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN107424120A (en) | A kind of image split-joint method in panoramic looking-around system | |
| US20220044356A1 (en) | Large-field-angle image real-time stitching method based on calibration | |
| CN104851076B (en) | Panoramic looking-around parking assisting system and camera installation method for commercial car | |
| CN110020985B (en) | Video stitching system and method of binocular robot | |
| CN103763479B (en) | The splicing apparatus and its method of real time high-speed high definition panorama video | |
| JP5739584B2 (en) | 3D image synthesizing apparatus and method for visualizing vehicle periphery | |
| CN103617606B (en) | For assisting the vehicle multi-angle panorama generation method of driving | |
| CN105354796B (en) | Image processing method and system for auxiliary of driving a vehicle | |
| CN105825475B (en) | 360 degree of full-view image generation methods based on single camera | |
| CN111080724A (en) | Infrared and visible light fusion method | |
| CN104463778B (en) | A kind of Panoramagram generation method | |
| CN101276465A (en) | Method for automatically split-jointing wide-angle image | |
| CN106994936A (en) | A kind of 3D panoramic parking assist systems | |
| CN108769578A (en) | A kind of real-time omnidirectional imaging system and method based on multi-path camera | |
| CN105933695A (en) | Panoramic camera imaging device and method based on high-speed interconnection of multiple GPUs | |
| CN109948398A (en) | The image processing method and panorama parking apparatus that panorama is parked | |
| CN102629372A (en) | 360 degree panoramic aerial view generation method used for assisting vehicle driving | |
| CN108171735B (en) | Billion pixel video alignment method and system based on deep learning | |
| CN106856000A (en) | A kind of vehicle-mounted panoramic image seamless splicing processing method and system | |
| CN106910208A (en) | A kind of scene image joining method that there is moving target | |
| CN114640801B (en) | Car end panoramic view angle assisted driving system based on image fusion | |
| CN105046647B (en) | Complete 360 ° of panorama vehicle monitoring systems of liquid crystal instrument and its method of work | |
| CN115239820A (en) | Split type flying vehicle aerial view real-time splicing and parking space detection method | |
| CN116245722A (en) | Panoramic image stitching system and method applied to heavy high-speed vehicle | |
| CN105657268A (en) | Multi-viewpoint video splicing and fusion algorithm based on multiple resolutions |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| RJ01 | Rejection of invention patent application after publication |
Application publication date: 20171201 |
|
| RJ01 | Rejection of invention patent application after publication |