CN102045546B - Panoramic parking assist system - Google Patents

Panoramic parking assist system Download PDF

Info

Publication number
CN102045546B
CN102045546B CN 201010590428 CN201010590428A CN102045546B CN 102045546 B CN102045546 B CN 102045546B CN 201010590428 CN201010590428 CN 201010590428 CN 201010590428 A CN201010590428 A CN 201010590428A CN 102045546 B CN102045546 B CN 102045546B
Authority
CN
China
Prior art keywords
image
camera
module
angle
system
Prior art date
Application number
CN 201010590428
Other languages
Chinese (zh)
Other versions
CN102045546A (en
Inventor
周立功
Original Assignee
广州致远电子股份有限公司
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 广州致远电子股份有限公司 filed Critical 广州致远电子股份有限公司
Priority to CN 201010590428 priority Critical patent/CN102045546B/en
Publication of CN102045546A publication Critical patent/CN102045546A/en
Application granted granted Critical
Publication of CN102045546B publication Critical patent/CN102045546B/en

Links

Abstract

The invention discloses a panoramic parking assist system, which is a technique applied to the fields of automotive electronics and auto safety driving assistance. The system comprises a camera input module, an image correction module, a bird-eye perspective module, an image mosaicking module, a histogram equalization module and an image display module. The system is used for effectively solving the problem of car crash accidents caused by that the driver can not fully observe the situations around the car in the process of parking. By using the system, a 360-degree top view around a car body in real time can be showed for the drivers, so that the drivers can clearly see the relative positions of obstacles around the car body and the distances between the obstacles and the car body; and the blind visual areas around the car body can be eliminated so as to improve the security and stability of car in the process of parking. Through integrating a plurality of machine vision and digital image processing algorithms and fusing a plurality of advanced techniques in the field of machine vision, and based on the advantages of low cost, high performance, strong portability and the like, the system is convenient to be popularized and applied widely.

Description

A kind of panorama parking assisting system

Technical field

The present invention relates to the situation around the automobile is showed the driver intuitively,, belong to vehicle electric field and field of machine vision, especially relate to automotive safety driver assistance field to improve the fail safe and the stability of automobile parking.

Background technology

In recent years, along with developing rapidly of automobile industry and improving constantly of people's living standard, the automobile quantity of China increases just year by year.Non-professional automobile driver's ratio also increases year by year among the automobile driver simultaneously.When moveed backward in crowded, narrow place in highway, street, parking lot, garage etc., the driver should look forward to the prospect, and looks back again, a little because of carelessness automobile collision accident will take place.

Therefore various car steering auxiliary systems are arisen at the historic moment, and as the reverse radar system of ultrasonic ranging, set up video camera at rear view of vehicle and set up back-sight visual system or the like.Shortcomings such as they have all alleviated driver's burden to a certain extent, but all still are subject to disturb, and are directly perceived inadequately.More satisfactory driver assistance is to access the picture of overlooking automobile.By being installed in vehicle body camera all around road surface situation is on every side taken, utilize image processing techniques that it is synthesized a complete picture of overlooking then and be presented on the liquid crystal display screen.Like this driver if in car, just can finish see whole car various piece all around, complete without any dead angle and blind area.This technology is called as the vehicle-mounted panoramic imaging technique, can bring great convenience for car steering and traffic safety.

Number of patent application 200810163310.X has proposed a kind of scaling method of panorama auxiliary parking system, adopted the camera calibration method of Zhang Zhengyou to carry out the calculating of camera confidential reference items and outer ginseng, but when image splices, what use is the joining method of fixed position, not strong for the adaptability of different automobile types and camera vibrations.Number of patent application 200910119475.1 has proposed a kind of panoramic visible parking system, has adopted fpga chip to carry out Video processing and synthetic, but has needed another MCU chip to assist processing.

The present invention supports to correct 180 degree ultra wide-angle imaging heads, and adopted algorithm of histogram equalization, make system stronger to the adaptability of vehicle and camera vibrations, make the panoramic picture of parking not have any blind area, for the driver presents real 360 degree automobiles situation all around, avoided the generation of collision accident effectively.

Summary of the invention

The object of the present invention is to provide a kind of panorama parking assisting system, can present the situation of motor vehicle environment more intuitively for the driver, to improve the process safe and the stability of vehicle parking.

For realizing this purpose, the present invention takes following technical scheme:

The invention provides a kind of panorama automobile auxiliary system, be used for automobile parking and carry out panoramic imagery and driver assistance, comprising:

The camera input module, by place car body all around a plurality of 180 degree ultra wide-angle imaging heads on the four direction car body peripheral situation is taken, the camera parameter is D1 form, pal mode, resolution is 720x576, and the angle of installing is adjusted to a suitable position according to the situation of camera output image;

Image correction module according to the inner parameter and the distortion model of camera, is carried out distortion correction with the inputted video image of each camera;

Get a bird's eye view the perspective module,, the image behind each camera distortion correction is carried out the angle perspective transform, generate overhead view image according to the setting angle and the perspective transform principle of camera;

The image concatenation module according to the size of the three-dimensional coordinate and the automobile of each camera installation site, by means of being laid on the ground graticule line of vehicle's surroundings, is pieced together the overhead view image on each direction in an image, forms a panorama vertical view;

Image histogram equalization module, by algorithm of histogram equalization, the phenomenons that the brightness that exists between each splicing regions is different are eliminated, to reach better display effect;

Image-display units is exported to display device with final panorama vertical view, and display device can be a conformable display, also can be LCDs.

Described camera input module, distortion correction module, perspective transform module, image concatenation module, image equalization module are connected successively with image display.

According to the order of connection of above-mentioned functional module, the concrete job step of native system is as follows:

1) a plurality of cameras is installed in the four direction of vehicle body, and the angle of installing is adjusted to a suitable position according to the situation of camera output image;

2) a plurality of camera output signals are connected with video input interface on this system hardware;

3) according to the inner parameter and the distortion model of camera, the inputted video image of each camera is carried out distortion correction;

4) according to the setting angle and the perspective transform principle of camera, the image behind each camera distortion correction is carried out the angle perspective transform, generate overhead view image;

5), the overhead view image of a plurality of cameras is spliced, to obtain a panorama vertical view according to the three-dimensional coordinate and the automobile dimension of camera installation site;

6) according to the balanced principle of image histogram, the panorama vertical view is carried out histogram equalization;

7) final panorama vertical view is exported to display device.

System can support four or eight camera inputs, is installed on body forward structure, trunk afterbody and vehicle body both sides respectively.Because vehicle and the not of uniform size of car cause, so the angle that camera is installed also should regulate according to the situation of vehicle body, and the counterparty can both be photographed to the situations of 180 degree, and makes the image around the vehicle body can be presented on the zone line of photographic images.

The oblique installation that has a down dip of the general employing of the installation of camera can obtain car body image all around so preferably, and concrete setting angle is big or small and different according to different vehicles and vehicle body.The difference of shooting angle, the display position of situation in image around can directly having influence on, the distortion correction parameter of simultaneously different display positions is also inequality, so the setting angle of camera is rather important.

In existing many automobile visual device,, on the angle of vehicle body 45 degree, can there be the blind area because the visual angle of the wide-angle imaging head that adopts is limited.The present invention is because the maximum ultra wide-angle imaging heads that can proofread and correct 180 degree, and therefore the image around the vehicle body on four angles of vehicle body also still can be seen clearly, so just provides condition for the panoramic picture that obtains a width of cloth and do not have blind spot.

The native system maximum can be supported the data input of eight road D1 form cameras, and eight cameras are installed in respectively around the vehicle body, make the present invention can be applicable to the bigger vehicles of vehicle body such as high capacity waggon, passenger vehicle, engineering truck.

Main process chip used in the present invention is a dsp chip, its dominant frequency height, and the efficient height is carried out in instruction, and a plurality of video input/output interfaces are arranged, and its powerful Video processing ability, can handle multi-path video data in real time.Therefore native system can be handled the input picture of the camera of four road or eight road D1 forms, pal mode (resolution is 720x576) in real time.

Video output signals of the present invention both can have been supported the output of CVBS signal, also can support the output of LVDS signal.Wherein the LVDS signal also is a signal format commonly used in the Vehicular video signal.

The core algorithm of described panorama parking assisting system mainly is divided into following three steps:

1) image distortion correction algorithm carries out the calculating of distortion parameter by the distortion model of fitting of a polynomial, obtains the one-to-one relationship of distortion point and check point.

The distortion of digital picture has been owing to having adopted wide-angle lens to introduce, therefore the principle that at first produces from the angle analysis distortion of pure optics.The off-axis point imaging is that angle pencil of ray or light pencil all have aberration to exist, even have only chief ray to pass through optical system, because the influence of spherical aberration, it still can not be consistent with desirable paraxial light.Therefore, the height of chief ray and Gaussian image hand-deliver point is not equal to desirable image height.When the magnification ratio on a pair of conjugate image plane is not normal value, will make to look like to have lost similitude with respect to thing.This defective is exactly the distortion of system.

The imaging of undistorted camera lens, its line magnification ratio and object image-forming depart from the range-independence of optical axis, and promptly the line enlargement ratio of each several part all is the same.And the lens imaging of distortion arranged, its line magnification ratio then changes with the variation that the ideal position of object leaves optical axis distance, like this, just make object through the distortion camera lens after imaging depart from ideal position, the size that departs from the ideal position distance is called distortion.

Distortion mainly comprises radial distortion and tangential distortion.Tangential distortion causes mainly due to the non-axial symmetry factor of actual optical system, as the optical material inhomogeneities, and optical element local deformation, part installation site and optical axis are asymmetric etc. in the optical system.So it is mainly caused by processing.Radial distortion when optical design, can be corrected to a certain degree usually, but this often can not satisfy required precision, and especially the system residual amount in big visual field can produce certain influence to the result.

Based on the basic mathematic model of geometric operation, we were divided into distortion correction for two steps usually.The first step is original image to be carried out pixel coordinate space carry out geometric transformation, the purpose of doing like this is that the distortion point on the original image is corresponded on the correct coordinate points, yet when our common practical operation, in order to make each pixel after the correction all have the original image pixel corresponding with it, we often the image of the target after proofreading and correct look for pixel corresponding with it the distortion original image; Second step was the pixel value that redefines new pixel.This is because through after the top coordinate transform, corresponding pixel not necessarily can drop on the discrete rounded coordinate point, therefore needs more reasonably to determine corresponding pixel value.Here we normally adopt the bilinear interpolation algorithm.

2) get a bird's eye view the perspective transform algorithm,, obtain image and the point of birds-eye perspective and the mapping matrix one by one of point that shooting obtains by calculating the homography matrix H of projection;

Get a bird's eye view conversion process, belong to the aerial prospective conversion, the image transitions that camera is tilted to photograph is the image that photographs vertically downward.Central projection method is used in perspective projection, finally converges to a projection line that is called as the point of projection centre along a series of, and the spot projection in the three-dimensional physical world is transformed in the two dimensional image plane.Projective transformation is a kind of specific homography conversion, is two width of cloth images that same three-dimensional body projects to respectively under two different projection planes are connected.Therefore the key of getting a bird's eye view conversion process is to obtain projection homography matrix H.

3) algorithm of histogram equalization by calculating the histogram of whole panorama splicing figure, is carried out histogram equalization, thereby eliminates the phenomenon that the brightness that produces when camera is taken on all directions differs.

Image histogram is a kind of crucial image analysis tool in the image processing, and it is described to be the statistical relationship of each gray scale and its frequency of occurrences in the digital picture.On mathematical meaning, image histogram is the function of each gray scale statistical property of image and image gray levels, and what its reflected is number of times or the probability that each gray scale occurs in the piece image.

The histogram equalization method is modified to the histogram that uniform gray level distributes to the histogram of original image by the greyscale transformation function, then by balanced histogram modification original image.When the histogram of image is even a distribution, the comentropy maximum of image, the amount of information maximum that this moment, image comprised, image seems just to seem clear that brightness is also consistent.This method is based on the cumulative distribution function, and its transforming function transformation function depends on the cumulative distribution function of image grey level histogram.It carries out same conversion to entire image, is also referred to as overall histogram equalization.

The concrete software flow of described panorama parking assisting system is:

1), the fault image on all directions is carried out distortion correction according to known distortion parameter and distortion model;

For the shooting angle of camera wider, in order there not to be blind area, all around the camera of installing on the direction is the wide-angle imaging heads of 180 degree, so the image of all directions photographs certainly exists very big distortion, the correction of fault image is just become the key of algorithm.

Suppose (x, y) be before proofreading and correct on the image a bit, adopt the fitting of a polynomial algorithm, point (x, y) with proofread and correct after corresponding points (u, v) the pass between is:

x = Σ i = 0 n Σ j = 0 n - i a ij u i v j y = Σ i = 0 n Σ j = 0 n - i b ij u i v j

Wherein, n is the fitting of a polynomial number of times, a Ij, b IjThe distortion correction that the system that is needs is regulated parameter.

In order to try to achieve best a Ij, b Ij, by least square method, allow error of fitting square minimum, in the hope of space coordinate transformation coefficient a Ij, b IjAsk for a IjFormula be:

ϵ = Σ i = 1 L [ x i - Σ i = 0 n Σ j = 0 n - i a ij u i v j ] 2

In addition, (u, v) (x y), in order to make the pixel value value more reasonable, has used the bilinear interpolation algorithm to carry out the correspondence of pixel here by calculating a floating number point owing to the integral point after proofreading and correct.The concrete principle of bilinear interpolation is as follows:

Suppose the x of x '=(int), the y of y '=(int),

Δ x=x-x ' then, Δ y=y-y '.

Suppose that f (x ', y ') is the pixel value of x ' row y ' row on the original input picture, f ' (u v) is the pixel value of the corresponding points on the image after proofreading and correct, and then uses the bilinear interpolation algorithm to get:

f′(u,v)=(1-Δx)*(1-Δy)*f(x′,y′)+(1-Δx)*Δy*f(x′,y′+1)

+Δx*(1-Δy)*f(x′+1,y′)+Δx*Δy*f(x′+1,y′+1)

2) according to the setting angle of camera with get a bird's eye view the perspective principle, with the image transform behind the distortion correction for getting a bird's eye view effect image;

After the distortion correction, in order to make panoramic picture that a kind of overall visual effect from overlooking is arranged in the air, need get a bird's eye view perspective transform to the image on all directions, make image seem down to take from aerial, so let us need obtain the homography matrix H of perspective transform just as camera;

If (x1, y1), (x2 y2) represents the plane of delineation behind the distortion correction and the corresponding points on birds-eye view plane respectively, and then both mapping relations are:

ρ x 2 y 2 1 = H * x 1 y 1 1

Promptly ρ x 2 y 2 1 = c 11 c 12 c 13 c 21 c 22 c 23 c 31 c 32 c 33 x 1 y 1 1

H - 1 = cos a cos b sin b - sin a cos b - cos a sin b cos b sin a sin b sin a 0 cos a

Wherein ρ is a scale factor, and a is the vertical dip angle that camera is installed, and b is the offset angle that camera is installed.Usually we think that the horizontal direction deflecting angle b of camera is 0, so we only need the vertical dip angle of determining that camera is installed, just can calculate the homography matrix H of perspective transform, also just can obtain a width of cloth counterparty to birds-eye view.

3),, the birds-eye view on all directions is spliced on same width of cloth image by means of being laid on the ground graticule line of vehicle's surroundings according to the three-dimensional coordinate of camera installation site on all directions and the size of automobile.

Image splicing problem is the difficult point in computer vision, the image processing.By means of being laid on ground graticule line around the automobile, the interface portion of arbitrary two width of cloth figure merged effectively with transition is connected, this method is simply quick, is convenient to implement and debugs.Therefore, native system adopts this method.

In order to obtain panoramic effect, system need look like the birds-eye view on all directions to be stitched together, and makes driver a glance just can see the car body situation on all directions on every side.Here, the length and width of our known automobile, three-dimensional coordinate with camera installation site on all directions, just can be with software flow 2) birds-eye view of resulting all directions generally pieces together in a rectangular area, but in order to obtain a satisfied splicing effect, just need be by means of being laid on the ground graticule line of vehicle's surroundings.In the process of system's Installation and Debugging, the quality quality of the splicing effect of all directions birds-eye view junction, determined the splicing effect of whole panorama sketch, so, we are according to the situation of ground graticule line in the coincidence and the transition of each junction, come the putting position and the effect of each birds-eye view are finely tuned, thereby reach a satisfied panorama splicing effect.

Can realize the satisfied transition in each junction and be connected by the step of following fine setting:

(1) can finely tune the height of each camera, make the corresponding area of getting a bird's eye view the zone carry out amplification slightly and dwindle, make that getting a bird's eye view the zone for four is on the same horizontal plane;

(2) can finely tune the camera level angle, make the corresponding left and right horizontal unanimity of getting a bird's eye view the zone;

(3) can finely tune the coordinate of camera apart from autocentre, the putting position of the birds-eye view of translation correspondence just makes the grid of junction connect more coherent.

4) according to algorithm of histogram equalization, panorama is spliced figure carry out luminance proportionization, to eliminate because the visual impact that the brightness difference of different directions photographic images causes.Concrete grammar is as follows:

(1) according to following formula, calculate the grey level histogram of original image, the gray value of original image is divided into 256 five equilibriums:

P k = n k n ( 0 ≤ k ≤ 255 )

Wherein n is the sum of all pixels of original image, and k represents k gray scale, n kThe number of the pixel that gray scale k occurs in the presentation video, P kIn all pixels of expression original image, the probability that gray scale k occurs;

(2) calculate the gray scale cumulative distribution Function Y of original image according to step (1) k, and obtain the greyscale transformation mapping table according to the cumulative distribution function:

Y k = Σ j = 0 k P k = Σ j = 0 k n j n ( 0 ≤ k ≤ 255 )

D k = Y k × 255 n + 0.5

Wherein, D kBe k gray scale transformation mapping corresponding gray scale value, add that 0.5 effect is to round up;

(3), the gray value of all pixels of original image is mapped to new gray value according to the resulting transformed mappings table of step (2).

The central idea that histogram equalization is handled is from becoming the even distribution in whole tonal ranges between certain gray area of relatively concentrating the grey level histogram of original image.Histogram equalization carries out non-linear stretching to image exactly, redistributes image pixel value, makes the pixel quantity in certain tonal range roughly the same.Histogram equalization is exactly the histogram distribution of given image to be changed over even distribution histogram distribute.

Description of drawings

Fig. 1 is the allomeric function module FB(flow block) of system of the present invention;

Fig. 2 is the software module FB(flow block) of system of the present invention;

Fig. 3 is enforcement principle and the camera coverage schematic diagram of the present invention when four cameras are implemented;

Fig. 4 is the hardware block diagram of the present invention when four cameras are implemented;

Fig. 5 is a panorama splicing composite diagram of the present invention;

Fig. 6 is the scheme of installation of the present invention when eight cameras are implemented;

Fig. 7 is the hardware block diagram of the present invention when eight cameras are implemented.

Embodiment

The following stated only is preferred embodiment of the present invention, does not therefore limit protection scope of the present invention.

Fig. 1 is the flow chart of allomeric function module of the present invention, connects successively between each module, and distortion correction module wherein, perspective transform module, image concatenation module and image equalization module all are to realize its function by software algorithm.

Fig. 2 has shown the software flow block diagram of described system, can clearly find out from figure, and the core algorithm of described panorama parking assisting system mainly is divided into following three steps:

1) image distortion correction;

2) get a bird's eye view perspective transform;

3) histogram equalization.

First embodiment

This embodiment realizes function of the present invention for using four 180 degree ultra wide-angle imaging heads.

Fig. 3 is enforcement principle and the camera coverage signal of the present invention when four cameras are implemented, the ultra wide-angle imaging head 110,120,130 and 140 of four 180 degree is installed on front truck mark, trunk handle, two rearview mirror belows respectively, captures four shooting areas and is respectively 110a, 120a, 130a and 140a.Because the angle of these four cameras is near 180 degree, therefore the visual angle of shooting area 110a, 120a, 130a and the 140a of four cameras is very big, thereby is provided as condition for constituting 360 degree panoramic pictures.In order to obtain car body all images all around, the image of four camera shootings has lap on the border, and these laps can be handled well when the synthetic splicing of image.

Fig. 4 has shown system's hardware block diagram when four cameras are implemented, and four-way CCD camera 201,202,203 and 204 is analog signal cameras, and output signal is the CVBS signal, and when system moved, primary processor 206 passed through I 2C bus configuration Video Decoder 205 makes Video Decoder 205 can receive the input of four road CVBS camera signals, and the analog signal of conversion input is a digital signal then, afterwards four way word signals is transferred to primary processor 206.Memory 207 is as the memory space of the program and the data of primary processor.Primary processor 206 with four way word signals successively through distortion correction, get a bird's eye view perspective, synthetic splice, after the software processes of histogram equalization, formed a width of cloth panorama birds-eye view picture, at last, show to LCDs 208 this width of cloth image data transmission, make around the driver can watch well.

Fig. 5 has shown when four camera embodiment, final panorama is got a bird's eye view splicing effect figure, system is placed on the body of a motor car overhead view image centre of stitching image, with the birds-eye view picture on the four direction according to the installation site of camera and the Aspect Ratio of vehicle body, be placed on pro rata on the corresponding direction, after synthetic splicing and histogram equalization, just the image around the vehicle body has been presented in face of the driver intuitively.

Second embodiment

This embodiment realizes function of the present invention for using eight 180 ultra wide-angle imaging heads.

Because the vehicle body of high capacity waggon, passenger vehicle, engineering truck is all long, the driver is difficult to see clearly tailstock portion and the other situation of car, if but four cameras only are installed, be difficult to again the surroundings of 360 degree is covered, therefore, present embodiment has used eight cameras, solves the implementation issue on oversize vehicle.

Fig. 6 is the schematic diagram of the present invention when eight cameras are implemented.The installation site was identical when camera 301,302,303,304 was implemented with four cameras, and 305,306,307 and 308 of cameras are installed near four diagonal angles of vehicle body, like this, make camera can access whole scenes.

Fig. 7 is the hardware block diagram of the present invention when eight cameras are implemented, and primary processor 411 has two video input ports, can receive the input of four road D1 form cameras respectively, therefore, uses two Video Decoders, the four-way CCD camera data of decoding respectively.Eight road cameras 401,402,403,404,405,406,407 and 408 are analog signal cameras, and output signal is the CVBS signal, and when system moved, primary processor 411 passed through I 2C bus configuration Video Decoder 409 and 410 makes Video Decoder 409 and 410 can receive the input of four road CVBS camera signals respectively, and the analog signal of conversion input is a digital signal then, respectively four way word signals is transferred to primary processor 411 afterwards.Memory 412 is as the memory space of the program and the data of primary processor.Primary processor 411 with eight way word signals successively through distortion correction, get a bird's eye view perspective, synthetic splice, after the software processes of histogram equalization, formed a width of cloth panorama birds-eye view picture, at last, show to LCDs 413 this width of cloth image data transmission, make around the driver can watch well.

In sum, the present invention combines the Processing Algorithm of multiple machine vision and Digital Image Processing, merged the multinomial advanced technology of field of machine vision, make the automobile parking process have higher fail safe and stability, and, be convenient to large scale application based on advantages such as low cost, high-performance and portability are strong.

Although do not enumerate more embodiment among the present invention, but for those skilled in the art and Yan Junke understands, any each technical characterictic of the present invention is carried out simple substitution to finish the behavior that realizes constructed effect, all should be considered as unconventional initiative spirit of the present invention not.

Claims (1)

1. panorama parking assisting system is characterized in that comprising at least as lower module:
The camera input module places the car body a plurality of ultra wide-angle imaging heads on the four direction all around;
The processing module that is used for image distortion correction;
Be used for image and get a bird's eye view the processing module of perspective transform;
Be used for the synthetic processing module of image splicing;
The processing module that is used for the image histogram equalization;
Be used for the module that image shows;
Described camera input module, distortion correction module, perspective transform module, image concatenation module, image equalization module are connected successively with image display;
Described a kind of panorama parking assisting system, concrete job step is as follows:
1) a plurality of cameras is installed in the four direction of vehicle body, and the angle of installing is adjusted to a suitable position according to the situation of camera output image;
2) a plurality of camera output signals are connected with video input interface on this system hardware;
3) according to the inner parameter and the distortion model of camera, the inputted video image of each camera is carried out distortion correction;
4) according to the setting angle and the perspective transform principle of camera, the image behind each camera distortion correction is carried out the angle perspective transform, generate overhead view image;
5), the overhead view image of a plurality of cameras is spliced, to obtain a panorama vertical view according to the three-dimensional coordinate and the automobile dimension of camera installation site;
6) according to the balanced principle of image histogram, the panorama vertical view is carried out histogram equalization;
7) final panorama vertical view is exported to display device;
Described panorama parking assisting system is according to the setting angle of each camera, utilization get a bird's eye view the perspective transform principle with the correction on each direction after image transitions for getting a bird's eye view vertical view, the conversion at formation visual angle, concrete grammar is the homography matrix H that obtains perspective transform:
If (x1, y1), (x2 y2) represents the plane of delineation behind the distortion correction and the corresponding points on birds-eye view plane respectively, and then both mapping relations are:
ρ x 2 y 2 1 = H * x 1 y 1 1
Promptly ρ x 2 y 2 1 = c 11 c 12 c 13 c 21 c 22 c 23 c 31 c 32 c 33 x 1 y 1 1
H - 1 = cos a cos b sin b - sin a cos b - cos a sin b cos b sin a sin b sin a 0 cos a
Wherein ρ is a scale factor, and a is the vertical dip angle that camera is installed, and b is the offset angle that camera is installed.
CN 201010590428 2010-12-15 2010-12-15 Panoramic parking assist system CN102045546B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN 201010590428 CN102045546B (en) 2010-12-15 2010-12-15 Panoramic parking assist system

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN 201010590428 CN102045546B (en) 2010-12-15 2010-12-15 Panoramic parking assist system

Publications (2)

Publication Number Publication Date
CN102045546A CN102045546A (en) 2011-05-04
CN102045546B true CN102045546B (en) 2013-07-31

Family

ID=43911259

Family Applications (1)

Application Number Title Priority Date Filing Date
CN 201010590428 CN102045546B (en) 2010-12-15 2010-12-15 Panoramic parking assist system

Country Status (1)

Country Link
CN (1) CN102045546B (en)

Families Citing this family (38)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN102291573A (en) * 2011-08-30 2011-12-21 山推工程机械股份有限公司 Panoramic operation monitoring method and system of engineering machinery
CN102736634B (en) * 2012-07-10 2014-09-03 浙江捷尚视觉科技股份有限公司 Camera angle regulation method for vehicle panorama
CN103661270A (en) * 2012-09-21 2014-03-26 北京酷信通科技有限公司 Car panoramic intelligent safeguard system
CN103065318B (en) * 2012-12-30 2016-08-03 深圳普捷利科技有限公司 The curved surface projection method and device of multiple-camera panorama system
CN103929613A (en) * 2013-01-11 2014-07-16 深圳市灵动飞扬科技有限公司 Three-dimensional stereoscopic aerial view driving auxiliary method, apparatus and system
CN103226827B (en) * 2013-03-21 2016-04-27 深圳市汉华安道科技有限责任公司 Camera external parameter correction method, device and auxiliary parking system
CN103253193B (en) * 2013-04-23 2015-02-04 上海纵目科技有限公司 Method and system of calibration of panoramic parking based on touch screen operation
CN103253194A (en) * 2013-05-17 2013-08-21 黄梅县云泰时代光学仪器有限公司 Traveling vehicle auxiliary system
CN104240225B (en) * 2013-06-19 2017-11-21 德尔福电子(苏州)有限公司 A kind of step scaling method of aerial view system camera one
KR101497933B1 (en) * 2013-08-19 2015-03-03 현대모비스(주) System and Method for compositing various images using clustering technique
CN103458192B (en) * 2013-09-04 2017-03-29 上海华凯展览展示工程有限公司 The method and system of perspective transform in a kind of vertical view arenas
CN103646385A (en) * 2013-11-21 2014-03-19 江西好帮手电子科技有限公司 Method and system for automatic stitching of panoramic parking image
CN103617606B (en) * 2013-11-26 2016-09-14 中科院微电子研究所昆山分所 For assisting the vehicle multi-angle panorama generation method of driving
TWI499999B (en) * 2014-03-27 2015-09-11 Univ Shu Te The 3D ring car image system based on probability calculation and its obtaining method
CN105313773A (en) * 2014-07-24 2016-02-10 深圳市驰晶科技有限公司 High-definition panoramic parking and driving assisting system
CN104735408B (en) * 2015-02-06 2019-05-24 广东光阵光电科技有限公司 A kind of police vehicle mounted surveillance method in no dead angle and monitoring system
CN104811663B (en) * 2015-04-17 2018-09-04 浙江吉利汽车研究院有限公司 The image processing method and processing unit of vehicle
CN106204416B (en) * 2015-05-04 2019-04-23 深圳市汉华安道科技有限责任公司 Panoramic parking assist system and its wide angle picture adjustment method and device
CN104851076B (en) * 2015-05-27 2018-10-09 武汉理工大学 Panoramic looking-around parking assisting system and camera installation method for commercial car
CN105338485A (en) * 2015-08-03 2016-02-17 浙江海洋学院 Auxiliary positioning device and method for mobile equipment
US9986150B2 (en) * 2015-09-30 2018-05-29 Ricoh Co., Ltd. Algorithm to estimate yaw errors in camera pose
CN105354796B (en) * 2015-10-12 2018-12-04 北京鑫洋泉电子科技有限公司 Image processing method and system for auxiliary of driving a vehicle
CN106683044B (en) * 2015-11-10 2020-04-28 中国航天科工集团第四研究院指挥自动化技术研发与应用中心 Image splicing method and device of multi-channel optical detection system
CN105554447B (en) * 2015-12-11 2018-10-23 北京天地玛珂电液控制系统有限公司 A kind of coal working face real-time video splicing system based on image processing techniques
CN105451000B (en) * 2015-12-27 2019-02-15 高田汽车电子(上海)有限公司 A kind of vehicle-mounted panoramic viewing system and method based on monocular rearview camera
CN106373091B (en) * 2016-09-05 2019-05-07 山东省科学院自动化研究所 Panorama gets a bird's eye view method for automatically split-jointing, system and the vehicle of image in parking
CN106541944B (en) * 2016-11-07 2019-03-05 纵目科技(上海)股份有限公司 A kind of warehouse compartment detection method, system and mobile device
CN106846243A (en) * 2016-12-26 2017-06-13 深圳中科龙智汽车科技有限公司 The method and device of three dimensional top panorama sketch is obtained in equipment moving process
CN107146255A (en) * 2017-04-05 2017-09-08 纵目科技(上海)股份有限公司 Panoramic picture error calibration method and device
CN107424120A (en) * 2017-04-12 2017-12-01 湖南源信光电科技股份有限公司 A kind of image split-joint method in panoramic looking-around system
TWI602154B (en) * 2017-04-26 2017-10-11 偉詮電子股份有限公司 Panoramic image stitching method and system thereof
CN107389088A (en) * 2017-05-27 2017-11-24 纵目科技(上海)股份有限公司 Error correcting method, device, medium and the equipment of vehicle-mounted inertial navigation
CN107277445B (en) * 2017-06-29 2020-05-12 深圳市元征科技股份有限公司 Vehicle-mounted equipment
US10643342B2 (en) * 2018-02-08 2020-05-05 Huawei Technologies Co., Ltd. Group optimization depth information method and system for constructing a 3D feature map
CN110874817A (en) * 2018-08-29 2020-03-10 上海商汤智能科技有限公司 Image stitching method and device, vehicle-mounted image processing device, electronic equipment and storage medium
CN109685000A (en) * 2018-12-21 2019-04-26 广州小鹏汽车科技有限公司 A kind of method for detecting parking stalls and device of view-based access control model
CN109697696B (en) * 2018-12-24 2019-10-18 北京天睿空间科技股份有限公司 Benefit blind method for panoramic video
CN110719411B (en) * 2019-12-16 2020-04-03 长沙智能驾驶研究院有限公司 Panoramic all-around view image generation method of vehicle and related equipment

Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN101236654A (en) * 2007-01-31 2008-08-06 三洋电机株式会社 Method and apparatus for camera calibration, and vehicle
CN101236655A (en) * 2007-02-01 2008-08-06 三洋电机株式会社 Camera calibration apparatus and vehicle
CN101425181A (en) * 2008-12-15 2009-05-06 浙江大学 Panoramic view vision auxiliary parking system demarcating method
CN101442618A (en) * 2008-12-31 2009-05-27 葛晨阳 Method for synthesizing 360 DEG ring-shaped video of vehicle assistant drive

Family Cites Families (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2007233996A (en) * 2006-02-06 2007-09-13 Olympus Imaging Corp Image compositing apparatus, image compositing method, image compositing program and recording medium

Patent Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN101236654A (en) * 2007-01-31 2008-08-06 三洋电机株式会社 Method and apparatus for camera calibration, and vehicle
CN101236655A (en) * 2007-02-01 2008-08-06 三洋电机株式会社 Camera calibration apparatus and vehicle
CN101425181A (en) * 2008-12-15 2009-05-06 浙江大学 Panoramic view vision auxiliary parking system demarcating method
CN101442618A (en) * 2008-12-31 2009-05-27 葛晨阳 Method for synthesizing 360 DEG ring-shaped video of vehicle assistant drive

Also Published As

Publication number Publication date
CN102045546A (en) 2011-05-04

Similar Documents

Publication Publication Date Title
CN103778649B (en) Imaging surface modeling for camera modeling and virtual view synthesis
US9866752B2 (en) Systems and methods for producing a combined view from fisheye cameras
CN103035005B (en) The scaling method that panorama is parked, and device, a kind of automatic calibration method
US9160981B2 (en) System for assisting driving of vehicle
US8242895B2 (en) Method and system for forming surrounding seamless bird-view image
DE60224473T2 (en) Driving assistance system
US20140152774A1 (en) Vehicle periphery monitoring device
US8514282B2 (en) Vehicle periphery display device and method for vehicle periphery image
CN101236655B (en) Camera calibration apparatus and vehicle, and method
DE102006003538B3 (en) Image acquisitions merging method for bird`s eye perspective, involves selecting image sections such that shadowing effects produced by moved objects are projected during transition in overall image from image section to other section
JP4766841B2 (en) Camera device and vehicle periphery monitoring device mounted on vehicle
CN103617606B (en) For assisting the vehicle multi-angle panorama generation method of driving
EP2623374B1 (en) Vision system for commercial vehicles for the display of the statutory fields of view of a main mirror and a wide-angle mirror
EP1850595B1 (en) Periphery supervising device, and periphery supervising method
US7486801B2 (en) Monitoring system for monitoring surroundings of vehicle
US20130058589A1 (en) Method and apparatus for transforming a non-linear lens-distorted image
US10523865B2 (en) Three dimensional rendering for surround view using predetermined viewpoint lookup tables
DE10296593B4 (en) Driving support device
CN100438623C (en) Image processing device and monitoring system
US7058235B2 (en) Imaging systems, program used for controlling image data in same system, method for correcting distortion of captured image in same system, and recording medium storing procedures for same method
KR100936557B1 (en) Perimeter monitoring apparatus and image display method for vehicle
JP5222597B2 (en) Image processing apparatus and method, driving support system, and vehicle
US6947059B2 (en) Stereoscopic panoramic image capture device
CN105141945B (en) Panorama camera chain (VPM) on-line calibration
KR100937750B1 (en) Drive assisting system

Legal Events

Date Code Title Description
DD01 Delivery of document by public notice

Addressee: Liu Jing

Document name: Notification of Passing Preliminary Examination of the Application for Invention

C06 Publication
PB01 Publication
DD01 Delivery of document by public notice

Addressee: Liu Jing

Document name: Notification of Publication of the Application for Invention

SE01 Entry into force of request for substantive examination
C10 Entry into substantive examination
CB02 Change of applicant information

Address after: 2 building, 7 floor, Huang Zhou Industrial Zone, Che Po Road, Guangdong, Guangzhou, Tianhe District 510660, China

Applicant after: Guangzhou Zhiyuan Electronic Co., Ltd.

Address before: 2 building, 7 floor, Huang Zhou Industrial Zone, Che Po Road, Guangdong, Guangzhou, Tianhe District 510660, China

Applicant before: Guangzhou ZHIYUAN Electronics Co., Ltd.

C53 Correction of patent for invention or patent application
GR01 Patent grant
C14 Grant of patent or utility model
CP03 Change of name, title or address
CP03 Change of name, title or address

Address after: Room 517, 1023 Gaopu Road, Tianhe Software Park, Tianhe District, Guangzhou City, Guangdong Province

Patentee after: Guangzhou ZHIYUAN Electronics Co., Ltd.

Address before: 510660 2/F, 7 buildings, Chepi Road Huangzhou Industrial Zone, Tianhe District, Guangzhou City, Guangdong Province

Patentee before: Guangzhou Zhiyuan Electronic Co., Ltd.