CN103247030A - Fisheye image correction method of vehicle panoramic display system based on spherical projection model and inverse transformation model - Google Patents

Abstract

A fisheye lens is required to be adopted by the vehicle panoramic display system to obtain a 180-degree wide visual field, however, the visual field is deformed severely. The invention discloses a fisheye image correction method of the vehicle panoramic display system based on a spherical projection model and an inverse transformation model. The method comprises the steps as follows: (1), confirming an area A which is required to be displayed in an aerial view, and establishing a world coordinate system to position the display area; (2), obtaining a coordinate B of the area required to be displayed in a spherical longitude and latitude mapping coordinate system according to camera mounting parameters; (3), confirming the position C of the coordinate B in an original image collected by the fisheye lens according to the fisheye lens spherical projection model; (4), establishing a coordinate transformation relation from C to A through inverse projection transformation; and (5), performing interpolation arithmetic for non-integer points through a bilinear interpolation method to obtain a complete aerial view in a certain direction of the vehicle. The algorithm provided by the invention is simple in implementation and strong in universality and instantaneity.

Description

Vehicle panorama display system correcting fisheye image method based on spherical projection model and inverse transformation model

Technical field

The present invention relates to vehicle electric field, is a kind of method of fish eye images distortion correction.

Background technology

Now, the improving constantly of people's living standard, the auto industry of China also develops rapidly, and according to the statistics of Traffic Administration Bureau of the Ministry of Public Security, the recoverable amount of national motor vehicle reaches about 1.99 hundred million, and wherein automobile is approximately 8,500 ten thousand.This also causes traffic hazard more and more, learn according to Public Security Department and State Statistics Bureau's statistics, annual because the dead number of days of traffic hazard of China is about 100,000 people, but also be ascendant trend year by year, and in these increasing traffic hazards, the accident that occurs during reversing surpasses half, this and the urban traffic situation that more and more blocks up now and automobilist more and more deprofessionalization have close relationship, women driver and the probability that accident occurs of just obtaining to move backward among the driver of driving license bigger.

At present, the radar for backing car technology is ripe, astern radar device also begins to popularize at all above the passenger car, now above the popularization commercial car, radar for backing car is a basic configuration, also loaded radar for backing car above the part minicar, as seen, its technology is quite ripe, and cost is also very cheap, generally be to install four ultrasonic radars above the vehicle rear bumper additional to pop one's head in perception rear view of vehicle barrier, show that vehicle points out the driver apart from distance and the corresponding voice of barrier driving an indoor charactron, but such radar for backing car can not perceive flying height than the barrier of the high part of radar probe, cause military road driver when such barrier occurring with regard to being easy to the demonstration of appearance mistake and wrong report, the reversing accident occurs.

High definition camera reverse aid is also being used more and more widely now, on middle-and-high-ranking passenger car, this device is popularized, in the height configuration of popularization passenger car, also can be contained in this reverse aid, this device is by being contained in an environmental information that the high definition camera is taken rear view of vehicle of the tailstock, the video information of taking is transferred in the pilothouse environmental information that display in the console shows rear view of vehicle, in display, can add range scale, be used for pointing out the distance of driver's car and car and obstruction, this and radar for backing car are compared, vivider, more humane, allow the driver that the rear environmental information is come into plain view, thus better direct drivers reversing, the probability of generation accident when having reduced reversing.But also can there be some blind areas in the image of a camera shooting, and the barrier of side can't detect.

Continuous development along with automotive engineering, the video of a direction can not satisfy the driver and drive requirement, need the four direction video to show simultaneously, produce a kind of panorama sketch of getting a bird's eye view of overlooking full car directly over the automobile, eliminate environment blind area around the car fully, not only satisfy reversing visual field demand, can also satisfy when vehicle advances when vulgar demand to environment, eliminate the blind area that the A post produces when turning, it is lighter that the driver is driven.At present, only on luxurious cars such as BMW X6 and Infiniti EX35, just introduce this panorama display system.This panorama display system needs the vision signal of the camera input more than 4 tunnel or 4 tunnel, this be installed in more than 4 the tunnel or 4 the tunnel vehicle all around each above the direction, be used for environmental information on each and every one direction of perception, and then each camera input video is spliced into a panorama general view, be presented at above the display of pilothouse console.Because camera installation site restriction, that can not install is too high, all each cameras can not adopt common perspective camera, need install additional fish eye lens again low-mounting height obtain environmental information in the wideer visual field, and serious distortion can appear in the image that fish eye lens is taken, need before the video-splicing distorted image proofreaied and correct in beginning and obtain common see-through view or aerial view, obtain the figure of panorama demonstration clearly in order to splice, correction to distorted image before must obtain reasonable skeleton view or general view, and this also has higher requirement to the correcting fisheye image algorithm.

Summary of the invention

The present invention is directed to the actual conditions that fish eye lens uses in automobile, fish eye lens is installed in stationkeeping on the car, location parameter also is convenient to measure relative to the earth, based on this utilization singularity, propose a kind of vehicle panorama display system correcting fisheye image method based on spherical projection model and inverse transformation model, algorithm of the present invention is realized simple, highly versatile, low to processor requirement, very strong real-time is arranged.

By the following technical solutions, realize foregoing invention:

A kind of vehicle panorama display system correcting fisheye image method based on spherical projection model and inverse transformation model, it is characterized in that following, comprise following steps: (1), the position of installing according to camera, determine the vehicle-periphery range of information a that the driver need see, and in world coordinate system and ideal image coordinate system, a is carried out coordinate setting; (2), the location parameter installed onboard according to camera, camera height h, the angle α camera optical axis and ground sets up how much contacts at fish eye lens hemisphere face and shooting coordinate system, determines the position coordinates b of a in sphere longitude and latitude mapping point system; (3), according to fish eye lens spherical projection model, determine in the sphere longitude and latitude mapping point that position coordinates b takes position coordinates c in the distorted image at fish eye lens; (4), basis is set up from a to c coordinate one-to-one relationship against projective transformation at last; (5), utilize the bilinear interpolation computing, carry out interpolation arithmetic to obtaining among the c non-integer coordinates, obtain a complete general view clearly.

Described step (1), conversion mutually between ideal image coordinate system and the world coordinate system, indication range a is fore-and-aft distance m rice in the world coordinate system, lateral separation n rice; Conversion generated ideal figure M*N, correspondence image matrix R, each pixel in the vertical ratio of corresponding actual range is: m/M, grid scale: n/N, pixel R among the image array R (i, j) respective coordinates:

$\left\{\begin{array}{c}\mathrm{Px}=j*n/N-n/2\\ \mathrm{Py}=m/2-i*m/M\end{array}\right.---(1-1)$

Described step (2) according to the camera installation parameter, is determined viewing area a in sphere longitude and latitude coordinate system, middle coordinate P (Px, Py) in the longitude and latitude coordinate system spherical co-ordinate (θ, Ф):

$\left\{\begin{array}{c}\mathrm{\θ}=H(\mathrm{Px},\mathrm{Py})\\ \mathrm{\Φ}=G(\mathrm{Px},\mathrm{Py})\end{array}\right.---(1-2)$

Wherein, and θ=H (Px, Py) and Ф=G (Px, Py) these two functions need be installed in the concrete position of car according to camera, all around four direction is all different, and setting height(from bottom) and optical axis are all different, and need set up corresponding mapping relations according to concrete position is H and G function.

Described step (3), based on the spherical projection model, fish eye images generates optical principle, obtain spherical co-ordinate (θ, coordinate R ' Ф) and in the flake original image (I, transformation relation J):

$\left\{\begin{array}{c}I=f*\sin \left(\mathrm{\θ}\right)\cos \left(\mathrm{\Φ}\right)\\ J=f*\sin \left(\mathrm{\θ}\right)*\sin \left(\mathrm{\Φ}\right)\end{array}\right.---(1-3)$

Wherein, f is the camera focal length, goes the hemisphere radius r in practice.

Described step (4), image shows that from needs image array R to world coordinate system, arrives sphere longitude and latitude coordinate system again through coordinate system conversion several times, arrive the original image coordinate system R ' that fish eye lens is taken at last, set up a kind of transformational relation at image array R and R ' at last:

$\left\{\begin{array}{c}I=f(i,j)\\ J=g(i,j)\end{array}\right.---(1-4)$

Wherein, function f and function g are the relation function of image from desirable figure to the original image inverse transformation.

Described step (5), from image array R (i, j) image coordinate is to original image R ' (I in, J) transformation results (I, J) I and J integer not necessarily all in, and practice shows that I and J extremely mostly are decimals, in this case, adopt bilinear interpolation, the non-integer pixel point is carried out interpolation arithmetic.

Description of drawings:

Fig. 1 is coordinate mapping relations figure.

Fig. 2 is unit ball and longitude and latitude coordinate system graph of a relation

Fig. 3 is the synoptic diagram of bilinear interpolation computing

Fig. 4 is preceding camera scheme of installation

Fig. 5 is the place ahead camera and the projection relation that needs to extract the zone

Embodiment

One, follows according to the camera installation site, with car sound geometric parameter, need to determine the scope of monitoring in the world coordinate system the inside, former vehicle front is example, as shown in Figure 4, need in pilothouse, show the long m rice in the place ahead by display, wide n rice, (vehicle heading is the long limit of rectangle) is with the information according to resolution of displayed images inside display, then to showing that coordinate system carries out conversion between image and the real world images, suc as formula (1-1).

Two, obtain real-world coordinates P (Px, Py) afterwards, the location parameter of installing onboard according to camera, camera height h, the angle α camera optical axis and ground, set up how much contacts at fish eye lens hemisphere face and shooting coordinate system, in sphere longitude and latitude mapping point system, determine P point coordinate (θ in the sphere longitude and latitude, Ф), formula (1-2) is introduced H and the G function represents transformation relation between the two, just is installed in car the place ahead camera below and is example derive H and G function, as Fig. 5, shown in, XYZ is world coordinate system, and rectangle ABCD is the zone that needs demonstration, and MG both z axle is the fish eye lens optical axis, coordinate system xGy is the parallel surface on fish eye lens internal image plane, ∠ GMP is the corresponding latitude Ф of P point among the △ GPM as seen from the figure, and the P point carries out projection with the direction of parallel z axle in the xGy of plane, and subpoint and G point line and x axle clamp angle are exactly longitude and latitude θ, can shift out H and G function onto according to geometric relationship, as follows:

$\mathrm{\Φ}=\arccos \left(\frac{{\left(\frac{h}{\sin \mathrm{\α}}\right)}^2+[h^2+{\left(\mathrm{Px}\right)}^2+{(\mathrm{Py}+\frac{h}{\tan \mathrm{\α}})}^2]-{\left(\mathrm{Px}\right)}^2-\left(\mathrm{Py}\right)^2}{2\left(\frac{h}{\sin \mathrm{\α}}\right)\sqrt{h^2+{\left(\mathrm{Px}\right)}^2+{(\mathrm{Py}+\frac{h}{\tan \mathrm{\α}})}^2}}\right)---(1-5)$

$\mathrm{\&theta;}=\left\{\begin{array}{cc}\mathrm{\&pi;}/2+\mathrm{ar}\cos \left(\frac{\mathrm{Px}}{\sin \mathrm{\&Phi;}\sqrt{h^2+{\left(\mathrm{Px}\right)}^2+{(\mathrm{Py}+\frac{h}{\tan \mathrm{\&alpha;}})}^2}}\right)& (\mathrm{Py}<=0)\\ \mathrm{\&pi;}/2-\mathrm{ar}\cos \left(\frac{\mathrm{Px}}{\sin \mathrm{\&Phi;}\sqrt{h^2+{\left(\mathrm{Px}\right)}^2+{(\mathrm{Py}+\frac{h}{\tan \mathrm{\&alpha;}})}^2}}\right)& (\mathrm{Px}>0)\end{array}\right.---(1-6)$

Wherein, formula (1-5) is the G function, formula (1-6) is the H function, and other three directions also can obtain identical corresponding relation according to identical principle, though the more complicated though this conversion computing is got up, but computing is carried out above the pc machine more like this, generates a kind of mapping table at last, and mapping table is stored in the middle of the panorama display system, thereby reduced the operand of system CPU, improved the real-time of display system.

Three, utilize the spherical projection model, find spherical co-ordinate (θ, Ф) in the flake original image position coordinates R ' (I, J), formula (1-3) is exactly corresponding relation between two coordinates of flake.

Four, from image array R (i, j) image coordinate is to original image R ' (I in, J) transformation results (I, J) I and J integer not necessarily all in, and practice shows that I and J extremely mostly are decimals, in this case, adopt bilinear interpolation, the non-integer pixel point is carried out interpolation arithmetic, as shown in Figure 3, is the principle of bilinear interpolation algorithm.Suppose pixel R ' (m, n), the m fraction part is a=m-i, the n fraction part is b=n-j.

Then R ' (m, n)=(1-a) * (1-b) * R ' (i, j)+a* (1-b) * R ' (i+1, j)+(1-a) * b*R ' (I, j+1)+a*b*R ' (i+1, j+1).

Claims (7)

1. vehicle panorama display system correcting fisheye image method based on spherical projection model and inverse transformation model, it is characterized in that following, comprise following steps: (1), the position of installing according to camera, determine the vehicle-periphery range of information a that the driver need see, and in world coordinate system and ideal image coordinate system, a is carried out coordinate setting; (2), the location parameter installed onboard according to camera, camera height h, the angle α camera optical axis and ground sets up how much contacts at fish eye lens hemisphere face and shooting coordinate system, determines the position coordinates b of a in sphere longitude and latitude mapping point system; (3), according to fish eye lens spherical projection model, determine in the sphere longitude and latitude mapping point that position coordinates b takes position coordinates c in the distorted image at fish eye lens; (4), basis is set up from a to c coordinate one-to-one relationship against projective transformation at last; (5), utilize the bilinear interpolation computing, carry out interpolation arithmetic to obtaining among the c non-integer coordinates, obtain a complete general view clearly.

2. according to the described vehicle panorama display system correcting fisheye image method based on spherical projection model and inverse transformation model of claim 1, it is characterized in that, described step (1), conversion mutually between ideal image coordinate system and the world coordinate system, indication range a is fore-and-aft distance m rice in the world coordinate system, lateral separation n rice; Conversion generated ideal figure M*N, correspondence image matrix R, each pixel in the vertical ratio of corresponding actual range is: m/M, grid scale: n/N, pixel R among the image array R (i, j) respective coordinates:

。

3. according to the described vehicle panorama display system correcting fisheye image method based on spherical projection model and inverse transformation model of claim 1, it is characterized in that, described step (2), according to the camera installation parameter, in sphere longitude and latitude coordinate system, determine viewing area a, in coordinate P (Px, Py) in the longitude and latitude coordinate system spherical co-ordinate (θ, Ф):

。

4. according to the described vehicle panorama display system correcting fisheye image method based on spherical projection model and inverse transformation model of claim 3, it is characterized in that, θ=H (Px, Py) and Ф=G (Px, Py) these two functions need be installed in the concrete position of car according to camera, all around four direction is all different, and setting height(from bottom) and optical axis are all different, and need set up corresponding mapping relations according to concrete position is H and G function.

5. according to the described vehicle panorama display system correcting fisheye image method based on spherical projection model and inverse transformation model of claim 1, it is characterized in that, described step (3), based on the spherical projection model, fish eye images generates optical principle, obtain spherical co-ordinate (θ, Ф) and the flake original image in coordinate R ' (I, transformation relation J):

Wherein, f is the camera focal length, goes the hemisphere radius r in practice.

6. according to the described vehicle panorama display system correcting fisheye image method based on spherical projection model and inverse transformation model of claim 1, it is characterized in that, described step (4), image is through coordinate system conversion several times, show that from needs image array R is to world coordinate system, arrive sphere longitude and latitude coordinate system again, arrive the original image coordinate system R ' that fish eye lens is taken at last, set up a kind of transformational relation at image array R and R ' at last:

Wherein, function f and function g are the relation function of image from desirable figure to the original image inverse transformation.

7. according to the described vehicle panorama display system correcting fisheye image method based on spherical projection model and inverse transformation model of claim 1, it is characterized in that, described step (5), from image array R (i, j) in image coordinate to original image R ' (I, J) transformation results (I, J) I and J integer not necessarily all in, and practice shows that I and J extremely mostly are decimals, in this case, adopt bilinear interpolation, the non-integer pixel point is carried out interpolation arithmetic.

Images