CN106060427B - Method for panoramic imaging and its device based on single camera - Google Patents
Method for panoramic imaging and its device based on single camera Download PDFInfo
- Publication number
- CN106060427B CN106060427B CN201610718973.8A CN201610718973A CN106060427B CN 106060427 B CN106060427 B CN 106060427B CN 201610718973 A CN201610718973 A CN 201610718973A CN 106060427 B CN106060427 B CN 106060427B
- Authority
- CN
- China
- Prior art keywords
- image
- module
- correction
- coordinate
- diameter
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 238000000034 method Methods 0.000 title claims abstract description 46
- 238000003384 imaging method Methods 0.000 title claims abstract description 20
- 238000012937 correction Methods 0.000 claims abstract description 114
- 238000006243 chemical reaction Methods 0.000 claims abstract description 16
- 238000012545 processing Methods 0.000 claims abstract description 16
- 235000004522 Pentaglottis sempervirens Nutrition 0.000 claims description 27
- 238000006073 displacement reaction Methods 0.000 claims description 22
- 239000011159 matrix material Substances 0.000 claims description 15
- 238000005259 measurement Methods 0.000 claims description 5
- 230000008901 benefit Effects 0.000 abstract description 5
- 238000013507 mapping Methods 0.000 description 6
- 238000005516 engineering process Methods 0.000 description 4
- 238000009434 installation Methods 0.000 description 4
- 230000000007 visual effect Effects 0.000 description 4
- 238000010586 diagram Methods 0.000 description 2
- 230000004927 fusion Effects 0.000 description 2
- 230000003287 optical effect Effects 0.000 description 2
- 206010039203 Road traffic accident Diseases 0.000 description 1
- 238000010276 construction Methods 0.000 description 1
- 230000001419 dependent effect Effects 0.000 description 1
- 230000000694 effects Effects 0.000 description 1
- 230000005484 gravity Effects 0.000 description 1
- 239000004973 liquid crystal related substance Substances 0.000 description 1
- 230000001105 regulatory effect Effects 0.000 description 1
- 238000005070 sampling Methods 0.000 description 1
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N25/00—Circuitry of solid-state image sensors [SSIS]; Control thereof
- H04N25/40—Extracting pixel data from image sensors by controlling scanning circuits, e.g. by modifying the number of pixels sampled or to be sampled
- H04N25/41—Extracting pixel data from a plurality of image sensors simultaneously picking up an image, e.g. for increasing the field of view by combining the outputs of a plurality of sensors
Landscapes
- Engineering & Computer Science (AREA)
- Multimedia (AREA)
- Signal Processing (AREA)
- Closed-Circuit Television Systems (AREA)
- Image Processing (AREA)
Abstract
A kind of method for panoramic imaging and its device based on single camera, its device includes the camera being successively electrically connected, distortion correction module, image mosaic update module, image conversion module and image processing module, and memory module and image mosaic update module are electrically connected.Compared with prior art, the present invention the present invention is only with a camera, and the prior art uses 4 cameras；When memory module receives new global image, memory module will use the new and old global image of new global image, and renewal speed is fast, reduces memory space, improves operational efficiency, can not only show that vehicle-surroundings image can also show image below vehicle.So that reversing process is distincter, driver's operation is more convenient, it is safer, simultaneously, it further reduced difficulty when driver's reversing, so that panorama reversing image precision it is higher, have many advantages, such as structure is simple, at low cost, interference is small, error is small, it is easy to use, improve move backward safety coefficient.
Description
Technical field
The present invention relates to vehicle-mounted parking assist system technical field, more particularly, to it is a kind of based on the panorama of single camera at
As method and device thereof.
Background technique
In automobile assistant driving field, in order to facilitate driver's reversing, the method generally used has car radar with after
Depending on camera, but these technologies can only all be realized and show backsight image when reversing automobile, cannot achieve 360 degree of full-view images
Demand.It is gradually popularized recently as full-view image technology, in order to realize 360 degree of full-view image reversings, people are in rear-camera
Increase multi-path camera, such as forward sight camera, left view camera, right view camera and rear-camera head in head parking system,
And by image mosaic technology, panorama reverse image is generated, so as to dead zone-eliminating, motor-vehicle accident is reduced, is greatly promoted
Traffic safety.Existing multi-cam full-view image technology has the following disadvantages: 4 cameras are used, structure is complicated,
At high cost, difficult to install, route is various to interfere greatly complete machine.
Summary of the invention
In order to overcome the above problem, the present invention provides a kind of method for panoramic imaging and its dress based on single camera to society
It sets, structure is simple, at low cost, easy for installation and interference is small.
A kind of technical solution of the invention is: providing a kind of method for panoramic imaging based on single camera, including walks as follows
It is rapid:
100. obtaining the image of rear of vehicle using single camera；
The image of 200. pairs of acquisitions carries out distortion correction and is converted into correction image；
The old global image of storage and the correction image are spliced and are formed new global image by 300., with it is described newly
Global image updates the old global image；
400. are converted into the new global image to get a bird's eye view image；
500. according to physical location by auto model image co-registration to it is described get a bird's eye view in image and formed panorama reversing image.
Further include step 600 as improvement of the present invention, shows the panorama reversing image.
As improvement of the present invention, in the step 200, barrel-type distortion correction, barrel shape are carried out to the image of acquisition
Distortion correction formula is as follows:
X=x'(1+k1x'2+k2y'2)
Y=y'(1+k1x'2+k2y′2) wherein, k1, k2For correction parameter, k1=-0.0000001, k2=-0.0000012,
k1Adjust the distortion in y-axis direction, k2The distortion of x-axis direction is adjusted, x ', y ' are the seat for needing distortion correction in the image obtained
Punctuate, the coordinate points in correction image described in x, y.
As improvement of the present invention, selection is located at the left diameter and right wing diameter of two sides in the correction image, if
Left diameter, right wing diameter displacement as, then prove vehicle reversing when do not turn；If the displacement of left diameter, right wing diameter is different
Sample then proves vehicle turning in reversing, according to the distance between left diameter, the displacement difference of right wing diameter and left diameter, right wing diameter
Rotation angle is calculated, the correction image is rotated according to the rotation angle.
As improvement of the present invention, the displacement of left diameter or right wing diameter is calculated using SSDA analogue method, formula is as follows:
The multiple rotary of template is completed using bilinear interpolation, formula is as follows:
f(x,y)=f00+x(f10-f00)+y(f01-f00)+xy(f00+f10+f11-f01)；
The prediction and calculating of backing track angle are carried out using double-template angular measurement, formula is as follows:
θ=arctan (newposy2-newposy1)/(xpos2-xpos1)), wherein θ is backing track angle, and x, y divide
It Wei not the coordinate points that corrected in image；x0、y0For the value of coordinate origin, f00For the corresponding rgb value of coordinate (0,0), f10For
The corresponding rgb value of coordinate (1,0), f01For the corresponding rgb value of coordinate (0,1), f11For the corresponding rgb value of coordinate (1,1),
(xpos1, newposy1) is the coordinate position for remembering the maximum target of frame medium contrast, and (xpos2, newposy2) is in present frame
The maximum target position of contrast.
As improvement of the present invention, in the step 300, the renewal speed of the new global image is greater than or equal to
The back-up speed of vehicle.
As improvement of the present invention, in the step 400, perspective matrix is obtained according to calibration point algorithm, has an X-rayed square
Battle array are as follows:
Wherein, a33It is 1, a11=1.1605, a21=1.3604, a31=-78.7857, a12=0, a22=2.2578, a32
=-19.5210, a13=0, a23=0.004；
According to perspective matrix projection algorithm, input picture is multiplied with specified perspective matrix, obtains following formula:
Wherein, u, v are the x-axis and y axial coordinate of the point in the new global image, x1、y1It is the corresponding birds-eye view
Coordinate as in, x=a11u+a21v+a31, y=a12u+a22v+a32, w=a13u+a23v+a33。
Another technical solution of the invention is: providing a kind of panoramic imaging device based on single camera, comprising:
Camera, the camera are used to obtain the image of rear of vehicle；
Distortion correction module, the distortion correction module carry out distortion correction to the image of acquisition and are converted into correction chart
Picture；
Image mosaic update module and memory module, described image splices update module will be old complete in the memory module
Office's image and the correction image are spliced and are formed new global image, update the memory module with the new global image
In the old global image；
Image conversion module, described image conversion module are converted into the new global image to get a bird's eye view image；
Image processing module, described image processing module get a bird's eye view auto model image co-registration according to physical location described in
In image and form panorama reversing image；
The camera, the distortion correction module, described image splicing update module, described image conversion module and institute
It states image processing module to be successively electrically connected, the memory module and described image splicing update module are electrically connected.
It further include display module as improvement of the present invention, described image processing module and the display module are electrical
Connection, the display module show the panorama reversing image.
It further include rotation correction module as improvement of the present invention, the distortion correction module passes through the rotation school
Positive module and described image splicing update module are electrically connected；Vehicle is not turned in reversing, and the rotation correction module is by institute
It states correction image and is transferred to described image splicing update module, vehicle is turned in reversing, and the rotation correction module will be described
Correction image is transferred to the rotation correction module, and the rotation correction module rotates it and is transferred to described image spelling
Connect update module.
The present invention is due to using distortion correction module, image mosaic update module, image conversion module and image procossing
Module, compared with prior art, the present invention is only with a camera, and the prior art uses 4 cameras；When storage mould
When block receives new global image, memory module will use the new and old global image of new global image, and renewal speed is fast, reduces
Memory space improves operational efficiency, can not only show that vehicle-surroundings image can also show image below vehicle.
In the process of work, the image that camera obtains first is carried out distortion correction and is converted into correction chart by the present invention
Picture prevents error, to cause unsafe accident this is done to keep correction image truer.Correction is completed
Afterwards, old global image and correction image are spliced and forms new global image, then be converted into getting a bird's eye view by new global image
Image finally just forms panorama reversing image, so that reversing process is distincter, driver's operation is more convenient, more pacifies
Entirely, it meanwhile, further reduced difficulty when driver's reversing, so that the precision of panorama reversing image is higher, there is structure
Simply, at low cost, interference is small, error is small, it is easy to use, improve reversing safety coefficient the advantages that.
Detailed description of the invention
Fig. 1 is the flow diagram of the method for panoramic imaging in the present invention based on single camera.
Fig. 2 is the schematic illustration that postrotational correction image and old global image are spliced in Fig. 1.
Fig. 3 is that new global image is converted into getting a bird's eye view the schematic illustration of image in Fig. 1.
Fig. 4 is the principle block diagram of the panoramic imaging device in the present invention based on single camera.
Wherein: 1. cameras；2. distortion correction module；3. image mosaic update module；4. image conversion module；5. image
Processing module；6. display module；7. rotation correction module；8. memory module.
Specific embodiment
In the description of the present invention, it is to be understood that, "center" in term, "upper", "lower", "front", "rear", " left side ",
The orientation or positional relationship of instructions such as " right sides " is to be based on the orientation or positional relationship shown in the drawings, and is merely for convenience of describing this hair
Bright and simplified description, rather than the device or element of indication or suggestion meaning must have a particular orientation, with specific orientation
Construction and operation, therefore be not considered as limiting the invention.In addition, term " first ", " second " are used for description purposes only,
It is not understood to indicate or imply relative importance.
In the description of the present invention, it should be noted that unless otherwise clearly defined and limited, term " installation " " connects
Connect ", " connected " shall be understood in a broad sense, for example, it may be being fixedly connected, be also possible to dismantling connection, or be integrally connected；It can be with
It is mechanical connection, is also possible to be electrically connected；It can be directly connected, be also possible to indirectly connected through an intermediary, can be
Connection inside two elements.For the ordinary skill in the art, above-mentioned term can be understood at this with concrete condition
The concrete meaning of invention.
Referring to Figure 1, it is a kind of method for panoramic imaging based on single camera that Fig. 1 is revealed, also refers to Fig. 2 and figure
3, Fig. 2 it is revealed be postrotational correction image and the schematic illustration spliced of old global image, Fig. 3 is revealed to be
New global image is converted into getting a bird's eye view the schematic illustration of image.The method for panoramic imaging based on single camera, including it is as follows
Step:
100. obtaining the image of rear of vehicle using single camera；
The image of 200. pairs of acquisitions carries out distortion correction and is converted into correction image；
The old global image of storage and the correction image are spliced and are formed new global image by 300., with it is described newly
Global image updates the old global image；
400. are converted into the new global image to get a bird's eye view image；
500. according to physical location by auto model image co-registration to it is described get a bird's eye view in image and formed panorama reversing image；
The 600. displays panoramas reversing image.
In this method, due to using a camera, and the camera generally uses wide-angle lens, depending on
Angle is being more than or equal to 130 degree to less than or equal between 180 degree.During use, visual angle, which is more than 100 degree, will make shooting
Image generates barrel-type distortion.So-called barrel-type distortion refers to: uniform grid placed in object plane, it is illuminated as object, if
Diaphragm is placed between object and lens, it can be seen that the magnifying power far from optical axis region is than low near optical axis, in as plane
There is evagination scene shown in figure.
In the step 200 of this method, barrel-type distortion correction, barrel-type distortion updating formula are carried out to the image of acquisition
It is as follows:
X=x ' (1+k1x’2+k2y’2)
Y=y'(1+k1x′2+k2y′2) wherein, k1, k2For correction parameter, k1=-0.0000001, k2=-0.0000012,
k1Adjust the distortion in y-axis direction, k2The distortion of x-axis direction is adjusted, x ', y ' are the seat for needing distortion correction in the image obtained
Punctuate, x, y are the coordinate points in the correction image.
In the step 200 of this method, selection is located at the left diameter and right wing diameter of two sides in the correction image,
If as the displacement of left diameter, right wing diameter, proving that vehicle is not turned in reversing；If the displacement of left diameter, right wing diameter
Different, i.e. the displacement of left diameter and right wing diameter is variant, then vehicle turning in reversing is proved, according to left diameter, right wing diameter
Displacement difference and the distance between left diameter, right wing diameter calculate rotation angle, according to the rotation angle rotation correction chart
Picture.That is, need to judge whether vehicle turns in the process of backing up, if vehicle is turned in the process of backing up,
It then needs to rotate the correction image according to the angle of turning；When vehicle is not turned in the process of backing up, then it is not required to
The correction image is rotated according to the angle of turning.
In this method, the displacement of left diameter or right wing diameter is calculated using SSDA analogue method, formula is as follows:
The multiple rotary of template is completed using bilinear interpolation, formula is as follows:
f(x,y)=f00+x(f10-f00)+y(f01-f00)+xy(f00+f10+f11-f01)；
The prediction and calculating of backing track angle (angle of reversing turning), formula are carried out using double-template angular measurement
It is as follows:
θ=arctan (newposy2-newposy1)/(xpos2-xpos1)), wherein θ is the (reversing of backing track angle
The angle of turning), x, y are respectively the coordinate points in the correction image.x0、y0For the value of coordinate origin, f00For coordinate (0,0)
Corresponding rgb value, f10For the corresponding rgb value of coordinate (1,0), f01For the corresponding rgb value of coordinate (0,1), f11For coordinate (1,1)
Corresponding rgb value, (xpos1, newposy1) be remember the maximum target of frame medium contrast coordinate position, (xpos2,
It newposy2) is the maximum target position of present frame medium contrast.
In the step 300 of this method, the renewal speed of the new global image is greater than or equal to the reversing of vehicle
Speed, that is to say, that the renewal speed of the new global image can be adjusted according to the back-up speed of vehicle, in this way can be with
Prevent it is updated slow due to the new global image, caused by unsafe accident.The field range of the new global image
The field range of partial visual field range and whole correction images including the old global image, about the old overall situation
The splicing of image and the correction image is the prior art, i.e. the splicing of image, herein no longer to the splicing skill of image
Art is explained.Herein it is emphasized that the new global image includes the partial visual field model of the old global image
It encloses, this is because the distance of reversing is longer during reversing, it is meant that the quantity for the image that the camera obtains is got over
It is more, then the quantity of the correction image is also more, if do not cut to the field range of the old global image, that
The field range of the old global image will be very wide, not only wastes resource in this way, it is also possible to reduce the new overall situation
The splicing speed of image.
When original state, the old global image is empty image, after camera work, is clapped the camera
First image taken the photograph carries out distortion correction, and as the old global image of first Zhang Suoshu.After normal work (reversing), to institute
State camera shooting present image carry out distortion correction, and by after distortion correction presently described correction image and a upper institute
The partial visual field range for stating old global image carries out being spliced to form the new global image.
In the step 400 of this method, perspective matrix, perspective matrix are obtained according to calibration point algorithm are as follows:
Wherein a33It is 1, a11=1.1605, a21=1.3604, a31=-78.7857, a12=0, a22=2.2578, a32=-
19.5210,a13=0, a23=0.004；
According to perspective matrix projection algorithm, input picture is multiplied with specified perspective matrix, obtains following formula:
Equation one,
Equation two,
Wherein, u, v are the x-axis and y-axis coordinate of the point in the new global image, X1、y1It is the corresponding birds-eye view
Coordinate as in, x=a11u+a21v+a31, y=a12u+a22v+a32, w=a13u+a23v+a33。
About a11To a32Sampling process be: firstly, in the new global image choose 4 points, according to mapping close
System, correspondingly chooses 4 mapping points in described get a bird's eye view in image.Then, by the coordinate of 4 points in the new global image
It is brought into equation one and equation two with the coordinate for getting a bird's eye view 4 mapping points in image, 8 solution of equation, 8 parameter finally obtains a11Extremely
a32Value.
In order to more easily obtain a11To a32Value, the new global image of selection is isosceles trapezoid, then described
Getting a bird's eye view image is then rectangle, that is to say, that by becoming rectangle after mapping, i.e. coordinate points in isosceles trapezoid pass through isosceles trapezoid
The coordinate points in rectangle have been reformed into after the conversion of equation one and equation two.4 points are chosen in the new global image, this 4
A point is 4 vertex of isosceles trapezoid, and 4 mapping points are chosen in rectangle, this 4 mapping points are 4 vertex of rectangle, and will
It is brought into equation one and equation two, and 8 solution of equation, 8 parameter is finally easy to obtain a11To a32Value, then a11=1.1605, a21
=1.3604, a31=-78.7857, a21=0, a22=2.2578, a32=-19.5210, a13=0, a23=0.004.
In the step 500 of this method, method that the auto model image is shown by Weighted Fusion with it is described
Get a bird's eye view image co-registration, that is to say, that the auto model image be it is transparent, the benefit designed in this way is, when there is one on the road
Well cover for sewer, after reversing automobile, well lid has been more than camera shooting head region and has been located at body bottom, but can still be seen positioned at vehicle
Well lid under body.Concrete methods of realizing is: by the value of the auto model image multiplied by 0.6, the original value for getting a bird's eye view image
Multiplied by 0.4, the auto model image and the image addition of getting a bird's eye view then are obtained into new pixel value, so that it may realize two
Image co-registration obtains panorama reversing image, and the coefficient of which image is big or closer to 1, the specific gravity of which image with regard to big,
But the coefficient of two images and it is necessary for 1.
In the step 500 of this method, be according to auto model image and it is described get a bird's eye view image scaled, by vehicle mould
The panorama reversing image is got a bird's eye view in image and formed to type image co-registration to described, after forming the panorama reversing image, also
The image that needs to move backward to the panorama is cut and adjusts the scale of the panorama reversing image, and the panorama made falls
Vehicle image meets the standard of display.
Fig. 4 is referred to, Fig. 4 revealed is a kind of panoramic imaging device based on single camera 1, comprising:
Camera 1, the camera 1 are used to obtain the image of rear of vehicle；
Distortion correction module 2, the image of 2 pairs of distortion correction module acquisitions carry out distortion correction and are converted into correction chart
Picture；
Image mosaic update module 3 and memory module 8, described image splices update module 3 will be in the memory module 8
Old global image and the correction image are spliced and are formed new global image, update the storage with the new global image
The old global image in module 8；
Described image conversion module 4, described image conversion module 4 are converted into the new global image to get a bird's eye view image；
Image processing module 5, described image processing module 5 is according to physical location by auto model image co-registration to the bird
It looks down from a height in image and forms panorama reversing image；
The camera 1, the distortion correction module 2, described image splice update module 3, described image conversion module 4
It is successively electrically connected with described image processing module 5, the memory module 8 is electrically connected with described image splicing update module 3.
It further include display module 6 in the present embodiment, described image processing module 5 and the display module 6 are electrically connected,
The display module 6 shows the panorama reversing image, and the display module 6 is liquid crystal display or LED display.
In the present embodiment, the camera 1 and horizontal plane are at 70 degree of installation angle.User during installation, with the bird
Image of looking down from a height is that the setting angle of the camera 1 is adjusted in standard, when the new global image is isosceles trapezoid, the bird
When image of looking down from a height is rectangle, then prove that the setting angle of the camera 1 has regulated, setting angle at this time is 70 degree.
In the present embodiment, the correction principle of the distortion correction module 2 is that barrel-type distortion updating formula is as follows:
X=x ' (1+k1x2+k2y’2)
Y=y'(1+k1x'2+k2y′2) wherein, k1, k2For correction parameter, k1=-0.0000001, k2=-0.0000012,
k1Adjust the distortion in y-axis direction, k2The distortion of x-axis direction is adjusted, x ', y ' are the seat for needing distortion correction in the image obtained
Punctuate, x, y are the coordinate points in the correction image.
Timing, input need the relevant parameter of coordinate points corrected, so that it may dependent coordinate point after being corrected
Parameter, to obtain the correction image.
It further include rotation correction module 7 in the present embodiment, the distortion correction module 2 passes through the rotation correction module 7
It is electrically connected with described image splicing update module 3；Vehicle is not turned in reversing, and the rotation correction module 7 is by the school
Positive image is transferred to described image splicing update module 3, and vehicle is turned in reversing, and the rotation correction module 7 is by the school
Positive image is transferred to the rotation correction module 7, and the rotation correction module 7 rotates it and is transferred to described image spelling
Connect update module 3.
The correction principle of the rotation correction module 7 is, in the correction image selection be located at two sides left diameter and
Right wing diameter, if as the displacement of left diameter, right wing diameter, proving that vehicle is not turned in reversing；If left diameter, right wing diameter
Displacement it is different, i.e. the displacement of left diameter and right wing diameter is variant, then prove vehicle reversing when turning, according to left diameter,
The distance between the displacement difference and left diameter of right wing diameter, right wing diameter calculate rotation angle, rotated according to the rotation angle described in
Correct image.That is, needing to judge whether vehicle turns in the process of backing up, if vehicle is in the process of backing up
Turning, then need to rotate the correction image according to the angle of turning；When vehicle is not turned in the process of backing up, then
It does not need to rotate the correction image according to the angle of turning.
The displacement of left diameter or right wing diameter is calculated using SSDA analogue method, formula is as follows:
The multiple rotary of template is completed using bilinear interpolation, formula is as follows:
f(x,y)=f00+x(f10-f00)+y(f01-f00)+xy(f00+f10+f11-f01)；
The prediction and calculating of backing track angle (angle of reversing turning), formula are carried out using double-template angular measurement
It is as follows:
θ=arctan (newposy2-newposy1)/(xpos2-xpos1)), wherein θ is the (reversing of backing track angle
The angle of turning), x, y are respectively the coordinate points in the correction image；x0、y0For the value of coordinate origin, f00For coordinate (0,
0) corresponding rgb value, f10For the corresponding rgb value of coordinate (1,0), f01For the corresponding rgb value of coordinate (0,1), f11For coordinate (1,
1) corresponding rgb value, (xpos1, newposy1) be remember the maximum target of frame medium contrast coordinate position, (xpos2,
It newposy2) is the maximum target position of present frame medium contrast.
When the rotation correction module 7 works, the correction image is rotated according to the angle θ.
In the present embodiment, the renewal speed that described image splices update module 3 is greater than or equal to the back-up speed of vehicle,
That is the renewal speed of described image splicing update module 3 can be adjusted according to the back-up speed of vehicle, it in this way can be with
Prevent from splicing the updated slow of update module 3 due to described image, caused by unsafe accident.Described image splicing updates mould
Block 3 is the prior art, i.e. the splicing of image, and no longer described image splicing update module 3 is explained herein.
In the present embodiment, the working principle of described image conversion module 4 is to obtain perspective matrix according to calibration point algorithm,
Perspective matrix are as follows:
Wherein a33It is 1；a11=1.1605, a21=1.3604, a31=-78.7857, a12=0, a22=2.2578, a32
=-19.5210, a13=0, a23=0.004；
According to perspective matrix projection algorithm, input picture is multiplied with specified perspective matrix, obtains following formula:
Equation one,
Equation two,
Wherein, u, v are the x-axis and y-axis coordinate of the point in the new global image, x1、y1It is the corresponding birds-eye view
Coordinate as in, x=a11u+a21v+a31, y=a12u+a22v+a32, w=a12u+a23v+a33。
When described image conversion module 4 works, it is only necessary to input the point coordinate in the new global image, will obtain institute
The point coordinate got a bird's eye view in image is stated, to get a bird's eye view image described in obtaining.
In the present embodiment, method that described image processing module 5 is shown by Weighted Fusion is by the auto model image
Image co-registration is got a bird's eye view with described, the benefit designed in this way is, when there is a well cover for sewer on the road, after reversing automobile, well lid is super
It has crossed 1 region of camera and has been located at body bottom, but can still see the well lid under vehicle body.Concrete methods of realizing please join
See the content in the above-mentioned method for panoramic imaging based on single camera 1.
The present invention is due to using the distortion correction module 2, described image splicing update module 3, described image conversion
Module 4 and described image processing module 5, compared with prior art, the present invention is only with a camera 1, and existing skill
Art uses 4 cameras 1；When the memory module 8 receives the new global image, the memory module 8 is just
The old global image can be updated with the new global image, renewal speed is fast, reduces memory space, improves operation effect
Rate can not only show that vehicle-surroundings image can also show image below vehicle.
In the process of work, the image that the camera 1 obtains first is carried out distortion correction and is converted into school by the present invention
Positive image prevents error, to cause unsafe accident this is done to keep the correction image truer.
After the completion of correction, the old global image and the correction image are spliced and form the new global image, then will
The new global image gets a bird's eye view image described in being converted into, and the panorama reversing image is finally just formed, so that reversing process is more
Distinctness, driver's operation is more convenient, safer, meanwhile, it further reduced difficulty when driver's reversing, so that
The precision of panorama reversing image is higher, have structure is simple, at low cost, interference is small, error is small, it is easy to use, improve
The advantages that safety coefficient of moving backward.
Claims (8)
1. a kind of method for panoramic imaging based on single camera, which comprises the steps of:
100. obtaining the image of rear of vehicle using single camera；
The image of 200. pairs of acquisitions carries out distortion correction and is converted into correction image；
The old global image of storage and the correction image are spliced and are formed new global image by 300., with the new overall situation
Old global image described in image update；
400. are converted into the new global image to get a bird's eye view image；
500. according to physical location by auto model image co-registration to it is described get a bird's eye view in image and formed panorama reversing image；
Selection is located at the left diameter and right wing diameter of two sides in the correction image, if as the displacement of left diameter, right wing diameter,
Then prove that vehicle is not turned in reversing；If the displacement of left diameter, right wing diameter is different, prove that vehicle turns in reversing
It is curved, rotation angle is calculated according to the distance between left diameter, the displacement difference of right wing diameter and left diameter, right wing diameter, according to the rotation
Gyration rotates the correction image；The displacement of left diameter or right wing diameter is calculated using SSDA analogue method, formula is as follows:
The multiple rotary of template is completed using bilinear interpolation, formula is as follows:
f(x, y)=f00+x(f10-f00)+y(f01-f00)+xy(f00+f10+f11-f01)；
The prediction and calculating of backing track angle are carried out using double-template angular measurement, formula is as follows:
θ=arctan (newposy2-newposy1)/(xpos2-xpos1)), wherein θ is backing track angle, and x, y are respectively
Coordinate points in the correction image；x0、y0For the value of coordinate origin, f00For the corresponding rgb value of coordinate (0,0), f10For coordinate
(1,0) corresponding rgb value, f01For the corresponding rgb value of coordinate (0,1), f11For the corresponding rgb value of coordinate (1,1), (xpos1,
Newposy1) for memory the maximum target of frame medium contrast coordinate position, (xpos2, newposy2) be present frame medium contrast most
Big target position.
2. the method for panoramic imaging according to claim 1 based on single camera, which is characterized in that it further include step 600,
Show the panorama reversing image.
3. the method for panoramic imaging according to claim 1 or 2 based on single camera, which is characterized in that in the step
In 200, barrel-type distortion correction is carried out to the image of acquisition, barrel-type distortion updating formula is as follows:
X=x ' (1+k1x′2+k2y′2)
Y=y ' (1+k1x′2+k2y′2)；Wherein, k1、k2For correction parameter, k1=-0.0000001, k2=-0.0000012, k1It adjusts
The distortion in whole y-axis direction, k2The distortion of x-axis direction is adjusted, x ', y ' are the coordinate for needing distortion correction in the image obtained
Point, x, y are the coordinate points in the correction image.
4. the method for panoramic imaging according to claim 1 or 2 based on single camera, which is characterized in that in the step
In 300, the renewal speed of the new global image is greater than or equal to the back-up speed of vehicle.
5. the method for panoramic imaging according to claim 1 or 2 based on single camera, which is characterized in that in the step
In 400, perspective matrix, perspective matrix are obtained according to calibration point algorithm are as follows:
Wherein a33It is 1, a11=1.1605, a21=1.3604, a31=-78.7857, a12=0, a22=2.2578, a32=-
19.5210 a13=0, a23=0.004；
According to perspective matrix projection algorithm, input picture is multiplied with specified perspective matrix, obtains following formula:
Wherein, u, v are the x-axis and y-axis coordinate of the point in the new global image, x1、y1It is corresponding described to get a bird's eye view image
In coordinate,
X=a11u+a21v+a31, y=a12u+a22v+a32, w=a13u+a23v+a33。
6. a kind of panoramic imaging device based on single camera characterized by comprising
Camera, the camera are used to obtain the image of rear of vehicle；
Distortion correction module, the distortion correction module carry out distortion correction to the image of acquisition and are converted into correction image；
Image mosaic update module and memory module, described image splice update module for the old global figure in the memory module
As being spliced and being formed new global image with the correction image, updated in the memory module with the new global image
The old global image；
Image conversion module, described image conversion module are converted into the new global image to get a bird's eye view image；
Auto model image co-registration is got a bird's eye view image according to physical location by image processing module, described image processing module described in
In and formed panorama reversing image；
The camera, the distortion correction module, described image splicing update module, described image conversion module and the figure
As processing module is successively electrically connected, the memory module and described image splicing update module are electrically connected；
Selection is located at the left diameter and right wing diameter of two sides in the correction image, if as the displacement of left diameter, right wing diameter,
Then prove that vehicle is not turned in reversing；If the displacement of left diameter, right wing diameter is different, prove that vehicle turns in reversing
It is curved, rotation angle is calculated according to the distance between left diameter, the displacement difference of right wing diameter and left diameter, right wing diameter, according to the rotation
Gyration rotates the correction image；The displacement of left diameter or right wing diameter is calculated using SSDA analogue method, formula is as follows:
The multiple rotary of template is completed using bilinear interpolation, formula is as follows:
f(x, y)=f00+x(f10-f00)+y(f01-f00)+xy(f00+f10+f11-f01)；
The prediction and calculating of backing track angle are carried out using double-template angular measurement, formula is as follows:
θ=arctan (newposy2-newposy1)/(xpos2-xpos1)), wherein θ is backing track angle, and x, y are respectively
Coordinate points in the correction image；x0、y0For the value of coordinate origin, f00For the corresponding rgb value of coordinate (0,0), f10For coordinate
(1,0) corresponding rgb value, f01For the corresponding rgb value of coordinate (0,1), f11For the corresponding rgb value of coordinate (1,1), (xpos1,
Newposy1) for memory the maximum target of frame medium contrast coordinate position, (xpos2, newposy2) be present frame medium contrast most
Big target position.
7. the panoramic imaging device according to claim 6 based on single camera, it is characterised in that: further include display mould
Block, described image processing module and the display module are electrically connected, and the display module shows the panorama reversing image.
8. the panoramic imaging device according to claim 6 or 7 based on single camera, it is characterised in that: further include rotation
Correction module, the distortion correction module are spliced update module by the rotation correction module and described image and are electrically connected；
Vehicle is not turned in reversing, and the correction image is transferred to described image splicing update module by the rotation correction module,
Vehicle is turned in reversing, and the correction image is transferred to the rotation correction module, the rotation by the rotation correction module
Turn correction module to rotate it and be transferred to described image splicing update module.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201610718973.8A CN106060427B (en) | 2016-08-25 | 2016-08-25 | Method for panoramic imaging and its device based on single camera |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201610718973.8A CN106060427B (en) | 2016-08-25 | 2016-08-25 | Method for panoramic imaging and its device based on single camera |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN106060427A CN106060427A (en) | 2016-10-26 |
| CN106060427B true CN106060427B (en) | 2019-05-21 |
Family
ID=57195548
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201610718973.8A Active CN106060427B (en) | 2016-08-25 | 2016-08-25 | Method for panoramic imaging and its device based on single camera |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN106060427B (en) |
Families Citing this family (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN107341787A (en) * | 2017-07-26 | 2017-11-10 | 珠海研果科技有限公司 | Method, apparatus, server and the automobile that monocular panorama is parked |
| CN107507131B (en) * | 2017-08-08 | 2021-02-23 | 广州市安晓科技有限责任公司 | 360-degree panoramic reverse image generation method based on single camera |
| CN109102464A (en) * | 2018-08-14 | 2018-12-28 | 四川易为智行科技有限公司 | panoramic image splicing method and device |
| CN109131082B (en) * | 2018-08-31 | 2022-03-15 | 深圳以恒科技有限公司 | Monocular panoramic parking image system completely based on vision and parking method thereof |
| CN109249859A (en) * | 2018-09-21 | 2019-01-22 | 奇瑞新能源汽车技术有限公司 | A kind of automobile single camera projection panorama display system and electric car |
| CN109389060B (en) * | 2018-09-26 | 2021-03-02 | 福州大学 | Vision-based vehicle surrounding collision early warning method |
| CN113221833B (en) * | 2021-06-01 | 2023-05-26 | 深圳市天双科技有限公司 | Method for realizing panoramic parking by rearview camera |
Family Cites Families (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102059978B (en) * | 2009-11-16 | 2014-09-10 | 财团法人工业技术研究院 | Assisted method and system for driving |
| WO2011087354A2 (en) * | 2010-01-15 | 2011-07-21 | Mimos Berhad | Vehicle parking and obstacle avoidance system using a single panoramic camera |
| CN103763517B (en) * | 2014-03-03 | 2017-02-15 | 惠州华阳通用电子有限公司 | Vehicle-mounted around view display method and system |
| US10525883B2 (en) * | 2014-06-13 | 2020-01-07 | Magna Electronics Inc. | Vehicle vision system with panoramic view |
| CN105825475B (en) * | 2016-04-01 | 2019-01-29 | 西安电子科技大学 | 360 degree of full-view image generation methods based on single camera |
| CN105763854B (en) * | 2016-04-18 | 2019-01-08 | 扬州航盛科技有限公司 | A kind of omnidirectional imaging system and its imaging method based on monocular cam |
-
2016
- 2016-08-25 CN CN201610718973.8A patent/CN106060427B/en active Active
Also Published As
| Publication number | Publication date |
|---|---|
| CN106060427A (en) | 2016-10-26 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN106060427B (en) | Method for panoramic imaging and its device based on single camera | |
| CN109741455B (en) | Vehicle-mounted stereoscopic panoramic display method, computer readable storage medium and system | |
| US9858639B2 (en) | Imaging surface modeling for camera modeling and virtual view synthesis | |
| JP5222597B2 (en) | Image processing apparatus and method, driving support system, and vehicle | |
| US10447948B2 (en) | Imaging system and display system | |
| CN105306805B (en) | Apparatus and method for correcting image distortion of camera for vehicle | |
| JP5455124B2 (en) | Camera posture parameter estimation device | |
| CN102045546B (en) | Panoramic parking assist system | |
| JP2008187566A (en) | Camera calibration apparatus and method and vehicle | |
| EP2061234A1 (en) | Imaging apparatus | |
| CN103871071A (en) | Method for camera external reference calibration for panoramic parking system | |
| WO2022141541A1 (en) | Digital automatic shading method and shading system for vehicle | |
| JP6614754B2 (en) | Method for converting an omnidirectional image from an omnidirectional camera placed on a vehicle into a rectilinear image | |
| EP3326146B1 (en) | Rear cross traffic - quick looks | |
| JP2009120135A (en) | Imaging device | |
| EP3623223A1 (en) | Image pickup system and display system | |
| JP7207038B2 (en) | IMAGING DEVICE, IMAGING OPTICAL SYSTEM AND MOVING OBJECT | |
| KR20190075034A (en) | Imaging Apparatus and method for Automobile | |
| JP2010187161A (en) | On-board camera system and image processing method | |
| JP2008034964A (en) | Image display apparatus | |
| KR20190026507A (en) | Imaging Apparatus and method for Automobile | |
| CN116740192A (en) | Calibration method and calibration system of vehicle-mounted looking-around system and intelligent automobile | |
| JP2009123131A (en) | Imaging apparatus | |
| JP2019026198A (en) | Head-up display device, and driver viewpoint detection method therefor | |
| JP2008230561A (en) | Photographing control device and photometry area adjustment method |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| C06 | Publication | ||
| PB01 | Publication | ||
| C10 | Entry into substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| TA01 | Transfer of patent application right |
Effective date of registration: 20171228 Address after: 518000 Guangdong city of Shenzhen province Baoan District Xin'an Street Community Lake Road Mei Ju Haiyu area C room 803 Applicant after: SHENZHEN ZHILIANCHE NET TECHNOLOGY CO.,LTD. Address before: 516600 Guangdong city of Shenzhen province Baoan District Xin'an Haiyu road street community new Mei Ju A District C Building Room 806 Applicant before: Liu Kaijian |
|
| TA01 | Transfer of patent application right | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant | ||
| CP03 | Change of name, title or address |
Address after: 211800, No. 111-22 Baihe Road, Pukou Economic Development Zone, Pukou District, Nanjing City, Jiangsu Province Patentee after: Nanjing Zhilian Zhixing Technology Co.,Ltd. Country or region after: China Address before: 518000, Room 803, Zone C, Huameiju, Xinhu Road, Haiyu Community, Xin'an Street, Bao'an District, Shenzhen City, Guangdong Province Patentee before: SHENZHEN ZHILIANCHE NET TECHNOLOGY CO.,LTD. Country or region before: China |
|
| CP03 | Change of name, title or address |