CN109447901A - A kind of method for panoramic imaging and device - Google Patents
A kind of method for panoramic imaging and device Download PDFInfo
- Publication number
- CN109447901A CN109447901A CN201811191264.4A CN201811191264A CN109447901A CN 109447901 A CN109447901 A CN 109447901A CN 201811191264 A CN201811191264 A CN 201811191264A CN 109447901 A CN109447901 A CN 109447901A
- Authority
- CN
- China
- Prior art keywords
- camera
- ground
- image data
- birds
- eye view
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
- 238000003384 imaging method Methods 0.000 title claims abstract description 46
- 238000000034 method Methods 0.000 title claims abstract description 32
- 238000012545 processing Methods 0.000 claims description 15
- 238000012937 correction Methods 0.000 claims description 9
- 238000012512 characterization method Methods 0.000 claims description 6
- 238000004364 calculation method Methods 0.000 claims description 5
- 230000004888 barrier function Effects 0.000 claims description 4
- 238000013519 translation Methods 0.000 claims description 4
- 238000005516 engineering process Methods 0.000 description 8
- 230000008859 change Effects 0.000 description 6
- 230000008569 process Effects 0.000 description 5
- 235000004522 Pentaglottis sempervirens Nutrition 0.000 description 3
- 238000010586 diagram Methods 0.000 description 3
- 230000003287 optical effect Effects 0.000 description 3
- 238000009434 installation Methods 0.000 description 2
- 230000004807 localization Effects 0.000 description 2
- 238000013507 mapping Methods 0.000 description 2
- 239000011159 matrix material Substances 0.000 description 2
- 238000006748 scratching Methods 0.000 description 2
- 230000002393 scratching effect Effects 0.000 description 2
- 230000009466 transformation Effects 0.000 description 2
- 238000004458 analytical method Methods 0.000 description 1
- 238000011161 development Methods 0.000 description 1
- 238000006073 displacement reaction Methods 0.000 description 1
- 239000007787 solid Substances 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T3/00—Geometric image transformations in the plane of the image
- G06T3/40—Scaling of whole images or parts thereof, e.g. expanding or contracting
- G06T3/4038—Image mosaicing, e.g. composing plane images from plane sub-images
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T5/00—Image enhancement or restoration
- G06T5/80—Geometric correction
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/30—Determination of transform parameters for the alignment of images, i.e. image registration
- G06T7/33—Determination of transform parameters for the alignment of images, i.e. image registration using feature-based methods
- G06T7/337—Determination of transform parameters for the alignment of images, i.e. image registration using feature-based methods involving reference images or patches
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/10—Image acquisition modality
- G06T2207/10004—Still image; Photographic image
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/30—Subject of image; Context of image processing
- G06T2207/30236—Traffic on road, railway or crossing
Landscapes
- Engineering & Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Theoretical Computer Science (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Image Processing (AREA)
- Closed-Circuit Television Systems (AREA)
Abstract
The present invention provides a kind of method for panoramic imaging and devices, wherein, by the moving distance and rotation angle that calculate camera, determine the position orientation relation on camera and ground, according to the position orientation relation of camera and ground, first correcting image is converted into birds-eye view, and birds-eye view and ground birds-eye view are spliced, obtain panoramic picture, even if camera and the position orientation relation on ground are changed, it can also determine the position orientation relation of camera and ground in real time according to the current frame image data got, to obtain correct panoramic picture, substantially increase the accuracy of the panoramic picture obtained under the conditions of the road conditions of actual complex.
Description
Technical field
The present invention relates to technical field of image processing, in particular to a kind of method for panoramic imaging and device.
Background technique
Currently, panoramic imaging techniques are gradually popularized with the development of image processing techniques, omnidirectional imaging system passes through setting
Multiple camera real-time image acquisitions, and to image carry out splicing generate omnidirectional imaging system surrounding enviroment panorama sketch
Picture.Certainly, omnidirectional imaging system can also acquire image using single camera, and obtain after handling acquired image
Panoramic picture.
In the related technology, using single camera omnidirectional imaging system in, need to keep camera in omnidirectional imaging system,
Under the premise of the positional relationship on vehicle and ground remains unchanged, more accurately panorama sketch is just can be generated in omnidirectional imaging system
Picture.
If the vehicle for installing omnidirectional imaging system is under the conditions of complicated road conditions, so that camera, vehicle and the position on ground
It sets relationship often to change, then above-mentioned omnidirectional imaging system often obtains the panoramic picture of mistake in actual use.
Summary of the invention
To solve the above problems, the embodiment of the present invention is designed to provide a kind of method for panoramic imaging and device.
In a first aspect, the embodiment of the invention provides a kind of method for panoramic imaging, comprising:
Determine the collected current frame image data of camera the first correcting image and the current frame image data it is upper
The matching double points of second correcting image of one frame image data；
The moving distance and rotation angle of the camera are calculated using the matching double points；
Moving distance and rotation angle based on the camera, the ground obstacle obtained in first correcting image are special
Point is levied, and determines the position orientation relation of camera and ground；
According to the position orientation relation of the camera and ground, first correcting image is converted into birds-eye view, and will get a bird's eye view
Figure is spliced with ground birds-eye view, obtains panoramic picture, wherein it is special that the ground obstacle is shown in the panoramic picture
The ground obstacle of sign point characterization.
Second aspect, the embodiment of the invention also provides a kind of panoramic imaging devices, comprising:
Determining module, for determining the first correcting image and the present frame of the collected current frame image data of camera
The matching double points of second correcting image of the previous frame image data of image data；
Computing module, for calculating the moving distance and rotation angle of the camera using the matching double points；
Processing module obtains in first correcting image for moving distance and rotation angle based on the camera
Ground obstacle characteristic point, and determine camera and ground position orientation relation；
First correcting image is converted to bird for the position orientation relation according to the camera and ground by splicing module
It looks down from a height figure, and birds-eye view and ground birds-eye view is spliced, obtain panoramic picture, wherein show in the panoramic picture
State the ground obstacle of ground obstacle characteristic point characterization.
In the scheme that the above-mentioned first aspect of the embodiment of the present invention is provided to second aspect, by the moving distance for calculating camera
With rotation angle, the position orientation relation on camera and ground is determined, according to the position orientation relation of camera and ground, by the first correcting image
Birds-eye view is converted to, and birds-eye view and ground birds-eye view are spliced, obtains panoramic picture, and must be kept in the related technology
Camera, vehicle are compared with the mode of the position relationship on ground in omnidirectional imaging system, even if camera and the pose on ground close
System is changed, and can also determine the position orientation relation of camera and ground in real time according to the current frame image data got, from
And correct panoramic picture is obtained, substantially increase the accuracy of the panoramic picture obtained under the conditions of the road conditions of actual complex.
To enable the above objects, features and advantages of the present invention to be clearer and more comprehensible, preferred embodiment is cited below particularly, and cooperate
Appended attached drawing, is described in detail below.
Detailed description of the invention
In order to more clearly explain the embodiment of the invention or the technical proposal in the existing technology, to embodiment or will show below
There is attached drawing needed in technical description to be briefly described, it should be apparent that, the accompanying drawings in the following description is only this
Some embodiments of invention for those of ordinary skill in the art without creative efforts, can be with
It obtains other drawings based on these drawings.
Fig. 1 is shown can be using a kind of structural schematic diagram of omnidirectional imaging system of the embodiment of the present invention；
Fig. 2 shows a kind of flow charts of method for panoramic imaging provided by the embodiment of the present invention 1；
Fig. 3 shows a kind of structural schematic diagram of panoramic imaging device provided by the embodiment of the present invention 2.
Icon: 100- monocular camera；102- image processing unit；104- image storage unit；106- image-display units；
300- determining module；302- computing module；304- processing module；306- splicing module.
Specific embodiment
Currently, if keeping panoramic imagery in the case where being equipped with the omnidirectional imaging system using single camera on vehicle
Under the premise of camera, vehicle and the positional relationship on ground remain unchanged in system, image data is obtained, and by image data
The displacement difference and the distance between left diameter and right wing diameter of left diameter and right wing diameter and perspective transformation matrix calculate vehicle
Angle of turn, to generate more accurately panoramic picture.But if pavement roughness or road surface have barrier all to have very much
It is different to may cause left and right path in the image data got, to obtain the panoramic picture of mistake.Moreover, if installation panorama
The vehicle of imaging system is under the conditions of complicated road conditions, so that camera, vehicle and the positional relationship on ground often change, then
Above-mentioned omnidirectional imaging system often obtains the panoramic picture of mistake in actual use.Based on this, the embodiment of the present application is proposed
A kind of method for panoramic imaging and device can also be according to getting even if camera and the position orientation relation on ground are changed
Current frame image data determine that the position orientation relation of camera and ground substantially increases to obtain correct panoramic picture in real time
The accuracy of the panoramic picture obtained under the conditions of the road conditions of actual complex.
In the method for panoramic imaging that application scheme proposes, by calculating the moving distance and rotation angle of camera, determine
First correcting image is converted to birds-eye view according to the position orientation relation of camera and ground by the position orientation relation of camera and ground out, and
Birds-eye view and ground birds-eye view are spliced, panoramic picture is obtained, even if camera and the position orientation relation on ground are changed,
The position orientation relation of camera and ground can also be determined in real time according to the current frame image data got, to obtain correctly complete
Scape image substantially increases the accuracy of the panoramic picture obtained under the conditions of the road conditions of actual complex.
The omnidirectional imaging system installed on vehicle shown in Figure 1, the omnidirectional imaging system include monocular camera 100,
Image processing unit 102, image storage unit 104 and image-display units 106.Monocular camera 100 is mounted on vehicle tail, uses
In the image for obtaining rear of vehicle.Image processing unit 102 is mainly responsible for pattern distortion correction, image mosaic, perspective image change
It changes equal image processing tasks and calculates vehicle movement and rotation angle bisecting analysis calculating task, be the core meter of this patent system
Calculate unit.The panoramic picture that image storage unit 104 is used to store the image after distortion correction and has spliced in real time.Image is aobvious
Show unit 106 for showing panoramic picture to driver in real time, to the comprehensive no dead angle exhibition vehicle of driver in a manner of vision
Periphery and bottom image.
In order to make the above objects, features, and advantages of the present application more apparent, with reference to the accompanying drawing and it is specific real
Mode is applied to be further described in detail the application.
Embodiment 1
The process of method for panoramic imaging shown in Figure 2, the present embodiment propose a kind of method for panoramic imaging, the panorama at
The executing subject of image space method is above-mentioned image processing unit.
The method for panoramic imaging that the present embodiment proposes, comprising the following specific steps
Step 200, the first correcting image and above-mentioned current frame image for determining the collected current frame image data of camera
The matching double points of second correcting image of the previous frame image data of data.
Specifically, the matching double points of the first correcting image and the second correcting image, above-mentioned steps 200 can have in order to obtain
The process of body execution following steps (1) to step (2):
(1) the collected current frame image data of camera are obtained, distortion correction behaviour is carried out to above-mentioned current frame image data
Make, obtains above-mentioned first correcting image；
(2) the second correcting image of the previous frame image data of above-mentioned current frame image data is obtained, and determines present frame
Of second correcting image of the previous frame image data of the first correcting image of image data and above-mentioned current frame image data
With point pair.
Pixel in above-mentioned steps (1), after obtaining above-mentioned first correcting image, on available first correcting image
The pixel coordinate A (u, v) of point, it is available to be somebody's turn to do according to the transformation formula of pixel coordinate in pattern distortion model and physical coordinates
Physical coordinates of the pixel on the first correcting image.It specifically, can be by following formula (1) pixel in the first correction
Physical coordinates on image:
Wherein, x ' indicates that physics abscissa of the pixel on the first correcting image, y ' indicate the pixel first
Physics ordinate on correcting image, u indicate that pixel abscissa of the pixel on the first correcting image, v indicate the pixel
Pixel ordinate o'clock on the first correcting image, CxAnd CyIndicate the intersection point of camera primary optical axis and imaging plane,Wherein f is camera focus, and dx and dy are respectively the width and height of the pixel.
The pixel is being obtained after the physical coordinates on the first correcting image, can be somebody's turn to do by following formula (2)
Physical coordinates of the pixel in above-mentioned current frame image data:
Wherein, x " indicates that physics abscissa of the pixel in above-mentioned current frame image data, y " indicate the pixel
Physics ordinate in above-mentioned current frame image data, k1And k2For the coefficient of radial distortion of camera lens, p1And p2It is tangential for camera lens
Distortion factor, r2=x '2+y′2。
Above-mentioned f, Cx、Cy、k1、k2、p1、p2, dx and dy be camera intrinsic parameter.Above-mentioned camera intrinsic parameter is pacified by camera
After being attached on vehicle and being demarcated to camera, accessed parameter relevant to camera.Above-mentioned camera intrinsic parameter is solid
Definite value is pre-stored in above-mentioned image processing unit.
The pixel is being obtained after the physical coordinates in above-mentioned current frame image data, following formula can be continued through
(3) pixel coordinate B (u of the pixel in above-mentioned current frame image data is obtainedd, vd):
Pixel coordinate B (u of the pixel in above-mentioned current frame image data is being obtained by above-mentioned formula (3)d, vd)
Afterwards, to the B (u of the obtained pixeld, vd) carry out interpolation calculation just completed to obtain the pixel size of the pixel
The distortion correction of the pixel.
Other pixels for successively traversing above-mentioned first correcting image can complete the distortion correction behaviour of current image date
Make.
Above-mentioned camera is exactly the monocular camera in above-mentioned omnidirectional imaging system.
In above-mentioned steps (2), the first correcting image and the second correcting image are determined by existing image matching algorithm
Matching double points.
Step 202, the moving distance that above-mentioned camera is calculated using above-mentioned matching double points and rotation angle.
In one embodiment, the moving distance and rotation angle of above-mentioned camera can be calculated by following formula (4):
Wherein, R indicates rotation angle, and t indicates moving distance, xiIndicate that the pixel coordinate of i-th of matching double points, K indicate
The matrix that camera intrinsic parameter is constituted, XiFor the world coordinates of the matching double points, Ω is the matching double points of all the known world coordinates.
Operator argmin, which solves R and t, is minimized right formula, so, the definition of above-mentioned formula (4) is in formula (4)
Right side of the equal signIn the case where being minimized, to seek the specific of R and t
Numerical value.
Optionally, when system is run for the first time, camera is located in the image data of two initial position acquisitions, image data
In the world coordinates of all characteristic points (contain matching double points) be unknown.It so just needs to all spies in image data
The world coordinates of sign point is estimated, and its estimated value is inputted in above-mentioned formula (4), and is solved using above-mentioned formula (4)
After the R and t of camera, R and t that then basis solves, the world coordinates of characteristic point is calculated using triangle telemetry, is denoted as
Ω_output；
If Ω _ input ≈ Ω _ output, the pose of camera and the world coordinates of all match points can be obtained, it is no
Then, above-mentioned formula (4) are re-entered using Ω _ output as Ω _ input, Ω _ output is calculated again, until Ω _ output
≈Ω_input。
After system spends two initial positions, image data and an adjacent upper position that camera is obtained in current location
The matching point set in the image data that place obtains is set, the characteristic point of part the known world coordinate is contained, with these characteristic points
World coordinates be that Ω _ input inputs above-mentioned formula (4), solution obtains the R and t of Current camera, and utilizes the R of Current camera
The world coordinates of the unknown characteristic point of another part world coordinates is calculated with t.
Certainly, above-mentioned steps 202 can also pass through simultaneous localization and mapping (Simultaneous
Localization and Mapping, SLAM) technology, so that it may using above-mentioned matching double points calculate the movement of above-mentioned camera away from
From with rotation angle, here no longer with repeat one by one.
After the moving distance and rotation angle that above-mentioned camera is calculated by above step 202, it can continue to execute
Following steps 204 determine the position orientation relation on camera and ground.Step 204, moving distance and rotation angle based on above-mentioned camera
Degree obtains the ground obstacle characteristic point in above-mentioned first correcting image, and determines the position orientation relation of camera and ground.
Specifically, above-mentioned steps 204 can execute following steps (1) to step (3):
(1) moving distance and rotation angle based on above-mentioned camera, each match point calculated in matching double points are sat in camera
Three-dimensional coordinate under mark system；
(2) three-dimensional coordinate according to each match point under above-mentioned camera coordinates system, by multiple coplanar in each match point
It is determined as terrain surface specifications point with point, and the non-terrain surface specifications point in each match point is determined as ground obstacle characteristic point；
(3) plane fitting operation is carried out to the above-mentioned terrain surface specifications point determined, obtains ground in camera coordinates system
Normal vector, so that it is determined that the position orientation relation of camera and ground.
In above-mentioned steps (1), pass through SLAM technology, so that it may moving distance and rotation angle based on above-mentioned camera,
Three-dimensional coordinate of each match point in matching double points under camera coordinates system is calculated, which is not described herein again.
Above-mentioned camera coordinates system, be using camera photocentre as origin, using from optical center be perpendicularly oriented to imaging plane direction as Z axis just
Direction, with parallel imaging plane transverse direction from left side be directed toward right side be X-axis positive direction, with parallel imaging plane longitudinal direction from
It is the coordinate system that Y-axis positive direction is formed that bottom is directed toward at top.
In above-mentioned steps (2), terrain surface specifications point should generally in same plane, so, x in each match point is sat
Mark and the essentially identical multiple match points of y-coordinate are determined as terrain surface specifications point.
In above-mentioned steps (3), plane fitting is carried out to terrain surface specifications point using least square method, obtains ground in camera
Normal vector in coordinate system.So, the Z axis (camera primary optical axis) of normal vector and camera coordinates system of the ground under camera coordinates system
Position orientation relation be exactly ground and camera position orientation relation.
After the position orientation relation for determining camera and ground by above step 204, following steps 206 can be continued to execute,
Obtain panoramic picture.
Step 206, according to the position orientation relation on above-mentioned camera and ground, above-mentioned first correcting image is converted into birds-eye view,
And splice birds-eye view and ground birds-eye view, obtain panoramic picture, wherein above-mentioned ground is shown in above-mentioned panoramic picture
The ground obstacle of barrier characteristic point characterization.
Specifically, panoramic picture, above-mentioned steps 206 can execute following steps (1) to step (2) in order to obtain:
(1) according to the position orientation relation of above-mentioned camera and ground, rotation and translation is carried out to above-mentioned first correcting image, it will be upper
It states the first correcting image and is converted to birds-eye view；
(2) above-mentioned ground birds-eye view is obtained, and above-mentioned birds-eye view and ground birds-eye view are spliced, obtains panorama sketch
Picture.
In above-mentioned steps (2), ground birds-eye view can be buffered in advance in above-mentioned image storage unit.
Step 200 more than passing through is to step 206, after obtaining panoramic picture, in order in reversing to driver with
Prompt, it is also necessary to position and the posture of vehicle are shown in panoramic picture.In order to shown in panoramic picture vehicle position and
Posture can continue to execute the process of following steps (1) to step (3):
(1) according to the moving distance of above-mentioned camera and rotation angle, the moving distance for installing the vehicle of above-mentioned camera is obtained
With rotation angle；
(2) position and direction according to the moving distance of vehicle and rotation angle calculation vehicle in panoramic picture；
(3) position and direction according to the above-mentioned vehicle being calculated in panoramic picture, by the vehicle mould of above-mentioned vehicle
Type is plotted in panoramic picture.
In above-mentioned steps (1), installation site of the camera on vehicle be it is fixed, so the moving distance and rotation of camera
Gyration is exactly the moving distance and rotation angle of vehicle.
In above-mentioned steps (2), the distance and rotation of position where camera, movement have been solved by SLAM technology
Angle.Camera is mounted on the fixation position of vehicle, and the physical size of vehicle is fixed.After obtaining the position of camera,
With camera position, position and direction of the vehicle in panoramic picture are calculated.
In above-mentioned steps (3), the vehicle ruler of vehicle is first obtained from image storage unit according to the vehicle model information of vehicle
It is very little, according to the image scaled of preset panoramic picture and actual environment, equal proportion scaling is carried out to the vehicle dimension of vehicle, is obtained
Vehicle dimension in panoramic picture draws auto model according to obtained vehicle dimension, and according to the above-mentioned vehicle being calculated
The auto model of drafting is placed into panoramic picture by the position and direction in panoramic picture.
Corresponding relationship, vehicle model information, the Yi Jiquan of vehicle model information and vehicle dimension are previously stored in image storage unit
The image scaled of scape image and actual environment.
Here, vehicle model information is that driver inputs when configuring above-mentioned omnidirectional imaging system, above-mentioned omnidirectional imaging system
After the vehicle model information for getting driver's input, vehicle model information can be cached in image storage unit.
It can be seen that the content that step (3) describes the position according to vehicle in panoramic picture by above step (1)
It sets and direction, the auto model of vehicle is drawn and shown in panoramic picture, so that driver not only will be seen that the week of vehicle
Surrounding environment can also determine the mode of operation being put in storage to vehicle backing, to the greatest extent may be used according to the position and direction of vehicle in panoramic picture
It is avoided that scratching and colliding with occur in the process of backing up in vehicle.
In conclusion the method for panoramic imaging that the present embodiment proposes, by calculating the moving distance and rotation angle of camera,
First correcting image is converted to and is got a bird's eye view according to the position orientation relation of camera and ground by the position orientation relation for determining camera and ground
Figure, and birds-eye view and ground birds-eye view is spliced, and obtains panoramic picture, and must keep panoramic imagery system in the related technology
Camera, vehicle are compared with the mode of the position relationship on ground in system, even if camera and the position orientation relation on ground are become
Change, can also determine the position orientation relation of camera and ground in real time according to the current frame image data got, to obtain correct
Panoramic picture, substantially increase the accuracy of the panoramic picture obtained under the conditions of the road conditions of actual complex.
Based on the same inventive concept, additionally provided in the embodiment of the present application panorama corresponding with above-mentioned method for panoramic imaging at
As device, the principle solved the problems, such as due to the method in the embodiment of the present application and the above-mentioned method for panoramic imaging of the embodiment of the present application 1
Described in image processing module function it is similar, therefore the implementation of the present embodiment device may refer to aforementioned method for panoramic imaging
Implementation, overlaps will not be repeated.
Embodiment 2
The structural schematic diagram of panoramic imaging device shown in Figure 3, the present embodiment propose a kind of panoramic imaging device, packet
It includes:
Determining module 300, for determining the first correcting image of the collected current frame image data of camera and above-mentioned working as
The matching double points of second correcting image of the previous frame image data of prior image frame data；
Computing module 302, for calculating the moving distance and rotation angle of above-mentioned camera using above-mentioned matching double points；
Processing module 304 obtains above-mentioned first correcting image for moving distance and rotation angle based on above-mentioned camera
In ground obstacle characteristic point, and determine camera and ground position orientation relation；
Splicing module 306 is converted to above-mentioned first correcting image for the position orientation relation according to above-mentioned camera and ground
Birds-eye view, and birds-eye view and ground birds-eye view are spliced, obtain panoramic picture, wherein show in above-mentioned panoramic picture
The ground obstacle of above-mentioned ground obstacle characteristic point characterization.
In one embodiment, above-mentioned determining module 300, is specifically used for:
The collected current frame image data of camera are obtained, distortion correction operation is carried out to above-mentioned current frame image data,
Obtain above-mentioned first correcting image；
The second correcting image of the previous frame image data of above-mentioned current frame image data is obtained, and determines current frame image
The match point of second correcting image of the previous frame image data of the first correcting image of data and above-mentioned current frame image data
It is right.
In one embodiment, above-mentioned processing module 304, is specifically used for:
Moving distance and rotation angle based on above-mentioned camera, calculate each match point in matching double points in camera coordinates system
Under three-dimensional coordinate；
According to three-dimensional coordinate of each match point under above-mentioned camera coordinates system, by multiple match points coplanar in each match point
It is determined as terrain surface specifications point, and the non-terrain surface specifications point in each match point is determined as ground obstacle characteristic point；
Plane fitting operation is carried out to the above-mentioned terrain surface specifications point determined, obtains normal direction of the ground in camera coordinates system
Amount, so that it is determined that the position orientation relation of camera and ground.
In one embodiment, above-mentioned splicing module, is specifically used for:
According to the position orientation relation of above-mentioned camera and ground, rotation and translation is carried out to above-mentioned first correcting image, it will be above-mentioned
First correcting image is converted to birds-eye view；
Above-mentioned ground birds-eye view is obtained, and above-mentioned birds-eye view and ground birds-eye view are spliced, obtains panoramic picture.
After obtaining panoramic picture, in order to reversing when to driver to prompt, it is also necessary to shown in panoramic picture
The position of vehicle and posture.In order to show position and the posture of vehicle, the panoramic imagery that the present embodiment proposes in panoramic picture
Device, further includes:
First operation module obtains installing above-mentioned camera for the moving distance and rotation angle according to above-mentioned camera
The moving distance and rotation angle of vehicle；
Second operation module, for position of the moving distance and rotation angle calculation vehicle according to vehicle in panoramic picture
It sets and direction；
Drafting module, for the position and direction according to the above-mentioned vehicle being calculated in panoramic picture, by above-mentioned vehicle
Auto model be plotted in panoramic picture.
By content described above as can be seen that position and direction according to vehicle in panoramic picture, in panorama sketch
The auto model of vehicle is drawn and shown as in, so that driver not only will be seen that the surrounding enviroment of vehicle, it can also basis
The position and direction of vehicle in panoramic picture determine the mode of operation being put in storage to vehicle backing, vehicle are avoided to move backward as far as possible
Occur scratching and colliding in the process.
In conclusion the panoramic imaging device that the present embodiment proposes, by calculating the moving distance and rotation angle of camera,
First correcting image is converted to and is got a bird's eye view according to the position orientation relation of camera and ground by the position orientation relation for determining camera and ground
Figure, and birds-eye view and ground birds-eye view is spliced, and obtains panoramic picture, and must keep panoramic imagery system in the related technology
Camera, vehicle are compared with the mode of the position relationship on ground in system, even if camera and the position orientation relation on ground are become
Change, can also determine the position orientation relation of camera and ground in real time according to the current frame image data got, to obtain correct
Panoramic picture, substantially increase the accuracy of the panoramic picture obtained under the conditions of the road conditions of actual complex.
The above description is merely a specific embodiment, but scope of protection of the present invention is not limited thereto, any
Those familiar with the art in the technical scope disclosed by the present invention, can easily think of the change or the replacement, and should all contain
Lid is within protection scope of the present invention.Therefore, the protection scope of the present invention shall be subject to the protection scope of the claims.
Claims (10)
1. a kind of method for panoramic imaging characterized by comprising
Determine the first correcting image of the collected current frame image data of camera and the previous frame of the current frame image data
The matching double points of second correcting image of image data；
The moving distance and rotation angle of the camera are calculated using the matching double points；
Moving distance and rotation angle based on the camera, obtain the ground obstacle feature in first correcting image
Point, and determine the position orientation relation of camera and ground；
According to the position orientation relation of the camera and ground, first correcting image is converted into birds-eye view, and by birds-eye view with
Ground birds-eye view is spliced, and panoramic picture is obtained, wherein the ground obstacle characteristic point is shown in the panoramic picture
The ground obstacle of characterization.
2. the method according to claim 1, wherein the determining camera collected current frame image data
The matching double points of second correcting image of the previous frame image data of the first correcting image and the current frame image data, packet
It includes:
The collected current frame image data of camera are obtained, distortion correction operation is carried out to the current frame image data, is obtained
First correcting image；
The second correcting image of the previous frame image data of the current frame image data is obtained, and determines current frame image data
The first correcting image and the current frame image data previous frame image data the second correcting image matching double points.
3. the method according to claim 1, wherein the moving distance and rotation angle based on the camera
Degree obtains the ground obstacle characteristic point in first correcting image, and determines the position orientation relation of camera and ground, comprising:
Moving distance and rotation angle based on the camera, calculate each match point in matching double points under camera coordinates system
Three-dimensional coordinate；
According to three-dimensional coordinate of each match point under the camera coordinates system, multiple match points coplanar in each match point are determined
For terrain surface specifications point, and the non-terrain surface specifications point in each match point is determined as ground obstacle characteristic point；
Plane fitting operation is carried out to the terrain surface specifications point determined, obtains normal vector of the ground in camera coordinates system,
So that it is determined that the position orientation relation of camera and ground.
4. the method according to claim 1, wherein the position orientation relation according to the camera and ground, is incited somebody to action
First correcting image is converted to birds-eye view, and birds-eye view and ground birds-eye view are spliced, and obtains panoramic picture, packet
It includes:
According to the position orientation relation of the camera and ground, rotation and translation is carried out to first correcting image, by described first
Correcting image is converted to birds-eye view；
The ground birds-eye view is obtained, and the birds-eye view and ground birds-eye view are spliced, obtains panoramic picture.
5. the method according to claim 1, wherein the method also includes:
According to the moving distance of the camera and rotation angle, moving distance and rotation angle that the vehicle of the camera is installed are obtained
Degree；
According to the position and direction of the moving distance of vehicle and rotation angle calculation vehicle in panoramic picture；
According to position and direction of the vehicle being calculated in panoramic picture, the auto model of the vehicle is plotted in
In panoramic picture.
6. a kind of panoramic imaging device characterized by comprising
Determining module, for determining the first correcting image and the current frame image of the collected current frame image data of camera
The matching double points of second correcting image of the previous frame image data of data；
Computing module, for calculating the moving distance and rotation angle of the camera using the match point；
Processing module obtains the ground in first correcting image for moving distance and rotation angle based on the camera
Face barrier characteristic point, and determine the position orientation relation of camera and ground；
First correcting image is converted to birds-eye view for the position orientation relation according to the camera and ground by splicing module,
And splice birds-eye view and ground birds-eye view, obtain panoramic picture, wherein the ground is shown in the panoramic picture
The ground obstacle of barrier characteristic point characterization.
7. device according to claim 6, which is characterized in that the determining module is specifically used for:
The collected current frame image data of camera are obtained, distortion correction operation is carried out to the current frame image data, is obtained
First correcting image；
The second correcting image of the previous frame image data of the current frame image data is obtained, and determines current frame image data
The first correcting image and the current frame image data previous frame image data the second correcting image matching double points.
8. device according to claim 6, which is characterized in that the processing module is specifically used for:
Moving distance and rotation angle based on the camera, calculate each match point in matching double points under camera coordinates system
Three-dimensional coordinate；
According to three-dimensional coordinate of each match point under the camera coordinates system, multiple match points coplanar in each match point are determined
For terrain surface specifications point, and the non-terrain surface specifications point in each match point is determined as ground obstacle characteristic point；
Plane fitting operation is carried out to the terrain surface specifications point determined, obtains normal vector of the ground in camera coordinates system,
So that it is determined that the position orientation relation of camera and ground.
9. device according to claim 6, which is characterized in that the splicing module is specifically used for:
According to the position orientation relation of the camera and ground, rotation and translation is carried out to first correcting image, by described first
Correcting image is converted to birds-eye view；
The ground birds-eye view is obtained, and the birds-eye view and ground birds-eye view are spliced, obtains panoramic picture.
10. device according to claim 6, which is characterized in that further include:
First operation module obtains the vehicle for installing the camera for the moving distance and rotation angle according to the camera
Moving distance and rotation angle；
Second operation module, for according to position in panoramic picture of the moving distance of vehicle and rotation angle calculation vehicle and
Direction；
Drafting module, for the position and direction according to the vehicle being calculated in panoramic picture, by the vehicle
Auto model is plotted in panoramic picture.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201811191264.4A CN109447901B (en) | 2018-10-12 | 2018-10-12 | Panoramic imaging method and device |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201811191264.4A CN109447901B (en) | 2018-10-12 | 2018-10-12 | Panoramic imaging method and device |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN109447901A true CN109447901A (en) | 2019-03-08 |
| CN109447901B CN109447901B (en) | 2023-12-19 |
Family
ID=65546424
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201811191264.4A Active CN109447901B (en) | 2018-10-12 | 2018-10-12 | Panoramic imaging method and device |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN109447901B (en) |
Cited By (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110610520A (en) * | 2019-08-29 | 2019-12-24 | 中德（珠海）人工智能研究院有限公司 | Visual positioning method and system based on double-dome camera |
| CN110969576A (en) * | 2019-11-13 | 2020-04-07 | 同济大学 | Highway pavement image splicing method based on roadside PTZ camera |
| CN111915910A (en) * | 2020-08-14 | 2020-11-10 | 山东领军智能交通科技有限公司 | Road traffic signal lamp based on Internet of things |
| CN113370993A (en) * | 2021-06-11 | 2021-09-10 | 北京汽车研究总院有限公司 | Control method and control system for automatic driving of vehicle |
| CN113837936A (en) * | 2020-06-24 | 2021-12-24 | 上海汽车集团股份有限公司 | Panoramic image generation method and device |
Citations (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN105763854A (en) * | 2016-04-18 | 2016-07-13 | 扬州航盛科技有限公司 | Omnidirectional imaging system based on monocular camera, and imaging method thereof |
| CN105825475A (en) * | 2016-04-01 | 2016-08-03 | 西安电子科技大学 | 360-degree panorama image generation method based on single pick-up head |
| US20170050564A1 (en) * | 2015-03-20 | 2017-02-23 | Boe Technology Group Co., Ltd. | Vehicle-mounted display system and vehicle |
| CN106476696A (en) * | 2016-10-10 | 2017-03-08 | 深圳市前海视微科学有限责任公司 | A kind of reverse guidance system and method |
| CN106846243A (en) * | 2016-12-26 | 2017-06-13 | 深圳中科龙智汽车科技有限公司 | The method and device of three dimensional top panorama sketch is obtained in equipment moving process |
| CN107145828A (en) * | 2017-04-01 | 2017-09-08 | 纵目科技（上海）股份有限公司 | Vehicle panoramic image processing method and device |
| CN107341787A (en) * | 2017-07-26 | 2017-11-10 | 珠海研果科技有限公司 | Method, apparatus, server and the automobile that monocular panorama is parked |
| CN108171655A (en) * | 2017-12-27 | 2018-06-15 | 深圳普思英察科技有限公司 | Reverse image joining method and device based on monocular cam |
-
2018
- 2018-10-12 CN CN201811191264.4A patent/CN109447901B/en active Active
Patent Citations (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20170050564A1 (en) * | 2015-03-20 | 2017-02-23 | Boe Technology Group Co., Ltd. | Vehicle-mounted display system and vehicle |
| CN105825475A (en) * | 2016-04-01 | 2016-08-03 | 西安电子科技大学 | 360-degree panorama image generation method based on single pick-up head |
| CN105763854A (en) * | 2016-04-18 | 2016-07-13 | 扬州航盛科技有限公司 | Omnidirectional imaging system based on monocular camera, and imaging method thereof |
| CN106476696A (en) * | 2016-10-10 | 2017-03-08 | 深圳市前海视微科学有限责任公司 | A kind of reverse guidance system and method |
| CN106846243A (en) * | 2016-12-26 | 2017-06-13 | 深圳中科龙智汽车科技有限公司 | The method and device of three dimensional top panorama sketch is obtained in equipment moving process |
| CN107145828A (en) * | 2017-04-01 | 2017-09-08 | 纵目科技（上海）股份有限公司 | Vehicle panoramic image processing method and device |
| CN107341787A (en) * | 2017-07-26 | 2017-11-10 | 珠海研果科技有限公司 | Method, apparatus, server and the automobile that monocular panorama is parked |
| CN108171655A (en) * | 2017-12-27 | 2018-06-15 | 深圳普思英察科技有限公司 | Reverse image joining method and device based on monocular cam |
Non-Patent Citations (2)
| Title |
|---|
| DAVIDE SCARAMUZZA: "Appearance-Guided Monocular Omnidirectional Visual Odometry for Outdoor Ground Vehicles", 《IEEE TRANSACTIONS ON ROBOTICS》 * |
| 马超群: "移动系统中的实时图像处理算法的研究", 《中国优秀硕士学位论文全文数据库》 * |
Cited By (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110610520A (en) * | 2019-08-29 | 2019-12-24 | 中德（珠海）人工智能研究院有限公司 | Visual positioning method and system based on double-dome camera |
| CN110969576A (en) * | 2019-11-13 | 2020-04-07 | 同济大学 | Highway pavement image splicing method based on roadside PTZ camera |
| CN113837936A (en) * | 2020-06-24 | 2021-12-24 | 上海汽车集团股份有限公司 | Panoramic image generation method and device |
| CN111915910A (en) * | 2020-08-14 | 2020-11-10 | 山东领军智能交通科技有限公司 | Road traffic signal lamp based on Internet of things |
| CN113370993A (en) * | 2021-06-11 | 2021-09-10 | 北京汽车研究总院有限公司 | Control method and control system for automatic driving of vehicle |
Also Published As
| Publication number | Publication date |
|---|---|
| CN109447901B (en) | 2023-12-19 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN109447901A (en) | A kind of method for panoramic imaging and device | |
| CN109902637B (en) | Lane line detection method, lane line detection device, computer device, and storage medium | |
| JP7240367B2 (en) | Methods, apparatus, electronic devices and storage media used for vehicle localization | |
| JP2021120864A (en) | Method and device for detecting obstacle, electronic apparatus, storage medium and computer program | |
| WO2018196391A1 (en) | Method and device for calibrating external parameters of vehicle-mounted camera | |
| US9536306B2 (en) | Vehicle vision system | |
| US10891795B2 (en) | Localization method and apparatus based on 3D color map | |
| CN112444242A (en) | Pose optimization method and device | |
| CN112614192B (en) | On-line calibration method of vehicle-mounted camera and vehicle-mounted information entertainment system | |
| CN109544630A (en) | Posture information determines method and apparatus, vision point cloud construction method and device | |
| CN110766760B (en) | Method, device, equipment and storage medium for camera calibration | |
| WO2020133172A1 (en) | Image processing method, apparatus, and computer readable storage medium | |
| CN112017246B (en) | Image acquisition method and device based on inverse perspective transformation | |
| CN112561841A (en) | Point cloud data fusion method and device for laser radar and camera | |
| CN112967344B (en) | Method, device, storage medium and program product for calibrating camera external parameters | |
| CN113610927B (en) | AVM camera parameter calibration method and device and electronic equipment | |
| CN109978954A (en) | The method and apparatus of radar and camera combined calibrating based on cabinet | |
| CN112017236A (en) | Method and device for calculating position of target object based on monocular camera | |
| CN116193108A (en) | Online self-calibration method, device, equipment and medium for camera | |
| CN113763481B (en) | Multi-camera visual three-dimensional map construction and self-calibration method in mobile scene | |
| WO2024147898A1 (en) | Parking space detection method and system | |
| CN113665591A (en) | Unmanned control method, device, equipment and medium | |
| CN114979464B (en) | Industrial camera view angle accurate configuration method and system adaptive to target area | |
| KR20210030751A (en) | Apparatus for providing top view image of parking space and method thereof | |
| CN114926550A (en) | Multi-vehicle rear-mounted camera viewing angle self-adaption method and system |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |