CN110288527B - Panoramic aerial view generation method of vehicle-mounted panoramic camera - Google Patents
Panoramic aerial view generation method of vehicle-mounted panoramic camera Download PDFInfo
- Publication number
- CN110288527B CN110288527B CN201910549926.9A CN201910549926A CN110288527B CN 110288527 B CN110288527 B CN 110288527B CN 201910549926 A CN201910549926 A CN 201910549926A CN 110288527 B CN110288527 B CN 110288527B
- Authority
- CN
- China
- Prior art keywords
- camera
- vehicle
- aerial view
- panoramic
- calibration scene
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 238000000034 method Methods 0.000 title claims abstract description 44
- 235000004522 Pentaglottis sempervirens Nutrition 0.000 claims abstract description 26
- 238000006243 chemical reaction Methods 0.000 claims abstract description 25
- 230000004927 fusion Effects 0.000 claims abstract description 22
- 239000011159 matrix material Substances 0.000 claims abstract description 21
- 238000012795 verification Methods 0.000 claims description 10
- 239000003550 marker Substances 0.000 claims description 4
- 238000012937 correction Methods 0.000 claims description 3
- 235000005121 Sorbus torminalis Nutrition 0.000 claims 2
- 244000152100 Sorbus torminalis Species 0.000 claims 2
- 238000010586 diagram Methods 0.000 description 7
- 240000004050 Pentaglottis sempervirens Species 0.000 description 2
- 238000004364 calculation method Methods 0.000 description 2
- 230000000694 effects Effects 0.000 description 2
- 230000008447 perception Effects 0.000 description 2
- 230000008859 change Effects 0.000 description 1
- 230000007547 defect Effects 0.000 description 1
- 238000013461 design Methods 0.000 description 1
- 230000007613 environmental effect Effects 0.000 description 1
- 230000006870 function Effects 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 230000008569 process Effects 0.000 description 1
- 238000012545 processing Methods 0.000 description 1
- 238000010187 selection method Methods 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T3/00—Geometric image transformations in the plane of the image
- G06T3/40—Scaling of whole images or parts thereof, e.g. expanding or contracting
- G06T3/4038—Image mosaicing, e.g. composing plane images from plane sub-images
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/80—Analysis of captured images to determine intrinsic or extrinsic camera parameters, i.e. camera calibration
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/10—Image acquisition modality
- G06T2207/10004—Still image; Photographic image
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20212—Image combination
- G06T2207/20221—Image fusion; Image merging
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/30—Subject of image; Context of image processing
- G06T2207/30248—Vehicle exterior or interior
- G06T2207/30252—Vehicle exterior; Vicinity of vehicle
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02T—CLIMATE CHANGE MITIGATION TECHNOLOGIES RELATED TO TRANSPORTATION
- Y02T10/00—Road transport of goods or passengers
- Y02T10/10—Internal combustion engine [ICE] based vehicles
- Y02T10/40—Engine management systems
Landscapes
- Engineering & Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Theoretical Computer Science (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Image Processing (AREA)
- Closed-Circuit Television Systems (AREA)
Abstract
The embodiment of the application relates to a panoramic aerial view generation method of a vehicle-mounted panoramic camera, which comprises the following steps: establishing a calibration scene according to the vehicle; the vehicle is provided with a plurality of vehicle-mounted looking-around cameras; setting corresponding checkerboards in a calibration scene according to a camera; a plurality of corner points are arranged in the checkerboard; obtaining a homography conversion matrix from each camera picture to a bird's eye view according to the calibration scene and the camera internal parameters; converting the plurality of camera pictures into a bird's eye view by utilizing a homography conversion matrix; and fusing the overlapping areas in the aerial view, and outputting the spliced circular aerial view. The calibration scene designed by the application has no strict limit on the size of the vehicle, and is particularly suitable for calibrating the external field environment; for the splicing and fusion of a plurality of aerial views, the method provided by the application has low consumption of computing resources and is easy to realize on an embedded platform.
Description
Technical Field
The application relates to the field of image data processing, in particular to a panoramic aerial view generation method of a vehicle-mounted panoramic camera.
Background
The vehicle-mounted all-around camera system is characterized in that a plurality of wide-angle cameras capable of covering all view fields around a vehicle are erected around the vehicle, multiple paths of video images collected at the same time are processed, and a vehicle body aerial view of 360 degrees around the vehicle is synthesized. The look-around system can provide richer perception information for the automatic driving vehicle.
The current general panoramic aerial view generation method of the vehicle-mounted panoramic camera is mainly divided into two parts, namely single-camera aerial view generation and multi-aerial view splicing.
For generating the aerial view of a single camera, a specific calibration scene is required to be arranged, and the projection change relation of the image to the aerial view is established by selecting a plurality of groups of mark point pairs. The arrangement of the calibration scene and the selection method of the mark point pairs directly influence the conversion accuracy of the aerial view. The existing scene arrangement schemes are complex, the selection of the mark point pairs adopts a manual mode, and the size of the vehicle is limited to a certain extent.
For the splicing and fusion of a plurality of aerial views, the traditional image splicing method needs to extract characteristic points and register images in real time, and the operations are complex, so that a large amount of calculation resources are consumed, and the real-time calculation on embedded equipment is not facilitated.
Disclosure of Invention
Aiming at the defects of the prior art, the application provides a vehicle-mounted panoramic aerial view generation method of a panoramic camera, which comprises a simple and flexible aerial view calibration scene, and a method for generating the panoramic aerial view of the vehicle-mounted panoramic camera is designed based on the scene, wherein the designed calibration scene has no strict limitation on the size of a vehicle, and is particularly suitable for calibrating an external field environment; for the splicing and fusion of a plurality of aerial views, the method provided by the application has low consumption of computing resources and is easy to realize on an embedded platform.
In view of this, the embodiment of the application provides a panoramic aerial view generating method for a vehicle-mounted panoramic camera, which includes:
establishing a calibration scene according to the vehicle; a plurality of vehicle-mounted looking-around cameras are arranged on the vehicle;
setting corresponding checkerboards in the calibration scene according to the arrangement of the cameras; a plurality of corner points are arranged in the checkerboard;
obtaining a homography conversion matrix from each camera picture to a bird's eye view according to the calibration scene and the camera internal parameters;
converting a plurality of camera pictures into a bird's eye view by utilizing the homography conversion matrix;
and fusing the overlapped areas in the aerial view, and outputting the spliced annular aerial view.
Preferably, the establishing the calibration scene according to the automatic driving vehicle specifically includes:
and planning a vehicle space occupying frame according to the vehicle.
Further preferably, the setting of the corresponding checkerboard in the calibration scene according to the camera specifically includes:
setting corresponding checkerboards in the calibration scene according to the camera and the vehicle space occupying frame.
Further preferably, a vehicle-mounted looking-around camera is arranged around the vehicle, each camera corresponds to two chessboards, and the distance between the two chessboards corresponding to each camera is equal;
after the setting of the corresponding checkerboard in the calibration scene according to the camera and the vehicle placeholder, the method further comprises:
judging whether each camera can acquire two checkerboards or not;
if the vehicle placeholder cannot be rescheduled.
Further preferably, after the setting of the corresponding checkerboard in the calibration scene according to the camera, the method further comprises:
obtaining a first configuration parameter according to the size of the vehicle space occupying frame;
obtaining a second configuration parameter according to the positions of the checkerboard;
and obtaining a third configuration parameter according to the positions of the vehicle space occupying frame and the checkerboard.
Further preferably, the homography conversion matrix for obtaining the image from each camera to the aerial view according to the calibration scene and the camera internal parameters specifically includes:
acquiring a camera picture;
carrying out image distortion correction on the camera picture according to the camera internal parameters to obtain a corrected camera picture;
acquiring checkerboard corner points in the corrected camera picture, and acquiring predefined positions of the checkerboard corner points in the aerial view in the calibration scene;
and obtaining a homography conversion matrix from the camera picture to the aerial view according to the corrected camera picture, the predefined positions of the checkerboard corner points in the calibration scene in the aerial view, and the first configuration parameter, the second configuration parameter and the third configuration parameter.
Further preferably, the converting the plurality of camera pictures to the aerial view by using the homography conversion matrix specifically includes:
and converting the corrected camera picture into a bird's eye view by utilizing the homography conversion matrix.
Preferably, the fusing the overlapping area in the aerial view specifically includes:
fusing RGB values of each pixel point of the overlapping area according to a preset algorithm; the preset algorithm is as follows:
α＝θ/90°
P IMG (fusion) ＝α*P IMG(1) +(1-α)*P IMG(2)
Wherein θ is the angle between any point P in the overlapping area and the X axis, P IMG (fusion) For the RGB value after point P fusion, P IMG(1) And P IMG(2) RGB values for overlapping area pixels, respectively.
Preferably, after the outputting the stitched round view, the method further includes:
verifying the spliced circular bird's eye view;
when the verification result meets the preset condition, the verification is passed;
and when the verification result does not meet the preset condition, the generation of the aerial view is performed again.
Further preferably, verifying the spliced circular bird's eye view specifically includes:
placing a marker in the overlapping area of the two cameras;
and checking whether the marker has ghost or not in the obtained surrounding bird's eye view.
The method for generating the panoramic aerial view of the vehicle-mounted panoramic camera provided by the embodiment of the application comprises a simple and flexible aerial view calibration scene, and a set of method for generating the panoramic aerial view of the vehicle-mounted panoramic camera is designed based on the scene. The calibration scene designed by the application has no strict limit on the size of the vehicle, and is particularly suitable for calibrating the external field environment; for the splicing and fusion of a plurality of aerial views, the method provided by the application has low consumption of computing resources and is easy to realize on an embedded platform.
Drawings
Fig. 1 is a flowchart of a method for generating panoramic aerial view of a vehicle-mounted panoramic camera according to an embodiment of the present application;
fig. 2 is a schematic view of a calibrating scene of an aerial view of an all-round camera according to an embodiment of the present application;
FIG. 3 is a schematic diagram of a checkerboard according to an embodiment of the present application;
FIG. 4 is a flowchart of a single fisheye camera calibration process according to an embodiment of the present application;
FIG. 5 is a bird's eye view of a single fisheye camera after calibration according to an embodiment of the present application;
FIG. 6 is a schematic diagram of a multi-aerial view fusion provided by an embodiment of the present application;
FIG. 7 is an example of a bird's eye view after stitching and fusion provided by an embodiment of the present application;
fig. 8 is a schematic diagram of ghost phenomena according to an embodiment of the present application.
Detailed Description
The technical scheme of the application is further described in detail through the drawings and the embodiments.
The vehicle-mounted panoramic camera panoramic aerial view generation method provided by the embodiment of the application is preferably applied to an automatic driving vehicle, and the images acquired by a plurality of cameras arranged on the automatic driving vehicle are processed and fused to obtain the panoramic aerial view around the vehicle, so that rich perception information is provided for the automatic driving vehicle.
Fig. 1 is a flowchart of a method for generating a panoramic aerial view of a vehicle-mounted panoramic camera according to an embodiment of the present application, as shown in fig. 1, where the method includes:
and 101, establishing a calibration scene according to the vehicle.
The vehicle is an automatic driving vehicle, and a plurality of vehicle-mounted looking-around cameras are arranged on the automatic driving vehicle, and in this example, the cameras are preferably fisheye cameras, but can also be other types of cameras, and the person skilled in the art can select the cameras according to the needs.
Before the aerial view conversion of the panoramic camera is performed, the arrangement of calibration scenes is needed, step 101 and step 102 both belong to the arrangement of calibration scenes, firstly, a vehicle space frame is planned according to the size of the vehicle, fig. 2 is a schematic view of the aerial view calibration scenes of the panoramic camera, as shown in fig. 2, and solid line frames in the view are space frames of the vehicle.
Step 102, setting corresponding checkerboards in the calibration scene according to the arrangement of the cameras.
Specifically, corresponding checkerboards are set in a calibration scene according to the positions of the cameras and the vehicle space occupying frames, the general positions of the checkerboards are set according to the positions of the cameras, each camera corresponds to at least one checkerboard, and then the specific positions of the checkerboards are determined according to the positions of the vehicle space occupying frames, so that one side of each checkerboard is attached to the space occupying frame.
Fig. 3 is a schematic diagram of a checkerboard according to an embodiment of the present application, in which each checkerboard is provided with a plurality of corner points, in this example, a checkerboard calibration plate is 90cm x 80cm (the number of corner points is 3*2), and a single black-and-white square is 20cm x 20cm. As shown in fig. 2 and 3, a vehicle-mounted looking-around camera is respectively arranged around the vehicle, namely front, back, left and right, each camera corresponds to two chessboards, that is, two chessboard calibration boards are respectively arranged in front of, behind, left and right of the vehicle, and the distances between the two chessboards corresponding to each camera are equal; preferably, the central lines of the front and rear groups of chessboards of the vehicle are consistent with the transverse central line of the vehicle space occupying frame, and the central lines of the left and right groups of chessboards of the vehicle are consistent with the connecting lines of the left and right rearview mirrors of the vehicle as much as possible, so that a symmetrical distribution mode is formed.
The number of cells in the checkerboard may be set as required by those skilled in the art, or the pitch between the checkerboards may be set as required.
After setting the checkerboard, the method further includes: judging whether each camera can acquire two checkerboards or not; if the vehicle space frame cannot be planned again, that is, the vehicle space frame has no fixed size, only two complete checkerboards can be seen by each fish-eye camera, so that the accuracy of an image fusion result can be ensured, and the vehicle can obtain more accurate environmental information. Subsequent steps are performed after ensuring that each camera sees two checkerboards.
After the calibration scene is arranged, a first configuration parameter is obtained according to the size of the vehicle space frame, a second configuration parameter is obtained according to the positions of the checkerboards, and a third configuration parameter is obtained according to the positions of the vehicle space frame and the checkerboards, specifically, the length and the width (the first configuration parameter) of the vehicle space frame, the distance (the second configuration parameter) between two adjacent checkerboards and the distance (the third configuration parameter) between the central lines of the left and right groups of checkerboards and the vehicle space frame in the schematic diagram are measured and used as configuration parameter input quantity for calculating a follow-up homography conversion matrix.
And step 103, obtaining a homography conversion matrix from each camera picture to the aerial view according to the calibration scene and the camera internal parameters.
As shown in fig. 4, first, a camera picture is acquired, where the camera picture refers to an original view of a camera; then acquiring camera internal parameters, wherein the camera internal parameters are preset internal parameter matrixes and distortion parameters of a camera, and carrying out image distortion correction on a camera picture according to the camera internal parameters to obtain a corrected camera picture; acquiring checkerboard corner points in the corrected camera picture, and acquiring predefined positions of the checkerboard corner points in the aerial view in the calibration scene; and calculating according to the checkered corner points obtained from the corrected camera picture, the predefined positions of the checkered corner points in the aerial view in the calibration scene, the first configuration parameters, the second configuration parameters and the third configuration parameters to obtain a homography conversion matrix from the camera picture to the aerial view, wherein the final output value is a conversion homography matrix used for converting the single fisheye camera into the aerial view.
Step 104, converting the plurality of camera pictures into a bird's eye view by utilizing the homography conversion matrix.
Single camera bird's eye view generation is to re-project information in the camera image plane onto the ground plane. Specifically, the corrected camera picture is converted into the aerial view by using the homography conversion matrix.
Fig. 5 is a schematic diagram of a single fisheye camera calibrated according to an embodiment of the present application, and after the conversion of the bird's eye view of the fisheye camera in four directions is successfully completed, a corresponding set of conversion parameters may be obtained, as shown in table 1 below. The set of parameters can then be used as input to the bird's eye view stitching and fusion module.
TABLE 1 calibration parameter definition
| Calibration effectiveness | Bird's eye view projection homography matrix | |
| Rear camera | 1(0) | H3*3(double) |
| Front camera | 1(0) | H3*3(double) |
| Left camera | 1(0) | H3*3(double) |
| Right camera | 1(0) | H3*3(double) |
And 105, fusing the overlapped areas in the aerial view, and outputting the spliced annular aerial view.
After the conversion of the aerial views of all the fish-eye cameras is completed, the aerial views can be spliced and fused, and the aerial views of the camera images are spliced and fused, so that the panoramic aerial view around the vehicle is finally formed. A schematic diagram of the principle of stitching and fusing of multiple pictures is shown in fig. 6. Four facing fisheye cameras may generate four birds-eye views, and two adjacent birds-eye views will have a partially overlapping region. The task of the stitching and fusion of the bird's eye view is thus to fuse the four overlapping areas. As shown in fig. 6, the front and right overlapping areas are overlapping areas a, the front and left overlapping areas are overlapping areas B, the left and rear overlapping areas are overlapping areas C, and the rear and right overlapping areas are overlapping areas D.
According to the application, the RGB values of each pixel point in the overlapping area are fused according to the following preset algorithm, and finally, a complete surrounding bird's eye view can be obtained.
α＝θ/90°
P IMG (fusion) ＝α*P IMG(1) +(1-α)*P IMG(2)
Wherein, the point P in FIG. 6 is any point in the overlapping area A, θ is the included angle between the point P and the X axis, 0.ltoreq.alpha.ltoreq.1, P IMG (fusion) For the RGB value after point P fusion, P IMG(1) And P IMG(2) RGB values of pixel points in overlapping areas are respectively overlapped, and P in overlapping areas A IMG(1) Can be expressed as P IMG (front) ，P IMG(2) Can be expressed as P IMG (rear) The bird's eye view effect after stitching and fusion is shown in fig. 7.
In order to ensure the fusion effect, after the generation of the surrounding bird's-eye view is completed, verifying the spliced surrounding bird's-eye view; when the verification result meets the preset condition, the verification is passed; and when the verification result does not meet the preset condition, the generation of the aerial view is performed again. The preset conditions can be set by a person skilled in the art according to the need, in the verification method of the application, the markers are placed in the overlapped area of the two cameras, the markers can be checkerboard or the like, and then whether the markers have ghost phenomenon or whether the ghost phenomenon is serious is checked in the obtained look-around aerial view; when there is a ghost, the bird's eye view is generated again. In a specific example, as shown in fig. 8, the ghost phenomenon appears at the circle in fig. 8, if the precision of the generated surrounding view aerial view does not meet the requirement, the previous operation steps can be repeated to re-generate the aerial view, thereby ensuring the fusion accuracy and providing accurate environment sensing information for the vehicle.
The method for generating the panoramic aerial view of the vehicle-mounted panoramic camera provided by the embodiment of the application comprises a simple and flexible aerial view calibration scene, and a set of method for generating the panoramic aerial view of the vehicle-mounted panoramic camera is designed based on the scene. The calibration scene designed by the application has no strict limit on the size of the vehicle, and is particularly suitable for calibrating the external field environment. For the splicing and fusion of a plurality of aerial views, the method provided by the application has low consumption of computing resources and is easy to realize on an embedded platform.
Those of skill would further appreciate that the various illustrative elements and algorithm steps described in connection with the embodiments disclosed herein may be implemented as electronic hardware, computer software, or combinations of both, and that the various illustrative elements and steps are described above generally in terms of function in order to clearly illustrate the interchangeability of hardware and software. Whether such functionality is implemented as hardware or software depends upon the particular application and design constraints imposed on the solution. Skilled artisans may implement the described functionality in varying ways for each particular application, but such implementation decisions should not be interpreted as causing a departure from the scope of the present application.
The steps of a method or algorithm described in connection with the embodiments disclosed herein may be embodied in hardware, in a software module executed by a processor, or in a combination of the two. The software module may be placed in a random access memory (RA vehicle-mounted panoramic aerial view generation method), a memory, a read-only memory (RO vehicle-mounted panoramic aerial view generation method), an electrically programmable RO vehicle-mounted panoramic aerial view generation method, an electrically erasable programmable RO vehicle-mounted panoramic aerial view generation method, a register, a hard disk, a removable disk, a CD-RO vehicle-mounted panoramic aerial view generation method, or any other form of storage medium known in the art.
The foregoing description of the embodiments has been provided for the purpose of illustrating the general principles of the application, and is not meant to limit the scope of the application, but to limit the application to the particular embodiments, and any modifications, equivalents, improvements, etc. that fall within the spirit and principles of the application are intended to be included within the scope of the application.
Claims (9)
1. A panoramic aerial view generation method of a vehicle-mounted panoramic camera is characterized by comprising the following steps:
establishing a calibration scene according to the vehicle; a plurality of vehicle-mounted looking-around cameras are arranged on the vehicle;
setting corresponding checkerboards in the calibration scene according to the arrangement of the cameras; a plurality of corner points are arranged in the checkerboard;
obtaining a homography conversion matrix from each camera picture to a bird's eye view according to the calibration scene and the camera internal parameters;
converting a plurality of camera pictures into a bird's eye view by utilizing the homography conversion matrix;
fusing the overlapping areas in the aerial view, and outputting a spliced circular aerial view;
the fusing the overlapping areas in the aerial view includes: fusing RGB values of each pixel point of the overlapping area according to a preset algorithm;
the preset algorithm is as follows:
α＝θ/90°
P IMG (fusion) ＝α*P IMG(1) +(1-α)*P IMG(2)
Wherein θ is the angle between any point P in the overlapping area and the X axis, P IMG (fusion) For the RGB value after point P fusion, P IMG(1) And P IMG(2) RGB values for overlapping area pixels, respectively.
2. The method for generating a panoramic aerial view of a vehicle-mounted panoramic camera according to claim 1, wherein the establishing a calibration scene according to a vehicle specifically comprises:
and planning a vehicle space occupying frame according to the vehicle.
3. The method for generating a panoramic aerial view of a vehicle-mounted panoramic camera according to claim 2, wherein the setting of the corresponding checkerboard in the calibration scene according to the camera is specifically:
setting corresponding checkerboards in the calibration scene according to the camera and the vehicle space occupying frame.
4. The method for generating the panoramic aerial view of the vehicle-mounted all-round camera according to claim 2, wherein the vehicle-mounted all-round camera is respectively arranged around the vehicle, each camera corresponds to two chequers, and the distance between the two chequers corresponding to each camera is equal;
after the setting of the corresponding checkerboard in the calibration scene according to the camera and the vehicle placeholder, the method further comprises:
judging whether each camera can acquire two checkerboards or not;
if the vehicle placeholder cannot be rescheduled.
5. The method for generating a panoramic aerial view of an on-board panoramic camera as recited in claim 2, further comprising, after said setting a corresponding checkerboard in said calibration scene according to said camera:
obtaining a first configuration parameter according to the size of the vehicle space occupying frame;
obtaining a second configuration parameter according to the positions of the checkerboard;
and obtaining a third configuration parameter according to the positions of the vehicle space occupying frame and the checkerboard.
6. The method for generating a panoramic aerial view of a vehicle-mounted panoramic camera according to claim 5, wherein the obtaining a homography conversion matrix from each camera picture to the aerial view according to the calibration scene and the camera internal parameters specifically comprises:
acquiring a camera picture;
carrying out image distortion correction on the camera picture according to the camera internal parameters to obtain a corrected camera picture;
acquiring checkerboard corner points in the corrected camera picture, and acquiring predefined positions of the checkerboard corner points in the aerial view in the calibration scene;
and obtaining a homography conversion matrix from the camera picture to the aerial view according to the corrected camera picture, the predefined positions of the checkerboard corner points in the calibration scene in the aerial view, and the first configuration parameter, the second configuration parameter and the third configuration parameter.
7. The method for generating a panoramic aerial view of a vehicle-mounted pan-around camera according to claim 6, wherein the converting the plurality of camera pictures into the aerial view by using the homography conversion matrix specifically comprises:
and converting the corrected camera picture into a bird's eye view by utilizing the homography conversion matrix.
8. The method for generating a panoramic aerial view of an on-vehicle panoramic camera of claim 1, wherein after said outputting the stitched panoramic aerial view, the method further comprises:
verifying the spliced circular bird's eye view;
when the verification result meets the preset condition, the verification is passed;
and when the verification result does not meet the preset condition, the generation of the aerial view is performed again.
9. The method for generating a panoramic aerial view of a vehicle-mounted panoramic camera of claim 8, wherein verifying the stitched panoramic aerial view specifically comprises:
placing a marker in the overlapping area of the two cameras;
and checking whether the marker has ghost or not in the obtained surrounding bird's eye view.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910549926.9A CN110288527B (en) | 2019-06-24 | 2019-06-24 | Panoramic aerial view generation method of vehicle-mounted panoramic camera |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910549926.9A CN110288527B (en) | 2019-06-24 | 2019-06-24 | Panoramic aerial view generation method of vehicle-mounted panoramic camera |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN110288527A CN110288527A (en) | 2019-09-27 |
| CN110288527B true CN110288527B (en) | 2023-10-24 |
Family
ID=68004679
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201910549926.9A Active CN110288527B (en) | 2019-06-24 | 2019-06-24 | Panoramic aerial view generation method of vehicle-mounted panoramic camera |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN110288527B (en) |
Families Citing this family (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110910311B (en) * | 2019-10-30 | 2023-09-26 | 同济大学 | Automatic splicing method of multi-path looking-around camera based on two-dimension code |
| CN112069980B (en) * | 2020-09-03 | 2022-01-25 | 三一专用汽车有限责任公司 | Obstacle recognition method, obstacle recognition system, and storage medium |
| CN112070886B (en) * | 2020-09-04 | 2023-04-25 | 中车大同电力机车有限公司 | Image monitoring method and related equipment for mining dump truck |
| CN112802109B (en) * | 2021-02-07 | 2024-05-10 | 的卢技术有限公司 | Method for generating aerial view panorama of automobile |
| CN113611143B (en) * | 2021-07-29 | 2022-10-18 | 同致电子科技（厦门）有限公司 | Parking memory system and map building system thereof |
| CN114202588B (en) * | 2021-12-09 | 2022-09-23 | 纵目科技（上海）股份有限公司 | Method and device for quickly and automatically calibrating vehicle-mounted panoramic camera |
| CN116029900A (en) * | 2022-12-30 | 2023-04-28 | 小米汽车科技有限公司 | Image processing method, device, storage medium and vehicle |
| CN117237237B (en) * | 2023-11-13 | 2024-07-05 | 深圳元戎启行科技有限公司 | Luminosity balancing method and device for vehicle-mounted 360-degree panoramic image |
Citations (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US6009190A (en) * | 1997-08-01 | 1999-12-28 | Microsoft Corporation | Texture map construction method and apparatus for displaying panoramic image mosaics |
| CN102629372A (en) * | 2012-02-22 | 2012-08-08 | 北京工业大学 | 360 degree panoramic aerial view generation method used for assisting vehicle driving |
| CN103035005A (en) * | 2012-12-13 | 2013-04-10 | 广州致远电子股份有限公司 | Panorama parking calibration method and device, and automatic calibration method |
| CN105741296A (en) * | 2016-02-02 | 2016-07-06 | 大连楼兰科技股份有限公司 | Auxiliary calibration method of 360-degre all-visual-angle aerial view panorama travelling crane |
| CN105894549A (en) * | 2015-10-21 | 2016-08-24 | 乐卡汽车智能科技（北京）有限公司 | Panorama assisted parking system and device and panorama image display method |
| CN106373091A (en) * | 2016-09-05 | 2017-02-01 | 山东省科学院自动化研究所 | Automatic panorama parking aerial view image splicing method, system and vehicle |
| CN106952311A (en) * | 2017-03-02 | 2017-07-14 | 山东省科学院自动化研究所 | Auxiliary parking system and method based on panoramic mosaic data mapping tables |
| CN109064404A (en) * | 2018-08-10 | 2018-12-21 | 西安电子科技大学 | It is a kind of based on polyphaser calibration panorama mosaic method, panoramic mosaic system |
-
2019
- 2019-06-24 CN CN201910549926.9A patent/CN110288527B/en active Active
Patent Citations (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US6009190A (en) * | 1997-08-01 | 1999-12-28 | Microsoft Corporation | Texture map construction method and apparatus for displaying panoramic image mosaics |
| CN102629372A (en) * | 2012-02-22 | 2012-08-08 | 北京工业大学 | 360 degree panoramic aerial view generation method used for assisting vehicle driving |
| CN103035005A (en) * | 2012-12-13 | 2013-04-10 | 广州致远电子股份有限公司 | Panorama parking calibration method and device, and automatic calibration method |
| CN105894549A (en) * | 2015-10-21 | 2016-08-24 | 乐卡汽车智能科技（北京）有限公司 | Panorama assisted parking system and device and panorama image display method |
| CN105741296A (en) * | 2016-02-02 | 2016-07-06 | 大连楼兰科技股份有限公司 | Auxiliary calibration method of 360-degre all-visual-angle aerial view panorama travelling crane |
| CN106373091A (en) * | 2016-09-05 | 2017-02-01 | 山东省科学院自动化研究所 | Automatic panorama parking aerial view image splicing method, system and vehicle |
| CN106952311A (en) * | 2017-03-02 | 2017-07-14 | 山东省科学院自动化研究所 | Auxiliary parking system and method based on panoramic mosaic data mapping tables |
| CN109064404A (en) * | 2018-08-10 | 2018-12-21 | 西安电子科技大学 | It is a kind of based on polyphaser calibration panorama mosaic method, panoramic mosaic system |
Non-Patent Citations (1)
| Title |
|---|
| 基于双目鱼眼相机的柱状投影全景行车记录仪;李剑等;《电子测量技术》;20171031(第10期);全文 * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN110288527A (en) | 2019-09-27 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN110288527B (en) | Panoramic aerial view generation method of vehicle-mounted panoramic camera | |
| CN108257183B (en) | Camera lens optical axis calibration method and device | |
| JP5739584B2 (en) | 3D image synthesizing apparatus and method for visualizing vehicle periphery | |
| US20130141526A1 (en) | Apparatus and Method for Video Image Stitching | |
| KR100914211B1 (en) | Distorted image correction apparatus and method | |
| CN105488766B (en) | Fisheye image bearing calibration and device | |
| CN112224132A (en) | Vehicle panoramic all-around obstacle early warning method | |
| CN109785390B (en) | Method and device for image correction | |
| CN108200360A (en) | A kind of real-time video joining method of more fish eye lens panoramic cameras | |
| CN110505468B (en) | Test calibration and deviation correction method for augmented reality display equipment | |
| CN109741241A (en) | Processing method, device, equipment and the storage medium of fish eye images | |
| CN110689476A (en) | Panoramic image splicing method and device, readable storage medium and electronic equipment | |
| KR101705558B1 (en) | Top view creating method for camera installed on vehicle and AVM system | |
| CN111243034A (en) | Panoramic auxiliary parking calibration method, device, equipment and storage medium | |
| CN114549666B (en) | AGV-based panoramic image splicing calibration method | |
| CN109785225B (en) | Method and device for correcting image | |
| CN112465915A (en) | Vehicle-mounted panoramic system calibration method | |
| CN110400255B (en) | Vehicle panoramic image generation method and system and vehicle | |
| CN113989392A (en) | Color chessboard calibration method and device of splicing camera and camera | |
| CN107886039A (en) | Parking system panoramic view generation method and device | |
| CN111798374A (en) | Image splicing method, device, equipment and medium | |
| WO2020114433A1 (en) | Depth perception method and apparatus, and depth perception device | |
| CN113114975B (en) | Image splicing method and device, electronic equipment and storage medium | |
| CN110428361A (en) | A kind of multiplex image acquisition method based on artificial intelligence | |
| CN111815752B (en) | Image processing method and device and electronic equipment |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| CB03 | Change of inventor or designer information |
Inventor after: Zhang Rong Inventor after: Xiong Qi Inventor after: Zhang Fang Inventor after: Li Xiaofei Inventor after: Zhang Dezhao Inventor after: Wang Xiao Inventor after: Huo Shuhao Inventor before: Zhang Rong Inventor before: Xiong Qi Inventor before: Zhang Fang Inventor before: Li Xiaofei Inventor before: Zhang Dezhao Inventor before: Wang Xiao Inventor before: Huo Shuhao |
|
| CB03 | Change of inventor or designer information | ||
| CB02 | Change of applicant information |
Address after: B4-006, maker Plaza, 338 East Street, Huilongguan town, Changping District, Beijing 100096 Applicant after: Beijing Idriverplus Technology Co.,Ltd. Address before: B4-006, maker Plaza, 338 East Street, Huilongguan town, Changping District, Beijing 100096 Applicant before: Beijing Idriverplus Technology Co.,Ltd. |
|
| CB02 | Change of applicant information | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |