CN110689506A - Panoramic stitching method, automotive panoramic stitching method and panoramic system thereof - Google Patents
Panoramic stitching method, automotive panoramic stitching method and panoramic system thereof Download PDFInfo
- Publication number
- CN110689506A CN110689506A CN201910782573.7A CN201910782573A CN110689506A CN 110689506 A CN110689506 A CN 110689506A CN 201910782573 A CN201910782573 A CN 201910782573A CN 110689506 A CN110689506 A CN 110689506A
- Authority
- CN
- China
- Prior art keywords
- coordinate
- image
- displacement
- panoramic
- spliced
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000000034 method Methods 0.000 title claims abstract description 45
- 238000006073 displacement reaction Methods 0.000 claims abstract description 96
- 238000012545 processing Methods 0.000 claims abstract description 22
- 230000009466 transformation Effects 0.000 claims abstract description 21
- 230000000694 effects Effects 0.000 claims abstract description 11
- 238000012937 correction Methods 0.000 claims abstract description 10
- 239000011159 matrix material Substances 0.000 claims abstract description 10
- 230000008859 change Effects 0.000 claims abstract description 5
- 238000005286 illumination Methods 0.000 claims abstract description 5
- 230000004927 fusion Effects 0.000 claims abstract description 3
- 230000008569 process Effects 0.000 claims abstract description 3
- 239000004744 fabric Substances 0.000 claims description 13
- 238000004422 calculation algorithm Methods 0.000 claims description 12
- 238000003708 edge detection Methods 0.000 claims description 6
- 230000000007 visual effect Effects 0.000 claims description 2
- 230000008901 benefit Effects 0.000 description 7
- 238000007500 overflow downdraw method Methods 0.000 description 6
- 230000006872 improvement Effects 0.000 description 3
- 238000009434 installation Methods 0.000 description 3
- 230000009286 beneficial effect Effects 0.000 description 2
- 238000010586 diagram Methods 0.000 description 2
- 238000013507 mapping Methods 0.000 description 2
- 238000013519 translation Methods 0.000 description 2
- 230000007812 deficiency Effects 0.000 description 1
- 238000011161 development Methods 0.000 description 1
- 238000005516 engineering process Methods 0.000 description 1
- 230000006870 function Effects 0.000 description 1
- 238000003384 imaging method Methods 0.000 description 1
- 238000012423 maintenance Methods 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 238000005096 rolling process Methods 0.000 description 1
- 238000007790 scraping Methods 0.000 description 1
- 230000001131 transforming effect Effects 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T5/00—Image enhancement or restoration
- G06T5/40—Image enhancement or restoration using histogram techniques
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T3/00—Geometric image transformations in the plane of the image
- G06T3/04—Context-preserving transformations, e.g. by using an importance map
- G06T3/047—Fisheye or wide-angle transformations
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T5/00—Image enhancement or restoration
- G06T5/80—Geometric correction
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/10—Segmentation; Edge detection
- G06T7/13—Edge detection
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/90—Determination of colour characteristics
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/10—Image acquisition modality
- G06T2207/10004—Still image; Photographic image
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20212—Image combination
- G06T2207/20221—Image fusion; Image merging
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/30—Subject of image; Context of image processing
- G06T2207/30248—Vehicle exterior or interior
- G06T2207/30252—Vehicle exterior; Vicinity of vehicle
- G06T2207/30264—Parking
Landscapes
- Engineering & Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Theoretical Computer Science (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Image Processing (AREA)
- Studio Devices (AREA)
- Closed-Circuit Television Systems (AREA)
Abstract
The invention discloses a panoramic stitching method, a vehicle panoramic stitching method and a panoramic system thereof, wherein the panoramic stitching method comprises the following steps: s1, wide-angle cameras arranged at different positions acquire shot images; s2, distortion correction straightening processing is carried out on the shot image; s3, setting a calibration reference object in the acquisition range of the wide-angle camera, performing point tracing calibration by using the calibration reference object, and tracing out a special image coordinate on the calibration reference object to obtain a change matrix; s4, carrying out perspective transformation processing on the transformation matrix to obtain a multi-angle spliced image; s5, carrying out color balance and fusion on the spliced images to obtain spliced images with consistent color and illumination, and respectively calculating the relative displacement of each spliced image in the panoramic spliced area to obtain panoramic spliced images; a vehicle panoramic stitching method and a panoramic system thereof assist in the advancing process of a vehicle and have better stitching effect compared with the prior art.
Description
Technical Field
The invention relates to the field of panoramic stitching, in particular to a panoramic stitching method, an automotive panoramic stitching method and a panoramic system thereof.
Background
With the rapid development of social economy and science and technology, automobiles play more and more important roles in the life of people and are more and more popularized, however, with the wide popularization of automobiles, the problems of parking, narrow lane meeting, obstacle avoidance, reversing blind areas and the like are faced, the problem of accidents such as people rolling caused by rubbing and scraping and the blind areas is more and more prominent, the existing social resources cannot meet the requirements, particularly for people with insufficient driving experience, how to more easily and safely park the automobiles in the parking spaces is a very troublesome event for drivers, and therefore, the vehicle-mounted 360-degree panoramic parking system is brought into operation.
The necessary panorama parking system must use the scale to measure automobile body length during prior art's calibration, car width isoparametric just can mark, this need not many people cooperative operation alone, and can't be nimble after the mark to the concatenation imaging effect of certain camera stretch, zoom, rotation or displacement, the loaded down with trivial details of installation and the unable manual debugging of concatenation effect cause in case the concatenation effect is not good must move the camera lens angle and mark again, this not only wastes time economic benefits still inconvenient later maintenance, after the car owner traveles a period, because the car vibrations cause the slight skew of camera lens angle, the panorama image that four cameras spliced out this moment will appear the error, the function use of parking is influenced in the distortion.
Disclosure of Invention
Therefore, in order to solve the above-mentioned deficiencies, the present invention provides a panorama stitching method, comprising the following steps:
s1, wide-angle cameras arranged at different positions acquire shot images;
s2, distortion correction straightening processing is carried out on the shot image;
s3, setting a calibration reference object in the acquisition range of the wide-angle camera, performing point tracing calibration by using the calibration reference object, and tracing out a special image coordinate on the calibration reference object to obtain a change matrix;
s4, carrying out perspective transformation processing on the transformation matrix to obtain a multi-angle spliced image;
s5, carrying out color balance and fusion on the spliced images to obtain the spliced images with consistent color illumination, and respectively calculating the relative displacement of each spliced image in a panoramic spliced area to obtain a panoramic spliced image.
Preferably, the calibration reference is a calibration cloth.
Preferably, in step S5, the algorithm for calculating the relative displacement is as follows:
s1, obtaining coordinate points through image edge detection operator processing;
s2, making a screenshot processing formula according to the coordinate values corresponding to the panoramic mosaic image by the coordinate points:
stipulating: s represents a source graph and D represents a mosaic graph
S coordinate (X-displacement, Y-displacement) ═ D coordinate (X, Y)
And S3, obtaining the displacement amount according to the coordinate point of the splicing map and the coordinate point of the source map by means of displacement of S coordinate-D coordinate.
The panoramic stitching method for the vehicle comprises a plurality of wide-angle cameras arranged around the vehicle, and the panoramic stitching images around the vehicle are formed by the wide-angle cameras through the panoramic stitching method.
Preferably, the change matrix is subjected to perspective transformation to obtain an original splicing image, a partial range of the original splicing image is intercepted to obtain the splicing image, and the splicing image is slightly smaller than the original splicing image.
Preferably, the algorithm for calculating the relative displacement is as follows:
s1, obtaining 8 vertex coordinates (1, 2, 3, 4, 5, 6, 7 and 8) which are source image coordinates through image edge detection operator processing;
s2, making a screenshot processing formula by using coordinate values of the 1, 2, 3, 4, 5, 6, 7 and 8 points corresponding to the coordinate values on the spliced image:
stipulating: s represents a source graph and D represents a mosaic graph
S coordinate 1 (X-displacement, Y-displacement) ═ D coordinate 1(X, Y)
S coordinate 2 (X-displacement, Y-displacement) ═ D coordinate 2(X, Y)
S coordinate 3 (X-displacement, Y-displacement) ═ D coordinate 3(X, Y)
S coordinate 4 (X-displacement, Y-displacement) ═ D coordinate 4(X, Y)
S coordinate 5 (X-displacement, Y-displacement) ═ D coordinate 5(X, Y)
S coordinate 6 (X-displacement, Y-displacement) ═ D coordinate 6(X, Y)
S coordinate 7 (X-displacement, Y-displacement) ═ D coordinate 7(X, Y)
S coordinate 8 (X-displacement, Y-displacement) ═ D coordinate 8(X, Y)
And S3, obtaining a displacement amount according to the coordinate point of the spliced image and the coordinate point of the source image by means of displacement of the S coordinate-D coordinate.
Preferably, the stitching image can be moved, zoomed, rotated, stretched, tilted and unfolded to adjust the stitching position, so as to obtain the panoramic stitching image with better stitching effect.
A panoramic system based on the vehicular panoramic stitching method comprises a plurality of wide-angle cameras and a system host, wherein the wide-angle cameras and the system host are arranged around a vehicle; the system host comprises an image acquisition sensor, a display data random access memory and an embedded memory, the wide-angle camera is connected with the image acquisition sensor, the image acquisition sensor is connected with a processor, and the processor is connected with the display data random access memory and the embedded memory; the wide-angle camera is used for obtaining original image information; the image acquisition sensor converts image information into an electric signal and transmits the electric signal to the processor; the processor processes the received electric signals to obtain required image information, and then transmits the required image information to the display data random access memory, the processed image information is displayed in a visual mode, and the image information which needs to be stored for a long time is transmitted to the embedded memory to be stored.
The invention has the following advantages: the invention provides a panoramic stitching method, a vehicle panoramic stitching method and a panoramic system thereof through improvement, compared with the prior art, the invention has the following improvement and advantages;
the method has the advantages that: the invention adopts the calibration cloth for auxiliary calibration, the calibration cloth is easy to carry, when the invention is used for panoramic splicing around the vehicle, the calibration does not need to measure the length and the width of the vehicle body, and the operation is simpler.
The method has the advantages that: in the prior art, a mapping table of a coordinate system needs to be calculated, the panoramic image is spliced through the mapping table look-up, and the panoramic spliced image cannot be finely adjusted by means of moving, zooming, rotating, stretching, inclining, expanding and the like after splicing.
The method has the advantages that: compared with the prior art, the method does not need to extract the characteristic points (such as a shif algorithm, a surf algorithm and a harris algorithm) of two images to be spliced in a splicing mode and then carry out characteristic point matching (such as a RANSAC algorithm) so as to obtain a transformation matrix.
The advantages are that: the invention adopts the picture type splicing method, and the splicing effect problem caused by the installation of the camera or the splicing problem caused by the movement of the camera can be solved through moving, zooming, rotating, stretching, inclining, unfolding and the like at the later stage; the prior art cannot be adjusted once the device is installed.
Drawings
FIG. 1 is a schematic diagram of the panoramic system of the present invention;
FIG. 2 is a schematic perspective transformation of the present invention;
fig. 3 is a schematic diagram of a splicing parameter according to an embodiment of the present invention.
Wherein: 1-front-view fisheye camera, 2-rear-view fisheye camera, 3-left-view fisheye camera, 4-right-view fisheye camera, 5-image acquisition sensor, 6-processor, 7-display data random memory, 8-embedded memory and 9-calibration cloth
Detailed Description
The present invention will be described in detail with reference to fig. 1 to 3, and the technical solutions in the embodiments of the present invention will be clearly and completely described, and it is obvious that the described embodiments are only a part of the embodiments of the present invention, and not all embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.
Example one
As shown in fig. 1, 4 wide-angle cameras, specifically, a fisheye camera, are arranged at front, back, left, and right positions of the automobile, and have a viewing angle range of 200 degrees, and respectively include a front-view fisheye camera 1, a rear-view fisheye camera 2, a left-view fisheye camera 3, and a right-view fisheye camera 4, wherein the 4 fisheye cameras obtain original image information and transmit the original image information to an image acquisition sensor 5, and the image acquisition sensor 5 converts the image information into an electrical signal and transmits the electrical signal to a processor 6.
The processor 6 carries out distortion correction processing on image information acquired by all the fisheye cameras, and the fisheye lens distortion correction can obtain internal and external parameters of the lens through a Zhang calibration (Zhang Zhengyou professor) method and then correct the fisheye lens image to obtain an image which is beneficial to observation or use of human eyes.
The method comprises the steps that calibration cloth 9 is arranged on the front, the rear, the left and the right of an automobile respectively, patterns on the calibration cloth 9 are composed of black and white squares, accurate points are most easily selected due to obvious color difference of the black and white squares, appropriate points in the calibration cloth 9 in 4 images after distortion correction are selected for point tracing calibration, 4 matrixes including transformation matrixes M1, M2, M3 and M4 are obtained through point tracing calibration respectively, and the matrixes correspond to the front, the rear, the left and the right of the automobile respectively and correspond to 4 directions;
after perspective transformation processing is performed on the 4 transformation matrices M1, M2, M3, and M4, 4 stitched images are obtained, respectively, and referring to fig. 2, a general formula of the perspective transformation is as follows:
the transformed coordinates x, y are respectively: x ═ x ', y ═ y '/w '
After the expansion, the method comprises the following steps:
wherein,
is a perspective transformation matrix;
representing a linear transformation, [ a ]31a32]For translation.
The color balance and fusion method is adopted for 4 spliced images to ensure that the color illumination is consistent, the existing mature color balance and fusion method is adopted for the color balance and fusion method, and the processor 6 is used for respectively calculating the relative displacement of each image in the panoramic spliced area to obtain the panoramic spliced image, and the method comprises the following steps:
referring to fig. 3, the width of the E region is 100 pixels; the width of the F area is 300 pixels; the height of the G region is 120 pixels; the width and height of the H area are 60 pixels; the width of the I region is 100 pixels. The circle represents the ideal position for placing the front fisheye camera picture in the stitched image, and the assumption is that:
coordinate 1(X ═ 40, Y ═ 60)
Coordinate 2(X ═ 40, Y ═ 120)
Coordinate 3 (100X, 60Y)
Coordinate 4 (100X, 120Y)
Coordinate 5(X ═ 200, Y ═ 60)
Coordinate 6 (X-200, Y-120)
Coordinate 7(X ═ 260, Y ═ 60)
Coordinate 8 (X260, Y120)
The calculation algorithm is as follows:
8 vertex coordinates (1, 2, 3, 4, 5, 6, 7, 8) obtained by processing an image edge detection operator are source map coordinates,
and (3) making a screenshot processing formula by using coordinate values of 1, 2, 3, 4, 5, 6, 7 and 8 points corresponding to the coordinate values on the panoramic mosaic image:
stipulating: s represents a source graph and D represents a mosaic graph
S coordinate 1 (X-displacement, Y-displacement) ═ D coordinate 1(X, Y)
S coordinate 2 (X-displacement, Y-displacement) ═ D coordinate 2(X, Y)
S coordinate 3 (X-displacement, Y-displacement) ═ D coordinate 3(X, Y)
S coordinate 4 (X-displacement, Y-displacement) ═ D coordinate 4(X, Y)
S coordinate 5 (X-displacement, Y-displacement) ═ D coordinate 5(X, Y)
S coordinate 6 (X-displacement, Y-displacement) ═ D coordinate 6(X, Y)
S coordinate 7 (X-displacement, Y-displacement) ═ D coordinate 7(X, Y)
S coordinate 8 (X-displacement, Y-displacement) ═ D coordinate 8(X, Y)
Since the coordinate points of the stitched image are known and the coordinates of the source map are known, the displacement is obtained by using the S coordinate-D coordinate, and assuming that the coordinate value of point 1 in the source map is X300 and Y250, the displacement is obtained by converting the formula
The displacement is obtained by using the S-coordinate 1 (X-300, Y-250) and the D-coordinate 1 (X-40, Y-60), the X-displacement is 260, and the Y-displacement is 190.
The processor transmits the spliced image and the panoramic spliced image to a display data random access memory 7 to visually display image information; the image information that needs to be stored for a long time is transferred to the embedded memory 8 for backup storage. Because of adopting the picture type splicing mode, if the algorithm splices errors, the image position can be adjusted through manual movement, scaling and rotation to achieve the best splicing effect.
Example two
As shown in fig. 1, 4 wide-angle cameras, specifically, a fisheye camera, are arranged at front, back, left, and right positions of the automobile, and have a viewing angle range of 200 degrees, and respectively include a front-view fisheye camera 1, a rear-view fisheye camera 2, a left-view fisheye camera 3, and a right-view fisheye camera 4, wherein the 4 fisheye cameras obtain original image information and transmit the original image information to an image acquisition sensor 5, and the image acquisition sensor 5 converts the image information into an electrical signal and transmits the electrical signal to a processor 6.
The processor 6 carries out distortion correction processing on image information acquired by all the fisheye cameras, and the fisheye lens distortion correction can obtain internal and external parameters of the lens through a Zhang calibration (Zhang Zhengyou professor) method and then correct the fisheye lens image to obtain an image which is beneficial to observation or use of human eyes.
Obtain the concatenation original image after carrying out distortion correction and handling, the certain range that can intercept the concatenation original image is as the concatenation image, and the outside as concatenation compensation area of scope, when because camera mounted position is not right or when taking place to remove, its concatenation position of accessible removal, zoom, rotation, flexible, slope, expansion adjustment can realize having certain adjustment degree of advance under the camera condition of not adjusting, because the wide-angle camera that uses, so can set up such concatenation compensation area.
The method comprises the steps that calibration cloth 9 is arranged on the front, the rear, the left and the right of an automobile respectively, patterns on the calibration cloth 9 are composed of black and white squares, accurate points are most easily selected due to obvious color difference of the black and white squares, appropriate points in the calibration cloth 9 in 4 images after distortion correction are selected for point tracing calibration, 4 matrixes including transformation matrixes M1, M2, M3 and M4 are obtained through point tracing calibration respectively, and the matrixes correspond to the front, the rear, the left and the right of the automobile respectively and correspond to 4 directions;
after perspective transformation processing is performed on the 4 transformation matrices M1, M2, M3, and M4, 4 stitched images are obtained, respectively, and referring to fig. 2, a general formula of the perspective transformation is as follows:
the transformed coordinates x, y are respectively: x ═ x ', y ═ y '/w '
After the expansion, the method comprises the following steps:
wherein,
transforming moments for perspectiveArraying;
representing a linear transformation, [ a ]31a32]For translation.
The color balance and fusion method is adopted for 4 spliced images to ensure that the color illumination is consistent, the existing mature color balance and fusion method is adopted for the color balance and fusion method, and the processor 6 is used for respectively calculating the relative displacement of each image in the panoramic spliced area to obtain the panoramic spliced image, and the method comprises the following steps:
referring to fig. 3, the width of the E region is 100 pixels; the width of the F area is 300 pixels; the height of the G region is 120 pixels; the width and height of the H area are 60 pixels; the width of the I region is 100 pixels. The circle represents the ideal position for placing the front fisheye camera picture in the stitched image, and the assumption is that:
coordinate 1(X ═ 40, Y ═ 60)
Coordinate 2(X ═ 40, Y ═ 120)
Coordinate 3 (100X, 60Y)
Coordinate 4 (100X, 120Y)
Coordinate 5(X ═ 200, Y ═ 60)
Coordinate 6 (X-200, Y-120)
Coordinate 7(X ═ 260, Y ═ 60)
Coordinate 8 (X260, Y120)
The calculation algorithm is as follows:
8 vertex coordinates (1, 2, 3, 4, 5, 6, 7, 8) obtained by processing an image edge detection operator are source map coordinates,
and (3) making a screenshot processing formula by using coordinate values of 1, 2, 3, 4, 5, 6, 7 and 8 points corresponding to the coordinate values on the panoramic mosaic image:
stipulating: s represents a source graph and D represents a mosaic graph
S coordinate 1 (X-displacement, Y-displacement) ═ D coordinate 1(X, Y)
S coordinate 2 (X-displacement, Y-displacement) ═ D coordinate 2(X, Y)
S coordinate 3 (X-displacement, Y-displacement) ═ D coordinate 3(X, Y)
S coordinate 4 (X-displacement, Y-displacement) ═ D coordinate 4(X, Y)
S coordinate 5 (X-displacement, Y-displacement) ═ D coordinate 5(X, Y)
S coordinate 6 (X-displacement, Y-displacement) ═ D coordinate 6(X, Y)
S coordinate 7 (X-displacement, Y-displacement) ═ D coordinate 7(X, Y)
S coordinate 8 (X-displacement, Y-displacement) ═ D coordinate 8(X, Y)
Since the coordinate points of the stitched image are known and the coordinates of the source map are known, the displacement is obtained by using the S coordinate-D coordinate, and assuming that the coordinate value of point 1 in the source map is X300 and Y250, the displacement is obtained by converting the formula
The displacement is obtained by using the S-coordinate 1 (X-300, Y-250) and the D-coordinate 1 (X-40, Y-60), the X-displacement is 260, and the Y-displacement is 190.
The processor transmits the spliced image and the panoramic spliced image to a display data random access memory 7 to visually display image information; the image information that needs to be stored for a long time is transferred to the embedded memory 8 for backup storage. Because of adopting the picture type splicing mode, if the algorithm splices errors, the image position can be adjusted through manual movement, scaling and rotation to achieve the best splicing effect.
The invention provides a panoramic splicing method, a vehicular panoramic splicing method and a panoramic system thereof through improvement, the invention adopts calibration cloth for auxiliary calibration, the calibration cloth is easy to carry, when the invention is used for panoramic splicing of images around automobiles, the calibration does not need to measure the length and width of an automobile body, and the operation is simpler; the invention can realize the fine adjustment of the panoramic mosaic image and achieve better mosaic effect; the method does not depend on the matching of the feature points, has lower requirement on the environment and has universality; the invention adopts the picture type splicing method, and can solve the problem of splicing effect caused by the installation of the camera or the splicing problem caused by the movement of the camera through moving, zooming, rotating, stretching, inclining, unfolding and the like at the later stage.
The previous description of the disclosed embodiments is provided to enable any person skilled in the art to make or use the present invention. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the generic principles defined herein may be applied to other embodiments without departing from the spirit or scope of the invention. Thus, the present invention is not intended to be limited to the embodiments shown herein but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.
Claims (8)
1. A panoramic stitching method comprises the following steps:
s1, wide-angle cameras arranged at different positions acquire shot images;
s2, distortion correction straightening processing is carried out on the shot image;
s3, setting a calibration reference object in the acquisition range of the wide-angle camera, performing point tracing calibration by using the calibration reference object, and tracing out a special image coordinate on the calibration reference object to obtain a change matrix;
s4, carrying out perspective transformation processing on the transformation matrix to obtain a multi-angle spliced image;
s5, carrying out color balance and fusion on the spliced images to obtain the spliced images with consistent color illumination, and respectively calculating the relative displacement of each spliced image in a panoramic spliced area to obtain a panoramic spliced image.
2. The panorama stitching method according to claim 1, wherein: the calibration reference object is a calibration cloth.
3. The panorama stitching method according to claim 1, wherein: in step S5, the algorithm for calculating the relative displacement is as follows:
s1, obtaining a source graph coordinate point through image edge detection operator processing;
s2, making a screenshot processing formula according to the coordinate values corresponding to the stitched image by the source image coordinate values:
stipulating: s represents a source graph and D represents a mosaic graph
S coordinate (X-displacement, Y-displacement) ═ D coordinate (X, Y)
And S3, obtaining a displacement amount according to the coordinate point of the spliced image and the coordinate point of the source image by means of displacement of the S coordinate-D coordinate.
4. A panoramic stitching method for a vehicle, which comprises a plurality of wide-angle cameras arranged around the vehicle, wherein the wide-angle cameras form a panoramic stitching image around the vehicle by the panoramic stitching method of any one of claims 1 to 3.
5. The vehicle panoramic stitching method according to claim 4, characterized in that: and performing perspective transformation on the change matrix to obtain an original splicing image, and intercepting partial range of the original splicing image to obtain the splicing image, wherein the splicing image is slightly smaller than the original splicing image.
6. The vehicle panoramic stitching method according to claim 4, characterized in that: the algorithm for calculating the relative displacement is as follows:
s1, obtaining 8 vertex coordinates (1, 2, 3, 4, 5, 6, 7 and 8) which are source image coordinates through image edge detection operator processing;
s2, making a screenshot processing formula by using coordinate values of the 1, 2, 3, 4, 5, 6, 7 and 8 points corresponding to the coordinate values on the spliced image:
stipulating: s represents a source graph and D represents a mosaic graph
S coordinate 1 (X-displacement, Y-displacement) ═ D coordinate 1(X, Y)
S coordinate 2 (X-displacement, Y-displacement) ═ D coordinate 2(X, Y)
S coordinate 3 (X-displacement, Y-displacement) ═ D coordinate 3(X, Y)
S coordinate 4 (X-displacement, Y-displacement) ═ D coordinate 4(X, Y)
S coordinate 5 (X-displacement, Y-displacement) ═ D coordinate 5(X, Y)
S coordinate 6 (X-displacement, Y-displacement) ═ D coordinate 6(X, Y)
S coordinate 7 (X-displacement, Y-displacement) ═ D coordinate 7(X, Y)
S coordinate 8 (X-displacement, Y-displacement) ═ D coordinate 8(X, Y)
And S3, obtaining a displacement amount according to the coordinate point of the spliced image and the coordinate point of the source image by means of displacement of the S coordinate-D coordinate.
7. The vehicle panoramic stitching method according to claim 5, characterized in that: the splicing position of the spliced image can be adjusted through moving, zooming, rotating, stretching, inclining and unfolding, and a panoramic spliced image with a better splicing effect is obtained.
8. A panoramic system based on the vehicular panoramic stitching method of any one of claims 4 to 7 comprises a plurality of wide-angle cameras and a system host which are arranged around a vehicle; the system host comprises an image acquisition sensor, a display data random access memory and an embedded memory, the wide-angle camera is connected with the image acquisition sensor, the image acquisition sensor is connected with a processor, and the processor is connected with the display data random access memory and the embedded memory; the wide-angle camera is used for obtaining original image information; the image acquisition sensor converts image information into an electric signal and transmits the electric signal to the processor; the processor processes the received electric signals to obtain required image information, and then transmits the required image information to the display data random access memory, the processed image information is displayed in a visual mode, and the image information which needs to be stored for a long time is transmitted to the embedded memory to be stored.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910782573.7A CN110689506A (en) | 2019-08-23 | 2019-08-23 | Panoramic stitching method, automotive panoramic stitching method and panoramic system thereof |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910782573.7A CN110689506A (en) | 2019-08-23 | 2019-08-23 | Panoramic stitching method, automotive panoramic stitching method and panoramic system thereof |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN110689506A true CN110689506A (en) | 2020-01-14 |
Family
ID=69108491
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201910782573.7A Pending CN110689506A (en) | 2019-08-23 | 2019-08-23 | Panoramic stitching method, automotive panoramic stitching method and panoramic system thereof |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN110689506A (en) |
Cited By (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111915537A (en) * | 2020-08-13 | 2020-11-10 | 歌尔光学科技有限公司 | Image processing method and device, image acquisition device and readable storage medium |
| CN111970537A (en) * | 2020-08-04 | 2020-11-20 | 威海精讯畅通电子科技有限公司 | Root system scanning image processing method and system |
| CN112581371A (en) * | 2021-01-27 | 2021-03-30 | 仲恺农业工程学院 | Panoramic real-time imaging splicing method based on novel structure of four-way camera |
| CN112672077A (en) * | 2020-12-14 | 2021-04-16 | 深圳市富中奇科技有限公司 | Panoramic image display method, liquid crystal instrument and storage medium |
| CN112738382A (en) * | 2021-01-25 | 2021-04-30 | 广州敏视数码科技有限公司 | Vehicle head and vehicle body panoramic all-around image splicing method |
| CN113191974A (en) * | 2021-04-29 | 2021-07-30 | 青岛科技大学 | Method and system for obtaining ship panoramic image based on machine vision |
| CN114289332A (en) * | 2022-01-20 | 2022-04-08 | 湖南视比特机器人有限公司 | Visual identification and positioning method and device for workpiece sorting and sorting system |
Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103035005A (en) * | 2012-12-13 | 2013-04-10 | 广州致远电子股份有限公司 | Panorama parking calibration method and device, and automatic calibration method |
| CN103139479A (en) * | 2013-02-25 | 2013-06-05 | 广东欧珀移动通信有限公司 | Method and device for finishing panorama preview scanning |
| CN104732542A (en) * | 2015-03-27 | 2015-06-24 | 安徽省道一电子科技有限公司 | Image processing method for panoramic vehicle safety system based on multi-camera self calibration |
| CN107492125A (en) * | 2017-07-28 | 2017-12-19 | 哈尔滨工业大学深圳研究生院 | The processing method of automobile fish eye lens panoramic view picture |
| CN108230248A (en) * | 2018-01-23 | 2018-06-29 | 深圳普捷利科技有限公司 | A kind of assessment of viewing system splicing effect and automatic fine tuning method based on self-adaptive features point registration |
| US20190068877A1 (en) * | 2017-08-28 | 2019-02-28 | Boe Technology Group Co., Ltd. | Mobile terminal image synthesis method, mobile terminal image synthesis apparatus and mobile terminal |
-
2019
- 2019-08-23 CN CN201910782573.7A patent/CN110689506A/en active Pending
Patent Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103035005A (en) * | 2012-12-13 | 2013-04-10 | 广州致远电子股份有限公司 | Panorama parking calibration method and device, and automatic calibration method |
| CN103139479A (en) * | 2013-02-25 | 2013-06-05 | 广东欧珀移动通信有限公司 | Method and device for finishing panorama preview scanning |
| CN104732542A (en) * | 2015-03-27 | 2015-06-24 | 安徽省道一电子科技有限公司 | Image processing method for panoramic vehicle safety system based on multi-camera self calibration |
| CN107492125A (en) * | 2017-07-28 | 2017-12-19 | 哈尔滨工业大学深圳研究生院 | The processing method of automobile fish eye lens panoramic view picture |
| US20190068877A1 (en) * | 2017-08-28 | 2019-02-28 | Boe Technology Group Co., Ltd. | Mobile terminal image synthesis method, mobile terminal image synthesis apparatus and mobile terminal |
| CN108230248A (en) * | 2018-01-23 | 2018-06-29 | 深圳普捷利科技有限公司 | A kind of assessment of viewing system splicing effect and automatic fine tuning method based on self-adaptive features point registration |
Non-Patent Citations (1)
| Title |
|---|
| 傅军栋等: "《实景图像拼接及其漫游控制技术》", 西南交通大学出版社, pages: 100 - 102 * |
Cited By (11)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111970537A (en) * | 2020-08-04 | 2020-11-20 | 威海精讯畅通电子科技有限公司 | Root system scanning image processing method and system |
| CN111970537B (en) * | 2020-08-04 | 2023-02-28 | 威海精讯畅通电子科技有限公司 | Root system scanning image processing method and system |
| CN111915537A (en) * | 2020-08-13 | 2020-11-10 | 歌尔光学科技有限公司 | Image processing method and device, image acquisition device and readable storage medium |
| CN112672077A (en) * | 2020-12-14 | 2021-04-16 | 深圳市富中奇科技有限公司 | Panoramic image display method, liquid crystal instrument and storage medium |
| CN112672077B (en) * | 2020-12-14 | 2024-03-15 | 深圳市富中奇科技有限公司 | Panoramic image display method, liquid crystal instrument and storage medium |
| CN112738382A (en) * | 2021-01-25 | 2021-04-30 | 广州敏视数码科技有限公司 | Vehicle head and vehicle body panoramic all-around image splicing method |
| CN112738382B (en) * | 2021-01-25 | 2022-07-12 | 广州敏视数码科技有限公司 | Vehicle head and vehicle body panoramic all-around image splicing method |
| CN112581371A (en) * | 2021-01-27 | 2021-03-30 | 仲恺农业工程学院 | Panoramic real-time imaging splicing method based on novel structure of four-way camera |
| CN112581371B (en) * | 2021-01-27 | 2022-03-22 | 仲恺农业工程学院 | Panoramic real-time imaging splicing method based on novel structure of four-way camera |
| CN113191974A (en) * | 2021-04-29 | 2021-07-30 | 青岛科技大学 | Method and system for obtaining ship panoramic image based on machine vision |
| CN114289332A (en) * | 2022-01-20 | 2022-04-08 | 湖南视比特机器人有限公司 | Visual identification and positioning method and device for workpiece sorting and sorting system |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN110689506A (en) | Panoramic stitching method, automotive panoramic stitching method and panoramic system thereof | |
| CN109741455B (en) | Vehicle-mounted stereoscopic panoramic display method, computer readable storage medium and system | |
| CN108263283B (en) | Method for calibrating and splicing panoramic all-round looking system of multi-marshalling variable-angle vehicle | |
| CN111369439B (en) | Panoramic all-around image real-time splicing method for automatic parking space identification based on all-around | |
| US9858639B2 (en) | Imaging surface modeling for camera modeling and virtual view synthesis | |
| CN110264395B (en) | Lens calibration method and related device of vehicle-mounted monocular panoramic system | |
| JP5739584B2 (en) | 3D image synthesizing apparatus and method for visualizing vehicle periphery | |
| KR101150546B1 (en) | Vehicle periphery monitoring device | |
| JP4560716B2 (en) | Vehicle periphery monitoring system | |
| CN107888894B (en) | A kind of solid is vehicle-mounted to look around method, system and vehicle-mounted control device | |
| JP4975592B2 (en) | Imaging device | |
| CN113362228A (en) | Method and system for splicing panoramic images based on improved distortion correction and mark splicing | |
| CN103839227B (en) | Fisheye image correcting method and device | |
| CN112348741B (en) | Panoramic image stitching method, panoramic image stitching equipment, storage medium, panoramic image display method and panoramic image display system | |
| CN102298771A (en) | Fish-eye image rapid correction method of panoramic parking auxiliary system | |
| CN110400255B (en) | Vehicle panoramic image generation method and system and vehicle | |
| CN106994936A (en) | A kind of 3D panoramic parking assist systems | |
| CN108596982A (en) | A kind of easy vehicle-mounted multi-view camera viewing system scaling method and device | |
| CN112070886B (en) | Image monitoring method and related equipment for mining dump truck | |
| CN111652937A (en) | Vehicle-mounted camera calibration method and device | |
| CN109131082B (en) | Monocular panoramic parking image system completely based on vision and parking method thereof | |
| CN106855999A (en) | The generation method and device of automobile panoramic view picture | |
| CN111768332A (en) | Splicing method of vehicle-mounted all-around real-time 3D panoramic image and image acquisition device | |
| CN106846243A (en) | The method and device of three dimensional top panorama sketch is obtained in equipment moving process | |
| CN107364393A (en) | Display methods, device, storage medium and the electronic equipment of vehicle rear view image |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| RJ01 | Rejection of invention patent application after publication |
Application publication date: 20200114 |
|
| RJ01 | Rejection of invention patent application after publication |