CN116912091A - Binocular fisheye image stitching method and system - Google Patents
Binocular fisheye image stitching method and system Download PDFInfo
- Publication number
- CN116912091A CN116912091A CN202310855895.6A CN202310855895A CN116912091A CN 116912091 A CN116912091 A CN 116912091A CN 202310855895 A CN202310855895 A CN 202310855895A CN 116912091 A CN116912091 A CN 116912091A
- Authority
- CN
- China
- Prior art keywords
- image
- grid
- stitching
- fisheye
- resolution
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000000034 method Methods 0.000 title claims abstract description 62
- 238000012937 correction Methods 0.000 claims abstract description 11
- 230000000750 progressive effect Effects 0.000 claims description 22
- 230000009466 transformation Effects 0.000 claims description 21
- 230000006870 function Effects 0.000 claims description 19
- 230000004913 activation Effects 0.000 claims description 12
- 238000011176 pooling Methods 0.000 claims description 10
- 238000003702 image correction Methods 0.000 claims description 8
- 230000000452 restraining effect Effects 0.000 claims description 8
- 238000002679 ablation Methods 0.000 claims description 6
- 238000001514 detection method Methods 0.000 claims description 6
- 238000013519 translation Methods 0.000 claims description 6
- 230000009191 jumping Effects 0.000 claims description 4
- 238000012545 processing Methods 0.000 abstract description 4
- 239000011159 matrix material Substances 0.000 description 8
- 238000010586 diagram Methods 0.000 description 5
- 238000006243 chemical reaction Methods 0.000 description 4
- 239000004973 liquid crystal related substance Substances 0.000 description 4
- 230000000694 effects Effects 0.000 description 3
- 238000003384 imaging method Methods 0.000 description 3
- 230000003044 adaptive effect Effects 0.000 description 2
- 238000013461 design Methods 0.000 description 2
- 239000000284 extract Substances 0.000 description 2
- 238000000605 extraction Methods 0.000 description 2
- 238000013507 mapping Methods 0.000 description 2
- 239000002699 waste material Substances 0.000 description 2
- 238000004364 calculation method Methods 0.000 description 1
- 230000007547 defect Effects 0.000 description 1
- 230000007123 defense Effects 0.000 description 1
- 238000011161 development Methods 0.000 description 1
- 238000012544 monitoring process Methods 0.000 description 1
- 238000012805 post-processing Methods 0.000 description 1
- 230000008569 process Effects 0.000 description 1
- 230000003068 static effect Effects 0.000 description 1
- 230000000007 visual effect Effects 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T3/00—Geometric image transformations in the plane of the image
- G06T3/40—Scaling of whole images or parts thereof, e.g. expanding or contracting
- G06T3/4038—Image mosaicing, e.g. composing plane images from plane sub-images
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/0464—Convolutional networks [CNN, ConvNet]
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/048—Activation functions
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/08—Learning methods
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/60—Analysis of geometric attributes
- G06T7/66—Analysis of geometric attributes of image moments or centre of gravity
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/80—Analysis of captured images to determine intrinsic or extrinsic camera parameters, i.e. camera calibration
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2200/00—Indexing scheme for image data processing or generation, in general
- G06T2200/32—Indexing scheme for image data processing or generation, in general involving image mosaicing
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20081—Training; Learning
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20084—Artificial neural networks [ANN]
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02T—CLIMATE CHANGE MITIGATION TECHNOLOGIES RELATED TO TRANSPORTATION
- Y02T10/00—Road transport of goods or passengers
- Y02T10/10—Internal combustion engine [ICE] based vehicles
- Y02T10/40—Engine management systems
Landscapes
- Engineering & Computer Science (AREA)
- Physics & Mathematics (AREA)
- Theoretical Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Molecular Biology (AREA)
- Evolutionary Computation (AREA)
- Health & Medical Sciences (AREA)
- Life Sciences & Earth Sciences (AREA)
- Artificial Intelligence (AREA)
- Biomedical Technology (AREA)
- Biophysics (AREA)
- Computational Linguistics (AREA)
- Data Mining & Analysis (AREA)
- Software Systems (AREA)
- General Health & Medical Sciences (AREA)
- Mathematical Physics (AREA)
- Computing Systems (AREA)
- General Engineering & Computer Science (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Geometry (AREA)
- Image Processing (AREA)
Abstract
The invention discloses a binocular fisheye image stitching method and a binocular fisheye image stitching system, belongs to the technical field of image processing, and aims to solve the technical problems that panoramic image stitching existing in binocular fisheye image stitching is long in time consumption and double images appear in panoramic image stitching. The method comprises the following steps: simultaneously acquiring two fisheye images through a binocular fisheye lens; acquiring parameters of a camera through a camera calibration method, and carrying out distortion correction on each fish-eye graph based on the parameters of the camera; performing image rough alignment and image reconstruction on the corrected fish-eye image based on the image stitching frame; the stitched image is rectangular by warping the stitched image from the predicted initial grid to the predefined target grid, resulting in a rectangular image.
Description
Technical Field
The invention relates to the technical field of image processing, in particular to a binocular fisheye image splicing method and system.
Background
Along with the development of science and technology, panoramic vision is a technical concept of extremely hot at present, and is widely applied in the field of virtual reality, and can influence the aspects of our life, and the application of panoramic vision is more and more extensive, including the application in the aspects of national defense, rescue, safety monitoring, medical treatment and the like. The premise of achieving omnibearing vision is to acquire a panoramic image containing all surrounding scene information. Such images are generally taken by expensive professional equipment and are obtained by complex post-processing, which is quite unfavorable for the use and spread of the general public. Thus, the panoramic picture is more conveniently acquired, which is a hot spot in the field of panoramic vision.
Panoramic image is taken as a live-action picture capable of providing a user with ultra-large field of view observation, wherein the generation of the panoramic image based on image stitching mainly comprises the following two modes:
(1) Specially-manufactured equipment adopting multiple lenses: the equipment is generally expensive, and the common public often cannot bear the equipment and is not suitable for popularization;
(2) Collecting a plurality of images through a single lens, splicing and the like: the method is complex in operation, low in efficiency and low in applicability in practical engineering application, and is only suitable for static scenes.
The fisheye lens has a wider visual angle than a common camera, and in order to simplify a camera placing device model for shooting a covering panorama in multiple directions, the binocular camera simultaneously shoots and acquires two fisheye images of the covering panorama in opposite directions, and a series of image processing processes are carried out to splice panoramic images. To obtain a better panoramic image, the binocular fisheye image needs to be corrected and spliced, which is a difficulty affecting the practical application of the fisheye lens.
The current fish-eye correction mode is mainly divided into a projection transformation model-based method and a calibration-based method. Based on spherical perspective projection, the objective function is optimized by fitting a polynomial, so that the correction model parameters are estimated, the corrected image is deduced, and the method is complex in calculation and poor in instantaneity. The calibration-based correction method mainly calibrates internal and external parameters of the fisheye image by means of external equipment, and realizes the fisheye image distortion correction by means of coordinate conversion between real coordinates and fisheye imaging plane coordinates.
How to solve the problems of long time consumption and double image in panoramic image stitching existing in binocular fisheye image stitching is a technical problem to be solved.
Disclosure of Invention
The technical task of the invention is to provide the binocular fisheye image splicing method and the binocular fisheye image splicing system aiming at the defects, so as to solve the technical problems of long time consumption of panoramic image splicing and double image occurrence in panoramic image splicing in binocular fisheye image splicing.
In a first aspect, the invention provides a binocular fisheye image stitching method, which comprises the following steps:
and (3) image acquisition: simultaneously acquiring two fisheye images through a binocular fisheye lens;
and (3) correcting an image: acquiring parameters of a camera through a camera calibration method, and carrying out distortion correction on each fisheye image based on the parameters of the camera to obtain corrected fisheye images;
and (3) image stitching: constructing an image stitching frame based on an unsupervised rough alignment method and an unsupervised image reconstruction method, and performing image rough alignment and image reconstruction on corrected fisheye images based on the image stitching frame to obtain stitched images;
image squaring: predicting an initial grid based on a complete convolution network and a residual error progressive regression strategy, predefining a target grid with a rigid shape, and squaring the spliced image in a mode of warping the spliced image from the predicted initial grid to the predefining target grid to obtain a rectangular image.
Preferably, the parameters of the camera are obtained through a camera calibration method, which comprises the following steps:
selecting a point as an origin of a world coordinate system, and constructing the world coordinate system;
for the fish-eye image, obtaining a three-dimensional coordinate of each pixel point in the fish-eye image in a world coordinate system through an image detection algorithm;
converting the three-dimensional coordinates of each pixel point into pixel coordinates through a camera coordinate system and an image plane coordinate system after rigid transformation and projection;
for the pixel coordinates of each pixel point, the position of the center of the image is determined, and the position is converted into a final pixel coordinate system through translation.
Preferably, the image rough alignment is performed on the corrected fisheye image based on the image stitching frame, and the method comprises the following steps:
constraining an unsupervised homography network based on ablation and introducing a stitching domain transformation layer for reducing image size;
for each corrected fisheye image, extracting the characteristics of the corrected fisheye image through a convolution network, taking the extracted characteristics as input, estimating the homography of the image through an unsupervised homography network, and obtaining a distorted image by distorting the corrected image and performing coarse alignment;
And putting the distorted image into a stitching domain transformation layer to reduce redundant black pixels in the image, so as to obtain a final distorted image.
Preferably, when the corrected fisheye image is reconstructed based on the image splicing frame, the double image of the characteristic to the pixel is eliminated through an unsupervised reconstruction network, the reconstruction network comprises a low-resolution deformation branch network and a high-resolution thinning branch network, the deformation rule of image splicing is learned through the low-resolution deformation branch network, and the resolution of the spliced image is improved through the high-resolution thinning branch network;
learning a deformation rule of image stitching through a low-resolution deformed branch network, comprising the following operations:
downsampling the warped image to reduce the resolution of the image to obtain a downsampled image;
inputting the downsampled image into an encoder-decoder structure consisting of a pooling layer and a deconvolution layer;
jumping connection is carried out through the encoder-decoder structure to deform the downsampled image, so that a content mask and a seam mask of the downsampled image are obtained, the content mask is used for restraining characteristics of the reconstructed image to be close to a distorted image, the seam mask is used for restraining the edge of an overlapped area to be natural and continuous, and the overlapped area is an overlapped area between two fisheye images;
The high-resolution refinement branch network consists of a convolution layer and a resource block, wherein the resource block consists of the convolution layer, a relu activation function, the convolution layer, a sum function and the relu activation function in sequence;
and calculating the content loss and the seam loss of the distorted image under the high-resolution thinning branch network, and thinning the spliced image by combining the content mask and the seam mask of the distorted image under the low resolution to obtain the spliced image of the two distorted fish-eye images.
Preferably, the stitched image is rectangular by warping the stitched image from the predicted initial grid to a predefined target grid, comprising the steps of:
taking the spliced image and a mask corresponding to the spliced image as input, and extracting advanced semantic features from the input through a feature extractor to obtain a feature image, wherein the feature extractor consists of stacked convolution pool blocks;
predicting horizontal motion and vertical motion of each vertex in the characteristic image based on a conventional grid through a grid motion regressor, wherein the grid motion regressor is of a full convolution structure;
constructing two regressors with the same structure based on a residual regression strategy, namely a first regressor and a second regressor, wherein the first regressor is used for predicting primary grid motion in a progressive manner, and the second regressor is used for predicting residual grid motion in a progressive manner;
And predicting primary grid motion through a first regression device, carrying out grid deformation on the feature map, predicting residual grid motion through a second regression device, carrying out grid deformation on the distorted feature map, and maintaining the shape of deformed grids based on intra-grid constraints and inter-grid constraints during grid deformation to obtain a rectangular feature map.
In a second aspect, the present invention provides a binocular fisheye image stitching system for stitching images by a binocular fisheye image stitching method according to any one of the first aspect, the system comprising:
the image acquisition module is used for simultaneously acquiring two fisheye images through the binocular fisheye lens;
the image correction module is used for obtaining parameters of a camera through a camera calibration method, and carrying out distortion correction on each fisheye graph based on the parameters of the camera to obtain corrected fisheye images;
the image stitching module is used for constructing an image stitching frame based on an unsupervised rough alignment method and an unsupervised image reconstruction method, and performing image rough alignment and image reconstruction on the corrected fisheye image based on the image stitching frame to obtain a stitched image;
And the image squaring module is used for predicting an initial grid based on a complete convolution network and a residual error progressive regression strategy, predefining a target grid with a rigid shape, and squaring the spliced image in a mode of warping the spliced image from the predicted initial grid to the predefining target grid to obtain a rectangular image.
Preferably, the image correction module is configured to perform the following to obtain parameters of the camera by a camera calibration method:
selecting a point as an origin of a world coordinate system, and constructing the world coordinate system;
for the fish-eye image, obtaining a three-dimensional coordinate of each pixel point in the fish-eye image in a world coordinate system through an image detection algorithm;
converting the three-dimensional coordinates of each pixel point into pixel coordinates through a camera coordinate system and an image plane coordinate system after rigid transformation and projection;
for the pixel coordinates of each pixel point, the position of the center of the image is determined, and the position is converted into a final pixel coordinate system through translation.
Preferably, the image stitching module is configured to perform coarse alignment of the corrected fisheye image as follows:
constraining an unsupervised homography network based on ablation and introducing a stitching domain transformation layer for reducing image size;
For each corrected fisheye image, extracting the characteristics of the corrected fisheye image through a convolution network, taking the extracted characteristics as input, estimating the homography of the image through an unsupervised homography network, and obtaining a distorted image by distorting the corrected image and performing coarse alignment;
and putting the distorted image into a stitching domain transformation layer to reduce redundant black pixels in the image, so as to obtain a final distorted image.
Preferably, the image stitching module is configured to perform the following steps
When the corrected fisheye image is subjected to image reconstruction, the image stitching module is used for eliminating double images from features to pixels through an unsupervised reconstruction network, the reconstruction network comprises a low-resolution deformation branch network and a high-resolution refinement branch network, the low-resolution deformation branch network is used for learning deformation rules of image stitching, and the high-resolution refinement branch network is used for improving resolution of stitched images;
the graphic stitching module is used for executing the following morphing rule for learning image stitching through a low-resolution morphing branch network:
downsampling the warped image to reduce the resolution of the image to obtain a downsampled image;
inputting the downsampled image into an encoder-decoder structure consisting of a pooling layer and a deconvolution layer;
Jumping connection is carried out through the encoder-decoder structure to deform the downsampled image, so that a content mask and a seam mask of the downsampled image are obtained, the content mask is used for restraining characteristics of the reconstructed image to be close to a distorted image, the seam mask is used for restraining the edge of an overlapped area to be natural and continuous, and the overlapped area is an overlapped area between two fisheye images;
the high-resolution refinement branch network consists of a convolution layer and a resource block, wherein the resource block consists of the convolution layer, a relu activation function, the convolution layer, a sum function and the relu activation function in sequence;
the image stitching module is used for calculating content loss and seam loss of the distorted image under the high-resolution thinning branch network, and thinning the stitched image by combining the content mask and the seam mask of the distorted image under the low resolution to obtain the stitched image of the two distorted fish-eye images.
Preferably, the image squaring module is configured to perform the following:
taking the spliced image and a mask corresponding to the spliced image as input, and extracting advanced semantic features from the input through a feature extractor to obtain a feature image, wherein the feature extractor consists of stacked convolution pool blocks;
Predicting horizontal motion and vertical motion of each vertex in the characteristic image based on a conventional grid through a grid motion regressor, wherein the grid motion regressor is of a full convolution structure;
constructing two regressors with the same structure based on a residual regression strategy, namely a first regressor and a second regressor, wherein the first regressor is used for predicting primary grid motion in a progressive manner, and the second regressor is used for predicting residual grid motion in a progressive manner;
and predicting primary grid motion through a first regression device, carrying out grid deformation on the feature map, predicting residual grid motion through a second regression device, carrying out grid deformation on the distorted feature map, and maintaining the shape of deformed grids based on intra-grid constraints and inter-grid constraints during grid deformation to obtain a rectangular feature map.
The binocular fisheye image splicing method has the following advantages: for the collected fisheye image, camera parameters are obtained through a camera calibration method, the fisheye image is corrected through the camera parameters, for the corrected image, the image coarse alignment and the image reconstruction are carried out on the basis of an unsupervised coarse alignment method and an unsupervised image reconstruction method, a spliced image is obtained, and the spliced image is subjected to squaring, so that the spliced image which is good in splicing effect, high in boundary rectangle degree and suitable for being watched by a user and subsequent image processing is obtained.
Drawings
In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings that are needed in the embodiments or the description of the prior art will be briefly introduced below, and it is obvious that the drawings in the following description are only some embodiments of the present invention, and that other drawings can be obtained according to these drawings without inventive effort for a person skilled in the art.
The invention is further described below with reference to the accompanying drawings.
Fig. 1 is a flow chart of a binocular fisheye image stitching method of embodiment 1;
fig. 2 is a relationship block diagram of four large coordinate systems in a binocular fisheye image stitching method of embodiment 1;
FIG. 3 is a table showing the transformation relationship of four coordinate systems in a binocular fisheye image stitching method according to embodiment 1;
fig. 4 is a functional block diagram of image stitching in a binocular fisheye image stitching method according to embodiment 1;
fig. 5 is a functional block diagram of image rectangle in a binocular fisheye image stitching method of embodiment 1;
FIG. 6 is a schematic diagram of intra-grid constraint in a binocular fisheye image stitching method of embodiment 1;
fig. 7 is a schematic diagram of inter-grid constraint in a binocular fisheye image stitching method of embodiment 1.
Detailed Description
The invention will be further described with reference to the accompanying drawings and specific examples, so that those skilled in the art can better understand the invention and implement it, but the examples are not meant to limit the invention, and the technical features of the embodiments of the invention and the examples can be combined with each other without conflict.
The embodiment of the invention provides a binocular fisheye image stitching method and a binocular fisheye image stitching system, which are used for solving the technical problems that panoramic image stitching is long in time consumption and double images appear in panoramic image stitching.
Example 1:
the invention discloses a binocular fisheye image splicing method, which comprises the following steps of:
s100, image acquisition: simultaneously acquiring two fisheye images through a binocular fisheye lens;
s200, correcting an image: acquiring parameters of a camera through a camera calibration method, and carrying out distortion correction on each fisheye image based on the parameters of the camera to obtain corrected fisheye images;
s300, image stitching: constructing an image stitching frame based on an unsupervised rough alignment method and an unsupervised image reconstruction method, and performing image rough alignment and image reconstruction on corrected fisheye images based on the image stitching frame to obtain stitched images;
S400, image squaring: predicting an initial grid based on a complete convolution network and a residual error progressive regression strategy, predefining a target grid with a rigid shape, and squaring the spliced image in a mode of warping the spliced image from the predicted initial grid to the predefining target grid to obtain a rectangular image.
In step S200 of the present embodiment, parameters of a camera are obtained by a camera calibration method, including the following steps:
(1) Selecting a point as an origin of a world coordinate system, and constructing the world coordinate system;
(2) For the fish-eye image, obtaining a three-dimensional coordinate of each pixel point in the fish-eye image in a world coordinate system through an image detection algorithm;
(3) Converting the three-dimensional coordinates of each pixel point into pixel coordinates through a camera coordinate system and an image plane coordinate system after rigid transformation and projection;
(4) For the pixel coordinates of each pixel point, the position of the center of the image is determined, and the position is converted into a final pixel coordinate system through translation.
The four coordinate systems related to this embodiment are shown in fig. 2, the conversion relationships of which are shown in fig. 3, and the internal reference matrix in fig. 3 is simplified as follows:
the overall conversion of the internal reference matrix of fig. 3 is as follows, i.e. the single point undistorted camera imaging model is as follows:
The camera parameters including the information of the internal reference matrix, the external reference matrix and the like are obtained through the camera calibration method, and the distortion of the fisheye image is corrected through the camera parameters.
Step S300 is to splice the corrected two fish-eye images to obtain a spliced image. In the step, an unsupervised depth image stitching framework is constructed, and two-stage operation is executed through the stitching framework: unsupervised coarse image alignment and unsupervised image reconstruction.
The first stage performs image coarse alignment on the corrected fisheye image by an unsupervised coarse image alignment method, and comprises the following steps:
(1) Constraining an unsupervised homography network based on ablation and introducing a stitching domain transformation layer for reducing image size;
(2) For each corrected fisheye image, extracting the characteristics of the corrected fisheye image through a convolution network, taking the extracted characteristics as input, estimating the homography of the image through an unsupervised homography network, and obtaining a distorted image by distorting the corrected image and performing coarse alignment;
(3) And putting the distorted image into a stitching domain transformation layer to reduce redundant black pixels in the image, so as to obtain a final distorted image.
The coarse image of the present embodiment is aligned with the complete image as input, ensuring that all overlapping areas are included in the input. The distorted target image is forced to be close to the reference image without refilling invalid pixels in the distorted image. In contrast, the content of the invalid pixels in the target image is warped in the reference image. At the same time, a transform layer is introduced to warp the input image in the stitching domain space so as to be roughly aligned with each other in the proposed stitching domain transform layer.
In order to solve the problem of space waste, a splice domain transform layer is used. The splicing area is defined as the minimum circumscribed rectangle of the spliced image, so that the space is saved to the greatest extent while the integrity of the image content is ensured. First, coordinates of 4 vertices in a warped object image are calculated by formula (1), whereinAnd->The object image and the kth vertex coordinates of the object image are warped, respectively.
Then, the size of the distorted image is found by equation 2, in whichIs in combination with->Vertex coordinates of reference images having the same value. Wherein the splice domain is set to H * ×W * Size of the product. In this way, the input image is transformed in the splicing domain space, so that the space occupied by the feature mapping in the subsequent reconstruction network is effectively reduced.
A and B respectively represent a distorted target image and a distorted target image, k represents the kth vertex coordinate, and Deltax k ,Δy k And the offset of the kth vertex is represented, M is a reference image, and the vertex coordinates of the reference image have the same value as the vertex coordinates of the target image.
In the second stage, there is often a ghost effect in the stitched result, since homography matrices cannot account for all arrangements of different depth levels. However, pixel-level misalignments can be eliminated to some extent at the feature level, and the warped image is used to reconstruct the stitched image from features to pixels. When the corrected fisheye image is subjected to image reconstruction, the double image of the characteristic to the pixel is eliminated through an unsupervised reconstruction network, the reconstruction network comprises a low-resolution deformation branch network and a high-resolution refinement branch network, the deformation rule of image splicing is learned through the low-resolution deformation branch network, and the resolution of the spliced image is improved through the high-resolution refinement branch network.
Learning a deformation rule of image stitching through a low-resolution deformed branch network, comprising the following operations:
(1) Downsampling the warped image to reduce the resolution of the image to obtain a downsampled image;
(2) Inputting the downsampled image into an encoder-decoder structure consisting of a pooling layer and a deconvolution layer;
(3) The downsampled image is deformed by a skip connection through the encoder-decoder structure to obtain a content mask and a seam mask for the downsampled image, the content mask being used to constrain features of the reconstructed image to approximate a warped image, the seam mask being used to constrain edges of an overlap region to be natural and continuous, wherein the overlap region is an overlap region between two fisheye images.
As a specific implementation, the warped image is first downsampled to a low resolution defined as 256×256, and then the stitched image is reconstructed using an encoder-decoder network consisting of 3 pooling layers and 3 deconvolution layers.
The design of the high resolution branch is to refine the stitched image, the branch is entirely composed of convolution layers, in particular, it is composed of three independent convolution layers and eight sum resource blocks, wherein the resource blocks are sequentially composed of convolution layers, relu activation functions, convolution layers, sum functions and relu activation functions. Wherein the number of filters of each layer is set to 64 except for the filter number of the last layer which is set to 3. The specific splicing flow is shown in fig. 4.
And when the image is reconstructed, calculating the content loss and the joint loss of the distorted image under the high-resolution thinning branch network, and thinning the spliced image by combining the content mask and the joint mask of the distorted image under the low resolution to obtain the spliced image of the two distorted fish-eye images.
Step S300 of squaring the stitched image, comprising the steps of:
(1) Taking the spliced image and a mask corresponding to the spliced image as input, and extracting advanced semantic features from the input through a feature extractor to obtain a feature image, wherein the feature extractor consists of stacked convolution pool blocks;
(2) Predicting horizontal motion and vertical motion of each vertex in the characteristic image based on a conventional grid through a grid motion regressor, wherein the grid motion regressor is of a full convolution structure;
(3) Constructing two regressors with the same structure based on a residual regression strategy, namely a first regressor and a second regressor, wherein the first regressor is used for predicting primary grid motion in a progressive manner, and the second regressor is used for predicting residual grid motion in a progressive manner;
(4) And predicting primary grid motion through a first regression device, carrying out grid deformation on the feature map, predicting residual grid motion through a second regression device, carrying out grid deformation on the distorted feature map, and maintaining the shape of deformed grids based on intra-grid constraints and inter-grid constraints during grid deformation to obtain a rectangular feature map.
As a specific implementation of specific squaring, the stitched image and mask are used as inputs, and the feature extractor extracts advanced semantic features from the inputs by stacking simple convolution tiles through a predefined feature extractor. After feature extraction, an adaptive pooling layer is utilized to fix the resolution of the feature map. Subsequently, a full convolution structure was designed as a grid motion regression variable to predict the horizontal and vertical motion of each vertex on a regular grid basis. Assuming that the grid resolution is u×v, the size of the output body is (u+1) × (v+1) ×2. And estimating accurate grid motion in a progressive mode through a residual progressive regression strategy, and taking the distorted result as the re-input of the network. The regressor, whose warped image consists of a full convolution structure, is not used as a direct input to the new network, as this doubles the computational complexity. In contrast, warping the intermediate feature map improves performance with a slight increase in computational effort. Then, the initial mesh motion and the residual mesh motion are predicted by two regressors having the same structure, respectively. Although they have the same structure, they are designated for different tasks due to the different input features. Finally, a rigid target mesh is predefined, and only an initial mesh is estimated to form mesh deformation, helping to form a compact single-stage solution.
To prevent in rectangular imagesContent warp, the predicted grid should not be exaggerated deformed. Thus, intra-mesh constraints and inter-mesh constraints are designed to maintain the shape of the deformed mesh. The grid internal constraints are constraints imposed on the size and orientation of the grid edges in the grid, as shown in FIG. 6, encouraging each horizontal edgeIs greater than a threshold(assuming that the stitched image has a resolution of H W), this constraint is described using penalty P (equation 3). At the same time, inter-grid constraints are employed to encourage neighboring grids to transform consistently, as shown in FIG. 7, to encourage two consecutive deformed grid edgesCollinear.
Wherein, the liquid crystal display device comprises a liquid crystal display device,for the threshold value i is a horizontal unit vector to the right, the vertical side in each grid +.>Each horizontal edgeIs arranged in the horizontal projection direction of the lens.
Example 2:
the invention discloses a binocular fisheye image stitching system, which comprises an image acquisition module, an image correction module, an image stitching module and an image squaring module, wherein the system performs the method disclosed in the embodiment 1 to stitch fisheye images.
The image acquisition module is used for simultaneously acquiring two fisheye images through the binocular fisheye lens.
The image correction module is used for obtaining parameters of the camera through a camera calibration method, and carrying out distortion correction on each fisheye graph based on the parameters of the camera to obtain corrected fisheye images.
In this embodiment, the image correction module is configured to perform the following to obtain the camera parameters:
(1) Selecting a point as an origin of a world coordinate system, and constructing the world coordinate system;
(2) For the fish-eye image, obtaining a three-dimensional coordinate of each pixel point in the fish-eye image in a world coordinate system through an image detection algorithm;
(3) Converting the three-dimensional coordinates of each pixel point into pixel coordinates through a camera coordinate system and an image plane coordinate system after rigid transformation and projection;
(4) For the pixel coordinates of each pixel point, the position of the center of the image is determined, and the position is converted into a final pixel coordinate system through translation.
In this embodiment, the reference matrix in the four-coordinate transformation relationship is simplified as follows:
the total conversion of the internal reference matrix is as follows, namely, a single-point undistorted camera imaging model is as follows:
the camera parameter comprises information such as an internal reference matrix, an external reference matrix and the like, and the image correction module is used for correcting the distortion of the fisheye image through the camera parameter.
The image stitching module is used for constructing an image stitching frame based on an unsupervised rough alignment method and an unsupervised image reconstruction method, and performing image rough alignment and image reconstruction on the corrected fisheye image based on the image stitching frame to obtain a stitched image.
The image stitching module of this embodiment is used for stitching the two corrected fisheye images to obtain a stitched image. The module is used for constructing an unsupervised depth image splicing frame, and two-stage operation is performed through the splicing frame: unsupervised coarse image alignment and unsupervised image reconstruction.
The first stage, the module is used for performing image coarse alignment on the corrected fisheye image through an unsupervised coarse image alignment method, and the following steps are executed:
(1) Constraining an unsupervised homography network based on ablation and introducing a stitching domain transformation layer for reducing image size;
(2) For each corrected fisheye image, extracting the characteristics of the corrected fisheye image through a convolution network, taking the extracted characteristics as input, estimating the homography of the image through an unsupervised homography network, and obtaining a distorted image by distorting the corrected image and performing coarse alignment;
(3) And putting the distorted image into a stitching domain transformation layer to reduce redundant black pixels in the image, so as to obtain a final distorted image.
The coarse image of the present embodiment is aligned with the complete image as input, ensuring that all overlapping areas are included in the input. The distorted target image is forced to be close to the reference image without refilling invalid pixels in the distorted image. In contrast, the content of the invalid pixels in the target image is warped in the reference image. At the same time, a transform layer is introduced to warp the input image in the stitching domain space so as to be roughly aligned with each other in the proposed stitching domain transform layer.
In order to solve the problem of space waste, a splice domain transform layer is used. The splicing area is defined as the minimum circumscribed rectangle of the spliced image, so that the space is saved to the greatest extent while the integrity of the image content is ensured. First, coordinates of 4 vertices in a warped object image are calculated by formula (1), whereinAnd->The object image and the kth vertex coordinates of the object image are warped, respectively.
Then, the size of the distorted image is found by equation 2, in whichIs in combination with->Vertex coordinates of reference images having the same value. Wherein the splice domain is set to H * ×W * Size of the product. In this way, the input image is transformed in the splicing domain space, so that the space occupied by the feature mapping in the subsequent reconstruction network is effectively reduced.
A and B respectively represent a distorted target image and a distorted target image, k represents the kth vertex coordinate, and Deltax k ,Δy k And the offset of the kth vertex is represented, M is a reference image, and the vertex coordinates of the reference image have the same value as the vertex coordinates of the target image.
In the second stage, there is often a ghost effect in the stitched result, since homography matrices cannot account for all arrangements of different depth levels. However, pixel-level misalignments can be eliminated to some extent at the feature level, and the warped image is used to reconstruct the stitched image from features to pixels. When the corrected fisheye image is subjected to image reconstruction, the image splicing module is used for eliminating double images from features to pixels through an unsupervised reconstruction network, the reconstruction network comprises a low-resolution deformation branch network and a high-resolution thinning branch network, the deformation rule of image splicing is learned through the low-resolution deformation branch network, and the resolution of the spliced image is improved through the high-resolution thinning branch network.
The image stitching module is used for executing the following morphing rules for learning image stitching through a low-resolution morphing branch network:
(1) Downsampling the warped image to reduce the resolution of the image to obtain a downsampled image;
(2) Inputting the downsampled image into an encoder-decoder structure consisting of a pooling layer and a deconvolution layer;
(3) The downsampled image is deformed by a skip connection through the encoder-decoder structure to obtain a content mask and a seam mask for the downsampled image, the content mask being used to constrain features of the reconstructed image to approximate a warped image, the seam mask being used to constrain edges of an overlap region to be natural and continuous, wherein the overlap region is an overlap region between two fisheye images.
As a specific implementation, the warped image is first downsampled to a low resolution defined as 256×256, and then the stitched image is reconstructed using an encoder-decoder network consisting of 3 pooling layers and 3 deconvolution layers.
The design of the high resolution branch is to refine the stitched image, the branch is entirely composed of convolution layers, in particular, it is composed of three independent convolution layers and eight sum resource blocks, wherein the resource blocks are sequentially composed of convolution layers, relu activation functions, convolution layers, sum functions and relu activation functions. Wherein the number of filters of each layer is set to 64 except for the filter number of the last layer which is set to 3.
When reconstructing the image, the image stitching module is used for calculating content loss and seam loss of the distorted image under the high-resolution thinning branch network, and thinning the stitched image by combining the content mask and the seam mask of the distorted image under the low resolution to obtain the stitched image of the two distorted fisheye images.
The image squaring module is used for predicting an initial grid based on a complete convolution network and a residual error progressive regression strategy, predefining a target grid with a rigid shape, and squaring the spliced image in a mode of warping the spliced image from the predicted initial grid to the predefining target grid to obtain a rectangular image.
In this embodiment, the image stitching module is configured to perform squaring of the stitched image as follows:
(1) Taking the spliced image and a mask corresponding to the spliced image as input, and extracting advanced semantic features from the input through a feature extractor to obtain a feature image, wherein the feature extractor consists of stacked convolution pool blocks;
(2) Predicting horizontal motion and vertical motion of each vertex in the characteristic image based on a conventional grid through a grid motion regressor, wherein the grid motion regressor is of a full convolution structure;
(3) Constructing two regressors with the same structure based on a residual regression strategy, namely a first regressor and a second regressor, wherein the first regressor is used for predicting primary grid motion in a progressive manner, and the second regressor is used for predicting residual grid motion in a progressive manner;
(4) And predicting primary grid motion through a first regression device, carrying out grid deformation on the feature map, predicting residual grid motion through a second regression device, carrying out grid deformation on the distorted feature map, and maintaining the shape of deformed grids based on intra-grid constraints and inter-grid constraints during grid deformation to obtain a rectangular feature map.
As a specific implementation, the image stitching module is configured to perform the following: taking the stitched image and mask as input, the feature extractor extracts advanced semantic features from the input by stacking simple convolved tiles through a predefined feature extractor. After feature extraction, an adaptive pooling layer is utilized to fix the resolution of the feature map. Subsequently, a full convolution structure was designed as a grid motion regression variable to predict the horizontal and vertical motion of each vertex on a regular grid basis. Assuming that the grid resolution is u×v, the size of the output body is (u+1) × (v+1) ×2. And estimating accurate grid motion in a progressive mode through a residual progressive regression strategy, and taking the distorted result as the re-input of the network. The regressor, whose warped image consists of a full convolution structure, is not used as a direct input to the new network, as this doubles the computational complexity. In contrast, warping the intermediate feature map improves performance with a slight increase in computational effort. Then, the initial mesh motion and the residual mesh motion are predicted by two regressors having the same structure, respectively. Although they have the same structure, they are designated for different tasks due to the different input features. Finally, a rigid target mesh is predefined, and only an initial mesh is estimated to form mesh deformation, helping to form a compact single-stage solution.
In order to prevent distortion of the content in the rectangular image, the predicted grid should not be exaggeratedly deformed. Thus, intra-mesh constraints and inter-mesh constraints are designed to maintain the shape of the deformed mesh. The grid internal constraints are constraints imposed on the size and orientation of the grid edges in the grid, as shown in FIG. 6, encouraging each horizontal edgeIs greater than the threshold ++>(assuming that the stitched image has a resolution of H W), this constraint is described using penalty P (equation 3). At the same time, inter-grid constraints are employed to encourage neighboring grids to transform consistently, encouraging two consecutive deformed grid edges +.>Collinear.
Wherein, the liquid crystal display device comprises a liquid crystal display device,is the threshold ofThe value i is a horizontal unit vector to the right, the vertical side in each grid +.>Each horizontal edgeIs arranged in the horizontal projection direction of the lens.
While the invention has been illustrated and described in detail in the drawings and in the preferred embodiments, the invention is not limited to the disclosed embodiments, but it will be apparent to those skilled in the art that many more embodiments of the invention can be made by combining the means of the various embodiments described above and still fall within the scope of the invention.
Claims (10)
1. The binocular fisheye image splicing method is characterized by comprising the following steps of:
And (3) image acquisition: simultaneously acquiring two fisheye images through a binocular fisheye lens;
and (3) correcting an image: acquiring parameters of a camera through a camera calibration method, and carrying out distortion correction on each fisheye image based on the parameters of the camera to obtain corrected fisheye images;
and (3) image stitching: constructing an image stitching frame based on an unsupervised rough alignment method and an unsupervised image reconstruction method, and performing image rough alignment and image reconstruction on corrected fisheye images based on the image stitching frame to obtain stitched images;
image squaring: predicting an initial grid based on a complete convolution network and a residual error progressive regression strategy, predefining a target grid with a rigid shape, and squaring the spliced image in a mode of warping the spliced image from the predicted initial grid to the predefining target grid to obtain a rectangular image.
2. The binocular fisheye image stitching method according to claim 1, wherein the parameters of the camera are obtained by a camera calibration method, comprising the steps of:
selecting a point as an origin of a world coordinate system, and constructing the world coordinate system;
for the fish-eye image, obtaining a three-dimensional coordinate of each pixel point in the fish-eye image in a world coordinate system through an image detection algorithm;
Converting the three-dimensional coordinates of each pixel point into pixel coordinates through a camera coordinate system and an image plane coordinate system after rigid transformation and projection;
for the pixel coordinates of each pixel point, the position of the center of the image is determined, and the position is converted into a final pixel coordinate system through translation.
3. The binocular fisheye image stitching method of claim 1, wherein the image coarse alignment of the corrected fisheye image based on the image stitching frame comprises the steps of:
constraining an unsupervised homography network based on ablation and introducing a stitching domain transformation layer for reducing image size;
for each corrected fisheye image, extracting the characteristics of the corrected fisheye image through a convolution network, taking the extracted characteristics as input, estimating the homography of the image through an unsupervised homography network, and obtaining a distorted image by distorting the corrected image and performing coarse alignment;
and putting the distorted image into a stitching domain transformation layer to reduce redundant black pixels in the image, so as to obtain a final distorted image.
4. A binocular fisheye image stitching method according to claim 3 wherein, when reconstructing the corrected fisheye image based on the image stitching frame, the ghost of the feature to the pixel is eliminated by an unsupervised reconstruction network, the reconstruction network comprises a low resolution deformed branch network and a high resolution refined branch network, the deformation rule of the image stitching is learned by the low resolution deformed branch network, and the resolution of the stitched image is improved by the high resolution refined branch network;
Learning a deformation rule of image stitching through a low-resolution deformed branch network, comprising the following operations:
downsampling the warped image to reduce the resolution of the image to obtain a downsampled image;
inputting the downsampled image into an encoder-decoder structure consisting of a pooling layer and a deconvolution layer;
jumping connection is carried out through the encoder-decoder structure to deform the downsampled image, so that a content mask and a seam mask of the downsampled image are obtained, the content mask is used for restraining characteristics of the reconstructed image to be close to a distorted image, the seam mask is used for restraining the edge of an overlapped area to be natural and continuous, and the overlapped area is an overlapped area between two fisheye images;
the high-resolution refinement branch network consists of a convolution layer and a resource block, wherein the resource block consists of the convolution layer, a relu activation function, the convolution layer, a sum function and the relu activation function in sequence;
and calculating the content loss and the seam loss of the distorted image under the high-resolution thinning branch network, and thinning the spliced image by combining the content mask and the seam mask of the distorted image under the low resolution to obtain the spliced image of the two distorted fish-eye images.
5. The binocular fisheye image stitching method of claim 4, wherein the stitched image is rectangular by warping the stitched image from a predicted initial grid to a predefined target grid, comprising the steps of:
taking the spliced image and a mask corresponding to the spliced image as input, and extracting advanced semantic features from the input through a feature extractor to obtain a feature image, wherein the feature extractor consists of stacked convolution pool blocks;
predicting horizontal motion and vertical motion of each vertex in the characteristic image based on a conventional grid through a grid motion regressor, wherein the grid motion regressor is of a full convolution structure;
constructing two regressors with the same structure based on a residual regression strategy, namely a first regressor and a second regressor, wherein the first regressor is used for predicting primary grid motion in a progressive manner, and the second regressor is used for predicting residual grid motion in a progressive manner;
and predicting primary grid motion through a first regression device, carrying out grid deformation on the feature map, predicting residual grid motion through a second regression device, carrying out grid deformation on the distorted feature map, and maintaining the shape of deformed grids based on intra-grid constraints and inter-grid constraints during grid deformation to obtain a rectangular feature map.
6. A binocular fisheye image stitching system, wherein image stitching is performed by a binocular fisheye image stitching method as claimed in any one of claims-5, the system comprising:
the image acquisition module is used for simultaneously acquiring two fisheye images through the binocular fisheye lens;
the image correction module is used for obtaining parameters of a camera through a camera calibration method, and carrying out distortion correction on each fisheye graph based on the parameters of the camera to obtain corrected fisheye images;
the image stitching module is used for constructing an image stitching frame based on an unsupervised rough alignment method and an unsupervised image reconstruction method, and performing image rough alignment and image reconstruction on the corrected fisheye image based on the image stitching frame to obtain a stitched image;
and the image squaring module is used for predicting an initial grid based on a complete convolution network and a residual error progressive regression strategy, predefining a target grid with a rigid shape, and squaring the spliced image in a mode of warping the spliced image from the predicted initial grid to the predefining target grid to obtain a rectangular image.
7. The binocular fisheye image stitching system of claim 6, wherein the image correction module is configured to perform the following to obtain parameters of the camera via a camera calibration method:
selecting a point as an origin of a world coordinate system, and constructing the world coordinate system;
for the fish-eye image, obtaining a three-dimensional coordinate of each pixel point in the fish-eye image in a world coordinate system through an image detection algorithm;
converting the three-dimensional coordinates of each pixel point into pixel coordinates through a camera coordinate system and an image plane coordinate system after rigid transformation and projection;
for the pixel coordinates of each pixel point, the position of the center of the image is determined, and the position is converted into a final pixel coordinate system through translation.
8. The binocular fisheye image stitching system of claim 6, wherein the image stitching module is configured to perform coarse image alignment of corrected fisheye images as follows:
constraining an unsupervised homography network based on ablation and introducing a stitching domain transformation layer for reducing image size;
for each corrected fisheye image, extracting the characteristics of the corrected fisheye image through a convolution network, taking the extracted characteristics as input, estimating the homography of the image through an unsupervised homography network, and obtaining a distorted image by distorting the corrected image and performing coarse alignment;
And putting the distorted image into a stitching domain transformation layer to reduce redundant black pixels in the image, so as to obtain a final distorted image.
9. The binocular fisheye image stitching system of claim 8, wherein the image stitching module is configured to perform the following
When the corrected fisheye image is subjected to image reconstruction, the image stitching module is used for eliminating double images from features to pixels through an unsupervised reconstruction network, the reconstruction network comprises a low-resolution deformation branch network and a high-resolution refinement branch network, the low-resolution deformation branch network is used for learning deformation rules of image stitching, and the high-resolution refinement branch network is used for improving resolution of stitched images;
the graphic stitching module is used for executing the following morphing rule for learning image stitching through a low-resolution morphing branch network:
downsampling the warped image to reduce the resolution of the image to obtain a downsampled image;
inputting the downsampled image into an encoder-decoder structure consisting of a pooling layer and a deconvolution layer;
jumping connection is carried out through the encoder-decoder structure to deform the downsampled image, so that a content mask and a seam mask of the downsampled image are obtained, the content mask is used for restraining characteristics of the reconstructed image to be close to a distorted image, the seam mask is used for restraining the edge of an overlapped area to be natural and continuous, and the overlapped area is an overlapped area between two fisheye images;
The high-resolution refinement branch network consists of a convolution layer and a resource block, wherein the resource block consists of the convolution layer, a relu activation function, the convolution layer, a sum function and the relu activation function in sequence;
the image stitching module is used for calculating content loss and seam loss of the distorted image under the high-resolution thinning branch network, and thinning the stitched image by combining the content mask and the seam mask of the distorted image under the low resolution to obtain the stitched image of the two distorted fish-eye images.
10. The binocular fisheye image stitching system of claim 9, wherein the image squaring module is configured to perform the following:
taking the spliced image and a mask corresponding to the spliced image as input, and extracting advanced semantic features from the input through a feature extractor to obtain a feature image, wherein the feature extractor consists of stacked convolution pool blocks;
predicting horizontal motion and vertical motion of each vertex in the characteristic image based on a conventional grid through a grid motion regressor, wherein the grid motion regressor is of a full convolution structure;
constructing two regressors with the same structure based on a residual regression strategy, namely a first regressor and a second regressor, wherein the first regressor is used for predicting primary grid motion in a progressive manner, and the second regressor is used for predicting residual grid motion in a progressive manner;
And predicting primary grid motion through a first regression device, carrying out grid deformation on the feature map, predicting residual grid motion through a second regression device, carrying out grid deformation on the distorted feature map, and maintaining the shape of deformed grids based on intra-grid constraints and inter-grid constraints during grid deformation to obtain a rectangular feature map.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202310855895.6A CN116912091A (en) | 2023-07-13 | 2023-07-13 | Binocular fisheye image stitching method and system |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202310855895.6A CN116912091A (en) | 2023-07-13 | 2023-07-13 | Binocular fisheye image stitching method and system |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN116912091A true CN116912091A (en) | 2023-10-20 |
Family
ID=88359653
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202310855895.6A Pending CN116912091A (en) | 2023-07-13 | 2023-07-13 | Binocular fisheye image stitching method and system |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN116912091A (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN117291809A (en) * | 2023-11-27 | 2023-12-26 | 山东大学 | Integrated circuit image stitching method and system based on deep learning |
-
2023
- 2023-07-13 CN CN202310855895.6A patent/CN116912091A/en active Pending
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN117291809A (en) * | 2023-11-27 | 2023-12-26 | 山东大学 | Integrated circuit image stitching method and system based on deep learning |
| CN117291809B (en) * | 2023-11-27 | 2024-03-15 | 山东大学 | Integrated circuit image stitching method and system based on deep learning |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN112767289B (en) | Image fusion method, device, medium and electronic equipment | |
| JP4620607B2 (en) | Image processing device | |
| DE102018120304A1 (en) | Method and system for image distortion correction for images taken by using a wide-angle lens | |
| CN110827200A (en) | Image super-resolution reconstruction method, image super-resolution reconstruction device and mobile terminal | |
| CN107451952B (en) | Splicing and fusing method, equipment and system for panoramic video | |
| CN101710932A (en) | Image stitching method and device | |
| CN116912091A (en) | Binocular fisheye image stitching method and system | |
| CN107895349B (en) | Endoscope video deblurring method based on synthesis | |
| JP2009290827A (en) | Image processing apparatus, and image processing method | |
| Sheng et al. | Depth-aware motion deblurring using loopy belief propagation | |
| CN111553841B (en) | Real-time video splicing method based on optimal suture line updating | |
| An et al. | Single-shot high dynamic range imaging via deep convolutional neural network | |
| Liu et al. | A single frame and multi-frame joint network for 360-degree panorama video super-resolution | |
| CN111866523B (en) | Panoramic video synthesis method and device, electronic equipment and computer storage medium | |
| Dai et al. | Image super-resolution via residual block attention networks | |
| CN115115516B (en) | Real world video super-resolution construction method based on Raw domain | |
| CN116612015A (en) | Model training method, image mole pattern removing method and device and electronic equipment | |
| Song et al. | Real-scene reflection removal with raw-rgb image pairs | |
| CN114581355A (en) | Method, terminal and electronic device for reconstructing HDR image | |
| Vien et al. | Bidirectional motion estimation with cyclic cost volume for high dynamic range imaging | |
| CN116403226A (en) | Unconstrained fold document image correction method, system, equipment and storage medium | |
| Xiong et al. | Hierarchical fusion for practical ghost-free high dynamic range imaging | |
| CN116596815A (en) | Image stitching method based on multi-stage alignment network | |
| Shen et al. | Over-exposure image correction with automatic texture synthesis | |
| CN115035013A (en) | Image processing method, image processing apparatus, terminal, and readable storage medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |