CN116883235A - Distributed photoelectric oriented image stitching method and device - Google Patents
Distributed photoelectric oriented image stitching method and device Download PDFInfo
- Publication number
- CN116883235A CN116883235A CN202310529250.3A CN202310529250A CN116883235A CN 116883235 A CN116883235 A CN 116883235A CN 202310529250 A CN202310529250 A CN 202310529250A CN 116883235 A CN116883235 A CN 116883235A
- Authority
- CN
- China
- Prior art keywords
- image
- distributed
- image data
- splicing
- current
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000000034 method Methods 0.000 title claims abstract description 58
- 238000007781 pre-processing Methods 0.000 claims abstract description 32
- 230000009466 transformation Effects 0.000 claims abstract description 31
- 230000004927 fusion Effects 0.000 claims abstract description 24
- 238000001514 detection method Methods 0.000 claims abstract description 16
- 238000007499 fusion processing Methods 0.000 claims abstract description 9
- 238000000605 extraction Methods 0.000 claims description 12
- 238000012545 processing Methods 0.000 claims description 11
- 238000001914 filtration Methods 0.000 claims description 10
- 230000005622 photoelectricity Effects 0.000 claims description 7
- 230000011218 segmentation Effects 0.000 claims description 7
- 230000002159 abnormal effect Effects 0.000 claims description 6
- 238000005070 sampling Methods 0.000 claims description 5
- 238000004590 computer program Methods 0.000 claims description 4
- 238000012937 correction Methods 0.000 claims description 4
- 230000032669 eclosion Effects 0.000 claims description 4
- 238000010586 diagram Methods 0.000 description 8
- 238000006243 chemical reaction Methods 0.000 description 6
- 238000013213 extrapolation Methods 0.000 description 4
- 230000006870 function Effects 0.000 description 4
- 230000000694 effects Effects 0.000 description 3
- 230000002708 enhancing effect Effects 0.000 description 3
- 238000003384 imaging method Methods 0.000 description 3
- 230000003993 interaction Effects 0.000 description 3
- 229910000831 Steel Inorganic materials 0.000 description 2
- 230000003044 adaptive effect Effects 0.000 description 2
- 238000011161 development Methods 0.000 description 2
- 230000018109 developmental process Effects 0.000 description 2
- 230000007613 environmental effect Effects 0.000 description 2
- 238000009499 grossing Methods 0.000 description 2
- 238000004519 manufacturing process Methods 0.000 description 2
- 230000008569 process Effects 0.000 description 2
- 239000010959 steel Substances 0.000 description 2
- XLYOFNOQVPJJNP-UHFFFAOYSA-N water Substances O XLYOFNOQVPJJNP-UHFFFAOYSA-N 0.000 description 2
- 235000002566 Capsicum Nutrition 0.000 description 1
- 239000006002 Pepper Substances 0.000 description 1
- 235000016761 Piper aduncum Nutrition 0.000 description 1
- 235000017804 Piper guineense Nutrition 0.000 description 1
- 244000203593 Piper nigrum Species 0.000 description 1
- 235000008184 Piper nigrum Nutrition 0.000 description 1
- 230000001133 acceleration Effects 0.000 description 1
- 230000005540 biological transmission Effects 0.000 description 1
- 230000002349 favourable effect Effects 0.000 description 1
- 239000000463 material Substances 0.000 description 1
- 239000011159 matrix material Substances 0.000 description 1
- 230000007246 mechanism Effects 0.000 description 1
- 239000000203 mixture Substances 0.000 description 1
- 230000003287 optical effect Effects 0.000 description 1
- 230000005693 optoelectronics Effects 0.000 description 1
- 230000008447 perception Effects 0.000 description 1
- 230000036544 posture Effects 0.000 description 1
- 238000004321 preservation Methods 0.000 description 1
- 238000003672 processing method Methods 0.000 description 1
- 238000000746 purification Methods 0.000 description 1
- 230000005855 radiation Effects 0.000 description 1
- 238000011160 research Methods 0.000 description 1
- 150000003839 salts Chemical class 0.000 description 1
- 230000007704 transition Effects 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T3/00—Geometric image transformation in the plane of the image
- G06T3/40—Scaling the whole image or part thereof
- G06T3/4038—Scaling the whole image or part thereof for image mosaicing, i.e. plane images composed of plane sub-images
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T5/00—Image enhancement or restoration
- G06T5/50—Image enhancement or restoration by the use of more than one image, e.g. averaging, subtraction
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20212—Image combination
- G06T2207/20221—Image fusion; Image merging
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02D—CLIMATE CHANGE MITIGATION TECHNOLOGIES IN INFORMATION AND COMMUNICATION TECHNOLOGIES [ICT], I.E. INFORMATION AND COMMUNICATION TECHNOLOGIES AIMING AT THE REDUCTION OF THEIR OWN ENERGY USE
- Y02D10/00—Energy efficient computing, e.g. low power processors, power management or thermal management
Abstract
The application provides a distributed photoelectric-oriented image stitching method and device, wherein the method comprises the following steps: acquiring distributed image data, wherein the distributed image data is an image of a region to be spliced, which is acquired under different position conditions by utilizing at least two image acquisition devices respectively; performing image preprocessing on the distributed image data; extracting feature points of the current image, and eliminating the error matching feature point pairs; binding adjustment is carried out on the current image, global binding adjustment is combined with a loop detection method, and accumulated errors are eliminated by comparing errors of a transformation relation generated by a minimum loop and a transformation relation of a connecting edge; and carrying out fusion processing on the images after the current splicing by adopting a fusion algorithm so as to finish the splicing of the distributed image data. Compared with the traditional method, the method meets the real-time requirement of splicing, eliminates the fuzzy ghost, reduces the projection distortion and improves the operation efficiency.
Description
Technical Field
The application relates to the technical field of distributed image processing, in particular to an image splicing method and device for distributed photoelectricity.
Background
The airborne infrared image with a large field of view range and high resolution has corresponding application value in various fields. However, the single detector cannot provide the coverage range required by the multi-user multi-window photoelectric system and meet the requirement of resolution by the limit of the current image infrared detection technology level. In complex and changeable environments, the single-machine photoelectric platform has limited scanning range, carried photoelectric load and other capabilities, and has obvious limitation on a shielded area, so that a multi-camera view field splicing mode is adopted to splice a scanned image sequence into a pair of panoramic images with a large view field range and high resolution at the same time, wide area detection is realized, rich environmental information is acquired, and large-range area coverage search is completed.
However, most of the existing image stitching algorithms are based on pure rotation differences or planar imaging, and have poor stitching applicability to distributed airborne images, which results in ghost images, artifacts and even stitching failure. In addition, although the characteristic-based image stitching algorithm can obtain higher stitching precision, the algorithm is low in operation efficiency and long in stitching time consumption, and the stitching real-time requirement cannot be met.
Disclosure of Invention
The application aims to solve the technical problem of how to realize effective distributed photoelectric image splicing and has the effects of high precision, low distortion and quick image splicing. In view of the above, the present application provides a distributed photoelectric oriented image stitching method and apparatus.
The technical scheme adopted by the application is that the distributed photoelectric-oriented image stitching method comprises the following steps:
acquiring distributed image data, wherein the distributed image data is an image of an area to be spliced, which is acquired under different position conditions by utilizing at least two image acquisition devices;
performing image preprocessing on the distributed image data;
extracting feature points of the current image, and eliminating the error matching feature point pairs;
binding adjustment is carried out on the current image, global binding adjustment is combined with a loop detection method, and accumulated errors are eliminated by comparing errors of a transformation relation generated by a minimum loop and a transformation relation of a connecting edge;
and carrying out fusion processing on the images after current splicing by adopting a fusion algorithm to finish the splicing of the distributed image data.
In one embodiment, the image preprocessing comprises denoising preprocessing and enhancement preprocessing, wherein the denoising preprocessing comprises mean value filtering denoising, gaussian filtering denoising and median filtering denoising; the enhancement preprocessing comprises histogram equalization, gray level correction, transformation and self-adaptive segmentation histogram enhancement.
In one embodiment, the extracting the feature points of the current image and eliminating the mismatching feature point pairs includes:
and extracting feature points of the current image by using a configured SIFT algorithm, and removing the error matching feature point pairs by using a configured random sampling consistency algorithm.
In one embodiment, the feature extraction further comprises: and extracting feature points of the current image by using SURF and ORB algorithms.
In one embodiment, the fusion algorithm comprises: weight average fusion and eclosion fusion.
In one embodiment, after the step of extracting feature points from the current image and rejecting the pair of mismatching feature points, the method further includes: and when the wrong characteristic point pair exists, purifying the abnormal characteristic point.
In one embodiment, the processing platform on which the method is based comprises: CPU, GPU, qtDesigner, pySide2, pyCharm.
Another aspect of the present application also provides an image stitching apparatus for distributed photoelectricity, including:
an acquisition unit configured to acquire distributed image data, wherein the distributed image data is an image of a region to be stitched acquired under different position conditions by using at least two image acquisition devices, respectively;
an onboard image preprocessing unit configured to perform image preprocessing on the distributed image data;
the characteristic point extraction unit is configured to extract characteristic points of the current image and reject the mismatching characteristic point pairs;
the binding adjustment unit is configured to carry out binding adjustment on the current image, combine global binding adjustment with a loop detection method and eliminate accumulated errors by comparing errors of a transformation relation generated by a minimum loop and a transformation relation of a connecting edge;
and the splicing and fusing unit is configured to perform fusion processing on the current spliced image by adopting a fusion algorithm so as to finish splicing the distributed image data.
Another aspect of the present application also provides an electronic device including: a memory, a processor and a computer program stored on the memory and executable on the processor, which when executed by the processor performs the steps of the distributed electro-optical oriented image stitching method as claimed in any one of the preceding claims.
Another aspect of the present application also provides a computer storage medium having stored thereon a computer program which, when executed by a processor, implements the steps of the distributed electro-optical oriented image stitching method as described in any of the above.
By adopting the technical scheme, the application has at least the following advantages:
the image stitching algorithm and the software thereof for the distributed photoelectricity have the advantages of high stitching precision, low fuzzy ghost, low projection distortion, high instantaneity, convenience for user interaction and the like, can effectively complete the large-scale wide-area panoramic image stitching function of the distributed airborne platform, effectively remove shielding, promote the covered situation perception information and realize the detection search of a large area.
Drawings
FIG. 1 is a flow chart of a distributed photoelectric oriented image stitching method according to an embodiment of the present application;
fig. 2 is a schematic diagram of implementing a distributed photoelectric oriented image stitching method according to an embodiment of the present application;
fig. 3 is a schematic diagram of a splicing result of a distributed photoelectric-oriented image splicing algorithm in a pure water field scene according to an embodiment of the present application;
fig. 4 is a schematic diagram of a splice result in a pure building scenario according to an embodiment of the present application;
FIG. 5 is a schematic diagram of a splice result in a field and building combination scenario according to an embodiment of the present application;
FIG. 6 is a diagram of a large-scale stitching result of 154 images according to an embodiment of the present application;
FIG. 7 is a schematic diagram of distributed optoelectronic image oriented stitching software according to an embodiment of the present application;
fig. 8 is a schematic diagram of a composition structure of an image stitching device facing distributed photoelectricity according to an embodiment of the application;
fig. 9 is a schematic diagram of an electronic device according to an embodiment of the present application.
Detailed Description
In order to further describe the technical means and effects adopted by the present application for achieving the intended purpose, the following detailed description of the present application is given with reference to the accompanying drawings and preferred embodiments.
It will be understood that the terms "comprises," "comprising," "includes," "including," "having," "containing," and/or "including," when used in this specification, specify the presence of stated features, integers, steps, operations, elements, and/or components, but do not preclude the presence or addition of one or more other features, integers, steps, operations, elements, components, and/or groups thereof. Furthermore, when a statement such as "at least one of the following" appears after a list of features that are listed, the entire listed feature is modified instead of modifying a separate element in the list. Furthermore, when describing embodiments of the present application, the use of "may" means "one or more embodiments of the present application. Also, the term "exemplary" is intended to refer to an example or illustration.
As used herein, the terms "substantially," "about," and the like are used as terms of a table approximation, not as terms of a table level, and are intended to illustrate inherent deviations in measured or calculated values that would be recognized by one of ordinary skill in the art.
Unless otherwise defined, all terms (including technical and scientific terms) used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this application belongs. It will be further understood that terms, such as those defined in commonly used dictionaries, should be interpreted as having a meaning that is consistent with their meaning in the context of the relevant art and will not be interpreted in an idealized or overly formal sense unless expressly so defined herein.
It should be noted that, without conflict, the embodiments of the present application and features of the embodiments may be combined with each other. The steps of the method flow described in the specification and the flow chart shown in the drawings of the specification are not necessarily strictly executed according to step numbers, and the execution order of the steps of the method may be changed. Moreover, some steps may be omitted, multiple steps may be combined into one step to be performed, and/or one step may be decomposed into multiple steps to be performed.
The steps of the method flow described in the specification and the flow chart shown in the drawings of the specification are not necessarily strictly executed according to step numbers, and the execution order of the steps of the method may be changed. Moreover, some steps may be omitted, multiple steps may be combined into one step to be performed, and/or one step may be decomposed into multiple steps to be performed.
In a first embodiment of the present application, as shown in fig. 1, an image stitching method for distributed photoelectricity includes the following specific steps:
step S1, acquiring distributed image data, wherein the distributed image data is an image of an area to be spliced, which is acquired under different position conditions by using at least two image acquisition devices respectively;
s2, performing image preprocessing on the distributed image data;
step S3, extracting feature points of the current image, and eliminating the error matching feature point pairs;
step S4, binding adjustment is carried out on the current image, global binding adjustment is combined with a loop detection method, and accumulated errors are eliminated by comparing errors of a conversion relation generated by a minimum loop and a conversion relation of a connecting edge;
and S5, performing fusion processing on the images after the current splicing by adopting a fusion algorithm to finish the splicing of the distributed image data.
The method provided by the present embodiment will be described in detail below with reference to fig. 1 to 2.
Step S1, acquiring distributed image data, wherein the distributed image data is an image of a region to be spliced, which is acquired under different position conditions by utilizing at least two image acquisition devices.
In this embodiment, the image acquiring apparatus may be an unmanned aerial vehicle, and a specific acquiring manner may be: the three frames of the machine fly at different heights and different speeds from different angles respectively, and the carried photoelectric pod looks down to shoot the images of the areas to be spliced. Each aircraft forward flies in the region to be spliced to acquire image data, and the three aircraft have a coincidence degree of 30% -50% between images acquired in the flying process. And transmitting the acquired image data to distributed image splicing software at the ground end through a wireless transmission device for splicing.
And S2, performing image preprocessing on the distributed image data.
In this embodiment, the image preprocessing may include denoising preprocessing and enhancement preprocessing.
Specifically, due to different production materials, manufacturing processes and the like of the infrared detector, noise interference is introduced in the image acquisition process, and the problems of narrow gray scale range, low contrast and the like of an infrared image are caused by an infrared radiation imaging mechanism, so that the acquired distributed infrared image needs to be preprocessed. According to the characteristics of distributed image data, a denoising and enhancing algorithm in image preprocessing is researched, a median filtering method with good effect of removing salt and pepper noise and a self-adaptive segmentation histogram enhancing method are selected, feature points are extracted from 938 and 2447 under two different scenes by using the enhancing algorithm, and the correct matching rate of the feature points is 0.89 and 0.97, so that a high-quality detail image is provided for subsequent feature point extraction and correct matching. And carrying out preprocessing such as image denoising and enhancement on the acquired image data in distributed image stitching software.
Exemplary image denoising preprocessing algorithms include, but are not limited to: mean filter denoising, gaussian filter denoising, median filter denoising and the like.
Accordingly, the enhancement preprocessing may divide the foreground and the background of the source image by using an adaptive histogram segmentation method, equalize the segmented front Jing Zi histogram, and correct the intensity range of the background sub-histogram. The module can overcome the phenomenon of excessive background enhancement, so that the gray level in the background can not be excessively fitted, the contrast between the foreground and the background in the image is effectively improved, the image detail information is improved, and high-quality image data is provided for subsequent feature point extraction and correct matching.
Exemplary image enhancement preprocessing includes, but is not limited to: histogram equalization, gray correction and transformation, adaptive segmentation histogram enhancement.
Step S3, extracting feature points of the current image, and eliminating the error matching feature point pairs;
in this embodiment, for the situation that parallax between distributed airborne images is larger and distortion is serious, the requirement on image registration accuracy is high, a scale-invariant feature transformation SIFT (Scale Invariant Feature Transform) algorithm is adopted as a feature point detection algorithm for distributed image stitching, and other feature extraction algorithms can be used, including but not limited to an acceleration robustness feature detection algorithm SURF (Speeded Up Robust Features), a feature detection algorithm ORB (Oriented FAST and Rotated BRIEF) for FAST feature point and rotation BRIEF feature description, and other feature extraction algorithms and variants thereof are adopted to perform feature extraction on the images. In addition, the error matching characteristic point pairs can be effectively removed by using a random sampling consistency algorithm, and the registration accuracy is further improved.
In some embodiments, after the step of extracting feature points from the current image and rejecting the pair of mismatching feature points, the method further includes: and when the wrong characteristic point pair exists, purifying the abnormal characteristic point.
Specifically, image registration and corresponding purification processes may be performed sequentially. The image registration part creates a corresponding description subset for the reference image and the observation image, measures the similarity of the key point descriptors by using a Euclidean distance method, and performs matching search by using a K-Dimensional KD (K-dimension) tree. Because of the error characteristic point pairs, the abnormal characteristic points need to be purified, the characteristic points are divided into an inner point and an outer point, 4 groups of matching characteristic point pairs are randomly extracted from an inner point set to construct a homography matrix, and projection errors are calculated until the errors reach the minimum. The module can obtain accurate matching characteristic point pairs, and is favorable for high-precision image stitching research.
Step S4, binding adjustment is carried out on the current image, global binding adjustment is combined with a loop detection method, and accumulated errors are eliminated by comparing errors of a conversion relation generated by a minimum loop and a conversion relation of a connecting edge;
in the embodiment, the method is improved on the basis of a global binding adjustment algorithm, the global binding adjustment and a loop detection method are combined, and the error of the transformation relation generated by the minimum loop and the error of the transformation relation of the connecting side are compared, so that accumulated errors are eliminated, and the large-scale image registration accuracy is improved.
In addition, aiming at the problems of large parallax between different angles, complex image space geometric transformation relationship and the like existing in the distributed airborne image stitching, the method is improved on the basis of being As close to a projection APAP (As-Projective-As-Possible) algorithm As Possible. Performing deformation processing on the image in the overlapping area before projection transformation to eliminate fuzzy double images; adopting linearization homography smooth extrapolation to global transformation to reduce projection distortion at the edge; in addition, deformation at the grid nodes of the image is calculated by using a grid dividing method, so that the operation efficiency of the algorithm is improved.
Accordingly, the deformation processing section applies to the homography transformation using the thin-plate spline method, for the target image I T Performing deformation treatment on the steel plate to obtain a steel plate,respectively solving deformation in the x direction and the y direction to finally obtain a whole target image I T Is a deformation processing function of (1). And (3) linearizing homography smoothing extrapolation to a global transformation part, and calculating and obtaining optimal similarity transformation by adopting a segmentation processing method. Specifically, first, the matched feature point pairs are thresholded to epsilon k Random sampling consistency processing of the system to eliminate abnormal characteristic point pairs. Next, a threshold size ε is used l Searching in the plane for the homography transformation with the largest number of inliers. If the search result appears the number epsilon of the inner points l ＜ε k And deleting the interior points from the set, and continuously performing iterative search until the number of the interior points is smaller than a given threshold value. And calculating single similarity transformation by using the matched interior point pairs, and selecting the minimum rotation angle as the rotation angle of the transformation. And finally, updating the global similarity transformation aiming at the unnatural area in the splicing result. The grid dividing part solves the deformation function at the grid node, and calculates the corresponding deformation value by adopting a bilinear interpolation method for the undivided pixel points.
And S5, performing fusion processing on the images after current splicing by adopting a fusion algorithm to finish splicing the distributed image data.
In this embodiment, the multi-resolution fusion with better preservation of the detail information of the fusion image and the most smooth transition at the splicing position can be selected as the image fusion algorithm of the text, and the spliced image is processed to obtain the seamless smooth panoramic high-resolution image, and the fusion algorithm includes but is not limited to weight average fusion, eclosion fusion and the like.
In some possible implementations, the processing platform on which the splicing algorithm provided in this embodiment is based includes: CPU, GPU, qtDesigner, pySide2, pyCharm; the development environment adopts an AMDRyzen74800H and 8-core 2.9GHz processor, the core image processing and video computing equipment is NVIDIAGeForce RTX2060, the video memory size is a 6GB hardware development platform and software platforms such as QtDesigner, pySide and PyCharm.
Compared with the traditional image stitching algorithm based on pure rotation difference or plane imaging, the image stitching method for the distributed photoelectric platform is not suitable for stitching of the distributed airborne images, has the phenomena of ghost, artifact and even stitching failure, is low in algorithm operation efficiency, cannot meet the stitching instantaneity requirement, and is used for performing deformation processing on images in overlapping areas by using a thin-plate spline method by using the image stitching algorithm for the distributed photoelectric platform and software thereof, so that fuzzy ghost is eliminated. Meanwhile, linearization homography smoothing extrapolation is adopted to global transformation, so that projection distortion is reduced. In addition, deformation at the grid nodes is calculated by using a grid division method, and the rest pixels are linearly interpolated, so that the operation efficiency of the algorithm is improved.
Specifically, some specific examples of the present embodiment are given below in conjunction with the drawings.
The splicing result of the distributed photoelectric oriented image splicing algorithm under the pure water field scene is shown in figure 3.
The splice results in a pure building scenario are shown in fig. 4.
The result of the splice in the paddy field and building combined scene is shown in fig. 5.
The result of large-scale stitching of 154 images is shown in fig. 6.
The distributed photoelectric image stitching software is shown in fig. 7.
In a specific embodiment, large-scale stitching of 154 images can be effectively achieved, and a high-resolution panoramic image with a resolution of 10K×10K is obtained, and the stitching time is 138s.
Specifically, when two images are spliced under different scenes, the average error of the spliced images under different scenes is 5177.3019, 5821.7354 and 4590.2185 respectively lower than that of the original APAP algorithm, the images have smaller alignment error and image detail information, the peak signal-to-noise ratio is 10.9902, 11.2912 and 11.5410 respectively, the structural similarity is 0.4378, 0.4201 and 0.4278 respectively, the image entropy is 5.9293, 8.0613 and 6.9243 respectively, the image entropy is higher than that of the APAP algorithm, and the obtained image has clearer contour and more abundant information; when large-scale image stitching is carried out, the improved algorithm carries out image stitching under a complex scene, so that the average error of the panoramic image is 3420.3491, the peak signal to noise ratio is 13.0385, the image entropy is 6.0063, large-scale stitching of 154 images can be effectively realized, the high-resolution panoramic image with the resolution of 10K multiplied by 10K is obtained, the stitching time is 138s, and the superiority of the improved distributed stitching algorithm is verified.
In summary, compared with the prior art, the method provided by the embodiment has the following advantages:
1) The method can effectively solve the problems that when the distributed airborne images are acquired, the images are easily interfered by external environmental factors and the flight postures of different platforms of aircrafts are affected, and larger parallax exists between the images;
2) The method has higher image splicing precision and eliminates the ghost blurring phenomenon existing in the overlapping area;
3) The problem that projection distortion is generated in a non-overlapping area at the edge of the panoramic image can be solved;
4) The splicing speed is high, and the time-consuming requirement of splicing is met;
5) The method has the function of splicing the wide-area panoramic images with high resolution, and can be used for realizing interaction between users and algorithms.
Before projection operation, the distributed image stitching algorithm adopted by the application uses a thin plate spline method to perform deformation treatment on the image in the overlapping area so as to eliminate fuzzy double images; adopting linearization homography smooth extrapolation to global transformation to reduce projection distortion; and calculating deformation at the grid nodes by using a grid division method, linearly interpolating the rest pixels by using points, and improving the operation efficiency of the algorithm. The splicing algorithm and the software provided by the application can effectively eliminate the fuzzy ghost and the projection distortion phenomenon of the edge area in the image, obtain the seamless smooth high-resolution wide-area panoramic image, have the advantages of high splicing precision, high speed, wide range, simple interaction, convenience in operation and the like, and have higher application value.
The second embodiment of the present application, corresponding to the first embodiment, introduces a distributed photoelectric-oriented image stitching device, as shown in fig. 8, including the following components:
an onboard image preprocessing unit configured to perform image preprocessing on the distributed image data;
the characteristic point extraction unit is configured to extract characteristic points of the current image and reject the mismatching characteristic point pairs;
the binding adjustment unit is configured to carry out binding adjustment on the current image, combine global binding adjustment with a loop detection method and eliminate accumulated errors by comparing errors of a transformation relation generated by a minimum loop and a transformation relation of a connecting edge;
and the splicing and fusing unit is configured to perform fusion processing on the current spliced image by adopting a fusion algorithm so as to finish splicing the distributed image data.
In this embodiment, the on-board image preprocessing unit is further configured to: the method is used for denoising pretreatment and enhancement pretreatment, wherein the denoising pretreatment comprises mean value filtering denoising, gaussian filtering denoising and median filtering denoising; the enhancement preprocessing comprises histogram equalization, gray level correction, transformation and self-adaptive segmentation histogram enhancement.
In this embodiment, the feature point extraction unit is further configured to:
and extracting feature points of the current image by using a configured SIFT algorithm, and removing the error matching feature point pairs by using a configured random sampling consistency algorithm.
In this embodiment, the feature point extraction unit is further configured to: and extracting feature points of the current image by using SURF and ORB algorithms.
In this embodiment, the fusion algorithm includes: weight average fusion and eclosion fusion.
In this embodiment, after the step of extracting the feature points of the current image and rejecting the mismatching feature point pairs, the method further includes: and when the wrong characteristic point pair exists, purifying the abnormal characteristic point.
In this embodiment, the processing platform includes: CPU, GPU, qtDesigner, pySide2, pyCharm.
A third embodiment of the present application, an electronic device, as shown in fig. 9, can be understood as a physical device, including a processor and a memory storing instructions executable by the processor, when the instructions are executed by the processor, performs the following operations:
step S1, acquiring distributed image data, wherein the distributed image data is an image of an area to be spliced, which is acquired under different position conditions by using at least two image acquisition devices respectively;
s2, performing image preprocessing on the distributed image data;
step S3, extracting feature points of the current image, and eliminating the error matching feature point pairs;
step S4, binding adjustment is carried out on the current image, global binding adjustment is combined with a loop detection method, and accumulated errors are eliminated by comparing errors of a conversion relation generated by a minimum loop and a conversion relation of a connecting edge;
and S5, performing fusion processing on the images after the current splicing by adopting a fusion algorithm to finish the splicing of the distributed image data.
In the fourth embodiment of the present application, the flow of the image stitching method for distributed photoelectricity in this embodiment is the same as that in the first, second or third embodiments, except that in engineering implementation, this embodiment may be implemented by means of software plus a necessary general hardware platform, and certainly may also be implemented by hardware, but in many cases, the former is a better implementation. Based on such understanding, the method of the present application may be embodied in the form of a computer software product stored on a storage medium (e.g., ROM/RAM, magnetic disk, optical disk) comprising instructions for causing an apparatus to perform the method of the embodiments of the present application.
While the application has been described in connection with specific embodiments thereof, it is to be understood that these drawings are included in the spirit and scope of the application, it is not to be limited thereto.
Claims (10)
1. The distributed photoelectric-oriented image stitching method is characterized by comprising the following steps of:
acquiring distributed image data, wherein the distributed image data is an image of an area to be spliced, which is acquired under different position conditions by utilizing at least two image acquisition devices;
performing image preprocessing on the distributed image data;
extracting feature points of the current image, and eliminating the error matching feature point pairs;
binding adjustment is carried out on the current image, global binding adjustment is combined with a loop detection method, and accumulated errors are eliminated by comparing errors of a transformation relation generated by a minimum loop and a transformation relation of a connecting edge;
and carrying out fusion processing on the images after current splicing by adopting a fusion algorithm to finish the splicing of the distributed image data.
2. The method for distributed photovoltaic-oriented image stitching according to claim 1, wherein,
the image preprocessing comprises denoising preprocessing and enhancement preprocessing, wherein the denoising preprocessing comprises mean value filtering denoising, gaussian filtering denoising and median filtering denoising; the enhancement preprocessing comprises histogram equalization, gray level correction, transformation and self-adaptive segmentation histogram enhancement.
3. The distributed photoelectric-oriented image stitching method according to claim 1, wherein the extracting feature points of the current image and eliminating the mismatching feature point pairs includes:
and extracting feature points of the current image by using a configured SIFT algorithm, and removing the error matching feature point pairs by using a configured random sampling consistency algorithm.
4. The distributed photovoltaic-oriented image stitching method according to claim 3, wherein the feature extraction further comprises: and extracting feature points of the current image by using SURF and ORB algorithms.
5. The distributed photovoltaic-oriented image stitching method according to claim 1, wherein the fusion algorithm comprises: weight average fusion and eclosion fusion.
6. The distributed photovoltaic-oriented image stitching method according to claim 1, wherein after the step of extracting feature points from the current image and rejecting pairs of mismatching feature points, the method further comprises: and when the wrong characteristic point pair exists, purifying the abnormal characteristic point.
7. The distributed photovoltaic-oriented image stitching method according to any one of claims 1 to 6, wherein the processing platform upon which the method is based comprises: CPU, GPU, qtDesigner, pySide2, pyCharm.
8. An image stitching device for distributed photoelectricity, comprising:
an acquisition unit configured to acquire distributed image data, wherein the distributed image data is an image of a region to be stitched acquired under different position conditions by using at least two image acquisition devices, respectively;
an onboard image preprocessing unit configured to perform image preprocessing on the distributed image data;
the characteristic point extraction unit is configured to extract characteristic points of the current image and reject the mismatching characteristic point pairs;
the binding adjustment unit is configured to carry out binding adjustment on the current image, combine global binding adjustment with a loop detection method and eliminate accumulated errors by comparing errors of a transformation relation generated by a minimum loop and a transformation relation of a connecting edge;
and the splicing and fusing unit is configured to perform fusion processing on the current spliced image by adopting a fusion algorithm so as to finish splicing the distributed image data.
9. An electronic device, the electronic device comprising: memory, a processor and a computer program stored on the memory and executable on the processor, which when executed by the processor, performs the steps of the distributed electro-optical oriented image stitching method according to any one of claims 1 to 7.
10. A computer storage medium, wherein a computer program is stored on the computer storage medium, which when executed by a processor, implements the steps of the distributed electro-optical oriented image stitching method according to any one of claims 1 to 7.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202310529250.3A CN116883235A (en) | 2023-05-11 | 2023-05-11 | Distributed photoelectric oriented image stitching method and device |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202310529250.3A CN116883235A (en) | 2023-05-11 | 2023-05-11 | Distributed photoelectric oriented image stitching method and device |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN116883235A true CN116883235A (en) | 2023-10-13 |
Family
ID=88264984
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202310529250.3A Pending CN116883235A (en) | 2023-05-11 | 2023-05-11 | Distributed photoelectric oriented image stitching method and device |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN116883235A (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN117541764A (en) * | 2024-01-09 | 2024-02-09 | 北京大学 | Image stitching method, electronic equipment and storage medium |
-
2023
- 2023-05-11 CN CN202310529250.3A patent/CN116883235A/en active Pending
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN117541764A (en) * | 2024-01-09 | 2024-02-09 | 北京大学 | Image stitching method, electronic equipment and storage medium |
| CN117541764B (en) * | 2024-01-09 | 2024-04-05 | 北京大学 | Image stitching method, electronic equipment and storage medium |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN110827200B (en) | Image super-resolution reconstruction method, image super-resolution reconstruction device and mobile terminal | |
| CN111145238B (en) | Three-dimensional reconstruction method and device for monocular endoscopic image and terminal equipment | |
| CN111062905B (en) | Infrared and visible light fusion method based on saliency map enhancement | |
| US20170011710A1 (en) | Unified optimization method for end-to-end camera image processing for translating a sensor captured image to a display image | |
| US11281897B2 (en) | Gesture shaking recognition method and apparatus, and gesture recognition method | |
| CN106875419B (en) | Weak and small moving target tracking loss re-detection method based on NCC matching frame difference | |
| CN109685045B (en) | Moving target video tracking method and system | |
| CN110503620B (en) | Image fusion method based on Fourier spectrum extraction | |
| Tang et al. | Single image dehazing via lightweight multi-scale networks | |
| CN111553841B (en) | Real-time video splicing method based on optimal suture line updating | |
| CN108961163A (en) | A kind of high-resolution satellite image super-resolution reconstruction method | |
| CN103841298A (en) | Video image stabilization method based on color constant and geometry invariant features | |
| CN111510691A (en) | Color interpolation method and device, equipment and storage medium | |
| CN116883235A (en) | Distributed photoelectric oriented image stitching method and device | |
| CN112907493A (en) | Multi-source battlefield image rapid mosaic fusion algorithm under unmanned aerial vehicle swarm cooperative reconnaissance | |
| Huang et al. | Removing reflection from a single image with ghosting effect | |
| CN107705295B (en) | Image difference detection method based on robust principal component analysis method | |
| CN109636731A (en) | A kind of removing method of image streaking, electronic equipment and storage medium | |
| Zhou et al. | Mesoscopic photogrammetry with an unstabilized phone camera | |
| CN105069764B (en) | A kind of image de-noising method and system based on Edge track | |
| CN109615584B (en) | SAR image sequence MAP super-resolution reconstruction method based on homography constraint | |
| CN114219706A (en) | Image fast splicing method based on reduction of grid partition characteristic points | |
| CN111008555B (en) | Unmanned aerial vehicle image small and weak target enhancement extraction method | |
| CN112927166A (en) | Infrared polarization angle image combined noise reduction method and device based on phase correction | |
| Wang et al. | Depth map restoration and upsampling for kinect v2 based on ir-depth consistency and joint adaptive kernel regression |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |