CN108399612A - Based on the pyramidal three light images intelligent method for fusing of bilateral filtering - Google Patents
Based on the pyramidal three light images intelligent method for fusing of bilateral filtering Download PDFInfo
- Publication number
- CN108399612A CN108399612A CN201810118085.1A CN201810118085A CN108399612A CN 108399612 A CN108399612 A CN 108399612A CN 201810118085 A CN201810118085 A CN 201810118085A CN 108399612 A CN108399612 A CN 108399612A
- Authority
- CN
- China
- Prior art keywords
- image
- pyramid
- pyramidal
- bilateral filtering
- fusing
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
- 238000001914 filtration Methods 0.000 title claims abstract description 58
- 230000002146 bilateral effect Effects 0.000 title claims abstract description 50
- 238000000034 method Methods 0.000 title claims abstract description 42
- 238000001228 spectrum Methods 0.000 claims abstract description 39
- 230000004927 fusion Effects 0.000 claims abstract description 34
- 238000005516 engineering process Methods 0.000 claims abstract description 21
- 238000012545 processing Methods 0.000 claims abstract description 19
- 238000004587 chromatography analysis Methods 0.000 claims abstract description 6
- 230000003287 optical effect Effects 0.000 claims abstract description 6
- 206010045178 Tunnel vision Diseases 0.000 claims abstract description 4
- 238000012937 correction Methods 0.000 claims abstract description 4
- 238000002156 mixing Methods 0.000 claims abstract description 3
- 230000008569 process Effects 0.000 claims description 16
- 239000000758 substrate Substances 0.000 claims description 12
- 238000005070 sampling Methods 0.000 claims description 11
- 238000004364 calculation method Methods 0.000 claims description 10
- 238000010586 diagram Methods 0.000 claims description 10
- 238000013461 design Methods 0.000 claims description 7
- PCHJSUWPFVWCPO-UHFFFAOYSA-N gold Chemical compound [Au] PCHJSUWPFVWCPO-UHFFFAOYSA-N 0.000 claims description 5
- 239000010931 gold Substances 0.000 claims description 5
- 229910052737 gold Inorganic materials 0.000 claims description 5
- 238000000354 decomposition reaction Methods 0.000 claims description 4
- 238000010606 normalization Methods 0.000 claims description 3
- 238000007500 overflow downdraw method Methods 0.000 claims description 3
- 238000004381 surface treatment Methods 0.000 claims description 3
- 238000013519 translation Methods 0.000 claims description 3
- 239000012466 permeate Substances 0.000 abstract description 3
- 210000003128 head Anatomy 0.000 description 21
- 238000000605 extraction Methods 0.000 description 6
- 241000269980 Pleuronectidae Species 0.000 description 5
- 238000007499 fusion processing Methods 0.000 description 5
- 238000006073 displacement reaction Methods 0.000 description 4
- 238000003384 imaging method Methods 0.000 description 4
- 230000008901 benefit Effects 0.000 description 3
- 230000010354 integration Effects 0.000 description 3
- 230000003595 spectral effect Effects 0.000 description 3
- 230000002159 abnormal effect Effects 0.000 description 2
- 238000004458 analytical method Methods 0.000 description 2
- 230000007547 defect Effects 0.000 description 2
- 238000009434 installation Methods 0.000 description 2
- 238000012544 monitoring process Methods 0.000 description 2
- 239000000047 product Substances 0.000 description 2
- 230000004075 alteration Effects 0.000 description 1
- 230000007812 deficiency Effects 0.000 description 1
- 238000001514 detection method Methods 0.000 description 1
- 230000004069 differentiation Effects 0.000 description 1
- 238000009826 distribution Methods 0.000 description 1
- 230000000694 effects Effects 0.000 description 1
- 238000010891 electric arc Methods 0.000 description 1
- 238000009776 industrial production Methods 0.000 description 1
- 238000007689 inspection Methods 0.000 description 1
- 239000000463 material Substances 0.000 description 1
- 239000000155 melt Substances 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 230000009467 reduction Effects 0.000 description 1
- 238000001931 thermography Methods 0.000 description 1
- 230000000007 visual effect Effects 0.000 description 1
Classifications
-
- G06T3/147—
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T3/00—Geometric image transformation in the plane of the image
- G06T3/60—Rotation of a whole image or part thereof
- G06T3/608—Skewing or deskewing, e.g. by two-pass or three-pass rotation
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T5/00—Image enhancement or restoration
- G06T5/50—Image enhancement or restoration by the use of more than one image, e.g. averaging, subtraction
-
- G06T5/70—
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/30—Determination of transform parameters for the alignment of images, i.e. image registration
- G06T7/33—Determination of transform parameters for the alignment of images, i.e. image registration using feature-based methods
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20016—Hierarchical, coarse-to-fine, multiscale or multiresolution image processing; Pyramid transform
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20024—Filtering details
- G06T2207/20028—Bilateral filtering
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20212—Image combination
- G06T2207/20221—Image fusion; Image merging
Landscapes
- Engineering & Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Theoretical Computer Science (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Image Processing (AREA)
- Image Analysis (AREA)
Abstract
The present invention proposes one kind and being based on the pyramidal three light images intelligent method for fusing of bilateral filtering, includes the following steps：It is designed first with parallel optical axis and sets up three light video camera heads, three light video camera head face Same Scenes is allowed to carry out video signal collection；Collected three tunnel vision signal is subjected to offset deviation and rotational distortion correction；Two-sided filter has been subjected to pyramid Image processing, when pyramid chromatography after treatment, has needed to merge three kinds of pyramid information that three spectrum pictures obtain.The present invention effectively permeates two-sided filter and Pyramid technology algorithm a new effective total algorithm, original three spectrum pictures fusion is become into the blending image that a width contains three kinds of all features of spectrum, and it exports outward, in this way, user, which only needs to observe this piece image, can complete acquisition to all target signatures in scene, substantially increase observation efficiency and user uses cheap property.
Description
Technical field
The present invention relates to intelligent detecting imaging and technical field of image processing, more particularly to a kind of to be based on bilateral filtering gold word
Three light image intelligent method for fusing of tower.
Background technology
It is international at present and domestic explosion-proof etc. with industry in community security protection, industrial production, security against fire, forest fire protection, safety check
The camera of different-waveband is widely used in the relevant application field of machine vision to be supervised to scene and target therein
It surveys.Since the target signature in nature can show different spectral signatures, it is completed to field using single camera
The monitoring process of scape and target can no longer meet modern military-civil technology requirement.Current existing solution is using more
The camera of a different-waveband is carried out while being observed to scene, for example is determined the target in scene using visible image capturing head
Position recycles thermal infrared imager to carry out thermal map analysis to the target；For another example used first during electric power network O＆M inspection
Thermal infrared imager detects the heat distribution of the electrical connection such as cabinet, blade joint, judges whether there is abnormal high temperature phenomenon and goes out
It is existing, recycle ultraviolet light video camera head is to whether there is high pressure arc discharge or electric leakage at abnormal high temperature phenomena such as.It is militarily more
Camera cooperation mode is also with a wide range of applications, such as when tracking missile target, is examined using thermal infrared imager
The guided missile during long-distance flight is surveyed, then the wake flame feature during missile flight is divided using ultraviolet light video camera head
Analysis, to effectively avoid the jamming bomb that guided missile is sent out from influencing the defect of infrared thermal imaging target identification, while also being capable of basis
Thermal image and wake flame feature effectively distinguish the type of airbound target.It can be seen that in industrial machine visual field, it is image more
The design concept that head cooperates, which has had been provided with, to be quite widely applied scene and has tentatively come into operation, and is the following vision
The important directions of imaging progress.
Current product solution main in the industry is to set up the more discrete monitorings of camera for Same Scene, such as together
When set up a visible image capturing head and a thermal infrared imager, vision signal is output in one and is controlled on host, in host
Software kit in simultaneously show this two-path video, observed simultaneously by user.Such maximum defect of system is following several
Point：1. whole system structure disperses, camera and middle control host are separate devices, and volume is excessive when erected on site and cost mistake
It is high；When 2. two-path video image is simultaneously displayed on display not due to the image device focal plane picture dot size of different-waveband
Together, even larger parallax can also be had by causing to be observed same scene, i.e., the scene in two-way image only has part
Identical, this just brings difficulty to target detection process；3. because user needs while observing two kinds of videos, therefore human eye needs
Observation is ceaselessly toggled between two-path video, it is not convenient to cause vision to user；4. system as, which exists, to be carried not
Yi Xing is merely able to be observed mounted on fixed position, can not carry, therefore largely limits its application scenarios
Expansion.In conclusion being badly in need of a small, light-weight, the Multi-spectral image fusion product that can carry and be fixedly mounted is filled up
The market vacancy.
It solves not generate parallax when the video of the Same Scene for acquiring a variety of wave band cameras is viewed by a user, again
Very easily user can be allowed to observe, best scheme is to carry out image co-registration process, is had been proposed in the industry at present
The image fusion technology of obstructed type.Wherein, it is most widely infrared in military-civil video solution and the double light of visible light
Compose integration technology.With advances in technology and user demand it is higher and higher, traditional double spectrum integration technologies are gradually not
The complex target characteristic information occurred in scene can be coped with completely, and therefore, one kind being capable of covering visible light, infrared light and ultraviolet
The high-precision intelligent integration technology of three imaging spectrals of light is particularly important.
Invention content
The purpose of the present invention aims to solve at least one of described technological deficiency.
For this purpose, a kind of based on bilateral filtering pyramidal three light image Intelligent Fusion side it is an object of the invention to propose
Method can merge the image of visible light, infrared light and ultraviolet light collection, and user only needs to observe this piece image
To complete the acquisition to all target signatures in scene, substantially increases observation efficiency and user uses cheap property.
To achieve the goals above, the present invention provides a kind of based on bilateral filtering pyramidal three light image Intelligent Fusion side
Method includes the following steps：
Step S1, first with parallel optical axis design set up three light video camera heads, allow three light video camera head face Same Scenes into
Row video signal collection；
Collected three tunnel vision signal is carried out offset deviation and rotational distortion corrects by step S2；
The bearing calibration of offset deviation and rotational distortion uses the image registration school under cartesian coordinate system and polar coordinate system
Normal operation method calculates target feature point in image, and calculates the pixel of identical characteristic point appearance simultaneously in three width spectrum pictures
Position, and calculated using the translation scaling of cartesian coordinate system and calculated with the twiddle factor under polar coordinate system, by three spectrograms
The angle position of picture, size are all adjusted to identical dimension, to carry out subsequent fusion calculation；
Two-sided filter has been carried out pyramid Image processing, i.e., the image result obtained two-sided filter by step S3
It carries out pyramid down-sampling layering again respectively, the result that layering obtains is subjected to bilateral filtering operation again, by repeatedly chromatographing connection
Close the mode of operation, it is ensured that all characteristic informations, which can be fully extracted out, in image carries out fusion calculation；
Step S4 needs three kinds of pyramid information for obtaining three spectrum pictures when pyramid chromatography after treatment
It is merged；Fusion method is to be weighted the corresponding pyramidal layer substrate of three spectrum pictures with substrate to merge, feature with
Feature is weighted fusion；Since pyramid top, per Weighted Fusion once with regard to carrying out pyramid inverse operation, by high-rise gold
Word tower dimension expands into next high level and is calculated with the secondary high-rise Weighted Fusion that carries out again, until all Pyramid technologies
It all calculates and finishes, three original at this time spectrum pictures, which just merge, becomes the fusion figure that a width contains three kinds of all features of spectrum
Picture, and exporting outward, in this way, user only needs to observe this piece image just and can complete to obtain all target signatures in scene
It takes.
Further, in step s 2, the timing for carrying out offset deviation and rotational distortion need to be by three spectrum pictures
Identical size is zoomed to be corrected again later.
Further, in step s3, two-sided filter has carried out mainly comprising the following steps for pyramid Image processing：
Since original three spectrum pictures, lower surface treatment is carried out to each spectrum picture：The image of most substrate passes through
First time bilateral filtering processing after obtained fundamental frequency layer and levels of detail two images, then by the fundamental frequency tomographic image obtained at this time into
Row pyramid down-sampling process, obtains second layer pyramid diagram picture, is then carried out respectively to the base and pyramid image of the second layer
Bilateral filtering processing, then the fundamental frequency layer and levels of detail two images of the second layer are obtained, then by the fundamental frequency image of the second layer
Continue bilateral filtering processing, and so on, until all having carried out bilateral filtering processing to all layers of pyramid.
Further, in step s 4, when the pyramid information of three spectrum pictures is merged, substrate image is chosen for
Visible images, it is infrared light image and ultraviolet light image feature to need to be superimposed.
Further, two-sided filter is a kind of effective nonlinear filter for distinguishing feature and noise in image.
Further, the calculation formula of two-sided filter is：
Wherein, k (i, j) represents normalization coefficient：
Here mark (i ', j ') ∈ Si,jIt is the adjacent element in image to represent (i ', j ') and (i, j)；Under normal circumstances
gsIt is selected as a standardized gaussian kernel function, i.e., the sum of all coefficients in two-sided filter are 1；In addition in intensity domain
Also a gaussian kernel function is used；The total weight k (i, j) of this template is by by two Gausses of spatial domain and intensity domain
The result of template is multiplied to obtain；Its range should be between 0-1.
Further, in step S3-S4, it is assumed that pyramidal l tomographic images are G1, the pyramidal structure of bilateral filtering
Formula is：Gl(i, j)=w (2i, 2j, σ) * Gl-1(2i, 2j), (1≤l≤N), (3)；
Wherein, N indicates that pyramidal level number, * indicate that convolution, i/j all indicate the coordinate of image, the expression side w (2i, 2j, σ)
Difference is the bilateral filtering kernel function of σ；
Therefore, G0、G1、…、GNJust constitute fundamental frequency layer pyramid, G0It is identical as original image；The size of current tomographic image
It is followed successively by the 1/4 of last layer image size, pyramidal total number of plies is N+1.
Further, by image GlThe image of quadruplication is obtained by interpolationIts picture size and Gl-1Size phase
Together；Then obtained by formula (3)：
In formula (2)：
It enables
By LP0、LP1、…、LPNThe pyramid of composition is bilateral filtering pyramid；Its each tomographic image is fundamental frequency layer
The difference of this tomographic image of pyramid and its high one layer of image image after interpolation is amplified, this process are equivalent to bandpass filtering, because
This bilateral filtering pyramid is also known as band logical pyramid decomposition；Finally, by by all every layers obtained of fundamental frequency layer pyramid
It is restored to get to the image after fusion by the method that pyramid is rebuild with obtained details coefficients.
Further, the formula of pyramid reconstruction is：
The present invention's is had the advantages that based on the pyramidal three light images intelligent method for fusing of bilateral filtering：
1, the present invention uses parallel optical axis when install three light video camera heads and designs, to ensure scene that camera is got
There is no the case where geometric distortion occur.
2, due to machine error that three light video camera heads are installed in structure, it is impossible to accomplish observed by three cameras
Scene is identical, therefore the present invention carries out offset deviation and rotational distortion correction first before carrying out fusion process, eliminates
Offset deviation and rotational distortion ensure that the high-precision of follow-up fusion process.
3, two-sided filter of the invention is a kind of effective nonlinear filter for distinguishing feature and noise in image, because
Its nonlinear filtering wave property, can be by spatial domain and intensity domain two indices jointly to the image slices in selected filter window
Plain value is arbitrated, distinguishing characteristic and noise information, and then in fusion process, avoids having brought more noise into so that most
Whole syncretizing effect greatly improves.
4, the present invention, i.e., will be double using the convergence strategy for being used as three light images based on bilateral filtering pyramid CT calculating method
Side filter and Pyramid technology algorithm effectively permeate a new effective total algorithm, by three original spectrum
Image co-registration becomes a width and contains the blending image of three kinds of all features of spectrum, and exports outward, in this way, user only needs to see
Acquisition to all target signatures in scene can be completed by examining this piece image, substantially increase observation efficiency and user uses
Cheap property.
5, the present invention can be good at carrying out the image from different spectrum segments to include image rectification, image co-registration mistake
The image for being simultaneously from different cameras and video are eliminated scene parallax and image size mismatch phenomenon by journey, and by being based on
Arbitrary two kinds in three kinds of spectrum or all information are melted in real time for the bilateral filtering pyramid algorith that designs of the present invention
It closes, this technology convergence strategy precision is high, and fusion results are outstanding.
6, of the invention since two-sided filter can be good at the fundamental frequency information (energy information) and feature letter in image
Breath (detailed information) is detached, therefore can realize high-precision image superposition in fusion；By itself and Pyramid technology
Technology is combined, good sifting property and multiple filtering operation characteristic；And traditional two-sided filter is just for common
Single image carries out a filtering operation.
The additional aspect of the present invention and advantage will be set forth in part in the description, and will partly become from the following description
Obviously, or practice through the invention is recognized.
Description of the drawings
The above-mentioned and/or additional aspect and advantage of the present invention will become in the description from combination following accompanying drawings to embodiment
Obviously and it is readily appreciated that, wherein：
Fig. 1 is the main process figure of the present invention；
Fig. 2 is that the carry out offset deviation of the present invention and rotational distortion correct schematic diagram；
Fig. 3 is the calculating process schematic diagram of the two-sided filter of the present invention；
Fig. 4 is that the complete pyramid of the present invention chromatographs process schematic；
Fig. 5 is the schematic diagram of detail extraction Pyramid Components during the pyramid of the present invention chromatographs；
Fig. 6 is the schematic diagram of Energy extraction Pyramid Components during the pyramid of the present invention chromatographs；
Fig. 7 is the pyramid algorithm for reconstructing schematic diagram of the present invention.
Specific implementation mode
The embodiment of the present invention is described below in detail, examples of the embodiments are shown in the accompanying drawings, wherein from beginning to end
Same or similar label indicates same or similar element or element with the same or similar functions.Below with reference to attached
The embodiment of figure description is exemplary, it is intended to for explaining the present invention, and is not considered as limiting the invention.
The present invention provides a kind of based on the pyramidal three light images intelligent method for fusing of bilateral filtering, refer to the attached drawing 1-7 institutes
Show, includes the following steps：
Step S1, first with parallel optical axis design set up three light video camera heads, allow three light video camera head face Same Scenes into
Row video signal collection.
The machine error installed in structure due to three light video camera heads, it is impossible to accomplish the field observed by three cameras
Scape is identical, and therefore, the present invention uses parallel optical axis design when installing three light video camera heads, to ensure that camera is got
Scene there is no there is the case where geometric distortion, additionally due to camera industrial structure installation site so that three light video camera heads
In space installation be there are physical displacement, to will appear displacement and the rotating deviation of scene in observation to unified scene,
There is also certain position deviations between the obtained scene information of three light video camera heads, it is therefore desirable to before carrying out fusion process
The part for obtaining same scene in the obtained scene information of three light video camera heads first carries out subsequent fusion treatment；Due to tying
Structure can not accomplish perfect Founder when installing three light video camera heads, and a degree of rotational distortion is there will necessarily be between three light video camera heads
Situation, therefore, it is also desirable to which this rotational distortion preferentially to be eliminated to the high-precision that can ensure follow-up fusion process.
Collected three tunnel vision signal is carried out offset deviation and rotational distortion corrects by step S2；
Since the camera picture dot size and inter-pixel of different spectrum are away from differing, cause to pass through camera lens to the same target
After focusing on focal plane corresponding number of pixels differ and the situation in different size that occurs, need carrying out displacement and rotation
Turn image scaling that deviation timing obtains three spectrum of correction to identical size.
Therefore when carrying out follow-up fusion treatment, it is necessary first to first be corrected displacement and rotating deviation.
The bearing calibration of offset deviation and rotational distortion uses the image registration school under cartesian coordinate system and polar coordinate system
Normal operation method calculates target feature point in image, and calculates the pixel of identical characteristic point appearance simultaneously in three width spectrum pictures
Position, and calculated using the translation scaling of cartesian coordinate system and calculated with the twiddle factor under polar coordinate system, by three spectrograms
The angle position of picture, size are all adjusted to identical dimension, to carry out subsequent fusion calculation.
Two-sided filter has been carried out pyramid Image processing, i.e., the image result obtained two-sided filter by step S3
It carries out pyramid down-sampling layering again respectively, the result that layering obtains is subjected to bilateral filtering operation again, it is ensured that acquisition
Three light images can detach feature and noise in each pyramidal layer, by the side for repeatedly chromatographing join operation
Formula, it is ensured that all characteristic informations, which can be fully extracted out, in image carries out fusion calculation.As seen in figures 3-6.
The present invention is the image fusion technology based on bilateral filtering pyramid algorith, wherein by two-sided filter and pyramid
Hierarchical algorithm effectively permeates a new effective total algorithm.Wherein two-sided filter is a kind of effective differentiation
The nonlinear filter of feature and noise in image.
As shown in figure 3, the calculation formula of two-sided filter is：
Wherein, k (i, j) represents normalization coefficient：
Here mark (i ', j ') ∈ Si,jIt is the adjacent element in image to represent (i ', j ') and (i, j)；Under normal circumstances
gsIt is selected as a standardized gaussian kernel function, i.e., the sum of all coefficients in two-sided filter are 1；In addition in intensity domain
Also a gaussian kernel function is used；The total weight k (i, j) of this template is by by two Gausses of spatial domain and intensity domain
The result of template is multiplied to obtain；Its range should be between 0-1.
σsWith σrIt indicates the standard deviation criteria of two gaussian kernel functions, controls the expansion range of two gaussian kernel functions.σs
Determine the scale of close region, thus must relationship proportional to the size of image, here the present invention choose image diagonal
The 2.5% of size.σrSelection it is more crucial because which represent the amplitudes of so-called details.If the range of signal fluctuation is small
In σr, then this signal fluctuation will be considered as details, i.e., it can be smooth by two-sided filter, and it has been split into levels of detail
In., whereas if the range of this fluctuation is more than σr, then due to the nonlinear characteristic of two-sided filter, this details will
It is effectively maintained to fundamental frequency layer.Here present invention selection human eye can differentiate the 20% of gray level, i.e., 25 are used as σrValue.
This value all has well adapting to property in view of human eye is to the resolution capability of gray scale, and for different scenes.
Since two-sided filter can be good at the fundamental frequency information (energy information) and characteristic information (details letter in image
Breath) it is detached, therefore can realize high-precision image superposition in fusion.Traditional two-sided filter is just for common
Single image carries out a filtering operation.Based on its good sifting property, the present invention carries out itself and Pyramid technology technology
It combines.
It is illustrated in figure 4 the schematic diagram of the complete pyramid chromatography process of the present invention one width picture of parsing, pyramid is most lower
Side is original image, per upper layer, has as been carried out current original image in of both detail extraction and Energy extraction
Hold, while image dimension being reduced to a quarter of artwork, Fig. 5 and Fig. 6 are illustrated respectively in details during pyramid chromatography
The Pyramid Components schematic diagram of extraction and Energy extraction.
Two-sided filter has carried out mainly comprising the following steps for pyramid Image processing：
Since original three spectrum pictures, lower surface treatment is carried out to each spectrum picture：The image of most substrate passes through
First time bilateral filtering processing after obtained fundamental frequency layer and levels of detail two images, then by the fundamental frequency tomographic image obtained at this time into
Row pyramid down-sampling process, obtains second layer pyramid diagram picture, is then carried out respectively to the base and pyramid image of the second layer
Bilateral filtering processing, then the fundamental frequency layer and levels of detail two images of the second layer are obtained, then by the fundamental frequency image of the second layer
Continue bilateral filtering processing, and so on, until all having carried out bilateral filtering processing to all layers of pyramid.
Since the process of pyramid down-sampling is a dimensionality reduction operation for reducing four times every time, pyramidal number of plies root
Depending on the size of initial pictures, for example, initial pictures dimension is 640 × 480, every time according to the speed down-sampling for reducing 4 times
When, the sum of all pixels to image at the 7th layer is 75, can not continue to be eliminated by 4, due to being to sum of all pixels when layer 5
4800, almost picture material cannot be distinguished in naked eyes, therefore total pyramid number of plies can be 5 layers, can also be according to circumstances
Select other numbers of plies.Its calculating process is as follows：
If original image is G0, with G0As pyramidal 0th layer, also known as basal layer, to original image carry out bilateral filtering and
Interlacing obtains the pyramidal first layer of fundamental frequency layer every the down-sampling of row；Low-pass filtering and down-sampling are carried out to the first tomographic image again,
Obtain the pyramidal second layer of fundamental frequency layer；Above procedure is repeated, fundamental frequency layer pyramid is constituted.Assuming that pyramidal 1st tomographic image
For G1, the pyramidal building process of bilateral filtering is：
Assuming that pyramidal l tomographic images are G1, the pyramidal structure formula of bilateral filtering is：
Gl(i, j)=w (2i, 2j, σ) * Gl-1(2i, 2j), (1≤l≤N), (3)；
Wherein, N indicates that pyramidal level number, * indicate that convolution, i/j all indicate the coordinate of image, the expression side w (2i, 2j, σ)
Difference is the bilateral filtering kernel function of σ；
Therefore, G0、G1、…、GNJust constitute fundamental frequency layer pyramid, G0It is identical as original image；The size of current tomographic image
It is followed successively by the 1/4 of last layer image size, pyramidal total number of plies is N+1.As it can be seen that the fundamental frequency layer pyramid decomposition of image is logical
It crosses and low-pass filtering is carried out to bottom layer image successively, then filter result made interlacing every 2 down-sampling of drop of row to realize.
Secondly, by image GlThe image of quadruplication is obtained by interpolationIts picture size and Gl-1Size it is identical；
Then obtained by formula (3)：
In formula (2)：
It enables
By LP0、LP1、…、LPNThe pyramid of composition is bilateral filtering pyramid；Its each tomographic image is fundamental frequency layer
The difference of this tomographic image of pyramid and its high one layer of image image after interpolation is amplified, this process are equivalent to bandpass filtering, because
This bilateral filtering pyramid is also known as band logical pyramid decomposition；Finally, by by all every layers obtained of fundamental frequency layer pyramid
It is restored to get to the image after fusion by the method that pyramid is rebuild with obtained details coefficients, as shown in Figure 7.
Pyramid rebuild formula be：
Step S4 needs three kinds of pyramid information for obtaining three spectrum pictures when pyramid chromatography after treatment
It is merged.As shown in Figure 7.
Fusion method is to be weighted the corresponding pyramidal layer substrate of three spectrum pictures with substrate to merge, feature and spy
Sign is weighted fusion；Since pyramid top, per Weighted Fusion once with regard to carrying out pyramid inverse operation, by high-rise golden word
Tower dimension expands into next high level and is calculated with the secondary high-rise Weighted Fusion that carries out again, until all Pyramid technologies are complete
Portion calculates and finishes, and three original at this time spectrum pictures, which just merge, becomes the fusion figure that a width contains three kinds of all features of spectrum
Picture, and exporting outward, in this way, user only needs to observe this piece image just and can complete to obtain all target signatures in scene
It takes, substantially increases observation efficiency and user uses cheap property.
It should be noted that visible image capturing head, infrared pick-up head, ultraviolet light camera shooting may be used in three light video camera heads
Head can also use the camera of other light.When the pyramid information of three spectrum pictures is merged, substrate image is chosen for
Visible images, it is infrared light image and ultraviolet light image feature to need to be superimposed.
The present invention based on the pyramidal three light images intelligent method for fusing of bilateral filtering can will be seen that light, infrared light and
The image and video obtained under three imaging spectrals of ultraviolet light carries out the technology of real-time high-precision fusion.The technology can be good at
Image from different spectrum segments is carried out to include image rectification, image co-registration process.The figure of different cameras will be simultaneously from
Picture and video eliminate scene parallax and image size mismatch phenomenon, and the bilateral filtering pyramid designed by being based upon the present invention
Algorithm comes to arbitrary two kinds or all information progress real time fusion in three kinds of spectrum, and this technology convergence strategy precision is high, melts
It is outstanding to close result.
Although the embodiments of the present invention has been shown and described above, it is to be understood that above-described embodiment is example
Property, it is not considered as limiting the invention, those skilled in the art are not departing from the principle of the present invention and objective
In the case of can make changes, modifications, alterations, and variations to the above described embodiments within the scope of the invention.The scope of the present invention
It is extremely equally limited by appended claims.
Claims (9)
1. one kind being based on the pyramidal three light images intelligent method for fusing of bilateral filtering, which is characterized in that include the following steps：
Step S1 designs first with parallel optical axis and sets up three light video camera heads, three light video camera head face Same Scenes is allowed to be regarded
Frequency signal acquisition；
Collected three tunnel vision signal is carried out offset deviation and rotational distortion corrects by step S2；
The bearing calibration of offset deviation and rotational distortion uses the image registration correction under cartesian coordinate system and polar coordinate system to calculate
Method calculates target feature point in image, and calculates the location of pixels of identical characteristic point appearance simultaneously in three width spectrum pictures,
And calculated using the translation scaling of cartesian coordinate system and calculated with the twiddle factor under polar coordinate system, by the angle of three spectrum pictures
Degree position, size are all adjusted to identical dimension, to carry out subsequent fusion calculation；
Two-sided filter has been carried out pyramid Image processing by step S3, i.e., the image result obtained two-sided filter is distinguished
Pyramid down-sampling layering is carried out again, the result that layering obtains is subjected to bilateral filtering operation again, by repeatedly chromatographing joint fortune
The mode of calculation, it is ensured that all characteristic informations, which can be fully extracted out, in image carries out fusion calculation；
Step S4 needs three kinds of pyramid information for obtaining three spectrum pictures to carry out when pyramid chromatography after treatment
Fusion；Fusion method is to be weighted the corresponding pyramidal layer substrate of three spectrum pictures with substrate to merge, feature and feature
It is weighted fusion；Since pyramid top, per Weighted Fusion once with regard to carrying out pyramid inverse operation, by high-rise pyramid
Dimension expands into next high level and is calculated with the secondary high-rise Weighted Fusion that carries out again, until all Pyramid technologies are whole
Calculating finishes, and three original at this time spectrum pictures, which just merge, becomes the blending image that a width contains three kinds of all features of spectrum,
And export outward, in this way, user only needs to observe this piece image just the acquisition that can be completed to all target signatures in scene.
2. being based on the pyramidal three light images intelligent method for fusing of bilateral filtering as described in claim 1, it is characterised in that：
In step S2, after three spectrum pictures need to be zoomed to identical size by the timing for carrying out offset deviation and rotational distortion
It is corrected again.
3. being based on the pyramidal three light images intelligent method for fusing of bilateral filtering as described in claim 1, it is characterised in that：
In step S3, two-sided filter has carried out mainly comprising the following steps for pyramid Image processing：
Since original three spectrum pictures, lower surface treatment is carried out to each spectrum picture：The image of most substrate passes through first
Fundamental frequency layer and levels of detail two images have been obtained after secondary bilateral filtering processing, the fundamental frequency tomographic image obtained at this time is then subjected to gold
Word tower down-sampling process, obtains second layer pyramid diagram picture, is then carried out respectively to the base and pyramid image of the second layer bilateral
It is filtered, then obtains the fundamental frequency layer and levels of detail two images of the second layer, then continue the fundamental frequency image of the second layer
Bilateral filtering processing is carried out, and so on, until all having carried out bilateral filtering processing to all layers of pyramid.
4. being based on the pyramidal three light images intelligent method for fusing of bilateral filtering as described in claim 1, it is characterised in that：
In step S4, when the pyramid information of three spectrum pictures is merged, substrate image is chosen for visible images, needs to be superimposed
It is infrared light image and ultraviolet light image feature.
5. being based on the pyramidal three light images intelligent method for fusing of bilateral filtering as described in claim 1, it is characterised in that：It is double
Side filter is a kind of effective nonlinear filter for distinguishing feature and noise in image.
6. being based on the pyramidal three light images intelligent method for fusing of bilateral filtering as described in claim 1, it is characterised in that：It is double
The calculation formula of side filter is：
Wherein, k (i, j) represents normalization coefficient：
Here mark (i ', j ') ∈ Si,jIt is the adjacent element in image to represent (i ', j ') and (i, j)；G under normal circumstancessQuilt
Selection is a standardized gaussian kernel function, i.e., the sum of all coefficients in two-sided filter are 1；In addition it is also adopted in intensity domain
With a gaussian kernel function；The total weight k (i, j) of this template is by by two Gaussian templates of spatial domain and intensity domain
Result be multiplied to obtain；Its range should be between 0-1.
7. being based on the pyramidal three light images intelligent method for fusing of bilateral filtering as described in claim 1, it is characterised in that：
In step S3-S4, it is assumed that pyramidal l tomographic images are G1, the pyramidal structure formula of bilateral filtering is：
Gl(i, j)=w (2i, 2j, σ) * Gl-1(2i, 2j), (1≤l≤N), (3)；
Wherein, N indicates that pyramidal level number, * indicate that convolution, i/j all indicate that the coordinate of image, w (2i, 2j, σ) indicate that variance is
The bilateral filtering kernel function of σ；
Therefore, G0、G1、…、GNJust constitute fundamental frequency layer pyramid, G0It is identical as original image；The size of current tomographic image is successively
It is the 1/4 of last layer image size, pyramidal total number of plies is N+1.
8. being based on the pyramidal three light images intelligent method for fusing of bilateral filtering as claimed in claim 7, it is characterised in that：It will
Image GlThe image of quadruplication is obtained by interpolationIts picture size and Gl-1Size it is identical；Then obtained by formula (3)：
In formula (2)：
It enables
By LP0、LP1、…、LPNThe pyramid of composition is bilateral filtering pyramid；Its each tomographic image is fundamental frequency layer gold word
The difference of this tomographic image of tower and its high one layer of image image after interpolation is amplified, this process are equivalent to bandpass filtering, therefore double
Side Filter Pyramid is also known as band logical pyramid decomposition；Finally, by by all every layers obtained of fundamental frequency layer pyramid and
The method that the details coefficients arrived are rebuild by pyramid is restored to get to the image after fusion.
9. being based on the pyramidal three light images intelligent method for fusing of bilateral filtering as claimed in claim 8, it is characterised in that：Gold
Word tower rebuild formula be：
Priority Applications (2)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201810118085.1A CN108399612B (en) | 2018-02-06 | 2018-02-06 | Three-light image intelligent fusion method based on bilateral filtering pyramid |
| PCT/CN2018/096023 WO2019153651A1 (en) | 2018-02-06 | 2018-07-17 | Bilateral filter pyramid based three-light image intelligent fusion method |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201810118085.1A CN108399612B (en) | 2018-02-06 | 2018-02-06 | Three-light image intelligent fusion method based on bilateral filtering pyramid |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN108399612A true CN108399612A (en) | 2018-08-14 |
| CN108399612B CN108399612B (en) | 2022-04-05 |
Family
ID=63095287
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201810118085.1A Active CN108399612B (en) | 2018-02-06 | 2018-02-06 | Three-light image intelligent fusion method based on bilateral filtering pyramid |
Country Status (2)
| Country | Link |
|---|---|
| CN (1) | CN108399612B (en) |
| WO (1) | WO2019153651A1 (en) |
Cited By (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110400262A (en) * | 2019-04-10 | 2019-11-01 | 泰州市康平医疗科技有限公司 | Identification device based on customization data processing |
| CN110956592A (en) * | 2019-11-14 | 2020-04-03 | 北京达佳互联信息技术有限公司 | Image processing method, image processing device, electronic equipment and storage medium |
| CN111462032A (en) * | 2020-03-31 | 2020-07-28 | 北方夜视技术股份有限公司 | Method for fusing uncooled infrared image and solar blind ultraviolet image and application |
| CN111814511A (en) * | 2019-04-10 | 2020-10-23 | 泰州市康平医疗科技有限公司 | Identification method based on customized data processing |
| CN113643219A (en) * | 2021-08-03 | 2021-11-12 | 武汉三江中电科技有限责任公司 | Image imaging method and device based on three-light fusion |
Families Citing this family (9)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111292284B (en) * | 2020-02-04 | 2024-03-01 | 淮阴师范学院 | Color image fusion method based on dual-tree-quaternion wavelet transformation |
| CN111462025B (en) * | 2020-02-26 | 2023-04-07 | 宁波大学 | Infrared and visible light image fusion method based on multi-scale low-rank matrix decomposition |
| CN111652818B (en) * | 2020-05-29 | 2023-09-29 | 浙江大华技术股份有限公司 | Pyramid-based image filtering method, pyramid-based image filtering device and storage medium |
| CN112116632B (en) * | 2020-09-21 | 2023-12-05 | 中国科学院长春光学精密机械与物理研究所 | Method, device and medium for tracking target along tail smoke of target |
| CN112135068A (en) * | 2020-09-22 | 2020-12-25 | 视觉感知(北京)科技有限公司 | Method and device for fusion processing of multiple input videos |
| CN112465705B (en) * | 2020-12-08 | 2022-08-19 | 福州大学 | Visual field expanding system and method based on two-aperture rotating biprism |
| CN114815760B (en) * | 2022-06-27 | 2022-09-09 | 天津德通电气股份有限公司 | Tracing disposal method of safety production tracing disposal system |
| CN116681633B (en) * | 2023-06-06 | 2024-04-12 | 国网上海市电力公司 | Multi-band imaging and fusion method |
| CN116996675B (en) * | 2023-09-27 | 2023-12-19 | 河北天英软件科技有限公司 | Instant messaging system and information processing method |
Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN104376546A (en) * | 2014-10-27 | 2015-02-25 | 北京环境特性研究所 | Method for achieving three-path image pyramid fusion algorithm based on DM642 |
| CN107607202A (en) * | 2017-08-31 | 2018-01-19 | 江苏宇特光电科技股份有限公司 | Three light merge intelligent imager and its method |
Family Cites Families (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US8928964B1 (en) * | 2012-01-30 | 2015-01-06 | Softronics, Ltd. | Three-dimensional image display |
| CN104182955B (en) * | 2014-09-05 | 2016-09-14 | 西安电子科技大学 | Image interfusion method based on steerable pyramid conversion and device thereof |
| CN105654448B (en) * | 2016-03-29 | 2018-11-27 | 微梦创科网络科技（中国）有限公司 | A kind of image interfusion method and system based on bilateral filtering and weight reconstruction |
-
2018
- 2018-02-06 CN CN201810118085.1A patent/CN108399612B/en active Active
- 2018-07-17 WO PCT/CN2018/096023 patent/WO2019153651A1/en active Application Filing
Patent Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN104376546A (en) * | 2014-10-27 | 2015-02-25 | 北京环境特性研究所 | Method for achieving three-path image pyramid fusion algorithm based on DM642 |
| CN107607202A (en) * | 2017-08-31 | 2018-01-19 | 江苏宇特光电科技股份有限公司 | Three light merge intelligent imager and its method |
Non-Patent Citations (3)
| Title |
|---|
| 佚名: "图像拉普拉斯金字塔融合（Laplacian Pyramid Blending）", 《HTTP://WWW.360DOC.COM/CONTENT/12/0911/09/10724725_235476158.SHTML》 * |
| 刘志强: "基于双边与高斯滤波混合分解的图像融合方法", 《系统工程与电子技术》 * |
| 刘贵喜: "多传感器图像融合方法研究", 《中国优秀博硕士论文全文数据库（博士） 信息科技辑》 * |
Cited By (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110400262A (en) * | 2019-04-10 | 2019-11-01 | 泰州市康平医疗科技有限公司 | Identification device based on customization data processing |
| CN111814511A (en) * | 2019-04-10 | 2020-10-23 | 泰州市康平医疗科技有限公司 | Identification method based on customized data processing |
| CN111814511B (en) * | 2019-04-10 | 2021-02-23 | 青岛大学附属医院 | Identification method based on customized data processing |
| CN110956592A (en) * | 2019-11-14 | 2020-04-03 | 北京达佳互联信息技术有限公司 | Image processing method, image processing device, electronic equipment and storage medium |
| CN111462032A (en) * | 2020-03-31 | 2020-07-28 | 北方夜视技术股份有限公司 | Method for fusing uncooled infrared image and solar blind ultraviolet image and application |
| CN111462032B (en) * | 2020-03-31 | 2023-03-31 | 北方夜视技术股份有限公司 | Method for fusing uncooled infrared image and solar blind ultraviolet image and application |
| CN113643219A (en) * | 2021-08-03 | 2021-11-12 | 武汉三江中电科技有限责任公司 | Image imaging method and device based on three-light fusion |
| CN113643219B (en) * | 2021-08-03 | 2023-11-24 | 武汉三江中电科技有限责任公司 | Image imaging method and device based on three-light fusion |
Also Published As
| Publication number | Publication date |
|---|---|
| CN108399612B (en) | 2022-04-05 |
| WO2019153651A1 (en) | 2019-08-15 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN108399612A (en) | Based on the pyramidal three light images intelligent method for fusing of bilateral filtering | |
| US11546576B2 (en) | Systems and methods for dynamic calibration of array cameras | |
| DE112020004810B4 (en) | Systems and methods for recording surface normals with polarization | |
| CN106548467B (en) | The method and device of infrared image and visual image fusion | |
| CN105118055B (en) | Camera position amendment scaling method and system | |
| CN106657910B (en) | A kind of panoramic video monitoring method of electricity substation | |
| CN109064404A (en) | It is a kind of based on polyphaser calibration panorama mosaic method, panoramic mosaic system | |
| Bochanski et al. | MASE: a new data-reduction pipeline for the Magellan Echellette spectrograph | |
| CN109029731A (en) | A kind of power equipment exception monitoring system and method based on multi-vision visual | |
| Bhattacharjee et al. | What you can't see can help you-extended-range imaging for 3d-mask presentation attack detection | |
| CN109447909A (en) | The infrared and visible light image fusion method and system of view-based access control model conspicuousness | |
| CN108682026A (en) | A kind of binocular vision solid matching method based on the fusion of more Matching units | |
| US8849059B2 (en) | Method and device for generating images having a reduced error rate, high resolution and improved contrast | |
| CN108074218A (en) | Image super-resolution method and device based on optical field acquisition device | |
| WO2001082593A1 (en) | Apparatus and method for color image fusion | |
| CN107976257B (en) | A kind of image display method of infrared thermal imager, device and infrared thermal imager | |
| EP3047424A2 (en) | Device, system and method for identifying a person | |
| CN107154014A (en) | A kind of real-time color and depth Panorama Mosaic method | |
| WO2012044619A1 (en) | Image capture using three-dimensional reconstruction | |
| JP2007330558A (en) | Spectral fundus measuring apparatus and its measuring method | |
| CN110132226A (en) | The distance and azimuth angle measurement system and method for a kind of unmanned plane line walking | |
| CN107578450A (en) | A kind of method and system for the demarcation of panorama camera rigging error | |
| AU2019221088A1 (en) | Method and system for calibrating a plenoptic camera system | |
| CN109087245A (en) | Unmanned aerial vehicle remote sensing image mosaic system based on neighbouring relations model | |
| CN109544498A (en) | A kind of image adaptive fusion method |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |