WO2019153651A1 - 基于双边滤波金字塔的三光图像智能融合方法 - Google Patents

基于双边滤波金字塔的三光图像智能融合方法 Download PDF

Info

Publication number
WO2019153651A1
WO2019153651A1 PCT/CN2018/096023 CN2018096023W WO2019153651A1 WO 2019153651 A1 WO2019153651 A1 WO 2019153651A1 CN 2018096023 W CN2018096023 W CN 2018096023W WO 2019153651 A1 WO2019153651 A1 WO 2019153651A1
Authority
WO
WIPO (PCT)
Prior art keywords
image
pyramid
layer
fusion
bilateral
Prior art date
Application number
PCT/CN2018/096023
Other languages
English (en)
French (fr)
Inventor
赵毅
张登平
钱晨
刘宁
杨超
马新华
谢小波
宋莽
Original Assignee
江苏宇特光电科技股份有限公司
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 江苏宇特光电科技股份有限公司 filed Critical 江苏宇特光电科技股份有限公司
Publication of WO2019153651A1 publication Critical patent/WO2019153651A1/zh

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T3/00—Geometric image transformations in the plane of the image
    • G06T3/14—Transformations for image registration, e.g. adjusting or mapping for alignment of images
    • G06T3/147—Transformations for image registration, e.g. adjusting or mapping for alignment of images using affine transformations
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T3/00—Geometric image transformations in the plane of the image
    • G06T3/60—Rotation of whole images or parts thereof
    • G06T3/608—Rotation of whole images or parts thereof by skew deformation, e.g. two-pass or three-pass rotation
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T5/00—Image enhancement or restoration
    • G06T5/50—Image enhancement or restoration using two or more images, e.g. averaging or subtraction
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T5/00—Image enhancement or restoration
    • G06T5/70—Denoising; Smoothing
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/30—Determination of transform parameters for the alignment of images, i.e. image registration
    • G06T7/33—Determination of transform parameters for the alignment of images, i.e. image registration using feature-based methods
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/20—Special algorithmic details
    • G06T2207/20016—Hierarchical, coarse-to-fine, multiscale or multiresolution image processing; Pyramid transform
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/20—Special algorithmic details
    • G06T2207/20024—Filtering details
    • G06T2207/20028—Bilateral filtering
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/20—Special algorithmic details
    • G06T2207/20212—Image combination
    • G06T2207/20221—Image fusion; Image merging

Definitions

  • the invention relates to the field of intelligent detection imaging and image processing technology, in particular to a three-light image intelligent fusion method based on bilateral filtering pyramid.
  • the existing solution is to use the cameras of different bands to observe the scene at the same time, for example, using a visible light camera to locate the target in the scene, and then using the infrared camera to perform heat map analysis on the target;
  • the infrared thermal imager is used to detect the heat distribution at the electrical connection joints such as the cabinet and the knife box, to determine whether there is abnormal high temperature, and then use the ultraviolet camera to detect whether there is high voltage arc discharge at abnormally high temperature. Or leakage or other phenomena.
  • multi-camera coordinated combat methods also have broad application prospects.
  • infrared cameras are used to detect missiles during long-distance flight, and then ultraviolet light cameras are used to detect tail flames during missile flight.
  • the analysis is carried out to effectively avoid the defects of the infrared thermal imaging target recognition caused by the missile's interference bombs, and can also effectively distinguish the types of flying targets according to the thermal image and the tail flame characteristics. It can be seen that in the field of industrial machine vision, the design concept of multi-camera collaborative work has already had a wide range of application scenarios and has been put into use, which is an important direction for the development of visual imaging in the future.
  • the main product solutions in the industry are to set up multiple cameras for discrete monitoring of the same scene.
  • a visible light camera and an infrared camera are installed at the same time, and the video signal is output to a central control host in the supporting software of the host. Both videos are displayed at the same time and are observed by the user at the same time.
  • the biggest defects of such a system are as follows: 1. The whole system structure is dispersed, the camera and the central control host are separate devices, and the volume is too large and the cost is too high when the field is set up; 2. Two video images are simultaneously displayed on the display.
  • the object of the present invention is to provide a three-light image intelligent fusion method based on a bilateral filtering pyramid, which can integrate images collected by visible light, infrared light and ultraviolet light, and the user only needs to observe this image to complete the scene.
  • the acquisition of all target features greatly improves the observation efficiency and the user's use of cheapness.
  • the present invention provides a three-light image intelligent fusion method based on a bilateral filtering pyramid, comprising the following steps:
  • Step S1 firstly designing a three-light camera by using a parallel optical axis, so that the three-light camera is performing video signal acquisition on the same scene;
  • Step S2 performing the displacement deviation and the rotation distortion correction on the collected three video signals
  • the correction method of displacement deviation and rotation distortion adopts the image registration correction algorithm in Cartesian coordinate system and polar coordinate system to calculate the target feature points in the image, and simultaneously calculate the pixel positions of the same feature points in the three spectral images. And using the translational scaling calculation of the Cartesian coordinate system and the rotation factor calculation in the polar coordinate system, the angular positions and sizes of the three spectral images are all adjusted to the same dimension for subsequent fusion calculation;
  • step S3 the bilateral filter is subjected to pyramidal tomography processing, that is, the image results obtained by the bilateral filter are respectively subjected to pyramid down sampling and layered, and the result obtained by the layering is subjected to bilateral filtering operation, and subjected to multiple tomographic joint operations.
  • pyramidal tomography processing that is, the image results obtained by the bilateral filter are respectively subjected to pyramid down sampling and layered, and the result obtained by the layering is subjected to bilateral filtering operation, and subjected to multiple tomographic joint operations.
  • Step S4 after the pyramid chromatography process is finished, the three pyramid information obtained by the three spectral images need to be fused; the fusion method is to weight-blend the pyramid layer base corresponding to the three spectral images and the base, and the features and features are weighted. Fusion; starting from the top of the pyramid, the pyramid inverse operation is performed once per weighted fusion, and the high-level pyramid dimension is expanded to the next secondary high-level and re-weighted fusion calculation with the secondary high-level until all the pyramid layers are calculated.
  • the original three-spectrum image is merged into a fused image containing all the features of the three spectra and output to the outside, so that the user only needs to observe this image to complete the acquisition of all target features in the scene.
  • step S2 when the correction of the displacement deviation and the rotational distortion is performed, the three spectral images need to be scaled to the same size before being corrected.
  • step S3 the main steps of the pyramidal chromatographic processing performed by the bilateral filter are:
  • each spectral image is processed as follows: the first base image is subjected to the first bilateral filtering process to obtain two images of the fundamental frequency layer and the detail layer, and then the fundamental frequency obtained at this time is obtained.
  • the layer image is subjected to a pyramid down sampling process to obtain a second layer pyramid image, and then the base pyramid image of the second layer is separately subjected to bilateral filtering processing, and then the second layer of the baseband layer and the detail layer are obtained, and then the image is The baseband image of the second layer continues to be bilaterally filtered, and so on, until all layers of the pyramid are bilaterally filtered.
  • step S4 when the pyramid information of the three spectral images is fused, the base image is selected as a visible light image, and the infrared light image and the ultraviolet light image feature are required to be superimposed.
  • the bilateral filter is an effective nonlinear filter that distinguishes features and noise in an image.
  • the notation (i',j') ⁇ S i,j represents (i',j') and (i,j) are adjacent elements in the image; in general g s is selected as a normalized Gaussian kernel
  • the function that is, the sum of all the coefficients in the bilateral filter is 1; in addition, a Gaussian kernel function is used in the intensity domain; the overall weight k(i,j) of this template is through two Gaussian templates of the spatial domain and the intensity domain. The results are multiplied; the range should be between 0-1.
  • N is the layer number of the pyramid
  • * is the convolution
  • i/j is the coordinate of the image
  • w(2i, 2j, ⁇ ) is the bilateral filtering kernel function with the variance ⁇ ;
  • G 0 , G 1 , ..., G N form the pyramid of the fundamental frequency layer, G 0 is the same as the original image; the size of the current layer image is 1/4 of the size of the image of the previous layer, and the total number of layers of the pyramid is N+1.
  • the image G l is interpolated to obtain an image that is magnified four times.
  • the image size is the same as the size of G l-1 ; it is obtained by formula (3):
  • the pyramid composed of LP 0 , LP 1 , ..., LP N is a bilateral filtering pyramid; each layer of the image is the difference between the image of the fundamental layer pyramid and the image of the higher layer of the image after interpolation and interpolation.
  • the process is equivalent to band-pass filtering, so the bilateral filtering pyramid is also called band-pass pyramid decomposition; finally, the fusion is obtained by the pyramidal reconstruction of all the fundamental frequency layers and the obtained detail components obtained by the pyramid reconstruction method. image.
  • the invention adopts a parallel optical axis design when installing a three-light camera to ensure that there is no geometric distortion in the scene acquired by the camera.
  • the present invention first performs displacement deviation and rotation distortion correction before the fusion process, eliminating displacement deviation and rotation. Distortion ensures high precision in the subsequent fusion process.
  • the bilateral filter of the present invention is an effective nonlinear filter for distinguishing features and noises in an image. Because of its nonlinear filtering characteristics, it can cope with the selected filtering window through two indexes of the spatial domain and the intensity domain. The pixel values of the image are arbitrated to distinguish the features and the noise information, so that in the fusion process, more noise is avoided, and the final fusion effect is greatly improved.
  • the present invention adopts a bilateral filtering pyramid tomography algorithm as a fusion strategy of three-light images, that is, the bilateral filter and the pyramid layering algorithm are effectively merged into a new and effective overall algorithm, and the original three-spectral image is merged. It becomes a fused image containing all the features of the three spectra and is output to the outside, so that the user only needs to observe this image to complete the acquisition of all target features in the scene, which greatly improves the observation efficiency and user use. Cheap.
  • the invention can well perform images including image correction and image fusion processes from different spectral segments, and simultaneously eliminate image parallax and image size mismatch phenomenon from images and videos of different cameras, and design based on the invention.
  • the bilateral filtering pyramid algorithm is used to fuse any two or all of the three spectra in real time.
  • the fusion strategy of this technology is extremely accurate and the fusion result is excellent.
  • the bilateral filter can well separate the fundamental frequency information (energy information) and the feature information (detail information) in the image, high-precision image superposition can be realized at the time of fusion;
  • the layering technique is combined, with good filtering characteristics and multiple filtering operations; while the traditional bilateral filter performs filtering operations only for ordinary single images.
  • Figure 1 is a flow chart of the main body of the present invention
  • FIG. 2 is a schematic diagram of performing displacement deviation and rotation distortion correction according to the present invention
  • FIG. 3 is a schematic diagram of a calculation process of a bilateral filter of the present invention.
  • Figure 4 is a schematic view of the complete pyramidal chromatography process of the present invention.
  • FIG. 5 is a schematic diagram of detail extraction pyramid components in a pyramidal tomography process of the present invention.
  • FIG. 6 is a schematic diagram of an energy extraction pyramid component in a pyramidal chromatography process of the present invention.
  • FIG. 7 is a schematic diagram of a pyramid reconstruction algorithm of the present invention.
  • the invention provides a three-light image intelligent fusion method based on a bilateral filtering pyramid. Referring to FIG. 1-7, the following steps are included:
  • step S1 the three-light camera is first set up by using the parallel optical axis to enable the three-light camera to perform video signal acquisition on the same scene.
  • the present invention adopts a parallel optical axis design when installing the three-light camera to ensure that the scene acquired by the camera is not There is a situation in which geometric distortion occurs.
  • the installation position of the industrial structure of the camera there is a physical displacement in the installation of the three-light camera in the space, so that the displacement and rotation deviation of the scene appear when the unified scene is observed, and the scene information obtained by the three-light camera There is also a certain positional deviation between the two.
  • Step S2 performing the displacement deviation and the rotation distortion correction on the collected three video signals
  • the size of the corresponding target is different when the same target is focused on the focal plane by the lens, and the displacement and rotation deviation are required.
  • the images obtained by the corrected three spectra are scaled to the same size during calibration.
  • the correction method of displacement deviation and rotation distortion adopts the image registration correction algorithm in Cartesian coordinate system and polar coordinate system to calculate the target feature points in the image, and simultaneously calculate the pixel positions of the same feature points in the three spectral images. And using the translational scaling calculation of the Cartesian coordinate system and the rotation factor calculation in the polar coordinate system, the angular positions and sizes of the three spectral images are all adjusted to the same dimension for subsequent fusion calculation.
  • step S3 the bilateral filter is subjected to pyramidal tomography processing, that is, the image results obtained by the bilateral filter are separately subjected to pyramid down sampling layering, and the layered result is subjected to bilateral filtering operation to ensure that the obtained three-light image is in the
  • the features and noises can be separated on each pyramid layer. Through multiple tomosynthesis operations, all the feature information in the image can be completely extracted for fusion calculation. As shown in Figure 3-6.
  • the invention is an image fusion technology based on a bilateral filtering pyramid algorithm, wherein the bilateral filter and the pyramid layering algorithm are effectively combined into a new and effective overall algorithm.
  • the bilateral filter is an effective nonlinear filter that distinguishes features and noise in an image.
  • the notation (i',j') ⁇ S i,j represents (i',j') and (i,j) are adjacent elements in the image; in general g s is selected as a normalized Gaussian kernel
  • the function that is, the sum of all the coefficients in the bilateral filter is 1; in addition, a Gaussian kernel function is used in the intensity domain; the overall weight k(i,j) of this template is through two Gaussian templates of the spatial domain and the intensity domain. The results are multiplied; the range should be between 0-1.
  • ⁇ s and ⁇ r represent the standard deviation parameters of the two Gaussian kernel functions, which control the expansion range of the two Gaussian kernel functions.
  • ⁇ s determines the scale of the adjacent area, so it must be proportional to the size of the image.
  • the present invention selects 2.5% of the diagonal size of the image.
  • the choice of ⁇ r is more critical because it represents the magnitude of the so-called detail. If the range of signal fluctuations is less than ⁇ r , then this signal fluctuation is considered to be the detail, which is smoothed by the bilateral filter and separated into the detail layer. Conversely, if the range of this fluctuation is greater than ⁇ r , this detail will be well preserved to the fundamental layer due to the nonlinear nature of the bilateral filter.
  • the present invention selects the human eye to distinguish 20% of the gray level, that is, 25 as the value of ⁇ r . This value takes into account the ability of the human eye to resolve grayscale and has good adaptability to different scenes.
  • the bilateral filter can well separate the fundamental frequency information (energy information) and the feature information (detail information) in the image, high-precision image superposition can be realized at the time of fusion.
  • the traditional bilateral filter performs only one filtering operation on a normal single image. Based on its good screening characteristics, the present invention combines it with pyramid layering techniques.
  • FIG. 4 is a schematic diagram of a complete pyramidal tomography process for parsing a picture according to the present invention.
  • the bottom of the pyramid is the original picture, and each upward layer is the content of the current original picture for detail extraction and energy extraction.
  • the image dimension is reduced to a quarter of the original image.
  • Figures 5 and 6 respectively show the pyramid component of the detail extraction and energy extraction in the pyramidal tomography process.
  • the main steps of the pyramidal chromatographic processing of the bilateral filter are:
  • each spectral image is processed as follows: the first base image is subjected to the first bilateral filtering process to obtain two images of the fundamental frequency layer and the detail layer, and then the fundamental frequency obtained at this time is obtained.
  • the layer image is subjected to a pyramid down sampling process to obtain a second layer pyramid image, and then the base pyramid image of the second layer is separately subjected to bilateral filtering processing, and then the second layer of the baseband layer and the detail layer are obtained, and then the image is The baseband image of the second layer continues to be bilaterally filtered, and so on, until all layers of the pyramid are bilaterally filtered.
  • the pyramid downsampling process is a dimensionality reduction operation that is reduced by four times each time, the number of layers of the pyramid depends on the size of the initial image.
  • the initial image dimension is 640 ⁇ 480, and each time it is reduced by 4 times.
  • the total number of pixels in the image at the 7th layer is 75, and it is no longer possible to be divided by 4.
  • the total number of pixels to the fifth layer is 4800, the image content can be almost invisible to the naked eye, so the total number of pyramid layers can be 5 layers, you can also choose other layers according to the situation.
  • the calculation process is as follows:
  • the original image be G 0
  • G 0 as the 0th layer of the pyramid, also known as the base layer
  • a layer of image is low-pass filtered and down-sampled to obtain a second layer of the baseband pyramid
  • the above process is repeated to form a baseband pyramid.
  • the construction process of the bilateral filtering pyramid is:
  • G l (i,j) w(2i,2j, ⁇ )*G l-1 (2i,2j), (1 ⁇ l ⁇ N), (3);
  • N is the layer number of the pyramid
  • * is the convolution
  • i/j is the coordinate of the image
  • w(2i, 2j, ⁇ ) is the bilateral filtering kernel function with the variance ⁇ ;
  • G 0 , G 1 , ..., G N form the pyramid of the fundamental frequency layer
  • G 0 is the same as the original image
  • the size of the current layer image is 1/4 of the size of the image of the previous layer
  • the total number of layers of the pyramid is N+1. It can be seen that the baseband pyramid decomposition of the image is realized by low-pass filtering the underlying image in turn, and then filtering the result as a down-sample of the interlaced interlace.
  • the image G l is interpolated to obtain an image that is magnified four times.
  • the image size is the same as the size of G l-1 ; it is obtained by formula (3):
  • the pyramid composed of LP 0 , LP 1 , ..., LP N is a bilateral filtering pyramid; each layer of the image is the difference between the image of the fundamental layer pyramid and the image of the higher layer of the image after interpolation and interpolation.
  • the process is equivalent to band-pass filtering, so the bilateral filtering pyramid is also called band-pass pyramid decomposition; finally, the fusion is obtained by the pyramidal reconstruction of all the fundamental frequency layers and the obtained detail components obtained by the pyramid reconstruction method.
  • the image is shown in Figure 7.
  • step S4 after the pyramidal chromatography processing is finished, the three pyramid information obtained by the three spectral images needs to be fused. As shown in Figure 7.
  • the fusion method is to weight-blend the pyramid layer base corresponding to the three spectral images and the base, and feature and feature are weighted and merged; from the top of the pyramid, the pyramid inverse operation is performed once per weighted fusion, and the high-level pyramid dimension is expanded to the next one.
  • the three-light camera can adopt a visible light camera, an infrared light camera, an ultraviolet light camera, or a camera of other light.
  • the pyramid information of the three spectral images is fused, the base image is selected as a visible light image, and the infrared light image and the ultraviolet light image feature need to be superimposed.
  • the three-light image intelligent fusion method based on the bilateral filtering pyramid of the invention can perform real-time high-precision fusion of images and videos obtained under three imaging spectra of visible light, infrared light and ultraviolet light.
  • This technology can well perform images including image correction and image fusion processes from different spectral segments. Image and video from different cameras are simultaneously eliminated to eliminate scene parallax and image size mismatch, and real-time fusion of any two or all of the three spectra is performed by a bilateral filtering pyramid algorithm designed according to the present invention.
  • the fusion strategy is extremely accurate and the fusion results are excellent.

Landscapes

  • Engineering & Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Theoretical Computer Science (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Image Processing (AREA)
  • Image Analysis (AREA)

Abstract

一种基于双边滤波金字塔的三光图像智能融合方法，包括以下步骤：首先利用平行光轴设计架设三光摄像头，让三光摄像头正对同一场景进行视频信号采集（S1）；将采集到的三路视频信号进行位移偏差和旋转畸变校正（S2）；将双边滤波器进行了金字塔层析处理（S3），当金字塔层析处理结束后，需要将三个光谱图像获得的三种金字塔信息进行融合（S4）。该方法将双边滤波器及金字塔分层算法有效的融合为一个新的行之有效的整体算法，将原来的三光谱图像融合变为一幅包含了三种光谱所有特征的融合图像，并向外输出，这样，用户只需要观察这一幅图像便可以完成对场景中所有目标特征的获取，大大提高了观察效率和用户使用便宜性。

Description

基于双边滤波金字塔的三光图像智能融合方法 技术领域
本发明涉及智能探测成像及图像处理技术领域，特别涉及一种基于双边滤波金字塔的三光图像智能融合方法。
背景技术
目前国际及国内在社区安防、工业生产、消防安全、森林防火、安检防爆等与工业机器视觉相关的应用领域中广泛使用了不同波段的摄像头来对场景及其中的目标进行监测。由于自然界中的目标特征能够表现出不同的光谱特征，因此，利用单一摄像头完成对场景及目标的监测过程已经无法满足现代军民用的技术要求。目前已有的解决方案是利用多个不同波段的摄像头对场景进行同时观测，比如利用可见光摄像头对场景中的目标进行定位，再利用红外热像仪对该目标进行热图分析；又比如在电力电网运维巡检过程中首先用红外热像仪检测机柜、闸刀盒等电气连接接头处的热量分布，判断是否有异常高温现象出现，再利用紫外光摄像头对异常高温处是否存在高压电弧放电或漏电等现象。在军事上多摄像头协同作战方式也具有广泛的应用前景，例如在追踪导弹目标时，利用红外热像仪检测远距离飞行过程中的导弹，继而利用紫外光摄像头对导弹飞行过程中的尾焰特征进行分析，从而有效的避免导弹发出的干扰弹影响红外热成像目标识别的缺陷，同时也能够根据热图像和尾焰特征有效的区分飞行目标的种类。由此可见，在工业机器视觉领域中，多摄像头协同工作的设计理念已 经具备了相当广泛的应用场景并已经初步投入使用，是未来视觉成像发展的重要方向。
目前业内主要的产品解决方案是针对同一场景架设多台摄像头分立监测，例如同时架设一台可见光摄像头及一台红外热像仪，将视频信号输出到一台中控主机上，在主机的配套软件中同时显示这两路视频，由用户同时观测。这样的系统最大的缺陷在于以下几点：1.整套系统结构分散，摄像头与中控主机均为分立设备，现场架设时体积过大且成本过高；2.两路视频图像同时显示在显示器上时由于不同波段的成像器件焦平面象元尺寸不同，导致即使是对相同场景进行观测也会存在较大的视差，即两路图像中的场景只有局部相同，这就给目标发现过程带来了难度；3.因为用户需要同时观测两种视频，因此人眼需要不停的在两路视频间来回切换观察，给用户造成了视觉不便利；4.这样的系统存在携带不易性，只能够安装在固定位置进行观察，无法随身携带，因此很大程度上限制了其应用场景的拓展。综上所述，急需一款体积小，重量轻，可携带及固定安装的多光谱融合产品来填补市场空白。
要解决对多种波段摄像头采集的同一场景的视频被用户观察时不产生视差，又要能够非常方便的让用户进行观察，最佳的方案是进行图像融合过程，目前业内已经提出了不通类型的图像融合技术。其中，在军民用视频解决方案中最广泛的是红外与可见光双光谱融合技术。随着科技的进步及用户需求的越来越高，传统的双光谱融合技术已经逐渐不能够完全应 对场景中出现的复杂目标特征信息，因此，一种能够覆盖可见光、红外光及紫外光三个成像光谱的高精度智能融合技术的出现显得尤为重要。
发明内容
本发明的目的旨在至少解决所述技术缺陷之一。
为此，本发明的目的在于提出一种基于双边滤波金字塔的三光图像智能融合方法,可以对可见光、红外光及紫外光采集的图像进行融合，用户只需要观察这一幅图像便可以完成对场景中所有目标特征的获取，大大提高了观察效率和用户使用便宜性。
为了实现上述目的，本发明提供一种基于双边滤波金字塔的三光图像智能融合方法，包括以下步骤：
步骤S1，首先利用平行光轴设计架设三光摄像头，让三光摄像头正对同一场景进行视频信号采集；
步骤S2，将采集到的三路视频信号进行位移偏差和旋转畸变校正；
位移偏差和旋转畸变的校正方法采用笛卡尔坐标系和极坐标系下的图像配准校正算法，计算图像中目标特征点，并在三幅光谱图像中同时计算相同的特征点出现的像素位置，并利用笛卡尔坐标系的平移缩放计算和极坐标系下的旋转因子计算，将三个光谱图像的角度位置、大小全部调整到相同的维度，以便进行后续的融合计算；
步骤S3，将双边滤波器进行了金字塔层析处理，即将双边滤波器得到的图像结果分别再进行金字塔下采样分层，将分层得到的结果再进行双边滤波操作，通过多次层析联合运算的方式，确保图像中所有的特征信息都 能够被完全提取出来进行融合计算；
步骤S4，当金字塔层析处理结束后，需要将三个光谱图像获得的三种金字塔信息进行融合；融合方法是将三个光谱图像对应的金字塔层基底与基底进行加权融合，特征与特征进行加权融合；从金字塔最顶层开始，每加权融合一次就进行金字塔逆运算，将高层金字塔维度扩大进入下一个次高层并与次高层进行再加权融合计算，直到所有的金字塔分层全部计算完毕，此时原来的三光谱图像就融合变为一幅包含了三种光谱所有特征的融合图像，并向外输出，这样，用户只需要观察这一幅图像便能够完成对场景中所有目标特征的获取。
进一步的，在步骤S2中，在进行位移偏差和旋转畸变的校正时需将三个光谱图像缩放到相同的大小之后再进行校正。
进一步的，在步骤S3中，双边滤波器进行了金字塔层析处理的主要步骤为：
从原始三个光谱图像开始，对每个光谱图像都进行下面处理：最基底的图像经过第一次双边滤波处理后得到了基频层和细节层两幅图像，继而将此时得到的基频层图像进行金字塔下采样过程，得到第二层金字塔图像，然后分别对第二层的基底金字塔图像进行双边滤波处理，再得到该第二层的基频层和细节层两幅图像，然后将该第二层的基频图像继续进行双边滤波处理，以此类推，直到对金字塔所有层都进行了双边滤波处理。
进一步的，在步骤S4中，三个光谱图像的金字塔信息进行融合时，基 底图像选取为可见光图像，需要叠加的为红外光图像及紫外光图像特征。
进一步的，双边滤波器是一种有效的区分图像中特征与噪声的非线性滤波器。
进一步的，双边滤波器的计算公式为：
Figure PCTCN2018096023-appb-000001
其中，k(i,j)代表归一化系数：
Figure PCTCN2018096023-appb-000002
这里的记号(i′,j′)∈S i,j代表(i′,j′)与(i,j)是图像中的相邻元素；一般情况下g s被选择是一个标准化的高斯核函数，即双边滤波器中的所有系数之和为1；此外在强度域亦采用一个高斯核函数；这个模板的总体权重k(i,j)是通过将空间域与强度域的两个高斯模板的结果相乘得到；其范围应该在0-1之间。
进一步的，在步骤S3-S4中，假设金字塔的第l层图像为G 1，双边滤波金字塔的构建公式为：G l(i,j)＝w(2i,2j,σ)*G l-1(2i,2j)，(1≤l≤N)，(3)；
其中，N表示金字塔的层号，*表示卷积，i/j都表示图像的坐标，w(2i,2j,σ)表示方差为σ的双边滤波核函数；
因此，G 0、G 1、…、G N就构成了基频层金字塔,G 0与原始图像相同；当前层图像的大小依次为上一层图像大小的1/4，金字塔的总层数为N+1。
进一步的，将图像G l经过内插得到放大四倍的图像
Figure PCTCN2018096023-appb-000003
其图像尺寸与G l-1的尺寸相同；则由公式(3)得到：
Figure PCTCN2018096023-appb-000004
公式(2)中：
Figure PCTCN2018096023-appb-000005
为整数，(5)；
令
Figure PCTCN2018096023-appb-000006
由LP 0、LP 1、…、LP N构成的金字塔即为双边滤波金字塔；它的每一层图像是基频层金字塔本层图像与其高一层的图像经过内插放大后图像的差，此过程相当于带通滤波，因此双边滤波金字塔又称为带通金字塔分解；最终，通过将得到的所有每层的基频层金字塔和得到的细节分量通过金字塔重建的方法复原，即得到融合后的图像。
进一步的，金字塔重建的公式为：
Figure PCTCN2018096023-appb-000007
本发明的基于双边滤波金字塔的三光图像智能融合方法具有以下有益效果：
1、本发明在安装三光摄像头时采用了平行光轴设计，以保证摄像头获取到的场景不存在出现几何畸变的情况。
2、由于三光摄像头在结构安装上的机械误差，不可能做到三个摄像头所观察到的场景完全相同，因此本发明在进行融合过程之前首先进行位移偏差和旋转畸变校正，消除位移偏差和旋转畸变，保证了后续融合过程的高精度。
3、本发明的双边滤波器是一种有效的区分图像中特征与噪声的非线性滤波器，因其非线性滤波特性，能够通过空间域和强度域两个指标共同对选定的滤波窗口内的图像像素值进行仲裁，区分特征及噪声信息，进而在 融合过程中，避免带入了较多的噪声，使得最终的融合效果大大提高。
4、本发明采用基于双边滤波金字塔层析算法来作为三光图像的融合策略，即将双边滤波器及金字塔分层算法有效的融合为一个新的行之有效的整体算法，将原来的三光谱图像融合变为一幅包含了三种光谱所有特征的融合图像，并向外输出，这样，用户只需要观察这一幅图像便可以完成对场景中所有目标特征的获取，大大提高了观察效率和用户使用便宜性。
5、本发明能够很好的将来自不同光谱段的图像进行包括图像校正、图像融合过程，将同时来自不同摄像头的图像及视频消除场景视差及图像大小失配现象，并通过基于为本发明设计的双边滤波金字塔算法来对三种光谱中的任意两种或全部信息进行实时融合，本技术融合策略精度极高，融合结果优秀。
6、本发明由于双边滤波器能够很好的将图像中的基频信息(能量信息)和特征信息(细节信息)进行分离，因此在融合时就可以实现高精度的图像叠加；将其与金字塔分层技术进行了结合，良好的筛选特性和多次滤波运算特性；而传统的双边滤波器只针对普通单幅图像进行一次滤波运算。
本发明附加的方面和优点将在下面的描述中部分给出，部分将从下面的描述中变得明显，或通过本发明的实践了解到。
附图说明
本发明的上述和/或附加的方面和优点从结合下面附图对实施例的描述中将变得明显和容易理解，其中：
图1为本发明的主体流程图；
图2为本发明的进行位移偏差和旋转畸变校正示意图；
图3为本发明的双边滤波器的计算过程示意图；
图4为本发明的完整金字塔层析过程示意图；
图5为本发明的金字塔层析过程中细节提取金字塔分量的示意图；
图6为本发明的金字塔层析过程中能量提取金字塔分量的示意图；
图7为本发明的金字塔重建算法示意图。
具体实施方式
下面详细描述本发明的实施例，所述实施例的示例在附图中示出，其中自始至终相同或类似的标号表示相同或类似的元件或具有相同或类似功能的元件。下面通过参考附图描述的实施例是示例性的，旨在用于解释本发明，而不能理解为对本发明的限制。
本发明提供一种基于双边滤波金字塔的三光图像智能融合方法，参考附图1-7所示，包括以下步骤：
步骤S1，首先利用平行光轴设计架设三光摄像头，让三光摄像头正对同一场景进行视频信号采集。
由于三光摄像头在结构安装上的机械误差，不可能做到三个摄像头所观察到的场景完全相同，因此，本发明在安装三光摄像头时采用了平行光轴设计，以保证摄像头获取到的场景不存在出现几何畸变的情况，此外由于摄像头工业结构安装位置，使得三光摄像头在空间中安装是存在物理位 移，从而对统一场景在观察时会出现场景的位移和旋转偏差，三光摄像头所得到的场景信息之间还存在一定的位置偏差，因此需要在进行融合过程之前首先获取三光摄像头所得到的场景信息中相同场景的部分进行后续的融合处理；由于在结构安装三光摄像头时无法做到完美的方正，三光摄像头之间必定存在一定程度的旋转畸变情况，因此，还需要将这种旋转畸变优先消除才能够保证后续融合过程的高精度。
步骤S2，将采集到的三路视频信号进行位移偏差和旋转畸变校正；
由于不同光谱的摄像头象元大小及象元间距不相同，导致对同一个目标经过镜头聚焦到焦平面上后对应的像素个数不相同而出现的大小不一样情况，需要在进行位移和旋转偏差校正时将校正的三个光谱获得的图像缩放到相同的大小。
因此在进行后续融合处理时，首先需要先将位移和旋转偏差进行校正。
位移偏差和旋转畸变的校正方法采用笛卡尔坐标系和极坐标系下的图像配准校正算法，计算图像中目标特征点，并在三幅光谱图像中同时计算相同的特征点出现的像素位置，并利用笛卡尔坐标系的平移缩放计算和极坐标系下的旋转因子计算，将三个光谱图像的角度位置、大小全部调整到相同的维度，以便进行后续的融合计算。
步骤S3，将双边滤波器进行了金字塔层析处理，即将双边滤波器得到的图像结果分别再进行金字塔下采样分层，将分层得到的结果再进行双边滤波操作，可以保证获得的三光图像在每一个金字塔层上都能够将特征和 噪声进行分离，通过多次层析联合运算的方式，确保图像中所有的特征信息都能够被完全提取出来进行融合计算。如图3-6所示。
本发明是基于双边滤波金字塔算法的图像融合技术，其中将双边滤波器及金字塔分层算法有效的融合为一个新的行之有效的整体算法。其中双边滤波器是一种有效的区分图像中特征与噪声的非线性滤波器。
如图3所示，双边滤波器的计算公式为：
Figure PCTCN2018096023-appb-000008
其中，k(i,j)代表归一化系数：
Figure PCTCN2018096023-appb-000009
这里的记号(i′,j′)∈S i,j代表(i′,j′)与(i,j)是图像中的相邻元素；一般情况下g s被选择是一个标准化的高斯核函数，即双边滤波器中的所有系数之和为1；此外在强度域亦采用一个高斯核函数；这个模板的总体权重k(i,j)是通过将空间域与强度域的两个高斯模板的结果相乘得到；其范围应该在0-1之间。
σ s与σ r表示两个高斯核函数的标准差参数，其控制两个高斯核函数的扩张范围。σ s决定了临近区域的尺度，所以必须与图像的大小成比例关系，这里本发明选取图像对角线尺寸的2.5％。σ r的选择更为关键，因为其代表了所谓细节的幅度。如果信号波动的范围小于σ r，那么这个信号波动就会被认为是细节，即会被双边滤波器平滑，且被分离到了细节层中。反之，如果这个波动的范围大于σ r，那么由于双边滤波器的非线性特性，这个细节将会被很好的保留到基频层。这里本发明选择人眼可以分辨灰度级 的20％，即25作为σ r的取值。这个取值考虑到人眼对灰度的分辨能力，且对于不同的场景都具有较好的适应性。
由于双边滤波器能够很好的将图像中的基频信息(能量信息)和特征信息(细节信息)进行分离，因此在融合时就可以实现高精度的图像叠加。传统的双边滤波器只针对普通单幅图像进行一次滤波运算。基于其良好的筛选特性，本发明将其与金字塔分层技术进行了结合。
如图4所示为本发明解析一幅图片的完整金字塔层析过程的示意图，金字塔最下方为原始图片，每向上一层，即为将当前原始图片进行了细节提取和能量提取两方面的内容，同时将图像维度缩减为原图的四分之一，图5和图6分别表示在金字塔层析过程中细节提取和能量提取的金字塔分量示意图。
双边滤波器进行了金字塔层析处理的主要步骤为：
从原始三个光谱图像开始，对每个光谱图像都进行下面处理：最基底的图像经过第一次双边滤波处理后得到了基频层和细节层两幅图像，继而将此时得到的基频层图像进行金字塔下采样过程，得到第二层金字塔图像，然后分别对第二层的基底金字塔图像进行双边滤波处理，再得到该第二层的基频层和细节层两幅图像，然后将该第二层的基频图像继续进行双边滤波处理，以此类推，直到对金字塔所有层都进行了双边滤波处理。
由于金字塔下采样的过程是一个每次缩小四倍的降维运算，因此金字塔的层数根据初始图像的大小而定，例如，初始图像维度为640×480，每 次按照缩小4倍的速度下采样时，到第7层时图像的像素总数为75，已经无法继续被4除尽，由于到第五层时像素总数为4800，已经几乎肉眼无法区分图像内容，因此总的金字塔层数可以为5层，也可以根据情况选择其他层数。其计算过程如下：
设原图像为G 0，以G 0作为金字塔的第0层，又称为基底层，对原图像进行双边滤波和隔行隔列的下采样，得到基频层金字塔的第一层；再对第一层图像进行低通滤波和下采样，得到基频层金字塔的第二层；重复以上过程，构成基频层金字塔。假设金字塔的第1层图像为G 1，双边滤波金字塔的构建过程为：
假设金字塔的第l层图像为G 1，双边滤波金字塔的构建公式为：G l(i,j)＝w(2i,2j,σ)*G l-1(2i,2j)，(1≤l≤N)，(3)；
其中，N表示金字塔的层号，*表示卷积，i/j都表示图像的坐标，w(2i,2j,σ)表示方差为σ的双边滤波核函数；
因此，G 0、G 1、…、G N就构成了基频层金字塔,G 0与原始图像相同；当前层图像的大小依次为上一层图像大小的1/4，金字塔的总层数为N+1。可见，图像的基频层金字塔分解是通过依次对底层图像进行低通滤波，再把滤波结果作隔行隔列的降2下采样来实现的。
其次，将图像G l经过内插得到放大四倍的图像
Figure PCTCN2018096023-appb-000010
其图像尺寸与G l-1的尺寸相同；则由公式(3)得到：
Figure PCTCN2018096023-appb-000011
公式(2)中：
Figure PCTCN2018096023-appb-000012
为整数，(5)；
令
Figure PCTCN2018096023-appb-000013
由LP 0、LP 1、…、LP N构成的金字塔即为双边滤波金字塔；它的每一层图像是基频层金字塔本层图像与其高一层的图像经过内插放大后图像的差，此过程相当于带通滤波，因此双边滤波金字塔又称为带通金字塔分解；最终，通过将得到的所有每层的基频层金字塔和得到的细节分量通过金字塔重建的方法复原，即得到融合后的图像，如图7所示。
金字塔重建的公式为：
Figure PCTCN2018096023-appb-000014
步骤S4，当金字塔层析处理结束后，需要将三个光谱图像获得的三种金字塔信息进行融合。如图7所示。
融合方法是将三个光谱图像对应的金字塔层基底与基底进行加权融合，特征与特征进行加权融合；从金字塔最顶层开始，每加权融合一次就进行金字塔逆运算，将高层金字塔维度扩大进入下一个次高层并与次高层进行再加权融合计算，直到所有的金字塔分层全部计算完毕，此时原来的三光谱图像就融合变为一幅包含了三种光谱所有特征的融合图像，并向外输出，这样，用户只需要观察这一幅图像便能够完成对场景中所有目标特征的获取，大大提高了观察效率和用户使用便宜性。
需要说明的是，三光摄像头可以采用可见光摄像头、红外光摄像头、紫外光摄像头，也可以采用其他光的摄像头。三个光谱图像的金字塔信息 进行融合时，基底图像选取为可见光图像，需要叠加的为红外光图像及紫外光图像特征。
本发明的基于双边滤波金字塔的三光图像智能融合方法能够将可见光、红外光及紫外光三个成像光谱下获得的图像及视频进行实时高精度融合的技术。该技术能够很好的将来自不同光谱段的图像进行包括图像校正、图像融合过程。将同时来自不同摄像头的图像及视频消除场景视差及图像大小失配现象，并通过基于为本发明设计的双边滤波金字塔算法来对三种光谱中的任意两种或全部信息进行实时融合，本技术融合策略精度极高，融合结果优秀。
尽管上面已经示出和描述了本发明的实施例，可以理解的是，上述实施例是示例性的，不能理解为对本发明的限制，本领域的普通技术人员在不脱离本发明的原理和宗旨的情况下在本发明的范围内可以对上述实施例进行变化、修改、替换和变型。本发明的范围由所附权利要求极其等同限定。

Claims (9)

  1. 一种基于双边滤波金字塔的三光图像智能融合方法，其特征在于，包括以下步骤：
    步骤S1，首先利用平行光轴设计架设三光摄像头，让三光摄像头正对同一场景进行视频信号采集；
    步骤S2，将采集到的三路视频信号进行位移偏差和旋转畸变校正；
    位移偏差和旋转畸变的校正方法采用笛卡尔坐标系和极坐标系下的图像配准校正算法，计算图像中目标特征点，并在三幅光谱图像中同时计算相同的特征点出现的像素位置，并利用笛卡尔坐标系的平移缩放计算和极坐标系下的旋转因子计算，将三个光谱图像的角度位置、大小全部调整到相同的维度，以便进行后续的融合计算；
    步骤S3，将双边滤波器进行了金字塔层析处理，即将双边滤波器得到的图像结果分别再进行金字塔下采样分层，将分层得到的结果再进行双边滤波操作，通过多次层析联合运算的方式，确保图像中所有的特征信息都能够被完全提取出来进行融合计算；
    步骤S4，当金字塔层析处理结束后，需要将三个光谱图像获得的三种金字塔信息进行融合；融合方法是将三个光谱图像对应的金字塔层基底与基底进行加权融合，特征与特征进行加权融合；从金字塔最顶层开始，每加权融合一次就进行金字塔逆运算，将高层金字塔维度扩大进入下一个次高层并与次高层进行再加权融合计算，直到所有的金字塔分层全部计算完 毕，此时原来的三光谱图像就融合变为一幅包含了三种光谱所有特征的融合图像，并向外输出，这样，用户只需要观察这一幅图像便能够完成对场景中所有目标特征的获取。
  2. 如权利要求1所述的基于双边滤波金字塔的三光图像智能融合方法，其特征在于：在步骤S2中，在进行位移偏差和旋转畸变的校正时需将三个光谱图像缩放到相同的大小之后再进行校正。
  3. 如权利要求1所述的基于双边滤波金字塔的三光图像智能融合方法，其特征在于：在步骤S3中，双边滤波器进行了金字塔层析处理的主要步骤为：
    从原始三个光谱图像开始，对每个光谱图像都进行下面处理：最基底的图像经过第一次双边滤波处理后得到了基频层和细节层两幅图像，继而将此时得到的基频层图像进行金字塔下采样过程，得到第二层金字塔图像，然后分别对第二层的基底金字塔图像进行双边滤波处理，再得到该第二层的基频层和细节层两幅图像，然后将该第二层的基频图像继续进行双边滤波处理，以此类推，直到对金字塔所有层都进行了双边滤波处理。
  4. 如权利要求1所述的基于双边滤波金字塔的三光图像智能融合方法，其特征在于：在步骤S4中，三个光谱图像的金字塔信息进行融合时，基底图像选取为可见光图像，需要叠加的为红外光图像及紫外光图像特征。
  5. 如权利要求1所述的基于双边滤波金字塔的三光图像智能融合方法，其特征在于：双边滤波器是一种有效的区分图像中特征与噪声的非线性滤 波器。
  6. 如权利要求1所述的基于双边滤波金字塔的三光图像智能融合方法，其特征在于：双边滤波器的计算公式为：
    Figure PCTCN2018096023-appb-100001
    其中，k(i,j)代表归一化系数：
    Figure PCTCN2018096023-appb-100002
    这里的记号(i′,j′)∈S i,j代表(i′,j′)与(i,j)是图像中的相邻元素；一般情况下g s被选择是一个标准化的高斯核函数，即双边滤波器中的所有系数之和为1；此外在强度域亦采用一个高斯核函数；这个模板的总体权重k(i,j)是通过将空间域与强度域的两个高斯模板的结果相乘得到；其范围应该在0-1之间。
  7. 如权利要求1所述的基于双边滤波金字塔的三光图像智能融合方法，其特征在于：在步骤S3-S4中，假设金字塔的第l层图像为G 1，双边滤波金字塔的构建公式为：G l(i,j)＝w(2i,2j,σ)*G l-1(2i,2j)，(1≤l≤N)，(3)；
    其中，N表示金字塔的层号，*表示卷积，i/j都表示图像的坐标，w(2i,2j,σ)表示方差为σ的双边滤波核函数；
    因此，G 0、G 1、…、G N就构成了基频层金字塔,G 0与原始图像相同；当前层图像的大小依次为上一层图像大小的1/4，金字塔的总层数为N+1。
  8. 如权利要求7所述的基于双边滤波金字塔的三光图像智能融合方法，其特征在于：将图像G l经过内插得到放大四倍的图像
    Figure PCTCN2018096023-appb-100003
    其图像尺寸与G l-1 的尺寸相同；则由公式(3)得到：
    Figure PCTCN2018096023-appb-100004
    公式(2)中：
    Figure PCTCN2018096023-appb-100005
    令
    Figure PCTCN2018096023-appb-100006
    由LP 0、LP 1、…、LP N构成的金字塔即为双边滤波金字塔；它的每一层图像是基频层金字塔本层图像与其高一层的图像经过内插放大后图像的差，此过程相当于带通滤波，因此双边滤波金字塔又称为带通金字塔分解；最终，通过将得到的所有每层的基频层金字塔和得到的细节分量通过金字塔重建的方法复原，即得到融合后的图像。
  9. 如权利要求8所述的基于双边滤波金字塔的三光图像智能融合方法，其特征在于：金字塔重建的公式为：
    Figure PCTCN2018096023-appb-100007
PCT/CN2018/096023 2018-02-06 2018-07-17 基于双边滤波金字塔的三光图像智能融合方法 WO2019153651A1 (zh)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN201810118085.1 2018-02-06
CN201810118085.1A CN108399612B (zh) 2018-02-06 2018-02-06 基于双边滤波金字塔的三光图像智能融合方法

Publications (1)

Publication Number Publication Date
WO2019153651A1 true WO2019153651A1 (zh) 2019-08-15

Family

ID=63095287

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2018/096023 WO2019153651A1 (zh) 2018-02-06 2018-07-17 基于双边滤波金字塔的三光图像智能融合方法

Country Status (2)

Country Link
CN (1) CN108399612B (zh)
WO (1) WO2019153651A1 (zh)

Cited By (11)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN111292284A (zh) * 2020-02-04 2020-06-16 淮阴师范学院 基于双树-四元数小波变换的彩色图像融合方法
CN111462025A (zh) * 2020-02-26 2020-07-28 宁波大学 基于多尺度低秩矩阵分解的红外与可见光图像融合方法
CN111462032A (zh) * 2020-03-31 2020-07-28 北方夜视技术股份有限公司 非制冷红外图像与日盲紫外图像融合方法及应用
CN111652818A (zh) * 2020-05-29 2020-09-11 浙江大华技术股份有限公司 一种基于金字塔的图像滤波方法、装置及存储介质
CN112116632A (zh) * 2020-09-21 2020-12-22 中国科学院长春光学精密机械与物理研究所 一种循着目标尾烟追踪目标的方法、装置及介质
CN112135068A (zh) * 2020-09-22 2020-12-25 视觉感知(北京)科技有限公司 一种多输入视频融合处理的方法和装置
CN112465705A (zh) * 2020-12-08 2021-03-09 福州大学 基于两孔径旋转双棱镜的视场扩大系统及方法
CN113095358A (zh) * 2021-03-05 2021-07-09 北京中电联达信息技术有限公司 一种图像融合方法及系统
CN114815760A (zh) * 2022-06-27 2022-07-29 天津德通电气股份有限公司 一种安全生产追踪处置系统及方法
CN116681633A (zh) * 2023-06-06 2023-09-01 国网上海市电力公司 一种多波段成像及融合方法
CN116996675A (zh) * 2023-09-27 2023-11-03 河北天英软件科技有限公司 一种即时通信系统及信息处理方法

Families Citing this family (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN110400262B (zh) * 2019-04-10 2020-11-06 诸暨良嘉环保科技咨询有限公司 基于定制数据处理的识别装置
CN111814511B (zh) * 2019-04-10 2021-02-23 青岛大学附属医院 基于定制数据处理的识别方法
CN110956592B (zh) * 2019-11-14 2023-07-04 北京达佳互联信息技术有限公司 图像处理方法、装置、电子设备和存储介质
CN113643219B (zh) * 2021-08-03 2023-11-24 武汉三江中电科技有限责任公司 一种基于三光融合的图像成像方法及装置

Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN104182955A (zh) * 2014-09-05 2014-12-03 西安电子科技大学 基于可操纵金字塔变换的图像融合方法及其装置
US8928964B1 (en) * 2012-01-30 2015-01-06 Softronics, Ltd. Three-dimensional image display
CN105654448A (zh) * 2016-03-29 2016-06-08 微梦创科网络科技（中国）有限公司 一种基于双边滤波及权值重建的图像融合方法及系统
CN107607202A (zh) * 2017-08-31 2018-01-19 江苏宇特光电科技股份有限公司 三光融合智能成像仪及其方法

Family Cites Families (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN104376546A (zh) * 2014-10-27 2015-02-25 北京环境特性研究所 基于dm642的三路图像金字塔融合算法的实现方法

Patent Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US8928964B1 (en) * 2012-01-30 2015-01-06 Softronics, Ltd. Three-dimensional image display
CN104182955A (zh) * 2014-09-05 2014-12-03 西安电子科技大学 基于可操纵金字塔变换的图像融合方法及其装置
CN105654448A (zh) * 2016-03-29 2016-06-08 微梦创科网络科技（中国）有限公司 一种基于双边滤波及权值重建的图像融合方法及系统
CN107607202A (zh) * 2017-08-31 2018-01-19 江苏宇特光电科技股份有限公司 三光融合智能成像仪及其方法

Cited By (19)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN111292284A (zh) * 2020-02-04 2020-06-16 淮阴师范学院 基于双树-四元数小波变换的彩色图像融合方法
CN111292284B (zh) * 2020-02-04 2024-03-01 淮阴师范学院 基于双树-四元数小波变换的彩色图像融合方法
CN111462025B (zh) * 2020-02-26 2023-04-07 宁波大学 基于多尺度低秩矩阵分解的红外与可见光图像融合方法
CN111462025A (zh) * 2020-02-26 2020-07-28 宁波大学 基于多尺度低秩矩阵分解的红外与可见光图像融合方法
CN111462032A (zh) * 2020-03-31 2020-07-28 北方夜视技术股份有限公司 非制冷红外图像与日盲紫外图像融合方法及应用
CN111462032B (zh) * 2020-03-31 2023-03-31 北方夜视技术股份有限公司 非制冷红外图像与日盲紫外图像融合方法及应用
CN111652818A (zh) * 2020-05-29 2020-09-11 浙江大华技术股份有限公司 一种基于金字塔的图像滤波方法、装置及存储介质
CN111652818B (zh) * 2020-05-29 2023-09-29 浙江大华技术股份有限公司 一种基于金字塔的图像滤波方法、装置及存储介质
CN112116632A (zh) * 2020-09-21 2020-12-22 中国科学院长春光学精密机械与物理研究所 一种循着目标尾烟追踪目标的方法、装置及介质
CN112116632B (zh) * 2020-09-21 2023-12-05 中国科学院长春光学精密机械与物理研究所 一种循着目标尾烟追踪目标的方法、装置及介质
CN112135068A (zh) * 2020-09-22 2020-12-25 视觉感知(北京)科技有限公司 一种多输入视频融合处理的方法和装置
CN112465705A (zh) * 2020-12-08 2021-03-09 福州大学 基于两孔径旋转双棱镜的视场扩大系统及方法
CN112465705B (zh) * 2020-12-08 2022-08-19 福州大学 基于两孔径旋转双棱镜的视场扩大系统及方法
CN113095358A (zh) * 2021-03-05 2021-07-09 北京中电联达信息技术有限公司 一种图像融合方法及系统
CN114815760A (zh) * 2022-06-27 2022-07-29 天津德通电气股份有限公司 一种安全生产追踪处置系统及方法
CN116681633A (zh) * 2023-06-06 2023-09-01 国网上海市电力公司 一种多波段成像及融合方法
CN116681633B (zh) * 2023-06-06 2024-04-12 国网上海市电力公司 一种多波段成像及融合方法
CN116996675A (zh) * 2023-09-27 2023-11-03 河北天英软件科技有限公司 一种即时通信系统及信息处理方法
CN116996675B (zh) * 2023-09-27 2023-12-19 河北天英软件科技有限公司 一种即时通信系统及信息处理方法

Also Published As

Publication number Publication date
CN108399612A (zh) 2018-08-14
CN108399612B (zh) 2022-04-05

Similar Documents

Publication Publication Date Title
WO2019153651A1 (zh) 基于双边滤波金字塔的三光图像智能融合方法
US11328188B2 (en) Target-image acquisition method, photographing device, and unmanned aerial vehicle
EP3265996B1 (en) Quantifying gas in passive optical gas imaging
CN107959805B (zh) 基于混合相机阵列的光场视频成像系统及视频处理方法
EP2779624B1 (en) Apparatus and method for multispectral imaging with three-dimensional overlaying
CN107292860B (zh) 一种图像处理的方法及装置
RU2677562C2 (ru) Система и способ моделирования и калибровки устройства формирования изображения
CN111741281B (zh) 图像处理方法、终端及存储介质
WO2021098080A1 (zh) 基于边缘特征的多光谱相机外参自校正算法
CN204967995U (zh) 用于机柜的监测系统
CN106952225B (zh) 一种面向森林防火的全景拼接方法
CN110660088A (zh) 一种图像处理的方法和设备
CN112184604B (zh) 一种基于图像融合的彩色图像增强方法
CN107784632A (zh) 一种基于红外热成像系统的红外全景图的生成方法
WO2016205419A1 (en) Contrast-enhanced combined image generation systems and methods
CN107976257A (zh) 一种红外热成像仪的图像显示方法、装置及红外热成像仪
WO2007129446A1 (ja) 画像処理方法、画像処理プログラム、画像処理装置、及び撮像装置
CN113630549A (zh) 变焦控制方法、装置、电子设备和计算机可读存储介质
CN115222785A (zh) 一种基于双目标定的红外与可见光图像配准方法
Zhou et al. LAMOST Fiber Positioning Unit Detection Based on Deep Learning
CN110910457B (zh) 基于角点特征的多光谱立体相机外参计算方法
Chen et al. An image fusion algorithm of infrared and visible imaging sensors for cyber-physical systems
CN109029380B (zh) 基于镀膜式多光谱相机的立体视觉系统及其标定测距方法
CN113066011B (zh) 一种图像处理方法、装置、系统、介质和电子设备
CN114596506A (zh) 一种无人机巡检设备及图像融合方法

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 18904634

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 18904634

Country of ref document: EP

Kind code of ref document: A1