WO2023134103A1 - 图像融合方法、设备以及存储介质 - Google Patents

图像融合方法、设备以及存储介质 Download PDF

Info

Publication number
WO2023134103A1
WO2023134103A1 PCT/CN2022/094716 CN2022094716W WO2023134103A1 WO 2023134103 A1 WO2023134103 A1 WO 2023134103A1 CN 2022094716 W CN2022094716 W CN 2022094716W WO 2023134103 A1 WO2023134103 A1 WO 2023134103A1
Authority
WO
WIPO (PCT)
Prior art keywords
image
visible light
infrared
domain
blocks
Prior art date
Application number
PCT/CN2022/094716
Other languages
English (en)
French (fr)
Inventor
杨思雨
Original Assignee
无锡英菲感知技术有限公司
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 无锡英菲感知技术有限公司 filed Critical 无锡英菲感知技术有限公司
Publication of WO2023134103A1 publication Critical patent/WO2023134103A1/zh

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T5/00—Image enhancement or restoration
    • G06T5/50—Image enhancement or restoration using two or more images, e.g. averaging or subtraction
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/10—Image acquisition modality
    • G06T2207/10048—Infrared image
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/20—Special algorithmic details
    • G06T2207/20048—Transform domain processing
    • G06T2207/20056—Discrete and fast Fourier transform, [DFT, FFT]
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/20—Special algorithmic details
    • G06T2207/20212—Image combination
    • G06T2207/20221—Image fusion; Image merging

Definitions

  • the present application relates to the technical field of image processing, and in particular to an image fusion method, device and storage medium.
  • Infrared imaging technology uses infrared radiation to sense the surrounding objective environment, and its images have the characteristics of good environmental adaptability, good concealment, and high recognition of camouflaged targets. Thanks to its imaging principle, infrared images have temperature representation characteristics, and after certain algorithm processing, you can feel the temperature distribution information of the target. However, limited by the difficulty of manufacturing the infrared detector and the influence of material purity, infrared images generally have low resolution, high noise, low image contrast, and narrow gray scale range, resulting in blurred contrast between the background and the monitored target. , the details of the monitored target are difficult to identify, and the image feature information is not clear, which makes it difficult to identify and analyze the target and scene in the image.
  • visible light images Compared with infrared images, visible light images have higher contrast and resolution, and have the advantages of rich spectral information, large dynamic range, more detailed information, and good visibility. However, the anti-interference ability of visible light images is poor. In low light, foggy days, and camouflaged targets, the effect of visible light images will obviously become unsatisfactory. Analyze and identify. It can be seen that infrared and visible light images are two types of images with complementary advantages and disadvantages.
  • the patent application number CN109478315A discloses a fusion image optimization system and method, which only extracts and utilizes the contour/edge information in the visible light image in the process of fusing the infrared image and the visible light image, while ignoring the information given by the visible light image. other large-scale information.
  • the patent application number CN105069768A discloses a visible light image and infrared image fusion processing system and image fusion method.
  • the image fusion scheme involves a series of operations such as low-pass filtering, detail enhancement, low-frequency fusion, and high-low frequency fusion. The overall process It is too cumbersome to output video streams in real time on devices with limited computing power.
  • the embodiments of the present invention provide an image fusion method, device and storage medium capable of obtaining more detailed information and realizing process simplification.
  • an image fusion method including:
  • the second aspect of the embodiment of the present invention provides an image fusion device, including a memory and a processor;
  • the processor executes the computer program instructions stored in the memory, it executes the steps of the image fusion method.
  • a computer-readable storage medium stores computer program instructions
  • the registered infrared image and visible light image are divided into multiple image blocks, and then each image block is transformed into the frequency domain by Fourier transform In , each image block is fused in the frequency domain, and finally each fused image block is merged into a fused image. Therefore, the fused image obtained by using the image fusion method fully distinguishes and extracts effective information of infrared and visible light images, and improves the utilization rate of image information.
  • FIG. 1 is a schematic flowchart of an image fusion method provided according to some embodiments of the present application.
  • FIG. 2 is a schematic flowchart of an image fusion method provided according to other embodiments of the present application.
  • Fig. 3 is a schematic diagram of the processing process of the image fusion method provided according to some other embodiments of the present application.
  • Fig. 4 is a schematic structural diagram of an image fusion device provided according to some embodiments of the present application.
  • Fig. 5 is a schematic structural diagram of an image fusion device provided according to some embodiments of the present application.
  • FIG. 6 is an infrared image used for image fusion according to an image fusion method provided by some embodiments of the present application.
  • Fig. 7 is a visible light image used for image fusion according to the image fusion method provided by some embodiments of the present application.
  • Fig. 8 is a fused image obtained after merging Fig. 6 and Fig. 7 according to the image fusion method provided by some embodiments of the present application.
  • connection should be understood in a broad sense, for example, it can be a fixed connection or a detachable connection. Connected, or integrally connected; it can be directly connected, or indirectly connected through an intermediary, and it can be the internal communication of two elements. Those of ordinary skill in the art can understand the specific meanings of the above terms in this application according to specific situations.
  • FIG. 1 is a schematic flowchart of an image fusion method provided by some embodiments of the present application.
  • the image fusion method is implemented by the image fusion device shown in FIG. 4 or the image fusion device shown in FIG. 5 .
  • the image fusion method includes S2, S4, S6, S8 and S10.
  • S2 can be realized by the image division module 11 in the image device shown in Figure 4, or store the image division program by the memory 21 in the image fusion device shown in Figure 5, and then by the processing in the fusion image device shown in Figure 5
  • the device 22 is implemented when executing the image division program stored in the memory 21.
  • the registered infrared image and visible light image mentioned in S2 refer to the infrared image and visible light image after aligning and mapping the original infrared image and visible light image into the same coordinate system.
  • the registered infrared image and visible light image are respectively divided into blocks using the image block algorithm, so as to divide the registered infrared image into multiple infrared image blocks, and divide the registered visible light image into corresponding number of visible light image blocks, that is, for each infrared image block, there is a corresponding visible light image block in position.
  • the positional correspondence mentioned here means that the positions of the infrared image block and the corresponding visible light image block correspond to each other in the same coordinate system.
  • the large-sized original infrared image and the original visible light image are divided into small-sized image blocks, which is beneficial to reduce the cost of each operation in the image fusion process.
  • Data volume and computational pressure are beneficial to reduce the cost of each operation in the image fusion process.
  • S4 Perform Fourier transform on each of the infrared image blocks and the visible light image blocks to obtain corresponding frequency-domain infrared image blocks and frequency-domain visible light image blocks.
  • S4 can be realized by the Fourier transform module 12 in the image device shown in Figure 4, or store the Fourier transform program by the memory 21 in the image fusion device shown in Figure 5, and then by the fused image shown in Figure 5
  • the processor 22 in the device is implemented when executing the Fourier transform program stored in the memory 21 .
  • a two-dimensional Fourier transform is performed on each infrared image block and each visible light image block, so that each image block can be transformed from the spatial domain to the frequency domain.
  • the fusion of images in the frequency domain can better distinguish and extract the effective key information of infrared images and visible light images. This is because, in the frequency domain, an image is decomposed into low-frequency parts representing overall information and high-frequency parts representing local details.
  • S6 can be realized by the image fusion module 13 in the image device shown in Figure 4, or store the image fusion program by the memory 21 in the image fusion equipment shown in Figure 5, and then by the processing in the fusion image equipment shown in Figure 5
  • the device 22 is implemented when executing the image fusion program stored in the memory 21.
  • the correspondence in the frequency-domain visible light image block corresponding to the frequency-domain infrared image block refers to that the position of the infrared image block corresponding to the frequency-domain infrared image block in the same coordinate system is the same as that of the frequency-domain visible light image
  • the positions of the visible light image blocks corresponding to the blocks in the above-mentioned same coordinate system are corresponding.
  • the registered infrared image is divided into K infrared image blocks, and the registered visible light image is correspondingly divided into K visible light image blocks, and the K infrared image blocks and K visible light image blocks are one by one
  • the nth infrared image block among the K infrared image blocks and the mth visible light image block among the K visible light image blocks have the same corresponding position in the above-mentioned same coordinate system
  • the nth infrared image block corresponds to
  • the visible light image block is the mth visible light image block, where n and m may be the same or different.
  • S8 Perform inverse Fourier transform on each of the frequency-domain fused image blocks to obtain corresponding spatial-domain fused image blocks.
  • S8 can be realized by the inverse Fourier transform module 14 in the image device shown in Figure 4, or store the inverse Fourier transform program by the memory 21 in the image fusion device shown in Figure 5, and then by the inverse Fourier transform module shown in Figure 5
  • the processor 22 in the image fusion device is implemented when executing the inverse Fourier transform program stored in the memory 21 .
  • each frequency domain fused image block is transformed back into the space domain by using the two-dimensional inverse Fourier transform, so that the fused image blocks can be subsequently merged in the space domain.
  • S10 can be realized by the image merging module 15 in the image device shown in Figure 4, or store the image merging program by the memory 21 in the image fusion device shown in Figure 5, and then by the processing in the fusion image device shown in Figure 5
  • the device 22 is implemented when executing the image merging program stored in the memory 21.
  • the registered infrared image and visible light image are divided into multiple image blocks, and then each image block is transformed into the frequency domain by Fourier transform, and each image block is fused in the frequency domain, and finally Merge individual fused image blocks into a fused image. Therefore, the fused image obtained by using the image fusion method provided in the present application can fully distinguish and extract effective information of infrared and visible light images, thereby improving the utilization rate of image information.
  • the image fusion method provided according to the embodiment of the present application is used to fuse the infrared image shown in FIG. 6 and the visible light image shown in FIG. 7 to obtain the fused image shown in FIG. 8 .
  • the infrared image shown in Figure 6 has temperature characterization characteristics, and the higher the temperature of the object, the brighter the brightness in the infrared image, as shown in Figure 6, the brightness of the person and the higher temperature object are brighter than other objects .
  • the infrared image has low resolution, high noise, low image contrast, and narrow gray scale range, which makes the contrast between the background and the monitored object blurred, and the details of the monitored object are difficult to identify , the image feature information is not clear.
  • FIG. 7 it is a visible light image obtained in the same scene as Figure 6. Compared with the infrared image, the visible light image shown in Figure 7 has higher contrast and resolution, more detailed information, and better visibility. advantage. However, it can also be clearly seen from FIG. 7 that the visible light image has poor anti-interference ability, and the brightness of the target object in the darker light area is relatively dark, and it is difficult to distinguish the characteristics of the target object.
  • the infrared image shown in Figure 6 and the visible light image shown in Figure 7 are aligned and mapped in the same coordinate system, and then the registered infrared image and The visible light image is divided into multiple image blocks, and then each image block is transformed into the frequency domain by Fourier transform, and then each image block is fused in the frequency domain, and finally each fused image block is merged into one as shown in Figure 8 Fused image shown. Comparing Figures 6 to 8, it can be seen that the fused image obtained by using the image fusion method provided by the embodiment of the present application can fully distinguish and extract effective information of infrared and visible light images, which improves the utilization rate of image information.
  • the image fusion method provided in the embodiment of the present application can solve the problem of obtaining a fused image that can clearly distinguish the target object based on the infrared image and the visible light image containing the target object when the target object is in a dark scene. Therefore, the image fusion method provided by this application can be widely used in mobile phone built-in modules, mobile phone plug-ins, vehicle vision systems, drone shooting systems, and other image capture devices or equipment mainly used outdoors.
  • S2 is specifically to divide the registered infrared image and visible light image into multiple image blocks evenly along the row direction and the column direction respectively, and obtain corresponding infrared image blocks and visible light image blocks.
  • the registered infrared image and visible light image are divided into N ⁇ M corresponding image blocks, that is, the registered infrared image and visible light image are divided into N rows and M columns, and each row corresponds to M images
  • the areas of the blocks are equal in size, and the areas of the N image blocks corresponding to each column are equal in size.
  • a sliding window of a preset size can be used to slide through the configured infrared image and visible light image in sequence according to a preset step size, and each time the sliding window slides one step, a corresponding infrared image block is obtained and visible image blocks.
  • Image blocks The method of dividing the registered infrared image and visible light image into image blocks is not particularly limited in this application.
  • the registered infrared image and visible light image are evenly divided into a plurality of image blocks along the row direction and the column direction respectively, and corresponding infrared image blocks and visible light image blocks are obtained, wherein adjacent There is a preset overlapping area between two infrared image blocks and two adjacent visible light image blocks. Setting an overlapping area of an appropriate size between adjacent image blocks can reduce the "pseudo-structure" that may be introduced in the fusion step (S6) of each frequency-domain image block in the frequency domain, that is, avoid the phenomenon of fusion image distortion .
  • the registered infrared image and visible light image are evenly divided into a plurality of image blocks along the row direction and the column direction respectively, and corresponding infrared image blocks and visible light image blocks are obtained, and any adjacent infrared images There is no overlapping area between blocks and adjacent visible light image blocks. Also or for the region prone to "false structure" (distortion) in the fusion process, the adjacent infrared image blocks and the adjacent visible light image blocks are set to have a preset overlapping area, while in the remaining other areas , then there will be no overlapping regions between any adjacent infrared image blocks and between adjacent visible light image blocks.
  • the result of the frequency-domain infrared image block and the corresponding frequency-domain visible light image block is weighted and averaged pixel by pixel according to a preset weight as the corresponding frequency-domain fusion image block , wherein the certain preset weight can be set according to the corresponding pixel positions of the frequency-domain infrared image block and the frequency-domain visible light image block.
  • each corresponding frequency domain image block select the pixel with a larger intensity absolute value in the infrared and visible light image block as the corresponding pixel of the frequency domain fusion image block; or select the frequency domain infrared image in the low frequency band
  • the pixels of the block and the high-frequency band select the pixels of the frequency-domain visible light image block to determine the corresponding pixels in the frequency-domain fusion image.
  • S6 specifically includes: weighting and averaging each of the frequency-domain infrared image blocks and the frequency-domain visible light image blocks corresponding to them according to preset weights pixel by pixel to obtain corresponding frequency-domain fused image blocks.
  • the preset weight is set according to the corresponding pixel position of the corresponding frequency domain image block. For example, the farther the corresponding pixel is from the origin, the smaller the corresponding preset weight is.
  • the pixel-by-pixel weighted averaging according to the preset weight includes weighting the pixels in the frequency-domain infrared image block with the first weight to determine the first weighted value, and weighting the pixels at the corresponding positions in the frequency-domain visible light image block with the second weight to determine the second weight. two weighted values, and then determine the average value of the first weighted value and the second weighted value, and then use the average value as the pixel at the corresponding position in the frequency domain fused image block. Wherein, the sum of the first weight and the second weight is 1.
  • S6 may also specifically include: comparing each of the frequency-domain infrared image blocks with the pixel values at the same position in the corresponding frequency-domain visible light image, and determining the corresponding position according to the pixel with a larger pixel value The corresponding pixels of the frequency domain fusion image are obtained. If pixel A in the frequency-domain infrared image block and pixel B in the corresponding visible light image block are pixels at the same position, compare the pixel values of the two pixels, and if the pixel value of pixel A is larger, compare pixel A The pixel of is used as the pixel at the corresponding position in the frequency domain fusion image block, otherwise, the pixel B is used as the pixel at the corresponding position in the frequency domain fusion image block. Corresponding positions here refer to positions having the same pixel coordinates.
  • S6 also It may specifically include: determining pixels at corresponding positions in the low-frequency band according to pixels in the frequency-domain infrared image block, determining pixels at corresponding positions in the high-frequency band based on pixels in the frequency-domain visible light image block, and obtaining corresponding frequency-domain fusion Image blocks.
  • S6 may also adopt the fusion rule of obtaining the corresponding frequency-domain fused image block by weighting and averaging each of the frequency-domain infrared image blocks and the corresponding frequency-domain visible light image blocks according to preset weights pixel by pixel.
  • the preset weight here is related to the frequency band where it is located. In the low frequency band, the first weight is greater than the second weight, while in the high frequency band, the first weight is smaller than the second weight.
  • each spatially fused image patch in the block consists of non-overlapping and overlapping regions.
  • S10 specifically includes: during the process of sequentially merging each spatially fused image block, determining the corresponding position in the fused image according to the pixels in the non-overlapping area of the spatially fused image block (with the non-overlapping The pixel corresponding to the position corresponding to the overlapping area), and the pixel corresponding to the corresponding position (position corresponding to the overlapping area) in the fusion image is determined according to the average pixels of the adjacent spatial fusion image blocks having the overlapping area in the corresponding overlapping area.
  • the pixels at the position corresponding to the overlapping area d in the fused image are divided by the spatially fused image block a and the spatially fused image block b
  • the average pixel in the overlapping area d is determined.
  • the method of determining the pixel at the position corresponding to the overlapping area d in the fused image can also be: the distance between the central position pixels of the adjacent two spatially fused image blocks a, b and the overlapping area d The distance is compared, and the pixel at the position corresponding to the overlapping area d in the fused image is determined according to the overlapping area pixels of the spatially fused image block corresponding to the shorter distance. If the center position of the spatially fused image block a is relatively close to the overlapping area d, then the pixels in the overlapping area of the spatially fused image block a are used as the pixels corresponding to the overlapping area d in the fused image.
  • the image fusion method further includes obtaining the configured Aligned infrared image and visible light image
  • the fusion device corresponding to Figure 4 also includes a registration acquisition module, which is used to realize the step of acquiring the registered infrared image and visible light image, and this step can also be performed by the image fusion in Figure 4 device implementation.
  • the step of acquiring the registered infrared image and visible light image specifically includes S11, S12, S13 and S14.
  • S11 Acquiring infrared images and visible light images of the same target in the same scene.
  • an optical device such as an infrared thermal imager
  • an infrared camera and a visible light camera can be used to shoot the same target in the scene at the same time, so as to obtain corresponding infrared images and visible light images .
  • Separate infrared cameras and visible light cameras can also be used to photograph the same target in the same scene at the same moment or at different moments to obtain corresponding infrared images and visible light images.
  • the infrared camera and visible light camera can be placed at the same position, and the optical axis of the lens is in the same direction and parallel to obtain infrared images and visible light images at the same angle.
  • the infrared camera and the visible light camera can also be placed in different positions to acquire infrared images and visible light images from different angles.
  • the distribution ratios of the infrared image and the visible light image can be the same or different.
  • the resolution of the collected image can be set when the corresponding photographing device collects the infrared image and the visible light image, or the resolution of the corresponding image can be adjusted and configured after the corresponding image is collected.
  • appropriate image processing can be performed on the corresponding image, such as image cropping or image stretching to make the infrared image shown and the shown infrared image
  • the visible light images are the same size.
  • the detailed information is mainly extracted from the visible light image, such as the outline of the target object.
  • Perform grayscale processing to obtain its corresponding grayscale image.
  • S14 Align and map the infrared image and the visible light image into the same coordinate system according to the coordinate mapping relationship, to obtain the registered infrared image and the visible light image.
  • the coordinate system of the infrared image and the visible light image may be different, that is, the coordinate systems of the two are different, and the pixels corresponding to the same target point in the infrared image and the visible light image display different pixel coordinates in different spatial coordinate systems, so that the relationship between the two can be established. coordinate mapping relationship.
  • the coordinate mapping relationship can be given in the following form: the pixel at row i1 and column j1 of the infrared image corresponds to the pixel at row i2 and column j2 of the visible light image, or the pixel at row i1 and column j1 of the visible light image corresponds to the pixel of the infrared image
  • the coordinate mapping relationship may also be represented in the form of a mapping table. In this application, the specific expression form of the coordinate mapping relationship is not particularly limited.
  • the infrared image and the infrared camera can be determined according to the parameter information (internal reference) and relative geometric position information of the infrared camera that collects the infrared image and the visible light camera that collects the visible light image.
  • aligning and mapping the infrared image and the visible light image into the same coordinate system according to the coordinate mapping relationship to obtain the registered infrared image and the visible light image includes: aligning and mapping the infrared image and the visible light image into the same coordinate system according to the mapping relationship, and assigning values to pixels corresponding to non-integer pixel coordinates in the same coordinate system based on a grayscale interpolation method. Because the pixel coordinates in the mapping relationship determined above are not integers, that is, when a pixel in the aligned image is projected into the original infrared image or visible light image, its corresponding coordinates are not integers.
  • the grayscale interpolation method is the nearest neighbor interpolation method, and assigning a value to a pixel corresponding to a non-integer pixel coordinate in the same coordinate system based on the nearest neighbor interpolation method is specifically assigning the nearest integer coordinate to the non-integer The pixel of is used as the corresponding pixel in the alignment mapping corresponding to the non-integer coordinates.
  • the gray level interpolation method may also be bilinear interpolation method, bicubic interpolation method and other interpolation methods.
  • the fused image obtained in S10 is a grayscale fused image.
  • the image fusion method further includes S20: transforming the fused image according to a preset pseudo-color mapping table For fusing pseudo-color images.
  • FIG. 3 shows some other embodiments of the image fusion method according to the present application.
  • the image fusion method provided by the present application mainly includes S01 to S010, and the processing process of the original infrared image and visible light image according to the image fusion method of this embodiment is shown in FIG. 3 .
  • S03 Determine the coordinate mapping relationship. Determine the coordinate mapping relationship between the original infrared image and the grayscale image corresponding to the original visible light image.
  • S05 Image segmentation.
  • the aligned and mapped infrared image and the grayscale image are respectively divided into a plurality of image blocks, so as to obtain corresponding infrared image blocks and visible light image blocks.
  • S06 Fourier transform. Fourier transform is performed on the infrared image block and the visible light image block respectively to obtain corresponding frequency domain image blocks.
  • S07 Image block fusion. Each frequency-domain infrared image block is fused with a corresponding frequency-domain visible light image block to obtain each corresponding frequency-domain fused image block.
  • S08 Fourier inverse transform. Inverse Fourier transform is performed on each frequency-domain fused image block to obtain each corresponding spatial-domain fused image block.
  • S09 Image block merging. Merge each space-domain fused image block according to the original order to recover the entire gray-scale fused image.
  • S010 Pseudo-color mapping.
  • the grayscale fusion image is mapped to the pseudo-color fusion image according to the preset pseudo-color mapping table.
  • the image fusion method provided by this application decomposes and extracts the information on the original infrared and visible light images through two-dimensional Fourier transform, fully utilizes all the information in the infrared and visible light images, and the overall process is relatively streamlined, which is conducive to computing power Handle live video streams on constrained devices.
  • the present application also provides an image fusion device.
  • the image fusion device includes an image division module 11 , a Fourier transform module 12 , an image fusion module 13 , an inverse Fourier transform module 14 and an image merging module 15 .
  • the image division module 11 is used for dividing the registered infrared image and visible light image into a plurality of image blocks respectively, and obtains corresponding infrared image blocks and visible light image blocks;
  • the Fourier transform module 12 is used for each infrared image The block and the visible light image block are respectively subjected to Fourier transform to obtain the corresponding frequency-domain infrared image block and frequency-domain visible light image block;
  • the image fusion module 13 is used to combine each of the frequency-domain infrared image blocks with its corresponding Frequency-domain visible light image blocks are fused according to preset fusion rules to obtain corresponding frequency-domain fused image blocks;
  • the inverse Fourier transform module 14 is used to perform Fourier inverse transform on each of the frequency-domain fused image blocks to obtain corresponding Spatially fused image blocks;
  • the image merging module 15 is used for merging each of the spatially fused image blocks to obtain a fused image.
  • the present application also provides an image fusion device, the memory 21 of which is stored in the processor 13 .
  • the processor 22 runs the computer program instructions stored in the memory 21, it executes the steps of the image fusion method described in any embodiment of the present application.
  • the present application also provides a computer-readable storage medium, the computer-readable storage medium stores computer program instructions; when the computer program instructions are executed by a processor, the items provided according to any embodiment of the present application are realized. The steps of the detection method.
  • the aforementioned processor may be a CPU (Central Processing Unit, Central Processing Unit), or an ASIC (Application Specific Integrated Circuit), or one or more integrated circuits configured to implement the embodiments of the present invention.
  • One or more processors included in the moving target detection device can be the same type of processor, such as one or more CPUs; it can also be different types of processors, such as one or more CPUs and one or more ASICs .
  • the above-mentioned memory may include high-speed RAM (Random Access Memory, Random Access Memory), and may also include NVM (Non-Volatile Memory, Non-Volatile Memory), such as at least one disk memory.
  • RAM Random Access Memory
  • NVM Non-Volatile Memory, Non-Volatile Memory
  • the present application also provides a computer-readable storage medium, the computer-readable storage medium stores computer program instructions, and when the computer program instructions are executed by a processor, the implementation of any one of the embodiments provided in the present application can be realized. Steps of the distance measuring method.

Landscapes

  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Image Processing (AREA)

Abstract

一种图像融合方法、设备以及存储介质，所述图像融合方法包括将已经配准的红外图像和可见光图像划分为多个图像块，然后通过傅里叶变换将各个图像块变换到频域中，在频域中对各个图像块进行融合，最后再将各个融合图像块合并成融合图像。因此，采用所述图像融合方法获得的融合图像充分区分和提取了红外和可见光图像的有效信息，提高了图像信息的利用率。

Description

图像融合方法、设备以及存储介质 技术领域
本申请涉及图像处理技术领域，尤其涉及一种图像融合方法、设备以及存储介质。
背景技术
红外成像技术利用红外辐射感知周围的客观环境，其图像具有环境适应性好、隐蔽性好、伪装目标辨识度高的特点。得益于其成像原理，红外图像具有温度表征特性，经过一定的算法处理，就能感受到目标的温度分布信息。但是受限于红外探测器的制作工艺难度和材料纯度影响，红外图像普遍存在分辨率低、噪声大、图像对比度低、灰度范围窄的现象，由此造成背景与被监测目标之间对比度模糊，被监测目标细节难以辨认，图像特征信息不明确，使得对图像中目标和场景的识别和分析存在一定的难度。相比于红外图像，可见光图像的对比度和分辨率都比较高，有光谱信息丰富、动态范围大、细节信息多、视觉性好等优点。但是可见光图像抗干扰能力差，在微光、雾天、目标有伪装等情况下，可见光图像的效果则会明显变得不尽人意，难以获取目标和场景的信息，甚至使得无法对目标和场景进行分析和识别。由此可见，红外和可见光图像是两类优缺点互补的图像。
同时用一个红外相机和可见光相机观察场景或物体可以在更广泛的环境条件下更加全面地了解其特征。然而在实际应用中，若采用两台显示器分别显示红外和可见光图像，会造成显示资源的浪费，而采用一台显示器分成两部分来分别显示红外图像和可见光图像又降低图像的显示质量。因此，把红外与可见光图像融合成一张图像的图像融合技术应运而生。
申请号为CN109478315A的专利公开了一种融合图像优化系统和方法，其在将红外图像和可见光图像进行融合的过程中仅提取并利用了可见光图像中的轮廓/边缘信息，而忽视了可见光图像给予的其他大尺度信息。此外，申请号为CN105069768A的专利公开了一种可见光图像与红外图像融合处理系统及图像融合方法，其图像融合方案涉及低通滤波、细节增强、低频融合、高低频融合等一系列操作，整体流程过于繁琐，难以在算力受限的设备上实时输出视频流。
技术问题
为了解决现有存在的技术问题，本发明实施例提供一种能够获取更多细节信息以及实现流程精简的图像融合方法、设备以及存储介质。
技术解决方案
本发明实施例第一方面，提供了一种图像融合方法，包括：
将已配准的红外图像和可见光图像分别划分为多个图像块，获得对应的红外图像块和可见光图像块；
对各个所述红外图像块和所述可见光图像块分别进行傅里叶变换，获得对应的频域红外图像块和频域可见光图像块；
将各个所述频域红外图像块分别与其对应的所述频域可见光图像块按预设融合规则进行融合，获得对应的频域融合图像块；
将各个所述频域融合图像块进行傅里叶逆变换，获得对应的空域融合图像块；
将各个所述空域融合图像块进行合并，获得融合图像。
本发明实施例第二方面，提供了一种图像融合设备，包括存储器和处理器；
所述处理器在运行所述存储器中存储的计算机程序指令时，执行所述的图像融合方法的步骤。
本发明实施例第三方面，提供了一种计算机可读存储介质，所述计算机可读存储介质存储有计算机程指令；
所述计算机程序指令被处理器执行时，实现所述的图像融合方法的步骤。
有益效果
由上可见，在本申请提供的图像融合方法、设备以及存储介质中，将已经配准的红外图像和可见光图像划分为多个图像块，然后通过傅里叶变换将各个图像块变换到频域中，在频域中对各个图像块进行融合，最后再将各个融合图像块合并成融合图像。因此，采用所述图像融合方法获得的融合图像充分区分和提取红外和可见光图像的有效信息，提高了图像信息的利用率。
附图说明
附图仅用于示出实施方式，而并不认为是对本发明的限制。而且在整个附 图中，用相同的参考符号表示相同的部件。在附图中：
图1为依据本申请一些实施例提供的图像融合方法的流程示意图；
图2为依据本申请另一些实施例提供的图像融合方法的流程示意图；
图3为依据本申请又一些实施例提供的图像融合方法的处理过程示意图；
图4为依据本申请一些实施例提供的图像融合装置的结构示意图；
图5为依据本申请一些实施例提供的图像融合设备的结构示意图；
图6为依据本申请一些实施例提供的图像融合方法的进行图像融合所使用的红外图像；
图7为依据本申请一些实施例提供的图像融合方法的进行图像融合所使用的可见光图像；
图8为依据本申请一些实施例提供的图像融合方法的对图6和图7进行融合后获得的融合图像。
本发明的实施方式
以下结合说明书附图及具体实施例对本申请技术方案做进一步的详细阐述。
除非另有定义，本文所使用的所有的技术和科学术语与属于本申请的技术领域的技术人员通常理解的含义相同。本文中在本申请的说明书中所使用的术语只是为了描述具体的实施例的目的，不是旨在于限制本申请的实现方式。本文所使用的术语“及/或”包括一个或多个相关的所列项目的任意的和所有的组合。
在本申请的描述中，需要理解的是，术语“中心”、“上”、“下”、“前”、“后”、“左”、“右”、“竖直”、“行”、“顶”、“底”、“内”、“外”等指示的方位或位置关系为基于附图所示的方位或位置关系，仅是为了便于描述本申请和简化描述，而不是指示或暗示所指的装置或元件必须具有特定的方位、以特定的方位构造和操作，因此不能理解为对本申请的限制。在本申请的描述中，除非另有说明，“多个”的含义是两个或两个以上。
在本申请的描述中，需要说明的是，除非另有明确的规定和限定，术语“安装”、“相连”、“连接”应做广义理解，例如，可以是固定连接，也可以是可拆卸连接，或一体地连接；可以是直接相连，也可以通过中间媒介间接相连，可 以是两个元件内部的连通。对于本领域的普通技术人员而言，可以根据具体情况理解上述术语在本申请中的具体含义。
图1为本申请一些实施例提供的图像融合方法的流程示意图，所述图像融合方法通过图4所示的图像融合装置或图5所示的图像融合设备实现。具体的，所述图像融合方法包括S2、S4、S6、S8以及S10。
S2：将已配准的红外图像和可见光图像分别划分为多个图像块，获得对应的红外图像块和可见光图像块。
S2可以由图4所示的图像装置中的图像划分模块11实现，或者由图5所示的图像融合设备中的存储器21存储图像划分程序，再由图5所示的融合图像设备中的处理器22在执行所述存储器21存储的图像划分程序时实现。
在S2中所述的已配准的红外图像和可见光图像是指将原始的红外图像和可见光图像对齐映射到同一坐标系中后的红外图像和可见光图像。按照位置区域对已配准的红外图像和可见光图像分别利用图像分块算法进行分块处理，以将配准的红外图像划分为多个红外图像块，以及将已配准的可见光图像划分为对应数的可见光图像块，即对于每一个红外图像块，均有一个在位置上对应的可见光图像块。这里说的位置上对应是指红外图像块与对应的可见光图像块在所述同一坐标系中的位置相对应。通过对配准后的红外图像和可见光图像分别进行分块处理，以将大尺寸的原始红外图像和原始可见光图像划分为各个小尺寸的图像块，有利于减小图像融合过程中每次运算的数据量和计算压力。
S4：对各个所述红外图像块和所述可见光图像块分别进行傅里叶变换，获得对应的频域红外图像块和频域可见光图像块。
S4可以由图4所示的图像装置中的傅里叶变换模块12实现，或者由图5所示的图像融合设备中的存储器21存储傅里叶变换程序，再由图5所示的融合图像设备中的处理器22在执行所述存储器21存储的傅里叶变换程序时实现。
对每一个所述红外图像块和每一个可见光图像块分别做二维傅里叶变换，可以将各个图像块从空域变换到频域中。在频域中对图像进行融合，可以更好的区分和提取红外图像以及可见光图像的有效关键信息。这是因为，在频域中，图像被分解成了代表整体信息的低频部分和代表局部细节的高频部分。在红外图像与可见光图像融合的过程中，需要尽可能保留住红外图像中的整体信息，而不去关注红外图像的细节信息，以及尽可能的关注可见光图像的细节信息， 而不去关注可见光图像的整体信息。由此可见，把各个红外图像块和可见光图像块分解到频域有利于区分和提取红外和可见光图像的有效信息。
S6：将各个所述频域红外图像块分别与其对应的所述频域可见光图像块按预设融合规则进行融合，获得对应的频域融合图像块。
S6可以由图4所示的图像装置中的图像融合模块13实现，或者由图5所示的图像融合设备中的存储器21存储图像融合程序，再由图5所示的融合图像设备中的处理器22在执行所述存储器21存储的图像融合程序时实现。
与所述频域红外图像块对应的所述频域可见光图像块中的对应是指，所述频域红外图像块对应的红外图像块在上述同一坐标系中的位置与所述频域可见光图像块对应的可见光图像块在上述同一坐标系中的位置相对应。
如在S2中将已配准的红外图像划分为K个红外图像块，则对应的将已配准的可见光图像也划分K个可见光图像块，K个红外图像块与K个可见光图像块一一对应，如K个红外图像块中的第n个红外图像块与K个可见光图像块中的第m个可见光图像块在上述同一坐标系中的对应位置相同，则第n个红外图像块对应的可见光图像块为第m个可见光图像块，其中，n与m可以相同也可以不同。在S4中将第n个红外图像块经过傅里叶变换后，获得对应的第n个频域红外图像块，将第m个可见光图像块经过傅里叶变换后，获得对应的第m个频域可见光图像块。然后在S6中将第n个频域红外图像块与第m个频域可见光图像块按照预设的融合规则进行融合，以获得对应的频域融合图像块。
S8：将各个所述频域融合图像块进行傅里叶逆变换，获得对应的空域融合图像块。
S8可以由图4所示的图像装置中的傅里叶逆变换模块14实现，或者由图5所示的图像融合设备中的存储器21存储傅里叶逆变换程序，再由图5所示的融合图像设备中的处理器22在执行所述存储器21存储的傅里叶逆变换程序时实现。
在完成各个对应的频域图像块之间的融合后，利用二维傅里叶逆变换将各个频域融合图像块再变换回空域中，以便后续在空域中对各个融合的图像块进行合并。
S10：将各个所述空域融合图像块进行合并，获得融合图像。
S10可以由图4所示的图像装置中的图像合并模块15实现，或者由图5所 示的图像融合设备中的存储器21存储图像合并程序，再由图5所示的融合图像设备中的处理器22在执行所述存储器21存储的图像合并程序时实现。
在对各个空域融合图像块进行合并的过程中，需要根据各个空域融合图像块在上述同一坐标系中对应的位置按照原来的顺序恢复成一整幅融合图像。
由上可见，将已经配准的红外图像和可见光图像划分为多个图像块，然后通过傅里叶变换将各个图像块变换到频域中，在频域中对各个图像块进行融合，最后再将各个融合图像块合并成融合图像。因此，采用本申请提供的图像融合方法获得的融合图像充分区分和提取红外和可见光图像的有效信息，提高了图像信息的利用率。
依据本申请实施例提供的图像融合方法用于将如图6所示的红外图像和如图7所示的可见光图像进行融合获得图8所示的融合图像。如图6所示的红外图像具有温度表征特性，温度越高的对象在所述红外图像中亮度越亮，如图6中的人和温度较高的物体的亮度相对于其它物体而言较亮。然而，从图6可以看出，红外图像存在分辨率低、噪声大、图像对比度低、灰度范围窄的现象，使得背景与被监测目标对象之间对比度模糊，被监测目标对象的细节难以辨认，图像特征信息不明确。因此，对红外图像中目标对象和场景的识别及分析存在一定的难度。如图7所示，其为与图6在同一场景下获得的可见光图像，相比于红外图像，图7所示的可见光图像的对比度和分辨率都比较高，细节信息多、视觉性好等优点。然而，从图7中也可以明显看出，可见光图像抗干扰能力差，光线较暗的区域中的目标对象的亮度较暗，难以分辨出目标对象的特征。因此，本申请实施例提供的图像融合方法，将如图6所示的红外图像和如图7所示的可见光图像对齐映射在同一坐标系中配准后，再将已经配准的红外图像和可见光图像划分为多个图像块，然后通过傅里叶变换将各个图像块变换到频域中，再在频域中对各个图像块进行融合，最后再将各个融合图像块合并成如如图8所示的融合图像。对比图6至图8,可以看出，采用本申请实施例提供的图像融合方法获得的融合图像充分区分和提取红外和可见光图像的有效信息，提高了图像信息的利用率。
本申请实施例提供的图像融合方法可解决目标对象在较暗场景中时，基于含目标对象的红外图像和可见光图像，获得可清晰分辨出所述目标对象的融合图像。因此，本申请提供的图像融合方法可广泛应用于诸如手机内置模组、手 机插件、车载视觉系统、无人机拍摄系统以及其它主要在户外使用的图像拍摄装置或设备中。
在一些实施例中，S2具体为将已配准的红外图像和可见光图像分别沿行方向和列方向均匀地划分成多个图像块，获得对应的红外图像块和可见光图像块。如分别将已配准的红外图像和可见光图像均划分为N×M个对应的图像块，即将已配准的红外图像和可见光图像均划分成N行和M列，每一行对应的M个图像块的面积大小相等，每一列对应的N个图像块的面积大小相等。
具体的，在一些实施例中，可以采用预设大小的滑动窗口按预设步长依序滑动遍历已配置的红外图像和可见光图像，每滑动一步所述滑动窗口，获得对应的一个红外图像块和可见光图像块。在其它实施例中，也可以定义所需划分的图像块数或尺寸，然后根据已配准的红外图像和可见光图像的尺寸从各自的像素原点开始划分，以获得对应的各个红外图像块和可见光图像块。对已配准的红外图像和可见光图像的具体划分成图像块的方式在本申请不做特别限定。
此外，在一些实施例中，将已配准的红外图像和可见光图像分别沿行方向和列方向均匀地划分成多个图像块，获得对应的红外图像块和可见光图像块，其中，相邻的两个红外图像块以及相邻的两个可见光图像块之间具有预设的重叠区域。将相邻的图像块之间设置适当大小的重叠区域，可以减小在频域中对各个频域图像块进行融合步骤(S6)中可能引入的“伪结构”，即避免融合图像失真的现象。
在其它实施例，将已配准的红外图像和可见光图像分别沿行方向和列方向均匀地划分成多个图像块，获得对应的红外图像块和可见光图像块，任意相邻的所述红外图像块之间以及相邻的所述可见光图像块之间无重叠区域。还或者对于在融合的过程中容易出现“伪结构”(失真)的区域，将相邻的红外图像块以及相邻的可见光图像块之间设置为具有预设重叠区域，而在剩余的其它区域，则将使任意相邻的所述红外图像块之间以及相邻的所述可见光图像块之间无重叠区域。
在S6中的预设融合规则可以有很多种选择，例如，把频域红外图像块和对应的频域可见光图像块按照某一预设权重逐像素加权平均后的结果作为对应频域融合图像块，其中所述某一预设权重大小可以根据频域红外图像块和频域可见光图像块的对应的像素位置设定。或者在进行各个对应的频域图像块的融合 过程中，选择红外和可见光图像块中具有较大强度绝对值的像素作为频域融合图像块的对应像素；还或者在低频段选择频域红外图像块的像素而高频段选择频域可见光图像块的像素来确定对应的频域融合图像中的像素。在本申请中，将对应的频域图像块之间进行融合的方式不做特别限定。
因此，在一些实施例中，S6具体包括：将各个所述频域红外图像块分别与其对应的所述频域可见光图像块按预设权重逐像素加权平均，获得对应的频域融合图像块。这里的预设权重根据其对应的频域图像块的对应像素位置而设定，如对应像素离原点越远，其对应的预设权重则越小。按预设权重逐像素加权平均包括将频域红外图像块中的像素与第一权重进行加权确定第一加权值，以及将频域可见光图像块中对应位置的像素与第二权重进行加权确定第二加权值，再确定第一加权值与第二加权值的平均值，然后将该平均值作为频域融合图像块中对应位置的像素。其中，第一权重与第二权重的和为1。
在一些实施例中，S6也可具体包括：将各个所述频域红外图像块分别与其对应的所述频域可见光图像中相同位置的像素值进行比较，根据像素值较大的像素确定对应位置的像素，获得对应的频域融合图像块。如频域红外图像块中的像素A与对应的可见光图像块中的像素B为相同位置的像素，则将两个像素的像素值进行比较，若像素A的像素值较大，则将像素A的像素作为频域融合图像块中对应位置的像素，否则以像素B作为频域融合图像块中对应位置的像素。这里的对应位置是指具有相同像素坐标的位置。
此外，由于在频域中低频部分着重代表整体信息，而高频部分着重代表局部细节信息。为了尽可能保留住红外图像中的整体信息，而不去关注红外图像的细节信息，以及尽可能的关注可见光图像的细节信息，而不去关注可见光图像的整体信息，在一些实施例中S6还可具体包括：在低频段根据所述频域红外图像块中的像素确定对应位置的像素，在高频段根据所述频域可见光图像块中的像素确定对应位置的像素，获得对应的频域融合图像块。在其它实施例中，S6也可采用将各个所述频域红外图像块分别与其对应的所述频域可见光图像块按预设权重逐像素加权平均，获得对应的频域融合图像块的融合规则，只是这里的预设权重与所处的频段相关，在低频段，第一权重大于第二权重，而在高频段，第一权重小于第二权重。
在图像块的划分过程中，若将相邻的图像块之间设置为具有预设重叠区域，则对应的相邻空域融合图像块之间也存在对应的重叠区域，即相邻的空域融合图像块中的每个空域融合图像块由非重叠区域和重叠区域构成。则在一些实施例中，S10具体包括：在将各个空域融合图像块依序进行合并的过程中，根据所述空域融合图像块的非重叠区域的像素确定所述融合图像中对应位置(与非重叠区域相对应的位置)的像素，根据具有重叠区域的相邻所述空域融合图像块在对应重叠区域的平均像素确定所述融合图像中对应位置(与重叠区域相对应的位置)的像素。若空域融合图像块a与空域融合图像块b相邻，且二者区域重叠区域d，则融合图像中与重叠区域d相对应的位置处的像素由空域融合图像块a与空域融合图像块b在重叠区域d中的平均像素确定。在其它实施例中，融合图像中与重叠区域d相对应的位置处的像素的确定方式还可以为：将相邻的两个所述空域融合图像块a、b的中心位置像素距重叠区域d的距离进行比较，根据较近距离对应的所述空域融合图像块的重叠区域像素确定融合图像中与重叠区域d相对应的位置处的像素。如若空域融合图像块a的中心位置距重叠区域d的位置相对较近，则将空域融合图像块a的重叠区域的像素作为融合图像中与重叠区域d相对应的位置处的像素。
如图2所示，其为依据本申请另一些实施例提供的图像融合方法的流程示意图，相比图1所示的图像融合方法，在本实施例中，图像融合方法还进一步包括获取已配准的红外图像和可见光图像，则图4对应的融合装置中还包括配准获取模块，用于实现获取已配准的红外图像和可见光图像的步骤，该步骤也可以由图4中的图像融合设备实现。获取已配准的红外图像和可见光图像的步骤具体包括S11、S12、S13以及S14。
S11：获取相同场景下相同目标的红外图像和可见光图像。
在本申请的一些实施例中，可以采用同时设置有红外相机和可见光相机的光学设备(如红外热成像仪)于同一时刻对场景中的相同目标进行拍摄，以获得对应的红外图像和可见光图像。也可以分别采用单独的红外相机和可见光相机在同一时刻或不同时刻拍摄同一场景中的相同目标，获得对应的红外图像和可见光图像。当使用不同设备获取红外图像和可见光图像时，可使红外相机和可见光相机放在同一位置，且镜头光轴同方向且平行，以获得同一角度的红外图像和可见光图像。红外相机和可见光相机也可以放置在不同位置，可以获取不同角度的红外图像和可见光图像。红外图像和可见光图像的分布率可相同也 可不同。可以在对应的拍摄装置采集红外图像和可见光图像时对其采集图像的分辨率进行设置，也可以在采集对应的图像后，对对应的图像的分辨率进行调整配置。此外，在从图像拍摄装置(红外相机、可见光相机)中获取其采集对应的图像后，还可对对应的图像进行适当的图像处理，如图像裁剪或图像拉伸使得所示红外图像和所示可见光图像的尺寸相同。
S12：将所述可见光图像转换为灰度图像。
在后续的图像融合过程中，主要从可见光图像中提取细节信息，如目标对象的轮廓等，为了简化后续的坐标映射关系以及傅里叶变换的计算复杂度，本申请实施例中，将可见光图像进行灰度化处理，以获得其对应的灰度图像。
S13：确定所述红外图像和所述可见光图像的灰度图像之间的坐标映射关系。
S14：根据所述坐标映射关系将所述红外图像和所述可见光图像对齐映射到同一坐标系中，获得配准的所述红外图像和所述可见光图像。
由于红外相机和可见光相机的焦距、视场以及分辨率不一致，在进行红外图像和可见光图像融合过程中，需要对红外图像和可见光图像进行逐像素点匹配对准(配准)，使得同一个像素对应相同的物体，以确保融合图像不出现失真和假影。红外图像和可见光图像的坐标系可能存在差异，即二者的坐标系不同，红外图像与可见光图像对应同一目标点的像素于不同的空间坐标系下显示不同像素坐标，从而可建立二者之间的坐标映射关系。坐标映射关系可以通过如下形式给出：红外图像的第i1行第j1列的像素对应可见光图像的第i2行第j2列的像素，或者可见光图像的第i1行第j1列的像素对应红外图像的第i2行第j2列的像素。坐标映射关系也可以通过映射表的形式表征，本申请中，对坐标映射关系的具体表现形式不做特别限定。
具体的，在一些实施例中，可根据采集所述红外图像的所述红外相机和采集所述可见光图像的所述可见光相机的参数信息(内参)和相对几何位置信息，确定所述红外图像和所述可见光图像的灰度图像之间的坐标映射关系。
进一步的，在一些实施例中，所述根据所述坐标映射关系将所述红外图像和所述可见光图像对齐映射到同一坐标系中，获得已配准的所述红外图像和所述可见光图像，包括：根据所述映射关系将所述红外图像和所述可见光图像对齐映射到相同坐标系中，并基于灰度级插值方法对所述同一坐标系中的非整数像素坐标对应的像素进行赋值。因为上确定的映射关系中存在像素坐标不是整 数的情况，即将对齐映射后的图像中的某个像素点投影到原始的红外图像或可见光图像中时，其对应的坐标并非整数。在一些实施例中，灰度级插值方法为最近邻插值方法，基于最近邻插值方法对所述同一坐标系中的非整数像素坐标对应的像素进行赋值具体为将离该非整数最近的整数坐标的像素作为该非整数坐标对应的对齐映射后中的对应像素。在其它一些实施例中，灰度级插值方法还可以为双线性插值方法、双立方插值方法等其它插值方法。
由于S12中对可见光图像进行了灰度化处理，因此S10中获得的融合图像为灰度融合图像。为了进一步提高融合图像的可辨识度，在一些实施例中，如图2所示，在S10之后，所述图像融合方法还包括S20：按照预设的伪彩映射表，将所述融合图像变换为融合伪彩图像。
为了进一步清楚描述本申请提供的图像融合方法，图3给出了依据本申请的图像融合方法的又一些实施例。在又一些实施例中，本申请提供的图像融合方法主要包括S01至S010，其根据本实施例的图像融合方法对原始的红外图像和可见光图像所做的处理过程如图3所示。
S01：分别获取原始的红外图像和原始的可见光图像。
S02：灰度化处理。将原始的可见光图像转化为灰度图像。
S03：确定坐标映射关系。确定原始的红外图像和原始的可见光图像对应的灰度图像之间的坐标映射关系。
S04：对齐映射。根据坐标映射关系将原始的红外图像和可见光图像的对应的灰度图像对齐映射到同一坐标系中。
S05：图像分块。分别将对齐映射后的红外图像和所述灰度图像划分为多个图像块，以获得对应的红外图像块和可见光图像块。
S06：傅里叶变换。分别对红外图像块和可见光图像块进行傅里叶变换，以获得对应的频域图像块。
S07：图像块融合。将各个频域红外图像块和对应的频域可见光图像块进行融合，以获得各个对应的频域融合图像块。
S08：傅里叶逆变换。分别对各个频域融合图像块进行傅里叶逆变换，以获得对应的各个空域融合图像块。
S09：图像块合并。将各个空域融合图像块按照原有顺序进行合并，以恢复整幅灰度化的融合图像。
S010：伪彩映射。根据预设的伪彩映射表，将灰度化的融合图像映射伪伪彩融合图像。
本申请提供的图像融合方法，通过二维傅里叶变换来分解和提取原始红外和可见光图像上的信息，充分利用了红外和可见光图像中所有信息，且整体流程相对精简，有利于在算力受限的设备上处理实时视频流。
需要说明的是，在本申请提供的图像融合方法中，上述各个步骤的先后顺序并不局限于各实施例中所示的顺序，其它实施例中，一些步骤的顺序可以互换。
如图4所示，在一些实施例中，本申请还提供了一种图像融合装置。所述图像融合装置包括图像划分模块11、傅里变换模块12、图像融合模块13、傅里叶逆变换模块14以及图像合并模块15。
其中，图像划分模块11用于将已配准的红外图像和可见光图像分别划分为多个图像块，获得对应的红外图像块和可见光图像块；傅里变换模块12用于对各个所述红外图像块和所述可见光图像块分别进行傅里叶变换，获得对应的频域红外图像块和频域可见光图像块；图像融合模块13用于将各个所述频域红外图像块分别与其对应的所述频域可见光图像块按预设融合规则进行融合，获得对应的频域融合图像块；傅里叶逆变换模块14用于将各个所述频域融合图像块进行傅里叶逆变换，获得对应的空域融合图像块；图像合并模块15用于将各个所述空域融合图像块进行合并，获得融合图像。
如图5所示，在一些实施例中，本申请还提供了一种图像融合设备，其存储器21存及处理器13。所述处理器22在运行所述存储器21中存储的计算机程序指令时，执行依据本申请任意一实施例中所述的图像融合方法的步骤。
此外，本申请还提供了一种计算机可读存储介质，所述计算机可读存储介质存储有计算机程指令；所述计算机程序指令被处理器执行时，实现依据本申请任意一实施例提供的项所述的检测方法的步骤。
上述处理器可能是CPU(中央处理器，Central Processing Unit)，或者是ASIC(特殊应用集成电路，Application Specific Integrated Circuit)，或者是被配置成实施本发明实施例的一个或多个集成电路。运动目标的检测设备 包括的一个或多个处理器，可以是同一类型的处理器，如一个或多个CPU；也可以是不同类型的处理器，如一个或多个CPU以及一个或多个ASIC。
上述存储器可能包含高速RAM(随机存取存储器，Random Access Memory)，也可能还包括NVM(非易失性存储器，Non-Volatile Memory)，例如至少一个磁盘存储器。
此外，本申请还提供了一种计算机可读存储介质，所述计算机可读存储介质存储有计算机程指令，所述计算机程序指令被处理器执行时，实现依据本申请提供的任意一实施例中所述的测距方法的步骤。
以上所述，仅为本申请的具体实施方式，但本申请的保护范围并不局限于此，任何熟悉本技术领域的技术人员在本申请揭露的技术范围之内，可轻易想到变化或替换，都应涵盖在本申请的保护范围之内。因此，本申请的保护范围应以所述权利要求的保护范围为准。

Claims (10)

  1. 一种图像融合方法，其特征在于，包括：
    将已配准的红外图像和可见光图像分别划分为多个图像块，获得对应的红外图像块和可见光图像块；
    对各个所述红外图像块和所述可见光图像块分别进行傅里叶变换，获得对应的频域红外图像块和频域可见光图像块；
    将各个所述频域红外图像块分别与其对应的所述频域可见光图像块按预设融合规则进行融合，获得对应的频域融合图像块；
    将各个所述频域融合图像块进行傅里叶逆变换，获得对应的空域融合图像块；
    将各个所述空域融合图像块进行合并，获得融合图像。
  2. 根据权利要求1所述的图像融合方法，其特征在于，所述将已配准的红外图像和可见光图像分别划分为多个图像块，获得对应的红外图像块和可见光图像块，包括：
    将已配准的红外图像和可见光图像分别沿行方向和列方向均匀地划分成多个图像块，获得对应的红外图像块和可见光图像块；
    相邻的所述红外图像块之间以及相邻的所述可见光图像块之间分别包括部分重叠区域，或，任意相邻的所述红外图像块之间以及任意相邻的所述可见光图像块之间无重叠区域。
  3. 根据权利要求1所述的图像融合方法，其特征在于，所述将各个所述频域红外图像块分别与其对应的所述频域可见光图像块按预设融合规则进行融合，获得对应的频域融合图像块，包括：
    将各个所述频域红外图像块分别与其对应的所述频域可见光图像块按预设权重逐像素加权平均，获得对应的频域融合图像块；或，
    将各个所述频域红外图像块分别与其对应的所述频域可见光图像中相同位置的像素值进行比较，根据像素值较大的像素确定对应位置的像素，获得对应的频域融合图像块；或，
    在低频段根据所述频域红外图像块中的像素确定对应位置的像素，在高频段根据所述频域可见光图像块中的像素确定对应位置的像素，获得对应的频域融合图像块。
  4. 根据权利要求1所述的图像融合方法，其特征在于，相邻的所述空域融合图像块之间具有重叠区域，所述将所述各个空域融合图像块进行合并，获得融合图像，包括：
    在将所述各个空域融合图像块依序进行合并的过程中，根据所述空域融合图像块的非重叠区域的像素确定所述融合图像中对应位置的像素，根据具有重叠区域的相邻所述空域融合图像块在对应重叠区域的平均像素确定所述融合图像中对应位置的像素；或，
    在将所述各个空域融合图像块依序进行合并的过程中，根据所述空域融合图像块的非重叠区域的像素确定所述融合图像中对应位置的像素，以及将相邻的两个所述空域融合图像块的中心位置像素距对应重叠区域的距离进行比较，根据较近距离对应的所述空域融合图像块的重叠区域像素确定所述融合图像中对应位置的像素。
  5. 根据权利要求1至4中任意一项所述的图像融合方法，其特征在于，所述图像融合方法还包括：
    获取相同场景下相同目标的红外图像和可见光图像；
    将所述可见光图像转换为灰度图像；
    确定所述红外图像和所述可见光图像的灰度图像之间的坐标映射关系；
    根据所述坐标映射关系将所述红外图像和所述可见光图像对齐映射到同一坐标系中，获得配准的所述红外图像和所述可见光图像。
  6. 根据权利要求5所述的图像融合方法，其特征在于，所述获取相同场景下相同目标的红外图像和可见光图像，包括：获取红外相机和可见光相机分别采集到的针对相同场景下相同目标的红外图像和可见光图像；
    所述确定所述红外图像和所述可见光图像的灰度图像之间的坐标映射关系，包括：
    根据所述红外相机和所述可见光相机的参数信息和相对几何位置信息，确定所述红外图像和所述可见光图像的灰度图像之间的坐标映射关系。
  7. 根据权利要求5所述的图像融合方法，其特征在于，所述根据所述坐标映射关系将所述红外图像和所述可见光图像对齐映射到同一坐标系中，获得已配准的所述红外图像和所述可见光图像，包括：
    根据所述映射关系将所述红外图像和所述可见光图像对齐映射到相同坐标系中，并基于灰度级插值方法对所述同一坐标系中的非整数像素坐标对应的像素进行赋值。
  8. 根据权利要求5所述的图像融合方法，其特征在于，在所述将各个所述空域融合图像块进行合并，获得融合图像之后，所述图像融合方法还包括：
    按照预设的伪彩映射表，将所述融合图像变换为融合伪彩图像。
  9. 一种图像融合设备，其特征在于，包括存储器和处理器；
    所述处理器在运行所述存储器中存储的计算机程序指令时，执行权利要求1至8中任意一项所述的图像融合方法的步骤。
  10. 一种计算机可读存储介质，其特征在于，所述计算机可读存储介质存储有计算机程指令；
    所述计算机程序指令被处理器执行时，实现权利要求1至8中任意一项所述的图像融合方法的步骤。
PCT/CN2022/094716 2022-01-14 2022-05-24 图像融合方法、设备以及存储介质 WO2023134103A1 (zh)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN202210042205.0 2022-01-14
CN202210042205.0A CN114372948A (zh) 2022-01-14 2022-01-14 图像融合方法、设备以及存储介质

Publications (1)

Publication Number Publication Date
WO2023134103A1 true WO2023134103A1 (zh) 2023-07-20

Family

ID=81143667

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2022/094716 WO2023134103A1 (zh) 2022-01-14 2022-05-24 图像融合方法、设备以及存储介质

Country Status (2)

Country Link
CN (1) CN114372948A (zh)
WO (1) WO2023134103A1 (zh)

Cited By (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN117201693A (zh) * 2023-11-01 2023-12-08 长春汽车工业高等专科学校 一种物联网图像压缩方法、装置、终端设备及介质
CN117576564A (zh) * 2023-11-27 2024-02-20 日照市农业科学研究院 用于茶叶种植的病虫害识别预警方法与系统
CN117575933A (zh) * 2024-01-08 2024-02-20 北京摩弈信息科技有限公司 一种图像融合方法和装置、电子设备
CN117788532A (zh) * 2023-12-26 2024-03-29 四川新视创伟超高清科技有限公司 一种安防领域基于fpga的超高清双光融合配准方法
CN117952841A (zh) * 2024-03-26 2024-04-30 山东省地质测绘院 基于人工智能的遥感图像自适应增强方法

Families Citing this family (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN114372948A (zh) * 2022-01-14 2022-04-19 无锡英菲感知技术有限公司 图像融合方法、设备以及存储介质

Citations (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN103500443A (zh) * 2013-10-10 2014-01-08 中国科学院上海技术物理研究所 一种基于傅里叶变换的红外偏振图像融合方法
US20170193635A1 (en) * 2014-05-28 2017-07-06 Peking University Shenzhen Graduate School Method and apparatus for rapidly reconstructing super-resolution image
CN109584193A (zh) * 2018-10-24 2019-04-05 航天时代飞鸿技术有限公司 一种基于目标预提取的无人机红外与可见光图像融合方法
CN112102217A (zh) * 2020-09-21 2020-12-18 四川轻化工大学 一种可见光图像与红外图像快速融合方法及系统
CN112258442A (zh) * 2020-11-12 2021-01-22 Oppo广东移动通信有限公司 图像融合方法、装置、计算机设备和存储介质
CN113393398A (zh) * 2021-06-21 2021-09-14 努比亚技术有限公司 一种图像降噪处理方法、设备及计算机可读存储介质
CN114372948A (zh) * 2022-01-14 2022-04-19 无锡英菲感知技术有限公司 图像融合方法、设备以及存储介质

Patent Citations (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN103500443A (zh) * 2013-10-10 2014-01-08 中国科学院上海技术物理研究所 一种基于傅里叶变换的红外偏振图像融合方法
US20170193635A1 (en) * 2014-05-28 2017-07-06 Peking University Shenzhen Graduate School Method and apparatus for rapidly reconstructing super-resolution image
CN109584193A (zh) * 2018-10-24 2019-04-05 航天时代飞鸿技术有限公司 一种基于目标预提取的无人机红外与可见光图像融合方法
CN112102217A (zh) * 2020-09-21 2020-12-18 四川轻化工大学 一种可见光图像与红外图像快速融合方法及系统
CN112258442A (zh) * 2020-11-12 2021-01-22 Oppo广东移动通信有限公司 图像融合方法、装置、计算机设备和存储介质
CN113393398A (zh) * 2021-06-21 2021-09-14 努比亚技术有限公司 一种图像降噪处理方法、设备及计算机可读存储介质
CN114372948A (zh) * 2022-01-14 2022-04-19 无锡英菲感知技术有限公司 图像融合方法、设备以及存储介质

Cited By (8)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN117201693A (zh) * 2023-11-01 2023-12-08 长春汽车工业高等专科学校 一种物联网图像压缩方法、装置、终端设备及介质
CN117201693B (zh) * 2023-11-01 2024-01-16 长春汽车工业高等专科学校 一种物联网图像压缩方法、装置、终端设备及介质
CN117576564A (zh) * 2023-11-27 2024-02-20 日照市农业科学研究院 用于茶叶种植的病虫害识别预警方法与系统
CN117576564B (zh) * 2023-11-27 2024-04-19 日照市农业科学研究院 用于茶叶种植的病虫害识别预警方法与系统
CN117788532A (zh) * 2023-12-26 2024-03-29 四川新视创伟超高清科技有限公司 一种安防领域基于fpga的超高清双光融合配准方法
CN117575933A (zh) * 2024-01-08 2024-02-20 北京摩弈信息科技有限公司 一种图像融合方法和装置、电子设备
CN117952841A (zh) * 2024-03-26 2024-04-30 山东省地质测绘院 基于人工智能的遥感图像自适应增强方法
CN117952841B (zh) * 2024-03-26 2024-06-04 山东省地质测绘院 基于人工智能的遥感图像自适应增强方法

Also Published As

Publication number Publication date
CN114372948A (zh) 2022-04-19

Similar Documents

Publication Publication Date Title
WO2023134103A1 (zh) 图像融合方法、设备以及存储介质
CN108898567B (zh) 图像降噪方法、装置及系统
CN109377469B (zh) 一种热成像融合可见光图像的处理方法、系统及存储介质
CN110717942B (zh) 图像处理方法和装置、电子设备、计算机可读存储介质
KR101643607B1 (ko) 영상 데이터 생성 방법 및 장치
CN109474780B (zh) 一种用于图像处理的方法和装置
WO2022100242A1 (zh) 图像处理方法、装置、电子设备和计算机可读存储介质
JP2018084982A (ja) 画像処理装置、情報処理方法及びプログラム
WO2013099628A1 (ja) 画像処理装置、画像処理システム、画像処理方法、および、プログラム
US11538175B2 (en) Method and apparatus for detecting subject, electronic device, and computer readable storage medium
CN111815517B (zh) 一种基于球机抓拍图片的自适应全景拼接方法
Várkonyi-Kóczy et al. Gradient-based synthesized multiple exposure time color HDR image
CN110766706A (zh) 图像融合方法、装置、终端设备及存储介质
US20230401855A1 (en) Method, system and computer readable media for object detection coverage estimation
CN112470189B (zh) 光场系统的遮挡消除
CN115760665A (zh) 图像的多尺度配准融合方法、装置、终端设备及存储介质
CN113159229B (zh) 图像融合方法、电子设备及相关产品
CN113628134B (zh) 图像降噪方法及装置、电子设备及存储介质
CN114742866A (zh) 图像配准方法、装置、存储介质及电子设备
US9392146B2 (en) Apparatus and method for extracting object
CN113592753B (zh) 基于工业相机拍摄的图像的处理方法、装置和计算机设备
CN114663284A (zh) 红外热成像全景图像处理方法、系统及存储介质
Wegner et al. Image based performance analysis of thermal imagers
Zhan et al. HDR synthesis technology for spaceborne CMOS cameras based on virtual digital TDI
Cho et al. Improvement on Demosaicking in Plenoptic Cameras by Use of Masking Information

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 22919742

Country of ref document: EP

Kind code of ref document: A1